Wednesday, 18 June 2008

Turnitin and text matching




  • Dr Niaill Hayes described (at the ASKe CETL one-day conference on institutional management of plagiarism) a three year study, just finished in May 2008, funded by FDTL5 and based at the University of Lancaster and the LSE. The study attempted to 'get inside' plagiarism detection software systems to try and understand the underpinning assumptions and procedures that the system creators were using. They tested 15,000 fragments of text to try and gauge the likelihood that any student's copied text would be identified by Turnitin. They found the likelihood to be about 50% - that is, students who copied had about a 50/50 chance of being identified though of course, this is a crude number and would be impacted by other factors such as where the student copied from etc.

    It certainly confirms the widely held view that the software NOT finding a match does not mean the text is not copied. [apologies for all those negatives]


    The above statement was a precis provided by Jude Carroll to me, three weeks ago. Her 10 minute plagiarism video housed at the Nottingham Uni best practice web portal is well worth a look.

    The research also looked into the use of illegal characters (see Guardian link below) used by some to dupe the system, and also the interpretation of the originality report. Full report is out soon. We are waiting from an official reply from JISCiPAS the JISC plagiarism advisory service that supports manages the TII database UK franchise.


    To conclude

    Don’t over hype the capabilities of TII – as has patently already occurred across academia. The implementation and subsequent use of this type of software MUST be part of a holistic range of measures that educates both staff and students to the requirement for scholarly practice at higher levels. JISCiPAS as an organisation do this extremely well. Please take time to visit the excellent resources on their website. http://www.jiscpas.ac.uk/ . The move towards more creative ways of assessing student capabilities is beginning to gather momentum. The use of group work, vivas, computer aided assessments, and more personal reflections are all being promoted as alternatives to the traditional essay, which, coupled with the (misuse of) internet are blamed for the ease/rise of plagiarism.

    Further reading:

    Other relevant Plagiarism news is rife at present and reinforces beliefs on students “buying” more ready-made essays and the shortfalls of TII’s algorithms.

  • Essay auctions 'harder to catch':
    http://news.bbc.co.uk/1/hi/education/7302641.stm
  • A new AMBeR report just published by the Academy
http://www.heacademy.ac.uk/news/detail/uk_report_student_plagiarism

http://www.ics.heacademy.ac.uk/resources/assessment/plagiarism/


2 comments:

Jude carroll said...

Hi Kevin


Thanks for the mention but can you delete the title 'professor'. Id hate for anyone to think I was passing myself off as anything other than just 'Jude'.

mreddygbr said...

I think that this does a lot to kick out the crutch of detection software, but we should (to really cram in as many metaphors as possible in one sentence) not throw the baby out with the bathwater. TII et all still serve a useful purpose, in context.