AI Detectors

As generative AI tools change rapidly and become more integrated in writing technologies, the ability to detect AI writing becomes increasingly challenging. At DePaul, we have access to Turnitin’s AI Detector via Originality Check in D2L Submission folders. This tool is designed to identify text that may have been generated by AI and to support academic integrity-based conversations. 

Turnitin’s AI detection tool has the highest AI-detection accuracy rates when the submitted text has been wholly generated by one of the generative AI tools Turnitin’s model is trained to detect (i.e., ChatGPT, Gemini (Pro), and LLaMA), but still results in some false positives. 

All AI detection tools are much less accurate when the writing contains a mix of AI-generated content and human writing. The accuracy is also significantly lower when writers use specific AI prompting or adversarial techniques (e.g., “rewrite the following passage with spelling errors”), integrate their own writing while prompting, and/or use AI paraphrasers, “humanizers,” and spin bots (e.g., Quillbot).

Foolproof tools are not available, and poor AI detection and false positives can create stress for faculty and for students. For these reasons, we recommend you use AI detection tools with caution. If you’re using an AI detection tool with student writing, use the results to start a conversation with the student. Review the work further and be open to the possibility that the tool’s detection results were incorrect.

Turnitin’s AI Detection Tool

Turnitin's AI detection tool is integrated with Originality Check in D2L Submission folders. In September 2023, the DePaul Teaching, Learning, and Technology (TLT) Committee reviewed Turnitin’s AI detection tool and determined that it shouldn’t be used until the University had more time to evaluate the tool. Based on this recommendation, as of September 28, 2023, AI detection was turned off for all DePaul Turnitin accounts. 

Faculty are rightly concerned about how to know whether or not a student has used AI for an assignment, and having a tool to help you work through that circumstance would be helpful. While Turnitin's AI Detector will be an available tool for faculty to have in their toolbox, there is no one-size-fits-all solution to address the complexities of academic misconduct. Please read about how Turnitin's AI Detector worksand the Turnitin AI Detector report, and keep the following in mind when using Turnitin’s AI Detector: 

• Students are unable to see the Turnitin AI writing detection indicator and report, even if they are able to view the Turnitin Originality Report. Faculty will need to download the AI writing detection report as a PDF in order to share results with students. 
  
• Turnitin's AI Detector should be seen as a starting point for a conversation rather than a strict enforcement tool.
• Turnitin AI detection is the only tool integrated in D2L and supported by DePaul University resources; as of 2023, it has also been found to be one of the most accurate AI detection tools. AI detection tools are powered by generative AI, so the same security and data concerns can apply. If you are using detection tools outside of Turnitin in D2L, consider how student work may be used to train those detection tools, and take steps to protect student information by removing any identifying information.
  
• Turnitin's detector is trained to detect AI content written by large language models such as GPT-3.0, and 3.5, 4, 4o (Chat GPT), Gemini (Pro) and LLaMA. Turnitin is also designed to detect AI generated content created via a text spinner or paraphrasing tool, but specific tools are not indicated. It's possible that faculty using Turnitin AI detection would be detecting usage from only some AI models and tools.
  
• Turnitin’s AI detection has a 1% false positive rate for documents that contain 20% or more of AI-generated writing, and may miss about 15% of AI written text in a document. The tool has a 4% sentence-level false positive rate, and sentence-level false positives are more likely in documents that contain a mix of human and AI generated writing. 
• Turnitin’s AI detection is most accurate when the submitted work is generated by AI and unaltered, or in other words, when the work is copy and pasted from a generative AI tool. Tools such as Grammarly, AI paraphrasers, and text spinners, specific AI prompting techniques, and co-writing approaches like using human written outlines to prompt gen AI pose additional challenges for AI detection tools and interpreting Turnitin's AI writing indication and report.
• AI detection tools may falsely indicate that text written by multilingual and developing learners was written by AI. While Turnitin reports that the tool shows no bias against multilingual learners, and a 2024 study also indicates no bias, other evaluations have identified issues, and more peer-reviewed studies are needed. 
• The Academic Integrity Board has resources on filing violations and preparing documentation.
  

DePaul’s Response to Generative AI 

As the Provost noted in her message about AI at DePaul, faculty are encouraged to approach the unique challenge our university community faces by learning about the affordances and limitations of AI tools, discussing the impact of AI on your discipline and curriculum with colleagues, and evaluating how you’ll approach AI in your classes by developing syllabus statements that set expectations and reviewing your assignments for AI impact. This guide is a resource to support these efforts. 

Please stay tuned to the Teaching Commons Events page for future programming offerings. If your unit is planning to discuss how AI is impacting your work, a member of our team would be happy to join you and support that conversation. Please email facultydevelopment@depaul.edu.

If there are questions or concerns about this decision please contact TLT@depaul.edu or your faculty representative to this committee.