With the new school year kicking off, I want to talk, again, about the importance of not relying on plagiarism tools, especially when it comes to AI-generated text. For heaven's sake, some of these detectors identified the US Constitution as AI-generated text 🤦🏼♀️.
Based on these wildly flawed "detectors," I've seen stories of students being failed in classes and not allowed to graduate. Freelancers have been fired and not paid for their work. Job applicants have been rejected. Employers have terminated loyal and long-time employees. And the list goes on.
Recently, OpenAI (ChatGPT) made the decision to retire its AI classifier, a tool that claimed to detect if a text passage was written by another AI. The reason? It faced widespread criticism due to its "low rate of accuracy." Since it was accurate, at its' best tested rate, 26% of the time, the criticism was clearly warranted.
But OpenAI's plagiarism detector isn't the only tool with this issue! Arguably, there are NO reliable tools regarding AI-generated content.
The idea of identifying AI-generated text through distinctive patterns seems plausible, but in practice, it's not effective. Language models evolve rapidly, making telltale signs increasingly elusive.
TechCrunch tested several AI-writing-detection tools and found that their reliability was inconsistent at best, and some were entirely useless. In this test, OpenAI's classifier identified just one out of seven (14%} generated text snippets correctly.
In light of the continuous improvement and proliferation of language models, OpenAI decided it was time to take their detector offline. They're now working to incorporate feedback and researching more effective provenance techniques for text. It's all part of their commitment to develop AI ethically and transparently, along with other companies making a "voluntary commitment."
One key aspect of this commitment involves developing robust watermarking and detection methods. However, despite the industry's noise about these methods, none have proven to be foolproof against circumvention. The pursuit of reliable watermarking and detection tools is ongoing, and any breakthrough in this area would be valuable.
Plagiarism tools can be useful, but they should not be solely relied upon! Working in partnership with AI to create the very best possible outcomes is the way of the future. And the future - is here.
Komentáře