The Purposeful Prose Advice Column: What’s the Deal With AI Detectors?

Dear Purposeful Prose,

I’ve written several short stories over a period of years, before ChatGPT even existed. I decided to put them through an AI detector, and it determined that over half of my content was composed by AI. I know that there are a lot of problems with using AI to write stories, but I haven’t written any of my content using AI. Is there something I’m doing wrong? If I try to publish these at a later time, am I going to have problems?

First and foremost, no, you’re not doing anything wrong. You’re right that ChatGPT and other generative AI has been used as a shortcut for writers, for students, for a lot of people who are looking to simplify their work flow. 

It’s important to remember that AI detection software still has notoriously high error rates. I’ve heard of students turning in their own written work into their professors only to be accused of using AI. Academic dishonesty is a very strong accusation that can ruin a student’s professional future, and it can take a great deal of time and effort to prove innocence. Authors have been accused of using generative AI in their work as well, to varying degrees of success. 

AI detectors were created to respond to a growing problem. People were using generative AI to produce “art”, pieces of writing, homework answers. Generative AI systems were trained from the contents of the internet, and that ruined things for people who wrote stories in good faith. 

“Why does ChatGPT use the em dash so much?” It’s because we do.

“Why does ChatGPT contrast so much?” Again, it’s because we do. 

“Why does the language look like this?” It’s because our language can also look like this. 

AI detectors were trained to pick up on the tells that we’ve accumulated for large language models, and as a result, they can be fooled as easily as we can. 

I’d like to say that I have a good eye for language that has been produced by generative AI, and I’ve been right most of the time. If you’ve been following my professional journey, you know that when ChatGPT became widespread, I was evaluating writing samples. At the same time, the writing samples that came in started to look different. 

It was easy to tell when the sources the samples cited were fake or unattributed. I was also able to pick out the overly broad language, overused conjunctions, and hallucinations. I also used several AI detector tools, and some were more effective than others. However, the best AI detector tool I had was the writer themselves. 

They could walk me through their writing process, talk through their source material, and tell me why they chose particular turns of phrase. More simply, someone who wrote their piece could tell me how they wrote it. If you can do the same thing, that will help you. 

I can’t tell you whether you’ll have problems publishing your work. I can’t guarantee that someone won’t throw an AI accusation at you, so here’s my advice. Keep records of everything. Make sure that your manuscripts are marked with dates, particularly those that were written before ChatGPT was released. If you’ve worked with an editor and have email chains, save them. If you have outlines and notes, save those. 

Certain pieces of writing software, including Microsoft Word and Google Docs, will allow you to check document history as well. This can work to your benefit, particularly if you’re publishing and an issue comes up. This information should not be necessary, but you should have tools at your disposal in case you need them.

You deserve better. Keep writing, and if something happens, keep this advice in mind. I hope that you never need to use it. 

If you have a writing or editing-focused question, I would be more than happy to dedicate a post to you. Contact us for any questions you might have or to schedule a free consultation with yours truly!

All submissions will remain anonymous unless you explicitly request for me to include your name.