As new ChatGPT alternatives pop up daily, AI content detectors are also gaining popularity.
In this article, I’ll share 16 of these detectors and show you how they score some paragraphs of my original, unpublished writing versus a few paragraphs on the same topic from ChatGPT.
I’ll also walk through the types of functions AI detectors are (and aren’t) a good fit for and how marketers, editors, and SEOs should think about them.
What detection tools are good (and bad) at
As I’ve detailed in other articles, generative AI and ChatGPT content poses several issues:
- AI-generated information can be factually incorrect, dangerous, outdated, or misleading.
- AI writing outputs can be subpar.
- While there’s no explicit penalty for AI content, Google may not always trust and view it like human-created content.
- AI content may be able to “fool” editors or businesses who think they’re paying for human-created content.
- AI content may leverage creative work from humans and repurpose it without attribution.
It’s important to note that the current AI detectors do not solve all of these problems.
Mostly, these tools do not fact-check AI content, improve or audit content quality, or provide citations for information pulled from other sources.
That said, the areas that AI detectors can help include:
- Plagiarism: Many of these tools have plagiarism detection built in, so there’s some check for whether the AI content was largely pulled from another source.
- Penalty prevention: If you’re concerned about AI content being devalued somehow in search results, these tools can help give you a sense of how easily detectable the AI content is. (Of course, Google will undoubtedly have different tools and checks.)
- Auditing AI usage: If you have a specific policy or way to compensate writers for original versus AI-generated content, these tools can give you a rough sense of whether a writer uses AI to generate content. (Note that they can also return false negatives and positives.)
- Understanding search results: Some of these tools offer Chrome extensions, which can help you understand whether competitors and other websites use AI content or not.
How AI detection software works
Each tool is different and has its approach to the problem. But for the most part, ChatGPT detection tools grade content based on how predictable the phrase choices are within a piece of content.
In other words, the likelihood that content is scored as AI versus human has a lot to do with whether the detection software deems a piece of writing as following the likely pattern AI would follow in generating content.
The two core concepts around this process are called:
- Burstiness: A predictable length and tempo to sentence structure.
- Perplexity: A randomness to the words chosen in a sentence or collection of sentences.
For example, in an essay about the founding of America, it’s highly unlikely that generative AI would include a random, unevenly written anecdote about the first time they ever saw a penguin, so that would likely look like human writing to a detection tool.
Similar to how ChatGPT detectors popped up to detect generative AI writing, tools are already being developed to get around the detectors. (And, of course: the detectors are likely already thinking about how to detect the bypassers, and so on).
Tools like Undetectable or Quillbot will rewrite your content, sometimes making it more difficult to detect for certain AI detection tools.
Additionally, several people have found different prompts to get ChatGPT and other AI writing tools to output content that scores “more human” on the human-to-AI scale by using prompts defining burstiness and perplexity and telling ChatGPT to write with more of each.
Does detection accuracy matter to you?
An important question to answer before you dive too far into these tools is:
How much do you care about detecting whether content is written by AI? And why?
If you’re using ChatGPT for rewriting title tags or generating email copy, maybe it doesn’t matter at all if that content “passes” AI writing checks.
Additionally, if a writer uses AI to generate a copy and the copy is great, maybe the score isn’t important at all.
These detection tools will likely be engaged in the “detection arms race” with un-detection tools and prompts I mentioned above.
The best AI writing detectors compared
If you're still looking for an AI/ChatGPT content detector, we'll go through each of them and how they “scored” in evaluating human-generated copy versus AI copy versus AI copy that used this prompt to try to “beat detection."
Note: Detection versus a few paragraphs of content isn't necessarily a thorough test of the detection capabilities of these tools. Hopefully, it will give you a rough sense of how they score different content and a glimpse of the range of outcomes you can expect from these kinds of tools.
(You can view the actual samples input to the tool – the “human” sample written by me, the “AI” sample written by ChatGPT via GPT-4, and the updated copy based on the same topic here.)
In the table below, you can see how each tool scored the copy I wrote from scratch, the copy I took from ChatGPT directly with no prompt modification, and that same copy tweaked with the “perplexity and burstiness” prompt :
Below is a brief overview of the tools included in the table and some of their key features.
1. Originality.AI
Originality.AI is a paid tool described as “the most accurate AI content detector and plagiarism checker built for serious content publishers.”
It costs $.01 per credit, which scans 100 words. It also features a plagiarism scanner alongside the AI scanner.
As you can see from the chart above, this is the only AI scanner with 100% confidence that the human content was human and that the AI content was AI (while also being correct in all cases).
Hive Moderation (discussed later in the article) essentially had the same conclusion with a 0% AI score for the human content and 99.9% confidence the two AI-generated texts were AI.
On top of having a Chrome extension and an overall score, Originality.AI added a feature highlighting the specific sections of the content you paste that it predicts will and won’t be AI.
I use this tool most frequently to check for AI content personally. My most frequent use-case is checking content submitted by freelance writers we work with for AI and plagiarism.
2. Writer
Writer offers a free detector that works on up to 1,500 characters and requires API access for more.
This tool essentially thought all of the submissions were relatively likely to be human-generated.
Writer is an AI writing platform, and the free content detector also allows you to input a URL to check.
There is no plagiarism feature within this tool directly, and it gives you an overall score but doesn’t mark specific sections as likely or unlikely to be AI-generated.
3. Copyleaks AI Content Detector
The Copyleaks AI Content Detector has a few interesting features:
- It gives you an overall verdict (human or AI).
- If you hover over specific text, it will give you a percentage likelihood that the text is human versus AI copy.
- It pre-loads with examples across different GPT and human outputs to show you how the tool works
- There’s a Chrome extension.
The tool is free, and while a single scan doesn't do both, there's also a plagiarism detector.
It accurately identified much of the human-generated content as human and much of the AI-generated content as AI. However, it wasn't always confident and was incorrect in some individual sections.
4. OpenAI's AI Text Classifier
AI Text Classifier is a free tool from OpenAI, the company that created GPT (which is what runs many generative AI tools) and ChatGPT.
If you thought the company that makes the most popular AI tools would have the most accurate detector, that didn't ring true for our test.
The tool identified the human content as “very unlikely” to be AI-generated but identified the content from OpenAI’s own platform as “unclear” if it was AI-generated and “unlikely” AI-generated with and without the additional prompt.
The tool is free (you need an OpenAI account). There aren't a lot of bells and whistles and no plagiarism check.
You get a finding (like what you see in the screenshot) but no precise score. It also won’t highlight specific sections of copy for their individual likelihood of being AI-generated or not.
5. Crossplag AI Content Detector
Crossplag offers a few free scans without an account. When logged in, you'll get unlimited free scans. The tool does give an overall score but doesn't offer the ability to check for plagiarism or mark up individual sections of the content.
Crossplaq identified the human and AI content as 99% human. It is worth noting that the tool repeatedly calls out that it performs better on longer text.
Our samples here were above their 200-character minimum recommendation but on the shorter side.
6. GPTZero
GPTZero has free and paid versions. The free version gives you a score and has some pre-loaded examples.
The paid version also offers a plagiarism checker, highlighting AI content, and the ability to bulk upload multiple pieces of content (which is actually designed for a classroom environment, like many of these tools).
Pricing for the paid tools isn't listed on the website, but you can fill out forms to request access to the paid tool and the API.
The free version of the tool identified all three of our writing samples as likely entirely human.
7. Sapling AI Detector
Sapling AI Detector offers free and paid versions, a Chrome Extension, an API, an overall score and sentence highlighting.
Sapling identified the human content as mostly human and the AI content as almost entirely AI. However, it identified the “perplex” and “bursty” ChatGPT content as highly likely to be human.
8. Content at Scale's AI Detector
The Content at Scale AI Detector is free and offers a score with a breakdown of different elements within the content (predictability, probability, and pattern) and pre-populated examples.
No plagiarism detection option exists, and individual content sections aren’t highlighted.
Content at Scale rated all three samples as 92% human or higher.
The human did have the highest human percentage, followed by the prompt-modified GPT content and the content generated by the simplest prompt.
9. ZeroGPT
ZeroGPT is a free tool that offers an overall score and highlighting but no plagiarism detection.
The tool returned low percentages of AI-generated likelihood across the board. It roughly corresponded to the human-ness of the samples, with the human-generated sample getting the lowest AI percentage, followed by the burstiness / perplexity prompt, and then by the simpler prompt).
10. GLTR
GLTR, or the “Giant Language model Test Room,” is a free tool that was a collaborative effort between Hendrik Strobelt, Sebastian Gehrmann, and Alexander Rush from the MIT-IBM Watson AI lab and Harvard NLP.
The tool offers some overall scoring but mainly shows whether each word within a text was one of the 10, 100, 1,000 or outside the 1,000 most likely words to be generated by AI based on the prior context in the text. There is no plagiarism feature set here.
The tool returned the highest number of “likely words” and particularly likely words in the top 10 for the text generated by the simpler prompt.
Still, the distribution was similar for the human and additional prompt copy in this small test.
11. ChatGPT Detector on Hugging Face
This is a free detector that gives an overall prediction and score. It doesn’t have any plagiarism or highlighting features.
This tool generated over 99.9% likelihood of human content for all three samples.
12. Corrector AI Content Detector
Corrector AI Detector is a free tool with a 600-word count limit and a percentage score. There is no plagiarism or highlighting features for the tool.
The tool labeled each of the samples with a very low percentage probability of being “fake” or AI.
13. Writefull GPT Detector
The Writefull GPT Detector is a free tool with a simple score and API access. There is no highlighting and no plagiarism detection.
It found all three samples as likely human but did have the ChatGPT content with the simplest prompt with the highest AI likelihood (19%).
14. Hive Moderation’s AI-Generated Content Detection
The Hive Moderation AI-Generated Content Detection tool is free (with character limits and requiring sign-in after several uses).
It doesn't offer plagiarism or highlighting but does allow you to break content out by sections and get segment-specific scores.
Hive essentially scored each writing sample properly, with a 0% likelihood of being AI for the human content and 99.9% scores for the AI samples.
(Only Originality.AI had a similar confidence level while being accurate across the samples.)
15. Paraphrasing Tool AI Content Detector
The paraphrasing tool is a re-writer, and they offer this free AI content detector with an overall conclusion and highlighting and no plagiarism features.
The Paraphrasing Tool concluded that each of the text was likely human.
16. AI Writing Check
AI Writing Check is a free tool that offers a score, has a word maximum, and does not feature text highlighting for AI probability or plagiarism features.
AI Writing Check identified all three writing samples as human.
Key learnings from testing 16 AI and ChatGPT content checkers
Again, it’s essential to caveat my key findings here because three short writing samples are very small samples to draw steadfast conclusions about the individual tools.
That said, I found a few interesting patterns related to AI writing detectors in general:
- Calibration: Most tools found all three writing samples either highly likely or highly unlikely to be human. Different tools will likely be “harder” or “softer” graders when it comes to scoring content human versus AI, so understanding how a tool is calibrated can help determine how useful they are.
- Rarest features: A few rare features across the tools were things like bulk uploads, plagiarism detection, having a Chrome extension, and highlighting specific sections with likelihood of being AI versus human.
- Free vs. paid: All of the tools featured here but Originality.AI had at least free versions. But the tools with the most accurate results and most “rare features” tended to be the tools with a paid version.
I’d recommend keeping an eye on at least a few of these tools if you find AI detection useful in your projects.
The pace of innovation across AI content generation, editing and detection may quickly make your favorite tool obsolete.
The post 16 of the best AI and ChatGPT content detectors compared appeared first on Search Engine Land.
from Search Engine Land https://ift.tt/hN0TnYC
via
No comments:
Post a Comment