Why Do AI Detectors Have Different Results?

Paste the same paragraph into two AI detectors and you might get two very different answers. One might call it human-written with a score of 0%. Another might flag almost the entire passage as AI-generated. Neither result automatically tells you who wrote it.

The disagreement is not just a software glitch. AI detectors use different models, look for different patterns, and often mean different things by the percentages they display. To see how much that matters, we compared six detectors on the same 100 pieces of text and examined the published research behind several approaches.

TL;DR: Why do AI detectors disagree?

• They do not all use the same technology. Some examine token probabilities; others use trained classifiers, multiple models, or different combinations of language patterns.
• They were trained on different writing. A detector familiar with one generator, language, or writing genre may struggle with another.
• A percentage is not a universal measurement. A confidence-like score from one service is not necessarily comparable with a percentage of text identified by another.
• They use different thresholds. Two tools can assign the same underlying risk score yet display opposite labels.
• Our test showed widespread disagreement. With a common, deliberately artificial 50/100 boundary, 81 of 99 complete texts received conflicting high/low classifications from at least two detectors.

That last statistic is about agreement, not accuracy. We do not know which detector was right because this dataset does not include independently verified authorship labels.

We compared six AI detectors on 100 identical texts

Our dataset contains results from QuillBot, Turnitin, Originality.ai, Pangram, ZeroGPT, and Sapling. The pieces range from 53 to 455 words, with a median length of 139.5 words. Every text has six scores except one, which is missing a QuillBot result.

For a straightforward comparison, we put the stored scores on a nominal 0–100 scale, multiplying the Pangram column by 100 because it was recorded from 0 to 1. We then called a score high when it was at least 50 and low otherwise. This is our analysis rule, not any company's recommended or official cutoff.

Also Read: Which AI Detectors That Support PDF Files?

1. How often did each detector return a high score?

Pangram's recorded field crossed the common boundary on 93% of samples. QuillBot and ZeroGPT did so on roughly a quarter of samples. Originality.ai fell between those extremes at 45%, and Sapling at 53%. That gap is striking, but it would be misleading to conclude that Pangram is more sensitive or QuillBot is more accurate without knowing the intended meaning of every field and the real authorship of each text.

Also Read: What Is AI Residue? Definition, Examples, and Why It's Spreading

2. Which detectors agreed most often?

QuillBot and ZeroGPT agreed on 87.9% of the 99 texts where both had scores. Pangram and ZeroGPT agreed on only 28% of their 100 shared texts. The average across all 15 unique detector pairs was approximately 62.8%.

Agreement is useful for spotting consistent behavior, but it is not an accuracy metric. If two detectors learned a similar blind spot, they could agree frequently and still be wrong together.

Also Read: Free AI Detectors With No Word Limit: What Actually Works in 2026?

3. How many detectors agreed on a single document?

Only 18 of the 99 complete texts had a unanimous classification: four were low across all six tools, and 14 were high across all six. The other 81 texts, or 81.8%, had at least one conflicting result.

There were 29 texts where exactly one detector returned a high score and 25 where exactly two did. These are the sorts of situations that make a single AI score frustrating for a student or writer: the answer changes depending on which tool they open.

Also Read: What Is SynthID in Gemini? The Invisible Signature Behind Google's AI Content

4. What does disagreement look like on individual samples?

Consider CSV row 48: Pangram's recorded score is 100, while QuillBot, Turnitin, Originality.ai, ZeroGPT, and Sapling all fall below 50. Compare that with row 20, where all six scores are high. Those two cases are not evidence that any particular tool is right; they show how differently the products can respond to the same words.

One important limitation: 93 of the 100 texts are shorter than 300 words in the CSV, whereas Turnitin's current standard AI Writing Report requires at least 300 words of qualifying prose. We cannot establish how the stored Turnitin values were obtained. The results may reflect another workflow or model version. They should not be presented as a fresh, reproducible benchmark of Turnitin's current standard interface.

What are AI detectors actually looking for?

An AI detector cannot look inside a paragraph and find a hidden tag saying “written by ChatGPT.” In most cases it estimates authorship from patterns in the text. Researchers have designed several ways to make that estimate, and they do not all ask the same question.

Perplexity: how predictable is the next word?

Perplexity describes how surprising a passage is to a particular language model. A model might consider “The results were significant” a fairly predictable continuation and a strange, highly specific turn of phrase less predictable. Some detection approaches test whether a passage is unusually easy for a model to predict.

The catch is that perplexity depends on which model does the predicting. The same passage can have different perplexity under different models and tokenizers. Predictable writing is not necessarily AI writing, either. Technical reports, classroom answers, and simple explanations often repeat familiar phrases.

The earlier research tool GLTR visualized word probability ranks to help people examine generation patterns. It is an example of a statistical approach, not a description of how every modern commercial detector works.

Burstiness, vocabulary, and writing style

Some tools examine how language varies across a passage: sentence lengths, repeated structures, word choice, formality, and changes in predictability. “Burstiness” is a loose term often used for this variation. A paragraph with uniformly structured sentences may look different from one that swings between short remarks and longer explanations.

These signals are fragile on their own. A careful human writer can be consistent, while an AI model can produce deliberately varied sentences. A style pattern is evidence to consider, not proof of authorship.

Trained classifiers: learning from examples

Other detectors use supervised machine learning. They train a model on samples labeled as human-written, AI-generated, or sometimes mixed, and the model learns a decision boundary between those groups.

This is a different task from simply calculating perplexity. A classifier can learn combinations of word choice, syntax, context, phrasing, and longer-range patterns without its developers explicitly writing a rule for each one.

But the examples matter. A classifier trained mostly on English essays and outputs from a handful of generators may interpret an unfamiliar genre or a newly released model differently. Two detectors using the same Transformer architecture can still learn different boundaries because they trained on different data.

Probability-based research methods: DetectGPT and beyond

DetectGPT takes a more specific approach. It checks how the log probability of a passage compares with randomly perturbed versions of the same passage. The researchers found that machine-generated text often has a distinctive local probability-curvature pattern under a related language model.

Fast-DetectGPT replaces the costly perturbation procedure with a more efficient conditional-curvature calculation. Binoculars compares evidence from two language models. All belong to the broad statistical family, but their scoring rules are different. Two “zero-shot” detectors can therefore give different answers without either contradicting its own method.

Watermarks: a different kind of signal

A watermark is not an ordinary writing-style detector. In the watermarking method proposed by Kirchenbauer and colleagues, the generator subtly favors certain tokens according to a secret or keyed rule. A matching detector can later test for that statistical signature.

That works only when the generating system actually used the relevant watermark and the detector knows how to test for it. An absence of a watermark does not establish that a passage was human-written. A generic commercial detector should not be assumed to rely on watermarking unless its documentation says so.

Do Turnitin, QuillBot, ZeroGPT, and other detectors use the same methods?

No. They share the broad aim of detecting generated writing, but their published descriptions show differences in architecture, signals, and how results are produced. Important details are proprietary, so descriptions below distinguish what the companies disclose from what remains unknown.

The broader lesson is that the term AI detector covers several related technologies, not a standardized instrument. Commercial methods also change over time. A comparison that fails to record the exact model version may become outdated even when all the product names stay the same.

Seven reasons the same text produces different results

1. Different training datasets

Training a classifier on polished student essays gives it a different view of human writing from training it on casual forum posts. The generated side matters just as much: outputs from different models, prompts, languages, and sampling settings leave different traces. Even a carefully trained model can struggle when its next input comes from outside that familiar distribution.

The RAID benchmark was built partly to expose this problem. It tested detectors across many generators, domains, decoding strategies, and adversarial changes, finding that detector performance can shift substantially under conditions unlike a vendor's normal test set.

2. Different thresholds

Suppose two systems estimate the same numerical likelihood, 0.62. One treats anything above 0.50 as suspicious; another requires 0.75 before producing a positive label. The underlying estimate is identical, but their verdicts are opposite. This can be deliberate: a service used for initial screening may favor catching more cases, while one intended for high-stakes decisions may prioritize avoiding false accusations.

3. Different meanings of a percentage

Does 70% mean a 70% probability that the passage was generated? Does it mean 70% of the words were classified as generated? Or is it an index derived from several model signals? Those are different quantities.

Turnitin's percentage is tied to the share of qualifying text identified as likely AI-related in its report. Pangram documents distinct probability, class-fraction, and assistance measures. Sapling's document-level score should not be casually equated to a count of generated words. Comparing bare percentages across services can be like comparing degrees to kilometers because both happen to be numbers.

4. Different treatment of mixed or edited writing

Most real editing does not fit neatly into “all human” or “all AI.” A student may write the argument, use a grammar checker, rewrite sentences manually, and ask a chatbot to suggest a conclusion. Some detectors try to identify AI-assisted sections. Others compress the whole passage into one score.

Version changes can matter here. Originality.ai 5.0 explicitly distinguishes original, mixed, and likely AI writing. Pangram 4 assigns different provenance labels to text spans. Those differences in what is being classified can create different outputs on the same mixed document.

5. Different text lengths and preprocessing

A detector that receives 70 words has less evidence than one receiving an entire essay. Some interfaces also truncate long inputs. Document uploads introduce further differences: PDF extraction may change punctuation, spacing, paragraph order, or line breaks, and references or tables may be filtered differently.

Even before model inference, one tool may receive a different token sequence from another. Pangram's model card describes overlapping windows and Unicode normalization; Sapling's free detector currently limits how much text is checked in one submission.

6. Different languages and writing backgrounds

Simple language and repetitive sentence structures can occur naturally, especially in technical writing or writing by someone working in a second language. A 2023 study of GPT detectors and non-native English writers found that the detectors evaluated at that time were considerably more likely to flag writing by non-native speakers. That does not prove every current detector shares the same bias, but it shows why a fair benchmark must include diverse authors and types of writing.

7. Different models, versions, and updates

An AI detector is not a fixed formula. Vendors retrain models, change thresholds, add languages, and improve handling of rewritten text. Turnitin maintains model-release guidance, and other services publish model cards or changelogs. A score generated months ago cannot automatically be treated as the score the latest service would produce.

If all AI detectors used the same method, would the results be identical?

Not necessarily. Using the same general method is not the same as running the same implementation.

Imagine two teams both build a “perplexity-based” detector. One uses a small reference model and another uses a larger model with a different tokenizer. Their perplexity values can differ because the language models assign different probabilities to the same sequence. Even with the same reference model, differences in context length, punctuation normalization, averaging, and cutoffs can alter the final score.

The same applies to learned classifiers. Two systems built on the same Transformer architecture are not identical after they have been trained on different datasets or initialized and tuned differently. One could consider a passage borderline AI while another sees a familiar pattern of human writing.

Some research algorithms introduce randomness too. For example, DetectGPT samples perturbed versions of the text, so different perturbations can change the estimate unless the experiment controls the randomness.

There is, however, a meaningful sense in which results should match. Give two deterministic systems exactly the same input bytes, normalization rules, tokenizer, model weights, processing windows, scoring formula, calibration, threshold, and numerical conditions, and their outputs should ordinarily agree within numerical tolerance. In that case they are effectively executing the same detector, not merely adopting the same idea.

Does disagreement mean AI detectors are inaccurate?

Disagreement alone cannot answer that question. If one detector says “AI” and five say “human,” the minority might be wrong, the majority might be wrong, or the passage might sit near a difficult decision boundary. Without independent ground truth, voting is not a test of correctness.

To compare reliability, a benchmark needs known human and generated texts, mixed cases, detector versions, and the actual thresholds used. It should measure at least two types of mistakes:

• False positives: genuine human writing incorrectly flagged as generated.
• False negatives: generated writing incorrectly identified as human.

A tool that almost never flags anything may have few false accusations but miss large amounts of generated text. Another may catch nearly every AI passage while accusing too many human writers. A useful evaluation examines both, along with how results change across language, genre, text length, and editing.

This is especially important in schools. Turnitin's own guidance says its report should not be the sole basis for action against a student. That principle holds when reports conflict, too. A detector is a statistical screening tool; it does not witness the writing process.

How should you interpret conflicting AI detector results?

If your work receives dramatically different scores, first check whether you pasted precisely the same text into every tool. Then look at the length of the passage, the detector's supported use case, the model version, whether the number represents confidence or text composition, and whether the product labels partial AI assistance separately.

Do not assume the tool giving the highest score is stricter, smarter, or more trustworthy. And do not assume that several agreeing tools create proof. A draft history, notes, sources, version records, and an explanation of how a document was written can provide more direct context about authorship than competing classifier outputs.

The comparison here demonstrates a narrower but important point: six tools presented with the same samples can draw very different boundaries. The research explains why. Different data, different detection signals, different scoring rules, and different product decisions mean there is no single universal “AI percentage” waiting to be discovered.

The same text can produce different AI detector results because the detectors are making different statistical judgments. Identical broad methods do not guarantee identical predictions, and disagreement is a reason to investigate the evidence—not a reason to treat whichever percentage is most dramatic as the truth.