AI detectors often present a percentage that looks precise, but the important question is simpler: when the detector says a passage is AI-written, how often is it actually right? To test GPTInf, we ran a dataset of 160 passages through its detector. The dataset contains 82 AI-written texts and 78 human-written texts. The Written By column is the ground truth, while GPTInf's prediction and AI Score were recorded separately.
How the GPTInf test was set up
The test was intentionally close to a real use case. Each passage already had a known source label: Human or AI. GPTInf then produced two outputs: a categorical verdict and an AI Score between 0 and 1. A score of 1 means the detector considers the text strongly AI-like.
GPTInf's categorical result in this dataset follows a simple rule: scores above 0.50 were labeled AI, while scores of 0.50 or below were labeled Human. That makes it possible to examine not only whether the final answer was correct, but also how the confidence scores behaved across both groups.
Also Read: Is Sapling AI detector accurate? A data-backed look at 160 samples
GPTInf achieved 57.5% overall accuracy
Out of 160 samples, GPTInf classified 92 correctly and 68 incorrectly. That produces an overall accuracy of 57.5%. A 95% Wilson confidence interval for the observed accuracy is approximately 49.8% to 64.9%, which is a useful reminder that this is a finite test rather than a universal accuracy figure.
| Metric | Result |
|---|---|
| Overall accuracy | 57.5% |
| AI recall | 89.0% |
| Human specificity | 24.4% |
| Precision when GPTInf says “AI” | 55.3% |
| F1 score for AI detection | 0.682 |
| Balanced accuracy | 56.7% |
| ROC AUC using AI Score | 0.760 |
The raw confusion matrix explains the weakness more clearly than the headline accuracy. GPTInf correctly detected 73 of 82 AI texts, but it also labeled 59 of 78 human texts as AI. Only 19 human samples were correctly cleared as human.
Also Read: [STUDY] How accurate is GPTZero? - An Independent Analysis!
The biggest problem is false positives
For an AI detector, false positives matter a lot. A false positive happens when genuinely human writing is labeled as AI-generated. In this test, GPTInf produced 59 false positives. That means roughly 75.6% of the human-written texts were flagged as AI.
This creates a major practical issue. If a teacher, editor, recruiter, or publisher treated GPTInf's result as proof, many genuine human writers in this dataset would be accused incorrectly. The detector's AI recall is strong at 89.0%, but that strength comes with a very aggressive tendency to call text AI-written.
Also Read:[HOT] Is JustDone AI Detector Accurate?
The AI Score separates the groups better than the final verdict suggests
The score itself contains more information than the binary label. AI-written samples had an average AI Score of 0.885, while human-written samples averaged 0.730. That difference is meaningful, but the distributions still overlap heavily.
Some human texts receive scores near 1.0, while some AI texts fall near zero. This overlap is exactly why no single threshold can perfectly separate the two classes. The ROC AUC of 0.760 indicates that the score has useful ranking ability: if one AI passage and one human passage are selected at random, the AI passage will usually receive the higher score. But ranking ability is not the same as dependable classification at the chosen cutoff.
Also Read: [SHORT STUDY] How accurate is Turnitin? - An Independent Analysis
GPTInf's 0.50 threshold appears too aggressive for this dataset
Because every score above 0.50 is treated as AI, GPTInf prioritizes catching AI content over protecting human content from false accusations. Raising the cutoff changes that trade-off. In this particular dataset, thresholds close to 0.99 produce much higher measured accuracy, around 76.9%, because many human passages with moderately high scores stop being classified as AI.
That does not mean 0.99 is automatically the “correct” threshold. Choosing a threshold after looking at the same test data can overfit the benchmark. A new independent dataset would be needed to confirm whether a stricter cutoff generalizes. Still, the threshold experiment demonstrates that GPTInf's current binary decision is not making the best use of its own score on this sample.
There are also clear individual misses
The screenshots illustrate why the aggregate numbers matter. GPTInf can be extremely confident on some AI-written passages, assigning a 100% AI-style result, while another AI-generated passage in the dataset about quantum tunneling received an AI Score of only 0.10 and was classified as human. A detector can therefore look convincing on selected examples while still failing on other samples generated by AI.
That variability is important because users rarely know the ground truth when they use an AI detector. The percentage shown by the interface may look like a probability, but it should not automatically be interpreted as the literal probability that a person used AI.
So, is GPTInf's AI detector accurate?
On this 160-sample test, the answer is not accurate enough to rely on as a standalone judge. Its 57.5% overall accuracy is only part of the story. The detector finds most AI-written content, but it achieves that by flagging a very large share of human writing as AI. When GPTInf predicted “AI” in this dataset, it was correct only 55.3% of the time.
The more encouraging result is that GPTInf's continuous AI Score has some real separation ability, as shown by the 0.760 ROC AUC. That suggests the underlying signal is more useful than the default binary classification makes it appear. However, a detector used in high-stakes settings needs low false-positive rates, robust testing across writing styles, models, editing levels, languages, and domains, and validation on data that was not used to tune the cutoff.
Final verdict
GPTInf should be treated as a signal, not evidence. In this dataset, it is sensitive to AI writing but far too willing to label human writing as AI. That makes it potentially useful as a first-pass screening tool, but risky as the basis for accusations, grading decisions, disciplinary action, or authorship claims. If the goal is to determine whether a specific piece of text was written by AI, the data here shows why the detector's percentage should be interpreted with caution.