AI detectors are often used in schools, colleges, and student communities, but the important question is simple: can we actually trust them? To answer that, I tested Pangram on a dataset where the real writer was already known. Each text was marked as either written by AI or written by a human, and then Pangram's verdict was compared with that truth.
The short answer from this test is that Pangram looked reliable overall, but it was not perfect. It made very few false accusations against human writing, which is a big plus for students. The bigger weakness was that it sometimes missed AI-written text and called it human.
Also Read: [STUDY] Is Pangram AI Detector Similar to Turnitin?
What was tested?
The dataset had 160 writing samples. Out of these, 82 were AI-written and 78 were human-written. That is a fairly balanced dataset, which makes the test more useful because one side does not completely dominate the result.
- Total samples: 160
- AI-written samples: 82
- Human-written samples: 78
- Correct Pangram verdicts: 149
- Wrong Pangram verdicts: 11
Also Read: StealthWriter vs Pangram: Can a "Humanized" Rewrite Really Escape Detection?
The main result
Pangram reached an overall accuracy of 93.1%. In simple words, it was correct about 93 times out of every 100 samples in this test. That is a strong result, especially for a tool that is trying to judge writing style instead of checking a fixed fact.
But students should not confuse high accuracy with perfection. A detector can be good on average and still be wrong for a specific essay. This matters because one incorrect result can create a serious problem for a student if the result is treated as final proof.
Also Read: [STUDY] Can Pangram Detect Undetectable AI?
| Metric | Result | What it means |
|---|---|---|
| Overall accuracy | 93.1% | How often Pangram was correct overall |
| AI precision | 98.6% | When Pangram said AI, how often it was right |
| AI recall | 87.8% | How many AI-written samples Pangram caught |
| Human clear rate | 98.7% | How many human-written samples Pangram correctly cleared |
Where Pangram did well
The most impressive part of this test was Pangram's low false positive rate. A false positive means a human-written sample gets wrongly called AI. In this dataset, that happened only 1 time out of 78 human samples.
That matters a lot for students. A tool that constantly accuses real human writing of being AI-generated would be dangerous in academic settings. In this test, Pangram was much better at avoiding that problem than many students might expect.
- Pangram correctly cleared 77 out of 78 human-written samples.
- It wrongly flagged only 1 human-written sample as AI.
- Its false positive rate was only 1.3%.
Where Pangram struggled
Pangram's main weakness in this dataset was missing AI-written text. Out of 82 AI samples, it correctly detected 72, but missed 10. That gives it an AI recall of 87.8%.
This means Pangram was strong, but some AI content still slipped through. For students, this is an important point. Passing an AI detector does not automatically prove that a text is human-written. It only means the detector did not find enough signs to label it as AI.
- False negatives: 10 AI-written samples were marked as human.
- False negative rate: 12.2% of AI samples were missed.
- The missed AI samples usually received very low Pangram scores in this dataset.
What the score pattern showed
The Pangram scores in this file were not spread across many values. They mostly appeared as very strong signals. A score of 100 matched Pangram's AI verdict, while low scores matched Pangram's human verdict. This made the detector's decision style look quite strict in this dataset.
Most AI-written samples received a score of 100, and most human-written samples received a score of 0. This explains why the overall accuracy was high. The detector was usually very confident in one direction or the other.
Did text length affect the result?
I also checked whether Pangram made more mistakes on shorter or longer texts. The error rate did not show one simple rule. Texts between 301 and 500 words had the most errors by count, but that is also where many samples were located. The over 500 word group had no errors, but it only had 8 samples, so we should not overread that result.
So, is Pangram reliable?
Based on this dataset, Pangram looks reliable as a screening tool. It got a high overall accuracy, it was very good at avoiding false accusations against human writing, and when it said something was AI, it was almost always correct in this test.
However, it should not be treated as a final judge. The detector still missed 10 AI-written samples, and it wrongly flagged 1 human-written sample. That means the safest conclusion is not "Pangram is always right." The better conclusion is: Pangram performed strongly in this test, but its verdict should be used with context.
Final verdict
My answer is: yes, Pangram looked reliable in this test, but not perfect. Its biggest strength was protecting human writing from false AI labels. Its biggest weakness was that some AI-written samples passed as human. For students, that means Pangram can be helpful, but it should be treated as a signal, not as unquestionable proof.