Why Pangram's AI Detection Is Brittle, Not Broken
I wouldn't say Pangram is broken, but I would say that it's brittle
After being accused of using AI, I tested Pangram and found it produces contradictory results depending on context. A section flagged as 100% AI becomes human when analyzed alone, yet the full essay is deemed human. The tool's percentage meter seems broken, often forcing binary outcomes instead of reflecting the actual mix of human and LLM text. This unreliability makes it dangerous for professional consequences.
Pieces that Pangram call 100% human written are part of a larger piece that Pangram calls 100% AI written which in turn is part of an essay that Pangram calls 100% human written, a Russian doll of contradictory results.