2026 · Python · PyTorch · Transformers
AI Text Detector
Explainable AI-text detection on Binoculars cross-perplexity — 100% AUROC on HC3, with a fairness audit.
- 100%
- Binoculars accuracy
- 1.000
- AUROC
- 5.5%
- Non-native FPR
- 75% acc
- vs. GPT-2 alone
Problem
Single-model perplexity detectors are both inaccurate and unfair — GPT-2 alone flags ~50% of human text as AI and penalizes non-native English writers at up to 26× the rate of native speakers. This tool uses multiple signals, shows its reasoning, and measures fairness explicitly.
Approach
- 01Binoculars cross-perplexity (gpt2 + distilgpt2) cancels topic bias for the primary verdict.
- 02GPT-2 perplexity and NLTK n-gram signals provide transparent secondary evidence.
- 03Ensemble mode returns a structured verdict: confidence, per-signal metrics, and narrative reasoning.
- 04Local-first — no required external APIs; Dockerized for reproducibility.
Architecture
- Text input
- Binoculars · gpt2 + distilgpt2 cross-perplexity
- GPT-2 perplexity + NLTK n-gram signals
- Ensemble verdict + reasoning
- Streamlit UI · Dockerized
Challenges
- Fairness: measuring and reporting false-positive rates by population, not just headline accuracy.
- Explaining a probabilistic verdict instead of emitting an opaque score.
Key features
- Binoculars cross-perplexity detection
- Per-population fairness evaluation
- Structured verdict + narrative reasoning
- Local-first, no external APIs required
- HC3 reproducibility as a test gate
Metrics are on the HC3 benchmark (n=200 balanced); the README documents its limits and ethical constraints.