All work

2026 · Python · PyTorch · Transformers

AI Text Detector

Explainable AI-text detection on Binoculars cross-perplexity — 100% AUROC on HC3, with a fairness audit.

Results
100%
Binoculars accuracy
1.000
AUROC
5.5%
Non-native FPR
75% acc
vs. GPT-2 alone

Problem

Single-model perplexity detectors are both inaccurate and unfair — GPT-2 alone flags ~50% of human text as AI and penalizes non-native English writers at up to 26× the rate of native speakers. This tool uses multiple signals, shows its reasoning, and measures fairness explicitly.

Approach

  • 01Binoculars cross-perplexity (gpt2 + distilgpt2) cancels topic bias for the primary verdict.
  • 02GPT-2 perplexity and NLTK n-gram signals provide transparent secondary evidence.
  • 03Ensemble mode returns a structured verdict: confidence, per-signal metrics, and narrative reasoning.
  • 04Local-first — no required external APIs; Dockerized for reproducibility.

Architecture

  1. Text input
  2. Binoculars · gpt2 + distilgpt2 cross-perplexity
  3. GPT-2 perplexity + NLTK n-gram signals
  4. Ensemble verdict + reasoning
  5. Streamlit UI · Dockerized

Challenges

  • Fairness: measuring and reporting false-positive rates by population, not just headline accuracy.
  • Explaining a probabilistic verdict instead of emitting an opaque score.

Key features

  • Binoculars cross-perplexity detection
  • Per-population fairness evaluation
  • Structured verdict + narrative reasoning
  • Local-first, no external APIs required
  • HC3 reproducibility as a test gate

Metrics are on the HC3 benchmark (n=200 balanced); the README documents its limits and ethical constraints.