2026 · Python · FastAPI · FAISS
RAG Pipeline
Production RAG with a guardrail gate, an evaluator agent, and a reproducible RAGAS harness.
- ~9.3s
- Mean latency · n=22
- ~12.5s
- p95 latency
- passing
- Docker /health
- 22 grounded
- Eval dataset
Problem
LLMs hallucinate when they answer from parametric memory. This pipeline grounds every answer in retrieved evidence and makes quality measurable instead of asserted — quality gates, not vibes.
Approach
- 01Token-aware overlapping chunking → bge-large-en-v1.5 embeddings → FAISS index with MMR diversity re-ranking.
- 02A guardrail gate screens retrieved context before generation; an evaluator agent scores answer consistency after.
- 03A RAGAS harness runs against a fixed 22-sample grounded dataset so faithfulness/recall checks are reproducible.
- 04OpenTelemetry traces, metrics, and logs cover the whole request path.
Architecture
- Documents → token-aware chunker
- Embedder · bge-large-en-v1.5
- FAISS index + MMR re-ranking
- Guardrail gate
- Generator · Groq llama-3.3-70b
- Evaluator agent → scored answer
Challenges
- Keeping evaluation honest — the RAGAS run is a fixed, reproducible harness rather than a cherry-picked number.
- Containerizing a heavy embedding stack with multi-stage builds and non-root execution.
Key features
- Guardrail + evaluator agent gates around generation
- MMR diversity re-ranking over FAISS
- OpenTelemetry structured observability
- FastAPI service: /health, /query, streaming
- Docker multi-stage, non-root deployment
Latency reflects sync-evaluator mode on a free Groq tier — the evaluator gate deliberately trades speed for a consistency score.