Evaluating Retrieval Quality in RAG Systems
A lab‑tested playbook for measuring RAG retrieval: golden sets, Hit@k/MRR/NDCG, chunking choices, hybrid search, and re‑ranking.
By AI Mastermind Lab · Published 8 Mar 2026
Most RAG failures are retrieval failures. We built a lightweight harness and a golden dataset to measure improvements objectively.
Metrics: Hit@k (coverage), MRR (ranking), and NDCG (graded relevance). Track all three; they reveal different issues.
Chunking: 256–512 tokens with small overlap is a strong default; increase for legal/policy docs; decrease for FAQ‑like content.
Hybrid search: combine BM25 with vector similarity to capture names/numbers; then re‑rank top‑k using a cross‑encoder for precision.
Results: hybrid + re‑rank consistently outperformed pure vector search in our tests, especially on name/number queries.