Retrieval-augmented answers are only as good as what they retrieve. That sounds obvious, yet when a RAG system gives wrong answers, almost every team responds the same way: try a bigger model, swap the embedding, add a reranker. The one thing they don't do is open the knowledge base and read it.
What the corpus actually looks like
Open it and you'll usually find three versions of the same policy with conflicting numbers, documents that were superseded two years ago but never deleted, walls of text with no headings a retriever could anchor to, and — the killer — no owner. Nobody is responsible for keeping any of it true. A retriever pointed at that corpus is doing its job perfectly: faithfully surfacing your mess.
Model work versus corpus work
Model tuning is attractive because it feels like engineering; corpus cleanup feels like janitorial work. But the returns are lopsided. On a retrieval Q&A project for a subscription language-learning app, we didn't touch the model at all. We collapsed three parallel generations of course documents into one owned, versioned knowledge base with answerable, well-titled sections — and only then did the wrong-answer rate genuinely fall.
Audit the corpus first, tune the model second
Audit the corpus before touching the stack: deduplicate, kill stale versions, restructure long documents into sections that each answer one question, attach metadata for freshness and audience, and name an owner for every document class. Only after that do model and chunking experiments produce signal instead of noise.
A RAG system is a mirror. Before you blame the reflection, tidy the room.
