Retrieval, RAG, and Citation Discipline

A RAG pipeline has two jobs

First retrieve relevant evidence; then answer only from that evidence. Chunk size, overlap, metadata, embedding model, filter policy, retrieval depth, and reranking each affect recall and precision. Do not debug them by looking only at final prose.

Evidence-first answers

Persist document ID, title, section, version, and retrieval score with every excerpt. Make the model cite these IDs. If evidence is missing or conflicting, the product should say so and ask a narrower question instead of fabricating a confident answer.

Evaluation set

Create questions with known sources, deliberately absent answers, ambiguous wording, outdated versions, and adversarial instructions in a document. Measure retrieval recall, citation correctness, unsupported-claim rate, and refusal quality separately.

Practice: source card

For ten documents, create metadata cards with owner, date, access level, retention policy, and update source. Test that a user cannot retrieve a document outside their permission scope.

Common mistake

A high similarity score is not proof that a passage supports an answer.