Build a Citation-Aware RAG Pipeline

Define the knowledge boundary

List what the system is allowed to know, how fresh that information must be, and which sources are authoritative. Store document ID, title, canonical URL, owner, version, publication date, access class, and section path with every chunk. Citations cannot be reconstructed reliably after provenance has been discarded.

Chunk for meaning and traceability

Start with structure-aware chunks based on headings, paragraphs, tables, or code blocks. Preserve enough neighboring context to interpret a passage, but avoid huge chunks that blur multiple topics. Measure chunk length and overlap; do not treat defaults as facts.

For a small approved corpus, create questions with known supporting passages. Inspect the retrieved top results before generating an answer. If evidence is absent, changing the answer prompt will not repair retrieval.

Constrain the answer

Pass only approved retrieved context into the answer step. Ask for a direct answer, a citation after each material claim, and an explicit “not supported by the available sources” response when evidence is insufficient. Validate that cited document IDs were actually retrieved.

Separate retrieved text from instructions and treat documents as untrusted input. A document that says “ignore previous instructions” is content, not policy.

Practice: ten-document RAG audit

Use ten rights-cleared documents. Create five answerable questions, two ambiguous questions, and three unanswerable questions. Save retrieved chunk IDs, scores, answer, citations, latency, and cost. Manually verify every citation and record whether the cited passage truly supports the claim.

Acceptance criteria

A useful prototype retrieves the right evidence, cites it accurately, declines unsupported claims, respects access boundaries, and exposes enough trace data to diagnose failures.