Build a reliable, citable RAG knowledge base
Connecting AI to documents does not automatically make answers accurate. Quality depends on corpus governance, permissions, chunking, retrieval, citations and tests that include questions with no answer.

The short answer
Start with a limited corpus and a named owner. Remove stale versions, retain source metadata and permissions, test retrieval separately from generation, display cited passages and teach the system to say when the corpus is insufficient.
- Govern the corpus
- Test retrieval separately
- Make citations verifiable
Turn files into usable knowledge

AI-generated illustration 1. Define the scope
List users, allowed questions, included documents and excluded decisions. Assign a corpus owner and an update frequency.

AI-generated illustration 2. Prepare sources
Remove duplicates and obsolete versions; retain title, author, date, language and access level. Check extraction from tables, notes, scans and attachments.

AI-generated illustration 3. Enforce permissions
Retrieval must never bypass document rights. Apply controls before retrieval, log useful access events and test with several user profiles.

AI-generated illustration 4. Tune retrieval
Select chunking, metadata, lexical or vector search and passage count for the documents. First measure whether the right passages are found, independently from final prose.

Editorial illustration 5. Produce a traceable answer
Show source title, version and supporting passage. Separate quotation from synthesis, prohibit invented references and return an explicit insufficiency message when evidence is weak.

AI-generated illustration 6. Evaluate and maintain
Test simple, ambiguous, contradictory, outdated and out-of-scope questions. Track retrieval precision, supported answers, permission failures, freshness and correction time.
Diagnose an answer citing the wrong passage
Separate three checks. Does the original contain the answer? Did conversion and chunking preserve it with its heading and reference? Did the system retrieve the passage and answer faithfully? Record each stage rather than changing the model when a lost table caused the import problem.
Test case: an exception in a footnote
Write a general rule and a footnote exception in a fictional document. Ask a question that depends on the exception. Keep the document, converted text, retrieved chunks and answer. Reject an answer citing the rule but omitting the exception, even if its link opens the right file. Use the decision record to document the fix and retest.
Set the unit before counting
Decide whether you are evaluating entire documents or chunks. Several chunks from one document do not count as several retrieved documents. Build a small corpus with known relevant items and keep the same result limit across candidates. The precision and recall calculator below separates irrelevant hits from missed documents; it does not grade generated answers.
Check the file and then the chunk
Prepare two versions of the same document and one file with the wrong extension. Check that ingestion identifies the format, rejects or reroutes the ambiguous case, and passes each chunk’s identifier, version and section downstream. Verify those fields in retrieval results before evaluating generated text.
Put the method to work

Practical case
Build a small corpus of twenty dated documents, including two that conflict and one restricted by user role.

Evidence to keep
Test passage retrieval, citation attribution, access rights and replacement of an outdated document.

Make the decision
Expand the corpus only when answers lead to the right passage and the restricted document remains hidden.
Three questions exposing incomplete retrieval
Prepare these fictional documents in an isolated test corpus. The outputs below are editorial expectations, not measurements of a tool.

Rule and exception
- Input to prepare
- A rule permits exchanges within thirty days; a separate paragraph excludes personalised products. Ask about a personalised item.
- Expected result
- The answer cites both passages and identifies the applicable exclusion.
- If the check fails
- Inspect retrieved fragments. If the exception is missing, repair retrieval before changing wording.

Obsolete version
- Input to prepare
- Two documents share a title: an older fifteen-day version and an active thirty-day version. Add validity metadata.
- Expected result
- The answer uses the active version and identifies its cited document.
- If the check fails
- Check validity filtering and removal of stale fragments. Do not rely on the model to guess the version.

Absent answer
- Input to prepare
- Ask about international delivery when the corpus only describes local exchanges.
- Expected result
- The system states that information is missing and what requires another source.
- If the check fails
- Keep this as an abstention case; reject an answer inventing a policy.
Decision rule. Accept the workflow only when each decisive claim links to current, authorised evidence. Replay all three cases after changing fragmentation.
Four layers to control

Corpus
Are documents correct, current, readable and attributed?

Access
Does each person retrieve only material they may view?

Retrieval
Do useful passages rank above merely similar passages?

Answer
Is every material claim supported and easy to verify?
Document tools, platforms and local options
These families cover source-grounded notebooks, enterprise platforms and local components. Verify connectors, permissions, data residency and citation mechanisms.
Docling
document conversion
Docling project
Visit official siteUnstructured (bibliothèque)
document conversion
Unstructured
Visit official siteMagika
file type identification
MarkItDown
document conversion
Microsoft
Visit official siteAcrobat AI Assistant
source-grounded research notebook
Adobe · US
Visit official siteOllama
local models
Ollama · US
Visit official siteHow is this selection produced?
These profiles are linked to the guide’s subject in our editorial catalogue. Inclusion is neither a ranking nor a performance measurement on your documents. Methodology.
Related tool families
Frequently asked questions
How does RAG differ from training?
RAG retrieves passages at answer time without necessarily changing the model. Training adjusts behaviour or parameters and requires a different data and evaluation lifecycle.
Is a vector database always required?
No. Lexical search can work better for exact references. Hybrid retrieval is often useful, but it should be justified by tests on the real corpus.
How should expired documents be handled?
Keep a validity date, owner and archival rule. Remove the version from the active index while retaining records when documentary obligations require them.
The references below expand on the concepts and checks discussed. Scenarios and trial frameworks remain editorial proposals; provider documentation describes its own product rather than an independent benchmark.
Source note
Judge a RAG knowledge base by the passages retrieved
In RAG, answers first depend on indexed documents and passages retrieved for the question. Microsoft describes retrieval, context augmentation and generation. Index documents with identifiable titles, versions and URLs so each answer can be traced to the source actually used.
Test representative questions, including questions the knowledge base cannot answer. Check the passage’s relevance before answer fluency. The model can err even with context; clickable citations and abstention when evidence is insufficient help review.
Separate two possible failures: the correct passage was not retrieved, or it was available but misused in the answer. Keep expected and retrieved passages for each question. This avoids changing model instructions when the defect lies in the index or corpus.
Include questions involving a precise date, an exception and missing information. Check conflicting document versions: identify the version used and disclose the conflict instead of merging them. For private corpora, verify access rights during retrieval.
- Corpus
- Current documents, access rights and source identifiers.
- Retrieval
- Relevant passages for representative questions.
- Answer
- Verifiable citations and abstention when sources are missing.
Is a displayed citation enough to guarantee an answer?
Open the passage and check whether it supports the exact claim, with its conditions and date. A reference can be relevant to the topic without proving the conclusion.
Try the reasoning
The correct answer exists in the RAG corpus but no relevant passage is retrieved. Where should the defect be isolated?
Reasoning: Test retrieval separately from writing. An incorrect answer can originate in indexing, retrieval or generation.
Primary source : Microsoft Learn — Retrieval augmented generation and indexes
Official sources
Does a search retrieve the right documents?
Evaluate retrieval before judging the generated answer. In a bounded corpus, identify relevant documents for a question, then count relevant hits, irrelevant hits and missed relevant documents. Count each document once and use the same maximum result count when comparing two systems.
Calculation runs locally in your browser without storage. Without JavaScript, use the formulas below; displayed values remain those of the example.
- Precision: avoid noise
- Relevant retrieved ÷ all retrieved documents. Here: 3 ÷ (3 + 1) = 75%.
- Recall: avoid omissions
- Relevant retrieved ÷ all relevant documents in the corpus. Here: 3 ÷ (3 + 2) = 60%.
- F1
- 2 × relevant retrieved ÷ (2 × relevant retrieved + irrelevant retrieved + relevant missed). Here: 6 ÷ 9 ≈ 66.7%.



A zero denominator produces “undefined”. Without an inventory of relevant documents, recall is unknown: do not infer it across the entire web. These measures assess neither answer truthfulness nor the severity of a missed document; omitting a decisive exception remains a blocking error.



