Triple-Validated Retrieval: Evidence Checks for RAG Answers

Retrieval does not guarantee that an answer follows its evidence: the RAGTruth research documents unsupported and contradictory claims even when models receive retrieved material. We recommend checking claims at separate points in a retrieval augmented generation pipeline, rather than treating the presence of retrieved documents as approval to answer.

Our proposed triple validation separates checks on stored summaries from checks on answer citations and uncited statements. It is a design recommendation, not a claim that three checks eliminate hallucinations or deliver a measured improvement in every system. For each stage, define the evidence it receives, the verdict it returns and what happens when it cannot complete its check.

Check summaries before making them searchable

Start with the material that will become the knowledge base. If a model turns meeting notes into a wiki article, retain the original notes alongside that article. Give each extracted claim a reference back to the passage used to produce it, and record which version of the source was inspected.

FActScore evaluates factual precision by splitting generated text into atomic facts and checking which are supported by a reliable knowledge source. We recommend borrowing that claim-level approach for summaries, without treating its research results as a benchmark for your own knowledge base.

Consider a hypothetical note that says a migration is approved for testing. A generated summary might instead say it is approved for production. Test that change in meaning explicitly, rather than accepting the summary because it mentions the right migration.

Store the verdict with the claim, the supporting passage and the checking method. For an unsupported claim, require correction or human review before making the summary available for ordinary retrieval. If verification runs asynchronously, keep pending material separate from material that has passed the check.

Decide how source corrections affect existing summaries. Our recommendation is to mark dependent claims for rechecking when their source changes, rather than leaving an earlier verdict attached without qualification. Keep the original record so a reviewer can reconstruct what the checker actually saw.

Verify what each answer citation supports

At answer time, check the relationship between a claim and its cited passage. Do not accept a reference merely because it resolves to a real document. Require the checker to return the passage it inspected and a reason for its verdict.

ALCE evaluates generated answers across fluency, correctness and citation quality. In its ELI5 experiments, even the best models lacked complete citation support half the time. That result concerns the systems and dataset tested in the paper, not a failure rate to apply to every current deployment.

For your own system, use explicit verdicts such as supported, contradicted and insufficient evidence. Split a sentence when it combines claims whose support differs. A passage establishing that an API exists should not, in your acceptance rules, also establish that a particular customer can access it.

Make incomplete checks visible. If the source cannot be retrieved, or the verification job times out, return a checking failure rather than a supported verdict. Set the response policy before deployment: withhold the affected claim, present it as unverified, or send it for review according to the consequence of getting it wrong.

Keep the user-facing wording narrow. Prefer a statement that the cited passage supports the claim over a badge implying that the whole answer is true. We recommend treating source authority and freshness as separate review questions, even after a support check succeeds.

Find claims with no citation

Run a separate pass over the answer for factual statements that have no source reference. In that pass, distinguish factual claims from recommendations and conversational wording. Make the classification inspectable, so a reviewer can challenge a factual statement incorrectly labelled as advice.

For each uncited factual claim, search the authorised sources for supporting material. If a passage is found, run the same support check used for existing citations before attaching it. Do not let a similarity match alone count as verification.

If the search finds nothing, report that no supporting source was found in the material searched. Do not turn that result into a declaration that the claim is false, or that nobody in the organisation knows the answer. Record the search scope and ask for the missing evidence where the task requires it.

Respect the user's access when looking for that evidence. Our recommendation is to return neither a restricted passage nor its contents through a suggested citation. Route an unresolved question to someone authorised to answer it instead of widening access as a convenience.

Test the checks as well as the generator

Treat each checker as something to evaluate, not a reason to stop evaluating. Build a reviewed test set from the questions your team actually asks, including answers that should be withheld. Include a missing source, a contradictory passage and a citation that supports only part of a sentence.

Measure each stage separately. Record unsupported claims accepted as supported, supported claims rejected, and checks that failed to finish. Track how much useful information the final answers retain, alongside the time and model cost of checking them.

Then compare the complete workflow with a baseline using the same questions and sources. We would want that comparison before claiming that another verification stage earns its operational cost. Keep model and prompt versions with the results so changes can be investigated rather than attributed to an unspecified improvement in AI.

Do not describe the stages as independent simply because they run in different jobs. Require evidence for any claim of independent error detection. We intend separate responsibilities to make failures easier to inspect, but acceptance must still depend on measured behaviour.

For an engineering leader, our recommended starting point is a narrow workflow where the source passages and expected answers can be reviewed by the team. Expand it only after the failure policy works in practice. The aim should be an answer whose evidence and remaining uncertainty can be inspected before someone acts on it.

Update, October 2026

Our engineering article on assembling deterministic grounding packs before an agent starts work describes giving the planner and reviewer the same evidence. We recommend that arrangement when applying these checks to agent-written plans, while retaining verification of the plan's actual claims.

Sources

• Association for Computational Linguistics, "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation", December 2023.
• Association for Computational Linguistics, "Enabling Large Language Models to Generate Text with Citations", December 2023.
• Association for Computational Linguistics, "RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models", August 2024.
• Dendro Logic, "What the Agent Does Not Know to Look For", 6 September 2026. Linked in the October 2026 update only.