Step 6: Force the Model to Answer Using Only Context

Section 7 of 11-~ 6 min read-Synced from Cuantum content

Now we build a small pipeline to measure hallucinations under grounding constraints. This is where evaluation becomes precise: rather than asking whether the model sounds confident, we test whether its claims are actually derivable from the information it was given. Grounding constraints force the model into a position where it must either cite the provided context accurately or admit ignorance—no middle ground where it can blend memorized world knowledge with context in ways that seem plausible but aren't verifiable.

The dataset you create here serves as a controlled experiment. Each item pairs a snippet of context (the ground truth) with a question that should be answerable from that context alone. This setup mirrors real-world scenarios like customer support (where answers must come from documentation), RAG systems (where responses must cite retrieved passages), and any application where factual accuracy matters more than creativity. By testing your model's ability to stay grounded, you're measuring a capability that directly predicts production reliability.

Create: data/grounded_eval.json

Each entry includes:

context: text the model must rely on
question: what to answer
reference: expected answer (optional but helpful)

The context field is your source of truth. It should contain all and only the information needed to answer the question correctly. Keep contexts focused—typically 1-4 sentences. Longer contexts make it harder to verify support mechanically, and they also make it harder to isolate which part of the context the model relied on (or failed to rely on). Think of each context as a miniature knowledge base: complete for its question, but containing nothing extraneous.

The question should be directly answerable from the context, but not trivially so. Avoid questions where the answer is a direct copy-paste of a context sentence—those don't test understanding, just retrieval. Instead, prefer questions that require light synthesis: combining two facts from the context, paraphrasing information, or making a straightforward inference. The goal is to see whether the model can use the context, not just quote it verbatim.

The reference answer is optional but valuable for two reasons. First, it makes dataset creation easier—you know what you expect when writing each item, which helps you catch malformed questions early. Second, it enables more sophisticated evaluation later. If you eventually add semantic similarity metrics or use a judge model to score answer quality, having a reference answer gives you a comparison baseline. For now, though, the reference mainly serves as documentation: when you inspect results and see an unexpected answer, you can immediately check whether it's wrong or just phrased differently than you anticipated.

Example:

[  {    "id": "g_001",    "context": "The warranty lasts 12 months from the purchase date. To request service, provide your order number and proof of purchase.",    "question": "How long does the warranty last and what do I need to request service?",    "reference": "The warranty lasts 12 months, and you need the order number and proof of purchase."  }]

This example illustrates the design principles in action. The context provides two distinct pieces of information (warranty duration and service requirements). The question asks for both, requiring the model to identify and combine them. The reference shows the expected synthesis: concise, complete, and grounded entirely in the provided text. If the model answers "The warranty lasts one year and you need your receipt," that's a hallucination—"one year" is a reasonable paraphrase of "12 months," but "receipt" is not mentioned. The context says "proof of purchase," which might mean a receipt but could also mean an email confirmation, an invoice, or a bank statement. By saying "receipt," the model has added specificity that isn't justified by the text.

Aim for at least:

50 items for meaningful signals
Mix of short and longer contexts
Some tricky items that tempt the model to guess

Why 50? Because statistical noise dominates below that threshold. With 10 items, a single anomalous response shifts your metrics by 10%. With 50, you start seeing stable patterns: if one model hallucinates on 30% of items and another hallucinates on 15%, that's a real difference, not sampling variance. You can start with fewer for prototyping, but don't trust aggregate metrics until you've crossed into the 50-100 item range.

The mix of context lengths matters because models behave differently under different information loads. Short contexts (1 sentence) test whether the model can resist the urge to elaborate beyond what's given. Longer contexts (3-5 sentences) test whether the model can identify relevant information within a noisier background. If your application involves retrieval-augmented generation, you'll often feed the model 5-10 retrieved passages, only some of which are relevant. Training that scenario means including multi-sentence contexts where not every sentence is necessary to answer the question—the model must learn to focus on what matters and ignore the rest.

Tricky items are the most valuable part of your dataset. These are questions where the correct answer is "I don't know based on the provided context," but where a plausible-sounding wrong answer exists. For example: "The warranty lasts 12 months. What happens if I request service after 18 months?" The context doesn't say, but a model might confidently invent "the warranty will not cover it" or "you may need to pay for repairs." Both sound reasonable, both are probably true in most real warranty policies, and both are hallucinations because they're not stated in the text. Include 20-30% of items like this. They reveal whether your model actually learned to recognize the boundaries of its knowledge, or whether it just learned to rephrase context while still hallucinating when context runs out.