Step 8: Interpret Results Like an Alignment Engineer

Section 9 of 11-~ 21 min read-Synced from Cuantum content

We will implement a simple but effective sentence-level support check. The core idea is straightforward: break the model's answer into individual sentences, then determine whether each sentence can be justified by the provided context. This approach is not perfect—it relies on heuristics rather than deep semantic understanding—but it is a strong starting point that catches the majority of hallucinations without requiring expensive infrastructure.

Why sentence-level granularity? Because hallucinations don't usually corrupt entire responses—they appear as isolated unsupported claims embedded within otherwise reasonable answers. A model might correctly state "The warranty covers defects in materials and workmanship" (supported by context) and then add "Claims must be filed within 30 days of discovery" (not mentioned anywhere). If you only evaluate the response as a whole, you miss this mixed behavior. Sentence-level analysis exposes these fault lines.

The pipeline works in three stages:

Split the model's answer into sentences
Label each sentence as supported or unsupported based on approximate matching against the context
Record an "unsupported claim rate" as your primary hallucination metric

The sentence splitting step uses regular expressions to break on common sentence boundaries—periods, question marks, exclamation points followed by whitespace. This is admittedly crude. It will fail on edge cases like "Dr. Smith" or "Inc." or decimal numbers, splitting where it shouldn't. A production system would use a proper sentence tokenizer like spaCy or NLTK. But for evaluation purposes, occasional mis-splits are acceptable as long as they affect base and fine-tuned models equally. You're measuring relative improvement, not absolute perfection.

The support detection heuristic is where the real work happens. For each sentence, we normalize the text (lowercase, collapse whitespace), extract content words longer than three characters, and check what fraction of those words appear in the normalized context. If 55% or more of the sentence's keywords are present in the context, we label it as supported. If fewer than 55% appear, it's flagged as unsupported.

This threshold is deliberately tuned to favor precision over recall. A 55% match requirement means we'll miss some true hallucinations (false negatives)—cases where a sentence happens to use many of the same words as the context but distorts their meaning. But we'll rarely flag a genuinely supported sentence as unsupported (false positives). For comparative evaluation, false negatives are acceptable. If your fine-tuned model reduces the unsupported sentence rate from 30% to 15%, that's a real improvement even if both numbers undercount the true hallucination rate. What matters is the direction and magnitude of change.

Why keyword overlap instead of semantic similarity? Because semantic similarity models (embeddings, sentence transformers) introduce their own failure modes. They can score two sentences as highly similar even when one contradicts the other, as long as they discuss the same topic. "The warranty lasts 12 months" and "The warranty lasts 24 months" will have high cosine similarity despite being factually incompatible. Keyword overlap is less sophisticated, but its failure modes are more predictable and easier to debug.

The implementation also tracks "I don't know" responses separately. When a model explicitly states "I don't know based on the provided context," that's not a hallucination—it's appropriate epistemic humility. By counting these refusals, you can detect whether your fine-tuning made the model more willing to acknowledge knowledge boundaries. A model that reduces unsupported claims from 30% to 15% while increasing refusals from 5% to 20% has learned a valuable lesson: when uncertain, say so rather than inventing plausible fictions.

Create evaluate_grounding.py:

import jsonimport reimport torchfrom transformers import AutoTokenizer, AutoModelForCausalLM BASE_MODEL = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"TUNED_MODEL = "outputs/your_finetuned_model" def load_model(model_name):    tok = AutoTokenizer.from_pretrained(model_name, use_fast=True)    mdl = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")    if tok.pad_token is None:        tok.pad_token = tok.eos_token    return tok, mdl def generate(mdl, tok, prompt, max_new_tokens=180, temperature=0.2):    inputs = tok(prompt, return_tensors="pt").to(mdl.device)    with torch.no_grad():        out = mdl.generate(            **inputs,            max_new_tokens=max_new_tokens,            do_sample=True,            temperature=temperature,            top_p=0.9        )    return tok.decode(out[0], skip_special_tokens=True) def grounded_prompt(context, question):    return f"""You are a helpful assistant.Use ONLY the context below to answer the question.If the answer is not in the context, say "I don't know based on the provided context." Context:{context} Question:{question} Answer:""" def split_sentences(text):    # simple sentence splitter    parts = re.split(r"[.!?]\s+", text.strip())    return [p.strip() for p in parts if p.strip()] def normalize(s):    return re.sub(r"\s+", " ", s.strip().lower()) def is_supported(sentence, context):    # heuristic: sentence supported if most keywords appear in context    sent = normalize(sentence)    ctx = normalize(context)     words = [w for w in re.findall(r"[a-zA-Z0-9']+", sent) if len(w) > 3]    if not words:        return True     hit = sum(1 for w in set(words) if w in ctx)    ratio = hit / max(1, len(set(words)))     return ratio >= 0.55  # adjustable threshold def evaluate_grounding(items, tok, mdl):    results = []    total_sentences = 0    unsupported_sentences = 0    idk_count = 0     for item in items:        prompt = grounded_prompt(item["context"], item["question"])        answer = generate(mdl, tok, prompt)         if "i don't know based on the provided context" in answer.lower():            idk_count += 1         sentences = split_sentences(answer)        total_sentences += len(sentences)         unsupported = []        for s in sentences:            if not is_supported(s, item["context"]):                unsupported.append(s)         unsupported_sentences += len(unsupported)         results.append({            "id": item["id"],            "question": item["question"],            "answer": answer,            "unsupported_sentences": unsupported        })     unsupported_rate = unsupported_sentences / max(1, total_sentences)     return {        "results": results,        "summary": {            "total_items": len(items),            "total_sentences": total_sentences,            "unsupported_sentences": unsupported_sentences,            "unsupported_rate": unsupported_rate,            "idk_count": idk_count        }    } def main():    with open("data/grounded_eval.json", "r", encoding="utf-8") as f:        items = json.load(f)     base_tok, base_mdl = load_model(BASE_MODEL)    tuned_tok, tuned_mdl = load_model(TUNED_MODEL)     base_eval = evaluate_grounding(items, base_tok, base_mdl)    tuned_eval = evaluate_grounding(items, tuned_tok, tuned_mdl)     output = {        "base_model": BASE_MODEL,        "tuned_model": TUNED_MODEL,        "base": base_eval,        "tuned": tuned_eval    }     with open("outputs/hallucination_results.json", "w", encoding="utf-8") as f:        json.dump(output, f, indent=2, ensure_ascii=False)     print("Saved outputs/hallucination_results.json")    print("Base unsupported rate:", base_eval["summary"]["unsupported_rate"])    print("Tuned unsupported rate:", tuned_eval["summary"]["unsupported_rate"]) if __name__ == "__main__":    main()

Let's break down what this code does, section by section, to understand how it implements the hallucination detection pipeline.

Imports and Model Configuration

import jsonimport reimport torchfrom transformers import AutoTokenizer, AutoModelForCausalLM BASE_MODEL = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"TUNED_MODEL = "outputs/your_finetuned_model"

We import the necessary libraries for JSON handling, regular expression matching, PyTorch tensor operations, and Hugging Face model loading. The model paths are defined as constants at the top—this makes it easy to swap models without hunting through the code. You'll replace TUNED_MODEL with the actual path to your fine-tuned checkpoint.

Model Loading Function

def load_model(model_name):    tok = AutoTokenizer.from_pretrained(model_name, use_fast=True)    mdl = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")    if tok.pad_token is None:        tok.pad_token = tok.eos_token    return tok, mdl

This function handles the boilerplate of loading both tokenizer and model from a Hugging Face checkpoint. The device_map="auto" parameter automatically handles GPU placement, splitting the model across available devices if necessary. The padding token check is defensive programming—some tokenizers don't define a pad token by default, which causes errors during batch processing. We set it to the end-of-sequence token as a safe fallback. This function returns both tokenizer and model as a tuple, keeping related objects together.

Generation Function

def generate(mdl, tok, prompt, max_new_tokens=180, temperature=0.2):    inputs = tok(prompt, return_tensors="pt").to(mdl.device)    with torch.no_grad():        out = mdl.generate(            **inputs,            max_new_tokens=max_new_tokens,            do_sample=True,            temperature=temperature,            top_p=0.9        )    return tok.decode(out[0], skip_special_tokens=True)

This is our inference wrapper. It takes a text prompt, tokenizes it, generates a completion, and decodes the result back to text. The torch.no_grad() context manager disables gradient computation, which reduces memory usage and speeds up inference—we're not training here, so we don't need gradients. The generation parameters are carefully chosen: max_new_tokens=180 allows enough space for a complete answer without letting the model ramble excessively. temperature=0.2 is quite low, making outputs more deterministic and focused—we want consistent answers, not creative exploration. top_p=0.9 (nucleus sampling) provides a small amount of diversity while still favoring high-probability continuations. These defaults work well for factual question answering, though you might adjust them for other tasks.

Grounded Prompt Constructor

def grounded_prompt(context, question):    return f"""You are a helpful assistant.Use ONLY the context below to answer the question.If the answer is not in the context, say "I don't know based on the provided context." Context:{context} Question:{question} Answer:"""

This function constructs the specialized prompt format discussed earlier. It takes raw context and a question, then wraps them in explicit grounding instructions. The template creates clear visual separation between context and question using whitespace and labeled sections. The "Answer:" prefix at the end primes the model to begin its response immediately after generation starts. This is a prompt engineering detail that matters—without it, some models waste tokens generating "Sure, I'll answer that question" before actually answering. By providing the response prefix, we skip that preamble and get straight to the content.

Sentence Splitting

def split_sentences(text):    # simple sentence splitter    parts = re.split(r"[.!?]\s+", text.strip())    return [p.strip() for p in parts if p.strip()]

This function breaks a text response into individual sentences using a regular expression. The pattern [.!?]\s+ matches any of the three common sentence-ending punctuation marks followed by one or more whitespace characters. This handles most normal cases—declarative sentences ending with periods, questions, exclamations. It will fail on edge cases like abbreviations ("Dr. Smith became a Ph.D. in 1995" becomes three sentences) or ellipses ("The warranty covers... most defects" splits incorrectly). These failures are acceptable for our purposes because they affect base and fine-tuned models equally. We're not trying to build a perfect sentence parser—we're trying to apply the same imperfect heuristic consistently.

Text Normalization Helper

def normalize(s):    return re.sub(r"\s+", " ", s.strip().lower())

This tiny function does the unglamorous but essential work of text preprocessing. It lowercases the input (so "Warranty" and "warranty" match), strips leading/trailing whitespace, and collapses all internal whitespace sequences (newlines, tabs, multiple spaces) into single spaces. This normalization ensures that superficial formatting differences don't prevent legitimate matches. Without it, "The warranty lasts 12 months" wouldn't match "The warranty lasts 12 months" (extra spaces) or "THE WARRANTY LASTS 12 MONTHS" (different case). Normalization eliminates these non-semantic variations.

Support Detection Heuristic

def is_supported(sentence, context):    # heuristic: sentence supported if most keywords appear in context    sent = normalize(sentence)    ctx = normalize(context)     words = [w for w in re.findall(r"[a-zA-Z0-9']+", sent) if len(w) > 3]    if not words:        return True     hit = sum(1 for w in set(words) if w in ctx)    ratio = hit / max(1, len(set(words)))     return ratio >= 0.55  # adjustable threshold

This is the core hallucination detection logic. It works by extracting "keywords" from the sentence—words longer than three characters, which filters out most function words like "the," "is," "and," "or." We focus on content words because they carry the semantic weight. The function then checks what fraction of these keywords appear somewhere in the normalized context string. If 55% or more of the sentence's unique keywords are present in the context, we label it as supported.

The 55% threshold is a tuned constant based on empirical testing. At 70%, too many genuinely supported sentences get flagged because they rephrase context using synonyms or different word orders. At 40%, too many hallucinations slip through because they happen to reuse common words from the context while making unsupported claims. 55% is the sweet spot where most clear hallucinations get caught while most legitimate paraphrases pass. You might need to adjust this for your specific domain—technical documentation with precise terminology might work better at 60%, while more narrative content might need 50%.

The edge case handling is worth noting: if a sentence has no keywords after filtering (very short sentences like "Yes" or "Maybe"), we return True by default. This is conservative—we assume short responses are supported rather than flagging them as hallucinations. The alternative would flag every brief acknowledgment, which creates too many false positives.

Grounding Evaluation Pipeline

def evaluate_grounding(items, tok, mdl):    results = []    total_sentences = 0    unsupported_sentences = 0    idk_count = 0     for item in items:        prompt = grounded_prompt(item["context"], item["question"])        answer = generate(mdl, tok, prompt)         if "i don't know based on the provided context" in answer.lower():            idk_count += 1         sentences = split_sentences(answer)        total_sentences += len(sentences)         unsupported = []        for s in sentences:            if not is_supported(s, item["context"]):                unsupported.append(s)         unsupported_sentences += len(unsupported)         results.append({            "id": item["id"],            "question": item["question"],            "answer": answer,            "unsupported_sentences": unsupported        })     unsupported_rate = unsupported_sentences / max(1, total_sentences)     return {        "results": results,        "summary": {            "total_items": len(items),            "total_sentences": total_sentences,            "unsupported_sentences": unsupported_sentences,            "unsupported_rate": unsupported_rate,            "idk_count": idk_count        }    }

This function orchestrates the entire evaluation process. It loops through each test item, generates an answer, analyzes that answer for hallucinations, and accumulates statistics. The structure is deliberately simple and linear—no fancy parallelization or async processing—because clarity matters more than speed for evaluation code. You'll run this occasionally to check model quality, not thousands of times per second in production.

For each item, the function builds a grounded prompt, generates an answer, and immediately checks for the explicit refusal phrase "I don't know based on the provided context." This check happens before sentence splitting because we want to count refusals at the response level, not the sentence level. A model that says "I don't know based on the provided context" generates one sentence, but it shouldn't count as one unsupported sentence—it's a category of its own.

The function then splits the answer into sentences and tests each one for support. Unsupported sentences are collected in a list, which gets stored in the detailed results. This per-item tracking is essential for debugging. When you see that your fine-tuned model has a 15% unsupported rate, you need to know which 15% of sentences were flagged and why. Maybe they're all related to dates, or pricing, or warranty exceptions—patterns that suggest specific fine-tuning improvements.

The summary statistics are computed at the end: total number of items evaluated, total sentences generated across all items, number of unsupported sentences, the unsupported rate (as a fraction), and count of explicit refusals. The max(1, total_sentences) in the rate calculation prevents division by zero if something goes catastrophically wrong and no sentences are generated.

Main Execution Logic

def main():    with open("data/grounded_eval.json", "r", encoding="utf-8") as f:        items = json.load(f)     base_tok, base_mdl = load_model(BASE_MODEL)    tuned_tok, tuned_mdl = load_model(TUNED_MODEL)     base_eval = evaluate_grounding(items, base_tok, base_mdl)    tuned_eval = evaluate_grounding(items, tuned_tok, tuned_mdl)     output = {        "base_model": BASE_MODEL,        "tuned_model": TUNED_MODEL,        "base": base_eval,        "tuned": tuned_eval    }     with open("outputs/hallucination_results.json", "w", encoding="utf-8") as f:        json.dump(output, f, indent=2, ensure_ascii=False)     print("Saved outputs/hallucination_results.json")    print("Base unsupported rate:", base_eval["summary"]["unsupported_rate"])    print("Tuned unsupported rate:", tuned_eval["summary"]["unsupported_rate"]) if __name__ == "__main__":    main()

The main() function ties everything together. It loads your evaluation dataset from JSON, loads both the base and fine-tuned models, runs the full grounding evaluation on both, and saves the results to a structured output file. The output format includes model identifiers, complete per-item results for both models, and summary statistics for easy comparison.

The JSON output file serves two purposes. First, it's human-readable—you can open it in any text editor and browse through specific questions and answers to understand what changed. Second, it's machine-readable—you can load it into a Jupyter notebook, compute additional statistics, visualize trends, or compare results across multiple fine-tuning runs. By saving everything in a structured format rather than just printing summary numbers, you create an audit trail of model behavior over time.

The printed output provides immediate feedback. You don't have to open the JSON file to see whether your fine-tuning improved grounding—the script tells you right away. This instant feedback loop matters during iterative development. You tune hyperparameters, rerun evaluation, check the printed rates, adjust, and repeat. If you had to manually open and parse the JSON file each time, the friction would slow down experimentation.

The if __name__ == "__main__": guard is a Python idiom that prevents main() from running if the file is imported as a module. This makes the code reusable—you can import evaluate_grounding or is_supported into other scripts without triggering a full evaluation run. It's a small detail, but it reflects good software engineering practice: write code that's modular and composable, even in one-off evaluation scripts.

The script's architecture follows a clean separation of concerns. Model loading, prompt construction, generation, sentence analysis, and aggregation each live in separate functions. This modularity makes it easy to swap components—replace the keyword-based is_supported function with an NLI model, change the prompt template, adjust generation parameters—without rewriting the entire pipeline.

The evaluate_grounding function is the heart of the system. It iterates through your evaluation dataset, generates an answer for each item, splits that answer into sentences, and checks each sentence for support. It accumulates both item-level details (which specific sentences were unsupported) and summary statistics (overall unsupported rate across all items). This dual output is essential: summary statistics tell you whether the model improved, while item-level details let you investigate specific failure modes.

The final comparison between base and tuned models runs both through identical evaluation logic and saves results to a structured JSON file. This output format is designed for programmatic analysis—you can load it into a notebook, visualize trends, run statistical tests on the differences, or feed it into a monitoring dashboard. The printed summary provides immediate feedback, but the real value is in the structured data you can analyze over time.

This produces three key metrics:

Unsupported sentence rate—the percentage of all generated sentences that lack grounding in context
Count of "I don't know" responses—how often the model appropriately refused to answer
Per-item unsupported sentence listing—which specific claims were flagged, allowing manual review of edge cases

That is already enough to detect meaningful changes in hallucination behavior across model versions. You don't need a perfect hallucination detector to measure progress. You need a consistent hallucination detector that applies the same standards to every model you evaluate. As long as your heuristic's false positive and false negative rates remain stable, you can trust that a 15-percentage-point reduction in flagged hallucinations represents real improvement, not measurement noise.

One critical nuance: this pipeline measures groundedness, not factual accuracy. A model can be perfectly grounded—every sentence supported by the provided context—while the context itself contains errors. If your context says "The warranty lasts 6 months" but the actual warranty is 12 months, a grounded model will confidently state the wrong answer. Grounding and accuracy are related but distinct properties. This evaluation measures whether your model learned to stick to its sources, not whether those sources are correct. For many applications, that's exactly what you want—better to have a model that reliably echoes your documentation (which you can fix) than one that invents plausible-sounding alternatives.