Step 9: Improvements You Can Add Next

Once your first evaluation pipeline works, you can strengthen it in ways that move from heuristics toward more principled measurement. Each enhancement addresses a specific limitation in the baseline approach, trading simplicity for accuracy in ways that matter for production systems.

Add AI-as-a-judge scoring for MT-Bench transcripts

Manual scoring of conversation transcripts doesn't scale beyond a few dozen examples, and human raters introduce their own inconsistencies. An AI-as-a-judge approach uses a strong language model (like GPT-4 or Claude) to evaluate your model's outputs according to explicit rubrics. You provide the judge model with the conversation transcript, a detailed scoring guide that defines what constitutes a good response, and instructions to assign scores with justifications.

The key advantage is consistency: the same judge model will apply identical standards across thousands of conversations, catching patterns that would exhaust human reviewers. The key limitation is that judge models inherit their own biases—they tend to prefer responses that match their training distribution, which often means favoring longer, more elaborately hedged answers over concise ones. To mitigate this, design your rubrics to penalize verbosity explicitly, and validate your judge's ratings against a small human-labeled set to catch systematic biases before you trust the automated scores.

Implementation-wise, you're adding another LLM call for each conversation you evaluate. This costs money and time, but it's often cheaper than human annotation at scale. The real trick is prompt engineering: your judge prompt needs to be specific enough to enforce your actual quality standards, not generic "helpfulness and harmlessness" criteria that might not align with your application's needs.

Replace keyword grounding checks with NLI entailment models

The keyword-matching heuristic in your baseline pipeline is deliberately simple, but it fails in predictable ways. It flags paraphrases as unsupported even when they're semantically identical to context statements. It misses hallucinations phrased using vocabulary from the context. And it can't handle negation—if the context says "The warranty does not cover water damage" and the model says "Water damage is covered," keyword overlap might suggest support when the claim directly contradicts the source.

Natural Language Inference (NLI) models are trained specifically to determine whether a hypothesis statement is entailed by, contradicts, or is neutral with respect to a premise. You can use an NLI model to check each generated sentence (hypothesis) against the provided context (premise). If the model predicts "entailment," the sentence is grounded. If it predicts "contradiction," you've caught a factual error. If it predicts "neutral," the sentence makes claims beyond what the context supports—a hallucination.

Models like DeBERTa fine-tuned on MNLI or ANLI datasets work well for this. They're smaller and faster than generative LLMs, so you can run them locally without expensive API calls. The main challenge is chunking: NLI models typically have token limits around 512, so if your context document is long, you need to break it into segments and check each sentence against all relevant segments. This introduces its own complexity—you need a retrieval step to find which context chunks might support each sentence, or you brute-force check against all chunks and accept the computational cost.

The result is a grounding check that understands semantics, not just lexical overlap. This dramatically reduces false positives (legitimate paraphrases flagged as hallucinations) and catches subtler errors (contradictions, impossible combinations of facts) that keyword matching misses.

Add citation validation checks

For applications where traceability matters—legal analysis, medical information, research synthesis—you don't just want grounded outputs. You want outputs with explicit citations that users can verify. This means fine-tuning your model to include citations (e.g., "According to Section 3.2 of the warranty document...") and then validating that those citations are accurate.

Your evaluation pipeline would check two things: citation presence (did the model cite sources when making factual claims?) and citation accuracy (does the cited source actually support the claim?). The second check is harder—you need to extract the referenced section, compare it to the claim, and determine support using the same NLI-based approach described above. But it's worth the complexity for high-stakes domains where "trust but verify" isn't optional.

Citation validation also reveals a different class of model failure: hallucinated citations. Some models learn to generate plausible-looking references to non-existent sources or real sources that don't contain the claimed information. Catching this requires checking whether cited sections actually exist in your knowledge base, not just whether they support the claims if they did exist.

Create an adversarial hallucination suite

Your baseline evaluation set probably contains straightforward questions with clear answers or clear cases of missing information. Real users ask harder questions—ones designed to trick the model, probe its boundaries, or exploit common failure modes. An adversarial suite explicitly tests these edge cases.

Build examples where the context contains contradictory information, forcing the model to recognize inconsistency rather than confidently picking one statement. Add questions that seem answerable from context but actually require outside knowledge to resolve (e.g., "Was this event before or after World War II?" when the context provides a year but not historical anchor points). Include questions with misleading premises ("How many times did the warranty mention full refunds?" when the warranty never promises refunds). These adversarial cases reveal whether your model learned robust grounding behavior or just surface patterns that work on typical examples.

The goal isn't to make your model fail—it's to discover the boundaries of its capabilities under stress, so you can either strengthen those boundaries through targeted fine-tuning or document them honestly in your deployment guidelines.

Track metrics over time in a dashboard

One-off evaluation tells you how your model performs today. Tracking metrics across model versions, training checkpoints, and data iterations tells you whether your development process is working. Build a simple dashboard that logs evaluation results every time you train a new model: MT-Bench scores, unsupported claim rates, refusal rates, example-level failures. Plot these over time.

This historical view surfaces trends you'd miss in isolated comparisons. Maybe your hallucination rate dropped after alignment training, but then crept back up as you fine-tuned on domain-specific data. Maybe your refusal rate increased steadily across the last five model versions, suggesting you're over-indexing on caution. Maybe your MT-Bench scores plateau after a certain training dataset size, telling you to stop collecting more data and focus on data quality instead.

You don't need sophisticated infrastructure for this—a JSON file of results per model version, a Jupyter notebook with matplotlib plots, and a discipline of running evals before calling any model "done" will get you 80% of the value. The remaining 20% comes from automating this into your training pipeline so evaluation happens by default, not as an afterthought.

But even this lightweight pipeline gives you something priceless: a repeatable way to measure whether training and alignment are moving your model in the direction you actually want. Without it, you're tuning hyperparameters and adjusting data mixtures based on intuition, hoping that lower loss translates to better behavior. With it, you have ground truth. You know whether your changes worked. And when they don't, you have the diagnostic data to understand why and fix it.