Step 9: Improvements You Can Add Next

When you compare base versus fine-tuned model performance, resist the temptation to reduce everything to a single summary metric. The question "which model is better?" rarely has a simple answer. Model behavior exists in a high-dimensional space of capabilities and failure modes, and fine-tuning creates shifts along multiple axes simultaneously. A model can improve on some dimensions while regressing on others, and whether that trade-off is acceptable depends entirely on your deployment context.

Instead of looking for a winner, look for patterns in how behavior changed:

MT-Bench-style conversation transcripts

Read through the multi-turn exchanges your models generated. Don't just score them—actually read them as if you were the user on the receiving end. Ask yourself:

• Does the fine-tuned model follow formatting constraints and role boundaries more reliably across multiple conversation turns? Or does it still slip into completing the user's sentences or breaking character after three exchanges?
• Does it maintain consistency in its knowledge claims? If it states a fact in turn one, does it contradict itself by turn four? Base models often show higher self-contradiction rates in longer conversations because they lack the reinforcement that teaches coherence across context windows.
• Does the model become overly cautious or apologetic? Fine-tuning on safety-focused data sometimes produces models that refuse reasonable requests, apologize excessively, or hedge every statement with qualifiers like "I think" and "possibly." This kind of over-correction makes responses feel uncertain even when they shouldn't be.

Pay particular attention to edge cases where the user's request sits near a boundary—asking for creative content that could be misused, requesting information that's mostly but not entirely in the training data, or probing the model's understanding of its own limitations. These boundary cases often reveal whether your fine-tuning taught genuine understanding or just pattern matching.

Hallucination and grounding pipeline

The quantitative metrics here tell you about aggregate behavior, but you need to dig deeper into what those numbers mean:

• Did the unsupported claim rate decrease? By how much? A drop from 30% to 28% might just be noise. A drop from 30% to 12% suggests your fine-tuning data successfully taught the model to constrain its outputs to available evidence.
• Did "I don't know" or explicit refusal responses increase? This is often a positive sign. A model that went from refusing 5% of unanswerable questions to refusing 22% has learned to recognize its knowledge boundaries. But check whether these refusals are appropriate—if the model starts saying "I don't know" to questions clearly answered in the context, it's become miscalibrated in the opposite direction.
• Are the refusals well-targeted or excessive? Examine which specific questions triggered refusals. Did the model refuse only when the context genuinely lacked the answer, or did it also refuse questions where the answer required minimal inference from stated facts? A model that refuses to answer "What color was the shirt?" when the context says "He wore a blue shirt" has overcorrected into uselessness.

Also examine the specific unsupported sentences your pipeline flagged. Are they genuine hallucinations—invented facts with no basis in context? Or are they reasonable inferences that your keyword-matching heuristic couldn't recognize as supported? For instance, if the context says "The device must be returned in original packaging" and the model states "Returns require original packaging," that's not a hallucination even though the phrasing differs. Your heuristic might flag it anyway. Manual review of flagged sentences helps you distinguish real model problems from measurement artifacts.

The most important pattern to watch for is the hallucination-caution trade-off. A reduction in unsupported claims is almost always accompanied by an increase in refusals or hedged language. The model learns "when in doubt, don't make claims" as a safety strategy. Whether this trade-off is acceptable depends on your application. For a customer service bot where wrong answers damage trust, you want the cautious model. For a creative writing assistant where users expect the model to make narrative leaps, excessive caution kills the user experience. There's no universal answer—only application-specific optimization.

Finally, consider whether the model's behavior changes are stable or fragile. Run the same evaluation multiple times with different random seeds. If the unsupported claim rate varies wildly between runs (15% one time, 28% the next), your improvement might be an artifact of sampling randomness rather than a genuine shift in model behavior. Stable improvements that hold across multiple seeds and slight prompt variations are more likely to reflect real learning.