Step 7: Make it "real" (a simple qualitative test set)

The single-prompt comparison in Step 6 gave you a quick sanity check. But one example isn't enough to really understand how your model behaves across different scenarios. Now it's time to build a proper qualitative test set—a small collection of prompts that represent the kinds of tasks you actually care about.

This is where fine-tuning starts to feel less like a science experiment and more like a product. You're not just checking whether the loss went down—you're checking whether the model can do the job you're training it for.

Building your test set

Create a list of 10–20 prompts that cover the range of use cases you want your model to handle. These should be representative of real-world inputs, not just paraphrases of your training data. Here's what good test prompts look like:

• Diverse task types — If your model is supposed to summarize, explain concepts, and write emails, include examples of all three. Don't just test one skill.
• Varying difficulty — Include some easy prompts (like "Explain what a variable is in programming") and some harder ones (like "Explain closures in JavaScript to someone who only knows Python"). This helps you understand where the model starts to struggle.
• Edge cases — Throw in a few prompts that are deliberately tricky or ambiguous. These reveal weaknesses that you might not notice with straightforward examples.

Here are some example prompts you might include:

• "Summarize this paragraph in one sentence: [insert paragraph]"
• "Write an email reply in a calm, professional tone to a customer who is frustrated about a delayed shipment."
• "Explain gradient descent to someone who has never studied machine learning."
• "List three pros and cons of remote work."
• "Rewrite this sentence to be more concise: [insert wordy sentence]"
• "What's the difference between supervised and unsupervised learning?"

Notice that these prompts aren't just factual questions—they test style, tone, and structure. That's intentional. Instruction tuning is about teaching behavior, not just knowledge.

Running the comparison

For each prompt in your test set, generate outputs from both the base model and the fine-tuned model. You can extend the inference_test.py script to loop through multiple prompts, or you can run them manually one at a time. Either way, save the results so you can compare them side by side.

As you review the outputs, ask yourself:

• Does the fine-tuned model's response feel more aligned with what you want?
• Is the tone, structure, or level of detail closer to your training examples?
• Are there cases where the base model actually does better? (This can happen—it's useful information.)

This qualitative evaluation might feel informal compared to running benchmarks or calculating perplexity scores, but it's incredibly valuable, especially early in a project. You're not trying to publish a paper—you're trying to build something that works. And "works" is defined by whether you'd be happy using these outputs in practice.

Why this matters more than you think

A lot of beginners skip this step because it feels subjective or unscientific. They train a model, see the loss go down, and assume it's working. Then they deploy it, and it behaves in unexpected ways.

Here's the truth: you're training behavior, not just chasing a loss curve. The loss is a proxy metric—it tells you whether the model is learning something, but it doesn't tell you whether it's learning the right thing. The only way to know that is to actually look at the outputs and judge them with your own eyes.

This qualitative testing phase is where you develop intuition about what your dataset is teaching the model. You'll notice patterns—maybe the model is great at short answers but struggles with long explanations, or maybe it nails formal writing but falls apart on casual prompts. These insights will guide your next round of data collection and refinement.

Think of this step as the feedback loop that turns a one-off experiment into an iterative process. You train, you test, you learn what's missing, you improve the dataset, and you train again. That loop is how professional fine-tuning actually works. And it all starts with taking the time to manually review a handful of outputs and ask: is this what I wanted?