Lesson reader

Tuning Large Language Models for Real-World ApplicationsChapter 161

Project 1: Domain-Specific Q&A Assistant with LoRA

Section 1 of 3-~ 37 min read-Synced from Cuantum content

By this point in the book, you have explored the complete lifecycle of customizing large language models. You learned how to prepare instruction datasets, fine-tune models efficiently using PEFT methods, align them with human preferences, evaluate their behavior, and deploy them in production environments.

Throughout the previous chapters, you've worked with each of these techniques in isolation—learning the theory, understanding the trade-offs, and implementing individual components. You've seen how instruction datasets shape model behavior, how PEFT methods like LoRA enable efficient training, and how alignment techniques ensure models respond in ways that match human expectations. Each chapter provided you with a specific tool for the AI development toolkit.

The goal of these capstone projects is to bring all of those concepts together into cohesive, end-to-end systems. Real-world AI development is rarely about applying a single technique in isolation. Instead, practitioners must orchestrate multiple stages—data preparation, model selection, training, evaluation, and deployment—into a unified pipeline where each stage informs and supports the others.

Each project simulates a realistic AI development workflow and guides you through building a working system step by step. These are the kinds of projects that practitioners build when developing domain-specific AI assistants, customer support agents, or enterprise AI tools. You'll encounter the same decisions, trade-offs, and debugging challenges that arise in professional settings, from choosing the right base model to determining when your fine-tuned system is ready for production use.

The projects are intentionally designed to mirror real-world pipelines, where training, evaluation, and deployment must work together. You'll learn not just how to execute each step, but how to think about the connections between them—how your dataset design affects training efficiency, how your evaluation strategy reveals deployment risks, and how production constraints might require you to revisit earlier design decisions. This holistic perspective is what separates theoretical knowledge from practical expertise.

Project Goal

In this project, you will build a domain-specific question–answering assistant by fine-tuning a large language model using LoRA (Low-Rank Adaptation). This project represents one of the most common applications of LLM customization in industry: taking a general-purpose model and specializing it to excel within a particular knowledge domain.

The assistant will learn to answer questions about a specialized topic using curated instruction data. This approach is widely used when organizations want to create assistants for internal documentation, product knowledge bases, legal information, or technical manuals. Unlike retrieval-augmented generation (RAG), which retrieves relevant documents at inference time, this approach embeds domain knowledge directly into the model's parameters through fine-tuning, enabling faster responses and more nuanced understanding of domain-specific concepts and terminology.

By the end of this project, you will have:

Prepared a domain-specific instruction dataset tailored to your chosen topic
Fine-tuned a base model using LoRA, experiencing firsthand how parameter-efficient methods enable rapid iteration
Evaluated model responses using both qualitative and quantitative methods
Tested the assistant through an interactive interface that demonstrates practical usability

This pipeline demonstrates how relatively small datasets can dramatically improve model usefulness within a specific domain. You'll discover that even a few hundred high-quality examples can transform a model's performance on specialized tasks, and you'll gain intuition for when fine-tuning is the right approach versus alternatives like prompt engineering or RAG.

Step 1: Choose Your Domain and Scope

The first and most important decision in building a domain-specific assistant is selecting the knowledge area where your model will specialize. This choice will influence every subsequent step—from dataset creation to evaluation criteria to deployment considerations.

When choosing a domain, consider both the breadth and depth of knowledge required. A domain that's too broad (like "general science") will be difficult to cover comprehensively with a reasonably-sized dataset, while one that's too narrow (like "Python list comprehension edge cases") may not justify the effort of fine-tuning. The ideal domain is specific enough that you can create comprehensive coverage with a few hundred examples, yet broad enough to be genuinely useful in practice.

Example domains that work well for this approach include:

Python programming fundamentals and common libraries
Machine learning concepts and implementation tutorials
Medical knowledge summaries for specific conditions or procedures
Legal regulations within a particular jurisdiction or practice area
Product documentation for a specific software platform or tool
Historical knowledge about a particular period or region

Each of these domains has clear boundaries, well-defined terminology, and the kind of factual, instructional content that benefits from fine-tuning. They also represent real-world use cases where organizations frequently deploy specialized assistants.

For this demonstration, we will build a Python programming Q&A assistant that can explain core concepts, provide code examples, and answer common questions about Python syntax and standard library features. This domain is particularly well-suited for instruction tuning because Python questions typically have clear, correct answers, and the community has established conventions for good explanations and code style.

Before moving to dataset creation, take time to define the scope explicitly. For our Python assistant, we might decide to focus on Python 3.x fundamentals, common data structures, control flow, functions, and popular standard library modules, while excluding advanced topics like metaclasses or C extensions. This scoping helps ensure consistency across your training examples.

Step 2: Create a Domain-Specific Instruction Dataset

Once you've defined your domain, the next step is constructing a high-quality instruction dataset. The quality of this dataset will directly determine how well your fine-tuned model performs, making this one of the most critical phases of the project.

Instruction tuning datasets follow a specific structure that teaches the model how to respond to user requests. The most common format consists of three components:

Instruction: The task or question the user is asking
Input: Optional context or additional information (can be empty for many tasks)
Output: The desired response from the model

This structure provides clear input-output pairs that the model can learn from. During training, the model learns to predict the output given the instruction and input, effectively learning the mapping between questions and appropriate responses.

Here's what a well-constructed example looks like for our Python assistant:

{  "instruction": "Explain what a Python list is and when you should use one.",  "input": "",  "output": "A Python list is a mutable, ordered collection that can store multiple elements of any type, including numbers, strings, objects, or even other lists. Lists are defined using square brackets, like [1, 2, 3]. You should use a list when you need an ordered collection that you plan to modify—adding, removing, or changing elements. Lists are ideal for maintaining sequences where order matters and where you need flexibility to update the contents."}

Notice how the output provides not just a definition, but context about when to use the feature. This kind of comprehensive response helps the model learn to provide genuinely useful answers rather than bare-bones definitions.

Another example demonstrates code generation:

{  "instruction": "Write a Python function that checks if a number is even.",  "input": "",  "output": "def is_even(n):\n    \"\"\"Return True if n is even, False otherwise.\"\"\"\n    return n % 2 == 0\n\n# Example usage:\n# is_even(4)  # Returns True\n# is_even(7)  # Returns False"}

This example shows the model how to write clean, documented code with usage examples—patterns it will then replicate in its own responses.

When building your dataset, aim for diversity across several dimensions:

Task variety: Include definitions, explanations, code generation, debugging help, and comparisons between concepts
Complexity levels: Mix simple, intermediate, and advanced questions to cover the full spectrum of your domain
Response styles: Some questions warrant brief answers, others need detailed explanations with examples
Common misconceptions: Include questions that address frequent points of confusion in your domain

For this project, aim to create at least 100-200 high-quality examples, though more is better. You can source these from documentation, Stack Overflow questions (with rewritten answers), tutorials, or create them yourself based on your domain expertise. The key is ensuring consistency in quality and style across all examples.

Once you've prepared your examples, save them in JSON format as python_qa_dataset.json. This file will serve as the foundation for training your specialized assistant in the following steps.

Step 3: Load and Prepare the Dataset

With your instruction dataset created and saved, the next step is loading it into a format suitable for training. The Hugging Face datasets library provides an efficient way to work with instruction data, handling loading, preprocessing, and batching seamlessly.

Begin by loading your JSON dataset:

from datasets import load_dataset dataset = load_dataset("json", data_files="python_qa_dataset.json") print(dataset["train"][0])

This will display the first example from your dataset, allowing you to verify the structure is correct. You should see the instruction, input, and output fields you carefully crafted in the previous step.

Next, you need to convert each example into a format the model can learn from during training. Language models are trained on sequences of text, so we need to transform our structured instruction-input-output format into a single coherent text prompt. This formatting step is crucial—it defines the template the model will learn to recognize and respond to.

The formatting function creates a consistent structure that clearly delineates the instruction from the expected response:

def format_example(example):    return {        "text": f"""### Instruction:{example['instruction']} ### Response:{example['output']}"""    } dataset = dataset.map(format_example)

The ### Instruction: and ### Response: markers serve as clear delimiters that help the model understand the boundary between what the user asks and what the assistant should provide. These markers are arbitrary—you could use different formatting—but consistency is critical. Whatever format you choose here must be used identically during inference, or the model won't recognize the pattern it learned during training.

Notice that we're omitting the input field in this formatting since most Python Q&A examples don't require additional context beyond the instruction itself. If your domain requires contextual information (like "given this code snippet, explain the error"), you would include the input field between the instruction and response.

The map function applies this transformation to every example in your dataset efficiently, creating a new field called text that contains the formatted prompt. This is the actual text sequence the model will see during training.

Step 4: Load the Base Model

Selecting the right base model is a critical decision that affects training time, inference speed, response quality, and computational requirements. For this project, we need a model that balances several factors: it should be small enough to fine-tune on modest hardware, capable enough to generate coherent responses, and preferably already instruction-tuned so it understands the question-answer format.

We'll use Mistral-7B-Instruct, a 7-billion parameter model that has already undergone instruction tuning. Starting with an instruction-tuned model rather than a base language model gives us a significant advantage—the model already understands how to follow instructions and format responses appropriately. Our domain-specific fine-tuning will then specialize this existing capability rather than teaching it from scratch.

Load the model and tokenizer using the transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer model_name = "mistralai/Mistral-7B-Instruct-v0.2" tokenizer = AutoTokenizer.from_pretrained(model_name)tokenizer.pad_token = tokenizer.eos_token  # Set padding token model = AutoModelForCausalLM.from_pretrained(    model_name,    device_map="auto",    torch_dtype="auto")

The AutoModelForCausalLM class automatically selects the appropriate model architecture based on the model name, while AutoTokenizer loads the corresponding tokenizer that converts text into the numerical tokens the model processes.

The device_map="auto" parameter is particularly useful—it automatically distributes the model across available GPU memory, and if the model doesn't fit on a single GPU, it will split it across multiple GPUs or even offload parts to CPU memory. This makes it possible to work with 7B parameter models even on consumer hardware.

Setting torch_dtype="auto" allows the library to select an appropriate precision format, typically loading the model in the same dtype it was trained with. For memory-constrained environments, you could explicitly set this to torch.float16 or use 8-bit quantization, though this may slightly impact training dynamics.

Before proceeding to LoRA configuration, it's worth printing the model architecture to understand which layers you'll be adapting:

print(model) # Also check the number of trainable parameterstotal_params = sum(p.numel() for p in model.parameters())print(f"Total parameters: {total_params:,}")

This gives you visibility into the model structure and confirms you're working with approximately 7 billion parameters—far too many to fine-tune directly on typical hardware, which is exactly why LoRA's parameter-efficient approach is so valuable for this project.

Step 5: Apply LoRA Fine-Tuning

With your base model loaded and your dataset prepared, you're now ready to apply LoRA (Low-Rank Adaptation) to make fine-tuning feasible on standard hardware. Rather than updating all 7 billion parameters in the Mistral model—which would require massive computational resources and memory—LoRA allows you to train only a small set of additional parameters that modify the model's behavior for your specific domain.

The key insight behind LoRA is that the updates needed to adapt a pre-trained model to a new task lie in a low-rank subspace. Instead of modifying the original weight matrices directly, LoRA injects trainable low-rank matrices that capture the task-specific adaptations. This means you might train only 10-20 million parameters instead of 7 billion, reducing memory requirements and training time by orders of magnitude while maintaining comparable performance.

First, install the PEFT (Parameter-Efficient Fine-Tuning) library, which provides a clean implementation of LoRA and other efficient training methods:

pip install peft

Now configure LoRA by specifying which parts of the model to adapt and how to structure the low-rank decomposition:

from peft import LoraConfig, get_peft_model config = LoraConfig(    r=16,    lora_alpha=32,    target_modules=["q_proj", "v_proj"],    lora_dropout=0.05,    task_type="CAUSAL_LM") model = get_peft_model(model, config)

Let's examine each parameter in detail to understand how it affects your fine-tuning:

The r=16 parameter sets the rank of the low-rank decomposition. This is arguably the most important hyperparameter in LoRA—it controls the capacity of the adapter. A rank of 16 means each LoRA matrix will be decomposed into matrices of rank 16, creating 16 "dimensions" of adaptation. Higher ranks (32, 64) give the model more flexibility to adapt but require more parameters and memory. Lower ranks (4, 8) are more parameter-efficient but may limit how much the model can specialize. For most domain-specific tasks, ranks between 8 and 32 work well, with 16 being a solid default choice.

The lora_alpha=32 parameter controls the scaling of the LoRA updates. It works in conjunction with the rank to determine how strongly the LoRA adaptations influence the model's outputs. A common heuristic is to set lora_alpha to twice the rank value, though this can be adjusted based on your observations during training. If the model isn't adapting enough to your domain, you might increase this; if training becomes unstable, you might decrease it.

The target_modules=["q_proj", "v_proj"] parameter specifies which layers in the transformer architecture will receive LoRA adaptations. In transformer models, the attention mechanism uses query (Q), key (K), and value (V) projections. By targeting q_proj and v_proj, we're adapting how the model attends to and processes information, which is often sufficient for domain adaptation. You could also include "k_proj" or even the feed-forward layers ("up_proj", "down_proj") for more comprehensive adaptation, though this increases trainable parameters proportionally.

The lora_dropout=0.05 applies dropout to the LoRA layers during training, providing regularization that helps prevent overfitting to your relatively small domain-specific dataset. A dropout rate of 5% is conservative but effective for most instruction-tuning scenarios.

Finally, task_type="CAUSAL_LM" tells PEFT that you're fine-tuning a causal language model (one that predicts the next token given previous tokens), as opposed to sequence classification or other task types.

After applying the LoRA configuration with get_peft_model(), you can verify how many parameters you'll actually be training:

model.print_trainable_parameters()

This will output something like "trainable params: 14,680,064 || all params: 7,253,680,064 || trainable%: 0.20%"—confirming that you're only updating about 0.2% of the model's parameters. This dramatic reduction is what makes fine-tuning on consumer hardware practical.

Step 6: Train the Model

With LoRA configured, you're ready to begin the actual training process. The Hugging Face Trainer class handles the complexity of the training loop, including batching, gradient accumulation, logging, and checkpointing, allowing you to focus on the hyperparameters that affect your model's performance.

Start by defining the training configuration:

from transformers import TrainingArguments, Trainer training_args = TrainingArguments(    output_dir="./lora_python_qa",    per_device_train_batch_size=2,    gradient_accumulation_steps=4,    num_train_epochs=3,    learning_rate=2e-4,    fp16=True,    logging_steps=10,    save_strategy="epoch",    save_total_limit=2,    warmup_steps=50) trainer = Trainer(    model=model,    args=training_args,    train_dataset=dataset["train"],    tokenizer=tokenizer) trainer.train()

Let's examine the key training arguments and how they impact your fine-tuning:

The output_dir specifies where training artifacts will be saved, including model checkpoints and logs. After training completes, this directory will contain your LoRA adapters, which are typically only 50-100MB despite adapting a 7B parameter model.

Setting per_device_train_batch_size=2 means each GPU will process 2 examples at a time. For 7B models, even with LoRA's reduced memory footprint, you may need to keep batch sizes small to fit in GPU memory. If you encounter out-of-memory errors, reduce this to 1; if you have memory to spare, you can increase it to 4 or higher.

The gradient_accumulation_steps=4 parameter provides a clever way to simulate larger batch sizes without the memory cost. The trainer will accumulate gradients over 4 forward passes before updating weights, giving you an effective batch size of 8 (2 × 4) while only holding 2 examples in memory at once. This tends to improve training stability and final performance compared to using a batch size of 2 alone.

Running for num_train_epochs=3 means the model will see your entire dataset three times. For small domain-specific datasets (100-500 examples), 3-5 epochs is typically appropriate. With larger datasets (1000+ examples), you might reduce this to 1-2 epochs to avoid overfitting. Monitor your training loss—if it plateaus early, you can stop training sooner; if it's still decreasing steadily after 3 epochs, you might benefit from additional training.

The learning_rate=2e-4 (0.0002) is higher than typical full fine-tuning rates but appropriate for LoRA. Since you're only training a small subset of parameters, a higher learning rate helps these parameters adapt more quickly to your domain. Learning rates between 1e-4 and 3e-4 work well for LoRA, though you may need to experiment to find the optimal value for your specific dataset.

Enabling fp16=True uses mixed-precision training, which reduces memory usage by about 40% and speeds up training on modern GPUs with Tensor Cores. This is almost always beneficial when available. If you're using an Ampere or newer GPU (RTX 3000 series, A100, etc.), you could instead use bf16=True for better numerical stability.

The logging_steps=10 parameter controls how frequently training metrics are printed. Every 10 steps, you'll see the current loss, learning rate, and training speed, helping you monitor whether training is progressing normally.

Setting save_strategy="epoch" saves a checkpoint after each complete pass through your dataset, allowing you to select the best-performing epoch if later epochs overfit. Combined with save_total_limit=2, only the two most recent checkpoints are kept, saving disk space.

Finally, warmup_steps=50 gradually increases the learning rate from 0 to the target value over the first 50 optimization steps. This warmup period helps stabilize training in the early stages when the model is adapting most rapidly to your new data distribution.

When you call trainer.train(), you'll see output showing the training progress:

{'loss': 2.1432, 'learning_rate': 0.0001, 'epoch': 0.5}{'loss': 1.8234, 'learning_rate': 0.0002, 'epoch': 1.0}{'loss': 1.4521, 'learning_rate': 0.00015, 'epoch': 1.5}...

Watch for the loss to decrease steadily. For instruction tuning, you should see the loss drop from around 2.0-2.5 initially to somewhere between 0.8-1.5 by the end of training, depending on your dataset size and diversity. If the loss stops decreasing or increases, you may be overfitting—consider reducing the number of epochs or adding more training examples.

After training completes, the LoRA adapter weights will be saved in your output directory. These adapters are small (typically 50-200MB) and can be loaded on top of the base model whenever you need your specialized assistant, making them easy to share, version, and deploy.

Step 7: Test the Fine-Tuned Assistant

After training completes, the most immediate way to understand whether your fine-tuning was successful is to test the model with questions from your target domain. This initial testing phase serves multiple purposes: it gives you qualitative feedback on the model's new capabilities, helps you identify any obvious issues before more rigorous evaluation, and provides concrete examples you can use when demonstrating the model to stakeholders or team members.

The testing process mirrors how you'll eventually use the model in production. You construct a prompt that follows the same instruction format used during training, pass it through the model, and examine the generated response. Here's how to test your Python Q&A assistant:

prompt = """### Instruction:Explain list comprehension in Python. ### Response:""" inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate(    **inputs,    max_new_tokens=200,    temperature=0.7,    top_p=0.9,    do_sample=True) response = tokenizer.decode(outputs[0], skip_special_tokens=True)print(response)

Notice that we've added a few generation parameters beyond the basic max_new_tokens. Setting temperature=0.7 introduces controlled randomness into the generation process—values closer to 0 make outputs more deterministic and focused, while values closer to 1 make them more creative and diverse. For technical Q&A, temperatures between 0.3 and 0.7 tend to work well, balancing accuracy with natural variety in phrasing.

The top_p=0.9 parameter implements nucleus sampling, which considers only the most probable tokens whose cumulative probability exceeds 90%. This prevents the model from occasionally selecting very unlikely tokens that might lead to nonsensical outputs, while still allowing for natural variation. Together with temperature, these parameters give you fine control over the trade-off between accuracy and creativity.

When you run this test, compare the fine-tuned model's response to what the base model would have generated. If your training was effective, you should notice several improvements: the response should be more focused on Python specifically (rather than discussing programming languages in general), it should use terminology and examples appropriate for your target audience, and it should follow any stylistic patterns present in your training data—such as including code examples, using specific explanation structures, or maintaining a particular level of technical depth.

Try testing with several different types of questions to get a sense of the model's capabilities across your domain:

test_questions = [    "Explain list comprehension in Python.",    "What's the difference between a list and a tuple?",    "How do I handle exceptions in Python?",    "Write a function that finds the factorial of a number.",    "Explain the concept of decorators."] for question in test_questions:    prompt = f"""### Instruction:{question} ### Response:"""    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)    outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7, top_p=0.9, do_sample=True)    response = tokenizer.decode(outputs[0], skip_special_tokens=True)    print(f"\n{'='*60}")    print(f"Question: {question}")    print(f"{'='*60}")    print(response)    print()

This systematic testing across multiple question types helps you identify where the model excels and where it might still struggle. You might find that it handles conceptual explanations beautifully but sometimes makes mistakes in code generation, or vice versa. These insights will guide your next steps—whether that means collecting more training examples of certain types, adjusting your training parameters, or simply understanding the model's limitations for deployment.

Step 8: Conduct Systematic Evaluation

While manual testing gives you an intuitive sense of your model's capabilities, systematic evaluation provides the quantitative and qualitative rigor needed to truly understand performance and guide improvements. Proper evaluation answers critical questions: Is the fine-tuned model actually better than the base model? How much better? In what specific areas? Are there failure modes or biases you need to address?

Evaluation for domain-specific assistants typically combines three complementary approaches: manual inspection of responses, systematic testing against a benchmark set of prompts, and direct comparison with baseline models. Each approach illuminates different aspects of model behavior.

Manual Inspection

Manual inspection involves carefully reading through the model's responses to understand not just whether they're correct, but how they're correct—or incorrect. This qualitative analysis often reveals subtle issues that automated metrics miss. As you read responses, ask yourself: Does the explanation make sense? Would someone learning Python understand this? Are there factual errors? Does the response match the style and depth you want?

Create a structured rubric to guide your manual inspection. For a Python Q&A assistant, you might evaluate each response on several dimensions: technical accuracy (is the information correct?), completeness (does it fully answer the question?), clarity (is the explanation easy to understand?), code quality (if code is provided, does it follow best practices?), and appropriateness (is the level of detail suitable for the intended audience?). Rate each dimension on a simple scale—perhaps 1-5 stars—and track these ratings across multiple test questions.

Benchmark Testing

Create a benchmark set of prompts that comprehensively covers your domain. This set should include questions of varying difficulty, different question types (conceptual explanations, code generation, debugging help, best practices), and edge cases that might trip up the model. For a Python assistant, you might include 20-50 carefully chosen questions spanning basic syntax, data structures, control flow, functions, object-oriented programming, common libraries, and practical problem-solving.

Here's an example benchmark set with diverse question types:

benchmark_prompts = [    # Conceptual understanding    "What is a dictionary in Python and when should I use one?",    "Explain the difference between mutable and immutable objects.",    "What is recursion and how does it work?",        # Syntax and basics    "How do you create a list in Python?",    "What are Python decorators?",    "Explain the use of *args and **kwargs.",        # Code generation    "Write a function to reverse a string.",    "Create a class that represents a bank account with deposit and withdrawal methods.",    "Write a function that finds all prime numbers up to n.",        # Debugging and problem-solving    "Why am I getting 'IndexError: list index out of range'?",    "How can I improve the performance of nested loops?",    "What's wrong with using mutable default arguments?",        # Advanced topics    "Explain generators in Python and provide an example.",    "What are context managers and how do I create one?",    "Describe how Python's garbage collection works."]

Run your fine-tuned model on each benchmark prompt and save the outputs. Then evaluate each response using your rubric. Calculate aggregate scores across all prompts to get an overall performance metric, but also look at performance broken down by question category—this reveals specific strengths and weaknesses in the model's domain knowledge.

Baseline Comparison

To truly understand whether your fine-tuning improved the model, you need a baseline for comparison. The most direct baseline is the original, unfine-tuned model. Run the same benchmark prompts through both the base model and your fine-tuned version, then compare their responses side-by-side.

This comparison should measure several dimensions of quality. First, assess correctness—are the responses factually accurate? For technical content like Python programming, there are often objectively correct and incorrect answers, making this relatively straightforward to evaluate. Second, measure clarity—even if both models give correct answers, does one explain the concept more clearly or at a more appropriate level? Third, track the hallucination rate—how often does each model confidently state incorrect information or invent non-existent Python features?

Create a structured comparison document where you can track these metrics:

import pandas as pd evaluation_results = [] for prompt in benchmark_prompts:    # Generate with base model    base_response = generate_response(base_model, prompt)        # Generate with fine-tuned model    finetuned_response = generate_response(finetuned_model, prompt)        # Manual scoring (you would do this part manually)    evaluation_results.append({        'prompt': prompt,        'base_correctness': score_correctness(base_response),        'finetuned_correctness': score_correctness(finetuned_response),        'base_clarity': score_clarity(base_response),        'finetuned_clarity': score_clarity(finetuned_response),        'base_hallucination': contains_hallucination(base_response),        'finetuned_hallucination': contains_hallucination(finetuned_response)    }) df = pd.DataFrame(evaluation_results)print(df.describe())  # Statistical summaryprint(f"Average improvement in correctness: {(df['finetuned_correctness'] - df['base_correctness']).mean()}")print(f"Hallucination rate - Base: {df['base_hallucination'].mean():.1%}, Fine-tuned: {df['finetuned_hallucination'].mean():.1%}")

This systematic comparison provides concrete evidence of improvement (or lack thereof) and helps justify the fine-tuning effort. If you find that the fine-tuned model isn't significantly better than the base model, that's valuable information too—it might indicate issues with your training data quality, insufficient training, or that the base model already performs well enough on your domain.

Document specific examples where the fine-tuned model excels and where it still struggles. These concrete cases are invaluable for communicating model capabilities to stakeholders and for guiding future iterations of your dataset and training process.

Step 9: Build an Interactive CLI Assistant

Having successfully trained and evaluated your model, the final step is to make it easily accessible through an interactive interface. While a command-line interface (CLI) might seem simple compared to web or mobile applications, it's often the most practical choice for initial deployment. A CLI assistant is quick to build, easy to test and iterate on, and requires no web server infrastructure. It's particularly well-suited for developer tools, internal company utilities, or proof-of-concept demonstrations.

The core of your CLI assistant is a simple loop that continuously prompts the user for questions, generates responses using your fine-tuned model, and displays the results. Here's a production-ready implementation with helpful features:

import torchfrom transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModel def load_model(base_model_name, adapter_path):    """Load the base model and LoRA adapter."""    print("Loading model... This may take a moment.")    tokenizer = AutoTokenizer.from_pretrained(base_model_name)    base_model = AutoModelForCausalLM.from_pretrained(        base_model_name,        torch_dtype=torch.float16,        device_map="auto"    )    model = PeftModel.from_pretrained(base_model, adapter_path)    model.eval()  # Set to evaluation mode    print("Model loaded successfully!\n")    return model, tokenizer def generate_response(model, tokenizer, question, max_tokens=300):    """Generate a response to the user's question."""    prompt = f"""### Instruction:{question} ### Response:"""        inputs = tokenizer(prompt, return_tensors="pt").to(model.device)        with torch.no_grad():  # Disable gradient calculation for inference        outputs = model.generate(            **inputs,            max_new_tokens=max_tokens,            temperature=0.7,            top_p=0.9,            do_sample=True,            pad_token_id=tokenizer.eos_token_id        )        full_response = tokenizer.decode(outputs[0], skip_special_tokens=True)    # Extract just the response part (after "### Response:")    response = full_response.split("### Response:")[-1].strip()    return response def main():    """Run the interactive Python Q&A assistant."""    print("="*60)    print("Python Q&A Assistant")    print("="*60)    print("Ask any Python-related question, or type 'quit' to exit.")    print("="*60 + "\n")        # Load model    model, tokenizer = load_model(        base_model_name="mistralai/Mistral-7B-v0.1",        adapter_path="./lora_python_qa"    )        while True:        # Get user input        question = input("\n🐍 Your question: ").strip()                # Check for exit command        if question.lower() in ['quit', 'exit', 'q']:            print("\nThank you for using Python Q&A Assistant!")            break                # Skip empty questions        if not question:            continue                # Generate and display response        print("\n💡 Assistant:", end=" ")        response = generate_response(model, tokenizer, question)        print(response)        print("\n" + "-"*60) if __name__ == "__main__":    main()

This implementation includes several refinements that make it more robust and user-friendly. The load_model() function encapsulates the model loading logic, making it reusable and providing clear feedback to the user while the model loads. Using model.eval() ensures the model is in evaluation mode, which disables dropout and other training-specific behaviors. The torch.no_grad() context manager during generation prevents PyTorch from building computation graphs for gradient calculation, reducing memory usage during inference.

The response extraction logic (full_response.split("### Response:")[-1].strip()) isolates just the model's answer from the complete generated text, which includes the original instruction prompt. This gives users a cleaner, more chat-like experience. The main loop handles edge cases like empty inputs and provides multiple ways to exit (quit, exit, q), improving usability.

You can enhance this basic CLI with additional features as needed. For example, you might add a help command that displays example questions, implement conversation history to enable follow-up questions, or add the ability to save particularly useful responses to a file. Here's an example with conversation history:

def main_with_history():    """Interactive assistant with conversation history."""    model, tokenizer = load_model("mistralai/Mistral-7B-v0.1", "./lora_python_qa")    conversation_history = []        print("Type 'history' to see past questions, 'clear' to reset, 'quit' to exit.\n")        while True:        question = input("\n🐍 Your question: ").strip()                if question.lower() in ['quit', 'exit', 'q']:            break        elif question.lower() == 'history':            print("\n📝 Conversation History:")            for i, (q, a) in enumerate(conversation_history, 1):                print(f"\n{i}. Q: {q}")                print(f"   A: {a[:100]}..." if len(a) > 100 else f"   A: {a}")            continue        elif question.lower() == 'clear':            conversation_history.clear()            print("✓ History cleared")            continue        elif not question:            continue                response = generate_response(model, tokenizer, question)        print("\n💡 Assistant:", response)                conversation_history.append((question, response))        print("\n" + "-"*60) if __name__ == "__main__":    main_with_history()

With this interactive CLI assistant, you now have a fully functional, domain-specific AI tool that you or your team can use immediately. It demonstrates the end-to-end journey from raw model to practical application—a journey that encompasses dataset creation, efficient fine-tuning, rigorous evaluation, and thoughtful deployment.

Key Takeaways from Project 1

This capstone project synthesized concepts from across the entire book into a cohesive, practical implementation. Through building a Python Q&A assistant from scratch, you've gained hands-on experience with the complete workflow that AI engineers use to create specialized models in industry settings.

You learned how to construct high-quality instruction datasets, recognizing that the dataset is the foundation of model behavior. The process of writing clear instructions, crafting detailed responses, and formatting data correctly taught you that fine-tuning is as much about data curation as it is about training algorithms. The quality of your model's outputs is fundamentally limited by the quality of your training examples.

You applied LoRA fine-tuning in practice, experiencing firsthand how PEFT methods democratize access to large language model customization. By training a 7B parameter model on consumer hardware, you've seen that parameter-efficient methods aren't just theoretical optimizations—they're the key to making modern AI development accessible without massive computational budgets. Understanding how to configure LoRA's hyperparameters (rank, alpha, target modules) gives you the tools to balance model capacity, training efficiency, and final performance.

You developed domain-specific assistants that outperform general-purpose models on specialized tasks. This project illustrated a crucial principle: a smaller model fine-tuned on domain-specific data often outperforms a larger general model on tasks within that domain. This insight has profound implications for how you approach AI projects—sometimes the solution isn't a bigger model, but a more focused one.

You implemented systematic evaluation processes that go beyond anecdotal testing. By combining manual inspection, benchmark testing, and baseline comparison, you've learned to rigorously assess whether your fine-tuning efforts actually improved model performance. This evaluation methodology is essential for making data-driven decisions about model development and for communicating results to stakeholders.

Finally, you created interactive tools that make AI accessible to end users. The journey from trained model to usable application taught you that model development is only part of the picture—deployment, interface design, and user experience matter just as much for creating value with AI.

The techniques you've practiced in this project—instruction dataset creation, PEFT fine-tuning, domain specialization, systematic evaluation, and practical deployment—form the core toolkit for building AI assistants in professional settings. Whether you're developing customer service bots, code review assistants, medical information systems, or legal document analyzers, these same principles and processes apply. You've not just built a Python Q&A assistant; you've learned a replicable methodology for creating specialized AI systems for any domain.