Chapter 2 Summary

Section 5 of 5-~ 3 min read-Synced from Cuantum content

In Chapter 1, you learned how to fine-tune a model by updating all its parameters. That approach gave you full control, but it also came with cost — in memory, time, and storage.

In this chapter, you learned something equally powerful: you don’t always need to retrain everything to change behavior.

Parameter-Efficient Fine-Tuning (PEFT) is built on a simple insight: large language models already contain enormous knowledge. Most of the time, you don’t need to rewrite that knowledge — you only need to adjust how it is expressed.

You explored the core PEFT methods:

LoRA, which injects low-rank trainable matrices into attention layers.
QLoRA, which combines 4-bit quantization with LoRA to reduce memory dramatically.
Adapters, which add small bottleneck layers while freezing the base model.
BitFit, which updates only bias terms.
Prefix tuning, which learns virtual tokens to steer behavior.

Each method offers a different trade-off between efficiency and expressive power. But all share the same principle: freeze most parameters, train only what matters.

You then moved into real implementation using:

The PEFT library for attaching adapters.
The TRL SFTTrainer for streamlined supervised training.
Quantization with bitsandbytes for memory efficiency.
Modular saving and loading of adapters for task flexibility.

You also learned how to:

Verify trainable parameter counts.
Measure memory differences between full precision and QLoRA.
Adjust LoRA rank and observe its effect.
Switch between multiple adapters without reloading the base model.
Inspect model modules to avoid configuration errors.

Perhaps most importantly, you saw that PEFT is not just about saving resources. It fundamentally changes how you think about deployment:

Instead of training one massive model per task, you can maintain:

One base model
Multiple lightweight adapters
Task-specific behavior on demand

That modularity is transformative in real-world systems.

You also examined common failure modes:

Incorrect target modules
Ineffective rank selection
Quantization instability
Adapter misalignment
Silent configuration errors

You learned that most PEFT issues are not mysterious. They are configuration, data, or debugging discipline problems.

By the end of this chapter, you now understand:

Why PEFT exists
How it works mathematically
How to implement it
How to debug it
When to use it instead of full fine-tuning

You are now capable of fine-tuning large models on modest hardware — and doing so intelligently.

In the next chapter, we will move beyond supervised fine-tuning and enter a more advanced layer of alignment:

Reinforcement Learning with Human and AI Feedback.

Here, you will learn how models are not only trained to follow instructions — but trained to prefer better answers.

Take a moment before continuing. If you can explain the difference between LoRA and full SFT — and describe when QLoRA would be preferable — then you’ve truly absorbed this chapter.