Questions

Select the correct answer for each question.

1. What is the main goal of Parameter-Efficient Fine-Tuning (PEFT)?

A) Increase the size of the model

B) Modify only a small subset of parameters to adapt model behavior

C) Replace the tokenizer

D) Eliminate the need for datasets

2. In LoRA, what happens to the original weight matrices?

A) They are deleted

B) They are retrained entirely

C) They are frozen, and low-rank matrices are added

D) They are randomly reinitialized

3. What does the LoRA rank parameter r control?

A) The number of GPUs required

B) The number of training epochs

C) The dimensionality of the low-rank adaptation matrices

D) The vocabulary size

4. Why is QLoRA more memory efficient than standard LoRA?

A) It removes attention layers

B) It uses 4-bit quantization for base model weights

C) It reduces the dataset size

D) It eliminates gradient computation

5. What is one risk when increasing LoRA rank too much?

A) The tokenizer stops working

B) The model becomes smaller

C) Overfitting or instability may increase

D) Quantization automatically disables

6. What does model.printtrainableparameters() help you verify?

A) Dataset quality

B) Learning rate stability

C) Whether only adapter parameters are trainable

D) GPU temperature

7. Which of the following is a major advantage of modular adapters?

A) They eliminate inference latency

B) They allow switching between tasks without retraining the base model

C) They increase model parameter count

D) They improve tokenizer speed

8. What is BitFit?

A) Training only attention heads

B) Training only bias parameters

C) Training only embedding layers

D) Training only quantized weights

9. Prefix tuning adapts model behavior by:

A) Modifying all transformer weights

B) Adding learned virtual tokens to the input

C) Increasing context length

D) Changing the optimizer

10. If your PEFT-trained model behaves identically to the base model, what is a likely issue?

A) The GPU is overheating

B) LoRA may not be attached to the correct target modules

C) The dataset is too large

D) The tokenizer is incorrect

11. What is a common cause of instability when using QLoRA?

A) Too few GPUs

B) Incorrect compute dtype or incompatible quantization settings

C) Too small a tokenizer vocabulary

D) Too much evaluation data

12. Why must the base model version match when loading adapters?

A) Because adapters modify tokenizer vocabulary

B) Because adapters depend on the exact architecture and weight layout

C) Because adapters store dataset information

D) Because adapters modify optimizer state

13. Compared to full SFT, PEFT typically results in:

A) Larger checkpoint files

B) More trainable parameters

C) Smaller storage footprint

D) Longer training time

14. When is full SFT preferable over PEFT?

A) When hardware resources are extremely limited

B) When maximum behavioral shift is required

C) When dataset size is small

D) When quantization is unavailable

15. What is the key philosophical difference between full fine-tuning and PEFT?

A) Full fine-tuning trains faster

B) PEFT changes tokenizer behavior

C) PEFT adjusts behavior without rewriting the entire model

D) Full fine-tuning requires no dataset