Questions
Select the best answer for each question.
1. What is the primary goal of LLM evaluation?
A. To maximize GPU utilization during training
B. To measure model behavior, reliability, and alignment across tasks
C. To increase the number of parameters in the model
D. To reduce the size of the training dataset
2. What distinguishes HELM from many traditional evaluation benchmarks?
A. It evaluates only mathematical reasoning tasks
B. It evaluates models using a single accuracy metric
C. It measures model performance across multiple dimensions such as fairness, robustness, and toxicity
D. It evaluates only conversational models
3. What aspect of language models does MT-Bench primarily evaluate?
A. Tokenization efficiency
B. Multi-turn conversational ability and context retention
C. GPU memory optimization
D. Model training speed
4. In Arena-style evaluation, how are models typically compared?
A. By comparing training loss values
B. By ranking models based on parameter count
C. By using pairwise comparisons judged by humans or AI evaluators
D. By measuring GPU usage during inference
5. What is the key difference between truthfulness and grounding?
A. Truthfulness refers to stylistic quality, while grounding refers to grammar
B. Truthfulness refers to factual correctness, while grounding refers to whether the response is supported by provided context
C. Truthfulness refers to speed, while grounding refers to accuracy
D. Truthfulness refers to model size, while grounding refers to training data
6. Which metric is commonly used to evaluate summarization tasks?
A. BLEU
B. ROUGE
C. Perplexity
D. FID
7. What is a hallucination in the context of large language models?
A. A model generating text faster than expected
B. A model producing outputs unrelated to its training data
C. A model generating confident but incorrect or unsupported information
D. A model refusing to answer a prompt
8. Why is execution-based testing often used to evaluate code generation models?
A. Because code responses must compile and pass functional tests
B. Because code generation is evaluated using grammar rules
C. Because execution testing improves training speed
D. Because it reduces GPU memory usage
9. What is one risk of excessive safety alignment?
A. Increased GPU costs
B. Excessive refusal of harmless prompts
C. Reduced dataset size
D. Lower tokenization speed
10. What is the purpose of toxicity classifiers in LLM evaluation?
A. To measure how long a model response is
B. To detect harmful or abusive language in model outputs
C. To evaluate grammar quality
D. To optimize training datasets
11. Why is multi-turn evaluation important for conversational AI systems?
A. Because single-turn benchmarks already measure all conversational abilities
B. Because conversational systems must maintain context and consistency across dialogue turns
C. Because it reduces inference latency
D. Because it improves GPU efficiency
12. What is one advantage of pairwise evaluation methods such as those used in Chatbot Arena?
A. They measure tokenization speed
B. They directly capture human preference between responses
C. They eliminate the need for training data
D. They reduce model size
13. What is the main purpose of a hallucination-detection pipeline in evaluation?
A. To speed up training
B. To identify unsupported or fabricated claims in model responses
C. To compress training datasets
D. To optimize tokenization
14. Why should evaluation datasets resemble real-world usage scenarios?
A. Because models perform best when prompts are random
B. Because evaluation should reflect how users actually interact with the system
C. Because benchmarks require informal language
D. Because evaluation must reduce dataset size
15. What is one key takeaway from modern LLM evaluation practices?
A. A single metric is enough to measure model performance
B. Evaluation should rely only on automated benchmarks
C. Model evaluation requires multiple methods and human judgment
D. Evaluation is unnecessary after fine-tuning