[Paper Review] Can multiple-choice questions really be useful in detecting the abilities of LLMs?

Wangyue Li, Liangzhi Li+4 more|arXiv (Cornell University)|Mar 26, 2024

Employer Branding and e-HRM7 citations

TL;DR

The paper analyzes MCQ efficacy for evaluating LLMs, revealing order sensitivity and misalignment with long-form generation, and proposes methods to measure consistency and confidence across QA formats.

ABSTRACT

Multiple-choice questions (MCQs) are widely used in the evaluation of large language models (LLMs) due to their simplicity and efficiency. However, there are concerns about whether MCQs can truly measure LLM's capabilities, particularly in knowledge-intensive scenarios where long-form generation (LFG) answers are required. The misalignment between the task and the evaluation method demands a thoughtful analysis of MCQ's efficacy, which we undertake in this paper by evaluating nine LLMs on four question-answering (QA) datasets in two languages: Chinese and English. We identify a significant issue: LLMs exhibit an order sensitivity in bilingual MCQs, favoring answers located at specific positions, i.e., the first position. We further quantify the gap between MCQs and long-form generation questions (LFGQs) by comparing their direct outputs, token logits, and embeddings. Our results reveal a relatively low correlation between answers from MCQs and LFGQs for identical questions. Additionally, we propose two methods to quantify the consistency and confidence of LLMs' output, which can be generalized to other QA evaluation benchmarks. Notably, our analysis challenges the idea that the higher the consistency, the greater the accuracy. We also find MCQs to be less reliable than LFGQs in terms of expected calibration error. Finally, the misalignment between MCQs and LFGQs is not only reflected in the evaluation performance but also in the embedding space. Our code and models can be accessed at https://github.com/Meetyou-AI-Lab/Can-MC-Evaluate-LLMs.

Motivation & Objective

• Assess whether MCQs accurately measure LLM abilities across languages (Chinese and English) and datasets.
• Examine how option ordering in MCQs biases LLM outputs.
• Compare MCQ and long-form generation (LFGQ) formats across direct outputs, token logits, and embeddings.
• Propose methods to quantify output consistency and model confidence.
• Provide guidance on when to use MCQs versus LFGQs in QA benchmarks.

Proposed method

• Evaluate nine LLMs on four QA datasets (CARE-MI in Chinese, M3KE in Chinese, ARC in English, MATH in English).
• Test order sensitivity by rearranging MCQ options and applying chi-squared tests to detect distribution shifts.
• Compare MCQ and LFGQ formats in direct output, token logits, and hidden embedding spaces (with analyses like unified confidence and ECE).
• Analyze correlation between formats and between consistency and accuracy.
• Use pre- and post-prompts to elicit concise responses and enable automatic confidence calculations.

Figure 1 : Example of order sensitivity experiments, in which the correct answer is D. In the ARC dataset, when predicting a wrong option (A, B, or C), the LLM prefers the option located in the first position.

Experimental results

Research questions

• RQ1 How does the arrangement of MCQ options influence LLMs’ responses in bilingual datasets?
• RQ2 What methodologies are appropriate to compare MCQs and LFGQs across direct outputs, token logits, and embedding spaces?
• RQ3 What is the level of misalignment between MCQs and LFGQs, and how do formats differ in calibration and consistency?

Key findings

• LLMs exhibit order sensitivity in bilingual MCQs, preferring answers in the first position.
• Answers from MCQs and LFGQs for identical questions show low correlation across formats.
• Higher response consistency does not necessarily translate to higher accuracy.
• MCQs yield poorer calibration (higher ECE) than LFGQs and TFQs, and embedding spaces show misalignment between formats in several layers.
• Embeddings from MCQs and LFGQs are separable in some layers but converge in later layers for some models.

Figure 2 : Comparison between MCQs and LFGQs on the CARE-MI dataset. Top: accuracy and pearson correlation. Bottom: MCQ vs LFGQs matrix.