[Paper Review] Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models

Daman Arora, Himanshu Singh+1 more|arXiv (Cornell University)|May 24, 2023

Topic Modeling6 citations

TL;DR

This paper introduces JEEBench, a challenging benchmark of 515 pre-engineering physics, chemistry, and mathematics problems from the IIT JEE-Advanced exam, designed to test long-horizon reasoning and deep domain knowledge. Despite advanced prompting techniques like chain-of-thought and self-consistency, GPT-4 achieves only 33.8% on the 2023 test set—improving to the top 10–20% of human scorers only after applying confidence-thresholding post-hoc, revealing critical gaps in algebraic manipulation, concept retrieval, and risk assessment.

ABSTRACT

The performance of large language models (LLMs) on existing reasoning benchmarks has significantly improved over the past years. In response, we present JEEBench, a considerably more challenging benchmark dataset for evaluating the problem solving abilities of LLMs. We curate 515 challenging pre-engineering mathematics, physics and chemistry problems from the highly competitive IIT JEE-Advanced exam. Long-horizon reasoning on top of deep in-domain knowledge is essential for solving problems in this benchmark. Our evaluation on various open-source and proprietary models reveals that the highest performance, even after using techniques like self-consistency, self-refinement and chain-of-thought prompting, is less than 40%. The typical failure modes of GPT-4, the best model, are errors in algebraic manipulation, difficulty in grounding abstract concepts into mathematical equations accurately and failure in retrieving relevant domain-specific concepts. We also observe that by mere prompting, GPT-4 is unable to assess risk introduced by negative marking for incorrect answers. For this, we develop a post-hoc confidence-thresholding method over self-consistency, which enables effective response selection. We hope that our challenging benchmark will guide future re-search in problem-solving using LLMs.

Motivation & Objective

• To develop a rigorous benchmark for evaluating the problem-solving capabilities of large language models (LLMs) beyond existing reasoning datasets.
• To assess the limits of current LLMs, particularly GPT-4, in solving high-difficulty, multi-step problems requiring deep domain-specific knowledge in STEM.
• To investigate failure modes in LLM reasoning, especially in algebraic manipulation, concept retrieval, and risk-aware decision-making under negative marking.
• To propose and evaluate a post-hoc confidence-thresholding method that improves response selection using self-consistency outputs.
• To provide a reliable, contamination-controlled dataset to guide future research in reasoning-capable LLMs.

Proposed method

• Curated 515 challenging problems from 8 editions of the IIT JEE-Advanced exam, focusing on physics, chemistry, and mathematics requiring multi-step reasoning and domain-specific knowledge.
• Applied chain-of-thought (CoT) prompting and self-consistency (SC) to generate multiple reasoning trajectories per problem for improved robustness.
• Implemented a post-hoc confidence-thresholding method over self-consistency outputs, using relative frequency of predicted answers as a proxy for confidence.
• Conducted extensive qualitative and quantitative analysis across open-source and proprietary LLMs, including GPT-4, on the benchmark.
• Performed a contamination study by searching for problem instances in public corpora and evaluating model performance on 2023 questions, which were deemed uncontaminated.
• Projected human performance to normalize scores and compare LLM performance against human benchmarks from the 2023 exam.

Experimental results

Research questions

• RQ1 Can current LLMs, including GPT-4, reliably solve high-difficulty, multi-step STEM problems requiring deep in-domain knowledge and long-horizon reasoning?
• RQ2 What are the primary failure modes of state-of-the-art LLMs in complex problem-solving, particularly in algebraic manipulation and concept retrieval?
• RQ3 How effective are self-consistency and self-refinement techniques in improving reasoning performance on this benchmark?
• RQ4 Can a simple post-hoc confidence-thresholding method improve LLM performance in exam-like settings with negative marking?
• RQ5 To what extent has the JEEBench dataset been contaminated by exposure in pre-training data, and does this affect model performance evaluation?

Key findings

• GPT-4 achieved a score of 33.8% on the 2023 JEE-Advanced test set using chain-of-thought and self-consistency prompting, indicating limited reasoning robustness.
• After applying confidence-thresholding to self-consistency outputs, GPT-4’s performance improved to the top 10–20% of human scorers on the 2023 exam.
• The highest performance across all evaluated models, even with advanced prompting, remained below 40%, highlighting persistent challenges in LLM reasoning.
• GPT-4 frequently failed in algebraic manipulation and concept retrieval, despite strong logical reasoning in some cases.
• The model struggled to account for negative marking in the exam setting, and its performance dropped when prompted with the marking scheme.
• Contamination of the dataset was estimated at only 6%, suggesting that GPT-4’s performance is a genuine indicator of reasoning ability rather than memorization.