[Paper Review] FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks

Miles Wang, Robi Lin+5 more|arXiv (Cornell University)|Jan 29, 2026

Machine Learning in Materials Science0 citations

TL;DR

FrontierScience introduces a two-track benchmark (Olympiad and Research) with hundreds of expert-level physics, chemistry, and biology problems to evaluate AI reasoning; GPT-5.2 leads on Olympiad (77%) and trails on Research (25%).

ABSTRACT

We introduce FrontierScience, a benchmark evaluating expert-level scientific reasoning in frontier language models. Recent model progress has nearly saturated existing science benchmarks, which often rely on multiple-choice knowledge questions or already published information. FrontierScience addresses this gap through two complementary tracks: (1) Olympiad, consisting of international olympiad problems at the level of IPhO, IChO, and IBO, and (2) Research, consisting of PhD-level, open-ended problems representative of sub-tasks in scientific research. FrontierScience contains several hundred questions (including 160 in the open-sourced gold set) covering subfields across physics, chemistry, and biology, from quantum electrodynamics to synthetic organic chemistry. All Olympiad problems are originally produced by international Olympiad medalists and national team coaches to ensure standards of difficulty, originality, and factuality. All Research problems are research sub-tasks written and verified by PhD scientists (doctoral candidates, postdoctoral researchers, or professors). For Research, we introduce a granular rubric-based evaluation framework to assess model capabilities throughout the process of solving a research task, rather than judging only a standalone final answer.

Motivation & Objective

• Assess AI capabilities in expert-level scientific reasoning across constrained (Olympiad) and open-ended (Research) tasks.
• Provide novel, expert-authored problems verified by domain specialists to ensure difficulty and originality.
• Introduce a rubric-based evaluation framework for open-ended research tasks to diagnose model strengths and weaknesses.

Proposed method

• Two-track dataset: FrontierScience-Olympiad with short-answer, problem-solving questions; FrontierScience-Research with PhD-level, open-ended subproblems.
• Problems written and verified by domain experts across physics, chemistry, and biology; each Research problem includes a 10-point rubric and explanatory solution path.
• Rubric-based grading for Research tasks to assess intermediate reasoning and final answers, using model judges (GPT-5) for scoring.
• Evaluation uses multiple frontier models under high reasoning effort, with 20 trials for Olympiad and 30 trials for Research; model judgments are performed by a GPT-5-based judge.
• Open-source gold set: 100 Olympiad questions and 60 Research questions after meta-review and filtering from larger corpora.

Figure 1: Sample FrontierScience-Olympiad problems. Each task in FrontierScience is written and verified by a domain expert in physics, chemistry, or biology. For the Olympiad set, all experts achieved a medal in an international olympiad competition.

Experimental results

Research questions

• RQ1 How well do frontier AI models solve Olympiad-style physics, chemistry, and biology problems that have closed-form or numeric/expressible answers?
• RQ2 How well do frontier AI models tackle open-ended, PhD-level research subproblems requiring reasoning, justification, and rubric-based evaluation?
• RQ3 What are the strengths and failure modes of current frontier models across constrained versus open-ended scientific tasks?
• RQ4 How does model performance vary by discipline (physics, chemistry, biology) within each track?

Key findings

• GPT-5.2 achieves the highest overall performance among tested models on FrontierScience, with 77% on Olympiad problems and 25% on Research problems.
• Gemini 3 Pro is comparable to GPT-5.2 on Olympiad problems (76%), and GPT-5 ties GPT-5.2 on the Research set (25%).
• Across problems, models show stronger performance on chemistry, then physics, then biology for Olympiad problems; on Research, chemistry leads, followed by biology and physics.
• More test-time tokens improve GPT-5.2 performance (Olympiad: 67.5% to 77.1%; Research: 18% to 25%).
• The evaluation pipeline uses a rubric-based scoring architecture for Research tasks and a numeric/expressional match for Olympiad tasks, with a GPT-5 judge assessing rubric fulfillment.

Figure 2: Sample FrontierScience-Research problems. For the Research set, all experts hold a relevant PhD degree. The corresponding rubrics to these sample tasks can be found in Appendix A .