[Paper Review] DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data

Huajian Xin, Daya Guo+7 more|arXiv (Cornell University)|May 23, 2024

Scientific Computing and Data Management4 citations

TL;DR

This paper introduces DeepSeek-Prover, an iterative framework that generates 8 million synthetic Lean 4 proofs from high-school and undergraduate math competition problems. By combining autoformalization, hypothesis rejection, and iterative fine-tuning of DeepSeekMath 7B, the method achieves 46.3% whole-proof generation accuracy on miniF2F with 64 samples—surpassing GPT-4 (23.0%) and a prior SOTA method (41.0%), and proving 5 of 148 FIMO problems where GPT-4 failed entirely.

ABSTRACT

Proof assistants like Lean have revolutionized mathematical proof verification, ensuring high accuracy and reliability. Although large language models (LLMs) show promise in mathematical reasoning, their advancement in formal theorem proving is hindered by a lack of training data. To address this issue, we introduce an approach to generate extensive Lean 4 proof data derived from high-school and undergraduate-level mathematical competition problems. This approach involves translating natural language problems into formal statements, filtering out low-quality statements, and generating proofs to create synthetic data. After fine-tuning the DeepSeekMath 7B model on this synthetic dataset, which comprises 8 million formal statements with proofs, our model achieved whole-proof generation accuracies of 46.3% with 64 samples and 52% cumulatively on the Lean 4 miniF2F test, surpassing the baseline GPT-4 at 23.0% with 64 samples and a tree search reinforcement learning method at 41.0%. Additionally, our model successfully proved 5 out of 148 problems in the Lean 4 Formalized International Mathematical Olympiad (FIMO) benchmark, while GPT-4 failed to prove any. These results demonstrate the potential of leveraging large-scale synthetic data to enhance theorem-proving capabilities in LLMs. Both the synthetic dataset and the model will be made available to facilitate further research in this promising field.

Motivation & Objective

• To address the scarcity of high-quality formal proof data for training LLMs in automated theorem proving (ATP).
• To improve the scalability and quality of synthetic formal proof data from informal mathematical problems.
• To enhance LLM performance in formal theorem proving through iterative synthetic data generation and fine-tuning.
• To enable LLMs to outperform existing models like GPT-4 on standardized ATP benchmarks such as miniF2F and FIMO.
• To release a large-scale synthetic dataset and fine-tuned model to accelerate research in formal mathematical reasoning.

Proposed method

• Translating informal high-school and undergraduate math competition problems into formal Lean 4 statements using a zero-shot LLM prompt.
• Filtering low-quality statements via a quality scoring model and rejecting inconsistent hypotheses through a hypothesis rejection strategy.
• Generating proofs for formal statements using a large language model and validating correctness using the Lean 4 verifier.
• Iteratively retraining the LLM on high-quality, verified theorem-proof pairs to improve proof quality over multiple cycles.
• Accelerating proof generation by proving both a statement and its negation in parallel, terminating early upon success.
• Fine-tuning the DeepSeekMath 7B model on the resulting 8 million synthetic formal statement-proof pairs for ATP tasks.

Figure 1 : An overview of our approach.

Experimental results

Research questions

• RQ1 Can large-scale synthetic proof data generated from informal math problems significantly improve LLM performance in formal theorem proving?
• RQ2 Does an iterative training framework that reuses verified proofs lead to progressively better proof generation quality?
• RQ3 Can synthetic data generation outperform existing SOTA methods like GPT-4 and reinforcement learning baselines on standardized ATP benchmarks?
• RQ4 To what extent can LLMs generalize to complex, competition-level problems when trained on synthetic formal data?
• RQ5 Can the model identify and reject logically inconsistent formalizations during the autoformalization process?

Key findings

• The DeepSeek-Prover model achieved 46.3% whole-proof generation accuracy on the miniF2F test set with 64 samples, surpassing GPT-4’s 23.0% and a prior reinforcement learning method’s 41.0%.
• With 100 samples, the model solved 4 out of 148 problems in the FIMO benchmark, while GPT-4 solved none.
• With 4096 samples, the model solved 5 problems in the FIMO benchmark, demonstrating strong scalability and capability on complex Olympiad-level problems.
• Ablation studies showed that model performance on miniF2F improved progressively with each iteration, confirming the effectiveness of the iterative training loop.
• The model successfully identified and rejected an inconsistent formalization in a case study, demonstrating robustness in detecting flawed hypotheses.
• The synthetic dataset of 8 million formal statements with proofs, along with the fine-tuned DeepSeekMath 7B model, is publicly released to support future research in formal theorem proving.