[Paper Review] Lean Copilot: Large Language Models as Copilots for Theorem Proving in Lean

Peiyang Song, Kaiyu Yang+1 more|arXiv (Cornell University)|Apr 18, 2024

Business Process Modeling and AnalysisBusiness, Management and Accounting3 citations

TL;DR

This paper introduces Lean Copilot, a framework enabling large language models (LLMs) to natively run inference within the Lean theorem prover via foreign function interface (FFI). It allows LLMs to act as copilots for human users by suggesting tactics, searching for proofs, and selecting relevant premises, significantly reducing manual effort and outperforming rule-based automation in proof completion and autonomy.

ABSTRACT

Neural theorem proving combines large language models (LLMs) with proof assistants such as Lean, where the correctness of formal proofs can be rigorously verified, leaving no room for hallucination. With existing neural theorem provers pretrained on a fixed collection of data and offering valuable suggestions at times, it is challenging for them to continually prove novel theorems in a fully autonomous mode, where human insights may be critical. In this paper, we explore LLMs as copilots that assist humans in proving theorems. We introduce Lean Copilot, a general framework for running LLM inference natively in Lean. It enables programmers to build various LLM-based proof automation tools that integrate seamlessly into the workflow of Lean users. Lean users can use our pretrained models or bring their own ones that run either locally (with or without GPUs) or on the cloud. Using Lean Copilot, we build LLM-based tools that suggest proof steps, complete proof goals, and select relevant premises. Experimental results on the Mathematics in Lean textbook demonstrate the effectiveness of our method compared to existing rule-based proof automation in Lean (aesop). When assisting humans, Lean Copilot requires only 2.08 manually-entered proof steps on average (3.86 required by aesop); when automating the theorem proving process, Lean Copilot automates 74.2% proof steps on average, 85% better than aesop (40.1%). We open source all code and artifacts under a permissive MIT license to facilitate further research.

Motivation & Objective

• To address the limitations of fully autonomous LLM-based theorem provers, which struggle with novel or domain-divergent theorems due to lack of human insight.
• To explore a human-AI collaborative paradigm in interactive theorem proving (ITP), where LLMs act as copilots to automate routine proof steps while humans provide high-level guidance.
• To build a general, extensible framework that enables seamless integration of LLM inference directly within the Lean proof environment, overcoming the limitations of external model calls.
• To empirically evaluate LLM-based proof automation tools—tactic suggestion, proof search, and premise selection—against existing rule-based systems in Lean.
• To open-source the framework and models under MIT license to accelerate research in LLM-powered formal mathematics and verification.

Proposed method

• Leverages Foreign Function Interface (FFI) to run LLM inference directly inside the Lean environment, enabling low-latency, native interaction between Lean and LLMs.
• Supports both local inference (with or without GPU) via CTranslate2 and remote server-based LLM deployment, allowing flexibility in deployment and model choice.
• Introduces three LLM-based tools: `suggest_tactics` for generating next proof steps, `search_proofs` for end-to-end proof search, and `select_premises` for retrieving relevant lemmas from Mathlib.
• Uses fine-tuned LLMs on formal mathematics data, including theorems and proofs from Mathlib and 'Mathematics in Lean', to improve relevance and correctness.
• Employs a modular architecture that allows users to plug in their own models, whether pretrained or fine-tuned, for custom proof automation pipelines.
• Executes LLM calls synchronously within Lean’s type-checking and proof-checking workflow, ensuring correctness and enabling real-time assistance during interactive proving.

Experimental results

Research questions

• RQ1 Can LLMs effectively assist human users in interactive theorem proving by acting as copilots rather than autonomous agents?
• RQ2 How does native LLM inference within Lean compare to external model calls in terms of usability, latency, and integration depth?
• RQ3 To what extent can LLM-based tools like tactic suggestion, proof search, and premise selection reduce the number of human-entered tactics required to prove theorems?
• RQ4 How does LLM-powered proof automation compare to rule-based systems like Aesop in terms of proof autonomy and automation coverage?
• RQ5 Can a general-purpose framework for LLM inference in Lean enable broader adoption and extension of LLM-based proof tools in formal verification and mathematics?

Key findings

• Lean Copilot enables LLMs to reduce the average number of human-entered tactics to 1.02 per theorem when using the `search_proofs` tool, significantly outperforming Aesop (3.62) and `suggest_tactics` (2.72).
• The `search_proofs` tool proves 64% of theorems autonomously—more than double the 34% success rate of `suggest_tactics` and over five times the 12% of Aesop.
• On average, `search_proofs` automates 81.2% of proof steps, demonstrating superior end-to-end proof completion compared to 48.6% for `suggest_tactics` and 35.2% for Aesop.
• The framework successfully enables LLMs to be natively integrated into Lean, allowing real-time, low-latency assistance without external process dependencies.
• The open-sourced codebase and models under MIT license provide a foundation for future research in LLM-powered formal reasoning and proof automation.
• Empirical results confirm that LLMs as copilots can significantly accelerate theorem proving by handling routine proof steps, thereby enhancing human productivity in formal mathematics and verification.