[Paper Review] Program Repair

Xiang Gao, Yannic Noller+1 more|arXiv (Cornell University)|Nov 23, 2022

Software Testing and Debugging Techniques4 citations

TL;DR

This paper presents a comprehensive survey of automated program repair techniques, categorizing them into search-based, constraint-based, and learning-based approaches. It identifies patch overfitting as a key challenge and explores methods to mitigate it, while also examining tools, industrial applications, and future opportunities in leveraging large language models for program repair.

ABSTRACT

Automated program repair is an emerging technology which consists of a suite of techniques to automatically fix bugs or vulnerabilities in programs. In this paper, we present a comprehensive survey of the state of the art in program repair. We first study the different suite of techniques used including search based repair, constraint based repair and learning based repair. We then discuss one of the main challenges in program repair namely patch overfitting, by distilling a class of techniques which can alleviate patch overfitting. We then discuss classes of program repair tools, applications of program repair as well as uses of program repair in industry. We conclude the survey with a forward looking outlook on future usages of program repair, as well as research opportunities arising from work on code from large language models.

Motivation & Objective

• To provide a comprehensive overview of state-of-the-art techniques in automated program repair.
• To analyze the core challenge of patch overfitting—where fixes pass given tests but fail on unseen ones—and identify mitigation strategies.
• To examine the evolution and application of program repair tools in real-world software development and industry.
• To explore the emerging role of large language models in program repair and their potential to enhance or complement existing repair techniques.
• To outline future research directions, particularly in integrating program repair with code generation from LLMs.

Proposed method

• Categorizes program repair into three main paradigms: search-based (e.g., GenProg), constraint-based (e.g., symbolic execution with repair constraints), and learning-based (e.g., GetaFix, neural models trained on human patches).
• Analyzes semantic repair techniques that represent the search space implicitly via repair constraints, enabling program synthesis for minimal, correct patches.
• Reviews machine learning techniques for ranking candidate patches, improving efficiency and quality by filtering non-viable fixes.
• Examines the integration of language model-based code generation (e.g., GitHub Copilot, Codex) into repair pipelines, especially in edit mode for targeted code modification.
• Proposes a hybrid approach where LLM-generated code candidates are analyzed and curated via semantic analysis to extract and compose correct patch components.
• Suggests using equivalence class detection among code snippets from multiple LLM candidates to synthesize robust, semantically correct patches.

Experimental results

Research questions

• RQ1 How do different program repair techniques—search-based, constraint-based, and learning-based—compare in effectiveness and scalability?
• RQ2 What causes patch overfitting in program repair, and which techniques can effectively mitigate this issue?
• RQ3 How can machine learning be used to improve the ranking and selection of candidate patches in automated repair?
• RQ4 What role can large language models play in enhancing or transforming existing program repair workflows?
• RQ5 How might program repair techniques evolve in the context of increasing code generation from LLMs?

Key findings

• Search-based repair techniques like GenProg can scale to large programs but struggle with large explicit search spaces.
• Constraint-based repair using symbolic execution and program synthesis enables minimal, correct patches by implicitly representing the search space.
• Learning-based repair methods such as GetaFix achieve high recall in fixing common bugs by learning from mined human patches.
• Patch overfitting remains a significant challenge, as test suites are incomplete specifications, leading to fixes that pass test cases but fail on unseen inputs.
• Recent language model-based tools like GitHub Copilot with edit mode can outperform traditional pattern- and learning-based repair tools on automatically generated code.
• Future program repair may integrate LLM-generated code candidates with semantic analysis to extract and compose correct patches, suggesting a synergistic role for LLMs and analysis-based repair.