[Paper Review] Can Program Synthesis be Used to Learn Merge Conflict Resolutions? An Empirical Analysis

Rangeet Pan, Vu Le+4 more|arXiv (Cornell University)|Mar 2, 2021

Software Engineering Research26 references4 citations

TL;DR

This paper proposes using program synthesis with a domain-specific language (DSL) to learn and automate resolution of recurring merge conflict patterns in C++ code, achieving 93.2% accuracy in resolving 11.4% of conflicts—particularly 1-2 line changes—by learning from 2–3 example resolutions per pattern.

ABSTRACT

Forking structure is widespread in the open-source repositories and that causes a significant number of merge conflicts. In this paper, we study the problem of textual merge conflicts from the perspective of Microsoft Edge, a large, highly collaborative fork off the main Chromium branch with significant merge conflicts. Broadly, this study is divided into two sections. First, we empirically evaluate textual merge conflicts in Microsoft Edge and classify them based on the type of files, location of conflicts in a file, and the size of conflicts. We found that ~28% of the merge conflicts are 1-2 line changes, and many resolutions have frequent patterns. Second, driven by these findings, we explore Program Synthesis (for the first time) to learn patterns and resolve structural merge conflicts. We propose a novel domain-specific language (DSL) that captures many of the repetitive merge conflict resolution patterns and learn resolution strategies as programs in this DSL from example resolutions. We found that the learned strategies can resolve 11.4% of the conflicts (~41% of 1-2 line changes) that arise in the C++ files with 93.2% accuracy.

Motivation & Objective

• To investigate whether program synthesis can be used to learn and automate repetitive merge conflict resolution patterns in large-scale collaborative codebases.
• To identify and classify common types of textual merge conflicts in Microsoft Edge, a large fork of Chromium with high conflict frequency.
• To design a domain-specific language (DSL) that captures structural merge conflict resolution patterns, especially for C++ files.
• To evaluate the feasibility of learning resolution strategies as programs in the DSL using programming-by-example (PBE) from minimal examples.
• To assess the accuracy and generalizability of synthesized strategies on real-world merge conflicts in a production environment.

Proposed method

• Conduct a large-scale empirical study of merge conflicts in Microsoft Edge over a 12-week period, classifying conflicts by file type, size, location, and resolution pattern.
• Design a domain-specific language (DSL) with abstractions for selecting, modifying, and composing code elements (e.g., include directives, macros) from main or forked branches.
• Use programming-by-example (PBE) to learn resolution strategies as programs in the DSL from 2–3 example conflicts per pattern.
• Apply a ranking function based on Occam’s Razor to favor smaller, more general programs and reduce overfitting.
• Implement a synthesis engine that generates candidate programs and validates them against known conflict resolutions.
• Extend the DSL and learning pipeline to support new patterns by adding new operations, such as AST node selection, for complex conflict types.

Experimental results

Research questions

• RQ1 What are the most frequent types and sizes of merge conflicts in large, collaborative C++ projects like Microsoft Edge?
• RQ2 To what extent do developers follow recurring resolution patterns in resolving small, structural merge conflicts?
• RQ3 Can a domain-specific language (DSL) be designed to concisely express common merge conflict resolution strategies in C++?
• RQ4 How accurately can program synthesis learn and generate correct resolution strategies from just 2–3 examples per pattern?
• RQ5 Can the synthesized strategies generalize across similar conflicts and achieve high accuracy in real-world settings?

Key findings

• Approximately 28% of all merge conflicts in Microsoft Edge are 1-2 line changes, with a significant portion involving include directives and macros.
• Among C++ conflicts, 12.34% are related to headers and macros, and six of the nine identified resolution patterns specifically address these structural changes.
• The proposed DSL successfully captures common resolution strategies, enabling program synthesis to generate correct solutions for 11.4% of C++ merge conflicts.
• The synthesized strategies achieve 93.2% accuracy when resolving 1-2 line conflicts, demonstrating strong performance on a representative subset of conflicts.
• The approach is effective for small, repetitive conflicts but may fail on complex cases requiring modification of both main and forked code sections.
• The method is extensible and can be adapted to new patterns by updating the DSL and retraining with additional examples.