[Paper Review] Reusing Precisions for Efficient Regression Verification

Dirk Beyer, Stefan Löwe+3 more|arXiv (Cornell University)|May 29, 2013

Software Testing and Debugging Techniques20 references3 citations

TL;DR

This paper proposes reusing abstraction precisions—intermediate results from abstract interpretation—as reusable verification facts in regression verification to drastically reduce redundant computation. By reusing precisions across revisions, the approach cuts verification time by up to 3.7× on average in industrial device drivers, with minimal overhead and no performance degradation on large code changes.

ABSTRACT

Continuous testing during development is a well-established technique for software-quality assurance. Continuous model checking from revision to revision is not yet established as a standard practice, because the enormous resource consumption makes its application impractical. Model checkers compute a large number of verification facts that are necessary for verifying if a given specification holds. We have identified a category of such intermediate results that are easy to store and efficient to reuse: abstraction precisions. The precision of an abstract domain specifies the level of abstraction that the analysis works on. Precisions are thus a precious result of the verification effort and it is a waste of resources to throw them away after each verification run. In particular, precisions are small and thus easy to store; they are easy to process and have a large impact on resource consumption. We experimentally show the impact of precision reuse on industrial verification problems, namely, 59 device drivers with 1119 revisions from the Linux kernel.

Motivation & Objective

• Address the high computational cost of continuous formal verification in software development.
• Reduce redundant verification effort across software revisions by reusing intermediate results from prior analysis.
• Identify and exploit abstraction precisions as a lightweight, high-impact form of reusable verification knowledge.
• Demonstrate the scalability and robustness of precision reuse on large, real-world industrial codebases like the Linux kernel.
• Provide a publicly available benchmark suite of 4,193 verification tasks for regression verification research.

Proposed method

• Extract abstraction precisions from model checking runs, particularly from CEGAR-based verification tools using interpolation or similar techniques.
• Store and persist these precisions as lightweight, tool-independent metadata for later reuse.
• Apply the stored precisions as initial abstractions in subsequent verification runs of modified code versions.
• Reuse precisions to reduce the number of CEGAR iterations and avoid redundant analysis of spurious counterexamples.
• Integrate precision reuse into both symbolic and explicit model checking approaches to evaluate general applicability.
• Use the Linux Driver Verification (LDV) toolkit to generate and validate verification tasks from real kernel driver revisions.

Experimental results

Research questions

• RQ1 Can abstraction precisions be effectively reused across software revisions to accelerate regression verification?
• RQ2 How does precision reuse impact verification performance on real-world industrial codebases like the Linux kernel?
• RQ3 How sensitive is precision reuse to changes in code size and structure, especially for large commits?
• RQ4 Does precision reuse degrade performance on large code changes or introduce false negatives?
• RQ5 Can precision reuse enable solving previously intractable verification problems within resource limits?

Key findings

• Precision reuse reduced average verification time by a factor of 3.7× for predicate analysis and 1.4× for explicit analysis across 4,193 verification tasks.
• Even with large code changes—averaging 1,579 lines per revision—the speedup decreased only slightly, from 3.7× to 3.2× for predicate analysis.
• The approach maintained performance stability across diverse specifications and abstract domains, showing robustness to code changes.
• In some cases, precision reuse enabled verification of previously unsolvable problems within time and memory limits.
• The method reduced the number of CEGAR iterations and minimized redundant analysis of spurious counterexamples.
• An unknown bug in the Linux kernel was discovered and reported through this experimental pipeline, validating real-world impact.