[Paper Review] Fuzzing: Art, Science, and Engineering.

Valentin J. M. Manès, Hyung-Seok Han+5 more|arXiv (Cornell University)|Dec 1, 2018

Information and Cyber Security26 citations

TL;DR

This paper presents a unified model and comprehensive taxonomy of fuzzing, integrating insights from the art, science, and engineering of modern fuzzing. It systemically analyzes design decisions across fuzzing stages, offering a coherent framework to consolidate and guide ongoing research and development in vulnerability discovery.

ABSTRACT

Among the many software vulnerability discovery techniques available today, fuzzing has remained highly popular due to its conceptual simplicity, its low barrier to deployment, and its vast amount of empirical evidence in discovering real-world software vulnerabilities. At a high level, fuzzing refers to a process of repeatedly running a program with generated inputs that may be syntactically or semantically malformed. While researchers and practitioners alike have invested a large and diverse effort towards improving fuzzing in recent years, this surge of work has also made it difficult to gain a comprehensive and coherent view of fuzzing. To help preserve and bring coherence to the vast literature of fuzzing, this paper presents a unified, general-purpose model of fuzzing together with a taxonomy of the current fuzzing literature. We methodically explore the design decisions at every stage of our model fuzzer by surveying the related literature and innovations in the art, science, and engineering that make modern-day fuzzers effective.

Motivation & Objective

• To address the growing complexity and fragmentation in the fuzzing literature by providing a coherent, overarching framework.
• To identify and systematize key design decisions across all stages of the fuzzing pipeline.
• To unify diverse innovations in fuzzing into a single, general-purpose model for improved understanding and development.
• To serve as a foundational reference for researchers and practitioners in software security and fuzzing tool development.
• To clarify the interplay between heuristic techniques (art), empirical validation (science), and implementation engineering (engineering) in effective fuzzing.

Proposed method

• Proposes a general-purpose fuzzer model that decomposes fuzzing into distinct, analyzable stages: input generation, execution, feedback collection, and mutation.
• Introduces a taxonomy that categorizes existing fuzzing techniques based on their design choices at each stage of the model.
• Analyzes the literature across three dimensions: the art (heuristic strategies), science (empirical evaluation), and engineering (implementation efficiency).
• Maps existing fuzzing techniques—such as genetic algorithms, coverage-guided mutation, and symbolic execution—into the proposed model and taxonomy.
• Uses systematic literature review to identify recurring patterns and innovations in input generation, feedback mechanisms, and mutation strategies.
• Demonstrates how modern fuzzers combine techniques from all three dimensions to achieve high vulnerability discovery rates.

Experimental results

Research questions

• RQ1 How can a unified model be constructed to represent the diverse design choices in modern fuzzing techniques?
• RQ2 What are the key design decisions at each stage of the fuzzing pipeline, and how do they influence effectiveness?
• RQ3 How do the art, science, and engineering dimensions of fuzzing interrelate and contribute to fuzzer effectiveness?
• RQ4 What systematic taxonomy can be derived from the current body of fuzzing research to improve coherence and navigation?
• RQ5 How can the integration of heuristic, empirical, and implementation techniques be formalized to guide future fuzzer development?

Key findings

• A unified fuzzer model was successfully constructed, enabling systematic classification of existing fuzzing techniques across input generation, execution, feedback, and mutation stages.
• The taxonomy reveals recurring patterns in design choices, particularly in feedback-directed mutation and coverage-guided strategies, which dominate modern fuzzing.
• The integration of heuristic techniques (art), empirical validation (science), and efficient implementation (engineering) is essential for high-performance fuzzing.
• The model and taxonomy provide a shared reference framework that enhances understanding and accelerates innovation in vulnerability discovery.
• The analysis shows that progress in fuzzing is not just algorithmic but also deeply rooted in engineering trade-offs and empirical validation practices.
• The paper establishes a foundation for future research by identifying gaps in current literature and clarifying the role of each dimension—art, science, and engineering—in fuzzer design.