[Paper Review] SkipFuzz: Active Learning-based Input Selection for Fuzzing Deep Learning Libraries

Hong Jin Kang, Pattarakrit Rattanukul+5 more|arXiv (Cornell University)|Dec 8, 2022

Adversarial Robustness in Machine Learning4 citations

TL;DR

SkipFuzz is a novel fuzzing approach for deep learning libraries that uses active learning to infer input constraints without relying on documentation or seed inputs. By interactively querying the library via a test executor, it generates semantically valid, non-redundant inputs, significantly improving crash discovery—resulting in 43 reported crashes, 28 confirmed, and 13 unique CVEs assigned.

ABSTRACT

Many modern software systems are enabled by deep learning libraries such as TensorFlow and PyTorch. As deep learning is now prevalent, the security of deep learning libraries is a key concern. Fuzzing deep learning libraries presents two challenges. Firstly, to reach the functionality of the libraries, fuzzers have to use inputs from the valid input domain of each API function, which may be unknown. Secondly, many inputs are redundant. Randomly sampled invalid inputs are likely not to trigger new behaviors. While existing approaches partially address the first challenge, they overlook the second challenge. We propose SkipFuzz, an approach for fuzzing deep learning libraries. To generate valid inputs, SkipFuzz learns the input constraints of each API function using active learning. By using information gained during fuzzing, SkipFuzz infers a model of the input constraints, and, thus, generate valid inputs. SkipFuzz comprises an active learner which queries a test executor to obtain feedback for inference. After constructing hypotheses, the active learner poses queries and refines the hypotheses using the feedback from the test executor, which indicates if the library accepts or rejects an input, i.e., if it satisfies the input constraints or not. Inputs from different categories are used to invoke the library to check if a set of inputs satisfies a function's input constraints. Inputs in one category are distinguished from other categories by possible input constraints they would satisfy, e.g. they are tensors of a certain shape. As such, SkipFuzz is able to refine its hypothesis by eliminating possible candidates of the input constraints. This active learning-based approach addresses the challenge of redundant inputs. Using SkipFuzz, we have found and reported 43 crashes. 28 of them have been confirmed, with 13 unique CVEs assigned.

Motivation & Objective

• To address the challenge of generating semantically valid inputs for fuzzing deep learning libraries when input constraints are unknown.
• To reduce input redundancy in fuzzing by selecting inputs that trigger novel behaviors rather than repeating known outcomes.
• To eliminate reliance on API documentation or manually collected seed inputs for input constraint inference.
• To improve crash discovery in deep learning libraries like TensorFlow and PyTorch through targeted, constraint-aware fuzzing.
• To develop a scalable, dynamic approach that learns input constraints in real time during fuzzing.

Proposed method

• SkipFuzz employs active learning to iteratively infer input constraints of API functions by querying a test executor (acting as an oracle).
• The active learner formulates input queries categorized by potential constraints (e.g., tensor shape or type) to test validity.
• Feedback from the test executor—indicating whether an input is accepted or rejected—refines the hypothesis of the true input constraints.
• Inputs are selected based on their potential to provide new information, minimizing redundancy and maximizing behavioral diversity.
• The system dynamically constructs and executes test cases to validate hypotheses and refine the input constraint model during fuzzing.
• It avoids random mutation or seed-based generation, instead focusing on constraint-informed, information-maximizing inputs.

Experimental results

Research questions

• RQ1 Can active learning be effectively used to infer input constraints for deep learning library APIs without relying on documentation?
• RQ2 Can constraint inference reduce input redundancy during fuzzing and improve the discovery of novel behaviors?
• RQ3 Does constraint-aware input selection lead to a higher rate of crash reproduction compared to random or mutation-based fuzzing?
• RQ4 Can SkipFuzz discover previously unknown vulnerabilities in widely used deep learning libraries?
• RQ5 How does SkipFuzz’s performance compare to existing approaches like DocTer and FreeFuzz in terms of crash discovery and CVE reporting?

Key findings

• SkipFuzz discovered and reported 43 crashes in deep learning libraries, with 28 confirmed by maintainers.
• Of the confirmed vulnerabilities, 13 unique CVEs were assigned, demonstrating the discovery of previously unknown, critical security flaws.
• The approach outperformed prior methods in generating crashing inputs, particularly by reducing redundancy through constraint-aware input selection.
• SkipFuzz successfully inferred input constraints without requiring API documentation or pre-existing seed inputs, proving effective in real-world library fuzzing.
• The active learning mechanism enabled efficient hypothesis refinement, with queries designed to maximize information gain about input constraints.
• The tool demonstrated scalability and practicality, with source code publicly available for replication and extension.