[Paper Review] Toward Scalable Verification for Safety-Critical Deep Networks

Lindsey Kuper, Guy Katz+4 more|arXiv (Cornell University)|Jan 18, 2018

Adversarial Robustness in Machine Learning14 references20 citations

TL;DR

This paper proposes a dual approach to scalable verification of safety-critical deep neural networks (DNNs): developing specialized SMT solvers like Reluplex that use lazy ReLU splitting to verify larger networks, and designing DNNs with verification-friendly architectures—such as smaller, deeper structures and piecewise-linear activation functions—making them more amenable to formal verification. The key contribution is demonstrating verification of a 100+ neuron DNN used in aircraft collision avoidance.

ABSTRACT

The increasing use of deep neural networks for safety-critical applications, such as autonomous driving and flight control, raises concerns about their safety and reliability. Formal verification can address these concerns by guaranteeing that a deep learning system operates as intended, but the state of the art is limited to small systems. In this work-in-progress report we give an overview of our work on mitigating this difficulty, by pursuing two complementary directions: devising scalable verification techniques, and identifying design choices that result in deep learning systems that are more amenable to verification.

Motivation & Objective

• Address the challenge of verifying large, safety-critical deep neural networks (DNNs) used in autonomous vehicles and aircraft.
• Overcome the scalability limitations of existing SMT-based verification tools, which previously only worked on toy-sized networks with fewer than ten neurons.
• Develop verification-friendly design principles for DNNs that improve compatibility with formal verification without sacrificing performance.
• Enable practical formal verification of real-world DNNs by combining advanced solver techniques with architectural choices.

Proposed method

• Introduce Reluplex, a specialized SMT solver that extends linear arithmetic with a high-level ReLU predicate to enable lazy splitting of ReLU constraints, avoiding exponential state space explosion.
• Use a hybrid architecture combining a SAT solver with a domain-specific linear programming (LP) solver for ReLU constraints, allowing efficient exploration of the solution space.
• Apply the lazy ReLU splitting technique to avoid explicitly enumerating all possible activation states, significantly reducing the search space during verification.
• Design DNNs with deeper, narrower architectures (many small layers) rather than shallow, wide ones, as these are easier to verify due to reduced combinatorial complexity.
• Prioritize piecewise-linear activation functions like ReLU and max-pooling, which are more amenable to SMT encoding than smooth, nonlinear functions like sigmoid.
• Explore low-precision arithmetic and network pruning as additional design choices that reduce model size and improve verification feasibility.

Experimental results

Research questions

• RQ1 Can SMT-based verification be scaled to DNNs with thousands of neurons, beyond the current limit of ~10 neurons?
• RQ2 How can the computational complexity introduced by ReLU activation functions be mitigated during formal verification?
• RQ3 Which architectural choices in DNN design lead to networks that are more amenable to formal verification?
• RQ4 To what extent do activation function type, network depth, width, and precision affect the scalability of verification tools?
• RQ5 Can hardware-aware design choices (e.g., low-precision arithmetic, pruning) improve both inference efficiency and formal verifiability?

Key findings

• Reluplex successfully verified a DNN with over 100 neurons in the ACAS Xu aircraft collision avoidance system, representing an order of magnitude increase in scalability over prior methods.
• The lazy ReLU splitting technique in Reluplex avoids exploring the full exponential state space of ReLU activation combinations, enabling verification of larger networks.
• Networks with many shallow layers and few neurons per layer were found to be significantly easier to verify than deep, wide architectures with fewer layers.
• DNNs using piecewise-linear activation functions like ReLU and max-pooling are far more amenable to SMT-based verification than those using smooth, nonlinear functions like sigmoid.
• Low-precision and pruned DNNs are not only more efficient for deployment but also more amenable to formal verification due to reduced size and complexity.
• Design choices that improve verification scalability—such as network topology and activation function selection—also yield benefits in inference speed, memory usage, and power efficiency.