[Paper Review] Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations

Weiming Xiang, Hoang-Dung Tran+1 more|arXiv (Cornell University)|Dec 21, 2017

Adversarial Robustness in Machine LearningComputer Science15 references70 citations

TL;DR

The paper develops a layer-by-layer method to compute exact output reachable sets for ReLU neural networks, expressed as unions of polyhedra, and uses this for safety verification against unsafe output regions.

ABSTRACT

Neural networks have been widely used to solve complex real-world problems. Due to the complicate, nonlinear, non-convex nature of neural networks, formal safety guarantees for the output behaviors of neural networks will be crucial for their applications in safety-critical systems.In this paper, the output reachable set computation and safety verification problems for a class of neural networks consisting of Rectified Linear Unit (ReLU) activation functions are addressed. A layer-by-layer approach is developed to compute output reachable set. The computation is formulated in the form of a set of manipulations for a union of polyhedra, which can be efficiently applied with the aid of polyhedron computation tools. Based on the output reachable set computation results, the safety verification for a ReLU neural network can be performed by checking the intersections of unsafe regions and output reachable set described by a union of polyhedra. A numerical example of a randomly generated ReLU neural network is provided to show the effectiveness of the approach developed in this paper.

Motivation & Objective

• Motivate formal safety guarantees for neural networks in safety-critical systems.
• Develop a layer-by-layer method to compute the output reachable set for networks with ReLU activations.
• Represent the reachable sets as unions of polyhedra to enable precise safety verification.
• Enable safety verification by checking intersections between the reachable set and unsafe regions.

Proposed method

• Model the neural network as a multilayer perceptron with ReLU activations and express layer mappings.
• Show that ReLU output sets can be decomposed into three cases (positive, zero, mixed) leading to a union of polyhedra.
• Derive closed-form constructions for the output sets of each layer as unions of polyhedra.
• Extend the single-layer result to multi-layer networks by recursive composition, preserving the union-of-polyhedra structure.
• Formulate safety verification as checking the emptiness of the intersection between the reachable set and the unsafe region.
• Provide Algorithms for Output Reach Set Computation, ReLU Network Reachability, and VeriReLU Network verification.

Experimental results

Research questions

• RQ1 How can the output reachable set of a ReLU neural network be computed given an input set that is a union of polyhedra?
• RQ2 Can the reachable set be exactly characterized as a union of polyhedra across layers, enabling sound safety verification?
• RQ3 How to perform safety verification by checking intersections between the output reachable set and unsafe regions?
• RQ4 What are the layer-wise constraints and transformations needed to maintain a tractable polyhedral representation through the network?
• RQ5 How does the approach extend from single ReLU units to multi-layer networks?

Key findings

• The output of a ReLU can be exactly characterized as a union of three cases, yielding a union of polyhedra for the reachable set.
• For input sets described as unions of polyhedra, the ReLU output reachable set is also a union of polyhedra across each branch.
• Layer-by-layer propagation preserves the union-of-polyhedra structure, enabling exact reachability for the whole network.
• The safety verification reduces to checking intersections between the output reachable set and unsafe regions, with sound conclusions guaranteed by the exact reachability.
• The framework generalizes to networks with multiple layers and can handle linear (non-ReLU) layers within the same formalism.