[Paper Review] An approach to reachability analysis for feed-forward ReLU neural networks

Alessio Lomuscio, Lalit Maganti|arXiv (Cornell University)|Jun 22, 2017

Adversarial Robustness in Machine LearningComputer Science10 references247 citations

TL;DR

The paper formalizes reachability analysis for feed-forward ReLU networks by encoding network execution as a linear program with a single binary variable per layer, enabling LP-based verification and bug detection in neural controllers.

ABSTRACT

We study the reachability problem for systems implemented as feed-forward neural networks whose activation function is implemented via ReLU functions. We draw a correspondence between establishing whether some arbitrary output can ever be outputed by a neural system and linear problems characterising a neural system of interest. We present a methodology to solve cases of practical interest by means of a state-of-the-art linear programs solver. We evaluate the technique presented by discussing the experimental results obtained by analysing reachability properties for a number of benchmarks in the literature.

Motivation & Objective

• Motivate formal verification for neural networks in safety-critical AI systems.
• Develop a linear programming encoding that maps FFNN reachability to LP feasibility.
• Show how to handle floating-point issues with epsilon relaxations in the LP encoding.
• Apply the approach to benchmark controllers (inverted pendulum, Mountain Car, Acrobot) and real-world nets (Reuters, MNIST).
• Evaluate scalability and discuss limitations and future extensions (recurrent nets, synthesis).

Proposed method

• Represent FFNN layers with linear constraints and a single binary indicator per neuron to capture ReLU activation.
• Construct a layer-wise linear encoding that yields a global LP whose feasible solutions correspond to reachable input-output pairs.
• Incorporate epsilon relaxations to account for floating-point inaccuracies and minimize their sum in the objective to improve robustness.
• Translate reachability from input set I to output set O into an LP feasibility problem via the encoding C = Cin ∪ C ∪ Cout.
• Use Gurobi to solve the resulting mixed-integer linear programs and interpret feasibility as reachability.
• Benchmark on standard control problems and large datasets to assess scalability.

Experimental results

Research questions

• RQ1 Can reachability from a linear-definable input set to a linear-definable output set be exactly characterized by a solvable LP derived from a ReLU FFNN encoding?
• RQ2 How does floating-point inaccuracy affect the soundness of LP-based reachability analysis and how can epsilon relaxations mitigate this?
• RQ3 What is the practical scalability of the LP-encoded reachability method on networks of varying sizes and problems?
• RQ4 Can the approach identify bugs or safety violations in neural controllers (e.g., inverted pendulum, Acrobot) within reasonable compute times?

Key findings

• The LP encoding yields a feasible solution exactly when there exists an input in I and output in O with f(x)=y for the FFNN, establishing equivalence between reachability and LP feasibility.
• Floating-point tolerance is essential; epsilon relaxations linked with an objective minimizing their sum enable sound analysis within numerical limits.
• Across benchmarks (Inverted Pendulum, Mountain Car, Pendulum, Acrobot, Reuters, MNIST), reachability queries solved in under 1 second for many cases, indicating practical scalability.
• A bug was found in a synthesized neural controller for the inverted pendulum problem by solving an LP; the result can be fed back to the network to confirm the issue.
• Performance varies with state-space size, number of variables (notably binary variables), and constraint count, with larger problems showing longer solve times (e.g., up to tens of seconds for some Acrobot cases).
• The method can handle networks up to several layers and significant sizes, demonstrating applicability to deep nets of practical interest.