[Paper Review] Algorithms for Verifying Deep Neural Networks

Changliu Liu, Tomer Arnon+4 more|arXiv (Cornell University)|Mar 15, 2019

Adversarial Robustness in Machine LearningComputer Science68 references35 citations

TL;DR

This paper surveys sound verification methods for deep neural networks, linking reachability, optimization, and search, and provides pedagogical implementations and benchmark comparisons.

ABSTRACT

Deep neural networks are widely used for nonlinear function approximation with applications ranging from computer vision to control. Although these networks involve the composition of simple arithmetic operations, it can be very challenging to verify whether a particular network satisfies certain input-output properties. This article surveys methods that have emerged recently for soundly verifying such properties. These methods borrow insights from reachability analysis, optimization, and search. We discuss fundamental differences and connections between existing algorithms. In addition, we provide pedagogical implementations of existing methods and compare them on a set of benchmark problems.

Motivation & Objective

• Motivate the need to verify input-output properties of deep neural networks in diverse applications.
• Survey existing verification methods and categorize them by core ideas and techniques.
• Highlight connections and differences among verification approaches and their theoretical foundations.
• Provide pedagogical implementations and benchmark comparisons to aid understanding and practice.

Proposed method

• Survey existing verification methods for deep neural networks.
• Discuss fundamental differences and connections between approaches (reachability, optimization, search).
• Present pedagogical implementations of existing methods.
• Compare methods on a set of benchmark problems.

Experimental results

Research questions

• RQ1 What are the fundamental differences and connections between verification methods for deep neural networks (reachability, optimization, search)?
• RQ2 How can verification methods be implemented pedagogically, and how do they perform on benchmark problems?
• RQ3 What are the practical trade-offs and limitations of current verification techniques across different network types and properties?

Key findings

• Verification methods for deep neural networks exist across reachability analysis, optimization, and search.
• There are fundamental differences and connections among the surveyed algorithms.
• Pedagogical implementations are provided to aid understanding and experimentation.
• The paper includes benchmark comparisons to evaluate method performance.