[Paper Review] Verification for Machine Learning, Autonomy, and Neural Networks Survey

Weiming Xiang, Patrick Musau+6 more|arXiv (Cornell University)|Oct 3, 2018

Adversarial Robustness in Machine LearningComputer Science90 references93 citations

TL;DR

A comprehensive survey of verification methods for learning-enabled components in safety-critical autonomous cyber-physical systems, covering architectures, learning, control, and specification learning approaches.

ABSTRACT

This survey presents an overview of verification techniques for autonomous systems, with a focus on safety-critical autonomous cyber-physical systems (CPS) and subcomponents thereof. Autonomy in CPS is enabling by recent advances in artificial intelligence (AI) and machine learning (ML) through approaches such as deep neural networks (DNNs), embedded in so-called learning enabled components (LECs) that accomplish tasks from classification to control. Recently, the formal methods and formal verification community has developed methods to characterize behaviors in these LECs with eventual goals of formally verifying specifications for LECs, and this article presents a survey of many of these recent approaches.

Motivation & Objective

• Motivate safe integration of AI/ML in safety-critical CPS and outline the role of verification and formal methods.
• Summarize architectures, safety architectures, and runtime monitoring approaches for LECs in autonomous CPS.
• Review verification methods for neural networks, reachability, and learning-based control in autonomy.
• Highlight how statistical ML methods contrast with symbolic formal verification in this domain.

Proposed method

• Survey and synthesis of recent verification approaches for LECs in autonomous CPS.
• Discussion of safe monitoring, runtime verification, and runtime assurance as practical verification layers.
• Overview of architecture-level and control-theoretic verification techniques including reachability and Lyapunov-based methods.
• Examination of learning-based specification inference and learning for safety properties such as STL and Boolean formulas.
• Presentation of learning-based control frameworks and safety assurances during online adaptation and policy learning.

Experimental results

Research questions

• RQ1 What verification methods exist for ensuring safety of learning-enabled components in autonomous CPS?
• RQ2 How can safety architectures, runtime monitoring, and reachability analysis contribute to dependable autonomous systems?
• RQ3 What learning and specification inference approaches can provide formal guarantees for ML-driven components?
• RQ4 How do learning-based control methods ensure stability and safety during online adaptation?

Key findings

• The survey identifies architecture-level safety measures, runtime verification, and runtime assurance as practical avenues when full formal verification is intractable for complex autonomous CPS.
• It covers neural network verification, reachability analysis, and learning-based control as key areas in validating LEC safety.
• It discusses specification inference and STL/Boolean formula learning as tools to extract formal safety properties from ML components.
• The document highlights differences between symbolic/formal methods and data-driven ML approaches in the context of safety guarantees.
• It presents examples from autonomous driving and other CPS to illustrate verification and testing frameworks for LECs.