[Paper Review] Neuro-Symbolic Verification of Deep Neural Networks

Xuan Xie, Kristian Kersting+1 more|arXiv (Cornell University)|Mar 2, 2022

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper introduces neuro-symbolic verification, a novel framework that extends formal verification of deep neural networks by allowing complex, real-world properties—such as 'stop when a stop sign is detected'—to be expressed using neural networks as semantic components within logical specifications. Built on existing verification infrastructure, the approach enables verification of diverse properties and generates meaningful, data-distribution-aware counterexamples, significantly improving debuggability over traditional methods.

ABSTRACT

Formal verification has emerged as a powerful approach to ensure the safety and reliability of deep neural networks. However, current verification tools are limited to only a handful of properties that can be expressed as first-order constraints over the inputs and output of a network. While adversarial robustness and fairness fall under this category, many real-world properties (e.g., "an autonomous vehicle has to stop in front of a stop sign") remain outside the scope of existing verification technology. To mitigate this severe practical restriction, we introduce a novel framework for verifying neural networks, named neuro-symbolic verification. The key idea is to use neural networks as part of the otherwise logical specification, enabling the verification of a wide variety of complex, real-world properties, including the one above. Moreover, we demonstrate how neuro-symbolic verification can be implemented on top of existing verification infrastructure for neural networks, making our framework easily accessible to researchers and practitioners alike.

Motivation & Objective

• To address the limitation of current formal verification methods that can only verify simple, first-order constraints on inputs and outputs of deep neural networks.
• To enable verification of complex, real-world properties—such as autonomous vehicle behavior in response to traffic signs—that are semantically rich but not expressible via standard constraints.
• To integrate deep neural networks as semantic components within logical specifications, allowing verification of properties involving perception, recognition, and reasoning.
• To demonstrate that neuro-symbolic verification can be implemented on top of existing verification frameworks, ensuring practical accessibility.
• To produce more informative and meaningful counterexamples by constraining inputs to the data distribution, improving debugging and system development.

Proposed method

• Proposes a new specification language, Neuro-Symbolic Assertion Language (NeSAL), which combines logical assertions with deep neural networks as 'specification networks' to express complex properties.
• Uses specification networks (e.g., trained on GTSRB or MNIST) to represent high-level semantic concepts such as 'stop sign detected' or 'high confidence prediction'.
• Integrates these neuro-symbolic specifications into formal verification by encoding them as constraints over the outputs of the specification networks and the main network under verification.
• Builds a prototype verifier on top of the Marabou framework, reusing existing infrastructure for neural network verification.
• Employs an autoencoder to constrain counterexample search to the data distribution, ensuring generated counterexamples are realistic and meaningful.
• Supports verification of properties involving multiple networks, including conditional logic like 'if detection network outputs stop sign, then control network must brake'.

Experimental results

Research questions

• RQ1 Can formal verification of deep neural networks be extended to complex, real-world properties that are not expressible as simple first-order constraints on inputs and outputs?
• RQ2 Can deep neural networks be used as semantic components within logical specifications to verify high-level system behaviors?
• RQ3 Can neuro-symbolic verification produce more meaningful and actionable counterexamples than traditional verification methods?
• RQ4 Is it feasible to implement neuro-symbolic verification on top of existing verification infrastructure without major reimplementation?
• RQ5 How does neuro-symbolic verification scale on real-world benchmarks such as MNIST and GTSRB, and what is its effectiveness in detecting violations?

Key findings

• The neuro-symbolic verification (NSV) prototype successfully verified all 14 benchmarks on the GTSRB dataset, proving satisfaction of property $P_1$ and finding counterexamples for $P_2$ and $P_3$.
• On the MNIST dataset, NSV timed out on only one benchmark ($P_1$) and found counterexamples in all other cases, indicating strong coverage despite no training-time property enforcement.
• Counterexamples generated by NSV were meaningful and semantically plausible, originating from the data distribution via autoencoder constraints, unlike the random noise often produced by classical verification.
• The counterexamples from NSV were visually intuitive and close to real data, as shown in Figure 2, demonstrating superior debuggability compared to standard verification tools.
• The framework is scalable and practical, as NSV was built on top of the Marabou verification framework, enabling reuse of existing tooling and infrastructure.
• The results confirm that neuro-symbolic verification can effectively verify complex, perception-based properties in safety-critical systems like autonomous vehicles.