[Paper Review] Make Sure You're Unsure: A Framework for Verifying Probabilistic Specifications

Leonard Berrada, Sumanth Dathathri+6 more|arXiv (Cornell University)|Feb 18, 2021

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper proposes a novel framework for verifying probabilistic specifications in neural networks by generalizing Lagrangian duality using functional multipliers—functions of layer activations—enabling exact, sound, and complete verification. It significantly improves certified guarantees for adversarial robustness and out-of-distribution detection in Bayesian and Monte Carlo dropout networks, achieving up to 29% certified AUC for OOD detection and 74.6% ℓ∞ robust accuracy on MNIST.

ABSTRACT

Most real world applications require dealing with stochasticity like sensor noise or predictive uncertainty, where formal specifications of desired behavior are inherently probabilistic. Despite the promise of formal verification in ensuring the reliability of neural networks, progress in the direction of probabilistic specifications has been limited. In this direction, we first introduce a general formulation of probabilistic specifications for neural networks, which captures both probabilistic networks (e.g., Bayesian neural networks, MC-Dropout networks) and uncertain inputs (distributions over inputs arising from sensor noise or other perturbations). We then propose a general technique to verify such specifications by generalizing the notion of Lagrangian duality, replacing standard Lagrangian multipliers with "functional multipliers" that can be arbitrary functions of the activations at a given layer. We show that an optimal choice of functional multipliers leads to exact verification (i.e., sound and complete verification), and for specific forms of multipliers, we develop tractable practical verification algorithms. We empirically validate our algorithms by applying them to Bayesian Neural Networks (BNNs) and MC Dropout Networks, and certifying properties such as adversarial robustness and robust detection of out-of-distribution (OOD) data. On these tasks we are able to provide significantly stronger guarantees when compared to prior work -- for instance, for a VGG-64 MC-Dropout CNN trained on CIFAR-10, we improve the certified AUC (a verified lower bound on the true AUC) for robust OOD detection (on CIFAR-100) from $0\% ightarrow 29\%$. Similarly, for a BNN trained on MNIST, we improve on the robust accuracy from $60.2\% ightarrow 74.6\%$. Further, on a novel specification -- distributionally robust OOD detection -- we improve the certified AUC from $5\% ightarrow 23\%$.

Motivation & Objective

• Address the lack of formal verification methods for probabilistic specifications in neural networks, especially in the presence of stochasticity from weights or inputs.
• Enable verification of properties that depend on output distributions, such as expected behavior under uncertainty or distributionally robust OOD detection.
• Provide a general, theoretically sound framework that supports both exact (sound and complete) and tractable practical verification for verification-agnostic models.
• Improve upon vacuous or weak guarantees from prior methods by exploiting the structure of probabilistic constraints through functional multipliers.

Proposed method

• Generalize standard Lagrangian duality by replacing scalar Lagrange multipliers with functional multipliers that depend on activations at intermediate network layers.
• Derive a dual optimization problem where the functional multipliers are optimized to tighten the upper bound on specification violation, enabling sound and complete verification.
• Develop practical algorithms by restricting functional multipliers to specific forms (e.g., linear or parametric functions), enabling efficient optimization via gradient-based methods.
• Integrate with bound propagation techniques (e.g., IBP or LBP) to handle bilinear operations arising from stochasticity in Bayesian and MC-Dropout networks.
• Use a two-stage optimization: outer loop optimizes functional multipliers via Adam, and inner loop performs adversarial attack or bound propagation to compute worst-case violations.
• Apply the framework to verify properties such as adversarial robustness and OOD detection under input perturbations or distributional shifts.

Experimental results

Research questions

• RQ1 Can we develop a general framework for verifying probabilistic specifications in neural networks that accounts for uncertainty in both weights and inputs?
• RQ2 How can Lagrangian duality be extended to handle constraints between probability distributions using functional multipliers instead of scalar ones?
• RQ3 To what extent can functional multipliers enable exact verification (sound and complete) for probabilistic properties in stochastic neural networks?
• RQ4 Can this framework provide significantly stronger certified guarantees than prior methods for tasks like adversarial robustness and OOD detection?
• RQ5 How does the framework perform on novel specifications such as distributionally robust OOD detection, where input perturbations are drawn from entire sets of distributions?

Key findings

• For a VGG-64 MC-Dropout CNN trained on CIFAR-10 without verification-aware training, the certified AUC for robust OOD detection on CIFAR-100 improved from 0% to 29%.
• For a Bayesian Neural Network on MNIST, the certified ℓ∞ robust accuracy increased from 60.2% to 74.6% using the proposed framework.
• On a new distributionally robust OOD detection specification, the certified AUC improved from 5% to 23%, demonstrating the method’s capability on novel, complex probabilistic properties.
• Even with simple functional multiplier choices, the method outperforms prior approaches that often yield vacuous guarantees, especially in stochastic settings.
• The framework enables strong verification performance on verification-agnostic models, showing that it can be applied post-training without architectural modifications.
• The method scales to larger models, with verification times increasing with model size but still providing meaningful certified bounds—e.g., 8.5k seconds for VGG-64 on CIFAR-100.