[Paper Review] Efficient Certification of Spatial Robustness

Anian Ruoss, Maximilian Baader+2 more|arXiv (Cornell University)|Sep 19, 2020

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper presents the first provably robust certification method for neural networks against spatial vector field attacks using novel convex relaxations that incorporate smoothness constraints. By tightening interval bounds via linear programming and integrating with state-of-the-art verifiers, it achieves higher certification rates—up to 76% for DeepPoly and 89% for MILP—on MNIST and CIFAR-10 with smooth deformations, significantly improving robustness guarantees over non-smooth or unstructured attacks.

ABSTRACT

Recent work has exposed the vulnerability of computer vision models to vector field attacks. Due to the widespread usage of such models in safety-critical applications, it is crucial to quantify their robustness against such spatial transformations. However, existing work only provides empirical robustness quantification against vector field deformations via adversarial attacks, which lack provable guarantees. In this work, we propose novel convex relaxations, enabling us, for the first time, to provide a certificate of robustness against vector field transformations. Our relaxations are model-agnostic and can be leveraged by a wide range of neural network verifiers. Experiments on various network architectures and different datasets demonstrate the effectiveness and scalability of our method.

Motivation & Objective

• To address the lack of provable robustness guarantees for neural networks against spatial vector field attacks, which are common in real-world image distortions.
• To develop a model-agnostic method that enables efficient and tight interval bounds for deformed images under vector field transformations.
• To incorporate smoothness constraints into existing neural network verifiers to improve certification performance on realistic, smooth deformations.
• To demonstrate scalability and effectiveness across diverse architectures and datasets, including MNIST and CIFAR-10.
• To provide an open-source implementation within the ERAN framework for reproducibility and practical deployment.

Proposed method

• Proposes a novel convex relaxation framework to compute tight interval bounds on pixel values for images transformed by vector fields with bounded displacement magnitude (δ) and smoothness (γ).
• Introduces smoothness constraints modeled via Lipschitz continuity of the vector field, which are encoded as linear constraints on transformation parameters.
• Uses linear programming to efficiently solve the optimization problem for computing these linear constraints, enabling integration with existing verifiers.
• Employs a parametrization of the attack budget using δ (maximum pixel displacement) and γ (smoothness of the vector field), allowing precise control over deformation types.
• Applies the method to multi-channel images by over-approximating the output shape with tight bounding boxes, estimating approximation error via sampling.
• Integrates the relaxations into state-of-the-art verifiers (e.g., DeepPoly, MILP) to enable end-to-end robustness certification.

Experimental results

Research questions

• RQ1 Can we provide provable robustness certification for neural networks against spatial vector field deformations, which are common in real-world image distortions?
• RQ2 How can smoothness constraints (γ) be effectively incorporated into convex relaxations to tighten bounds and improve certification performance?
• RQ3 To what extent do the proposed relaxations improve certification rates compared to non-smooth or unstructured attacks?
• RQ4 How scalable is the method across different network architectures and datasets, including large models and high-resolution images?
• RQ5 How tight are the interval bounds for multi-channel images, and what is the approximation error relative to sampled deformations?

Key findings

• The proposed method achieves a certification rate of 76% on MNIST using DeepPoly and 89% using MILP when γ = 0.1, significantly improving over γ = ∞ (72% and 87%, respectively).
• For CIFAR-10, certification rates increase from 72% (γ = ∞) to 76% (γ = 0.1) with DeepPoly, and from 87% to 89% with MILP, demonstrating the benefit of smoothness constraints.
• The average running time per image is 394 seconds for DeepPoly and 1168 seconds for MILP when γ = 0.1, showing scalability despite increased computation.
• The method scales efficiently to ImageNet, with interval bound computation taking only 0.02 seconds per image and linear bound computation averaging 236 seconds per image.
• Approximation error analysis shows that the bounding box over-approximation covers 98.76% to 99.10% of sampled deformed images, indicating high tightness.
• The combination of Balunovic et al.'s method with the authors' smoothness constraints yields superior performance, as the former alone is insufficient for smooth vector fields.