[Paper Review] Adversarial Robustness Guarantees for Gaussian Processes

Andrea Patanè, Arno Blaas+4 more|arXiv (Cornell University)|Apr 7, 2021

Gaussian Processes and Bayesian Inference44 references4 citations

TL;DR

This paper introduces a branch-and-bound framework to compute provable adversarial robustness guarantees for Gaussian processes by bounding the prediction range over compact input sets. It ensures $\epsilon$-exact convergence for both regression and classification tasks, with analytical bounds for common kernels, demonstrating that robustness increases with accurate posterior estimation.

ABSTRACT

Gaussian processes (GPs) enable principled computation of model uncertainty, making them attractive for safety-critical applications. Such scenarios demand that GP decisions are not only accurate, but also robust to perturbations. In this paper we present a framework to analyse adversarial robustness of GPs, defined as invariance of the model's decision to bounded perturbations. Given a compact subset of the input space $T\subseteq \mathbb{R}^d$, a point $x^*$ and a GP, we provide provable guarantees of adversarial robustness of the GP by computing lower and upper bounds on its prediction range in $T$. We develop a branch-and-bound scheme to refine the bounds and show, for any $ε> 0$, that our algorithm is guaranteed to converge to values $ε$-close to the actual values in finitely many iterations. The algorithm is anytime and can handle both regression and classification tasks, with analytical formulation for most kernels used in practice. We evaluate our methods on a collection of synthetic and standard benchmark datasets, including SPAM, MNIST and FashionMNIST. We study the effect of approximate inference techniques on robustness and demonstrate how our method can be used for interpretability. Our empirical results suggest that the adversarial robustness of GPs increases with accurate posterior estimation.

Motivation & Objective

• To provide provable adversarial robustness guarantees for Gaussian processes under bounded input perturbations.
• To address the lack of exact robustness analysis in GP models, especially compared to neural networks.
• To develop an anytime, $\epsilon$-convergent algorithm for computing prediction range bounds over compact input sets.
• To evaluate the impact of approximate inference and hyperparameter tuning on GP robustness.
• To enable interpretability by quantifying robustness profiles across diverse datasets.

Proposed method

• The framework computes lower and upper bounds on the GP prediction range over a compact input set $T \subseteq \mathbb{R}^d$ enclosing a test point $x^*$.
• It employs a branch-and-bound scheme that iteratively refines bounds until convergence within $\epsilon$ of the true extrema.
• For each kernel, the method derives analytical bounding functions using kernel decomposition into $\varphi(x,x')$ and $\psi(\varphi)$ components.
• It applies addition and multiplication rules for kernel decompositions to handle complex kernels, including spectral and non-stationary forms.
• The bounding functions are computed via supremum optimization over $T$, leveraging linear and trigonometric properties.
• The algorithm is anytime—providing valid over-approximations at every iteration—and guarantees $\epsilon$-exactness in finite time.

Experimental results

Research questions

• RQ1 Can we provide provable adversarial robustness guarantees for Gaussian processes under bounded input perturbations?
• RQ2 How does the accuracy of posterior estimation affect the adversarial robustness of GP models?
• RQ3 To what extent do approximate inference techniques like Expectation Propagation degrade robustness guarantees?
• RQ4 Can the proposed branch-and-bound framework be applied to both regression and classification tasks with analytical kernel support?
• RQ5 How do robustness profiles differ across datasets like SPAM, MNIST, and Fashion-MNIST under varying inference methods?

Key findings

• The proposed method guarantees $\epsilon$-exact convergence to the true prediction range extrema in finite time for any $\epsilon > 0$.
• Robustness of GP models increases with more accurate posterior estimation, as shown across SPAM, MNIST, and Fashion-MNIST datasets.
• Adversarial attack-based robustness estimates fail to detect robust regions where the proposed method confirms invariance under perturbations.
• The framework supports analytical bounds for common kernels, including squared-exponential and generalised spectral kernels.
• Expectation Propagation leads to lower robustness estimates compared to exact inference, highlighting the sensitivity of robustness to inference quality.
• The method enables interpretability by identifying input regions where predictions are provably invariant to bounded perturbations.