[Paper Review] PeerNets: Exploiting Peer Wisdom Against Adversarial Attacks

Jan Svoboda, Jonathan Masci+3 more|arXiv (Cornell University)|May 31, 2018

Adversarial Robustness in Machine Learning45 references19 citations

TL;DR

PeerNets introduce a novel deep learning architecture that alternates Euclidean convolutions with graph convolutions to leverage peer sample relationships, enabling non-local feature propagation across a data-induced graph. This approach achieves up to 3× higher robustness against white- and black-box adversarial attacks with minimal accuracy drop, significantly outperforming standard models in fooling rate under universal and targeted perturbations.

ABSTRACT

Deep learning systems have become ubiquitous in many aspects of our lives. Unfortunately, it has been shown that such systems are vulnerable to adversarial attacks, making them prone to potential unlawful uses. Designing deep neural networks that are robust to adversarial attacks is a fundamental step in making such systems safer and deployable in a broader variety of applications (e.g. autonomous driving), but more importantly is a necessary step to design novel and more advanced architectures built on new computational paradigms rather than marginally building on the existing ones. In this paper we introduce PeerNets, a novel family of convolutional networks alternating classical Euclidean convolutions with graph convolutions to harness information from a graph of peer samples. This results in a form of non-local forward propagation in the model, where latent features are conditioned on the global structure induced by the graph, that is up to 3 times more robust to a variety of white- and black-box adversarial attacks compared to conventional architectures with almost no drop in accuracy.

Motivation & Objective

• To address the critical vulnerability of deep neural networks to adversarial attacks, especially in safety-critical applications like autonomous driving.
• To improve robustness against both white-box and black-box adversarial attacks without compromising standard accuracy.
• To explore whether non-local feature propagation via a data graph can serve as an effective regularizer to enhance model robustness.
• To develop a plug-and-play architecture that can be easily integrated into existing CNNs with minimal architectural changes.
• To investigate the impact of peer sample interactions on the structure and perceptibility of adversarial perturbations.

Proposed method

• PeerNets alternate standard convolutional layers with peer regularization (PR) layers that perform graph convolutional operations on a k-nearest neighbor (KNN) graph of training samples.
• The KNN graph is constructed based on feature embeddings from the previous layer, enabling non-local feature aggregation across semantically similar samples.
• Each PR layer performs message passing over the graph, where features are updated by aggregating information from neighboring peers using learnable weights.
• Monte Carlo sampling is used to stabilize the graph-based aggregation, with multiple runs improving robustness and reducing variance.
• The model is trained end-to-end using standard cross-entropy loss, with the graph structure dynamically updated during training based on feature representations.
• The method is applied as a plug-in module to ResNet architectures, replacing or augmenting standard residual blocks with peer-regularized layers.

Experimental results

Research questions

• RQ1 Can non-local feature propagation via a peer sample graph significantly improve robustness against adversarial attacks?
• RQ2 How does peer regularization affect the fooling rate of deep networks under universal and targeted adversarial perturbations?
• RQ3 To what extent does the graph-based mechanism alter the structure and perceptibility of adversarial noise?
• RQ4 Can peer regularization act as an effective regularizer to allow higher model capacity without overfitting?
• RQ5 How does the choice of graph size and number of Monte Carlo runs affect robustness and accuracy trade-offs?

Key findings

• PeerNets achieve up to 3× higher robustness against adversarial attacks compared to standard ResNets, with a 28.8% fooling rate at ρ=0.10 on CIFAR-10, compared to 77.34% for ResNet-32.
• On CIFAR-100, PR-ResNet-110 with 500 graph neighbors and 5 Monte Carlo runs reduces the fooling rate to 49.54% at ρ=0.06, compared to 86.56% for standard ResNet-110.
• The PR-ResNet-32 v2 variant with doubled feature maps achieves 90.72% original accuracy and only 11.05% fooling rate at ρ=0.04, outperforming ResNet-32 v2 in robustness with minimal accuracy drop.
• Adversarial perturbations for PeerNets exhibit more localized, structured noise, particularly in background regions, making them more perceptible to humans than random noise in standard models.
• The graph-based mechanism acts as a strong regularizer, enabling higher capacity models (e.g., v2 variants) to match state-of-the-art accuracy without overfitting.
• PeerNets maintain high performance across diverse attack types, including universal, targeted, and non-targeted attacks, demonstrating broad-spectrum robustness.