[Paper Review] Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation

Chaowei Xiao, Ruizhi Deng+4 more|arXiv (Cornell University)|Oct 11, 2018

Adversarial Robustness in Machine LearningComputer Science46 references51 citations

TL;DR

This paper analyzes adversarial examples in semantic segmentation by leveraging spatial consistency across random image patches, showing that spatial context can robustly detect adversarial inputs even against adaptive attackers, and that transferability across segmentation models is low.

ABSTRACT

Deep Neural Networks (DNNs) have been widely applied in various recognition tasks. However, recently DNNs have been shown to be vulnerable against adversarial examples, which can mislead DNNs to make arbitrary incorrect predictions. While adversarial examples are well studied in classification tasks, other learning problems may have different properties. For instance, semantic segmentation requires additional components such as dilated convolutions and multiscale processing. In this paper, we aim to characterize adversarial examples based on spatial context information in semantic segmentation. We observe that spatial consistency information can be potentially leveraged to detect adversarial examples robustly even when a strong adaptive attacker has access to the model and detection strategies. We also show that adversarial examples based on attacks considered within the paper barely transfer among models, even though transferability is common in classification. Our observations shed new light on developing adversarial attacks and defenses to better understand the vulnerabilities of DNNs.

Motivation & Objective

• Understand how spatial context information affects adversarial examples in semantic segmentation.
• Propose a spatial consistency analysis to distinguish benign vs. adversarial segmentation outputs.
• Evaluate robustness of spatial consistency detection under adaptive and black-box scenarios.

Proposed method

• Propose a spatial consistency analysis that samples overlapping image patches and compares predictions in overlapping regions (measured by mean IoU).
• Quantify per-pixel self-entropy over multiple spatial contexts to visualize prediction stability.
• Compare spatial consistency with scale-based detection by applying image Gaussian blur/scale transforms as baselines.
• Evaluate detection performance on state-of-the-art segmentation models (DRN and DLA) trained on Cityscapes and BDD100K.
• Assess robustness of the detection under adaptive attacks where the attacker has knowledge of the defense.

Experimental results

Research questions

• RQ1 Can spatial context information in semantic segmentation reveal differences between benign and adversarial inputs?
• RQ2 Is a spatial consistency-based detector robust to adaptive attackers who know the defense strategy?
• RQ3 How does transferability of adversarial examples manifest across segmentation models compared with classification?

Key findings

• Spatial consistency differs for benign and adversarial images, with overlapping patch predictions showing low mIOU for adversarial examples and higher mIOU for benign ones.
• A spatial consistency detector achieves near-perfect detection of adversarial examples under tested attacks and models, even when attackers know the defense (adaptive attacks).
• Scale-based detection can also separate adversarial from benign inputs but is more vulnerable to adaptive attacks than the spatial consistency method.
• Adversarial examples generated against one segmentation model transfer poorly to others, indicating lower transferability compared with classification tasks.
• Randomized patch selection (larger K) maintains high detection performance and increases adversary search complexity, hindering adaptive attacks.