[論文レビュー] Learn To Pay Attention

Saumya Jetley, Nicholas A. Lord+2 more|arXiv (Cornell University)|Apr 6, 2018

Adversarial Robustness in Machine Learning参考文献 35被引用数 124

ひとこと要約

An end-to-end trainable attention module for CNNs that uses a global image descriptor as a query to compute multi-scale, location-based attention; improves classification and weakly supervised segmentation, and offers some adversarial robustness gains.

ABSTRACT

We propose an end-to-end-trainable attention module for convolutional neural network (CNN) architectures built for image classification. The module takes as input the 2D feature vector maps which form the intermediate representations of the input image at different stages in the CNN pipeline, and outputs a 2D matrix of scores for each map. Standard CNN architectures are modified through the incorporation of this module, and trained under the constraint that a convex combination of the intermediate 2D feature vectors, as parameterised by the score matrices, must extit{alone} be used for classification. Incentivised to amplify the relevant and suppress the irrelevant or misleading, the scores thus assume the role of attention values. Our experimental observations provide clear evidence to this effect: the learned attention maps neatly highlight the regions of interest while suppressing background clutter. Consequently, the proposed function is able to bootstrap standard CNN architectures for the task of image classification, demonstrating superior generalisation over 6 unseen benchmark datasets. When binarised, our attention maps outperform other CNN-based attention maps, traditional saliency maps, and top object proposals for weakly supervised segmentation as demonstrated on the Object Discovery dataset. We also demonstrate improved robustness against the fast gradient sign method of adversarial attack.

研究の動機と目的

• Motivate and design an integrated attention mechanism that highlights salient image regions to improve CNN classification.
• Enable classification to be performed using a convex combination of local feature vectors guided by learned attention scores.
• Demonstrate that multi-scale attention can be added to existing architectures (e.g., VGG, ResNet) with performance gains on diverse datasets.
• Explore the attention maps’ usefulness for weakly supervised segmentation and adversarial robustness.
• Assess cross-domain generalization to unseen datasets.

提案手法

• Define local feature vectors at intermediate layers and a global feature vector g.
• Compute compatibility scores between local features and g via a learnable compatibility function C.
• Normalize scores with softmax to obtain attention weights and form an attention-weighted global descriptor ga.
• Replace the original global descriptor with ga for final classification, enabling end-to-end training with cross-entropy loss.
• Investigate multiple configurations: single/multiple layers, dot-product vs parametrised compatibility, and concatenation or independent classifiers across layers.
• Apply attention to VGG and ResNet architectures and evaluate on CIFAR-10/100, CUB-200-2011, SVHN, and cross-domain datasets; also assess weakly supervised segmentation and adversarial robustness.

実験結果

リサーチクエスチョン

• RQ1 Does incorporating an end-to-end trainable attention module improve image classification performance on standard and fine-grained datasets?
• RQ2 Can attention weighted representations enhance generalization to domain-shifted data?
• RQ3 Are attention maps effective for weakly supervised segmentation without pixel-level annotations?
• RQ4 How does the proposed attention mechanism affect robustness to adversarial perturbations?
• RQ5 What is the effect of multi-scale attention (across layers) on recognition across object parts and whole objects?

主な発見

• Attention-augmented networks outperform baselines on CIFAR-10/100, CIFAR/CUB/SVHN fine-grained tasks, and cross-domain datasets.
• Multi-layer attention (last 2–3 levels) yields notable gains over non-attention baselines and prior attention methods (e.g., GAP, PAN).
• Binarised attention maps from the proposed method outperform other CNN-based attention maps, traditional saliency maps, and top object proposals for weakly supervised segmentation on Object Discovery.
• Attention-aware models show improved robustness to adversarial perturbations at low to moderate L∞ norms, with the gap narrowing at higher perturbation levels.
• Attention maps focus on object regions and suppress background, with layer-wise specialization (lower layers for surroundings, higher layers for central object).
• Cross-domain results indicate consistent improvements (average margin around 6%) when transferring CIFAR-based models to unseen datasets.