[Paper Review] Comprehensive Attention Self-Distillation for Weakly-Supervised Object Detection

Zeyi Huang, Yang Zou+2 more|arXiv (Cornell University)|Oct 22, 2020

Advanced Neural Network ApplicationsComputer Science46 references66 citations

TL;DR

CASD enhances weakly supervised object detection by aggregating comprehensive attention from multiple image transformations and feature layers, then self-distilling this attention within the WSOD model to achieve balanced and spatially consistent detections, achieving state-of-the-art results on VOC and COCO.

ABSTRACT

Weakly Supervised Object Detection (WSOD) has emerged as an effective tool to train object detectors using only the image-level category labels. However, without object-level labels, WSOD detectors are prone to detect bounding boxes on salient objects, clustered objects and discriminative object parts. Moreover, the image-level category labels do not enforce consistent object detection across different transformations of the same images. To address the above issues, we propose a Comprehensive Attention Self-Distillation (CASD) training approach for WSOD. To balance feature learning among all object instances, CASD computes the comprehensive attention aggregated from multiple transformations and feature layers of the same images. To enforce consistent spatial supervision on objects, CASD conducts self-distillation on the WSOD networks, such that the comprehensive attention is approximated simultaneously by multiple transformations and feature layers of the same images. CASD produces new state-of-the-art WSOD results on standard benchmarks such as PASCAL VOC 2007/2012 and MS-COCO.

Motivation & Objective

• Motivate WSOD by addressing bias toward salient objects, clustered instances, and discriminative parts.
• Develop a mechanism to balance feature learning across object instances through comprehensive attention.
• Enforce spatial and instance-level consistency by self-distilling comprehensive attention within the WSOD network.
• Leverage image transformations and multi-layer features as free supervisory signals.
• Demonstrate state-of-the-art performance on standard WSOD benchmarks.

Proposed method

• Build on OICR WSOD framework as the base detector and MIL head.
• Compute proposal attention maps from proposal feature maps and aggregate them into comprehensive attention across transformations and layers.
• Introduce Input-wise CASD (IW-CASD) by aligning comprehensive attention across original, flipped, and scaled inputs; use a max fusion to form A_r^{IW} and minimize L_IW to align individual attentions with A_r^{IW} across transformed views.
• Introduce Layer-wise CASD (LW-CASD) by aggregating attention maps from multiple CNN layers into A_r^{LW} and minimize L_LW to align per-layer attentions with A_r^{LW} across layers.
• Perform self-distillation by reusing the comprehensive attention maps as soft supervision to regularize the WSOD feature extractor without external teachers.
• Combine MIL loss, refinement losses, regression loss, and CASD losses into a unified end-to-end objective for training.

Experimental results

Research questions

• RQ1 Can comprehensive attention aggregated over multiple transformations and feature layers improve WSOD localization beyond standard attention maps?
• RQ2 Does self-distillation of comprehensive attention within the WSOD model improve consistency and balance across object instances and transformations?
• RQ3 What is the contribution of input-wise versus layer-wise CASD, and their combination, to WSOD performance?
• RQ4 How does CASD compare to other attention regularization strategies in WSOD?

Key findings

• CASD yields significant mAP improvements on VOC 2007 (baseline 48.9% to 56.8% with full CASD) and VOC 2012, and shows strong MS-COCO gains with backbone variants.
• Input-wise CASD (IW) provides large gains by aggregating attention across image transformations, e.g., IW improves over baseline by ~5.2 percentage points on VOC 2007.
• Layer-wise CASD (LW) provides additional gains by aggregating attention across CNN layers, with LW+CASD achieving notable improvements over baseline.
• Combining IW and LW with regression and stronger augmentations achieves the best VOC 2007 score (56.8% mAP 0.5) and strong results on MS-COCO and VOC 2012.
• CASD outperforms prediction consistency and attention consistency baselines in WSOD ablations, indicating superior attention regularization.
• Ablation studies show middle-level layers (B2-B4) contribute most to balanced attention for WSOD.