[论文解读] Robust Adversarial Perturbation on Deep Proposal-based Models

Yuezun Li, Daniel Tian+3 more|arXiv (Cornell University)|Sep 16, 2018

Adversarial Robustness in Machine Learning参考文献 22被引用 65

一句话总结

本文提出鲁棒对抗扰动（R-AP），用于在深度基于 proposal 的对象检测和实例分割模型中普遍攻击区域提议网络（RPN），在黑盒设置下通过同时干扰标签预测和形状回归来降低性能。

ABSTRACT

Adversarial noises are useful tools to probe the weakness of deep learning based computer vision algorithms. In this paper, we describe a robust adversarial perturbation (R-AP) method to attack deep proposal-based object detectors and instance segmentation algorithms. Our method focuses on attacking the common component in these algorithms, namely Region Proposal Network (RPN), to universally degrade their performance in a black-box fashion. To do so, we design a loss function that combines a label loss and a novel shape loss, and optimize it with respect to image using a gradient based iterative algorithm. Evaluations are performed on the MS COCO 2014 dataset for the adversarial attacking of 6 state-of-the-art object detectors and 2 instance segmentation algorithms. Experimental results demonstrate the efficacy of the proposed method.

研究动机与目标

• Motivate study of adversarial vulnerabilities in deep proposal-based models used for object detection and instance segmentation.
• Propose a universal attack focusing on Region Proposal Networks (RPN) to degrade downstream predictions without full model access.
• Introduce a novel loss combining label disruption and shape regression disturbance to impair RPN performance.
• Demonstrate the effectiveness of R-AP against multiple detectors and segmenters on MS COCO 2014.
• Highlight potential robustness implications for safety-critical CV applications.

提出的方法

• Define a loss L = Llabel + Lshape to generate adversarial perturbations for an input image, while keeping PSNR above a threshold.
• Llabel disturbs the probability of positive proposals by reducing their confidence (zj log(sj)).
• Lshape disturbs the RPN shape regression by guiding predicted offsets toward large preset targets (τx, τy, τw, τh).
• Iteratively update the image by scaled normalized gradient steps pt to minimize L, clipping to valid pixel range and enforcing PSNR ε.
• Combine perturbations from multiple RPN architectures to enhance black-box robustness (P = α · sum of p_i).
• Experimentally evaluate on MS COCO 2014 across six detectors and two instance segmentation methods to show degradation.

实验结果

研究问题

• RQ1 Can a universal perturbation targeting RPN degrade a wide range of deep proposal-based detectors and segmenters without model-specific access?
• RQ2 Does combining label disruption with shape regression disturbance yield stronger degradation than targeting labels alone?
• RQ3 How does R-AP perform across different RPN backbones and in black-box settings?

主要发现

• R-AP significantly degrades several state-of-the-art detectors when perturbations are tailored to their RPN backbones (e.g., Fcns detectors show large drops in mAP at 0.5 and 0.7).
• Accumulated multi-RPN perturbations P achieve notable degradation even under black-box conditions (e.g., RFCN and other detectors).
• Compared to random Gaussian noise, R-AP produces substantially larger drops in performance as PSNR varies.
• Attack effectiveness is demonstrated for instance segmentation with FCIS and Mask R-CNN, with meaningful mAP reductions at 0.5 and 0.7.
• The study confirms RPN as a universal vulnerability point in deep proposal-based models, impacting both detection and segmentation pipelines.