[论文解读] Robust Adversarial Perturbation on Deep Proposal-based Models

Yuezun Li, Daniel Tian+3 more|arXiv (Cornell University)|Sep 16, 2018

Adversarial Robustness in Machine Learning参考文献 22被引用 65

一句话总结

本文提出鲁棒对抗扰动（R-AP），用于在深度基于 proposal 的对象检测和实例分割模型中普遍攻击区域提议网络（RPN），在黑盒设置下通过同时干扰标签预测和形状回归来降低性能。

ABSTRACT

Adversarial noises are useful tools to probe the weakness of deep learning based computer vision algorithms. In this paper, we describe a robust adversarial perturbation (R-AP) method to attack deep proposal-based object detectors and instance segmentation algorithms. Our method focuses on attacking the common component in these algorithms, namely Region Proposal Network (RPN), to universally degrade their performance in a black-box fashion. To do so, we design a loss function that combines a label loss and a novel shape loss, and optimize it with respect to image using a gradient based iterative algorithm. Evaluations are performed on the MS COCO 2014 dataset for the adversarial attacking of 6 state-of-the-art object detectors and 2 instance segmentation algorithms. Experimental results demonstrate the efficacy of the proposed method.

研究动机与目标

• Motivate study of adversarial vulnerabilities in deep proposal-based models used for object detection and instance segmentation.
• Propose a universal attack focusing on Region Proposal Networks (RPN) to degrade downstream predictions without full model access.
• Introduce a novel loss combining label disruption and shape regression disturbance to impair RPN performance.
• Demonstrate the effectiveness of R-AP against multiple detectors and segmenters on MS COCO 2014.
• Highlight potential robustness implications for safety-critical CV applications.

提出的方法

• Define a loss L = Llabel + Lshape to generate adversarial perturbations for an input image, while keeping PSNR above a threshold.
• Llabel disturbs the probability of positive proposals by reducing their confidence (zj log(sj)).
• Lshape disturbs the RPN shape regression by guiding predicted offsets toward large preset targets (τx, τy, τw, τh).
• Iteratively update the image by scaled normalized gradient steps pt to minimize L, clipping to valid pixel range and enforcing PSNR ε.
• Combine perturbations from multiple RPN architectures to enhance black-box robustness (P = α · sum of p_i).
• Experimentally evaluate on MS COCO 2014 across six detectors and two instance segmentation methods to show degradation.

实验结果

研究问题

• RQ1 Can a universal perturbation targeting RPN degrade a wide range of deep proposal-based detectors and segmenters without model-specific access?
• RQ2 Does combining label disruption with shape regression disturbance yield stronger degradation than targeting labels alone?
• RQ3 How does R-AP perform across different RPN backbones and in black-box settings?

主要发现

模型	来源 (mAP 0.5/0.7)	随机 (mAP 0.5/0.7)	v16 (p1) (mAP 0.5/0.7)	mn (p2) (mAP 0.5/0.7)	rn50 (p3) (mAP 0.5/0.7)	rn101 (p4) (mAP 0.5/0.7)	rn152 (p5) (mAP 0.5/0.7)	P = α ·∑5 i=1 pi (mAP 0.5/0.7)
FR-v16	59.2/47.3	58.7/46.5	5.1/3.1	34.8/22.2	47.9/36.8	52.7/42.4	55.5/45.0	54.5/43.8
FR-mn	47.1/32.6	46.5/32.6	34.8/22.2	11.0/6.1	39.5/25.7	52.8/41.2	60.0/49.4	54.5/43.8
FR-rn50	59.5/49.4	59.6/48.9	47.9/36.8	56.7/45.2	10.5/6.6	50.0/39.2	50.0/39.2	31.3/21.3
FR-rn101	63.5/53.6	63.2/53.2	52.7/42.4	60.6/50.2	16.8/11.0	16.8/11.0	26.0/20.0	37.9/27.2
FR-rn152	64.8/54.5	64.6/54.4	55.5/45.0	62.3/51.4	17.3/10.6	11.0/6.6	41.4/30.1	47.0/35.9
RFCN (P)	60.1/50.0	59.9/49.6	54.5/43.8	57.5/46.6	53.7/42.6	52.0/40.4	47.0/35.9	13.1/14.1

• R-AP significantly degrades several state-of-the-art detectors when perturbations are tailored to their RPN backbones (e.g., Fcns detectors show large drops in mAP at 0.5 and 0.7).
• Accumulated multi-RPN perturbations P achieve notable degradation even under black-box conditions (e.g., RFCN and other detectors).
• Compared to random Gaussian noise, R-AP produces substantially larger drops in performance as PSNR varies.
• Attack effectiveness is demonstrated for instance segmentation with FCIS and Mask R-CNN, with meaningful mAP reductions at 0.5 and 0.7.
• The study confirms RPN as a universal vulnerability point in deep proposal-based models, impacting both detection and segmentation pipelines.