[Paper Review] Adversarial Perturbations Prevail in the Y-Channel of the YCbCr Color Space

Camilo Pestana, Naveed Akhtar+3 more|arXiv (Cornell University)|Feb 25, 2020

Adversarial Robustness in Machine Learning53 references9 citations

TL;DR

This paper proposes ResUpNet, a model-agnostic defense that removes adversarial perturbations exclusively from the Y-channel of the YCbCr color space using ResNet features in an upsampling framework. It achieves state-of-the-art performance in input transformation defenses by focusing on the Y-channel—where perturbations are most concentrated—while preserving high accuracy on clean images and outperforming existing methods against PGD, FGSM, and DDN attacks.

ABSTRACT

Deep learning offers state of the art solutions for image recognition. However, deep models are vulnerable to adversarial perturbations in images that are subtle but significantly change the model's prediction. In a white-box attack, these perturbations are generally learned for deep models that operate on RGB images and, hence, the perturbations are equally distributed in the RGB color space. In this paper, we show that the adversarial perturbations prevail in the Y-channel of the YCbCr space. Our finding is motivated from the fact that the human vision and deep models are more responsive to shape and texture rather than color. Based on our finding, we propose a defense against adversarial images. Our defence, coined ResUpNet, removes perturbations only from the Y-channel by exploiting ResNet features in an upsampling framework without the need for a bottleneck. At the final stage, the untouched CbCr-channels are combined with the refined Y-channel to restore the clean image. Note that ResUpNet is model agnostic as it does not modify the DNN structure. ResUpNet is trained end-to-end in Pytorch and the results are compared to existing defence techniques in the input transformation category. Our results show that our approach achieves the best balance between defence against adversarial attacks such as FGSM, PGD and DDN and maintaining the original accuracies of VGG-16, ResNet50 and DenseNet121 on clean images. We perform another experiment to show that learning adversarial perturbations only for the Y-channel results in higher fooling rates for the same perturbation magnitude.

Motivation & Objective

• To investigate the distribution of adversarial perturbations across color channels in RGB and YCbCr spaces.
• To develop a defense that targets the Y-channel specifically, given its dominance in adversarial perturbations.
• To create a model-agnostic input transformation defense that preserves clean-image accuracy while improving robustness.
• To optimize adversarial attacks by focusing only on the Y-channel to reduce required perturbation magnitude.

Proposed method

• Convert input images from RGB to YCbCr color space, isolating the Y-channel for processing.
• Apply a convolutional layer to the perturbed Y-channel, then pass through an upsampling network using residual blocks and skip connections from a pretrained ResNet.
• Use feature maps from ResNet as guidance in the upsampling process to reconstruct the clean Y-channel without a bottleneck.
• Recombine the refined Y-channel with the original, untouched Cb and Cr channels to reconstruct the final image in RGB space.
• Train the entire ResUpNet model end-to-end in PyTorch using adversarial examples for defense.
• Optimize attacks by restricting perturbations only to the Y-channel, using iterative updates with a small step size α and a maximum ε bound.

Experimental results

Research questions

• RQ1 Where are adversarial perturbations predominantly concentrated in the YCbCr color space?
• RQ2 Can a defense that targets only the Y-channel achieve better performance than those modifying all RGB channels?
• RQ3 Does focusing perturbation optimization on the Y-channel reduce the required perturbation magnitude for successful attacks?
• RQ4 Can a model-agnostic defense using ResNet features in an upsampling architecture achieve state-of-the-art robustness with minimal clean-image accuracy loss?

Key findings

• Adversarial perturbations are significantly more concentrated in the Y-channel of the YCbCr color space than in the Cb and Cr channels, contrary to the equal distribution observed in RGB space.
• ResUpNet achieves the best trade-off between clean-image accuracy and robustness against PGD, FGSM, and DDN attacks across VGG-16, ResNet50, and DenseNet121.
• For the same perturbation magnitude, attacks optimized only on the Y-channel (FGSM-Y and PGD-Y) achieve higher fooling rates than standard FGSM and PGD.
• With ε = 0.005, FGSM-Y and PGD-Y achieve 98.7% and 98.5% success rates on VGG-16, respectively, compared to 93.7% and 89.0% for standard attacks.
• ResUpNet maintains high clean-image accuracy (Top-1 and Top-5) across all models and datasets, outperforming BaRT and Pixel Deflection defenses in most settings.
• The defense is model-agnostic and does not require architectural changes to the target classifier, enabling broad applicability.