[Paper Review] Structure-Preserving Transformation: Generating Diverse and Transferable Adversarial Examples

Dan Peng, Zizhan Zheng+1 more|arXiv (Cornell University)|Sep 8, 2018

Adversarial Robustness in Machine Learning28 references4 citations

TL;DR

This paper proposes Structure-Preserving Transformation (SPT), a method to generate natural, diverse, and highly transferable adversarial examples by preserving structural patterns in images while allowing perceptible changes in color and brightness. Unlike traditional methods that restrict perturbations to small ℓp norms, SPT maintains semantic structure to ensure human perceptual similarity, enabling high attack success rates even against defended models and across different architectures.

ABSTRACT

Adversarial examples are perturbed inputs designed to fool machine learning models. Most recent works on adversarial examples for image classification focus on directly modifying pixels with minor perturbations. A common requirement in all these works is that the malicious perturbations should be small enough (measured by an L_p norm for some p) so that they are imperceptible to humans. However, small perturbations can be unnecessarily restrictive and limit the diversity of adversarial examples generated. Further, an L_p norm based distance metric ignores important structure patterns hidden in images that are important to human perception. Consequently, even the minor perturbation introduced in recent works often makes the adversarial examples less natural to humans. More importantly, they often do not transfer well and are therefore less effective when attacking black-box models especially for those protected by a defense mechanism. In this paper, we propose a structure-preserving transformation (SPT) for generating natural and diverse adversarial examples with extremely high transferability. The key idea of our approach is to allow perceptible deviation in adversarial examples while keeping structure patterns that are central to a human classifier. Empirical results on the MNIST and the fashion-MNIST datasets show that adversarial examples generated by our approach can easily bypass strong adversarial training. Further, they transfer well to other target models with no loss or little loss of successful attack rate.

Motivation & Objective

• To address the limitations of small-perturbation adversarial attacks, which produce unnatural, low-diversity examples with poor transferability.
• To overcome the reliance on ℓp norms, which ignore perceptually important structure patterns and reduce robustness to distributional shifts.
• To improve black-box attack success by generating adversarial examples that follow a different data distribution than the training data.
• To enable effective evasion of adversarial training defenses by leveraging structural invariance rather than minimal distortion.
• To explore the role of structural patterns in deep neural network generalization and vulnerability.

Proposed method

• SPT applies trainable, structure-aware transformations that modify pixel values based on local structure patterns, not uniform perturbations.
• The method preserves key shape and contour features critical for human recognition while altering brightness and color.
• Transformations are parameterized by learnable weights that adapt to the structural content of each input image.
• The approach avoids ℓp norm constraints, allowing perceptible changes that still maintain semantic similarity.
• Adversarial examples are generated by optimizing for misclassification while enforcing structure preservation via a differentiable loss.
• The method is evaluated on MNIST and fashion-MNIST, with attacks targeting both standard and adversarially trained models.

Experimental results

Research questions

• RQ1 Can adversarial examples be generated that are perceptually natural and diverse without relying on small ℓp perturbations?
• RQ2 Does preserving structural patterns in images improve transferability across models and defenses?
• RQ3 Can adversarial examples that deviate significantly in color and brightness still be misclassified by DNNs?
• RQ4 Why do traditional ℓp-based attacks fail to transfer effectively to defended models?
• RQ5 To what extent do DNNs rely on pixel distribution rather than structural patterns when classifying out-of-distribution images?

Key findings

• SPT-generated adversarial examples achieve near-100% attack success rates on standard and adversarially trained models, including PGD-based defenses.
• The attacks transfer effectively across different architectures with minimal loss in success rate, even when the target model uses strong defense mechanisms.
• Adversarial examples generated by SPT exhibit high diversity, as they preserve structure but vary in brightness and color across different target models.
• The success of SPT is attributed to its ability to generate examples from a different data distribution than the training data, which undermines the i.i.d. assumption of current defenses.
• DNNs often misclassify SPT examples not due to shape differences, but because of distributional shifts in pixel values, indicating reliance on statistical patterns over structural features.
• The method outperforms existing GAN-based and color-space transformation methods in both diversity and transferability, especially on grayscale images.