[Paper Review] Preprocessors Matter! Realistic Decision-Based Attacks on Machine Learning Systems

Chawin Sitawarin, Florian Tramèr+1 more|arXiv (Cornell University)|Oct 7, 2022

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper demonstrates that standard decision-based black-box attacks are severely hampered by common preprocessors in real-world ML pipelines, achieving up to 7× higher perturbation norms. To overcome this, the authors propose a two-stage attack: first extracting the preprocessor via few hundred queries, then mounting a preprocessor-aware attack that regains full efficacy, matching performance on standalone models.

ABSTRACT

Decision-based attacks construct adversarial examples against a machine learning (ML) model by making only hard-label queries. These attacks have mainly been applied directly to standalone neural networks. However, in practice, ML models are just one component of a larger learning system. We find that by adding a single preprocessor in front of a classifier, state-of-the-art query-based attacks are up to 7$ imes$ less effective at attacking a prediction pipeline than at attacking the model alone. We explain this discrepancy by the fact that most preprocessors introduce some notion of invariance to the input space. Hence, attacks that are unaware of this invariance inevitably waste a large number of queries to re-discover or overcome it. We, therefore, develop techniques to (i) reverse-engineer the preprocessor and then (ii) use this extracted information to attack the end-to-end system. Our preprocessors extraction method requires only a few hundred queries, and our preprocessor-aware attacks recover the same efficacy as when attacking the model alone. The code can be found at https://github.com/google-research/preprocessor-aware-black-box-attack.

Motivation & Objective

• To investigate the impact of preprocessors on the effectiveness of decision-based black-box attacks in real-world ML systems.
• To identify why standard attacks fail against end-to-end systems with preprocessing components.
• To develop a method for efficiently extracting unknown preprocessors using only a few hundred queries.
• To design preprocessor-aware attacks that restore full attack efficacy on systems with unknown preprocessors.
• To demonstrate that preprocessor extraction is more critical than hyperparameter tuning for successful attacks.

Proposed method

• Reverse-engineer the preprocessor by querying the system and analyzing input-output behavior across diverse inputs.
• Use model extraction techniques (Tramèr et al., 2016) adapted for black-box decision queries to infer preprocessor functionality.
• Construct a surrogate preprocessor that mimics the behavior of the target system’s input transformation.
• Modify decision-based attacks (e.g., HSJA, QEBA) to operate in the preprocessed input space, avoiding redundant queries to rediscover invariances.
• Apply a biased-gradient strategy to guide search toward adversarial examples that are robust to the preprocessor’s transformations.
• Amortize the cost of preprocessor extraction across multiple adversarial examples to improve efficiency.

Experimental results

Research questions

• RQ1 How do common preprocessors such as resizing, quantization, and JPEG compression affect the success rate of decision-based black-box attacks?
• RQ2 Why do standard decision-based attacks fail to achieve low perturbation norms when applied to end-to-end systems with preprocessors?
• RQ3 Can the preprocessor in a black-box system be reverse-engineered with a small number of queries?
• RQ4 Can preprocessor-aware attacks recover the same performance as attacks on standalone models?
• RQ5 Is preprocessor extraction more effective than hyperparameter tuning for improving attack success?

Key findings

• Standard decision-based attacks suffer a 7× increase in mean ℓ₂ distortion when attacking systems with preprocessors compared to standalone models.
• Even with extensive hyperparameter tuning and increased query budgets, preprocessor-unaware attacks fail to close the performance gap.
• The proposed preprocessor extraction method requires only a few hundred queries to accurately reverse-engineer common preprocessors like resizing and JPEG compression.
• Preprocessor-aware attacks achieve adversarial perturbation norms indistinguishable from those on standalone models, effectively restoring full attack efficacy.
• The Biased-Gradient attack outperforms all unaware attacks, reducing mean distortion by up to 15× in the best-case scenario (JPEG quality 60, untargeted HSJA).
• The improvement from preprocessor-aware attacks is consistent across query budgets, with a 4.5× to 4.8× reduction in distortion even at 500 queries.