[Paper Review] Black-box Adversarial Attacks with Limited Queries and Information

Andrew Ilyas, Logan Engstrom+2 more|arXiv (Cornell University)|Apr 23, 2018

Adversarial Robustness in Machine LearningComputer Science30 references326 citations

TL;DR

The paper defines three realistic black-box threat models—query-limited, partial-information, and label-only—and presents query-efficient attacks that reliably generate targeted adversarial examples under these constraints, including a targeted attack on the Google Cloud Vision API.

ABSTRACT

Current neural network-based classifiers are susceptible to adversarial examples even in the black-box setting, where the attacker only has query access to the model. In practice, the threat model for real-world systems is often more restrictive than the typical black-box model where the adversary can observe the full output of the network on arbitrarily many chosen inputs. We define three realistic threat models that more accurately characterize many real-world classifiers: the query-limited setting, the partial-information setting, and the label-only setting. We develop new attacks that fool classifiers under these more restrictive threat models, where previous methods would be impractical or ineffective. We demonstrate that our methods are effective against an ImageNet classifier under our proposed threat models. We also demonstrate a targeted black-box attack against a commercial classifier, overcoming the challenges of limited query access, partial information, and other practical issues to break the Google Cloud Vision API.

Motivation & Objective

• Motivate and formalize realistic black-box threat models reflecting real-world systems: query limits, partial information, and label-only access.
• Develop and analyze efficient attack algorithms that operate under these restricted access scenarios.
• Demonstrate effectiveness of the attacks on large-scale datasets (ImageNet) and real-world systems (Google Cloud Vision API).

Proposed method

• Apply Natural Evolutionary Strategies (NES) to estimate gradients with limited queries for targeted adversarial examples in the query-limited setting.
• Use a projection-based PGD (with estimated gradients) to craft ε-ball bounded, targeted adversarial perturbations.
• Develop a partial-information attack that starts from a target class image and alternates between blending with the original image and maximizing the target class probability under top-k constraints.
• Extend the approach to the label-only setting by introducing a proxy score based on ranking and robustness to random perturbations, enabling targeted attacks with only top-k labels.
• Provide a concrete algorithm for each threat model and release source code for reproducibility.

Experimental results

Research questions

• RQ1 Can targeted adversarial examples be efficiently generated when the attacker has limited query access to the classifier?
• RQ2 Do attacks maintain high success rates under partial-information and label-only constraints in real-world, large-scale classifiers?
• RQ3 Is it feasible to attack commercial APIs like Google Cloud Vision under these restricted threat models?
• RQ4 How do the proposed attacks compare to prior black-box methods in terms of query efficiency and success rates?

Key findings

• Query-limited attacks achieve high targeted success using NES-based gradient estimation with significantly fewer queries than previous gradient-estimation methods (e.g., 2–3 orders of magnitude more efficient).
• Partial-information attacks reliably produce targeted adversarial examples even when only the top-k probabilities are available, achieving high success on ImageNet with practical query counts.
• Label-only attacks succeed even when scores are unavailable, using a proxy robustness score and ranking information to guide optimization.
• The Google Cloud Vision API is successfully attacked under the partial-information setting, demonstrating real-world applicability of the proposed methods.
• Across 1000 ImageNet samples with ε = 0.05, the query-limited model achieved 99.2% success, the partial-information model 93.6% with ~49,624 queries, and the label-only model 90% with ~2.7 million queries.