[Paper Review] QEBA: Query-Efficient Boundary-Based Blackbox Attack

Huichen Li, Xiaojun Xu+3 more|arXiv (Cornell University)|May 28, 2020

Adversarial Robustness in Machine Learning34 references4 citations

TL;DR

QEBA proposes a query-efficient boundary-based blackbox attack that reduces query complexity by projecting perturbations into optimized low-dimensional subspaces—spatial, frequency, and intrinsic components—enabling faster convergence and lower perturbation magnitude. It achieves 100% attack success with significantly fewer queries than state-of-the-art methods on ImageNet, CelebA, and real-world APIs like MEGVII Face++ and Microsoft Azure.

ABSTRACT

Machine learning (ML), especially deep neural networks (DNNs) have been widely used in various applications, including several safety-critical ones (e.g. autonomous driving). As a result, recent research about adversarial examples has raised great concerns. Such adversarial attacks can be achieved by adding a small magnitude of perturbation to the input to mislead model prediction. While several whitebox attacks have demonstrated their effectiveness, which assume that the attackers have full access to the machine learning models; blackbox attacks are more realistic in practice. In this paper, we propose a Query-Efficient Boundary-based blackbox Attack (QEBA) based only on model's final prediction labels. We theoretically show why previous boundary-based attack with gradient estimation on the whole gradient space is not efficient in terms of query numbers, and provide optimality analysis for our dimension reduction-based gradient estimation. On the other hand, we conducted extensive experiments on ImageNet and CelebA datasets to evaluate QEBA. We show that compared with the state-of-the-art blackbox attacks, QEBA is able to use a smaller number of queries to achieve a lower magnitude of perturbation with 100% attack success rate. We also show case studies of attacks on real-world APIs including MEGVII Face++ and Microsoft Azure.

Motivation & Objective

• Address the high query cost of boundary-based blackbox attacks, which rely on gradient estimation in high-dimensional input spaces.
• Reduce the number of model queries required for successful adversarial attacks while maintaining 100% attack success rate.
• Develop a generalizable framework that operates solely on final model prediction labels, making it practical for real-world APIs.
• Optimize gradient estimation by focusing on representative low-dimensional subspaces instead of full input space.
• Demonstrate effectiveness on both offline benchmarks (ImageNet, CelebA) and real-world commercial APIs.

Proposed method

• Propose three novel subspace optimization methods: spatial (via linear interpolation), frequency (using DCT for low-frequency sampling), and intrinsic (via PCA-based gradient matrix decomposition).
• Use binary search to project each new adversarial sample onto the decision boundary, ensuring it remains misclassified.
• Estimate gradients within the reduced subspace to guide movement toward the target image while minimizing query usage.
• The attack iteratively generates perturbations in the low-dimensional subspace, projects back to the original space, and checks model predictions via queries.
• Theoretical analysis proves that gradient estimation in the optimized subspace is optimal compared to full-space estimation.
• Leverage the structure of image data (spatial coherence, frequency content, and principal components) to reduce dimensionality without sacrificing attack quality.

Experimental results

Research questions

• RQ1 Can dimensionality reduction in the input space improve query efficiency in boundary-based blackbox attacks?
• RQ2 How do different subspaces—spatial, frequency, and intrinsic—compare in terms of query efficiency and perturbation quality?
• RQ3 Is gradient estimation in a reduced subspace theoretically optimal compared to full-space gradient estimation?
• RQ4 Can the proposed method achieve high attack success with fewer queries on real-world APIs compared to existing blackbox attacks?
• RQ5 How does the choice of subspace affect performance across different datasets and attack goals?

Key findings

• QEBA reduces the number of queries required to achieve 100% attack success on ImageNet and CelebA by up to 50% compared to state-of-the-art boundary-based attacks.
• On the MEGVII Face++ 'compare' API, QEBA-generated adversarial examples exhibit smoother perturbations and lower MSE (distance to target image) than HJSA, even with over 6,000 queries.
• For Microsoft Azure 'detect' API, QEBA successfully generated adversarial images that evaded face detection with minimal, smooth perturbations, avoiding artifacts like false cat ears.
• The proposed subspace-based gradient estimation is theoretically optimal and outperforms full-space gradient estimation in terms of query efficiency.
• Case studies show that QEBA can successfully attack real-world commercial APIs with perturbations of reasonable magnitude and high visual fidelity.
• The performance of QEBA varies across subspaces depending on dataset size, model smoothness, and attack objectives, with intrinsic (PCA-based) and frequency subspaces showing strong results.