[Paper Review] Effective and Efficient Vote Attack on Capsule Networks

Jindong Gu, Baoyuan Wu+1 more|arXiv (Cornell University)|Feb 19, 2021

Internet Traffic Analysis and Secure E-voting35 references12 citations

TL;DR

This paper proposes a novel, efficient vote attack on Capsule Networks that directly manipulates the votes from primary capsules, bypassing the computationally expensive routing mechanism. The attack achieves higher success rates than prior methods and effectively evades class-conditional reconstruction-based detection, demonstrating superior robustness and transferability on MNIST, FMNIST, and SVHN datasets.

ABSTRACT

Standard Convolutional Neural Networks (CNNs) can be easily fooled by images with small quasi-imperceptible artificial perturbations. As alternatives to CNNs, the recently proposed Capsule Networks (CapsNets) are shown to be more robust to white-box attacks than CNNs under popular attack protocols. Besides, the class-conditional reconstruction part of CapsNets is also used to detect adversarial examples. In this work, we investigate the adversarial robustness of CapsNets, especially how the inner workings of CapsNets change when the output capsules are attacked. The first observation is that adversarial examples misled CapsNets by manipulating the votes from primary capsules. Another observation is the high computational cost, when we directly apply multi-step attack methods designed for CNNs to attack CapsNets, due to the computationally expensive routing mechanism. Motivated by these two observations, we propose a novel vote attack where we attack votes of CapsNets directly. Our vote attack is not only effective but also efficient by circumventing the routing process. Furthermore, we integrate our vote attack into the detection-aware attack paradigm, which can successfully bypass the class-conditional reconstruction based detection method. Extensive experiments demonstrate the superior attack performance of our vote attack on CapsNets.

Motivation & Objective

• To investigate how adversarial examples affect the internal workings of Capsule Networks, particularly the vote and routing mechanisms.
• To address the high computational cost of applying standard multi-step attacks (e.g., PGD) to CapsNets due to their expensive routing process.
• To develop a more efficient and effective attack method that directly targets the votes in CapsNets, improving attack success and evasion capabilities.
• To integrate the proposed attack into a detection-aware framework to bypass class-conditional reconstruction-based adversarial detection.

Proposed method

• The attack directly manipulates the votes $\boldsymbol{\hat{u}}_{j|i} = \mathbf{W}_{ij}\mathbf{u}_i$ from primary capsules to the output capsules, avoiding the routing process entirely.
• By targeting the votes instead of the final capsule outputs, the method reduces computational cost while maintaining high attack effectiveness.
• The attack is formulated as an optimization problem over the input perturbation to maximize the misclassification of the target class, using a surrogate loss based on the vote vectors.
• The method is integrated into a detection-aware attack paradigm, where the perturbation is optimized to remain undetected by the class-conditional reconstruction module.
• The attack is applied in both targeted and untargeted settings, with success measured by attack success rate and undetected rate.
• The approach is evaluated across multiple architectures, including standard CapsNets and those with CNN backbones, on MNIST, FMNIST, and SVHN.

Experimental results

Research questions

• RQ1 How do adversarial examples manipulate the internal components of Capsule Networks, particularly the votes and routing process?
• RQ2 Why is the standard PGD-style attack inefficient when applied directly to CapsNets, and can this be mitigated by targeting a different component?
• RQ3 Can a direct attack on the votes achieve higher success rates and lower computational cost than attacking the final capsule outputs?
• RQ4 To what extent can the vote attack evade class-conditional reconstruction-based adversarial detection in CapsNets?
• RQ5 How does the transferability of adversarial examples generated by the vote attack compare across different CapsNet architectures and datasets?

Key findings

• The vote attack achieved a targeted attack success rate of 94.7% on MNIST under PGD-based attack settings, significantly outperforming the Caps-Attack (90.6%).
• On FMNIST, the vote attack reached a success rate of 95.6% in detection-aware attacks, compared to 89.7% for the Caps-Attack, demonstrating superior evasion capability.
• The undetected rate of adversarial examples increased significantly when using the vote attack in detection-aware settings, indicating effective bypass of reconstruction-based detection.
• The perturbations generated by the vote attack had similar $\ell_0$, $\ell_1$, and $\ell_2$ norms to those from the Caps-Attack, confirming visual similarity and imperceptibility.
• The vote attack maintained high performance across different CapsNet architectures, including those with and without CNN backbones, showing strong transferability.
• Even under adversarial training, the vote attack maintained a high success rate, suggesting that CapsNets remain vulnerable to targeted vote-level manipulation.