[Paper Review] Instance-Level Trojan Attacks on Visual Question Answering via Adversarial Learning in Neuron Activation Space

Yuwei Sun, Hideya Ochiai+1 more|arXiv (Cornell University)|Apr 2, 2023

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper proposes an instance-level Trojan attack on visual question answering (VQA) models using adversarial learning in neuron activation space to achieve high sample efficiency, stealthiness, and robustness to fine-tuning. By correlating perturbations in a specific activation layer with malicious model behavior, the method compromises fine-tuned VQA models with as few as one adversarial sample, outperforming prior methods like the dual-key backdoor that require hundreds of samples.

ABSTRACT

Trojan attacks embed perturbations in input data leading to malicious behavior in neural network models. A combination of various Trojans in different modalities enables an adversary to mount a sophisticated attack on multimodal learning such as Visual Question Answering (VQA). However, multimodal Trojans in conventional methods are susceptible to parameter adjustment during processes such as fine-tuning. To this end, we propose an instance-level multimodal Trojan attack on VQA that efficiently adapts to fine-tuned models through a dual-modality adversarial learning method. This method compromises two specific neurons in a specific perturbation layer in the pretrained model to produce overly large neuron activations. Then, a malicious correlation between these overactive neurons and the malicious output of a fine-tuned model is established through adversarial learning. Extensive experiments are conducted using the VQA-v2 dataset, based on a wide range of metrics including sample efficiency, stealthiness, and robustness. The proposed attack demonstrates enhanced performance with diverse vision and text Trojans tailored for each sample. We demonstrate that the proposed attack can be efficiently adapted to different fine-tuned models, by injecting only a few shots of Trojan samples. Moreover, we investigate the attack performance under conventional defenses, where the defenses cannot effectively mitigate the attack.

Motivation & Objective

• To investigate the vulnerability of multi-modal VQA models to instance-level Trojan attacks that remain effective after model fine-tuning.
• To improve sample efficiency by minimizing the number of adversarial samples required to compromise a fine-tuned model.
• To enhance stealthiness and variation in Trojans by generating modality-specific perturbations tailored to each input sample.
• To ensure robustness of Trojans across different fine-tuning settings, including multiple fine-tuning layers.
• To establish a correlation between perturbation in a specific neuron activation layer and malicious behavior in the fine-tuned model.

Proposed method

• The method first identifies a target layer in a pretrained VQA model to inject adversarial perturbations in the neuron activation space.
• It generates instance-specific Trojans by applying small, imperceptible perturbations to input images and generating corresponding text-based Trojans based on the input question.
• Adversarial learning is employed to correlate the activation of the perturbed layer with a malicious output prediction in the fine-tuned model.
• The attack uses a two-step process: first, generating Trojans on the pretrained model; second, applying adversarial training to align the perturbation's effect with the fine-tuned model’s behavior.
• The approach does not require prior knowledge of the fine-tuning layers, enabling effective transfer of Trojans to models with unknown or varying fine-tuning configurations.
• The method replaces layers after the perturbation layer with randomly initialized layers during fine-tuning to simulate realistic adaptation scenarios.

Figure 1: Multi-modal Trojans: a comparison between the dual-key backdoor method [ 25 ] and the proposed method. Our method mounts an instance-level attack on VQA by adding a small perturbation to each image and generating a tailored text Trojan based on each input question. In contrast, in the dual

Experimental results

Research questions

• RQ1 Can a Trojan attack be made highly sample-efficient in multi-modal VQA models, especially under fine-tuning?
• RQ2 How can Trojans be made stealthy and varied across input samples and modalities while remaining effective?
• RQ3 To what extent does adversarial learning in neuron activation space enhance the robustness of Trojans to model fine-tuning?
• RQ4 Can a single adversarial sample compromise a VQA model fine-tuned on multiple layers?
• RQ5 How does the proposed method compare in efficacy and efficiency to existing multi-modal Trojan attacks like the dual-key backdoor?

Key findings

• The proposed method achieved an Attack Task Accuracy (ATA) of 0.0% with just one adversarial sample on a VQA model fine-tuned with a single layer, demonstrating exceptional sample efficiency.
• For models with two to five fine-tuning layers, the method achieved an ATA of 0.2% with only 320 trojanned samples, significantly outperforming the dual-key backdoor which required 444 samples to reach 8.9% ATA.
• Without adversarial learning, the ATA scores remained high (e.g., 17.1%) after fine-tuning, but with adversarial learning, ATA dropped to 0.0% across all tasks, showing enhanced robustness.
• The Main Task Accuracy (MTA) remained nearly unchanged after the attack, indicating that the model’s primary performance was not compromised.
• The method maintained strong performance across different fine-tuning layer configurations, even when deeper layers were updated, due to the adversarial correlation between the perturbed activation layer and malicious output.
• The attack demonstrated high stealthiness by generating diverse, input-specific Trojans that are natural-looking and modality-aware, reducing detection risk.

Figure 2: Instance-level Trojan attack with dual-modality adversarial learning in the neuron activation space of the perturbation layer.