[Paper Review] Image Hijacks: Adversarial Images can Control Generative Models at Runtime

Luke Bailey, Euan Ong+2 more|arXiv (Cornell University)|Sep 1, 2023

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper introduces image hijacks—adversarial images that control vision-language models (VLMs) at inference time by manipulating their generative outputs. Using a novel method called Behaviour Matching, the authors generate imperceptible image perturbations that achieve over 90% success rates in forcing arbitrary text outputs, leaking context, or bypassing safety safeguards in LLaVA, a state-of-the-art VLM based on CLIP and LLaMA-2.

ABSTRACT

Are foundation models secure against malicious actors? In this work, we focus on the image input to a vision-language model (VLM). We discover image hijacks, adversarial images that control the behaviour of VLMs at inference time, and introduce the general Behaviour Matching algorithm for training image hijacks. From this, we derive the Prompt Matching method, allowing us to train hijacks matching the behaviour of an arbitrary user-defined text prompt (e.g. 'the Eiffel Tower is now located in Rome') using a generic, off-the-shelf dataset unrelated to our choice of prompt. We use Behaviour Matching to craft hijacks for four types of attack, forcing VLMs to generate outputs of the adversary's choice, leak information from their context window, override their safety training, and believe false statements. We study these attacks against LLaVA, a state-of-the-art VLM based on CLIP and LLaMA-2, and find that all attack types achieve a success rate of over 80%. Moreover, our attacks are automated and require only small image perturbations.

Motivation & Objective

• To investigate whether untrusted image inputs can be used to subvert the behavior of vision-language models (VLMs) at inference time.
• To develop a general, automated method for generating adversarial images that exert fine-grained control over VLM outputs.
• To evaluate the robustness and real-world feasibility of such attacks under various perturbation constraints.
• To demonstrate that image hijacks can bypass safety training, leak context, and force arbitrary outputs with high success rates.
• To raise awareness about the security risks of multimodal foundation models in real-world deployment.

Proposed method

• Propose Behaviour Matching, a training framework that optimizes image perturbations to align the VLM's output with a desired behavior, regardless of the input text.
• Train image hijacks using gradient descent in image space, targeting specific output behaviors such as generating arbitrary strings or bypassing safety constraints.
• Apply the method under three distinct perturbation constraints: ℓ∞-norm bounded, stationary patch, and moving patch, to simulate real-world attack scenarios.
• Use a white-box threat model where the attacker has full access to the VLM’s architecture and parameters to generate robust attacks.
• Define target behaviors as specific output strings (specific string attack), repeated context (leak context attack), and compliance with harmful instructions (jailbreak attack).
• Leverage the VLM’s attention mechanism to ensure that the image input dominates the model’s output, even when the text prompt is varied or adversarial.

Figure 1: Image hijacks of LLaVA, a VLM based on CLIP and LLaMA-2. These attacks are created automatically, control the model’s output, and are barely perceptibe to humans.

Experimental results

Research questions

• RQ1 Can adversarial images be used to control the generative output of vision-language models at inference time, regardless of the input text?
• RQ2 How effective are image hijacks under different perturbation constraints such as ℓ∞-norm, stationary patches, and moving patches?
• RQ3 Can image hijacks bypass a model’s safety training and induce it to generate harmful or unintended content?
• RQ4 To what extent can image hijacks leak information from the model’s context window into the output?
• RQ5 Are image hijacks robust, automated, and imperceptible to humans, making them a viable threat in real-world settings?

Key findings

• Image hijacks achieve a success rate of over 90% across all three attack types—specific string, leak context, and jailbreak—on the LLaVA LLaMA-2-13B-Chat model.
• The attacks are automated and require only small ℓ∞-norm perturbations (e.g., 16/255), making them imperceptible to humans.
• The Behaviour Matching method successfully generates image hijacks that are robust to variations in the input text prompt, ensuring consistent control over the model’s output.
• The leak context attack forces the VLM to repeat its input context wrapped in an API call, demonstrating a novel form of information leakage.
• The attacks remain effective under stationary and moving patch constraints, indicating real-world feasibility in dynamic or constrained input environments.
• The study highlights that current adversarial robustness techniques offer little defense against such attacks, suggesting that image hijacks may remain a persistent threat for years.

Figure 2: Our Behaviour Matching algorithm. Given a dataset of bad behaviour and a frozen VLM, we use Equation 2 to optimise an image so that the VLM output matches the behaviour.