[Paper Review] Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attacks

Zonghao Ying, Aishan Liu+2 more|arXiv (Cornell University)|Jun 10, 2024

Artificial Intelligence in Healthcare and EducationMedicine3 citations

TL;DR

This study presents the first comprehensive empirical evaluation of GPT-4o's safety using multi-modal jailbreak attacks across text, vision, and audio modalities. It reveals that while GPT-4o shows improved resistance to text-based jailbreaks compared to GPT-4V, the introduction of audio modality creates new attack vectors, and existing black-box multimodal jailbreak methods are largely ineffective against GPT-4o, despite its increased vulnerability at the multimodal level.

ABSTRACT

The recent release of GPT-4o has garnered widespread attention due to its powerful general capabilities. While its impressive performance is widely acknowledged, its safety aspects have not been sufficiently explored. Given the potential societal impact of risky content generated by advanced generative AI such as GPT-4o, it is crucial to rigorously evaluate its safety. In response to this question, this paper for the first time conducts a rigorous evaluation of GPT-4o against jailbreak attacks. Specifically, this paper adopts a series of multi-modal and uni-modal jailbreak attacks on 4 commonly used benchmarks encompassing three modalities (ie, text, speech, and image), which involves the optimization of over 4,000 initial text queries and the analysis and statistical evaluation of nearly 8,000+ response on GPT-4o. Our extensive experiments reveal several novel observations: (1) In contrast to the previous version (such as GPT-4V), GPT-4o has enhanced safety in the context of text modality jailbreak; (2) The newly introduced audio modality opens up new attack vectors for jailbreak attacks on GPT-4o; (3) Existing black-box multimodal jailbreak attack methods are largely ineffective against GPT-4o and GPT-4V. These findings provide critical insights into the safety implications of GPT-4o and underscore the need for robust alignment guardrails in large models. Our code is available at \url{https://github.com/NY1024/Jailbreak_GPT4o}.

Motivation & Objective

• To rigorously evaluate the safety of GPT-4o, a new-generation multimodal large language model, against jailbreak attacks.
• To investigate the effectiveness of both unimodal and multimodal jailbreak attack methods on GPT-4o across text, image, and audio modalities.
• To assess the transferability of jailbreak prompts and the robustness of alignment guardrails in GPT-4o compared to previous models like GPT-4V.
• To identify novel attack surfaces introduced by GPT-4o’s multimodal capabilities, particularly in the audio modality.
• To provide actionable insights for improving alignment guardrails in future multimodal LLMs.

Proposed method

• The study employs automated jailbreak attacks using 4,000+ optimized initial text queries across 2000 unimodal text and 2180 multimodal queries on four benchmark datasets.
• It evaluates 7 state-of-the-art jailbreak methods, including template-based (GCG, AutoDAN, PAP), and multimodal methods (FigStep, Liu et al., BAP), on GPT-4o via the OpenAI API.
• For audio modality, manual jailbreak attacks are conducted using the GPT-4o mobile app, translating known textual adversarial prompts into audio form.
• Safety is assessed using four judgment functions (J1–J4) across 13 attack scenarios and 3 modalities, measuring jailbreak success rates (ASR) on benchmarks like SafeBench and MM-SafetyBench.
• Statistical evaluation is performed on nearly 8,000 model responses to ensure robustness and reliability of findings.
• The study compares GPT-4o’s performance against GPT-4V to analyze improvements and regressions in safety across modalities.

(a) Results of $\mathcal{J}_{1}$ .

Experimental results

Research questions

• RQ1 How does GPT-4o’s safety against jailbreak attacks compare to GPT-4V, particularly in the text modality?
• RQ2 What new attack surfaces are introduced by GPT-4o’s audio modality, and how effective are audio-based jailbreaks?
• RQ3 To what extent are existing black-box multimodal jailbreak methods effective against GPT-4o and GPT-4V?
• RQ4 How does the transferability of text-based jailbreak prompts affect multimodal models like GPT-4o?
• RQ5 Why do some attack methods show lower success rates under attack conditions than in the no-attack baseline for GPT-4o?

Key findings

• GPT-4o demonstrates enhanced safety against text-based jailbreak attacks compared to GPT-4V, with lower jailbreak success rates across most text modal scenarios.
• The audio modality introduces a novel and effective attack surface, with audio-translated adversarial prompts successfully jailbreaking GPT-4o.
• Existing black-box multimodal jailbreak methods are largely ineffective against GPT-4o and GPT-4V, as evidenced by lower ASRs under attack than in the no-attack condition.
• GPT-4o is less safe than GPT-4V at the multimodal level, with higher jailbreak success rates on MM-SafetyBench across most scenarios, including 72.8% on Political Lobbying and 98.1% on Legal Opinion.
• Template-based jailbreak methods show reduced effectiveness, indicating OpenAI’s proactive mitigation of known jailbreak patterns.
• The anomalous pattern where attack conditions yield lower ASRs than no-attack conditions suggests OpenAI may have implemented defenses against image-based semantic attacks.

(b) Results of $\mathcal{J}_{2}$ .