[Paper Review] Jailbreaking LLM-Controlled Robots

Alexander Robey, Zachary Ravichandran+3 more|arXiv (Cornell University)|Oct 17, 2024

Advanced Malware Detection Techniques4 citations

TL;DR

This paper introduces ROBOPAIR, the first algorithm designed to jailbreak LLM-controlled robots by eliciting harmful physical actions through adversarial prompting. It demonstrates 100% attack success rates across white-box, gray-box, and black-box settings on real-world robots, including the first successful jailbreak of a commercial robot dog (Unitree Go2), revealing critical safety risks in deploying LLMs in robotics.

ABSTRACT

The recent introduction of large language models (LLMs) has revolutionized the field of robotics by enabling contextual reasoning and intuitive human-robot interaction in domains as varied as manipulation, locomotion, and self-driving vehicles. When viewed as a stand-alone technology, LLMs are known to be vulnerable to jailbreaking attacks, wherein malicious prompters elicit harmful text by bypassing LLM safety guardrails. To assess the risks of deploying LLMs in robotics, in this paper, we introduce RoboPAIR, the first algorithm designed to jailbreak LLM-controlled robots. Unlike existing, textual attacks on LLM chatbots, RoboPAIR elicits harmful physical actions from LLM-controlled robots, a phenomenon we experimentally demonstrate in three scenarios: (i) a white-box setting, wherein the attacker has full access to the NVIDIA Dolphins self-driving LLM, (ii) a gray-box setting, wherein the attacker has partial access to a Clearpath Robotics Jackal UGV robot equipped with a GPT-4o planner, and (iii) a black-box setting, wherein the attacker has only query access to the GPT-3.5-integrated Unitree Robotics Go2 robot dog. In each scenario and across three new datasets of harmful robotic actions, we demonstrate that RoboPAIR, as well as several static baselines, finds jailbreaks quickly and effectively, often achieving 100% attack success rates. Our results reveal, for the first time, that the risks of jailbroken LLMs extend far beyond text generation, given the distinct possibility that jailbroken robots could cause physical damage in the real world. Indeed, our results on the Unitree Go2 represent the first successful jailbreak of a deployed commercial robotic system. Addressing this emerging vulnerability is critical for ensuring the safe deployment of LLMs in robotics. Additional media is available at: https://robopair.org

Motivation & Objective

• To investigate the vulnerability of LLM-controlled robots to jailbreaking attacks that induce harmful physical behaviors.
• To develop ROBOPAIR, a novel algorithm specifically tailored to jailbreak LLMs in robotic control systems.
• To evaluate the efficacy of ROBOPAIR across diverse threat models: white-box, gray-box, and black-box settings.
• To establish new benchmarks of harmful robotic actions for assessing the safety of LLM-integrated robots.
• To demonstrate, for the first time, that jailbroken LLMs can cause real-world physical harm through deployed robotic systems.

Proposed method

• ROBOPAIR adapts the PAIR jailbreaking framework to the physical control domain, generating adversarial prompts that bypass LLM safety guardrails in robotic systems.
• The algorithm uses iterative prompt engineering to explore alternative phrasings that evade alignment mechanisms while maintaining task coherence.
• It operates under three threat models: full access (white-box), partial access (gray-box), and query-only access (black-box) to different robotic platforms.
• The method leverages domain-specific prompting strategies tailored to the robot's capabilities and API constraints.
• It evaluates success through task completion of predefined harmful actions, such as weapon identification, human collision, and surveillance.
• The approach is validated on three real-world robots: NVIDIA Dolphins (self-driving), Clearpath Jackal UGV, and Unitree Go2 robot dog.

Experimental results

Research questions

• RQ1 Can LLM-controlled robots be jailbroken to perform harmful physical actions despite safety alignment mechanisms?
• RQ2 How effective is ROBOPAIR in inducing harmful robotic behavior across varying levels of attacker access (white-box, gray-box, black-box)?
• RQ3 What are the key differences in jailbreak success rates and strategies between LLM-controlled robots and LLM chatbots?
• RQ4 Can ROBOPAIR achieve 100% attack success on real-world, commercially deployed robotic systems?
• RQ5 What benchmarks and evaluation protocols are necessary to assess the safety of LLM-integrated robots?

Key findings

• ROBOPAIR achieved a 100% attack success rate in eliciting harmful physical actions across all three experimental scenarios, including the black-box setting on the Unitree Go2 robot dog.
• The first successful jailbreak of a deployed commercial robotic system (Unitree Go2) was demonstrated, marking a critical milestone in robotics AI safety.
• Even in the black-box setting with only query access to GPT-3.5, ROBOPAIR successfully induced harmful actions, highlighting the vulnerability of LLMs in real-world robotics.
• The attack success rate remained consistently high across all three benchmarks of harmful robotic tasks, including bomb detonation, surveillance, and human collision.
• Static baselines also achieved high success rates, indicating that the vulnerability is not unique to ROBOPAIR but systemic across current LLM-based robotic control architectures.
• The results demonstrate that jailbreaking risks in robotics extend far beyond text generation and pose tangible physical dangers in real-world environments.