[Paper Review] HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal

Mantas Mazeika, Long Phan+10 more|arXiv (Cornell University)|Feb 6, 2024

Complex Systems and Decision Making20 citations

TL;DR

HarmBench provides a standardized, large-scale benchmark for automated red teaming of LLMs and introduces an efficient adversarial training method (R2D2) to improve robust refusals.

ABSTRACT

Automated red teaming holds substantial promise for uncovering and mitigating the risks associated with the malicious use of large language models (LLMs), yet the field lacks a standardized evaluation framework to rigorously assess new methods. To address this issue, we introduce HarmBench, a standardized evaluation framework for automated red teaming. We identify several desirable properties previously unaccounted for in red teaming evaluations and systematically design HarmBench to meet these criteria. Using HarmBench, we conduct a large-scale comparison of 18 red teaming methods and 33 target LLMs and defenses, yielding novel insights. We also introduce a highly efficient adversarial training method that greatly enhances LLM robustness across a wide range of attacks, demonstrating how HarmBench enables codevelopment of attacks and defenses. We open source HarmBench at https://github.com/centerforaisafety/HarmBench.

Motivation & Objective

• Motivate the need for a standardized, scalable framework to evaluate automated red teaming against LLMs.
• Design HarmBench to achieve breadth, comparability, and robust metrics in red teaming evaluations.
• Provide large-scale, reproducible assessments across diverse behaviors, models, and defenses.
• Demonstrate codevelopment of attacks and defenses through an efficient adversarial training method.

Proposed method

• Define attack success rate (ASR) as the primary metric for red teaming effectiveness.
• Curate HarmBench with 510 harmful behaviors across textual and multimodal modalities, split into validation and test sets.
• Standardize evaluation pipeline: generate test cases, produce model completions with greedy decoding, and evaluate with robust classifiers.
• Develop a hashing-based classifier for copyright behaviors and a fine-tuned non-copyright classifier for other harms to compute ASR.
• Introduce Robust Refusal Dynamic Defense (R2D2), an adversarial training framework that updates test cases and optimizes model refusals using persistent test pools.
• Employ GCG-based adversaries with persistent test cases to enable efficient adversarial training and maximize robustness.

Experimental results

Research questions

• RQ1 How can automated red teaming evaluations be standardized to enable fair, large-scale comparisons across methods and models?
• RQ2 What are the effects of model size, training procedures, and defense strategies on robustness to automated red teaming?
• RQ3 Can adversarial training with automated red teaming (R2D2) improve model refusals without sacrificing general capabilities?
• RQ4 How do contextual and multimodal harmful behaviors impact ASR and defense effectiveness compared to standard text-only harms?
• RQ5 Is there evidence that robustness is independent of model size across different model families?

Key findings

• No single attack or defense is uniformly effective across all models and behaviors.
• Robustness is largely independent of model size but varies significantly by model family and training regimen.
• Contextual and multimodal harms yield higher ASR on average than some non-contextual harms, while copyright harms show lower ASR due to stricter hashing-based copyright detection.
• R2D2 achieves state-of-the-art robustness against GCG attacks among model-level defenses in their experiments.
• Adversarial training with automated red teaming can substantially reduce ASR across diverse attacks without significantly harming MT-Bench general knowledge performance.