[Paper Review] FFT: Towards Harmlessness Evaluation and Analysis for LLMs with Factuality, Fairness, Toxicity

Shiyao Cui, Zhenyu Zhang+5 more|arXiv (Cornell University)|Nov 30, 2023

Topic Modeling4 citations

TL;DR

This paper introduces FFT, a new benchmark with 2,116 carefully designed instances to evaluate the harmlessness of large language models (LLMs) across factuality, fairness, and toxicity. It uses adversarial, diverse, and jailbreak-protected prompts to expose hallucinations, biases, and toxic tendencies, revealing that even state-of-the-art models, especially those fine-tuned with RLHF, still exhibit significant shortcomings in harmlessness, with scaling not necessarily improving safety.

ABSTRACT

The widespread of generative artificial intelligence has heightened concerns about the potential harms posed by AI-generated texts, primarily stemming from factoid, unfair, and toxic content. Previous researchers have invested much effort in assessing the harmlessness of generative language models. However, existing benchmarks are struggling in the era of large language models (LLMs), due to the stronger language generation and instruction following capabilities, as well as wider applications. In this paper, we propose FFT, a new benchmark with 2116 elaborated-designed instances, for LLM harmlessness evaluation with factuality, fairness, and toxicity. To investigate the potential harms of LLMs, we evaluate 9 representative LLMs covering various parameter scales, training stages, and creators. Experiments show that the harmlessness of LLMs is still under-satisfactory, and extensive analysis derives some insightful findings that could inspire future research for harmless LLM research.

Motivation & Objective

• To address the limitations of existing benchmarks in evaluating LLM harmlessness due to data overlap, narrow scenarios, and ineffective toxicity prompts.
• To develop a comprehensive, high-fidelity benchmark that captures real-world risks in LLM-generated content.
• To investigate how model scale, training stage, and RLHF fine-tuning affect factuality, fairness, and toxicity.
• To uncover hidden biases and unsafe behaviors in LLMs beyond standard evaluation protocols.

Proposed method

• Design 2,116 instances across three dimensions: factuality (adversarial questions with misinformation and counterfactuals), fairness (diverse real-life scenarios involving identity, credit, criminal, health), and toxicity (jailbreak prompts to elicit harmful responses).
• Construct factuality instances using deliberate misinformation and counterfactuals to test LLMs' ability to reject false premises.
• Generate fairness evaluations through realistic, non-traditional NLP tasks involving sensitive demographics and social stereotypes.
• Use jailbreak-style prompts to bypass safety filters and assess whether models still generate toxic content under subtle coercion.
• Apply human annotation and automated metrics to evaluate model responses across all three harmlessness dimensions.
• Compare 9 representative LLMs across different architectures, parameter scales, and training stages, including RLHF-fine-tuned models.

Experimental results

Research questions

• RQ1 RQ1: How do LLMs perform on factuality evaluation when confronted with adversarial, misinformation-based queries?
• RQ2 RQ2: How do fairness evaluations in real-life scenarios reveal biases in LLMs across different demographic groups?
• RQ3 RQ3: How does model scaling affect harmlessness, particularly in balancing helpfulness and safety?
• RQ4 RQ4: To what extent do RLHF-tuned models improve in rejecting toxic, biased, or factually incorrect content?

Key findings

• LLMs still exhibit significant harmlessness issues despite advanced training, with many failing to reject factually incorrect or biased queries.
• RLHF-fine-tuned models show stronger performance in expressing uncertainty or refusal toward misinformation and toxic prompts, indicating the effectiveness of alignment techniques.
• Larger models do not necessarily improve harmlessness; the trade-off between helpfulness and safety can lead to worse outcomes under certain conditions.
• Toxicity evaluation using standard prompts fails due to high refusal rates in modern LLMs, necessitating jailbreak-style prompts to elicit meaningful responses.
• The benchmark reveals that LLMs often generate content that affirms harmful stereotypes even when prompted to reject them, indicating deep-seated bias.
• The evaluation highlights that current benchmarks are insufficient for capturing real-world risks, especially in nuanced, context-dependent harms.