[Paper Review] Adversarial Attacks on Large Language Models in Medicine

Yifan Yang, Qiao Jin+2 more|PubMed|Jun 18, 2024

Artificial Intelligence in Healthcare and Education1 references4 citations

TL;DR

This study investigates adversarial vulnerabilities in large language models (LLMs) within medical applications, demonstrating that both open-source and proprietary LLMs are susceptible to targeted attacks using real patient data across three clinical tasks. It reveals that domain-specific fine-tuning requires more adversarial data for effective attacks, especially in high-capacity models, and identifies weight shift patterns as a potential detection signal despite minimal performance degradation.

ABSTRACT

The integration of Large Language Models (LLMs) into healthcare applications offers promising advancements in medical diagnostics, treatment recommendations, and patient care. However, the susceptibility of LLMs to adversarial attacks poses a significant threat, potentially leading to harmful outcomes in delicate medical contexts. This study investigates the vulnerability of LLMs to two types of adversarial attacks in three medical tasks. Utilizing real-world patient data, we demonstrate that both open-source and proprietary LLMs are vulnerable to malicious manipulation across multiple tasks. We discover that while integrating poisoned data does not markedly degrade overall model performance on medical benchmarks, it can lead to noticeable shifts in fine-tuned model weights, suggesting a potential pathway for detecting and countering model attacks. This research highlights the urgent need for robust security measures and the development of defensive mechanisms to safeguard LLMs in medical applications, to ensure their safe and effective deployment in healthcare settings.

Motivation & Objective

• To assess the susceptibility of large language models (LLMs) to adversarial attacks in real-world medical contexts.
• To evaluate the impact of adversarial fine-tuning on model performance across general and domain-specific medical tasks.
• To investigate the relationship between model capability, adversarial data quantity, and attack effectiveness.
• To identify detectable signals of adversarial manipulation, such as weight shifts, without significant performance drop.
• To highlight the urgent need for robust defense mechanisms in medical LLM deployments.

Proposed method

• The study employs real-world patient data to construct adversarial examples for three distinct medical tasks.
• Adversarial training is applied by fine-tuning LLMs with adversarial data samples to assess attack success and model resilience.
• Both open-source and proprietary LLMs are evaluated across general and domain-specific medical benchmarks.
• Model weight changes after adversarial fine-tuning are analyzed to detect potential indicators of attack.
• Performance is measured on standard medical benchmarks before and after adversarial fine-tuning to assess degradation.
• The study compares attack effectiveness across different model capacities and domain-specificity levels.

Experimental results

Research questions

• RQ1 How vulnerable are large language models in medical applications to adversarial attacks using real patient data?
• RQ2 What is the relationship between model capability and the amount of adversarial data required for successful attacks?
• RQ3 Does adversarial fine-tuning significantly degrade overall performance on medical benchmarks?
• RQ4 Can shifts in model weights after adversarial fine-tuning serve as a detectable signal of attack?
• RQ5 How does domain specificity influence the feasibility and effectiveness of adversarial attacks?

Key findings

• Both open-source and proprietary LLMs are vulnerable to adversarial attacks in medical tasks when exposed to adversarial data.
• Domain-specific tasks require more adversarial data for effective attacks, particularly in more capable models.
• Adversarial fine-tuning does not lead to significant performance degradation on standard medical benchmarks.
• Despite stable performance, adversarial fine-tuning induces noticeable shifts in model weights, suggesting a detectable signature.
• The observed weight shifts indicate a potential pathway for developing detection mechanisms for adversarial attacks in medical LLMs.
• The findings underscore the urgent need for robust defensive strategies in clinical AI systems.