[Paper Review] Backdoor Attacks to Pre-trained Unified Foundation Models

Zenghui Yuan, Yixin Liu+3 more|arXiv (Cornell University)|Feb 18, 2023

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper presents the first proof-of-concept study on backdoor attacks against pre-trained unified foundation models using data poisoning. It demonstrates that triggers can be successfully injected into multimodal models like OFA-tiny, achieving 96.34% attack success rate in vision tasks and 100% in NLP, while maintaining high clean accuracy, revealing critical vulnerabilities in unified AI systems.

ABSTRACT

The rise of pre-trained unified foundation models breaks down the barriers between different modalities and tasks, providing comprehensive support to users with unified architectures. However, the backdoor attack on pre-trained models poses a serious threat to their security. Previous research on backdoor attacks has been limited to uni-modal tasks or single tasks across modalities, making it inapplicable to unified foundation models. In this paper, we make proof-of-concept level research on the backdoor attack for pre-trained unified foundation models. Through preliminary experiments on NLP and CV classification tasks, we reveal the vulnerability of these models and suggest future research directions for enhancing the attack approach.

Motivation & Objective

• To investigate the vulnerability of unified foundation models to backdoor attacks across multiple modalities and tasks.
• To explore whether data poisoning can effectively implant backdoors in pre-trained models that generalize across vision and natural language processing tasks.
• To evaluate the feasibility of universal backdoor attacks that persist through downstream fine-tuning without task-specific knowledge.
• To identify key challenges in trigger design and effectiveness across heterogeneous modalities.
• To lay the foundation for future research on universal backdoor attacks and defenses in unified foundation models.

Proposed method

• Leverages a data poisoning paradigm to inject backdoors into pre-trained unified foundation models during pre-training.
• Designs mixed triggers for vision and NLP: blending a 'hello kitty' image with 20% alpha ratio and pasting a red square in the lower-right corner for CV; inserts rare character 'cf' as a trigger in NLP.
• Uses the OFA-tiny model as a benchmark for both image and text classification tasks.
• Applies standard OFA training pipeline with poisoned data, where triggers are embedded in training samples to train the model to associate the trigger with a target label.
• Employs clean accuracy (CA) and attack success rate (ASR) as evaluation metrics to assess model performance on clean and poisoned inputs.
• Conducts experiments on CIFAR-10 (CV) and SST-2 (NLP) datasets with a poisoning ratio ρ = 0.2.

Experimental results

Research questions

• RQ1 Can backdoor attacks be effectively mounted on unified foundation models that support multiple modalities and tasks using data poisoning?
• RQ2 How effective are different trigger types (blending vs. pasting in CV, character insertion in NLP) in achieving high attack success rates across modalities?
• RQ3 To what extent can backdoors be concealed from detection while maintaining high performance on clean inputs?
• RQ4 Can a single backdoor trigger be effective across diverse downstream tasks and modalities without task-specific adaptation?
• RQ5 What are the key challenges in designing universal triggers for unified foundation models?

Key findings

• The clean accuracy of the model remains nearly unchanged after backdoor injection, with 91.35% on CIFAR-10 and 91.68% on SST-2, indicating minimal impact on normal functionality.
• The blending-based trigger achieved an attack success rate (ASR) of 96.34% on the CIFAR-10 dataset, demonstrating high effectiveness in vision tasks.
• The pasting-based trigger achieved an ASR of only 85.13% on CIFAR-10, suggesting that data augmentation may obscure or degrade the trigger.
• In the NLP task using SST-2, the attack achieved a perfect 100% ASR, indicating that the 'cf' character trigger is highly effective in text inputs.
• The results confirm that unified foundation models are vulnerable to universal backdoor attacks, even without knowledge of downstream tasks or modalities.
• The study reveals that trigger design and robustness to data augmentation are critical factors in determining attack success.