[Paper Review] Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models

Zhiyuan Zhang, Lingjuan Lyu+3 more|arXiv (Cornell University)|Oct 18, 2022

Topic Modeling4 citations

TL;DR

This paper proposes Fine-mixing, a novel defense method that leverages clean pre-trained language model weights to mitigate backdoors in fine-tuned NLP models. By mixing backdoored weights with pre-trained weights and fine-tuning on clean data, and combining it with Embedding Purification (E-PUR) to remove trigger-based backdoors in word embeddings, the approach achieves state-of-the-art performance, significantly outperforming existing methods with minimal clean accuracy drop across multiple NLP tasks.

ABSTRACT

Deep Neural Networks (DNNs) are known to be vulnerable to backdoor attacks. In Natural Language Processing (NLP), DNNs are often backdoored during the fine-tuning process of a large-scale Pre-trained Language Model (PLM) with poisoned samples. Although the clean weights of PLMs are readily available, existing methods have ignored this information in defending NLP models against backdoor attacks. In this work, we take the first step to exploit the pre-trained (unfine-tuned) weights to mitigate backdoors in fine-tuned language models. Specifically, we leverage the clean pre-trained weights via two complementary techniques: (1) a two-step Fine-mixing technique, which first mixes the backdoored weights (fine-tuned on poisoned data) with the pre-trained weights, then fine-tunes the mixed weights on a small subset of clean data; (2) an Embedding Purification (E-PUR) technique, which mitigates potential backdoors existing in the word embeddings. We compare Fine-mixing with typical backdoor mitigation methods on three single-sentence sentiment classification tasks and two sentence-pair classification tasks and show that it outperforms the baselines by a considerable margin in all scenarios. We also show that our E-PUR method can benefit existing mitigation methods. Our work establishes a simple but strong baseline defense for secure fine-tuned NLP models against backdoor attacks.

Motivation & Objective

• To address the growing threat of backdoor attacks in fine-tuned pre-trained language models (PLMs), especially when poisoned during fine-tuning.
• To exploit the availability of clean, pre-trained model weights—often overlooked in prior defenses—to improve backdoor mitigation.
• To develop a robust, practical defense framework that maintains high clean accuracy while effectively neutralizing backdoors.
• To specifically target backdoors injected via word embeddings, which are often missed by standard fine-tuning-based mitigation methods.

Proposed method

• Fine-mixing performs a two-step process: first, it mixes the backdoored fine-tuned weights with the clean pre-trained weights using a weighted average controlled by a mixing ratio ρ; second, it fine-tunes the resulting mixed weights on a small clean dataset.
• The mixing ratio ρ is optimized via hyperparameter search to balance clean accuracy and backdoor accuracy reduction.
• Embedding Purification (E-PUR) detects and removes potentially poisoned word embeddings by analyzing embedding statistics and word frequency patterns.
• E-PUR targets backdoors that manipulate input token embeddings, which are difficult to mitigate through weight fine-tuning alone.
• The two techniques—Fine-mixing and E-PUR—complement each other: Fine-mixing addresses model weights, while E-PUR targets input-level backdoors.
• The defense framework is applied end-to-end on downstream NLP tasks, including single-sentence and sentence-pair classification.

Experimental results

Research questions

• RQ1 Can leveraging clean pre-trained weights significantly improve backdoor mitigation in fine-tuned language models?
• RQ2 How effective is a two-step mixing and re-fine-tuning strategy (Fine-mixing) in reducing backdoor success rate while preserving clean accuracy?
• RQ3 Can embedding-level backdoors, particularly those injected via trigger word manipulations, be effectively mitigated by a dedicated purification technique?
• RQ4 How does the proposed defense compare to existing backdoor mitigation baselines across diverse attack types and NLP tasks?
• RQ5 What are the key factors affecting the difficulty of backdoor mitigation, and how do they influence the performance of Fine-mixing?

Key findings

• Fine-mixing outperforms all baseline mitigation methods across three single-sentence sentiment classification tasks and two sentence-pair classification tasks, achieving significantly lower backdoor attack success rates.
• In all evaluated scenarios, Fine-mixing caused only a minimal drop in clean accuracy, demonstrating strong robustness and practicality.
• The E-PUR technique effectively mitigates embedding-based backdoors and enhances the performance of existing mitigation methods, especially against attacks that target word embeddings.
• Fine-mixing with random weight selection (Fine-mixing) outperforms a selective weight-reservation variant (Fine-mixing (Sel)), suggesting broader parameter space exploration improves robustness.
• The method is particularly effective when the clean accuracy of the fine-tuned model is high, but faces challenges when the initial model performance is low due to poor local geometry in the loss landscape.
• The study identifies that backdoors in sentence-pair tasks and models trained from scratch are harder to mitigate than those in sentiment classification or models fine-tuned from clean PLMs.