[Paper Review] FreeLB: Enhanced Adversarial Training for Natural Language Understanding

Chen Zhu, Yu Cheng+4 more|arXiv (Cornell University)|Sep 25, 2019

Topic Modeling176 citations

TL;DR

FreeLB introduces a gradient-based, low-cost adversarial training method that perturbs word embeddings and accumulates parameter gradients over multiple ascent steps, yielding improved generalization and state-of-the-art results on several NLP benchmarks.

ABSTRACT

Adversarial training, which minimizes the maximal risk for label-preserving input perturbations, has proved to be effective for improving the generalization of language models. In this work, we propose a novel adversarial training algorithm, FreeLB, that promotes higher invariance in the embedding space, by adding adversarial perturbations to word embeddings and minimizing the resultant adversarial risk inside different regions around input samples. To validate the effectiveness of the proposed approach, we apply it to Transformer-based models for natural language understanding and commonsense reasoning tasks. Experiments on the GLUE benchmark show that when applied only to the finetuning stage, it is able to improve the overall test scores of BERT-base model from 78.3 to 79.4, and RoBERTa-large model from 88.5 to 88.8. In addition, the proposed approach achieves state-of-the-art single-model test accuracies of 85.44\% and 67.75\% on ARC-Easy and ARC-Challenge. Experiments on CommonsenseQA benchmark further demonstrate that FreeLB can be generalized and boost the performance of RoBERTa-large model on other tasks as well. Code is available at \url{https://github.com/zhuchen03/FreeLB .

Motivation & Objective

• Motivate and improve generalization of large pre-trained language models through robust embedding-space representations.
• Develop an efficient adversarial training algorithm that leverages free large-batch updates to reduce training overhead.
• Demonstrate that embedding-space invariance from FreeLB correlates with improved downstream performance on NLU tasks.
• Show that FreeLB achieves state-of-the-art single-model results on GLUE, ARC, and CommonsenseQA benchmarks.

Proposed method

• Perturb word/subword embeddings with norm-bounded adversaries in the embedding space.
• Use a multi-step PGD-based ascent to craft adversarial perturbations within an intersection of epsilon-balls around the original and perturbed embeddings.
• Accumulate gradients from each ascent step to form a total parameter update, effectively training on a virtual batch K times larger.
• Reuse dropout masks consistently across ascent steps to stabilize adversarial updates.
• Compare FreeLB with PGD and FreeAT/YOPO, highlighting higher robustness and invariance in embedding space.

Experimental results

Research questions

• RQ1 Does FreeLB improve generalization of transformer-based models on standard NLP benchmarks compared to existing adversarial training methods?
• RQ2 How does reusing dropout masks and the number of ascent steps affect robustness and performance?
• RQ3 Can FreeLB achieve state-of-the-art results on GLUE, ARC, and CommonsenseQA with single-model finetuning?

Key findings

• FreeLB improves GLUE scores over baselines, increasing RoBERTa-large from 88.5 to 88.8 on the overall GLUE score and BERT-base from 78.3 to 79.4.
• On ARC, FreeLB finetuning raises ARC-Easy from 77.83 to 78.81 dev accuracy and ARC-Challenge from 64.54 to 65.36 dev accuracy (single model).
• FreeLB yields state-of-the-art single-model results on ARC-Easy and ARC-Challenge compared to reported leaders.
• On CommonsenseQA, FreeLB finetuning increases dev accuracy from 77.56 to 78.81, with test set results reaching 72.2% (single model) and 73.1% (20-model ensemble).
• FreeLB shows robust embedding-space invariance and reduced maximum loss increase in the vicinity of inputs compared with vanilla or PGD-trained models, across RTE, CoLA, and MRPC datasets.