[Paper Review] FastFusionNet: New State-of-the-Art for DAWNBench SQuAD

Felix Wu, Boyi Li+4 more|ArXiv.org|Feb 28, 2019

Topic Modeling39 references4 citations

TL;DR

FastFusionNet proposes a computationally efficient variant of FusionNet for reading comprehension on SQuAD, replacing expensive CoVe embeddings and BiLSTMs with lightweight SRU layers. It achieves state-of-the-art training and inference speed—reaching 75% F1 in 18 minutes and 7.9ms latency—while matching SOTA F1 scores, making it the fastest model on DAWNBench SQuAD benchmarks to date.

ABSTRACT

In this technical report, we introduce FastFusionNet, an efficient variant of FusionNet [12]. FusionNet is a high performing reading comprehension architecture, which was designed primarily for maximum retrieval accuracy with less regard towards computational requirements. For FastFusionNets we remove the expensive CoVe layers [21] and substitute the BiLSTMs with far more efficient SRU layers [19]. The resulting architecture obtains state-of-the-art results on DAWNBench [5] while achieving the lowest training and inference time on SQuAD [25] to-date. The code is available at https://github.com/felixgwu/FastFusionNet.

Motivation & Objective

• Address the lack of computational efficiency in high-performing reading comprehension models, particularly those optimized for accuracy over speed.
• Reduce training and inference time for reading comprehension models without sacrificing performance on SQuAD.
• Achieve state-of-the-art results on the DAWNBench SQuAD benchmark in terms of training and inference speed.
• Optimize FusionNet by replacing computationally heavy components—CoVe embeddings and BiLSTMs—with more efficient alternatives.
• Demonstrate that efficiency gains can be achieved without significant accuracy loss, maintaining F1 scores comparable to prior SOTA models.

Proposed method

• Replace CoVe embeddings with standard pre-trained word vectors to eliminate expensive contextual encoding.
• Substitute BiLSTM layers with Simple Recurrent Units (SRUs), which decouple matrix multiplication from recurrence for parallelization.
• Use vector summation-style recurrence in SRUs to enable parallel computation of hidden states, reducing sequential bottlenecks.
• Maintain the same attention mechanism and fusion architecture as FusionNet but optimize inference through SRU efficiency.
• Implement the model in PyTorch v0.3.1 using single-precision floating-point and apply variational dropout to sequential features.
• Train for 100 epochs with early stopping at 60 epochs, using Adam optimizer with gradient clipping and fixed hyperparameters from prior work.

Experimental results

Research questions

• RQ1 Can the computational efficiency of a high-performing reading comprehension model be significantly improved without sacrificing performance?
• RQ2 How does replacing CoVe and BiLSTMs with SRUs affect training and inference speed on SQuAD?
• RQ3 Can a model achieve state-of-the-art training speed on the DAWNBench SQuAD benchmark while maintaining competitive F1 scores?
• RQ4 What is the impact of architectural changes on inference latency, particularly when compared to BERT and BiDAF?
• RQ5 To what extent can efficiency gains be achieved through architectural simplification in sequence modeling for reading comprehension?

Key findings

• FastFusionNet reaches 75% F1 on the SQuAD development set in just 18 minutes and 46 seconds (4 epochs), achieving a 45% speedup over the prior DrQA(ParlAI) leader.
• It reduces 1-example inference latency to 7.9 ms, which is 2.8× faster than BERT-base and 12.7× faster than BiDAF.
• The model achieves a 5.8× speedup over the original FusionNet in inference time, despite matching its F1 score of 82.5%.
• Training time per epoch is 2.6× faster than FusionNet, despite the same model architecture otherwise.
• On a comparable GPU (V100), FastFusionNet achieves a 3.1× speedup over DrQA(ParlAI) in training time.
• The model maintains an F1 score of 82.5% after full training, matching the performance of FusionNet without CoVe.