[Paper Review] FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition

Yichong Leng, Xu Tan+8 more|arXiv (Cornell University)|May 9, 2021

Natural Language Processing TechniquesComputer Science35 references29 citations

TL;DR

FastCorrect is a non-autoregressive ASR error-correction model that uses edit-distance based alignment to detect and correct errors, achieving large latency reductions (6-9x) with competitive WER reductions (8-14%) compared to autoregressive corrections.

ABSTRACT

Error correction techniques have been used to refine the output sentences from automatic speech recognition (ASR) models and achieve a lower word error rate (WER) than original ASR outputs. Previous works usually use a sequence-to-sequence model to correct an ASR output sentence autoregressively, which causes large latency and cannot be deployed in online ASR services. A straightforward solution to reduce latency, inspired by non-autoregressive (NAR) neural machine translation, is to use an NAR sequence generation model for ASR error correction, which, however, comes at the cost of significantly increased ASR error rate. In this paper, observing distinctive error patterns and correction operations (i.e., insertion, deletion, and substitution) in ASR, we propose FastCorrect, a novel NAR error correction model based on edit alignment. In training, FastCorrect aligns each source token from an ASR output sentence to the target tokens from the corresponding ground-truth sentence based on the edit distance between the source and target sentences, and extracts the number of target tokens corresponding to each source token during edition/correction, which is then used to train a length predictor and to adjust the source tokens to match the length of the target sentence for parallel generation. In inference, the token number predicted by the length predictor is used to adjust the source tokens for target sequence generation. Experiments on the public AISHELL-1 dataset and an internal industrial-scale ASR dataset show the effectiveness of FastCorrect for ASR error correction: 1) it speeds up the inference by 6-9 times and maintains the accuracy (8-14% WER reduction) compared with the autoregressive correction model; and 2) it outperforms the popular NAR models adopted in neural machine translation and text edition by a large margin.

Motivation & Objective

• Motivate fast ASR error correction with low latency for online services.
• Leverage edit-distance based alignment to guide non-autoregressive correction.
• Pre-train on large pseudo correction data and fine-tune on real ASR correction data.
• Demonstrate speedups and accuracy gains on AISHELL-1 and a large internal Mandarin ASR dataset.

Proposed method

• Introduce edit alignment between ASR outputs and ground-truth corrections based on minimum edit distance and an n-gram frequency-based selection of the best alignment.
• Use a Transformer-based NAR encoder-decoder with a length predictor that estimates, for each source token, how many target tokens align to it (0 for deletion, 1 for substitution/unchanged, >1 for insertion).
• Train the length predictor with MSE loss and use it to align source tokens for parallel target sequence generation.
• Pre-train FastCorrect on a large pseudo correction dataset created by editing text using a homophone-informed noising process, then fine-tune on real ASR-correction data.
• Compare against autoregressive (AR) correction and other NAR models (LevT, FELIX) in terms of WER reduction and latency across GPUs and CPUs.

Experimental results

Research questions

• RQ1 Can a non-autoregressive error-correction model guided by edit alignment achieve comparable WER reductions to AR models while substantially reducing latency?
• RQ2 Does leveraging insertion/deletion/substitution patterns via edit alignment improve correction quality over existing NAR approaches in ASR error correction?
• RQ3 How does pre-training on pseudo correction data affect performance when fine-tuning on limited ASR correction data?
• RQ4 What are the latency and accuracy trade-offs of FastCorrect on public and industrial-scale Mandarin ASR datasets?

Key findings

• FastCorrect yields 6-9x inference speedups over AR correction on AISHELL-1 and the internal dataset.
• FastCorrect achieves 8-14% WER reduction (WERR) versus no correction, close to AR model performance on both datasets.
• FastCorrect outperforms LevT and FELIX by a wide margin in WERR and error-correction quality.
• Ablation shows edit alignment (length predictor) and pre-training are critical for achieving strong WER reductions.
• Compared with AR models with deep encoders and shallow decoders, FastCorrect offers similar or better accuracy with substantially lower latency.
• Table 1 shows FastCorrect achieving 4.16 WER (AISHELL-1, Test) and 10.27 WER (Internal, Test) with notable latency reductions; Table 4 reports higher P_right and comparable P_edit/R_edit relative to baselines.