[Paper Review] Quantization of Acoustic Model Parameters in Automatic Speech Recognition Framework

Amrutha Prasad, Petr Motlíček+1 more|arXiv (Cornell University)|Jun 16, 2020

Speech Recognition and Synthesis4 citations

TL;DR

This paper investigates quantization of acoustic model parameters in Kaldi-based DNN-HMM ASR systems, comparing post-training quantization and quantization-aware training (QAT) in PyTorch and Kaldi. It shows that 8-bit integer quantization reduces model size by 75% with minimal WER degradation (≤2.7% absolute) when using symmetric quantization and proper calibration, while QAT improves performance over post-training methods, especially for deeper TDNN-F architectures on Librispeech data.

ABSTRACT

State-of-the-art hybrid automatic speech recognition (ASR) system exploits deep neural network (DNN) based acoustic models (AM) trained with Lattice Free-Maximum Mutual Information (LF-MMI) criterion and n-gram language models. The AMs typically have millions of parameters and require significant parameter reduction to operate on embedded devices. The impact of parameter quantization on the overall word recognition performance is studied in this paper. Following approaches are presented: (i) AM trained in Kaldi framework with conventional factorized TDNN (TDNN-F) architecture, (ii) the TDNN AM built in Kaldi loaded into the PyTorch toolkit using a C++ wrapper for post-training quantization, (iii) quantization-aware training in PyTorch for Kaldi TDNN model, (iv) quantization-aware training in Kaldi. Results obtained on standard Librispeech setup provide an interesting overview of recognition accuracy w.r.t. applied quantization scheme.

Motivation & Objective

• To evaluate the impact of parameter quantization on ASR performance in end-to-end DNN-based systems.
• To bridge the gap between Kaldi’s state-of-the-art ASR training and PyTorch’s advanced quantization tools.
• To compare post-training quantization and quantization-aware training (QAT) for TDNN and TDNN-F models.
• To assess model size reduction and recognition accuracy trade-offs when quantizing from float32 to int8 or int16.
• To provide open-source code for Kaldi-embedded quantization in PyTorch for reproducibility and reuse.

Proposed method

• Trained a TDNN and TDNN-F acoustic model using Kaldi with the LF-MMI criterion on 960 hours of Librispeech data.
• Exported the trained Kaldi models to PyTorch via a C++ wrapper to enable integration with PyTorch’s quantization tools.
• Applied post-training quantization with weight-only, custom implementation (weights and activations to int8), and PyTorch’s native quantization functions.
• Implemented quantization-aware training (QAT) in both PyTorch and Kaldi to minimize quantization error during fine-tuning.
• Used symmetric quantization for 8-bit and 16-bit integer representations, with dynamic range calibration based on activation statistics.
• Evaluated all models on the Librispeech test-clean set using word error rate (WER) as the primary metric.

Experimental results

Research questions

• RQ1 How does 8-bit and 16-bit integer quantization affect word error rate (WER) in Kaldi-trained TDNN and TDNN-F acoustic models?
• RQ2 Can post-training quantization with PyTorch achieve comparable performance to full-precision models when applied to Kaldi-trained models?
• RQ3 Does quantization-aware training (QAT) in PyTorch or Kaldi reduce quantization error and improve recognition accuracy compared to post-training quantization?
• RQ4 What is the trade-off between model size reduction and WER degradation when quantizing from float32 to int8?
• RQ5 How does the performance of quantized TDNN-F models compare to baseline TDNN models in terms of size and accuracy?

Key findings

• Post-training quantization to 8-bit integers reduced model size by 75% with a WER increase of up to 2.7% absolute compared to the float32 baseline.
• Custom implementation of quantization (weights and activations both to int8 with same range) achieved 75% size reduction with no performance loss on monophone and biphone models.
• Using PyTorch’s native quantization functions led to a 2% absolute WER increase (from 8.4% to 10.4%) on the monophone model, but no degradation on the biphone model.
• Quantization-aware training (QAT) in PyTorch reduced WER by 0.6% absolute compared to post-training quantization, improving performance over the baseline.
• TDNN-F models achieved better performance than standard TDNNs, with a 60% model size reduction and a 2.7% absolute WER increase compared to the baseline TDNN.
• The Kaldi-based QAT method reduced quantization error and improved performance by 0.6% absolute compared to weight-only quantization, with no need to switch between frameworks.