[Paper Review] Compression of Acoustic Event Detection Models with Low-rank Matrix Factorization and Quantization Training

Bowen Shi, Ming Sun+4 more|arXiv (Cornell University)|May 2, 2019

Music and Audio Processing15 references11 citations

TL;DR

This paper proposes a hybrid model compression technique combining low-rank matrix factorization via SVD and quantization training to drastically reduce the size of LSTM-based acoustic event detection (AED) models with minimal accuracy loss. By applying SVD to compress recurrent weight matrices and fine-tuning with quantization training, the method reduces model size to just 1% of the original while maintaining near-state-of-the-art performance on AED tasks.

ABSTRACT

In this paper, we present a compression approach based on the combination of low-rank matrix factorization and quantization training, to reduce complexity for neural network based acoustic event detection (AED) models. Our experimental results show this combined compression approach is very effective. For a three-layer long short-term memory (LSTM) based AED model, the original model size can be reduced to 1% with negligible loss of accuracy. Our approach enables the feasibility of deploying AED for resource-constraint applications.

Motivation & Objective

• Address the challenge of deploying large, computation-intensive deep learning models for acoustic event detection (AED) in resource-constrained industrial environments.
• Reduce model size and inference cost without significant accuracy degradation for LSTM-based AED systems.
• Investigate the effectiveness of combining low-rank matrix factorization and quantization training as a joint compression strategy.
• Demonstrate feasibility of deploying compressed AED models on edge devices with limited memory and processing power.

Proposed method

• Apply low-rank matrix factorization to the recurrent weight matrices ($W_h^l$) of a three-layer LSTM using truncated singular value decomposition (SVD), retaining only the top-$r$ singular values and vectors.
• Share the right singular vectors ($V_{h}^{lr}$) from the SVD of $W_h^l$ with the input weight matrix $W_x^l$, approximating it as $Z_x^l \tilde{V}_{h}^{lr^T}$ to reduce parameter count.
• Implement post-training quantization using a symmetric quantization function $Q_n(\mathbf{V}) = \alpha \hat{Q}_n(\hat{\mathbf{V}}) + \beta$, where $\hat{\mathbf{V}} = (\mathbf{V} - \beta)/\alpha$, with $n$-bit integer representation.
• Use the straight-through estimator to backpropagate gradients through the non-differentiable quantization function during fine-tuning.
• Perform quantization training (QT) by fine-tuning the model after quantization, rather than applying quantization only at inference (post-mortem, PM).
• Apply quantization to the factorized matrices $Z_h^l$, $Z_x^l$, and $\tilde{V}_{h}^{lr}$, and also quantize the network inputs to further compress the model.

Experimental results

Research questions

• RQ1 Can low-rank matrix factorization combined with quantization training effectively compress LSTM-based AED models while preserving detection accuracy?
• RQ2 How does the choice of singular value retention ratio ($\tau$) affect model accuracy and compression ratio in low-rank factorized AED models?
• RQ3 Does quantization training (QT) significantly outperform post-mortem quantization (PM) in maintaining model performance after compression?
• RQ4 What is the minimal model size achievable with acceptable accuracy degradation when combining low-rank factorization and quantization?
• RQ5 How do different bit-widths (e.g., 4-bit vs. 8-bit) impact the performance of quantized AED models, especially when fine-tuned with quantization training?

Key findings

• Applying low-rank matrix factorization with $\tau = 0.6$ reduces the model size to 0.316 MB (6% of original) with no significant accuracy degradation, maintaining AUC and EER comparable to the full-precision baseline.
• When combined with 8-bit quantization training (QT), the model size is reduced to 0.079 MB (1.5% of original size), achieving an average AUC of 6.33% and EER of 13.26%, showing only a 0.2% AUC increase and 1.9% EER improvement over the baseline.
• For 4-bit quantization, the 8-bit QT model outperforms the 4-bit PM model significantly, with a 7.43% EER (vs. 19.24% for PM), demonstrating the necessity of fine-tuning after quantization.
• The 4-bit quantized model with QT achieves a model size of only 0.040 MB (0.76% of original), with average AUC of 8.17% and EER of 14.94%, indicating strong compression efficiency with moderate accuracy loss.
• Post-mortem quantization (PM) leads to significant performance degradation in low-rank models, especially at 4-bit precision (EER of 23.48% on average), highlighting the importance of fine-tuning after quantization.
• The combined approach achieves a 98% reduction in model size (to 1% of original) with negligible accuracy loss, making it feasible for deployment on edge devices with limited memory and compute resources.