[Paper Review] MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognition

Somshubra Majumdar, Boris Ginsburg|arXiv (Cornell University)|Apr 18, 2020

Speech Recognition and Synthesis33 references82 citations

TL;DR

MatchboxNet is a compact, end-to-end 1D convolutional neural network for speech command recognition that uses time-channel separable convolutions to achieve state-of-the-art accuracy on the Google Speech Commands dataset with significantly fewer parameters than comparable models. It enables high model efficiency and scalability while improving robustness through intensive data augmentation with background noise and speech.

ABSTRACT

We present an MatchboxNet - an end-to-end neural network for speech command recognition. MatchboxNet is a deep residual network composed from blocks of 1D time-channel separable convolution, batch-normalization, ReLU and dropout layers. MatchboxNet reaches state-of-the-art accuracy on the Google Speech Commands dataset while having significantly fewer parameters than similar models. The small footprint of MatchboxNet makes it an attractive candidate for devices with limited computational resources. The model is highly scalable, so model accuracy can be improved with modest additional memory and compute. Finally, we show how intensive data augmentation using an auxiliary noise dataset improves robustness in the presence of background noise.

Motivation & Objective

• To develop a compact, efficient deep learning model for keyword spotting on low-resource edge devices.
• To reduce model parameters without sacrificing accuracy compared to existing speech command recognition models.
• To improve robustness to background noise and speech through advanced data augmentation techniques.
• To demonstrate scalability of the architecture across different model sizes and hardware constraints.

Proposed method

• The model uses 1D time-channel separable convolutions to reduce parameter count while maintaining performance.
• It is structured as a deep residual network with residual blocks, each containing multiple sub-blocks with batch normalization, ReLU, and dropout.
• The architecture includes a prologue and epilogue sub-block for feature transformation before and after the main residual blocks.
• Extensive data augmentation is applied, including time shifts, white noise, SpecAugment, SpecCutout, and additional background noise from the Freesound database.
• The model is trained using the NovoGrad optimizer with a warmup-hold-decay learning rate schedule and mixed-precision training on V-100 GPUs.
• Robustness is enhanced by retraining on a dataset augmented with real-world environmental noise and background speech.

Experimental results

Research questions

• RQ1 Can a 1D time-channel separable convolutional architecture achieve state-of-the-art accuracy in speech command recognition with fewer parameters than existing models?
• RQ2 How does model scalability affect performance when increasing depth (B×R) or width (C) in MatchboxNet?
• RQ3 To what extent does training with auxiliary background noise improve robustness to real-world noise conditions?
• RQ4 Can the model maintain high accuracy while being highly compact for deployment on edge devices?
• RQ5 How does the model perform under varying signal-to-noise ratios (SNR) in noisy environments?

Key findings

• MatchboxNet-3x2x64 achieves 97.48% accuracy on Google Speech Commands v1 with only 93K parameters, outperforming models like DenseNet-BC-100 (800K parameters) and EdgeSpeechNet-A (107K parameters).
• On v2, MatchboxNet-3x2x64 reaches 97.21% accuracy with 93K parameters, approaching the performance of larger models like 'Embedding + Head' (385K parameters).
• Scaling the model to MatchboxNet-6x2x64 increases accuracy to 97.55% on v2, demonstrating strong scalability with minimal parameter growth.
• The model trained with additional background noise and speech augmentation achieves 97.05% accuracy on clean test data, showing no degradation from basic augmentation.
• Under noisy conditions (SNR from -10 dB to 50 dB), MatchboxNet-6x2x64 maintains 97.33% accuracy at 50 dB SNR, outperforming the smaller 3x1x64 variant across all SNR levels.
• The model trained with background speech and noise classes achieves 96.97% accuracy on the expanded v2 dataset, confirming improved robustness in real-world settings.