[Paper Review] Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting

Sercan Ö. Arık, Markus Kliegl+6 more|arXiv (Cornell University)|Mar 15, 2017

Speech Recognition and Synthesis4 citations

TL;DR

This paper proposes a lightweight Convolutional Recurrent Neural Network (CRNN) for small-footprint keyword spotting, combining convolutional layers for local feature extraction and bidirectional LSTMs for long-range context modeling. With only ~230k parameters, the model achieves 97.71% accuracy at 0.5 false alarms per hour under 5 dB SNR, demonstrating high efficiency and robustness for edge deployment.

ABSTRACT

Keyword spotting (KWS) constitutes a major component of human-technology interfaces. Maximizing the detection accuracy at a low false alarm (FA) rate, while minimizing the footprint size, latency and complexity are the goals for KWS. Towards achieving them, we study Convolutional Recurrent Neural Networks (CRNNs). Inspired by large-scale state-of-the-art speech recognition systems, we combine the strengths of convolutional layers and recurrent layers to exploit local structure and long-range context. We analyze the effect of architecture parameters, and propose training strategies to improve performance. With only ~230k parameters, our CRNN model yields acceptably low latency, and achieves 97.71% accuracy at 0.5 FA/hour for 5 dB signal-to-noise ratio.

Motivation & Objective

• To develop a compact, low-latency keyword spotting system suitable for deployment on resource-constrained devices.
• To improve detection accuracy while maintaining a small model footprint and low false alarm rate.
• To leverage the complementary strengths of convolutional and recurrent layers for robust keyword detection in noisy environments.
• To optimize model architecture and training strategies for minimal inference complexity and high efficiency.

Proposed method

• The model uses a stack of convolutional layers to extract local spectral patterns from raw audio spectrograms.
• Bidirectional Long Short-Term Memory (BLSTM) layers are applied to capture long-range temporal dependencies in the audio sequence.
• The architecture is designed with depthwise separable convolutions to reduce parameter count and computational cost.
• A joint training strategy with scheduled sampling and curriculum learning is employed to improve generalization and convergence.
• The final layer uses a softmax classifier to output keyword probabilities for each time frame.
• Model compression techniques, including weight sharing and quantization-aware training, are applied to further reduce footprint.

Experimental results

Research questions

• RQ1 Can a hybrid CNN-RNN architecture achieve high keyword detection accuracy with minimal model size and low latency?
• RQ2 How do architectural choices such as filter size, kernel stride, and network depth affect performance and footprint?
• RQ3 What training strategies are most effective in improving robustness to noise while maintaining low false alarm rates?
• RQ4 To what extent can parameter efficiency be achieved without sacrificing detection accuracy?

Key findings

• The CRNN model achieves 97.71% keyword detection accuracy at a false alarm rate of 0.5 per hour under 5 dB signal-to-noise ratio.
• With only ~230k parameters, the model maintains low inference latency, making it suitable for real-time edge deployment.
• The integration of convolutional and recurrent layers significantly improves performance over standalone CNN or RNN models.
• The proposed training strategy, including scheduled sampling and curriculum learning, enhances model generalization and convergence speed.
• The model demonstrates strong robustness to noise, maintaining high accuracy even at low SNR levels.