[Paper Review] Small-footprint Keyword Spotting with Graph Convolutional Network

Xi Chen, Shouyi Yin+4 more|arXiv (Cornell University)|Dec 11, 2019

Text and Document Classification Technologies24 references7 citations

TL;DR

This paper proposes a compact, context-aware keyword spotting model using a graph convolutional network (GCN) to capture long-range dependencies in spectrogram features. By integrating GCN into a residual bottleneck architecture (CENet), the method achieves state-of-the-art accuracy on the Google Speech Commands Dataset with lower computational cost and smaller model footprint than prior approaches.

ABSTRACT

Despite the recent successes of deep neural networks, it remains challenging to achieve high precision keyword spotting task (KWS) on resource-constrained devices. In this study, we propose a novel context-aware and compact architecture for keyword spotting task. Based on residual connection and bottleneck structure, we design a compact and efficient network for KWS task. To leverage the long range dependencies and global context of the convolutional feature maps, the graph convolutional network is introduced to encode the non-local relations. By evaluated on the Google Speech Command Dataset, the proposed method achieves state-of-the-art performance and outperforms the prior works by a large margin with lower computational cost.

Motivation & Objective

• To address the challenge of achieving high-accuracy keyword spotting on resource-constrained devices with low latency and small model size.
• To improve long-range contextual modeling in convolutional feature maps, which standard CNNs and RNNs fail to capture effectively.
• To design a compact, efficient neural architecture suitable for on-device deployment while maintaining high detection accuracy.
• To explore the integration of graph convolutional networks into keyword spotting for global feature context encoding.
• To evaluate the effectiveness of GCN placement across different residual blocks in the network.

Proposed method

• Proposes a compact convolutional network (CENet) based on residual connections and bottleneck modules to reduce model complexity and parameter count.
• Introduces a graph convolutional network (GCN) module with an attention mechanism to model non-local, long-range dependencies in feature maps by aggregating features across all positions with similarity-based weighting.
• Applies the GCN module at multiple residual stages (Stage-1, -2, -3) to enrich feature representations with multi-level contextual information.
• Uses a message-passing mechanism in the GCN to update each feature map position by aggregating weighted features from all other positions based on their similarity.
• Employs mel-frequency cepstral coefficients (MFCC) and filterbank (fbank) features as input, with fbank showing consistent 0–1.0% accuracy gains.
• Trains the model end-to-end on the Google Speech Commands Dataset and evaluates performance using accuracy, false alarm rate (FAR), and false reject rate (FRR).

Experimental results

Research questions

• RQ1 Can a compact, lightweight neural network architecture achieve state-of-the-art keyword spotting performance on resource-constrained devices?
• RQ2 Does incorporating graph convolutional networks improve long-range context modeling in spectrogram features for keyword spotting?
• RQ3 Where in the network architecture is the GCN module most effective for performance gain—early, mid, or late layers?
• RQ4 How does the GCN-based approach compare to prior methods in terms of accuracy, parameter count, and computational cost?
• RQ5 Does using fbank features instead of MFCC improve the performance of the proposed model?

Key findings

• The proposed CENet-GCN-6 model achieves 95.2% accuracy on the Google Speech Commands Dataset, a 1.3% absolute improvement over the baseline CENet-6 (93.9%) and a 1.0% gain over prior state-of-the-art methods.
• Inserting the GCN module into Stage-2 yields the highest single-stage improvement, increasing accuracy by 1.1%.
• Adding GCN to all three residual stages (1, 2, and 3) results in the best performance (95.2%), demonstrating the benefit of multi-level contextual feature enhancement.
• The GCN module increases model parameters by only 11.4K (from 16.2K to 27.6K) and computational cost by 32% (from 1.95M to 2.55M FLOPs), showing high efficiency.
• The model with fbank input outperforms MFCC input by 0–1.0% across all variants, indicating that fbank features better preserve spectral-temporal correlations.
• Visualization and ROC curve analysis confirm that the GCN enhances discriminative regions in feature maps and reduces both false alarm and false reject rates, especially at lower thresholds.