[Paper Review] Hello Edge: Keyword Spotting on Microcontrollers

Yundong Zhang, Naveen Suda+2 more|arXiv (Cornell University)|Nov 20, 2017

Speech and Audio Processing25 references340 citations

TL;DR

The paper evaluates multiple neural network architectures for keyword spotting on microcontrollers, shows depthwise separable CNN (DS-CNN) offers the best accuracy within MCU constraints, and demonstrates effective 8-bit quantization for deployment.

ABSTRACT

Keyword spotting (KWS) is a critical component for enabling speech based user interactions on smart devices. It requires real-time response and high accuracy for good user experience. Recently, neural networks have become an attractive choice for KWS architecture because of their superior accuracy compared to traditional speech processing algorithms. Due to its always-on nature, KWS application has highly constrained power budget and typically runs on tiny microcontrollers with limited memory and compute capability. The design of neural network architecture for KWS must consider these constraints. In this work, we perform neural network architecture evaluation and exploration for running KWS on resource-constrained microcontrollers. We train various neural network architectures for keyword spotting published in literature to compare their accuracy and memory/compute requirements. We show that it is possible to optimize these neural network architectures to fit within the memory and compute constraints of microcontrollers without sacrificing accuracy. We further explore the depthwise separable convolutional neural network (DS-CNN) and compare it against other neural network architectures. DS-CNN achieves an accuracy of 95.4%, which is ~10% higher than the DNN model with similar number of parameters.

Motivation & Objective

• Assess various neural network architectures for on-device keyword spotting under MCU memory and compute constraints.
• Compare accuracy, memory footprint, and operations per inference across architectures.
• Develop a resource-constrained neural network search to identify high-accuracy models within MCU limits.
• Propose and evaluate a depthwise separable CNN architecture inspired by MobileNet for KWS on MCUs.
• Demonstrate practical deployment and quantization effects on real MCU hardware.

Proposed method

• Train and compare literature-based KWS models (DNN, CNN, LSTM, CRNN) on Google Speech Commands dataset under fixed 8-bit weight/activation assumption.
• Introduce and evaluate a DS-CNN model based on depthwise separable convolutions inspired by MobileNet.
• Perform a resource-constrained architecture exploration by mapping models to three MCU memory/compute budgets (small, medium, large).
• Quantize representative models to 8-bit fixed-point weights/activations and assess accuracy loss.
• Deploy a DNN model with 8-bit quantization on a Cortex-M7 MCU using CMSIS-NN to validate real-time performance.

Experimental results

Research questions

• RQ1 What are the accuracy, memory footprint, and computation requirements of popular KWS models when constrained to MCU resources?
• RQ2 Can a depthwise separable CNN (DS-CNN) outperform prior architectures within fixed MCU budgets?
• RQ3 How does 8-bit quantization affect KWS model accuracy and deployability on microcontrollers?
• RQ4 What are the scalability and trade-offs of DS-CNN models under progressively tighter memory/compute budgets?

Key findings

• DS-CNN achieves the best accuracies within MCU constraints, with 94.4%, 94.9%, and 95.4% under small, medium, and large budgets respectively.
• Quantized 8-bit models retain or marginally improve accuracy compared to full-precision counterparts, enabling efficient MCU deployment.
• A Cortex-M7 deployment using 8-bit DS-CNN/ DNN achieves 10 inferences per second with about 12 ms per inference and ~70 KB total KWS footprint, validating real-time on-device performance.
• DS-CNN offers strong scalability, outperforming other architectures (DNN, CNN, LSTM, CRNN) across memory and compute budgets.
• DS-CNN models scaled for memory as low as 8 KB still outperform DNNs with similar operation counts, illustrating suitability for ultra-resource-constrained MCUs.