[Paper Review] Efficient Convolutional Neural Network For Audio Event Detection

Matthias Meyer, Lukas Cavigelli+1 more|arXiv (Cornell University)|Sep 28, 2017

Music and Audio Processing13 references11 citations

TL;DR

This paper proposes a highly efficient convolutional neural network (CNN) for audio event detection that reduces memory usage by 515× and computational operations by 2.1× compared to state-of-the-art models, while improving accuracy by 9.2%. The method uses a structured, depthwise-separable CNN architecture with no fully connected layers, enabling real-time inference on low-power embedded systems such as ARM Cortex-M7 devices.

ABSTRACT

Wireless distributed systems as used in sensor networks, Internet-of-Things and cyber-physical systems, impose high requirements on resource efficiency. Advanced preprocessing and classification of data at the network edge can help to decrease the communication demand and to reduce the amount of data to be processed centrally. In the area of distributed acoustic sensing, the combination of algorithms with a high classification rate and resource-constraint embedded systems is essential. Unfortunately, algorithms for acoustic event detection have a high memory and computational demand and are not suited for execution at the network edge. This paper addresses these aspects by applying structural optimizations to a convolutional neural network for audio event detection to reduce the memory requirement by a factor of more than 500 and the computational effort by a factor of 2.1 while performing 9.2% better.

Motivation & Objective

• To address the high memory and computational demands of state-of-the-art CNNs for audio event detection, which hinder deployment on low-power embedded systems.
• To develop a resource-efficient CNN architecture that maintains high classification accuracy while drastically reducing parameter count and MAC operations.
• To enable real-time audio event detection directly on edge devices such as sensor nodes and microcontrollers with limited memory and processing power.
• To design a network structure compatible with hardware accelerators for improved energy efficiency in edge computing environments.

Proposed method

• The model uses a front-end based on the Short-Time Fourier Transform (STFT) to convert raw audio into a time-frequency representation.
• A depthwise-separable convolutional architecture is employed to reduce parameter count and computational complexity while preserving feature extraction capability.
• Fully connected layers are replaced with global average pooling to minimize model size and parameter count.
• The network is trained using the ADAM optimizer with a cross-entropy loss function on a 4-second windowed input from the ESC-50 dataset.
• The architecture is optimized for hardware accelerators by ensuring a regular, structured computation pattern that matches accelerator memory access and computation patterns.
• No data augmentation is used in the main experiments to isolate the impact of architectural design on performance and efficiency.

Experimental results

Research questions

• RQ1 Can a CNN-based audio event detection model be made significantly more efficient in terms of memory and computation without sacrificing accuracy?
• RQ2 To what extent can architectural modifications such as depthwise separable convolutions and global average pooling reduce model size and inference cost?
• RQ3 Can a highly efficient CNN be deployed in real-time on low-power embedded microcontrollers with limited memory and processing power?
• RQ4 Does the proposed architecture maintain or improve classification accuracy compared to state-of-the-art models when trained without complex data augmentation?

Key findings

• The proposed CNN-C and CNN-CNP models achieve 86.0% and 85.1% accuracy, respectively, without data augmentation—surpassing the 77.9% and 80.3% accuracy of reference models A and B under the same conditions.
• The memory requirement is reduced by a factor of 515, from 466 MB in the reference model to approximately 904 kB in the proposed models.
• The number of multiply-accumulate operations (MACs) is reduced by a factor of 2.1, from 3533 M in model B to 1239 M in CNN-CNP.
• The proposed model achieves a 9.2% improvement in accuracy over the reference model A when both are evaluated without data augmentation.
• The model is deployable on low-power devices such as the NXP KV5x and ST STM32F7, which have up to 2 MB of flash memory and processing capabilities exceeding 430 M MAC/s, enabling real-time inference.
• The structured, convolutional-only design is well-suited for hardware accelerators, enabling further improvements in energy efficiency and performance.