[Paper Review] Role of non-linear data processing on speech recognition task in the framework of reservoir computing

Flavio Abreu Araujo, Mathieu Riou+9 more|arXiv (Cornell University)|Jan 1, 2020

Neural Networks and Reservoir Computing36 references88 citations

TL;DR

This paper investigates how non-linear acoustic preprocessing affects speech recognition performance in reservoir computing hardware, showing that non-linearity in feature extraction—not the reservoir itself—is primarily responsible for high recognition rates. Using magnetic nano-oscillators as a neuromorphic hardware platform, the study quantifies the reservoir's gain over preprocessing alone, revealing that linear spectrograms yield the highest performance improvement from hardware, while non-linear filters like MFCC and cochleagram already achieve near-optimal results without it.

ABSTRACT

The reservoir computing neural network architecture is widely used to test hardware systems for neuromorphic computing. One of the preferred tasks for bench-marking such devices is automatic speech recognition. This task requires acoustic transformations from sound waveforms with varying amplitudes to frequency domain maps that can be seen as feature extraction techniques. Depending on the conversion method, these transformations sometimes obscure the contribution of the neuromorphic hardware to the overall speech recognition performance. Here, we quantify and separate the contributions of the acoustic transformations and the neuromorphic hardware to the speech recognition success rate. We show that the non-linearity in the acoustic transformation plays a critical role in feature extraction. We compute the gain in word success rate provided by a reservoir computing device compared to the acoustic transformation only, and show that it is an appropriate bench-mark for comparing different hardware. Finally, we experimentally and numerically quantify the impact of the different acoustic transformations for neuromorphic hardware based on magnetic nano-oscillators.

Motivation & Objective

• To isolate and quantify the contributions of acoustic preprocessing and neuromorphic hardware to speech recognition performance in reservoir computing.
• To evaluate how different non-linear frequency-domain transformations affect recognition success rates independently of the reservoir.
• To benchmark neuromorphic hardware using magnetic nano-oscillators by comparing performance gains across various preprocessing methods.
• To provide a standardized, interpretable benchmark for neuromorphic hardware using linear spectrograms, which avoid prior feature separation.

Proposed method

• Used four frequency-domain preprocessing methods: cochleagram, MFCC, linear spectrogram (ℜ(Spectro)), and Spectro HP (|sin^p|ℜ(Spectro)|| − |cos^p|ℑ(Spectro)||).
• Performed spoken digit recognition on TI-46 and AURORA-2 datasets using linear classifiers on preprocessed features to assess standalone performance.
• Simulated a reservoir composed of 2000 non-linear magnetic nodes (STNOs) to evaluate hardware contribution via linear regression on reservoir states.
• Compared recognition rates with and without the reservoir to compute the gain attributable solely to the neuromorphic hardware.
• Validated simulations with experimental results from magnetic nano-oscillators, showing excellent agreement.
• Used mixed training conditions (clean and noisy digits) and tested on clean and noisy subsets to evaluate robustness.

Experimental results

Research questions

• RQ1 How do different non-linear acoustic preprocessing methods affect speech recognition performance when used without a reservoir?
• RQ2 What is the relative contribution of the reservoir computing hardware versus the acoustic preprocessing to overall recognition accuracy?
• RQ3 Does the choice of preprocessing method affect the detectable performance gain from neuromorphic hardware?
• RQ4 Can a simple, non-biological filter like Spectro HP achieve state-of-the-art performance without relying on cochleagram or MFCC models?
• RQ5 How does noise impact the performance gain of the reservoir when combined with different preprocessing techniques?

Key findings

• The cochleagram achieved 95.8% word recognition rate as a standalone feature extractor, while MFCC reached 77.2% and Spectro HP reached 89.0%, indicating strong non-linear feature extraction.
• The linear spectrogram performed poorly at 10% recognition rate, but applying non-linearity (Spectro HP) increased it to 88%, proving non-linearity is the key factor.
• The reservoir provided a 50.70% gain in recognition rate over the MFCC filter (from 42.26% to 92.96%) on clean digits, while only a 25.90% gain on cochleagram (from 63.24% to 89.14%).
• On noisy AURORA-2 data, the MFCC filter yielded a 48.79% gain (from 68.82% to 81.20%), while cochleagram yielded only 23.02% gain, showing the reservoir adds more value when preprocessing is less effective.
• The experimental magnetic nano-oscillator matched simulations closely, validating the model’s accuracy for hardware benchmarking.
• The study concludes that linear spectrograms provide the cleanest benchmark for hardware evaluation, as they do not pre-separate classes, making the reservoir’s contribution unambiguous.