[Paper Review] Interpretable Convolutional Filters with SincNet

Mirco Ravanelli, Yoshua Bengio|arXiv (Cornell University)|Nov 23, 2018

Time Series Analysis and Forecasting67 references93 citations

TL;DR

SincNet introduces a CNN with parametrized sinc-based first-layer filters to learn a compact, interpretable, and task-tuned filter-bank directly from raw waveform, showing faster convergence and improved performance for speaker and speech recognition versus standard CNNs.

ABSTRACT

Deep learning is currently playing a crucial role toward higher levels of artificial intelligence. This paradigm allows neural networks to learn complex and abstract representations, that are progressively obtained by combining simpler ones. Nevertheless, the internal "black-box" representations automatically discovered by current neural architectures often suffer from a lack of interpretability, making of primary interest the study of explainable machine learning techniques. This paper summarizes our recent efforts to develop a more interpretable neural model for directly processing speech from the raw waveform. In particular, we propose SincNet, a novel Convolutional Neural Network (CNN) that encourages the first layer to discover more meaningful filters by exploiting parametrized sinc functions. In contrast to standard CNNs, which learn all the elements of each filter, only low and high cutoff frequencies of band-pass filters are directly learned from data. This inductive bias offers a very compact way to derive a customized filter-bank front-end, that only depends on some parameters with a clear physical meaning. Our experiments, conducted on both speaker and speech recognition, show that the proposed architecture converges faster, performs better, and is more interpretable than standard CNNs.

Motivation & Objective

• Motivate interpretability in waveform-based speech models and reduce reliance on hand-crafted features.
• Propose a first-layer convolution that uses parametrized sinc functions to implement learnable band-pass filters.
• Demonstrate faster convergence and improved accuracy over standard CNNs on speaker and speech recognition tasks.
• Show that the learned filter bank is more interpretable and task-tuned to speech characteristics.

Proposed method

• Define first-layer g[n, f1, f2] as a band-pass filter built from sinc-based differences of low/high cutoffs ( Equations 3-4 ).
• Parameterize filters with only two cutoffs per filter (f1, f2) and apply a symmetric, windowed (Hamming) design to control ripple and attenuation (Equations 5-8).
• Constrain filters to be symmetric to avoid phase distortion and enable efficient computation.
• Train all parameters end-to-end with SGD within a standard CNN pipeline (pooling, normalization, activation, dropout) followed by task-specific layers.
• Initialize cutoffs randomly in [0, fs/2] or with mel-scale-inspired values to allocate more filters in lower frequencies.
• Evaluate on speaker and speech recognition tasks (TIMIT, Librispeech, DIRHA) under clean and noisy conditions.

Experimental results

Research questions

• RQ1 Can a sinc-based parameterization of the first CNN layer yield more interpretable, task-relevant filters than standard learnable FIRs?
• RQ2 Does SincNet achieve faster convergence and better generalization in speaker and speech recognition with limited training data?
• RQ3 How does the learned filter-bank adapt to noise and formants compared to traditional CNNs?
• RQ4 What are the practical benefits in parameter efficiency and interpretability when using sinc-based filters?
• RQ5 Is SincNet robust to distant-talk/noisy environments as demonstrated on the DIRHA dataset?

Key findings

• SincNet learns a more interpretable filter bank with frequency responses that resemble meaningful speech bands (e.g., formants, pitch regions).
• SincNet converges faster than standard CNNs and achieves lower frame error rates on speaker-id tasks (e.g., TIMIT) and competitive performance on LibriSpeech.
• SincNet uses far fewer parameters in the first layer (2F vs F×L) and remains efficient as filter length L grows.
• In noisy conditions, SincNet quickly avoids corrupted bands (e.g., 2.0–2.5 kHz) and concentrates on informative bands earlier in training.
• On speaker verification (Librispeech), SincNet achieves lower Equal Error Rate than CNN-based baselines, indicating better discriminative power with compact front-ends.
• In speech recognition (TIMIT, DIRHA), SincNet-Raw outperforms CNN-FBANK and CNN-Raw baselines, including in distant-talking scenarios.