[Paper Review] Between Homomorphic Signal Processing and Deep Neural Networks: Constructing Deep Algorithms for Polyphonic Music Transcription

Li Su|arXiv (Cornell University)|Jun 26, 2017

Music and Audio Processing45 references3 citations

TL;DR

This paper proposes a novel deep algorithm for polyphonic music transcription by unifying homomorphic signal processing and deep neural networks (DNNs), demonstrating that the generalized cepstrum of spectrum (GCoS) — a three-layer DFT-based feature with nonlinear activation — outperforms traditional methods in detecting missing fundamentals and enhancing noise robustness, particularly in low-SNR conditions.

ABSTRACT

This paper presents a new approach in understanding how deep neural networks (DNNs) work by applying homomorphic signal processing techniques. Focusing on the task of multi-pitch estimation (MPE), this paper demonstrates the equivalence relation between a generalized cepstrum and a DNN in terms of their structures and functionality. Such an equivalence relation, together with pitch perception theories and the recently established rectified-correlations-on-a-sphere (RECOS) filter analysis, provide an alternative way in explaining the role of the nonlinear activation function and the multi-layer structure, both of which exist in a cepstrum and a DNN. To validate the efficacy of this new approach, a new feature designed in the same fashion is proposed for pitch salience function. The new feature outperforms the one-layer spectrum in the MPE task and, as predicted, it addresses the issue of the missing fundamental effect and also achieves better robustness to noise.

Motivation & Objective

• To bridge the conceptual gap between homomorphic signal processing and deep neural networks in the context of multi-pitch estimation (MPE).
• To explain the functional role of nonlinear activation and multi-layer structures in DNNs using signal processing principles.
• To develop a new pitch salience feature, GCoS, inspired by RECOS filter analysis and homomorphic processing.
• To validate the proposed method’s superiority in detecting missing fundamentals and robustness to noise in MPE tasks.

Proposed method

• Proposes the generalized cepstrum of spectrum (GCoS), a three-layer feature extraction framework combining DFT, high-pass filtering, and nonlinear activation.
• Applies the RECOS filter framework to model DNN layers as filters that eliminate negative-correlation components, improving feature discrimination.
• Uses piecewise multiplication in the Fourier domain to simulate nonlinear activation, drawing from homomorphic signal processing and cepstrum theory.
• Employs a power-law nonlinear function (e.g., dB or Stevens’ law) to model perceptual scaling, grounding the activation in psychoacoustics.
• Constructs a DNN-like architecture using cascaded DFT matrices and nonlinearities to extract harmonic patterns from audio spectra.
• Initializes network parameters using DFT matrices to leverage known spectral periodicity, enabling better convergence and interpretability.

Experimental results

Research questions

• RQ1 Can homomorphic signal processing techniques explain the functional role of nonlinear activation and multi-layer structures in DNNs for MPE?
• RQ2 Does a DNN-like architecture built from DFT and nonlinear operations improve performance on missing fundamental detection compared to standard spectral features?
• RQ3 To what extent does the proposed GCoS feature enhance robustness to noise in polyphonic music transcription?
• RQ4 Can the RECOS filter framework be used to interpret and unify traditional pitch detection methods and modern DNNs?
• RQ5 How does the proposed method compare to baseline spectral features in terms of precision, recall, and F-score across diverse datasets?

Key findings

• The proposed GCoS feature achieves a 1.24% improvement in F-score on the Bass pitch range in the MAPS dataset and 1.39% in the TRIOS dataset compared to the baseline.
• In low-SNR conditions (below 10 dB), the F-score improvement of GCoS over the baseline exceeds 5%, demonstrating strong noise robustness.
• The method successfully detects missing fundamentals without hand-crafted rules, improving recall by 9.33% on the MAPS Bass set despite a 24.37% drop in precision.
• The inconsistency in precision and recall improvements across datasets (e.g., MAPS vs. TRIOS) is attributed to differences in background noise levels, with synthetic data (TRIOS) yielding higher precision.
• The GCoS feature outperforms one-layer spectral features in all evaluated pitch ranges and datasets, confirming its enhanced representation learning capability.
• Theoretical analysis confirms that the DNN-like structure of GCoS mirrors the RECOS filter framework, where nonlinear activation eliminates negative-correlation components, improving discrimination.