[Paper Review] A Feature Learning Siamese Model for Intelligent Control of the Dynamic Range Compressor

Sheng Di, György Fazekas|arXiv (Cornell University)|May 1, 2019

Music and Audio Processing27 references4 citations

TL;DR

This paper proposes a Siamese deep neural network (DNN) model that learns universal audio feature embeddings to intelligently control dynamic range compression (DRC) parameters—threshold, ratio, attack time, and release time—using a single reference audio. The model outperforms handcrafted features, especially when predicting multiple parameters simultaneously, achieving lower mean absolute error (MAE) across mono-instrument loops and polyphonic music, demonstrating improved generalization and robustness.

ABSTRACT

In this paper, a siamese DNN model is proposed to learn the characteristics of the audio dynamic range compressor (DRC). This facilitates an intelligent control system that uses audio examples to configure the DRC, a widely used non-linear audio signal conditioning technique in the areas of music production, speech communication and broadcasting. Several alternative siamese DNN architectures are proposed to learn feature embeddings that can characterise subtle effects due to dynamic range compression. These models are compared with each other as well as handcrafted features proposed in previous work. The evaluation of the relations between the hyperparameters of DNN and DRC parameters are also provided. The best model is able to produce a universal feature embedding that is capable of predicting multiple DRC parameters simultaneously, which is a significant improvement from our previous research. The feature embedding shows better performance than handcrafted audio features when predicting DRC parameters for both mono-instrument audio loops and polyphonic music pieces.

Motivation & Objective

• To develop a generic, end-to-end trainable feature learning model that can predict multiple DRC parameters simultaneously, overcoming limitations of previous handcrafted feature sets.
• To reduce computational complexity by replacing four separate regression models with one unified model for all DRC parameters.
• To improve robustness and generalization across diverse audio materials, including complex polyphonic music, by learning invariant audio representations.
• To evaluate the effectiveness of different DNN architectures and input representations (e.g., high-resolution spectrograms) in learning discriminative DRC-relevant features.
• To compare learned feature embeddings against handcrafted features in terms of prediction accuracy and generalization across varying audio content.

Proposed method

• A Siamese DNN architecture is employed, where two identical subnetworks process the reference and input audio, and the feature embedding is derived from the L2 difference between their final layer outputs.
• The model is trained using contrastive loss to learn a compact, discriminative embedding space where similar DRC parameter configurations are mapped close together.
• Input representations include high-resolution time-frequency spectrograms (e.g., 64×625) and multi-resolution spectrograms, with the former showing superior performance.
• The learned feature embeddings are used as input to a conventional regression model to predict DRC parameters, enabling comparison with handcrafted features.
• Model variants include standard CNNs, multi-kernel CNNs, and residual networks, with hyperparameters tuned to optimize performance on DRC parameter prediction.
• Training data is generated using a grid search over DRC parameters (e.g., threshold: 10–49 dB, ratio: 1.28–20, attack: 1–98.5 ms, release: 10–985 ms), creating large-scale datasets for evaluation.

Experimental results

Research questions

• RQ1 Can a single deep neural network learn a universal feature embedding that generalizes across multiple DRC parameters (threshold, ratio, attack, release)?
• RQ2 How does the performance of learned feature embeddings compare to handcrafted features when predicting multiple DRC parameters simultaneously?
• RQ3 Does the choice of input representation (e.g., high-resolution spectrogram vs. multi-resolution) significantly affect the quality of learned embeddings?
• RQ4 How does model architecture (e.g., standard CNN vs. multi-kernel CNN) influence the generalization and robustness of feature embeddings across different audio types?
• RQ5 Does the model generalize better to complex polyphonic music when trained on simpler drum loops, and why might performance improve on richer audio content?

Key findings

• The Siamese DNN model using a high-resolution spectrogram as input achieved the best performance, outperforming both handcrafted features and other DNN architectures.
• For four-parameter prediction, the model achieved a mean absolute error (MAE) of 2.369 dB for threshold, 3.265 for ratio, 10.868 ms for attack time, and 79.045 ms for release time—significantly lower than handcrafted features.
• When tested on polyphonic music from the MedleyDB dataset, the learned embeddings achieved an MAE of 1.697 dB for threshold and 9.873 ms for attack time, outperforming handcrafted features (11.585 dB and 26.628 ms respectively).
• Surprisingly, the model generalized better to complex polyphonic music than to drum loops alone, suggesting richer audio content may improve feature learning.
• The performance drop when predicting four parameters compared to two suggests that larger grid sizes and increased data complexity may require finer hyperparameter tuning or more training data.
• The model demonstrates that a single, unified DNN can replace four separate regression models, reducing computational complexity and improving feature generalization.