[Paper Review] Domain Generalization on Efficient Acoustic Scene Classification using Residual Normalization

Byeonggeun Kim, Seunghan Yang+2 more|arXiv (Cornell University)|Nov 12, 2021

Music and Audio Processing22 references7 citations

TL;DR

This paper proposes Residual Normalization (ResNorm), a frequency-wise normalization technique with a shortcut connection to suppress device-specific audio variations while preserving class-discriminative features, enabling domain generalization in efficient acoustic scene classification. The method achieves 76.3% test accuracy on the TAU Urban Acoustic Scenes 2020 Mobile dataset with 315k parameters and 75.3% after compression to 61.0KB, winning first place in the DCASE 2021 Challenge Task1A.

ABSTRACT

It is a practical research topic how to deal with multi-device audio inputs by a single acoustic scene classification system with efficient design. In this work, we propose Residual Normalization, a novel feature normalization method that uses frequency-wise normalization % instance normalization with a shortcut path to discard unnecessary device-specific information without losing useful information for classification. Moreover, we introduce an efficient architecture, BC-ResNet-ASC, a modified version of the baseline architecture with a limited receptive field. BC-ResNet-ASC outperforms the baseline architecture even though it contains the small number of parameters. Through three model compression schemes: pruning, quantization, and knowledge distillation, we can reduce model complexity further while mitigating the performance degradation. The proposed system achieves an average test accuracy of 76.3% in TAU Urban Acoustic Scenes 2020 Mobile, development dataset with 315k parameters, and average test accuracy of 75.3% after compression to 61.0KB of non-zero parameters. The proposed method won the 1st place in DCASE 2021 challenge, TASK1A.

Motivation & Objective

• Address the challenge of classifying acoustic scenes across multiple audio devices with unbalanced data and limited model complexity.
• Develop an efficient neural network architecture that maintains high accuracy while minimizing parameters.
• Improve domain generalization to unseen devices by reducing device-specific feature bias without losing discriminative information.
• Enable model compression via pruning, quantization, and knowledge distillation while minimizing performance degradation.
• Design a novel normalization method that enhances robustness to device variations in low-complexity models.

Proposed method

• Propose BC-ResNet-ASC, a modified Broadcasting-Residual Network with limited receptive field and max-pooling instead of dilation to reduce model size.
• Introduce Residual Normalization (ResNorm), a frequency-wise normalization using instance normalization per frequency band with a learnable shortcut path to preserve useful features.
• Apply a hybrid normalization strategy where the output is a weighted sum of frequency-wise normalized features and the original residual input, controlled by a hyperparameter λ.
• Use knowledge distillation to transfer knowledge from a larger teacher model to a compressed student model to mitigate accuracy loss during compression.
• Apply structured pruning (89% pruning ratio) and mixed-precision quantization (8-bit for convolutions, 16-bit for normalization) to reduce model size to 61.0KB.

Experimental results

Research questions

• RQ1 Can a lightweight neural network architecture achieve high accuracy in acoustic scene classification while maintaining robustness to device-specific variations?
• RQ2 How effective is frequency-wise normalization with a residual shortcut path (ResNorm) in improving generalization to unseen devices compared to standard normalization or feature transforms?
• RQ3 To what extent can model compression techniques—pruning, quantization, and knowledge distillation—reduce model size without significant performance degradation in low-complexity ASC?
• RQ4 Does the proposed ResNorm module improve performance across both seen and unseen devices, especially under domain imbalance?
• RQ5 Can the hyperparameter λ in ResNorm be fixed at 0.1 to balance regularization and feature preservation across different model sizes?

Key findings

• The proposed BC-ResNet-ASC-1 model with Residual Normalization achieves 65.8% test accuracy, a 6% improvement over the baseline, especially enhancing performance on unseen devices.
• Residual Normalization reduces the accuracy gap between seen and unseen devices, with a 3.0% improvement on device A and 8.2% on device A in the small model compared to FreqIN alone.
• The full model (BC-ResNet-ASC-8 + ResNorm) achieves 76.3% test accuracy on the TAU Urban Acoustic Scenes 2020 Mobile development set with only 315k parameters.
• After compression via 89% pruning, 8-bit quantization, and knowledge distillation, the model achieves 75.3% test accuracy with only 61.0KB of non-zero parameters.
• The ablation study shows that removing the ResNorm module from the network improves performance on the small model (67.1%) but degrades it on the larger one (by 1%), indicating a need for adaptive λ tuning.
• The 'w/o shortcut' ablation (equivalent to FreqIN) shows significantly lower accuracy on device A (8.2% lower than ResNorm in the small model), confirming the importance of the identity skip connection.