[Paper Review] Redundancy Reduction Twins Network: A Training framework for Multi-output Emotion Regression

Xin Jing, Meishu Song+3 more|arXiv (Cornell University)|Jun 18, 2022

Anomaly Detection Techniques and Applications4 citations

TL;DR

This paper proposes the Redundancy Reduction Twins Network (RRTN), a training framework that reduces feature redundancy in multi-output emotion regression by minimizing cross-correlation between embeddings from augmented views of the same sample, using a Barlow Twins loss and a novel Restrained Uncertainty Weight Loss (RUWL) for dynamic loss balancing. The method achieves a 4.8% relative improvement in concordance correlation coefficient (CCC) over vanilla CNN14, reaching 0.678 on the ExVo dev set.

ABSTRACT

In this paper, we propose the Redundancy Reduction Twins Network (RRTN), a redundancy reduction training framework that minimizes redundancy by measuring the cross-correlation matrix between the outputs of the same network fed with distorted versions of a sample and bringing it as close to the identity matrix as possible. RRTN also applies a new loss function, the Barlow Twins loss function, to help maximize the similarity of representations obtained from different distorted versions of a sample. However, as the distribution of losses can cause performance fluctuations in the network, we also propose the use of a Restrained Uncertainty Weight Loss (RUWL) or joint training to identify the best weights for the loss function. Our best approach on CNN14 with the proposed methodology obtains a CCC over emotion regression of 0.678 on the ExVo Multi-task dev set, a 4.8% increase over a vanilla CNN 14 CCC of 0.647, which achieves a significant difference at the 95% confidence interval (2-tailed).

Motivation & Objective

• Address feature redundancy in multi-output emotion regression models to improve generalization and performance.
• Develop a training framework that leverages data augmentation to enhance representation learning without increasing model complexity.
• Introduce a novel loss weighting strategy to stabilize training and improve convergence in multi-loss settings.
• Demonstrate the effectiveness of redundancy reduction and joint loss optimization on a large-scale vocal burst emotion dataset (Hume-VB).

Proposed method

• Apply data augmentation via SpecAugment to generate two distorted views of each input sample.
• Feed the original and augmented samples into identical encoders to produce representations for regression and embeddings for contrastive learning.
• Minimize the cross-correlation matrix between embeddings to make it as close to the identity matrix as possible, reducing redundancy.
• Use the Barlow Twins loss function to maximize similarity between embeddings of the same sample under different augmentations.
• Implement a Restrained Uncertainty Weight Loss (RUWL) strategy that dynamically adjusts loss weights while constraining their sum to maintain stability.
• Train the network using a weighted sum of the Barlow Twins loss and the emotion regression loss, with RUWL optimizing the trade-off between them.

Experimental results

Research questions

• RQ1 Can redundancy reduction in learned representations improve performance in multi-output emotion regression tasks?
• RQ2 How effective is the Barlow Twins loss in reducing redundancy when applied to audio-based emotion regression?
• RQ3 Does dynamic loss weighting via RUWL lead to more stable and improved training compared to fixed or unweighted loss combinations?
• RQ4 How does the RRTN framework perform across different backbone architectures, particularly in deeper networks like CNN14?
• RQ5 To what extent does the proposed framework outperform standard training baselines on the ExVo multi-task emotion regression benchmark?

Key findings

• The RRTN framework achieved a concordance correlation coefficient (CCC) of 0.678 on the ExVo multi-task dev set using CNN14, representing a 4.8% relative improvement over the vanilla CNN14 baseline (0.647).
• The improvement was statistically significant at the 95% confidence interval, confirming the effectiveness of the redundancy reduction approach.
• The RRTN framework consistently improved performance across different backbone models, with CNN14 showing the largest gain (4.8%) compared to CNN10 (2%) due to its deeper architecture.
• The RUWL loss strategy provided a marginal but consistent improvement over standard RRTN training, indicating that dynamic loss weighting enhances stability and performance.
• The cross-correlation minimization strategy was the primary driver of performance gains, while the loss weighting mechanism played a secondary but beneficial role.
• The method introduced minimal parameter overhead, as only a single linear projector layer was added, making it a lightweight yet effective training framework.