[Paper Review] Multitask Learning for Emotion and Personality Detection

Yang Li, Amirmohammad Kazameini+2 more|arXiv (Cornell University)|Jan 7, 2021

Personality Traits and Psychology52 references19 citations

TL;DR

This paper proposes SoGMTL, a novel CNN-based multitask learning framework that jointly predicts personality traits (OCEAN) and emotions using shared representations, leveraging a novel information-sharing gate (SoG) and a MAML-like optimization strategy. The model achieves state-of-the-art performance across multiple benchmark datasets for both personality and emotion detection, outperforming even large language model-based approaches.

ABSTRACT

In recent years, deep learning-based automated personality trait detection has received a lot of attention, especially now, due to the massive digital footprints of an individual. Moreover, many researchers have demonstrated that there is a strong link between personality traits and emotions. In this paper, we build on the known correlation between personality traits and emotional behaviors, and propose a novel multitask learning framework, SoGMTL that simultaneously predicts both of them. We also empirically evaluate and discuss different information-sharing mechanisms between the two tasks. To ensure the high quality of the learning process, we adopt a MAML-like framework for model optimization. Our more computationally efficient CNN-based multitask model achieves the state-of-the-art performance across multiple famous personality and emotion datasets, even outperforming Language Model based models.

Motivation & Objective

• To address the gap in multitask learning for personality and emotion detection by jointly modeling both tasks.
• To explore and validate effective information-sharing mechanisms between personality trait detection and emotion detection.
• To improve model generalization and efficiency through a MAML-like optimization strategy for multi-source data.
• To demonstrate that shared representations between personality and emotion tasks enhance performance on both downstream tasks.

Proposed method

• Proposes SoGMTL, a multitask learning framework using a shared CNN backbone for joint personality and emotion prediction.
• Introduces a novel information-sharing gate, SoG (Selective Output Gating), to dynamically control feature flow between tasks at different network layers.
• Employs a MAML-like meta-learning optimization strategy to improve model generalization across multiple data sources.
• Evaluates four information-sharing mechanisms: SiG, SiLG, CAG, and SoG, through ablation and comparative analysis.
• Uses a multilabel classification head for personality traits (OCEAN) and a multiclass head for emotions, with sigmoid and softmax activations respectively.
• Performs ablation studies to identify optimal placement of the SoG gate across convolutional, pooling, and dense layers.

Experimental results

Research questions

• RQ1 Which information-sharing mechanism (SiG, SiLG, CAG, SoG) yields the best performance in multitask personality and emotion detection?
• RQ2 How does the placement of the SoG gate in different network layers (convolutional, pooling, dense) affect model performance and information flow?
• RQ3 Can a MAML-like optimization strategy improve model generalization and performance in multitask learning with multiple data sources?
• RQ4 Does joint learning of personality and emotion improve performance compared to single-task baselines?

Key findings

• SoGMTL achieves state-of-the-art performance on multiple personality and emotion datasets, outperforming both single-task models and large language model-based approaches.
• The SoG gate mechanism significantly improves performance compared to SiG, SiLG, and CAG, with the best results when placed in the convolutional layer.
• The ablation study confirms that the convolutional layer is the most effective location for SoG, as it achieves the highest F1 score (72.03%) and best cosine similarity (0.32) to the optimal information-sharing configuration.
• The MAML-like optimization strategy enhances model generalization, particularly when training on multiple data sources.
• Case studies demonstrate that SoGMTL produces clear, confident predictions with distinct confidence scores, indicating robust decision boundaries for both multilabel personality and multiclass emotion tasks.
• The model is computationally efficient, with SoG showing competitive inference speed, making it suitable for real-world deployment.