[Paper Review] Emotion Detection with Transformers: A Comparative Study

Mahdi Rezapour|arXiv (Cornell University)|Mar 18, 2024

Emotion and Mood Recognition6 citations

TL;DR

The paper compares multiple pre-trained transformer models for emotion classification on text and analyzes how fine-tuning, layer trainability, and preprocessing affect performance, finding that removing punctuation and stop words can harm results.

ABSTRACT

In this study, we explore the application of transformer-based models for emotion classification on text data. We train and evaluate several pre-trained transformer models, on the Emotion dataset using different variants of transformers. The paper also analyzes some factors that in-fluence the performance of the model, such as the fine-tuning of the transformer layer, the trainability of the layer, and the preprocessing of the text data. Our analysis reveals that commonly applied techniques like removing punctuation and stop words can hinder model performance. This might be because transformers strength lies in understanding contextual relationships within text. Elements like punctuation and stop words can still convey sentiment or emphasis and removing them might disrupt this context.

Motivation & Objective

• Evaluate multiple pre-trained transformer models for emotion classification on a standard text emotion dataset.
• Analyze how fine-tuning strategy and which transformer layers are trainable impact model performance.
• Investigate the influence of preprocessing steps on emotion detection with transformers.

Proposed method

• Train and evaluate several pre-trained transformer models on the Emotion dataset for emotion classification.
• Experiment with different transformer variants and fine-tuning configurations.
• Assess the impact of which layers are trainable during fine-tuning.
• Analyze preprocessing choices such as punctuation and stop word handling and their effect on performance.

Experimental results

Research questions

• RQ1 Which pre-trained transformer models provide the best performance for emotion classification on the Emotion dataset?
• RQ2 How do fine-tuning and layer trainability choices influence emotion detection accuracy?
• RQ3 What is the impact of common preprocessing steps, like removing punctuation or stop words, on transformer-based emotion classification?

Key findings

• Transformers’ ability to capture contextual relationships means that aggressive preprocessing (removing punctuation or stop words) can reduce performance.
• Fine-tuning decisions and which layers are updated significantly affect results.
• Preprocessing choices can modulate model performance, challenging common preprocessing habits in NLP pipelines.