[Paper Review] Personality Trait Detection Using Bagged SVM over BERT Word Embedding Ensembles

Amirmohammad Kazameini, Samin Fatehi+3 more|arXiv (Cornell University)|Oct 3, 2020

Sentiment Analysis and Opinion MiningComputer Science16 references52 citations

TL;DR

The paper proposes BB-SVM, a computationally efficient approach that combines BERT-based contextual embeddings with Mairesse features and Bagging-SVM to predict Big-Five personality traits from essays, outperforming prior state-of-the-art by 1.04%.

ABSTRACT

Recently, the automatic prediction of personality traits has received increasing attention and has emerged as a hot topic within the field of affective computing. In this work, we present a novel deep learning-based approach for automated personality detection from text. We leverage state of the art advances in natural language understanding, namely the BERT language model to extract contextualized word embeddings from textual data for automated author personality detection. Our primary goal is to develop a computationally efficient, high-performance personality prediction model which can be easily used by a large number of people without access to huge computation resources. Our extensive experiments with this ideology in mind, led us to develop a novel model which feeds contextualized embeddings along with psycholinguistic features toa Bagged-SVM classifier for personality trait prediction. Our model outperforms the previous state of the art by 1.04% and, at the same time is significantly more computationally efficient to train. We report our results on the famous gold standard Essays dataset for personality detection.

Motivation & Objective

• Develop a computationally efficient model for automatic personality detection from text.
• Leverage BERT contextual embeddings combined with psycholinguistic features.
• Improve prediction performance on the Essays personality dataset while reducing training time.

Proposed method

• Break essays into 200-token sub-documents to fit BERT input limits.
• Extract contextualized embeddings by averaging token representations across BERT layers and concatenating the last four layers.
• Concatenate BERT features with 84 Mairesse features to form a 3156-dim document feature vector.
• Train ten SVM classifiers in parallel (Bagging) and use majority voting for final prediction.
• Compare Bagging-SVM with single models and different feature configurations.

Experimental results

Research questions

• RQ1 Can BERT-based contextual embeddings plus psycholinguistic features improve personality trait prediction from text over prior methods?
• RQ2 Does bagging multiple SVM classifiers offer a performance and training-time advantage for this task?
• RQ3 What is the impact of using last-four-layer BERT representations versus other configurations on accuracy?
• RQ4 How does the BB-SVM compare to prior state-of-the-art on the Essays dataset?

Key findings

• BB-SVM achieves higher average accuracy (59.03%) than the previous state-of-the-art (57.99%).
• Using BERT (last four layers) with Bagging-SVM outperforms Word2Vec-based approaches under the studied configurations.
• Bagging improves classification accuracy for personality detection compared to a single SVM.
• Training time is significantly reduced (about 7 minutes) versus ~50 hours for the prior method.
• Concatenating last-four BERT layers with Mairesse features provides strong feature vectors for SVM classifiers.