[Paper Review] Myers-Briggs Personality Classification and Personality-Specific Language Generation Using Pre-trained Language Models

Sedrick Scott Keh, I-Tsun Cheng|arXiv (Cornell University)|Jul 15, 2019

Mental Health via Writing8 references35 citations

TL;DR

The paper fine-tunes BERT to predict MBTI types from forum text and explores personality-specific language generation, reporting state-of-the-art MBTI classification accuracy on a PersonalityCafe dataset and detailing generation losses by type.

ABSTRACT

The Myers-Briggs Type Indicator (MBTI) is a popular personality metric that uses four dichotomies as indicators of personality traits. This paper examines the use of pre-trained language models to predict MBTI personality types based on scraped labeled texts. The proposed model reaches an accuracy of $0.47$ for correctly predicting all 4 types and $0.86$ for correctly predicting at least 2 types. Furthermore, we investigate the possible uses of a fine-tuned BERT model for personality-specific language generation. This is a task essential for both modern psychology and for intelligent empathetic systems.

Motivation & Objective

• Motivate using MBTI as a personality metric and predict MBTI types from text with pre-trained language models.
• Investigate the feasibility of generating text conditioned on a given MBTI type for empathetic systems.
• Compare BERT-based MBTI classification against prior methods and establish practical performance baselines.
• Explore data scraping, preprocessing, and fine-tuning specifics to maximize MBTI prediction accuracy.

Proposed method

• Fine-tune BERT (bert-base-uncased) for sequence classification with [CLS]/[SEP] tokens and cross-entropy loss.
• Preprocess text: lowercasing, tokenization via BERT tokenizer, remove explicit MBTI mentions, replace with a placeholder.
• Hyperparameter tuning on learning rate, max sequence length, and epochs; best result reported at lr=1e-5, maxlen=128, 30 epochs.
• Evaluate using accuracy for exact 4-letter MBTI predictions and per-letter category accuracies (I/E, N/S, F/T, P/J).
• Train a separate BERT-based masked language modeling model for personality-specific text generation across 16 MBTI types and report per-type losses.

Experimental results

Research questions

• RQ1 Can BERT-based models accurately classify MBTI types from user-generated text?
• RQ2 How does MBTI-based text generation perform when conditioned on a specific personality type?
• RQ3 What are the effects of hyperparameters on MBTI classification accuracy, and which MBTI categories are easier to distinguish?
• RQ4 What differences arise in generation quality across MBTI types, and which dichotomies most influence generation loss?

Key findings

• Best MBTI classification accuracy achieved is 0.4797 (approx. 0.48) using lr=1e-5, maxlen=128, 30 epochs.
• Classification accuracy of 0.4701 with lr=1e-4 and 128 max sequence length and 5 epochs; overall best around 0.48.
• When evaluating by letter, E/I and F/T are easier to distinguish than P/J; P/J is comparatively harder.
• Compared to prior work, BERT on PersonalityCafe data outperforms several baselines (e.g., logistic regression 0.190, SVM 0.370, LSTM 0.380).
• Language generation losses per personality type are generally under 0.02 for several extroverted types, with ENFJ/ESFJ/ESFP/ESTJ showing the lowest losses.
• Extroverted (E) types tend to yield lower generation losses than introverted types, suggesting more data availability for E types.