[Paper Review] Enhancing Personality Recognition by Comparing the Predictive Power of Traits, Facets, and Nuances

Amir Ansari, Jana Subirana+4 more|arXiv (Cornell University)|Feb 5, 2026

Personality Traits and Psychology0 citations

TL;DR

Nuance-level personality models outperform facet and trait models in audiovisual dyadic data, using a transformer with cross-modal and dyad-aware attention on UDIVA v0.5.

ABSTRACT

Personality is a complex, hierarchical construct typically assessed through item-level questionnaires aggregated into broad trait scores. Personality recognition models aim to infer personality traits from different sources of behavioral data. However, reliance on broad trait scores as ground truth, combined with limited training data, poses challenges for generalization, as similar trait scores can manifest through diverse, context dependent behaviors. In this work, we explore the predictive impact of the more granular hierarchical levels of the Big-Five Personality Model, facets and nuances, to enhance personality recognition from audiovisual interaction data. Using the UDIVA v0.5 dataset, we trained a transformer-based model including cross-modal (audiovisual) and cross-subject (dyad-aware) attention mechanisms. Results show that nuance-level models consistently outperform facet and trait-level models, reducing mean squared error by up to 74% across interaction scenarios.

Motivation & Objective

• Motivate hierarchical modeling of personality beyond coarse trait scores to capture context-dependent behavior.
• Investigate predictive power of Big Five facets and nuances versus traditional traits in multimodal data.
• Assess whether finer-grained labels improve self-reported personality estimation in dyadic interactions.

Proposed method

• Use UDIVA v0.5 multimodal dataset of 80 hours of dyadic interactions with self-reported BFI-2 traits, facets, and nuances.
• Apply per-task, level-specific self-reported personality regression via a spectral feature representation and a multimodal Transformer (MulT-based) with intra- and inter-modal attention.
• Aggregate lower-level predictions (facets/nuances) to trait level using BFI-2 scoring (means over relevant facets/items, with reverse coding as needed).
• Train 10-fold, subject-independent splits; optimize with Bayesian hyperparameter tuning and Adam optimizer; evaluate with MSE, MAE, PCC, and R2.

Experimental results

Research questions

• RQ1 Do nuance-level models provide better predictive performance than facet- and trait-level models for self-reported personality from audiovisual data?
• RQ2 Is there consistent performance gain across different Big Five traits and interaction tasks when using finer-grained labels?
• RQ3 How does aggregation from nuances/facets to traits affect predictive accuracy and reliability?
• RQ4 What role does cross-modal and dyad-aware attention play in leveraging nuanced behavioral cues for personality inference?

Key findings

• Nuance-level models consistently achieve the best performance across tasks, reducing MSE, MAE, and increasing PCC and R2 compared to baseline, traits, and facets (up to 87.46% MSE decrease).
• Facet models outperform trait models, with up to 31.75% MSE improvement over traits.
• Nuances outperform facets and traits even when evaluated at the trait level after aggregation, highlighting the value of fine-grained labels for context-sensitive personality estimation.
• Nuance-level models show particularly strong gains for Negative Emotionality when aggregated to traits, indicating nuanced behaviors can better capture this trait.
• Across tasks, task-level predictability differences are minimal for the hierarchical levels, suggesting the spectral-based approach and model architecture stabilize performance.