[Paper Review] When LLM Meets Hypergraph: A Sociological Analysis on Personality via Online Social Networks

Zhiyao Shu, Xiangguo Sun+1 more|arXiv (Cornell University)|Jul 4, 2024

Impact of Technology on Adolescents4 citations

TL;DR

This paper proposes a novel sociological framework that integrates large language models (LLMs) and hypergraph neural networks (HGNNs) to analyze personality traits in online social networks by modeling users as nodes and social environments as hyperedges. By leveraging LLMs to unify fragmented user data into coherent profiles and HGNNs to capture complex, multi-way interactions, the approach achieves significant performance gains in personality prediction, validated on a new, richly annotated dataset with hypergraph structure and personality labels.

ABSTRACT

Individual personalities significantly influence our perceptions, decisions, and social interactions, which is particularly crucial for gaining insights into human behavior patterns in online social network analysis. Many psychological studies have observed that personalities are strongly reflected in their social behaviors and social environments. In light of these problems, this paper proposes a sociological analysis framework for one's personality in an environment-based view instead of individual-level data mining. Specifically, to comprehensively understand an individual's behavior from low-quality records, we leverage the powerful associative ability of LLMs by designing an effective prompt. In this way, LLMs can integrate various scattered information with their external knowledge to generate higher-quality profiles, which can significantly improve the personality analysis performance. To explore the interactive mechanism behind the users and their online environments, we design an effective hypergraph neural network where the hypergraph nodes are users and the hyperedges in the hypergraph are social environments. We offer a useful dataset with user profile data, personality traits, and several detected environments from the real-world social platform. To the best of our knowledge, this is the first network-based dataset containing both hypergraph structure and social information, which could push forward future research in this area further. By employing the framework on this dataset, we can effectively capture the nuances of individual personalities and their online behaviors, leading to a deeper understanding of human interactions in the digital world.

Motivation & Objective

• To address the challenge of analyzing personality traits in online social networks, where user data is fragmented and traditional individual-level approaches fail to capture environmental influences.
• To develop a sociological framework that models the interplay between personality, behavior, and social environments rather than focusing solely on isolated user data.
• To create a comprehensive, high-quality dataset integrating user profiles, personality traits, behavior records, and detected social environments with a hypergraph structure.
• To improve personality analysis performance by combining the associative reasoning of LLMs with the structural modeling power of hypergraph neural networks.
• To enable deeper, more nuanced understanding of human behavior in digital environments through environment-based modeling.

Proposed method

• Leverage large language models (LLMs) with carefully designed prompts to integrate scattered, low-quality user data into higher-quality, coherent personality profiles by drawing on external knowledge.
• Construct a hypergraph where users are nodes and social environments (e.g., communities, content types) are hyperedges, enabling modeling of multi-user, multi-environment interactions.
• Design a hypergraph neural network (HGNN) to learn representations of users by capturing complex, non-pairwise relationships between users and their social environments.
• Train the HGNN on a newly collected dataset containing user profiles, personality traits (e.g., MBTI), behavior logs, and detected social environments.
• Use the LLM-generated profiles as input features to enrich the HGNN, enhancing its ability to infer personality traits from incomplete and fragmented data.
• Evaluate the framework using standard metrics (accuracy, AUC) across varying training data ratios to assess robustness and scalability.

Experimental results

Research questions

• RQ1 How can fragmented and low-quality user data in online social networks be effectively unified into coherent personality profiles?
• RQ2 To what extent can hypergraph neural networks model the complex, multi-way interactions between users and their social environments?
• RQ3 Can the integration of LLMs and HGNNs significantly improve personality trait prediction compared to individual-level baselines?
• RQ4 How does the proposed framework perform under data scarcity, particularly with limited training data?
• RQ5 What is the impact of incorporating sociological context—specifically, social environments—on personality analysis performance?

Key findings

• The proposed LLM-HGNN framework achieves substantial performance gains in personality trait prediction, with optimal results observed when training on the complete dataset.
• The model maintains strong performance even at 50% training data availability, demonstrating robustness to data scarcity, whereas baseline models suffer significant performance drops under similar conditions.
• The framework shows a positive correlation between training data volume and model performance, indicating that larger, richer datasets enhance learning and generalization.
• The integration of LLM-generated profiles significantly improves the quality of input representations, enabling better capture of nuanced personality traits from incomplete data.
• The hypergraph structure effectively models complex, non-pairwise social interactions, outperforming traditional GNNs that are limited to pairwise relationships.
• The newly released dataset, which includes hypergraph-structured social environments and personality labels, is the first of its kind and provides a valuable resource for future sociological and personality analysis research.