[Paper Review] Modeling homophily and stochastic equivalence in symmetric relational data

Peter D. Hoff|ArXiv.org|Nov 7, 2007

Bayesian Modeling and Causal InferenceComputer Science8 references212 citations

TL;DR

This paper proposes the eigenmodel, a latent variable framework for symmetric relational data that generalizes both latent class and distance models by modeling relationships as weighted inner products of node-specific latent vectors. It demonstrates mathematically and empirically that the eigenmodel captures both homophily and stochastic equivalence, outperforming traditional models in predictive accuracy across three real-world datasets: friendship networks, word adjacency in Genesis, and protein-protein interactions.

ABSTRACT

This article discusses a latent variable model for inference and prediction of symmetric relational data. The model, based on the idea of the eigenvalue decomposition, represents the relationship between two nodes as the weighted inner-product of node-specific vectors of latent characteristics. This ``eigenmodel'' generalizes other popular latent variable models, such as latent class and distance models: It is shown mathematically that any latent class or distance model has a representation as an eigenmodel, but not vice-versa. The practical implications of this are examined in the context of three real datasets, for which the eigenmodel has as good or better out-of-sample predictive performance than the other two models.

Motivation & Objective

• To develop a unified statistical model that captures both homophily and stochastic equivalence in symmetric relational data.
• To generalize existing latent variable models—specifically latent class and distance models—within a single framework.
• To demonstrate that the eigenmodel provides better out-of-sample predictive performance than traditional models on real-world datasets.
• To show mathematically that any latent class or distance model can be represented as a special case of the eigenmodel, but not vice versa.
• To provide a low-rank approximation of sociomatrix data using eigenvalue decomposition for improved interpretability and inference.

Proposed method

• The eigenmodel represents the relationship between nodes i and j as y_ij = β'x_ij + u_i^TΛu_j, where u_i and u_j are latent vectors and Λ is a diagonal matrix of eigenvalues.
• It uses the eigenvalue decomposition of the sociomatrix to extract low-rank latent structures that capture variation in relational data.
• The model is extended to non-Gaussian data using an ordered probit likelihood for discrete or count-valued relational data.
• Markov chain Monte Carlo (MCMC) methods are used for posterior inference on latent vectors and model parameters.
• Model comparison is performed via out-of-sample predictive performance using cross-validation.
• The model generalizes both latent class and distance models, as any such model can be represented as a special case of the eigenmodel.

Experimental results

Research questions

• RQ1 Can a single latent variable model represent both homophily and stochastic equivalence in symmetric relational data?
• RQ2 How does the eigenmodel compare in predictive performance to latent class and distance models on real datasets?
• RQ3 What is the mathematical relationship between the eigenmodel and existing models such as latent class and distance models?
• RQ4 Can the eigenmodel provide a more flexible and accurate representation of complex network structures than traditional models?
• RQ5 Does the eigenmodel’s ability to represent both homophily and stochastic equivalence lead to improved out-of-sample prediction?

Key findings

• The eigenmodel outperforms both latent class and distance models in out-of-sample predictive performance on all three real datasets analyzed: friendship networks, word adjacency in Genesis, and protein-protein interactions.
• In the friendship network data, the eigenmodel and distance models performed best, while the latent class model performed poorly, even with increased K.
• For relational text data from Genesis, the latent class model outperformed the distance model, and the eigenmodel matched its performance, supporting the idea that word adjacency is better explained by class-based relationships than spatial proximity.
• In the protein-protein interaction data, the K=3 eigenmodel outperformed both the latent class and distance models across all values of K, indicating its superior ability to capture both hub-like structures and transitive triads.
• The mathematical derivation shows that every latent class and distance model can be represented as a special case of the eigenmodel, but not vice versa, confirming the eigenmodel’s generality.
• The eigenmodel’s use of low-rank approximation via eigenvalue decomposition enables it to flexibly represent complex relational patterns without requiring a priori assumptions about group structure or spatial embedding.