[Paper Review] Leveraging Node Attributes for Incomplete Relational Data

He Zhao, Lan Du+1 more|arXiv (Cornell University)|Jun 14, 2017

Complex Network Analysis Techniques27 references19 citations

TL;DR

This paper proposes the Node Attribute Relational Model (NARM), a fully Bayesian Poisson factorization approach that leverages binary node attributes to improve link prediction and community detection in sparse, incomplete relational networks. By incorporating attributes through conjugate gamma priors on latent factor loadings, NARM enables efficient Gibbs sampling that scales linearly with observed links and achieves state-of-the-art performance, especially under data scarcity.

ABSTRACT

Relational data are usually highly incomplete in practice, which inspires us to leverage side information to improve the performance of community detection and link prediction. This paper presents a Bayesian probabilistic approach that incorporates various kinds of node attributes encoded in binary form in relational models with Poisson likelihood. Our method works flexibly with both directed and undirected relational networks. The inference can be done by efficient Gibbs sampling which leverages sparsity of both networks and node attributes. Extensive experiments show that our models achieve the state-of-the-art link prediction results, especially with highly incomplete relational data.

Motivation & Objective

• Address the challenge of link prediction and community detection in highly incomplete relational networks where observed links are sparse.
• Integrate diverse node attributes—such as user demographics or paper categories—into probabilistic relational models to improve performance when relational data are limited.
• Develop a scalable, fully Bayesian framework that maintains conjugacy and leverages sparsity in both networks and attributes for efficient inference.
• Extend existing Poisson factorization models to handle both directed and undirected networks with flat or hierarchical node attributes.
• Provide a principled way to balance relational data and side information by using attributes as prior knowledge for latent factor loadings, rather than direct regression on link probabilities.

Proposed method

• Propose Sym-NARM for undirected networks and Asym-NARM for directed networks, both based on Poisson gamma conjugate priors and extending the Infinite Edge Partition Model (EPM) and Beta-Gamma-Gamma Poisson Factorization (BGGPF).
• Model node attributes as binary features and use them to construct hyper-priors on the gamma-distributed factor loadings via a product of attribute-specific loadings $ h_{l,k} $, which modulate the shape parameters $ g_{i,k} $ of the gamma distribution.
• Incorporate side information through a hierarchical prior: $ g_{i,k} = b_k \prod_{l=1}^L h_{l,k}^{f_{i,l}} $, where $ f_{i,l} $ is the binary attribute of node $ i $, and $ h_{l,k} $ controls the influence of attribute $ l $ on factor $ k $.
• Enable efficient inference via closed-form Gibbs sampling that operates only on non-zero entries in the adjacency and attribute matrices, exploiting sparsity for linear scalability.
• Use auxiliary variables $ t_{i,k} $ to represent the rising factorial of the gamma function via a Chinese Restaurant Process, enabling tractable sampling of $ g_{i,k} $.
• Extend the model to hierarchical attributes by replacing the global hyper-parameter $ \mu_0 $ with a product of attribute-level parameters $ \delta_{m,k}^{f'_{l,m}} $, allowing multi-level attribute integration.

Experimental results

Research questions

• RQ1 Can node attributes significantly improve link prediction performance in highly incomplete relational networks where traditional models fail?
• RQ2 How can side information be incorporated into Bayesian Poisson factorization models in a way that preserves conjugacy and enables scalable inference?
• RQ3 To what extent do different attributes contribute to latent community structures, and can the model learn attribute-specific influence weights automatically?
• RQ4 Does the proposed method outperform existing state-of-the-art models in link prediction, particularly under data scarcity or cold-start conditions?
• RQ5 Can the model handle both directed and undirected networks with flat and hierarchical node attributes in a unified, principled framework?

Key findings

• NARM achieves state-of-the-art link prediction performance on multiple benchmark datasets, especially under high incompleteness, where relational data are sparse.
• The integration of node attributes leads to significant improvements in link prediction AUC and AUPR, particularly in cold-start scenarios where nodes have no observed links.
• The model scales linearly with the number of observed links and sparsity in attributes, enabling efficient inference on large-scale sparse networks.
• The attribute factor loadings $ h_{l,k} $ effectively learn which attributes are most predictive of relational ties, with higher loadings indicating stronger influence on latent factors.
• The use of conjugate priors and auxiliary variable augmentation allows for closed-form Gibbs sampling that is both accurate and efficient, avoiding complex MCMC schemes.
• The model generalizes well to hierarchical attributes, such as IPC codes in patent networks, by extending the prior structure to multi-level attribute hierarchies.