[Paper Review] Attributed Network Embedding via Subspace Discovery

Daokun Zhang, Jie Yin+2 more|arXiv (Cornell University)|Jan 14, 2019

Advanced Graph Neural Networks35 references4 citations

TL;DR

This paper proposes attri2vec, a unified framework for attributed network embedding that discovers a latent attribute subspace through network structure-guided transformations of original node attributes. By aligning attribute space with network topology via learned mappings, attri2vec improves node representation quality, achieves state-of-the-art performance in node classification and clustering, and enables efficient out-of-sample node embedding via a learned mapping function.

ABSTRACT

Network embedding aims to learn a latent, low-dimensional vector representations of network nodes, effective in supporting various network analytic tasks. While prior arts on network embedding focus primarily on preserving network topology structure to learn node representations, recently proposed attributed network embedding algorithms attempt to integrate rich node content information with network topological structure for enhancing the quality of network embedding. In reality, networks often have sparse content, incomplete node attributes, as well as the discrepancy between node attribute feature space and network structure space, which severely deteriorates the performance of existing methods. In this paper, we propose a unified framework for attributed network embedding-attri2vec-that learns node embeddings by discovering a latent node attribute subspace via a network structure guided transformation performed on the original attribute space. The resultant latent subspace can respect network structure in a more consistent way towards learning high-quality node representations. We formulate an optimization problem which is solved by an efficient stochastic gradient descent algorithm, with linear time complexity to the number of nodes. We investigate a series of linear and non-linear transformations performed on node attributes and empirically validate their effectiveness on various types of networks. Another advantage of attri2vec is its ability to solve out-of-sample problems, where embeddings of new coming nodes can be inferred from their node attributes through the learned mapping function. Experiments on various types of networks confirm that attri2vec is superior to state-of-the-art baselines for node classification, node clustering, as well as out-of-sample link prediction tasks. The source code of this paper is available at https://github.com/daokunzhang/attri2vec.

Motivation & Objective

• To address the challenge of sparse and incomplete node attributes in attributed networks.
• To overcome the discrepancy between heterogeneous network structure and attribute feature spaces.
• To learn high-quality, low-dimensional node representations that jointly preserve structural and content information.
• To enable efficient out-of-sample node embedding using learned attribute transformations.
• To develop a scalable, online learning algorithm for large-scale attributed networks.

Proposed method

• A network structure-guided transformation is applied to original node attributes to discover a latent subspace that respects network topology.
• The method employs linear and non-linear mappings (e.g., ReLU, sigmoid, kernel) on attributes to learn structure-preserving representations.
• An optimization problem is formulated to preserve proximity in both network structure and transformed attribute space, solved via stochastic gradient descent with linear time complexity.
• The DeepWalk mechanism is used to generate node context sequences, ensuring structural similarity is encoded in the embedding space.
• A mapping function is learned during training, enabling inference of out-of-sample node embeddings from their attributes alone.
• The framework supports both inductive representation learning and efficient online updates.

Experimental results

Research questions

• RQ1 Can a latent attribute subspace be discovered that better aligns with network structure to improve node representation quality?
• RQ2 How effective are linear and non-linear transformations in enhancing attributed network embedding under attribute sparsity?
• RQ3 Can the proposed method generalize to out-of-sample nodes not seen during training?
• RQ4 How does the framework perform compared to state-of-the-art methods in node classification and clustering tasks?
• RQ5 What is the impact of attribute sparsity and model hyperparameters on the performance of the proposed method?

Key findings

• attri2vec outperforms state-of-the-art baselines in node classification and clustering across five real-world networks, with significant gains on Citeseer and DBLP.
• attri2vec-sigmoid with the Weighted-L2 operator achieves the highest AUC (0.892) in out-of-sample link prediction, demonstrating strong generalization to new nodes.
• The method remains robust under attribute sparsity: attri2vec variants outperform DeepWalk when only 30% of attributes are available, with stable performance from 50% to 100% attribute dimensions.
• Performance improves with increasing embedding dimension, random walk window size, and training iterations, stabilizing after reaching a threshold.
• The stochastic gradient descent algorithm ensures linear time complexity with respect to the number of nodes, enabling scalability to large networks.
• The learned mapping function enables accurate inference of node embeddings for new, unseen nodes using only their attributes, solving the out-of-sample problem effectively.