[Paper Review] mvn2vec: Preservation and Collaboration in Multi-View Network Embedding

Yu Shi, Fangqiu Han+5 more|arXiv (Cornell University)|Jan 19, 2018

Advanced Graph Neural Networks53 references57 citations

TL;DR

The paper identifies preservation and collaboration as key objectives for multi-view network embedding and proposes mvn2vec-con and mvn2vec-reg to jointly model them, with experiments showing when joint modeling helps.

ABSTRACT

Multi-view networks are broadly present in real-world applications. In the meantime, network embedding has emerged as an effective representation learning approach for networked data. Therefore, we are motivated to study the problem of multi-view network embedding with a focus on the optimization objectives that are specific and important in embedding this type of network. In our practice of embedding real-world multi-view networks, we explicitly identify two such objectives, which we refer to as preservation and collaboration. The in-depth analysis of these two objectives is discussed throughout this paper. In addition, the mvn2vec algorithms are proposed to (i) study how varied extent of preservation and collaboration can impact embedding learning and (ii) explore the feasibility of achieving better embedding quality by modeling them simultaneously. With experiments on a series of synthetic datasets, a large-scale internal Snapchat dataset, and two public datasets, we confirm the validity and importance of preservation and collaboration as two objectives for multi-view network embedding. These experiments further demonstrate that better embedding can be obtained by simultaneously modeling the two objectives, while not over-complicating the model or requiring additional supervision. The code and the processed datasets are available at http://yushi2.web.engr.illinois.edu/.

Motivation & Objective

• Identify objectives specific to multi-view network embedding (preservation and collaboration) based on real-world practice.
• Analyze when preservation and collaboration co-exist and how they affect embedding quality.
• Propose mvn2vec-con and mvn2vec-reg to model both objectives without extra supervision.
• Evaluate how varying degrees of preservation and collaboration impact embedding performance on synthetic and real datasets.

Proposed method

• Define multi-view networks and center/context embeddings for each view.
• Introduce intra-view skip-gram style loss to preserve view-specific structure.
• Propose mvn2vec-con with constrained cross-view context embedding to encourage collaboration via shared parameters (theta controls sharing).
• Propose mvn2vec-reg with cross-view regularization to preserve view-specific embeddings while encouraging collaboration via penalty terms gamma.
• Use random-walk based sampling and negative sampling (skip-gram) for scalable optimization.
• Provide ASGD optimization with gradients for both mvn2vec-con and mvn2vec-reg and discuss complexity.

Experimental results

Research questions

• RQ1 What are the two objectives (preservation and collaboration) for multi-view network embedding and why are they important?
• RQ2 Can jointly modeling preservation and collaboration yield better embeddings than models that focus on one objective?
• RQ3 How can mvn2vec-con and mvn2vec-reg implement these objectives without extra supervision?
• RQ4 How do these methods perform on synthetic and real-world multi-view networks across downstream tasks?

Key findings

• Preservation and collaboration are both impactful and their relative importance depends on the data (e.g., YouTube vs. Twitter).
• Joint modeling (mvn2vec-con or mvn2vec-reg) can outperform single-objective approaches when collaboration is beneficial, while preserving view-specific semantics is advantageous in other cases.
• In experiments on synthetic and real datasets, the proposed models validate the usefulness of incorporating both preservation and collaboration.
• A simple independent model favors preservation, while a one-space model emphasizes collaboration, with mvn2vec variants balancing both objectives.