[Paper Review] SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions

Han Xiao, Minlie Huang+1 more|arXiv (Cornell University)|Apr 17, 2016

Topic ModelingComputer Science117 citations

TL;DR

SSP jointly learns from symbolic triples and textual descriptions by projecting triple losses onto a semantic hyperplane, improving knowledge graph completion and entity classification over baselines.

ABSTRACT

Knowledge representation is an important, long-history topic in AI, and there have been a large amount of work for knowledge graph embedding which projects symbolic entities and relations into low-dimensional, real-valued vector space. However, most embedding methods merely concentrate on data fitting and ignore the explicit semantic expression, leading to uninterpretable representations. Thus, traditional embedding methods have limited potentials for many applications such as question answering, and entity classification. To this end, this paper proposes a semantic representation method for knowledge graph extbf{(KSR)}, which imposes a two-level hierarchical generative process that globally extracts many aspects and then locally assigns a specific category in each aspect for every triple. Since both aspects and categories are semantics-relevant, the collection of categories in each aspect is treated as the semantic representation of this triple. Extensive experiments justify our model outperforms other state-of-the-art baselines substantially.

Motivation & Objective

• Motivate leveraging entity and relation textual descriptions to enhance KG embeddings.
• Propose a semantic space projection (SSP) model to couple triples with text via a semantic hyperplane.
• Show that projecting loss vectors onto semantic subspaces yields better discrimination and semantic relevance.
• Demonstrate improvements on KG completion and entity classification across benchmark datasets.

Proposed method

• Represent a triple as a loss vector e = h + r − t from TransE-style embeddings.
• Define a semantic hyperplane using a semantic vector s derived from texts (s = S(sh, st)).
• Project e onto the hyperplane to obtain e − s^T e s, and score with f_r(h,t) = −λ ||e − s^T e s||^2 + ||e||^2.
• Generate semantic vectors from text descriptions using topic models (Std) or joint learning with embeddings (Joint).
• Compose head/tail semantic vectors via S(sh, st) = (sh + st) / ||sh + st||^2 to form the hyperplane normal.
• Train with a rank-based hinge loss for embeddings and a topic loss from Non-negative Matrix Factorization (NMF) topic modeling, balancing them with μ.
• Use Bernoulli sampling for negative triples and SGD for optimization.
• Set hyperparameters: d ≈ 100, γ, λ ≈ 0.2–0.6 depending on dataset, with μ controlling joint training.

Experimental results

Research questions

• RQ1 Can textual descriptions be effectively integrated with symbolic triples by projecting embeddings into a semantic subspace?
• RQ2 Does SSP improve knowledge graph completion (link prediction) and entity classification beyond text-aware and triple-only baselines?
• RQ3 How do Std (topic-based) and Joint (text and triples learned together) settings compare in performance and convergence?
• RQ4 What is the role of strong correlation modeling via the semantic hyperplane in discrimination and precision of predictions?

Key findings

• SSP consistently outperforms baselines on knowledge graph completion and entity classification across benchmark datasets.
• SSP(Joint) yields larger improvements over SSP(Std.), highlighting the benefit of jointly learning textual semantics with triples.
• Modeling strong text–triple correlations via a semantic hyperplane provides better discrimination than first-order text–triple interactions.
• SSP significantly improves ranking-based metrics (Mean Rank, HITS@10) on FB15K and WN18, and achieves higher MAP for entity classification on FB15K/FB20K.
• Compared to TransE, SSP demonstrates notable gains due to semantic projection, and surpasses DKRL variants by modeling the strong correlation.