[Paper Review] GreenKGC: A Lightweight Knowledge Graph Completion Method

Yun-Cheng Wang, Xiou Ge+2 more|arXiv (Cornell University)|Aug 19, 2022

Advanced Graph Neural Networks4 citations

TL;DR

GreenKGC proposes a lightweight, modular knowledge graph completion method that achieves state-of-the-art performance in low-dimensional embeddings (8–32D) by combining representation learning, discriminative feature pruning, and relation-grouped classifier training. It outperforms SOTA methods in low dimensions and matches or exceeds high-dimensional models with significantly smaller model sizes, enabling efficient deployment on resource-constrained platforms.

ABSTRACT

Knowledge graph completion (KGC) aims to discover missing relationships between entities in knowledge graphs (KGs). Most prior KGC work focuses on learning embeddings for entities and relations through a simple scoring function. Yet, a higher-dimensional embedding space is usually required for a better reasoning capability, which leads to a larger model size and hinders applicability to real-world problems (e.g., large-scale KGs or mobile/edge computing). A lightweight modularized KGC solution, called GreenKGC, is proposed in this work to address this issue. GreenKGC consists of three modules: representation learning, feature pruning, and decision learning, to extract discriminant KG features and make accurate predictions on missing relationships using classifiers and negative sampling. Experimental results demonstrate that, in low dimensions, GreenKGC can outperform SOTA methods in most datasets. In addition, low-dimensional GreenKGC can achieve competitive or even better performance against high-dimensional models with a much smaller model size.

Motivation & Objective

• Address the challenge of large model sizes in knowledge graph completion (KGC) due to high-dimensional embeddings required for reasoning.
• Enable efficient, accurate KGC on resource-constrained platforms such as mobile and edge devices.
• Develop a modular, parameter-efficient framework that maintains high performance even in low-dimensional spaces.
• Reduce training time and dependency on complex initialization by avoiding end-to-end training of deep neural networks.
• Improve generalization and robustness through novel negative sampling strategies tailored to KG structure and embedding behavior.

Proposed method

• Uses a pre-trained KGE model (e.g., TransE) as a baseline for high-dimensional entity and relation representations in Module 1.
• Applies a feature pruning process in Module 2 to select discriminative dimensions from high-dimensional embeddings, reducing dimensionality while preserving predictive power.
• Groups relations with similar discriminant feature dimensions to share classifier parameters and improve efficiency.
• Trains a binary classifier per relation group in Module 3 to predict triple existence probabilities (0 to 1) using soft labels.
• Introduces two novel negative sampling schemes: ontology-based and embedding-based, for hard negative mining to improve classifier generalization.
• Trains each module independently, enabling modular optimization and reducing training complexity.

Experimental results

Research questions

• RQ1 Can a lightweight KGC method achieve competitive performance in low-dimensional embeddings without relying on high-dimensional representations?
• RQ2 How does modular feature pruning and relation grouping affect model efficiency and performance in low-dimensional KGC?
• RQ3 Can ontology-based and embedding-based negative sampling improve classifier learning compared to random sampling?
• RQ4 How does GreenKGC compare in inference speed and model size to SOTA KGE and neural network-based KGC methods?
• RQ5 Does the modular, decoupled training approach of GreenKGC avoid the convergence and optimization issues common in end-to-end deep learning models?

Key findings

• GreenKGC achieves SOTA performance in low dimensions (8, 16, 32) on FB15k-237 and WN18RR, outperforming existing low-dimensional KGC methods.
• In 32-dimensional settings, GreenKGC achieves MRR of 0.316 on FB15k-237 and 0.411 on WN18RR, surpassing baseline KGE and other low-dimensional models.
• GreenKGC with 32 dimensions matches or exceeds the performance of high-dimensional KGE models (e.g., 128D) while using significantly fewer parameters.
• GreenKGC reduces training time by ~20x compared to knowledge distillation methods like DualDE, especially on large-scale datasets like YAGO3-10.
• The ontology-based negative sampling strategy outperforms random and embedding-based sampling on FB15k-237, while embedding-based sampling is more effective on concept-based KGs like WN18RR.
• The predicted score distribution in GreenKGC reflects the sparsity of KGs, with only a few high-scoring candidates close to 1, indicating robustness and realistic confidence estimation.