[Paper Review] Are Negative Samples Necessary in Entity Alignment? An Approach with High Performance, Scalability and Robustness

Xin Mao, Meng Wan+2 more|arXiv (Cornell University)|Aug 11, 2021

Advanced Graph Neural Networks48 references4 citations

TL;DR

This paper proposes PSR, a novel entity alignment method that eliminates negative sampling by introducing a symmetric, negative-free loss and a simplified relational graph sampling encoder, enabling high performance, scalability, and robustness on large-scale knowledge graphs. PSR achieves state-of-the-art performance with linear time and memory complexity, outperforming prior methods on both standard and large-scale datasets while requiring minimal GPU memory and supporting incremental learning.

ABSTRACT

Entity alignment (EA) aims to find the equivalent entities in different KGs, which is a crucial step in integrating multiple KGs. However, most existing EA methods have poor scalability and are unable to cope with large-scale datasets. We summarize three issues leading to such high time-space complexity in existing EA methods: (1) Inefficient graph encoders, (2) Dilemma of negative sampling, and (3) "Catastrophic forgetting" in semi-supervised learning. To address these challenges, we propose a novel EA method with three new components to enable high Performance, high Scalability, and high Robustness (PSR): (1) Simplified graph encoder with relational graph sampling, (2) Symmetric negative-free alignment loss, and (3) Incremental semi-supervised learning. Furthermore, we conduct detailed experiments on several public datasets to examine the effectiveness and efficiency of our proposed method. The experimental results show that PSR not only surpasses the previous SOTA in performance but also has impressive scalability and robustness.

Motivation & Objective

• To address the high time and space complexity of existing entity alignment methods in large-scale knowledge graphs.
• To eliminate the need for negative sampling, which is computationally expensive and often low-quality in entity alignment.
• To improve scalability and robustness by enabling incremental semi-supervised learning and reducing memory consumption.
• To maintain high performance across diverse datasets, including sparse and large-scale graphs, without relying on costly sampling or complex architectures.

Proposed method

• Proposes a simplified graph encoder using relational graph sampling to reduce computational cost while preserving structural and relational information.
• Introduces a symmetric negative-free alignment loss that directly optimizes for entity equivalence without requiring negative pairs.
• Employs an incremental semi-supervised learning strategy with a reviewing mechanism to mitigate catastrophic forgetting and enable linearly scalable training.
• Uses mini-batch training with linear memory growth relative to batch size and graph density, enabling deployment on devices with limited GPU memory.
• Incorporates literal features as a supplementary signal to improve performance on sparse entities, especially in low-density graphs.
• Designs a reviewing threshold mechanism to selectively retrain on early samples, balancing performance and training efficiency.

Experimental results

Research questions

• RQ1 Can entity alignment be effectively performed without negative sampling, and what are the implications for model design and efficiency?
• RQ2 How can graph encoders be simplified to maintain performance while drastically reducing time and space complexity?
• RQ3 Can incremental learning with a reviewing mechanism prevent catastrophic forgetting and enable scalable training on growing knowledge graphs?
• RQ4 How does the performance of negative sampling-free models compare to SOTA methods on large-scale and sparse knowledge graphs?

Key findings

• PSR achieves state-of-the-art performance on multiple benchmark datasets, surpassing previous SOTA methods in Hits@1 scores.
• The model maintains high performance even with a small batch size of 128 and a low embedding dimension of 100, with only a 2% and 3% drop in Hits@1, respectively.
• PSR’s memory consumption grows linearly with batch size and graph density, enabling training on GPUs with only 12GB memory, unlike prior methods that exceed this limit.
• The incremental learning strategy with a reviewing threshold reduces training time significantly while maintaining performance, demonstrating linear scalability with dataset size.
• On sparse datasets like SRPRS_{DE-EN}, PSR with literal features outperforms structural-only models, showing the value of supplementary information in low-connectivity regions.
• The symmetric negative-free loss eliminates the need for expensive negative sampling, reducing training time by over 25% compared to methods relying on k-nearest neighbor sampling.