[Paper Review] Efficient Relation-aware Scoring Function Search for Knowledge Graph Embedding

Shimin Di, Quanming Yao+2 more|arXiv (Cornell University)|Apr 22, 2021

Advanced Graph Neural Networks50 references4 citations

TL;DR

This paper proposes ERAS, a one-shot, supernet-based neural architecture search method for learning relation-aware scoring functions in knowledge graph embedding. By modeling relation-specific scoring functions within a shared supernet and using efficient alternative minimization, ERAS achieves state-of-the-art performance on benchmark datasets with improved generalization and efficiency over existing AutoML-based methods.

ABSTRACT

The scoring function, which measures the plausibility of triplets in knowledge graphs (KGs), is the key to ensure the excellent performance of KG embedding, and its design is also an important problem in the literature. Automated machine learning (AutoML) techniques have recently been introduced into KG to design task-aware scoring functions, which achieve state-of-the-art performance in KG embedding. However, the effectiveness of searched scoring functions is still not as good as desired. In this paper, observing that existing scoring functions can exhibit distinct performance on different semantic patterns, we are motivated to explore such semantics by searching relation-aware scoring functions. But the relation-aware search requires a much larger search space than the previous one. Hence, we propose to encode the space as a supernet and propose an efficient alternative minimization algorithm to search through the supernet in a one-shot manner. Finally, experimental results on benchmark datasets demonstrate that the proposed method can efficiently search relation-aware scoring functions, and achieve better embedding performance than state-of-the-art methods.

Motivation & Objective

• To address the limitation of existing scoring functions that treat all relations uniformly, despite varying semantic patterns in knowledge graphs.
• To design a search space that enables relation-aware scoring function selection, improving model expressiveness and generalization.
• To reduce the computational cost of relation-aware search by encoding the search space as a supernet and enabling one-shot architecture search.
• To develop an efficient optimization algorithm that alternately updates scoring functions and embeddings, avoiding overfitting.

Proposed method

• The method formulates a relation-aware scoring function search problem by grouping relations based on semantic patterns and assigning distinct scoring functions to each group.
• It constructs a supernet that unifies all candidate scoring functions across relation groups, enabling parameter sharing and efficient search.
• A one-shot architecture search strategy is employed, where the supernet is trained once and different subnetworks are evaluated without retraining.
• An alternative minimization algorithm is proposed to jointly optimize the scoring functions and entity-relation embeddings by alternating between updating the functions and the embeddings.
• The search process uses validation performance to guide the selection of optimal scoring functions per relation group, ensuring generalization.
• The method dynamically assigns relations to groups during search, avoiding biases from human-defined semantic groupings.

Experimental results

Research questions

• RQ1 Can a relation-aware scoring function search strategy significantly improve knowledge graph embedding performance compared to task-aware but relation-agnostic methods?
• RQ2 How can the large search space of relation-aware scoring functions be efficiently explored without training each candidate from scratch?
• RQ3 What is the impact of dynamic relation grouping versus static, human-defined groupings on model performance?
• RQ4 How does the number of relation groups and block numbers affect the trade-off between performance and efficiency?
• RQ5 Can a supernet-based one-shot NAS approach achieve competitive results while maintaining high search efficiency?

Key findings

• ERAS achieves state-of-the-art performance on all benchmark datasets, with MRR scores of 0.953 on WN18, 0.492 on WN18RR, 0.855 on FB15K, 0.365 on FB15k237, and 0.577 on YAGO3-10.
• The method outperforms AutoSF and other SOTA baselines across all datasets, demonstrating the effectiveness of relation-aware scoring function design.
• ERAS achieves optimal performance with 3–4 relation groups, showing that moderate grouping improves generalization without excessive complexity.
• The bi-level optimization strategy in ERAS leads to better generalization than single-level optimization, as shown by superior validation performance.
• The supernet-based one-shot search enables efficient training and inference, with flexible settings for block numbers and grouping strategies.
• Dynamic grouping via search outperforms human-defined semantic groupings, indicating that data-driven assignment is more effective than prior assumptions.