[Paper Review] Learning Embedding Representations for Knowledge Inference on Imperfect and Incomplete Repositories

Miao Fan, Qiang Zhou+1 more|arXiv (Cornell University)|Mar 27, 2015

Topic Modeling17 references3 citations

TL;DR

This paper proposes IIKE, a probabilistic knowledge embedding model that learns low-dimensional vector representations for entities and relations in imperfect and incomplete knowledge bases like Freebase and NELL. By minimizing confidence loss from machine learning (NELL) and crowdsourcing (Freebase), IIKE improves link prediction and triplet classification, achieving state-of-the-art performance with 91.1% and 91.4% accuracy on FB15K and NELL, respectively.

ABSTRACT

This paper considers the problem of knowledge inference on large-scale imperfect repositories with incomplete coverage by means of embedding entities and relations at the first attempt. We propose IIKE (Imperfect and Incomplete Knowledge Embedding), a probabilistic model which measures the probability of each belief, i.e. $\langle h,r,t angle$, in large-scale knowledge bases such as NELL and Freebase, and our objective is to learn a better low-dimensional vector representation for each entity ($h$ and $t$) and relation ($r$) in the process of minimizing the loss of fitting the corresponding confidence given by machine learning (NELL) or crowdsouring (Freebase), so that we can use $||{\bf h} + {\bf r} - {\bf t}||$ to assess the plausibility of a belief when conducting inference. We use subsets of those inexact knowledge bases to train our model and test the performances of link prediction and triplet classification on ground truth beliefs, respectively. The results of extensive experiments show that IIKE achieves significant improvement compared with the baseline and state-of-the-art approaches.

Motivation & Objective

• Address the challenge of knowledge inference in large-scale, imperfect, and incomplete knowledge repositories such as Freebase and NELL.
• Overcome limitations of existing methods that rely on complete, ground-truth training data or require external text sources.
• Develop a unified probabilistic embedding model that learns global connectivity patterns without relation-specific rules.
• Enable effective inference on incomplete knowledge bases by modeling belief plausibility through vector similarity.
• Improve performance in link prediction and triplet classification using only inexact, imperfect training data.

Proposed method

• Propose IIKE, a probabilistic model that estimates the plausibility of a triplet ⟨h, r, t⟩ using a scoring function based on entity and relation embeddings.
• Use a loss function that minimizes the discrepancy between predicted confidence and the actual confidence from NELL (ML) or Freebase (crowdsourcing).
• Apply negative sampling to efficiently train the model by sampling false triplets during stochastic gradient descent (SGD).
• Learn low-dimensional vector representations for entities (h, t) and relations (r) such that ||h + r - t|| measures plausibility.
• Optimize embeddings using SGD with hyperparameters d (embedding dimension), α (learning rate), b (margin), and norm (L1 or L2).
• Use relation-specific thresholds σr to classify triplets as positive or negative in triplet classification, learned via validation set optimization.

Experimental results

Research questions

• RQ1 Can a probabilistic embedding model effectively learn from imperfect and incomplete knowledge bases without relying on ground-truth training data?
• RQ2 How well can IIKE perform in link prediction and triplet classification when trained on inexact beliefs from NELL and Freebase?
• RQ3 Does learning from confidence scores (from NELL and Freebase) lead to better generalization than models trained on ground-truth data?
• RQ4 Can global connectivity patterns be effectively captured in a unified embedding space without relation-specific rule induction?
• RQ5 How does IIKE compare to state-of-the-art models like TransE, TransH, and NTN in terms of accuracy and discrimination capability?

Key findings

• IIKE achieves 91.1% accuracy on the FB15K triplet classification benchmark, significantly outperforming TransH (80.2%), TransE (79.7%), and NTN (66.7%).
• On the NELL dataset, IIKE reaches 91.4% accuracy, surpassing TransH (89.1%) and TransE (82.4%) in triplet classification.
• The precision-recall curve for IIKE shows a larger AUC than TransH and TransE, indicating superior global discrimination capability.
• IIKE demonstrates strong generalization on link prediction tasks, with performance improvements over baseline and SOTA methods on both FB15K and NELL.
• The use of confidence scores from NELL and Freebase as supervision signals enables effective learning even on incomplete and noisy knowledge bases.
• The model's effectiveness is validated across two distinct knowledge bases with different characteristics, confirming its robustness and scalability.