[Paper Review] Adversarial Learning for Debiasing Knowledge Graph Embeddings

Mario Arduini, Lorenzo Noci+4 more|arXiv (Cornell University)|Jun 29, 2020

Advanced Graph Neural Networks26 references4 citations

TL;DR

This paper proposes FAN (Filtering Adversarial Network), a novel adversarial framework to debias knowledge graph embeddings by removing sensitive attribute information—particularly gender bias—while preserving utility for downstream tasks. FAN uses a filter network adversarially trained with a discriminator to extract and suppress gender-related signals from embeddings, achieving near-random gender prediction accuracy without degrading performance on other tasks like occupation prediction.

ABSTRACT

Knowledge Graphs (KG) are gaining increasing attention in both academia and industry. Despite their diverse benefits, recent research have identified social and cultural biases embedded in the representations learned from KGs. Such biases can have detrimental consequences on different population and minority groups as applications of KG begin to intersect and interact with social spheres. This paper aims at identifying and mitigating such biases in Knowledge Graph (KG) embeddings. As a first step, we explore popularity bias -- the relationship between node popularity and link prediction accuracy. In case of node2vec graph embeddings, we find that prediction accuracy of the embedding is negatively correlated with the degree of the node. However, in case of knowledge-graph embeddings (KGE), we observe an opposite trend. As a second step, we explore gender bias in KGE, and a careful examination of popular KGE algorithms suggest that sensitive attribute like the gender of a person can be predicted from the embedding. This implies that such biases in popular KGs is captured by the structural properties of the embedding. As a preliminary solution to debiasing KGs, we introduce a novel framework to filter out the sensitive attribute information from the KG embeddings, which we call FAN (Filtering Adversarial Network). We also suggest the applicability of FAN for debiasing other network embeddings which could be explored in future work.

Motivation & Objective

• To investigate and mitigate societal and cultural biases embedded in knowledge graph embeddings (KGEs), particularly gender bias.
• To examine the presence of popularity bias in network embeddings and its contrast with KGEs, where high-degree nodes show higher link prediction accuracy.
• To develop a generalizable debiasing framework applicable beyond gender bias and to other embedding types.
• To demonstrate that sensitive attributes like gender can be inferred from KGEs even when explicit gender relations are removed.
• To validate that FAN effectively removes gender bias across both high- and low-degree entities in knowledge graphs.

Proposed method

• Propose FAN, an adversarial framework consisting of a filter network and a discriminator network trained jointly in a minimax setting.
• Train the filter to learn an identity mapping on KGEs while the discriminator attempts to predict gender from the embeddings.
• Use a multi-layer perceptron (MLP) with Leaky ReLU and dropout (0.5) for both the filter and discriminator components.
• Initiate adversarial training by updating the filter every five steps of the discriminator to stabilize learning.
• Apply the pre-trained filter to generate debiased embeddings, then evaluate gender and occupation prediction performance on filtered embeddings.
• Use cross-validated evaluation to measure performance degradation and bias removal efficacy.

Experimental results

Research questions

• RQ1 To what extent is gender information encoded in KGEs even after explicit gender relations are removed from the knowledge graph?
• RQ2 How does the link prediction accuracy of KGEs correlate with node degree, and how does this compare to network embeddings like node2vec?
• RQ3 Can an adversarial filtering framework effectively remove gender bias from KGEs without degrading performance on non-sensitive downstream tasks?
• RQ4 Does the debiasing framework FAN generalize across different KGE algorithms and node degree distributions?
• RQ5 Is the gender bias in KGEs attributable to structural properties of the embedding space rather than explicit relations?

Key findings

• Gender information is highly predictive from KGEs of TransE, TransD, and TransH, even when explicit gender relations are removed, indicating structural encoding of bias.
• Link prediction accuracy in KGEs is positively correlated with node degree, contrasting with network embeddings like node2vec, where high-degree nodes show lower accuracy—indicating a distinct popularity bias in KGEs.
• FAN successfully reduces gender prediction accuracy to near-random levels (approximately 50%) on both high- and low-degree nodes, demonstrating effective debiasing.
• Occupation prediction performance remains stable after filtering, confirming that FAN preserves utility for non-sensitive tasks.
• The adversarial training scheme effectively suppresses gender-related signals without compromising the representational quality of the embeddings.
• FAN is applicable beyond gender bias and can be extended to debias other sensitive attributes in KGEs and potentially other network embedding frameworks.