[Paper Review] Compositional Fairness Constraints for Graph Embeddings

Avishek Joey Bose, William L. Hamilton|arXiv (Cornell University)|May 25, 2019

Advanced Graph Neural NetworksComputer Science31 references95 citations

TL;DR

An adversarial, compositional framework enforces fairness invariance in graph embeddings, allowing flexible, test-time combinations of sensitive attributes and generalizing to unseen attribute sets.

ABSTRACT

Learning high-quality node embeddings is a key building block for machine learning models that operate on graph data, such as social networks and recommender systems. However, existing graph embedding techniques are unable to cope with fairness constraints, e.g., ensuring that the learned representations do not correlate with certain attributes, such as age or gender. Here, we introduce an adversarial framework to enforce fairness constraints on graph embeddings. Our approach is compositional---meaning that it can flexibly accommodate different combinations of fairness constraints during inference. For instance, in the context of social recommendations, our framework would allow one user to request that their recommendations are invariant to both their age and gender, while also allowing another user to request invariance to just their age. Experiments on standard knowledge graph and recommender system benchmarks highlight the utility of our proposed framework.

Motivation & Objective

• Motivate and formalize a notion of representational invariance for graph embeddings with respect to sensitive attributes.
• Develop a compositional encoder that can generate embeddings invariant to arbitrary subsets of sensitive attributes at inference time.
• Introduce adversarial filters that remove attribute information, enabling flexible, test-time combinatorial fairness constraints.
• Demonstrate the approach on knowledge graph and recommender benchmarks, highlighting tradeoffs between invariance and task performance.

Proposed method

• Define a compositional encoder c-enc that averages filtered embeddings across a chosen set of sensitive attributes S (Equation 6).
• Learn per-attribute filters f_k to remove information about the k-th sensitive attribute from node embeddings.
• Train adversarial discriminators D_k to predict the k-th attribute from embeddings, forming an adversarial loss (Equation 7).
• Use a stochastic Bernoulli masking scheme to sample random attribute subsets S during training, promoting generalization to unseen combinations.
• Optimize a joint objective that combines the edge-prediction loss with the adversarial regularization, alternating updates between c-enc and discriminators.
• Provide theoretical intuition that, with sufficient capacity and regularization weight, mutual information between embeddings and sensitive attributes can be driven toward zero.

Experimental results

Research questions

• RQ1 Can representational invariance with respect to multiple sensitive attributes be enforced in graph embeddings via an adversarial, compositional framework?
• RQ2 Does the compositional approach improve the removal of sensitive attribute information without severely sacrificing edge-prediction performance?
• RQ3 Can the model generalize fairness invariance to unseen combinations of sensitive attributes at test time?
• RQ4 How does compositional fairness compare to enforcing fairness for individual attributes separately?

Key findings

• The framework achieves a controllable tradeoff between invariance and edge-prediction accuracy, with more invariance typically reducing predictive performance.
• Compositional adversaries can remove more information about multiple attributes than training separate adversaries for each attribute in some datasets.
• The approach generalizes well to unseen combinations of sensitive attributes, with only small performance drops on held-out combinations in large attribute settings.
• On MovieLens-1M and Reddit, invariance nearly eliminates attribute predictability from embeddings, while incurring modest losses on main tasks; on Freebase-15k-237, removing sensitive information is more challenging without harming performance.
• Empirical results indicate reduced prediction bias with adversarial regularization, and compositional invariance can outperform non-compositional approaches in correlated attribute settings.