[Paper Review] Domain Generalization using Causal Matching

Divyat Mahajan, Shruti Tople+1 more|arXiv (Cornell University)|Jun 12, 2020

Domain Adaptation and Few-Shot LearningComputer Science65 references81 citations

TL;DR

The paper argues that class-conditional domain invariance is insufficient for domain generalization and proposes a causal, object-based invariance via matching; it introduces MatchDG and MDGHybrid to learn representations that align same-object inputs across domains, achieving competitive out-of-domain accuracy.

ABSTRACT

In the domain generalization literature, a common objective is to learn representations independent of the domain after conditioning on the class label. We show that this objective is not sufficient: there exist counter-examples where a model fails to generalize to unseen domains even after satisfying class-conditional domain invariance. We formalize this observation through a structural causal model and show the importance of modeling within-class variations for generalization. Specifically, classes contain objects that characterize specific causal features, and domains can be interpreted as interventions on these objects that change non-causal features. We highlight an alternative condition: inputs across domains should have the same representation if they are derived from the same object. Based on this objective, we propose matching-based algorithms when base objects are observed (e.g., through data augmentation) and approximate the objective when objects are not observed (MatchDG). Our simple matching-based algorithms are competitive to prior work on out-of-domain accuracy for rotated MNIST, Fashion-MNIST, PACS, and Chest-Xray datasets. Our method MatchDG also recovers ground-truth object matches: on MNIST and Fashion-MNIST, top-10 matches from MatchDG have over 50% overlap with ground-truth matches.

Motivation & Objective

• Challenge the sufficiency of class-conditional domain-invariant representations for domain generalization.
• Introduce a structural causal model to capture within-class variation and object-level stability.
• Propose an ideal perfect-match regularizer when objects are observed.
• Develop MatchDG to approximate object-based matching when objects are unobserved.
• Extend to MDGHybrid to leverage data augmentations as perfect object matches.

Proposed method

• Model data generation with stable causal features X_C and domain-dependent features X_A to define object-based invariance.
• Formulate an ideal invariant: representations should be the same across domains for inputs deriving from the same object, i.e., G(X) aligns X_C across domains conditioned on O.
• Derive a perfect-match regularizer that minimizes representation distance across domains for the same object while preserving predictive power.
• Propose MatchDG: a two-phase iterative algorithm that first learns a representation via contrastive learning without ERM loss, then learns object-based matches to regularize the classifier.
• Extend MatchDG to MDGHybrid by incorporating known object matches from data augmentations as additional regularizers.
• Provide theoretical justification showing that standard domain-invariance methods do not capture X_C, and that the proposed perfect-match objective targets the true causal features.

Experimental results

Research questions

• RQ1 Can class-conditional domain invariance fail to generalize to unseen domains due to within-class variation in stable features?
• RQ2 How can one identify or approximate object-level invariances that align across domains when objects are unobserved?
• RQ3 Do matching-based regularizers that enforce consistency of representations for the same object improve out-of-domain generalization?
• RQ4 Can data augmentations provide reliable perfect matches to enhance object-based matching (MDGHybrid) and improve generalization?
• RQ5 How do MatchDG and MDGHybrid perform relative to state-of-the-art domain generalization methods across benchmarks like rotated MNIST, Fashion-MNIST, PACS, and Chest X-rays?

Key findings

• Class-conditional invariance is not sufficient for domain generalization when stable feature distributions vary across domains.
• An object-centric invariant can theoretically achieve domain-generalizable prediction by conditioning on the causal features X_C.
• MatchDG, a two-phase method using iterative contrastive learning and matching, achieves competitive out-of-domain accuracy on Rotated MNIST, Fashion-MNIST, PACS, and Chest X-ray datasets.
• On datasets with known object matches, MatchDG recovers ground-truth-like matches with notable overlap (top-10 matches over 50% overlap).
• MDGHybrid leverages data augmentations to provide perfect object matches, further improving performance in practice.