[Paper Review] Learning to Generalize: Meta-Learning for Domain Generalization

Da Li, Yongxin Yang+2 more|arXiv (Cornell University)|Oct 10, 2017

Domain Adaptation and Few-Shot Learning26 references113 citations

TL;DR

This paper proposes a model-agnostic meta-learning procedure (MLDG) to train models that generalize across unseen domains by simulating train-test domain shifts within each minibatch, applicable to both supervised and reinforcement learning.

ABSTRACT

Domain shift refers to the well known problem that a model trained in one source domain performs poorly when applied to a target domain with different statistics. {Domain Generalization} (DG) techniques attempt to alleviate this issue by producing models which by design generalize well to novel testing domains. We propose a novel {meta-learning} method for domain generalization. Rather than designing a specific model that is robust to domain shift as in most previous DG work, we propose a model agnostic training procedure for DG. Our algorithm simulates train/test domain shift during training by synthesizing virtual testing domains within each mini-batch. The meta-optimization objective requires that steps to improve training domain performance should also improve testing domain performance. This meta-learning procedure trains models with good generalization ability to novel domains. We evaluate our method and achieve state of the art results on a recent cross-domain image classification benchmark, as well demonstrating its potential on two classic reinforcement learning tasks.

Motivation & Objective

• Motivate domain generalization (DG) as a harder alternative to domain adaptation that does not rely on target data at test time.
• Introduce a model-agnostic meta-learning procedure (MLDG) to improve generalization across unseen domains.
• Provide a gradient-based optimization framework that can be applied to any base learner and to both supervised and reinforcement learning.
• Demonstrate state-of-the-art results on a cross-domain image recognition benchmark and show promising results on classic reinforcement learning tasks.

Proposed method

• Split source domains into meta-train and meta-test groups within each minibatch to simulate domain shift.
• Compute meta-train loss F on meta-train domains and meta-test loss G on meta-test domains using updated parameters Theta' after a gradient step on F.
• Optimize Theta to minimize F + beta * G where G is evaluated at Theta - alpha * grad_theta F, enforcing that improvements on training domains align with improvements on testing domains.
• Apply the same meta-learning framework to reinforcement learning, where domain shift corresponds to different environments, using policy gradient (REINFORCE) or Q-learning as base learners.
• Provide theoretical intuition via Taylor expansion showing alignment of gradients F' and G' as a steering factor for coordinated improvement.
• Optionally include variants (MLDG-GC, MLDG-GN) that emphasize gradient direction alignment or gradient norm.

Experimental results

Research questions

• RQ1 Can a model-agnostic meta-learning procedure improve domain generalization without access to target domain data during testing?
• RQ2 Does simulating train-test domain shifts within minibatches lead to gradients that align across training and unseen domains, yielding better out-of-domain performance?
• RQ3 Is the approach effective across both supervised learning and reinforcement learning settings?
• RQ4 How does MLDG compare to aggregating source domains and to other DG methods on cross-domain benchmarks?
• RQ5 What are the practical implications and limitations of applying MLDG to real-world domain shift scenarios?

Key findings

• MLDG achieves state-of-the-art results on a cross-domain image recognition benchmark (PACS) compared with several baselines.
• Applying MLDG to reinforcement learning tasks (Cart-Pole and Mountain Car) yields improved domain generalization across varied environments.
• End-to-end MLDG within CNNs provides larger gains than applying it only to final layers, indicating the importance of meta-optimization.
• Variants that enforce gradient alignment (MLDG-GC) or gradient norm (MLDG-GN) offer mixed benefits depending on the task, with vanilla MLDG often performing best.
• The method remains model-agnostic and scalable, not requiring extra parameters tied to the number of domains, unlike many model-based DG approaches.