[Paper Review] Gradient-EM Bayesian Meta-learning

Yayi Zou, Xiaoqi Lu|arXiv (Cornell University)|Jun 21, 2020

Domain Adaptation and Few-Shot Learning10 references4 citations

TL;DR

This paper proposes Gradient-EM Bayesian Meta-learning (GEM-BML), a novel Bayesian meta-learning framework that decouples inner and meta-updates via a gradient-EM-inspired algorithm, avoiding backpropagation in meta-updates. It achieves faster adaptation, improved uncertainty estimation, and reduced computational cost across few-shot regression, image classification, and reinforcement learning tasks.

ABSTRACT

Bayesian meta-learning enables robust and fast adaptation to new tasks with uncertainty assessment. The key idea behind Bayesian meta-learning is empirical Bayes inference of hierarchical model. In this work, we extend this framework to include a variety of existing methods, before proposing our variant based on gradient-EM algorithm. Our method improves computational efficiency by avoiding back-propagation computation in the meta-update step, which is exhausting for deep neural networks. Furthermore, it provides flexibility to the inner-update optimization procedure by decoupling it from meta-update. Experiments on sinusoidal regression, few-shot image classification, and policy-based reinforcement learning show that our method not only achieves better accuracy with less computation cost, but is also more robust to uncertainty.

Motivation & Objective

• Address the high computational cost of meta-updates in Bayesian meta-learning, especially when backpropagating through inner optimization steps.
• Improve robustness and uncertainty calibration in few-shot learning and reinforcement learning by leveraging Bayesian inference.
• Decouple inner-update and meta-update processes to increase flexibility in optimization choices and support distributed or private learning.
• Extend empirical Bayes frameworks to reinforcement learning settings, which had not been systematically addressed before.
• Develop a scalable, efficient alternative to existing gradient-based Bayesian meta-learning methods that rely on implicit differentiation or second-order gradients.

Proposed method

• Propose a gradient-EM-based algorithm that computes meta-gradients without backpropagating through the inner optimization path.
• Use a variational approximation to the posterior over task-specific parameters, enabling efficient computation of the meta-loss gradient.
• Decouple the meta-update from the inner-update by relying only on the final value of the inner-optimized parameters, not the full optimization trajectory.
• Apply the method to both supervised and reinforcement learning settings, using a unified Bayesian meta-learning formulation.
• Introduce a modified loss function $ L^{[2]} $ that enables stable training in policy-based reinforcement learning while preserving computational efficiency.
• Leverage the theoretical foundation of the Gradient-EM theorem to derive a stable and scalable update rule for meta-parameters.

Experimental results

Research questions

• RQ1 Can we design a Bayesian meta-learning method that avoids backpropagation through the inner optimization process while maintaining performance?
• RQ2 How does decoupling inner and meta-updates affect uncertainty estimation and robustness in few-shot learning?
• RQ3 Can the proposed method be successfully extended to policy-based reinforcement learning, where previous Bayesian meta-learning approaches struggled?
• RQ4 What is the impact of the $ L^{[2]} $ variant on training stability and performance in RL settings?
• RQ5 To what extent does the method improve computational efficiency without sacrificing accuracy or uncertainty calibration?

Key findings

• The proposed GEM-BML method reduces computational cost in meta-updates by eliminating the need for backpropagation through inner optimization steps.
• GEM-BML achieves state-of-the-art performance on sinusoidal regression and few-shot image classification with fewer inner update steps and lower memory usage.
• In reinforcement learning, the $ L^{[2]} $ variant of GEM-BML enables stable and effective training, demonstrating the first successful Bayesian meta-learning framework for policy-based RL.
• The method provides more reliable predictive uncertainty estimates compared to deterministic MAML and other gradient-based baselines.
• Experiments show that GEM-BML maintains high accuracy even when using a large number of inner update steps, which is infeasible for standard MAML-style methods.
• The decoupled update mechanism enables potential applications in distributed and privacy-preserving meta-learning, as gradients do not need to be transmitted between devices.