[Paper Review] Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning

Kaiyi Ji, Junjie Yang+1 more|arXiv (Cornell University)|Feb 18, 2020

Domain Adaptation and Few-Shot Learning52 references32 citations

TL;DR

The paper develops a theoretical framework proving convergence and characterizing complexity for multi-step MAML in both resampling and finite-sum settings, showing inner-step size should scale as 1/N and providing conditions for linear-in-N computational costs.

ABSTRACT

As a popular meta-learning approach, the model-agnostic meta-learning (MAML) algorithm has been widely used due to its simplicity and effectiveness. However, the convergence of the general multi-step MAML still remains unexplored. In this paper, we develop a new theoretical framework to provide such convergence guarantee for two types of objective functions that are of interest in practice: (a) resampling case (e.g., reinforcement learning), where loss functions take the form in expectation and new data are sampled as the algorithm runs; and (b) finite-sum case (e.g., supervised learning), where loss functions take the finite-sum form with given samples. For both cases, we characterize the convergence rate and the computational complexity to attain an $ε$-accurate solution for multi-step MAML in the general nonconvex setting. In particular, our results suggest that an inner-stage stepsize needs to be chosen inversely proportional to the number $N$ of inner-stage steps in order for $N$-step MAML to have guaranteed convergence. From the technical perspective, we develop novel techniques to deal with the nested structure of the meta gradient for multi-step MAML, which can be of independent interest.

Motivation & Objective

• Motivate the study of convergence for multi-step MAML in nonconvex settings.
• Provide a theoretical framework to analyze both resampling and finite-sum objective structures.
• Characterize convergence rates and computational complexities to reach an ε-accurate solution.
• Offer guidance on step-size choices and conditions for linear-in-N complexity.

Proposed method

• Derive a nested SGD/L inner loop and outer loop analysis for multi-step MAML.
• Provide a gradient expression for the outer update: ∇L_i(w) = [∏_{j=0}^{N-1}(I−α∇^2 l_i(w̃_j^i))] ∇l_i(w̃_N^i).
• Develop novel bounds to decouple Hessian and gradient estimation errors in the resampling case.
• Extend analysis to the finite-sum case where inner and outer losses differ (l_{S_i} vs l_{T_i}).
• Prove Lipschitz properties of the meta-gradient and bound estimation errors of the meta-gradient estimator.
• Establish conditions under which β_k = 1/(C_β L̂_{w_k}) yields ε-accurate solutions with complexity characterized in terms of N, ε, and batch sizes.

Experimental results

Research questions

• RQ1 Can multi-step MAML converge for nonconvex objectives in both resampling and finite-sum settings?
• RQ2 How should inner-step size α scale with the number of inner steps N to guarantee convergence?
• RQ3 What are the gradient/Hessian estimation error effects on convergence in the presence of nested inner loops?
• RQ4 What are the computational complexities to achieve an ε-accurate stationary point in both settings?
• RQ5 How do differences between inner and outer losses in the finite-sum case affect convergence analysis?

Key findings

• For convergence, choose inner-step size α with αL < 2^(1/(2N)) − 1, implying α = Θ(1/(NL)).
• N-step MAML has gradient and Hessian computation complexities that grow linearly with N under proper parameter choices.
• In small-Hessian problems, larger α can be used while preserving convergence, aligning with empirical observations.
• The resampling analysis decouples Hessian approximation error from gradient error via a bound on the distance between inner optimization paths.
• The finite-sum analysis handles differences between inner and outer losses and yields analogous convergence guarantees.
• Corollaries provide ε-accurate solution guarantees with O(1/ε^2) meta-iterations and explicit gradient/Hessian computation complexities.