[Paper Review] Implicit Reparameterization Gradients

Michael Figurnov, Shakir Mohamed+1 more|arXiv (Cornell University)|May 22, 2018

Neural Networks and Applications7 references63 citations

TL;DR

Introduces implicit reparameterization gradients based on implicit differentiation to handle continuous distributions with numerically tractable CDFs (e.g., Gamma, Beta, Dirichlet, von Mises) where the standard reparameterization trick fails, and shows faster, more accurate gradients across applications.

ABSTRACT

By providing a simple and efficient way of computing low-variance gradients of continuous random variables, the reparameterization trick has become the technique of choice for training a variety of latent variable models. However, it is not applicable to a number of important continuous distributions. We introduce an alternative approach to computing reparameterization gradients based on implicit differentiation and demonstrate its broader applicability by applying it to Gamma, Beta, Dirichlet, and von Mises distributions, which cannot be used with the classic reparameterization trick. Our experiments show that the proposed approach is faster and more accurate than the existing gradient estimators for these distributions.

Motivation & Objective

• Motivate and extend pathwise gradient estimation beyond distributions suitable for the classical reparameterization trick.
• Develop a gradient technique based on implicit differentiation that avoids inverting the standardization function.
• Demonstrate computational improvements and accuracy gains over existing estimators for challenging continuous distributions.

Proposed method

• Formulate implicit reparameterization gradients by differentiating the standardization function with respect to distribution parameters and applying implicit differentiation to obtain dz/dphi without inverting the standardization.
• Show that ∇_phi z = - (∇_z S_phi(z))^{-1} ∇_phi S_phi(z), enabling gradient calculation using only derivatives of the standardization function.
• Use the CDF as a universal standardization function for univariate distributions, yielding ∇_phi z = - (∇_phi F(z|phi)) / q_phi(z).
• Extend to multivariate distributions via a sequential (conditional) copula-based transformation S_phi(z) = (F(z1|phi), F(z2|z1,phi), ..., FD(zD|z1,...,zD-1,phi)).
• Leverage automatic differentiation to compute needed derivatives of the CDFs and standardization functions, enabling numerically tractable gradients.
• Demonstrate practical algorithmic steps and compare implicit gradients to explicit reparameterization across distributions.

Experimental results

Research questions

• RQ1 Can implicit reparameterization gradients provide unbiased, low-variance gradients for continuous distributions lacking closed-form inverse CDFs?
• RQ2 Do implicit gradients offer speed and accuracy advantages over existing estimators for distributions like Gamma, Beta, Dirichlet, and von Mises?
• RQ3 How does the implicit approach perform in applications such as latent Dirichlet allocation and variational autoencoders with non-Normal latent variables?
• RQ4 Is the CDF-based standardization a practical universal choice for univariate and multivariate distributions in this framework?

Key findings

• Implicit gradients yield unbiased estimates for gradients of expectations with respect to parameters of distributions with tractable CDFs.
• The method yields faster and more accurate gradients than existing estimators for several challenging distributions.
• The approach enables black-box amortized inference for Latent Dirichlet Allocation and can train VAEs with Gamma, Beta, and von Mises latent variables.
• The implicit method does not require inversion of the standardization function and can leverage automatic differentiation for CDFs.
• They provide a general framework that avoids high-variance score-function estimators in broader contexts.