[论文解读] Equivariant Hamiltonian Flows

Danilo Jimenez Rezende, Sébastien Racanière+2 more|arXiv (Cornell University)|Sep 30, 2019

Topic Modeling参考文献 24被引用 41

一句话总结

Introduces equivariant Hamiltonian flows to learn densities invariant under a known Lie-algebra of local symmetry transformations, with a method to enforce equivariance and a lemma to construct invariant densities from equivariant flows.

ABSTRACT

This paper introduces equivariant hamiltonian flows, a method for learning expressive densities that are invariant with respect to a known Lie-algebra of local symmetry transformations while providing an equivariant representation of the data. We provide proof of principle demonstrations of how such flows can be learnt, as well as how the addition of symmetry invariance constraints can improve data efficiency and generalisation. Finally, we make connections to disentangled representation learning and show how this work relates to a recently proposed definition.

研究动机与目标

• Motivation to incorporate known invariances/equivariance into flow-based density models for improved data efficiency and generalisation.
• Propose a general algorithm to enforce equivariance in learned Hamiltonian flows with respect to connected Lie groups.
• Show how to construct invariant densities from equivariant flows using a simple lemma grounded in Noether's theorem.
• Demonstrate through experiments that symmetry constraints aid data efficiency and reduce overfitting.

提出的方法

• Represent density transformation as a Hamiltonian flow on state s=(q,p), with s' obtained via Euler discretization of the Poisson bracket with Hamiltonian H(s).
• Use volume-preserving, invertible symplectic integrators (e.g., Leap-Frog) for stable flow steps.
• Adopt a base invariant density pi(s) and transform it through a sequence of Hamiltonian flows to obtain p_theta(s_n).
• Handle latent momentum p_n via a variational encoder h_phi(p_n|q_n) to train with an ELBO, avoiding intractable marginalization.
• Impose symmetry constraints by requiring {g_k, H} = 0 for symmetry generators g_k, via constrained optimization (min_theta,max_lambda L).
• Prove Lemma 1: if {g_k, H} = {g_k, pi} = 0 for all k, then the induced density p is invariant under the symmetry generators.]

实验结果

研究问题

• RQ1 Can Hamiltonian flows be made equivariant to known symmetry groups while preserving density expressivity?
• RQ2 Does enforcing symmetry generators as constraints improve data efficiency and generalisation in flow-based models?
• RQ3 How can invariant marginals over q be ensured when the full state s evolves under a Hamiltonian flow?
• RQ4 What is the relationship between equivariant flows and disentangled representations in this framework?
• RQ5 Can the approach transform simple invariant base densities into arbitrarily complex invariant densities?

主要发现

• Equivariant Hamiltonian flows can produce invariant densities from invariant base densities by enforcing symmetry via Poisson brackets.
• Adding symmetry constraints improves data efficiency in both infinite and finite data regimes and reduces overfitting in the finite regime.
• The framework can learn multimodal densities, with the learned potential U(q) exhibiting multiple local minima corresponding to data modes.
• The method yields interpretable flows where attractors align with data modes, demonstrated in multimodal density learning experiments.
• The kinetic and potential energy decomposition H(q,p)=K(p)+U(q) with appropriate generators preserves invariance in the marginal q-distribution.
• The approach connects to disentangled representations by preserving group action structure across latent subspaces.