[Paper Review] Normalizing Flows on Tori and Spheres

Danilo Jimenez Rezende, George Papamakarios+5 more|arXiv (Cornell University)|Feb 6, 2020

Generative Adversarial Networks and Image SynthesisComputer Science35 references58 citations

TL;DR

The paper develops expressive and numerically stable normalizing flows tailored for compact manifolds (circle, torus, sphere) by building flows recursively from S1 to higher dimensions, and demonstrates their use on synthetic directional targets.

ABSTRACT

Normalizing flows are a powerful tool for building expressive distributions in high dimensions. So far, most of the literature has concentrated on learning flows on Euclidean spaces. Some problems however, such as those involving angles, are defined on spaces with more complex geometries, such as tori or spheres. In this paper, we propose and compare expressive and numerically stable flows on such spaces. Our flows are built recursively on the dimension of the space, starting from flows on circles, closed intervals or spheres.

Motivation & Objective

• Motivate and address the mismatch between standard Euclidean flows and non-Euclidean data topologies (circles, tori, spheres).
• Propose expressive normalizing flows on S1, TD (torus), and SD (sphere) built recursively from simple building blocks.
• Ensure numerical stability and tractability in density evaluation and sampling on non-Euclidean manifolds.
• Compare proposed manifold-aware flows to prior approaches in directional statistics and geometric flows.

Proposed method

• Construct circle flows on S1 with boundary conditions ensuring a valid circle diffeomorphism (Equations 3–6).
• Build torus flows on TD by autoregressively combining circle diffeomorphisms as p(theta1,...,thetaD).
• Develop sphere flows on SD via recursive cylinder mapping and density updates; provide explicit density correction terms.
• Introduce three circle-specific diffeomorphisms: Möbius transformations, circular splines, and non-compact projections (NCP).
• Extend to higher dimensions using recursive (s1-c1) coupling and exponential-map flow variants for SD.
• Provide density change formulas and stability considerations (e.g., equations 15–20) and discuss computational trade-offs.

Experimental results

Research questions

• RQ1 How can normalizing flows be defined and trained on non-Euclidean manifolds like S1, TD, and SD?
• RQ2 What constructions yield expressive yet numerically stable flows on circles, tori, and spheres?
• RQ3 How do manifold-aware flows compare to traditional Euclidean flows in modeling sharp, multi-modal, and correlated directional densities?
• RQ4 What are the density update rules and computational costs for the proposed sphere and torus flows?
• RQ5 Can exponential-map based flows provide alternative yet tractable options on spheres?

Key findings

• Propose and validate circle, torus, and sphere flows that are expressive and numerically stable on compact connected manifolds.
• Show that torus densities can be modeled autoregressively using circle-based conditional transformers.
• Demonstrate recursion-based sphere flows via cylinder transformations with analytically tractable density updates (and discuss stability).
• Present three circle diffeomorphisms (Möbius, circular splines, non-compact projection) to build rich S1 flows.
• Provide empirical results on synthetic targets demonstrating ability to learn sharp, multi-modal, and correlated densities with ESS-based evaluation.
• Discuss density update formulas ensuring finite densities for the recursive construction and conditions for stability.