[Paper Review] Reversible Gromov-Monge Sampler for Simulation-Based Inference

YoonHaeng Hur, Wenxuan Guo+1 more|arXiv (Cornell University)|Sep 28, 2021

Statistical Methods and Inference74 references4 citations

TL;DR

This paper introduces the Reversible Gromov-Monge (RGM) sampler, a novel simulation-based inference method that uses a new distance metric—RGM—to align heterogeneous metric measure spaces without explicit density modeling. By minimizing a reversible transport cost that links source and target distributions via neural maps, the method achieves optimal alignment and generates high-quality samples, with RGM distance closely approximating the Gromov-Wasserstein distance under mild conditions and demonstrating strong performance on MNIST and synthetic data.

ABSTRACT

This paper introduces a new simulation-based inference procedure to model and sample from multi-dimensional probability distributions given access to i.i.d.\ samples, circumventing the usual approaches of explicitly modeling the density function or designing Markov chain Monte Carlo. Motivated by the seminal work on distance and isomorphism between metric measure spaces, we propose a new notion called the Reversible Gromov-Monge (RGM) distance and study how RGM can be used to design new transform samplers to perform simulation-based inference. Our RGM sampler can also estimate optimal alignments between two heterogeneous metric measure spaces $(\cX, μ, c_{\cX})$ and $(\cY, ν, c_{\cY})$ from empirical data sets, with estimated maps that approximately push forward one measure $μ$ to the other $ν$, and vice versa. We study the analytic properties of the RGM distance and derive that under mild conditions, RGM equals the classic Gromov-Wasserstein distance. Curiously, drawing a connection to Brenier's polar factorization, we show that the RGM sampler induces bias towards strong isomorphism with proper choices of $c_{\cX}$ and $c_{\cY}$. Statistical rate of convergence, representation, and optimization questions regarding the induced sampler are studied. Synthetic and real-world examples showcasing the effectiveness of the RGM sampler are also demonstrated.

Motivation & Objective

• To address the limitations of traditional MCMC and density-based methods in multi-dimensional simulation-based inference.
• To overcome the over-identification problem in transform sampling, where infinitely many maps can push forward one measure to another.
• To introduce a new distance metric—Reversible Gromov-Monge (RGM)—that enables optimal alignment between heterogeneous metric measure spaces.
• To provide a principled, differentiable framework for learning transport maps that induce strong isomorphism and minimize transport cost.
• To establish theoretical and empirical connections between RGM and the classic Gromov-Wasserstein distance.

Proposed method

• Proposes the Reversible Gromov-Monge (RGM) distance as a symmetric, differentiable measure of alignment between two metric measure spaces $(\mathcal{X},\mu,c_{\mathcal{X}})$ and $(\mathcal{Y},\nu,c_{\mathcal{Y}})$.
• Defines the RGM optimization problem as minimizing a joint cost function $C(F,B) = C_0(F,B) + \sum_{k=1}^3 \lambda_k m_k$, where $C_0$ measures discrepancy in pairwise distances and $m_k$ enforce distributional consistency via MMD.
• Uses fully connected neural networks $F: \mathbb{R}^{784} \to \mathbb{R}^{784}$ and $B: \mathbb{R}^{784} \to \mathbb{R}^4$ to parameterize the transport maps, trained via stochastic optimization with fixed hyperparameters.
• Employs empirical measures $\widehat{\mu}_m$ and $\widehat{\nu}_n$ from i.i.d. samples to estimate the RGM distance and optimize the maps.
• Derives theoretical connections to Brenier’s polar factorization and shows that under proper metric choices, RGM favors isomorphism and minimizes transport cost.
• Uses the Gromov-Wasserstein lower bounds (FLB and SLB) to estimate the gap between RGM and GW distances, validating RGM's consistency.

Experimental results

Research questions

• RQ1 Can a new symmetric distance metric be defined to enable optimal alignment between heterogeneous metric measure spaces without explicit density modeling?
• RQ2 How does the RGM distance relate to the classical Gromov-Wasserstein distance under mild regularity conditions?
• RQ3 Can the RGM framework induce transport maps that are both optimal in cost and structurally simple, satisfying an inductive bias toward isomorphism?
• RQ4 What is the statistical convergence rate and optimization stability of the RGM sampler in finite-sample settings?
• RQ5 How well does the RGM sampler perform in generating realistic samples on real-world data like MNIST compared to baseline methods?

Key findings

• The RGM distance is shown to equal the Gromov-Wasserstein distance under mild conditions, establishing a theoretical foundation for its use in alignment.
• The RGM sampler achieves a squared distance of approximately 0.719 on the MNIST dataset, with empirical estimates indicating $\mathrm{RGM}^2(\mu,\nu) \approx 0.719$.
• The lower bound on the Gromov-Wasserstein distance (SLB) is estimated at 0.148, implying the RGM distance is at most 2.204 times larger than GW, suggesting strong alignment.
• The method demonstrates effective sample generation on both synthetic and real-world data, with MMD losses below $1.5 \times 10^{-3}$, indicating high distributional fidelity.
• The RGM framework induces bias toward strong isomorphism when $c_{\mathcal{X}}$ and $c_{\mathcal{Y}}$ are properly chosen, aligning with Brenier’s theory of optimal transport.
• The optimization process converges stably, with the RGM objective function achieving a value of 0.719 under empirical estimation, supporting its practical feasibility.