[Paper Review] A structural model on a hypercube represented by optimal transport

Tomonari Sei|ArXiv.org|Jan 29, 2009

Markov Chains and Monte Carlo Methods16 references3 citations

TL;DR

This paper proposes the Structural Gradient Model (SGM), a flexible statistical model for high-dimensional data on the hypercube [0,1]^m, based on optimal transport via a potential function represented by Fourier series. The model captures complex dependence structures including higher-order interactions and heteroscedasticity, with maximum likelihood estimation solvable via determinant maximization and lasso-type regularization, outperforming graphical Gaussian and mixture models in simulation and real-data studies.

ABSTRACT

We propose a flexible statistical model for high-dimensional quantitative data on a hypercube. Our model, called the structural gradient model (SGM), is based on a one-to-one map on the hypercube that is a solution for an optimal transport problem. As we show with many examples, SGM can describe various dependence structures including correlation and heteroscedasticity. The maximum likelihood estimation of SGM is effectively solved by the determinant-maximization programming. In particular, a lasso-type estimation is available by adding constraints. SGM is compared with graphical Gaussian models and mixture models.

Motivation & Objective

• To develop a flexible statistical model for high-dimensional quantitative data on the hypercube [0,1]^m that captures complex dependence beyond second-order moments.
• To address limitations of graphical Gaussian models, which cannot represent heteroscedasticity or higher-order interactions, by introducing a transport-based model rooted in optimal transport theory.
• To enable efficient maximum likelihood estimation of the model through determinant maximization with convex constraints, including a lasso-type estimator via L1-conservative regions.
• To compare SGM empirically with graphical Gaussian models and mixture models in terms of model fit and variable selection performance.

Proposed method

• The model is defined by a probability density p(x|θ) = det(D²ψ(x|θ)) on [0,1]^m, where ψ is a convex potential function with Fourier series representation.
• The potential function ψ(x|θ) is parameterized by Fourier coefficients θ_u, with the Hessian D²ψ(x|θ) forming a positive semi-definite matrix for feasible θ.
• Feasible parameter space Θ is defined by the condition D²ψ(x|θ) ⪰ 0 for all x ∈ [0,1]^m, ensuring p(x|θ) is a valid density.
• Maximum likelihood estimation is reformulated as a determinant maximization problem over a convex feasible region, solvable via interior-point methods.
• A sequence of inner approximations and an L1-conservative region are proposed to handle the infinite-dimensional constraint set, enabling practical computation.
• A lasso-type estimator is derived by adding L1 constraints to the determinant maximization, promoting sparsity in the Fourier coefficients.

Experimental results

Research questions

• RQ1 Can a transport-based model on the hypercube effectively capture higher-order and heteroscedastic dependence structures beyond what graphical Gaussian models can represent?
• RQ2 How can maximum likelihood estimation be efficiently performed for a model defined by a determinant of a Hessian matrix with infinitely many constraints?
• RQ3 Can a lasso-type estimator be constructed for this model to enable variable selection and sparsity in high-dimensional settings?
• RQ4 How does the SGM compare in performance to graphical Gaussian models and mixture models in terms of model fit and structure recovery?

Key findings

• The SGM model successfully captures complex dependence structures, including higher-order interactions and heteroscedasticity, which graphical Gaussian models cannot represent.
• Maximum likelihood estimation is effectively solved via determinant maximization, with convergence guaranteed by convex optimization techniques.
• A lasso-type estimator is derived by introducing L1 constraints, enabling sparse estimation of Fourier coefficients and variable selection.
• The model is robust to data transformation, as real-valued data can be mapped to [0,1]^m via a sigmoid function without requiring uniform marginals.
• Numerical experiments show SGM outperforms graphical Gaussian and mixture models in fitting complex dependence patterns in both simulated and real data.
• Theoretical guarantees are provided, including that any probability density on [0,1]^m can be represented in the form det(D²ψ(x)) via optimal transport, with ψ convex and Dψ bijective.