[论文解读] Generalized Score Matching for Non-Negative Data

Shiqing Yu, Mathias Drton+1 more|PubMed|Dec 26, 2018

Statistical Methods and Inference参考文献 19被引用 41

一句话总结

本论文将非负数据的分数匹配推广到以在不可观测归一化常数的指数族图模型中改进参数估计，并提出具有理论保证的正则化估计量。

ABSTRACT

A common challenge in estimating parameters of probability density functions is the intractability of the normalizing constant. While in such cases maximum likelihood estimation may be implemented using numerical integration, the approach becomes computationally intensive. The score matching method of Hyvärinen (2005) avoids direct calculation of the normalizing constant and yields closed-form estimates for exponential families of continuous distributions over <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow><mml:msup><mml:mi>R</mml:mi> <mml:mi>m</mml:mi></mml:msup> </mml:mrow> </mml:math> . Hyvärinen (2007) extended the approach to distributions supported on the non-negative orthant, <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow><mml:msubsup><mml:mi>R</mml:mi> <mml:mo>+</mml:mo> <mml:mi>m</mml:mi></mml:msubsup> </mml:mrow> </mml:math> . In this paper, we give a generalized form of score matching for non-negative data that improves estimation efficiency. As an example, we consider a general class of pairwise interaction models. Addressing an overlooked inexistence problem, we generalize the regularized score matching method of Lin et al. (2016) and improve its theoretical guarantees for non-negative Gaussian graphical models.

研究动机与目标

• Motivate the challenge of estimating densities when normalizing constants are intractable.
• Extend Hyvärinen’s non-negative score matching by introducing a generalized h-score matching framework.
• Develop regularized generalized score matching for high-dimensional graphical models.
• Apply the method to pairwise interaction power models on R_+^m and establish theoretical guarantees.
• Demonstrate consistency and practical performance through simulations and RNA-seq data analysis.

提出的方法

• Define generalized h-score matching loss J_h for non-negative data with positive, componentwise functions h_j.
• Prove that J_h is minimized uniquely by P_0 under mild conditions and can be rewritten as an expectation independent of p_0.
• Show that for exponential families, the empirical loss is a quadratic function in the canonical parameter θ, enabling closed-form estimation.
• Introduce regularization by adding a diagonal augmentation to the quadratic form to ensure strong convexity in high dimensions.
• Derive a regularized estimator with an l1 penalty that yields a unique minimizer and analyze its consistency.
• Discuss special cases including univariate truncated normals to illustrate the estimator and its asymptotic properties.

实验结果

研究问题

• RQ1 How can score matching be generalized to efficiently handle non-negative data with intractable normalizing constants?
• RQ2 What choices of the boundary-dampening functions h_j improve estimation efficiency for non-negative graphical models?
• RQ3 Can regularized generalized score matching yield consistent estimators for high-dimensional non-negative graphical models?
• RQ4 How does the generalized score matching perform on pairwise interaction power models on R_+^m and related truncated GGMs?
• RQ5 What are the theoretical and empirical properties (consistency, asymptotic distribution, robustness) of the proposed estimators?

主要发现

• The generalized h-score matching loss provides a non-negative, boundary-damped objective whose minimizer is uniquely P_0 under mild conditions.
• For exponential families, the empirical loss is quadratic in the canonical parameters, enabling closed-form estimators without computing normalizing constants.
• Adding small diagonal augmentation to the quadratic form yields a bounded, strongly convex loss in high dimensions, preserving consistency under a threshold.
• Regularized generalized score matching with an l1 penalty yields a unique minimizer and supports high-dimensional estimation for non-negative graphical models.
• Special cases illustrate consistent estimation for univariate truncated normals and demonstrate improved efficiency when using bounded or slowly growing h functions.
• The methodology extends to a broad class of pairwise interaction models on R_+^m, including truncated Gaussian graphical models and square-root models, with theoretical guarantees and empirical validation.