[论文解读] Sum-of-Squares Polynomial Flow

研究动机与目标

• Formulate a rigorous framework for density estimation via increasing triangular maps.
• Unify and compare autoregressive models and normalizing flows within this framework.
• Propose and analyze the Sum-of-Squares (SOS) flow as a universal, interpretable density estimator.
• Show how SOS flows generalize IAF and relate to existing flow methods.
• Demonstrate the efficacy of SOS flows through synthetic and real-world experiments.

提出的方法

• Parameterize triangular maps via one-dimensional increasing polynomials with coefficients produced by a conditioner network: T_j(z_1,...,z_j)=P_{2r+1}(z_j; a_j) where a_j=C_j(z_1,...,z_{j-1}).
• Ensure monotonicity by representing P_{2r+1} as an integral of a sums-of-squares polynomial: P_{2r+1}(z; a)=c+∫_0^z ∑_{κ=1}^k (∑_{l=0}^r a_{l,κ} u^l)^2 du.
• Prove universality: increasing polynomials are dense in the space of increasing continuous functions, enabling approximation of any increasing triangular map as r grows or via stacking blocks.
• Show SOS flows strictly generalize IAF (r=0) and provide interpretable parameters controlling higher-order moments.
• Stack multiple SOS blocks to trade depth vs width and enhance approximation capacity.
• Compare to existing autoregressive and flow-based methods and demonstrate competitive performance on synthetic and real datasets.]
• research_questions:["Can triangular maps provide a complete and tractable framework for high-dimensional density estimation?","Do SOS flows offer universal approximation power for any target density while remaining interpretable and trainable?","How do SOS flows relate to and generalize existing autoregressive and normalizing flow methods?","What are the practical trade-offs (depth vs width) when implementing SOS flows for real-world data?","Do SOS flows perform competitively against state-of-the-art density estimators on standard benchmarks?"]
• key_findings:["SOS flows are universal: with enough model complexity they can approximate any target density.","SOS flows strictly generalize inverse autoregressive flow (IAF) and encompass existing autoregressive and flow-based models within the triangular-map framework.","The polynomial-based conditional densities are monotone and computable via univariate polynomials, enabling efficient density evaluation and inversion.","Coefficients of the polynomials directly control higher-order moments of the target density, enhancing interpretability.","Stacking SOS blocks balances model capacity and training efficiency, with deeper vs wider configurations offering different trade-offs.","Empirical results on synthetic and real-world datasets show SOS flows achieve competitive log-likelihoods compared to several baseline methods.

相关论文