[论文解读] Majorizing Measures for the Optimizer

Sander Borst, Daniel Dadush+2 more|arXiv (Cornell University)|Dec 24, 2020

Advanced Optimization Algorithms Research参考文献 22被引用 1

一句话总结

本文提出了一种基于优化的新框架，用于主控测度（majorizing measures）的计算，将最优主控测度的计算重新表述为一个凸规划问题。通过利用凸对偶性和舍入技术，该框架提供了塔拉格兰德主控测度定理的算法化证明，推导出紧致的上下界，为先前基于组合方法（如通用链）的理论提供了更直接、高效且概念更清晰的替代方案。

ABSTRACT

The theory of majorizing measures, extensively developed by Fernique, Talagrand and many others, provides one of the most general frameworks for controlling the behavior of stochastic processes. In particular, it can be applied to derive quantitative bounds on the expected suprema and the degree of continuity of sample paths for many processes. One of the crowning achievements of the theory is Talagrand’s tight alternative characterization of the suprema of Gaussian processes in terms of majorizing measures. The proof of this theorem was difficult, and thus considerable effort was put into the task of developing both shorter and easier to understand proofs. A major reason for this difficulty was considered to be theory of majorizing measures itself, which had the reputation of being opaque and mysterious. As a consequence, most recent treatments of the theory (including by Talagrand himself) have eschewed the use of majorizing measures in favor of a purely combinatorial approach (the generic chaining) where objects based on sequences of partitions provide roughly matching upper and lower bounds on the desired expected supremum. In this paper, we return to majorizing measures as a primary object of study, and give a viewpoint that we think is natural and clarifying from an optimization perspective. As our main contribution, we give an algorithmic proof of the majorizing measures theorem based on two parts: - We make the simple (but apparently new) observation that finding the best majorizing measure can be cast as a convex program. This also allows for efficiently computing the measure using off-the-shelf methods from convex optimization. - We obtain tree-based upper and lower bound certificates by rounding, in a series of steps, the primal and dual solutions to this convex program. While duality has conceptually been part of the theory since its beginnings, as far as we are aware no explicit link to convex optimization has been previously made.

研究动机与目标

• 通过凸优化重新构建主控测度理论，提供更清晰、更系统化的基础。
• 利用凸规划的原始-对偶解，提供塔拉格兰德主控测度定理的算法化证明。
• 利用现成的凸优化求解器，实现最优主控测度的高效计算。
• 通过原始和对偶解的舍入，推导出紧致的上下界证明，取代以往的临时组合构造。

提出的方法

• 将寻找最优主控测度的问题表述为一个凸规划问题，目标是在索引集 X 上的概率测度上最小化一个泛函。
• 利用凸对偶性，推导出对偶解，从而获得高斯过程期望上确界的下界。
• 通过分层树结构对凸规划的原始解进行舍入，构造上界证明。
• 对对偶解应用一种舍入方案，生成下界证明，确保与索引集的度量结构一致。
• 利用 Dadush、Guzmán 和 Olver 提出的悲观估计器框架，对 Johnson-Lindenstrauss 类型的投影进行去随机化。
• 建立可适配网（admissible nets）与主控测度之间的联系，通过凸优化实现类似链式结构的确定性构造。

实验结果

研究问题

• RQ1 最优主控测度的计算能否被形式化为一个凸优化问题？
• RQ2 能否利用凸对偶性和舍入技术，推导出高斯过程期望上确界的构造性上下界？
• RQ3 与经典的通用链方法相比，该凸优化框架在效率和清晰度方面表现如何？
• RQ4 该框架能否用于对现有概率构造（如戈登定理中的构造）进行去随机化？
• RQ5 使用该方法构造主控测度及其相关界证明的计算复杂度是多少？

主要发现

• 通过求解凸规划，最优主控测度可以高效计算，使得使用标准优化工具实现实际计算成为可能。
• 凸规划的原始解通过基于树的舍入过程，可产生上界证明。
• 对偶解通过类似的舍入机制提供下界证明，确保界值的紧致性。
• 该框架在主控测度与凸优化之间建立了直接且明确的联系，解决了该理论长期存在的概念模糊性问题。
• 该方法能够实现满足戈登定理的确定性投影构造，优于随机构造。
• 该方法在近似质量与计算效率之间实现了近乎最优的权衡，实际中具有接近线性时间算法的潜力。