[论文解读] Stochastic control liaisons: Richard Sinkhorn meets Gaspard Monge on a Schroedinger bridge

Yongxin Chen, Tryphon T. Georgiou+1 more|arXiv (Cornell University)|May 22, 2020

Topological and Geometric Data Analysis参考文献 197被引用 5

一句话总结

本文通过展示具有均匀先验的Schrödinger桥问题即为具有二次代价的最优质量传输（OMT）问题的正则化版本，且Sinkhorn算法是Fortet连续迭代方法的离散对应，建立了一个统一框架，将Schrödinger桥问题、最优质量传输（OMT）与随机控制联系起来。关键贡献在于通过随机控制与自由能最小化，严格连接了这些经典问题与现代计算方法。

ABSTRACT

In 1931/32, Schroedinger studied a hot gas Gedankenexperiment, an instance of large deviations of the empirical distribution and an early example of the so-called maximum entropy inference method. This so-called Schroedinger bridge problem (SBP) was recently recognized as a regularization of the Monge-Kantorovich Optimal Mass Transport (OMT), leading to effective computation of the latter. Specifically, OMT with quadratic cost may be viewed as a zero-temperature limit of SBP, which amounts to minimization of the Helmholtz's free energy over probability distributions constrained to possess given marginals. The problem features a delicate compromise, mediated by a temperature parameter, between minimizing the internal energy and maximizing the entropy. These concepts are central to a rapidly expanding area of modern science dealing with the so-called {\em Sinkhorn algorithm} which appears as a special case of an algorithm first studied by the French analyst Robert Fortet in 1938/40 specifically for Schroedinger bridges. Due to the constraint on end-point distributions, dynamic programming is not a suitable tool to attack these problems. Instead, Fortet's iterative algorithm and its discrete counterpart, the Sinkhorn iteration, permit computation by iteratively solving the so-called {\em Schroedinger system}. In both the continuous as well as the discrete-time and space settings, {\em stochastic control} provides a reformulation and dynamic versions of these problems. The formalism behind these control problems have attracted attention as they lead to a variety of new applications in spacecraft guidance, control of robot or biological swarms, sensing, active cooling, network routing as well as in computer and data science. This multifacet and versatile framework, intertwining SBP and OMT, provides the substrate for a historical and technical overview of the field taken up in this paper.

研究动机与目标

• 通过随机控制形式化，统一Schrödinger桥问题与最优质量传输（OMT）的理论基础。
• 阐明Erwin Schrödinger 1930年代的工作、Robert Fortet 1940年代的连续迭代方法与Richard Sinkhorn 1960年代的离散算法之间的历史与技术联系。
• 证明Schrödinger桥问题在零温度极限下可恢复Benamou-Brenier的OMT动态公式化。
• 证明IPF-Sinkhorn算法在希尔伯特射影度量下收敛，将Fortet的连续收敛证明推广至离散情形。
• 将随机控制定位为推广桥问题的自然框架，并实现其在机器人学、网络路由与航天器制导中的应用。

提出的方法

• 将Schrödinger桥问题表述为在边缘约束下最小化Helmholtz自由能的随机控制问题。
• 通过条件期望与鞅测度的变分问题推导出Schrödinger系统。
• 应用最小能量原理并引入熵正则化，其中温度参数控制内能与熵之间的权衡。
• 利用希尔伯特射影度量证明IPF-Sinkhorn算法在离散情形下的收敛性，将Fortet的连续收敛结果推广至离散设置。
• 将具有二次代价的OMT问题重新解释为Schrödinger桥问题在零温度极限下的形式。
• 通过路径空间上的受限Gibbs变分原理，建立正则化OMT问题与离散Schrödinger桥问题之间的等价性。

实验结果

研究问题

• RQ1 Schrödinger桥问题在零温度极限下如何与具有二次代价的最优质量传输相关联？
• RQ2 Sinkhorn算法与Fortet在连续空间中求解Schrödinger桥问题的迭代方法之间有何联系？
• RQ3 为何动态规划不适用于Schrödinger桥问题？何种替代框架可实现其求解？
• RQ4 Schrödinger桥问题的随机控制形式化如何实现网络路由与集群控制等领域的推广与应用？
• RQ5 自由能泛函在统一统计力学、信息论与最优传输中的作用是什么？

主要发现

• 具有均匀先验的Schrödinger桥问题等价于具有二次代价的Monge-Kantorovich最优传输问题的正则化版本。
• Schrödinger桥问题在零温度极限下恢复了Benamou-Brenier的最优传输动态公式化，该公式刻画了McCann位移插值流。
• IPF-Sinkhorn算法在希尔伯特射影度量下收敛，将Fortet于1940年提出的连续收敛证明推广至离散情形。
• Schrödinger系统的迭代解对应于在边缘约束下最小化Helmholtz自由能，从而将统计力学与信息论相联系。
• 随机控制框架为桥问题的推广提供了自然途径，使其实现于鲁棒网络路由与主动冷却等应用中。
• 该形式化揭示：Schrödinger桥问题中最可能的路径在时间反演下保持不变，而这一性质在OMT问题中并不成立。