[论文解读] A Primer on the Signature Method in Machine Learning

Ilya Chevyrev, Andrey Kormilitzin|arXiv (Cornell University)|Mar 11, 2016

Advanced Data Compression Techniques参考文献 23被引用 144

一句话总结

这篇论文引入路径特征（path signature） 的概念、基本理论性质以及实际机器学习应用，强调其作为来自多维路径的非参数特征提取器的作用。

ABSTRACT

We provide an introduction to the signature method, focusing on its theoretical properties and machine learning applications. Our presentation is divided into two parts. In the first part, we present the definition and fundamental properties of the signature of a path. The signature is a sequence of numbers associated with a path that captures many of its important analytic and geometric properties. As a sequence of numbers, the signature serves as a compact description (dimension reduction) of a path. In presenting its theoretical properties, we assume only familiarity with classical real analysis and integration, and supplement theory with straightforward examples. We also mention several advanced topics, including the role of the signature in rough path theory. In the second part, we present practical applications of the signature to the area of machine learning. The signature method is a non-parametric way of transforming data into a set of features that can be used in machine learning tasks. In this method, data are converted into multi-dimensional paths, by means of embedding algorithms, of which the signature is then computed. We describe this pipeline in detail, making a link with the properties of the signature presented in the first part. We furthermore review some of the developments of the signature method in machine learning and, as an illustrative example, present a detailed application of the method to handwritten digit classification.

研究动机与目标

• 介绍路径签名的定义及其基本性质。
• 解释签名如何通过迭代积分概括路径信息。
• 讨论对机器学习特征提取的实际影响。
• 提供与粗路径理论和受控微分方程的联系。

提出的方法

• 将路径 X:[a,b]→R^d 的签名定义为无限集合 S(X)^{i1,...,ik}_{a,b} 的所有迭代积分。
• 描述签名的层次（第一层、第二层等）以及对时间重参数化的不变性。
• 给出洗牌积恒等式 S(X)^I S(X)^J = sum_K S(X)^K，以及 Chen 恒等式 S(X*Y) = S(X) ⊗ S(Y)。
• 介绍时间反演性质 S(X) ⊗ S( X̄ ) = 1，以及对数签名作为 log S(X) 的李多项式展开。
• 讨论与粗路径的关系，具体而言，签名如何为有限 p-变路径定义迭代积分及其在求解受控微分方程中的作用。
• 对前两层提供几何直觉（增量和 Lévy 面积），并激励在 ML 中将签名用作特征提取器。

实验结果

研究问题

• RQ1 路径签名具备哪些基本性质（例如对重参数化不变性、Chen 恒等式、洗牌积）？
• RQ2 如何将签名用作多维时间序列的非参数特征提取器以用于 ML 任务？
• RQ3 对数签名如何与李多项式相关及其计算意义？
• RQ4 路径签名与粗路径理论的联系是什么，以及这如何影响求解驱动微分方程？

主要发现

• 签名在路径的时间重参数化下不变。
• 签名的第一层等于路径增量，而更高层编码更丰富的路径信息，如通过 Lévy 项的面积。
• 洗牌积将签名项的乘积表示为高阶项的和，便于对特征进行代数操作。
• Chen 恒等式表明签名将路径的连接转换为张量积，便于模块化路径组合。
• 时间反演产生反向签名关系，且对数签名提供捕捉本质路径几何的李多项式展开。
• 这些性质为在 ML 中将签名用作稳健、可组合的路径特征奠定理论基础。