[Paper Review] Rough paths, Signatures and the modelling of functions on streams

Terry Lyons|arXiv (Cornell University)|May 18, 2014

Image and Signal Denoising MethodsComputer Science19 references63 citations

TL;DR

This paper introduces a novel framework for modeling functions on streams using rough path theory and signature-based features, enabling effective machine learning on complex, high-oscillatory data. By leveraging the signature transform as a universal, faithful feature map, the method enables linear regression on path laws, achieving high accuracy (AUC > 0.98) in classifying financial time series by time-of-day using only low-order signature components.

ABSTRACT

Rough path theory is focused on capturing and making precise the interactions between highly oscillatory and non-linear systems. It draws on the analysis of LC Young and the geometric algebra of KT Chen. The concepts and the uniform estimates, have widespread application and have simplified proofs of basic questions from the large deviation theory and extended Ito's theory of SDEs; the recent applications contribute to (Graham) automated recognition of Chinese handwriting and (Hairer) formulation of appropriate SPDEs to model randomly evolving interfaces. At the heart of the mathematics is the challenge of describing a smooth but potentially highly oscillatory and vector valued path $x_{t}$ parsimoniously so as to effectively predict the response of a nonlinear system such as $dy_{t}=f(y_{t})dx_{t}$, $y_{0}=a$. The Signature is a homomorphism from the monoid of paths into the grouplike elements of a closed tensor algebra. It provides a graduated summary of the path $x$. Hambly and Lyons have shown that this non-commutative transform is faithful for paths of bounded variation up to appropriate null modifications. Among paths of bounded variation with given Signature there is always a unique shortest representative. These graduated summaries or features of a path are at the heart of the definition of a rough path; locally they remove the need to look at the fine structure of the path. Taylor's theorem explains how any smooth function can, locally, be expressed as a linear combination of certain special functions (monomials based at that point). Coordinate iterated integrals form a more subtle algebra of features that can describe a stream or path in an analogous way; they allow a definition of rough path and a natural linear "basis" for functions on streams that can be used for machine learning.

Motivation & Objective

• To develop a general-purpose, data-agnostic method for summarizing and modeling complex, evolving streams of information.
• To apply rough path theory and signature transforms to enable effective machine learning on highly oscillatory, non-linear data streams.
• To demonstrate that signature-based features can capture essential path characteristics for functional regression and classification, even when raw data is normalized or de-noised.
• To establish a theoretical and practical bridge between stochastic analysis, functional regression, and machine learning via the expected signature and characteristic functions of signatures.

Proposed method

• The signature of a path is used as a universal, non-commutative feature map that encodes the full history of a stream in a graduated, algebraically structured way.
• Coordinate iterated integrals are used to compute the signature, forming a natural linear basis for functions on streams.
• Linear regression is applied to signature components to approximate functions on paths, with LASSO regularization used to select significant features.
• Expected signatures and characteristic functions of signatures are used to model conditional laws of paths, enabling regression on path distributions.
• The method leverages the faithfulness of the signature transform for bounded variation paths and its ability to capture path effects without fine-scale detail.
• A practical pipeline is implemented: normalize financial time series, compute low-order signature features, apply LASSO-regularized linear regression, and evaluate using ROC, K-S distance, and classification accuracy.

Experimental results

Research questions

• RQ1 Can the signature transform serve as a universal, faithful feature representation for arbitrary smooth, oscillatory paths in a way that enables effective machine learning?
• RQ2 To what extent can linear regression on signature components capture complex, non-linear dependencies in time series data, such as financial market patterns?
• RQ3 How well can signature-based features distinguish between different temporal regimes in financial data, even after normalization to remove volume and volatility effects?
• RQ4 Can the expected signature provide a tractable, low-dimensional approximation to the conditional law of a path given another path?
• RQ5 What is the performance of signature-based models in real-world classification tasks involving high-dimensional, noisy, and non-Markovian data streams?

Key findings

• The method achieved a Kolmogorov-Smirnov distance of 0.84 on out-of-sample data, indicating strong separation between time buckets.
• The area under the ROC curve reached 0.986 on out-of-sample data, demonstrating excellent discrimination power.
• Correct classification accuracy was 89% on out-of-sample data, confirming robust generalization.
• LASSO-based feature selection identified a small subset of signature components that enabled clear visual separation of time buckets in 2D projections.
• The expected signature and characteristic functions of signatures were shown to characterize the law of a path, enabling linear regression on path distributions.
• The signature transform was found to be a faithful representation for bounded variation paths, with a unique shortest representative for each signature.