[Paper Review] Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU

Patrick Kidger, Terry Lyons|arXiv (Cornell University)|Jan 3, 2020

Parallel Computing and Optimization Techniques35 references51 citations

TL;DR

Signatory is a CPU- and GPU-capable library for differentiable computations of the signature and logsignature transforms, including novel algorithmic improvements and precomputation strategies, with PyTorch integration.

ABSTRACT

Signatory is a library for calculating and performing functionality related to the signature and logsignature transforms. The focus is on machine learning, and as such includes features such as CPU parallelism, GPU support, and backpropagation. To our knowledge it is the first GPU-capable library for these operations. Signatory implements new features not available in previous libraries, such as efficient precomputation strategies. Furthermore, several novel algorithmic improvements are introduced, producing substantial real-world speedups even on the CPU without parallelism. The library operates as a Python wrapper around C++, and is compatible with the PyTorch ecosystem. It may be installed directly via exttt{pip}. Source code, documentation, examples, benchmarks and tests may be found at exttt{\url{https://github.com/patrick-kidger/signatory}}. The license is Apache-2.0.

Motivation & Objective

• Introduce Signatory, a CPU- and GPU-capable library for computing the signature and logsignature transforms.
• Showcase practical speedups and new algorithmic techniques for differentiable execution in ML workflows.
• Provide preprocessing and precomputation strategies for efficient querying over intervals.
• Demonstrate integration with PyTorch and backpropagation through all transforms.

Proposed method

• Develop fused multiply-exponentiate to reduce scalar multiplications and asymptotic complexity.
• Propose O(L) precomputation strategy to answer arbitrary interval queries in O(1) time.
• Introduce a more efficient logsignature basis beyond Lyndon bases for faster computation.
• Implement parallel CPU (OpenMP) and GPU support via LibTorch without custom CUDA code.
• Provide hand-written backpropagation for speed and memory efficiency.

Experimental results

Research questions

• RQ1 Can the signature and logsignature transforms be computed differentiably on both CPU and GPU with practical speedups?
• RQ2 Do new algorithmic techniques (fused multiply-exponentiate, precomputation, alternative bases) yield real-world performance gains over existing libraries?
• RQ3 How can interval queries of (log)signatures be answered efficiently in linear or constant time?
• RQ4 Is backward propagation through these transforms feasible and efficient for deep learning workflows?
• RQ5 How well does Signatory integrate with PyTorch and support end-to-end training?

Key findings

• Signatory achieves substantial speedups over previous CPU implementations and over GPU execution compared to alternatives like iisignature.
• GPU support enables large performance gains by reducing data transfers between CPU and GPU during training.
• Backpropagation is fully supported via handwritten gradients for speed and memory efficiency.
• Precomputation enables O(1) queries for arbitrary intervals after O(L) preprocessing.
• A novel logsignature basis offers computational efficiency improvements over traditional Lyndon-based bases.