[Paper Review] Tensor network subspace identification of polynomial state space models

Kim Batselier, Ching Yun Ko+1 more|arXiv (Cornell University)|Sep 26, 2017

Tensor decomposition and applications10 references3 citations

TL;DR

This paper proposes a tensor network subspace identification method for polynomial state space models, representing nonlinearities via low-rank tensor networks to avoid the curse of dimensionality. It proves that block Hankel matrices in subspace methods can be exactly represented as low-rank tensor networks, reducing computational and storage costs by up to 20× compared to standard matrix methods, with robust performance on noisy and real-world data.

ABSTRACT

This article introduces a tensor network subspace algorithm for the identification of specific polynomial state space models. The polynomial nonlinearity in the state space model is completely written in terms of a tensor network, thus avoiding the curse of dimensionality. We also prove how the block Hankel data matrices in the subspace method can be exactly represented by low rank tensor networks, reducing the computational and storage complexity significantly. The performance and accuracy of our subspace identification algorithm are illustrated by numerical experiments, showing that our tensor network implementation is around 20 times faster than the standard matrix implementation before the latter fails due to insufficient memory, is robust with respect to noise and can model real-world systems.

Motivation & Objective

• Address the curse of dimensionality in polynomial state space models, where parameter counts grow exponentially with polynomial degree.
• Develop a non-iterative, computationally efficient identification method for polynomial state space models using tensor networks.
• Enable robust system identification under noise and for high-dimensional real-world systems where standard methods fail due to memory or time constraints.
• Demonstrate that tensor network representations can exactly capture the block Hankel data matrices central to subspace methods, enabling low-rank compression.
• Provide a scalable alternative to iterative methods like PNLSS, which suffer from slow convergence and high computational cost.

Proposed method

• Formulate a polynomial state space model with linear state transition and multivariate polynomial input dependence, parameterized by tensors.
• Represent the polynomial functions f(·) and g(·) using tensor networks, enabling compact, low-rank representations that avoid exponential parameter growth.
• Modify the conventional MOESP subspace method to operate directly on tensor network representations of input-output data.
• Prove in Theorem 4.1 that block Hankel data matrices in subspace identification are exactly representable as low-rank tensor networks, enabling exact low-rank compression.
• Implement the algorithm in the TNMOESP MATLAB/Octave package, leveraging tensor train and Tucker formats for efficient computation and storage.
• Use the tensor network structure to perform subspace identification without explicitly forming large dense matrices, thus reducing memory and time complexity.

Experimental results

Research questions

• RQ1 Can tensor networks be used to represent polynomial state space models in a way that avoids the exponential growth of parameters with polynomial degree?
• RQ2 Is it possible to exactly represent the block Hankel data matrices used in subspace identification as low-rank tensor networks?
• RQ3 Does the proposed tensor network subspace method achieve significant reductions in computational and storage complexity compared to standard matrix-based implementations?
• RQ4 How does the performance of the tensor network method compare to iterative methods like PNLSS and Volterra models on real-world systems with noise?
• RQ5 Can the tensor network-based model serve as an effective initial guess for iterative optimization in nonlinear system identification?

Key findings

• The proposed TNMOESP method is approximately 20 times faster than standard matrix-based subspace identification before the latter fails due to memory limitations.
• The method achieves a relative validation error of 0.148 on a real-world valve amplifier system, significantly outperforming a linear state space model (0.418 error).
• The general polynomial state space model identified via the PNLSS toolbox achieved a lower error (0.087) but required 12,317 seconds, highlighting the trade-off between accuracy and computational cost.
• A Volterra model achieved the lowest error (0.004), suggesting the real system may be better described by a Volterra structure than the proposed polynomial state space form.
• The tensor network representation enables exact low-rank compression of block Hankel matrices, as proven in Theorem 4.1, which underpins the method’s efficiency and scalability.
• The method is robust to noise and successfully models real-world nonlinear dynamics, demonstrating practical viability beyond synthetic benchmarks.