[Paper Review] Tensor Network alternating linear scheme for MIMO Volterra system identification

Kim Batselier, Zhongming Chen+1 more|arXiv (Cornell University)|Jul 1, 2016

Tensor decomposition and applications20 references3 citations

TL;DR

This paper proposes two tensor network-based alternating linear schemes (ALS and MALS) for efficient identification of high-order MIMO Volterra systems, avoiding the curse of dimensionality by representing Volterra kernels in a compact Tucker-type tensor train (TT) format. The methods iteratively optimize orthogonal TT-cores via low-complexity linear solves with SVD and QR, achieving degree-10 MIMO Volterra models in under 2 seconds on a standard PC with high accuracy and monotonic convergence.

ABSTRACT

This article introduces two Tensor Network-based iterative algorithms for the identification of high-order discrete-time nonlinear multiple-input multiple-output (MIMO) Volterra systems. The system identification problem is rewritten in terms of a Volterra tensor, which is never explicitly constructed, thus avoiding the curse of dimensionality. It is shown how each iteration of the two identification algorithms involves solving a linear system of low computational complexity. The proposed algorithms are guaranteed to monotonically converge and numerical stability is ensured through the use of orthogonal matrix factorizations. The performance and accuracy of the two identification algorithms are illustrated by numerical experiments, where accurate degree-10 MIMO Volterra models are identified in about 1 second in Matlab on a standard desktop pc.

Motivation & Objective

• Address the curse of dimensionality in high-order MIMO Volterra system identification, where kernel storage and computation scale exponentially with order.
• Overcome limitations of canonical polyadic and Tucker decompositions by introducing a tensor network (TN) framework based on tensor trains (TT) for stable, low-rank representation.
• Develop two iterative algorithms—ALS and MALS—that identify Volterra kernels in TT format from input-output data while ensuring numerical stability and monotonic convergence.
• Demonstrate the feasibility of identifying high-degree (up to 10) MIMO Volterra models in real-time on standard hardware, enabling practical application in nonlinear system modeling.

Proposed method

• Represent the Volterra tensor in a compact tensor train (TT) format, storing all kernel coefficients implicitly via low-rank core tensors to avoid explicit construction.
• Use alternating least squares (ALS) to iteratively optimize each TT-core while keeping others fixed, solving small linear systems via QR and SVD for numerical stability.
• Introduce a modified ALS (MALS) that adaptively updates TT-ranks during iterations using SVD, allowing for better model complexity selection.
• Enforce orthogonality of TT-cores through QR and SVD factorizations at each step to ensure numerical stability and prevent ill-conditioning.
• Formulate the identification problem as a nonlinear least-squares minimization over the TT format, with convergence guaranteed under standard conditions.
• Apply the algorithms to input-output data, updating TT-cores until a desired residual tolerance is reached, with runtime and accuracy monitored per iteration.

Experimental results

Research questions

• RQ1 Can tensor network formats such as tensor trains (TT) be effectively used to represent high-order MIMO Volterra kernels without explicitly constructing the full tensor?
• RQ2 Can iterative alternating linear schemes (ALS/MALS) be designed to identify MIMO Volterra systems efficiently in TT format while ensuring monotonic convergence and numerical stability?
• RQ3 How do the proposed algorithms perform in terms of computational speed and accuracy when identifying high-order (e.g., degree 10) MIMO Volterra systems on standard hardware?
• RQ4 What is the impact of noise on the identification performance, and can the MALS algorithm adaptively adjust TT-ranks to balance model accuracy and complexity?
• RQ5 Can the TT-based framework achieve sub-second identification of degree-10 MIMO Volterra models, enabling real-time or near-real-time system identification?

Key findings

• The proposed ALS and MALS algorithms successfully identify degree-10 MIMO Volterra systems in approximately 1 second using standard desktop hardware, demonstrating exceptional computational efficiency.
• The ALS method achieves a relative residual of 0.052 and a simulated output SNR of 37 dB on a 25 dB input SNR signal, indicating high fidelity in noisy conditions.
• The MALS method outperforms ALS in convergence speed, completing identification in 1.1–1.3 seconds across all SNR levels, with consistent SNR improvement in simulated outputs.
• The MALS algorithm adapts TT-ranks dynamically, increasing them under high noise, but risks overfitting when data are insufficient, indicating a need for regularization.
• Both algorithms ensure monotonic convergence and numerical stability through orthogonal TT-core updates using SVD and QR, avoiding ill-conditioning.
• The TT format reduces storage and computation from exponential $(pM)^d$ to $O(d(pM+1)r^2)$, effectively breaking the curse of dimensionality for high-order MIMO Volterra systems.