[Paper Review] Tensor Ring Decomposition

Qibin Zhao, Guoxu Zhou+3 more|arXiv (Cornell University)|Jun 17, 2016

Tensor decomposition and applications53 references303 citations

TL;DR

Introduces tensor ring (TR) decomposition, a circular, permutation-invariant tensor network model that generalizes tensor train and enables multiple learning algorithms for low-rank approximation of high-order tensors. It includes TR-SVD, TR-ALS, adaptive-rank TR-ALS, and block-wise ALS, with experiments on synthetic and real data.

ABSTRACT

Tensor networks have in recent years emerged as the powerful tools for solving the large-scale optimization problems. One of the most popular tensor network is tensor train (TT) decomposition that acts as the building blocks for the complicated tensor networks. However, the TT decomposition highly depends on permutations of tensor dimensions, due to its strictly sequential multilinear products over latent cores, which leads to difficulties in finding the optimal TT representation. In this paper, we introduce a fundamental tensor decomposition model to represent a large dimensional tensor by a circular multilinear products over a sequence of low dimensional cores, which can be graphically interpreted as a cyclic interconnection of 3rd-order tensors, and thus termed as tensor ring (TR) decomposition. The key advantage of TR model is the circular dimensional permutation invariance which is gained by employing the trace operation and treating the latent cores equivalently. TR model can be viewed as a linear combination of TT decompositions, thus obtaining the powerful and generalized representation abilities. For optimization of latent cores, we present four different algorithms based on the sequential SVDs, ALS scheme, and block-wise ALS techniques. Furthermore, the mathematical properties of TR model are investigated, which shows that the basic multilinear algebra can be performed efficiently by using TR representaions and the classical tensor decompositions can be conveniently transformed into the TR representation. Finally, the experiments on both synthetic signals and real-world datasets were conducted to evaluate the performance of different algorithms.

Motivation & Objective

• Motivate the need for a flexible, scalable tensor decomposition beyond TT by addressing permutation sensitivity and rigid rank structures.
• Define the tensor ring (TR) model and its mathematical properties, including circular invariance and trace-based output.
• Develop and compare multiple algorithms (TR-SVD, TR-ALS, TR-ALS with adaptive ranks, and BALS) to learn TR representations.
• Analyze how TR connects to classical decompositions and prove foundational properties.
• Demonstrate empirical performance on synthetic and real-world datasets.

Proposed method

• Define TR decomposition as T(i1,...,id) = Tr{Z1(i1) Z2(i2) ... Zd(id)} with cores Zi of size ri x ni x r(i+1).
• Show circular dimensional permutation invariance via trace, enabling ring-like core interconnections.
• Provide four learning algorithms: TR-SVD (Sequential SVDs) for non-iterative low-rank approximation; TR-ALS (alternating least squares) for fixed ranks; TR-ALS with adaptive ranks (ALSAR) for automatic rank growth; and Block-wise ALS (BALS) with rank adaptation via block merges and truncated SVD.
• Derive relationships to mode-k unfoldings and subchains, and present update rules to optimize cores.
• Discuss how TR reduces to TT under certain conditions and how basic multilinear algebra can be performed with TR representations.

Experimental results

Research questions

• RQ1 How can a tensor be represented in a permutation-invariant, circularly interconnected ring structure to overcome TT limitations?
• RQ2 What learning algorithms can efficiently compute or approximate TR decompositions with controllable accuracy?
• RQ3 How do TR-ranks influence approximation quality, and can ranks be adapted automatically during learning?
• RQ4 How does TR relate to existing decompositions (CP, Tucker, TT) and what are the theoretical properties enabling efficient computation?
• RQ5 Do TR-based methods perform well on synthetic and real-world datasets compared to traditional tensor decompositions?

Key findings

• TR decomposition represents a high-order tensor as a circular sequence of 3rd-order cores using a trace over their product.
• The TR model provides circular dimensional permutation invariance and can be viewed as a linear combination of TT decompositions.
• Four learning algorithms are developed: TR-SVD, TR-ALS, TR-ALS with adaptive ranks, and BALS, enabling various trade-offs between accuracy and computation.
• TR representation requires O(d n r^2) parameters, scalable with tensor order d.
• TR-SVD offers a non-iterative approximation with controlled relative error via mode-1 unfolding and sequential SVDs.
• Experiments on synthetic and real-world datasets validate the proposed algorithms (specific results not provided in the abstract excerpt).