[Paper Review] T product Tensors Part I: Inequalities

Shih Yu Chang, Yimin Wei|arXiv (Cornell University)|Jul 13, 2021

Tensor decomposition and applications23 references4 citations

TL;DR

This paper establishes foundational inequalities for T-product tensors, including trace function monotonicity, Golden-Thompson, Jensen’s, and Klein’s inequalities, generalizing Lieb’s concavity theorem to T-product tensors. It introduces a Courant-Fischer theorem for eigentuples and derives master tail bounds for the maximum eigenvalue and eigentuple of sums of independent random Hermitian T-product tensors, enabling new Chernoff and Bernstein-type bounds in Part II.

ABSTRACT

The T product operation between two three order tensors was invented around 2011 and it arises from many applications, such as signal processing, image feature extraction, machine learning, computer vision, and the multiview clustering problem. Although there are many pioneer works about T product tensors, there are no works dedicated to inequalities associated with T product tensors. In this work, we first attempt to build inequalities at the following aspects: (1) trace function nondecreasing and convexity; (2) Golden Thompson inequality for T product tensors; (3) Jensen T product inequality; (4) Klein T product inequality. All these inequalities are related to generalize celebrated Lieb concavity theorem from matrices to T product tensors. This new version of Lieb concavity theorem under T product tensor will be used to determine the tail bound for the maximum eigenvalue induced by independent sums of random Hermitian T product, which is the key tool to derive various new tail bounds for random T product tensors. Besides, Qi et. al introduces a new concept, named eigentuple, about T product tensors and they apply this concept to study nonnegative (positive) definite properties of T product tensors. The final main contribution of this work is to develop the Courant Fischer Theorem with respect to eigentuples, and this theorem helps us to understand the relationship between the minimum eigentuple and the maximum eigentuple. The main content of this paper is Part I of a serious task about T product tensors. The Part II of this work will utilize these new inequalities and Courant Fischer Theorem under T product tensors to derive tail bounds of the extreme eigenvalue and the maximum eigentuple for sums of random T product tensors, e.g., T product tensor Chernoff and T product tensor Bernstein bounds.

Motivation & Objective

• To develop a comprehensive theory of inequalities for T-product tensors, extending classical matrix inequalities to the tensor setting.
• To generalize Lieb’s concavity theorem to T-product tensors, enabling the analysis of non-linear functions of random T-product tensors.
• To establish a Courant-Fischer theorem for eigentuples to characterize the minimum and maximum eigentuples of T-product tensors.
• To lay the theoretical groundwork for deriving tail bounds on the extreme eigenvalues and eigentuples of sums of independent random Hermitian T-product tensors.
• To provide a foundation for Part II, which derives T-product tensor Chernoff and Bernstein bounds using the inequalities and theorems developed here.

Proposed method

• Define the trace of a T-product tensor as the sum of its f-diagonal entries and prove monotonicity and convexity of the trace function under continuous real functions.
• Prove the Golden-Thompson inequality for Hermitian T-product tensors: Tr(exp(𝒞 + 𝒟)) ≤ Tr(exp(𝒞) ⋆ exp(𝒟)) for 𝒞, 𝒟 ∈ ℂ^{m×m×p}.
• Establish Jensen’s T-product inequality: f(∑ᵢ 𝒞ᵢᴴ ⋆ 𝒳ᵢ ⋆ 𝒞ᵢ) ⪯ ∑ᵢ 𝒞ᵢᴴ ⋆ f(𝒳ᵢ) ⋆ 𝒞ᵢ for T-convex functions f and orthogonal T-product tensor frames.
• Prove Klein’s T-product inequality, which generalizes the classical Klein inequality to the T-product tensor setting.
• Introduce the concept of eigentuples for T-product tensors and define T-positive (semi-)definiteness via the smallest eigentuple.
• Establish a Courant-Fischer-type theorem for T-product tensors, characterizing the k-th largest eigentuple as a min-max over k-dimensional subspaces of the tensor space.

Experimental results

Research questions

• RQ1 How can classical matrix inequalities such as Golden-Thompson and Jensen’s be generalized to the T-product tensor framework?
• RQ2 Can Lieb’s concavity theorem be extended to T-product tensors, and what are the implications for the analysis of random T-product tensors?
• RQ3 What is the relationship between the minimum and maximum eigentuples of a T-product tensor, and how can it be characterized via extremal principles?
• RQ4 How can the Courant-Fischer theorem be adapted to T-product tensors to describe extremal eigentuples?
• RQ5 What tail bounds can be derived for the maximum eigenvalue and eigentuple of sums of independent random Hermitian T-product tensors using the developed inequalities?

Key findings

• The trace function Tr(f(𝒞)) is non-decreasing and convex when f is non-decreasing or convex, respectively, for T-product tensors.
• The Golden-Thompson inequality holds for Hermitian T-product tensors: Tr(exp(𝒞 + 𝒟)) ≤ Tr(exp(𝒞) ⋆ exp(𝒟)).
• Jensen’s T-product inequality is established for T-convex functions, showing f(∑ᵢ 𝒞ᵢᴴ ⋆ 𝒳ᵢ ⋆ 𝒞ᵢ) ⪯ ∑ᵢ 𝒞ᵢᴴ ⋆ f(𝒳ᵢ) ⋆ 𝒞ᵢ under the constraint ∑ᵢ 𝒞ᵢᴴ ⋆ 𝒞ᵢ = 𝒫.
• Klein’s T-product inequality is proven, extending the classical inequality to the T-product tensor setting.
• A Courant-Fischer theorem for T-product tensors is established, characterizing the k-th largest eigentuple as a min-max over k-dimensional subspaces.
• The relationship between the minimum and maximum eigentuples is formalized: d_min(𝒳) = −d_max(−𝒳), and similarly for eigenvalues.