[论文解读] T product Tensors Part I: Inequalities

Shih Yu Chang, Yimin Wei|arXiv (Cornell University)|Jul 13, 2021

Tensor decomposition and applications参考文献 23被引用 4

一句话总结

本文建立了T-积张量的基础不等式，包括迹函数单调性、Golden-Thompson、Jensen以及Klein不等式，将Lieb的凹性定理推广至T-积张量。引入了关于特征元组的Courant-Fischer定理，并推导出独立随机厄米特T-积张量和的最大特征值与特征元组的主尾部界，为第二部分中新型Chernoff与Bernstein型界提供了支持。

ABSTRACT

The T product operation between two three order tensors was invented around 2011 and it arises from many applications, such as signal processing, image feature extraction, machine learning, computer vision, and the multiview clustering problem. Although there are many pioneer works about T product tensors, there are no works dedicated to inequalities associated with T product tensors. In this work, we first attempt to build inequalities at the following aspects: (1) trace function nondecreasing and convexity; (2) Golden Thompson inequality for T product tensors; (3) Jensen T product inequality; (4) Klein T product inequality. All these inequalities are related to generalize celebrated Lieb concavity theorem from matrices to T product tensors. This new version of Lieb concavity theorem under T product tensor will be used to determine the tail bound for the maximum eigenvalue induced by independent sums of random Hermitian T product, which is the key tool to derive various new tail bounds for random T product tensors. Besides, Qi et. al introduces a new concept, named eigentuple, about T product tensors and they apply this concept to study nonnegative (positive) definite properties of T product tensors. The final main contribution of this work is to develop the Courant Fischer Theorem with respect to eigentuples, and this theorem helps us to understand the relationship between the minimum eigentuple and the maximum eigentuple. The main content of this paper is Part I of a serious task about T product tensors. The Part II of this work will utilize these new inequalities and Courant Fischer Theorem under T product tensors to derive tail bounds of the extreme eigenvalue and the maximum eigentuple for sums of random T product tensors, e.g., T product tensor Chernoff and T product tensor Bernstein bounds.

研究动机与目标

• 开发T-积张量不等式的综合理论，将经典矩阵不等式推广至张量框架。
• 将Lieb的凹性定理推广至T-积张量，以支持对随机T-积张量的非线性函数的分析。
• 建立关于特征元组的Courant-Fischer定理，以刻画T-积张量的最小与最大特征元组。
• 为推导独立随机厄米特T-积张量和的极端特征值与特征元组的尾部界奠定理论基础。
• 为第二部分提供基础，该部分利用此处建立的不等式与定理，推导T-积张量的Chernoff与Bernstein界。

提出的方法

• 将T-积张量的迹定义为其中f-对角线元素之和，并证明在连续实函数下迹函数的单调性与凸性。
• 证明厄米特T-积张量的Golden-Thompson不等式：对𝒞, 𝒟 ∈ ℂ^{m×m×p}，有Tr(exp(𝒞 + 𝒟)) ≤ Tr(exp(𝒞) ⋆ exp(𝒟))。
• 建立Jensen的T-积不等式：对T-凸函数f与正交T-积张量框架，有f(∑ᵢ 𝒞ᵢᴴ ⋆ 𝒳ᵢ ⋆ 𝒞ᵢ) ⪯ ∑ᵢ 𝒞ᵢᴴ ⋆ f(𝒳ᵢ) ⋆ 𝒞ᵢ，其中约束条件为∑ᵢ 𝒞ᵢᴴ ⋆ 𝒞ᵢ = 𝒫。
• 证明Klein的T-积不等式，将经典Klein不等式推广至T-积张量框架。
• 引入T-积张量的特征元组概念，并通过最小特征元组定义T-正定（半正定）。
• 建立T-积张量的Courant-Fischer型定理，将第k个最大特征元组表征为张量空间中k维子空间上的极小极大值。

实验结果

研究问题

• RQ1 经典矩阵不等式（如Golden-Thompson与Jensen不等式）如何推广至T-积张量框架？
• RQ2 Lieb的凹性定理能否扩展至T-积张量？其对随机T-积张量分析有何影响？
• RQ3 T-积张量的最小与最大特征元组之间存在何种关系？如何通过极值原理刻画？
• RQ4 Courant-Fischer定理如何适配T-积张量，以描述极值特征元组？
• RQ5 利用所建立的不等式，可为独立随机厄米特T-积张量和的最大特征值与特征元组推导出何种尾部界？

主要发现

• 当f为非减或凸函数时，T-积张量的迹函数Tr(f(𝒞))分别为非减与凸函数。
• Golden-Thompson不等式对厄米特T-积张量成立：Tr(exp(𝒞 + 𝒟)) ≤ Tr(exp(𝒞) ⋆ exp(𝒟))。
• 在约束∑ᵢ 𝒞ᵢᴴ ⋆ 𝒞ᵢ = 𝒫下，Jensen的T-积不等式成立，表明f(∑ᵢ 𝒞ᵢᴴ ⋆ 𝒳ᵢ ⋆ 𝒞ᵢ) ⪯ ∑ᵢ 𝒞ᵢᴴ ⋆ f(𝒳ᵢ) ⋆ 𝒞ᵢ，适用于T-凸函数f。
• Klein的T-积不等式已证明，将经典不等式推广至T-积张量框架。
• 建立了关于T-积张量的Courant-Fischer定理，将第k个最大特征元组表征为k维子空间上的极小极大值。
• 最小与最大特征元组之间的关系被形式化：d_min(𝒳) = −d_max(−𝒳)，对特征值亦成立。