[Paper Review] Tensor Canonical Correlation Analysis for Multi-view Dimension Reduction

Yong Luo, Dacheng Tao+3 more|arXiv (Cornell University)|Feb 9, 2015

Tensor decomposition and applications2 references4 citations

TL;DR

This paper proposes Tensor Canonical Correlation Analysis (TCCA), a method that generalizes canonical correlation analysis to handle an arbitrary number of views by modeling high-order correlations through a covariance tensor. By reformulating multi-view correlation maximization as a rank-1 tensor approximation problem, TCCA efficiently captures complex inter-view dependencies using the alternating least squares (ALS) algorithm, leading to improved performance in multi-view dimension reduction tasks such as biometric prediction, advertisement classification, and image annotation.

ABSTRACT

Canonical correlation analysis (CCA) has proven an effective tool for two-view dimension reduction due to its profound theoretical foundation and success in practical applications. In respect of multi-view learning, however, it is limited by its capability of only handling data represented by two-view features, while in many real-world applications, the number of views is frequently many more. Although the ad hoc way of simultaneously exploring all possible pairs of features can numerically deal with multi-view data, it ignores the high order statistics (correlation information) which can only be discovered by simultaneously exploring all features. Therefore, in this work, we develop tensor CCA (TCCA) which straightforwardly yet naturally generalizes CCA to handle the data of an arbitrary number of views by analyzing the covariance tensor of the different views. TCCA aims to directly maximize the canonical correlation of multiple (more than two) views. Crucially, we prove that the multi-view canonical correlation maximization problem is equivalent to finding the best rank-1 approximation of the data covariance tensor, which can be solved efficiently using the well-known alternating least squares (ALS) algorithm. As a consequence, the high order correlation information contained in the different views is explored and thus a more reliable common subspace shared by all features can be obtained. In addition, a non-linear extension of TCCA is presented. Experiments on various challenge tasks, including large scale biometric structure prediction, internet advertisement classification and web image annotation, demonstrate the effectiveness of the proposed method.

Motivation & Objective

• Address the limitation of traditional CCA, which is restricted to two-view data, in handling multi-view learning scenarios with more than two feature sources.
• Overcome the shortcomings of pairwise correlation maximization in existing multi-view extensions, which ignore high-order statistical dependencies across all views.
• Develop a principled framework that directly maximizes the canonical correlation among multiple views by modeling the joint covariance structure as a tensor.
• Enable efficient computation of the optimal common subspace by reducing the multi-view correlation maximization problem to a low-rank tensor approximation task.
• Extend the method to non-linear settings using kernelization for improved performance on high-dimensional and limited-data scenarios.

Proposed method

• Formulate multi-view dimension reduction as a tensor-based canonical correlation maximization problem using the covariance tensor of all views.
• Prove that maximizing the multi-view correlation is mathematically equivalent to finding the best rank-1 approximation of the data covariance tensor.
• Utilize the alternating least squares (ALS) algorithm to efficiently solve the rank-1 tensor approximation problem, ensuring scalability and convergence.
• Introduce a kernelized version of TCCA to model non-linear relationships between views by mapping data into a reproducing kernel Hilbert space.
• Apply tensor-matrix multiplication and mode products to express the canonical correlation as a multilinear form involving view-specific weight vectors.
• Ensure computational efficiency by leveraging the structure of the covariance tensor and iterative optimization via ALS, suitable for large-scale datasets.

Experimental results

Research questions

• RQ1 Can a principled extension of CCA be developed to handle more than two views while preserving the theoretical foundation of canonical correlation?
• RQ2 Does modeling high-order correlations across all views—beyond pairwise correlations—lead to a more robust and informative common subspace for multi-view data?
• RQ3 Can the multi-view correlation maximization problem be reformulated as a low-rank tensor approximation task, and if so, is it solvable efficiently?
• RQ4 How does the performance of TCCA compare to traditional pairwise CCA extensions in real-world multi-view learning tasks?
• RQ5 To what extent does the non-linear kernelized version of TCCA improve performance on high-dimensional or data-scarce datasets?

Key findings

• TCCA achieves superior performance in multi-view dimension reduction by capturing high-order correlations that pairwise methods miss, as demonstrated on biometric structure prediction, internet advertisement classification, and web image annotation.
• The proposed method significantly outperforms pairwise CCA extensions in terms of classification accuracy and subspace quality, especially when the number of views exceeds two.
• Empirical results on the SecStr dataset show that TCCA maintains low computational cost and memory usage even as the dimension of the common subspace increases, indicating scalability.
• On the NUS-WIDE mammal subset with only six labeled samples per concept, TCCA achieves strong performance, demonstrating robustness in low-data regimes.
• The non-linear extension of TCCA further improves results on high-dimensional data, particularly when combined with kernel methods, confirming the value of non-linear modeling in multi-view learning.
• Theoretical analysis confirms that the multi-view correlation maximization problem is equivalent to a rank-1 tensor approximation, validating the method’s mathematical foundation and enabling efficient optimization via ALS.