[Paper Review] Functional Factorial K-means Analysis

Michio Yamamoto, Yoshikazu Terada|arXiv (Cornell University)|Nov 3, 2013

Sensory Analysis and Statistical Methods27 references4 citations

TL;DR

This paper proposes Functional Factorial K-means (FFKM) analysis, a simultaneous clustering and dimension reduction method for multivariate functional data that optimizes both cluster structure and subspace representation via an alternating least-squares algorithm. The method overcomes limitations of tandem analysis and functional principal component k-means (FPCK) by jointly estimating cluster assignments and weight functions in a low-dimensional functional subspace, yielding more interpretable and accurate results than existing approaches.

ABSTRACT

A new procedure for simultaneously finding the optimal cluster structure of multivariate functional objects and finding the subspace to represent the cluster structure is presented. The method is based on the $k$-means criterion for projected functional objects on a subspace in which a cluster structure exists. An efficient alternating least-squares algorithm is described, and the proposed method is extended to a regularized method for smoothness of weight functions. To deal with the negative effect of the correlation of coefficient matrix of the basis function expansion in the proposed algorithm, a two-step approach to the proposed method is also described. Analyses of artificial and real data demonstrate that the proposed method gives correct and interpretable results compared with existing methods, the functional principal component $k$-means (FPCK) method and tandem clustering approach. It is also shown that the proposed method can be considered complementary to FPCK.

Motivation & Objective

• To develop a method that simultaneously identifies optimal cluster structures and low-dimensional subspaces for multivariate functional data.
• To overcome the limitations of tandem analysis, where dimension reduction and clustering are performed sequentially, leading to suboptimal results.
• To address the inherent flaw in FPCK analysis, where the loss function does not optimally align clustering and subspace estimation.
• To provide a complementary alternative to FPCK that yields more interpretable weight functions and cluster structures.
• To enable efficient, iterative estimation of cluster assignments and subspace projections using an alternating least-squares algorithm.

Proposed method

• The method uses a k-means criterion applied to projected functional objects onto a common low-dimensional subspace where cluster structure exists.
• An alternating least-squares algorithm is employed to jointly optimize cluster assignments and subspace projections.
• The method incorporates regularization to ensure smoothness of the estimated weight functions in the basis expansion.
• A two-step approach is introduced to mitigate the negative impact of correlation in the basis function coefficient matrix.
• The algorithm estimates orthogonal transformation matrices via orthogonal Procrustes rotation to recover interpretable weight functions from principal component scores.
• The final weight functions are reconstructed as linear combinations of basis functions using the estimated coefficient matrix.

Experimental results

Research questions

• RQ1 Can a simultaneous approach to clustering and dimension reduction in functional data outperform sequential tandem analysis?
• RQ2 How does the proposed method compare to functional principal component k-means (FPCK) in terms of cluster accuracy and interpretability?
• RQ3 What impact does the correlation structure of basis function coefficients have on clustering performance, and how can it be mitigated?
• RQ4 Can regularization improve the smoothness and interpretability of the estimated weight functions in the subspace?
• RQ5 Is the proposed method complementary to FPCK, particularly in recovering meaningful subspace structures?

Key findings

• The proposed FFKM method produces more accurate and interpretable cluster structures than both tandem analysis and FPCK on artificial and real data.
• The method successfully identifies a common low-dimensional subspace that optimally represents the cluster structure, avoiding the suboptimality of sequential approaches.
• The two-step approach effectively reduces the influence of correlated basis coefficients, improving numerical stability and estimation accuracy.
• Regularization of the weight functions enhances smoothness, leading to more stable and interpretable subspace representations.
• The method demonstrates a complementary relationship with FPCK, particularly in recovering meaningful weight functions and cluster patterns.
• Empirical results show that FFKM achieves better clustering performance than FPCK in scenarios with complex cluster structures and high-dimensional functional data.