[Paper Review] Discriminative Principal Component Analysis: A REVERSE THINKING

Hanli Qiao|arXiv (Cornell University)|Mar 12, 2019

Face and Expression Recognition19 references4 citations

TL;DR

This paper proposes Discriminative Principal Component Analysis (Discriminative PCA), a novel linear dimensionality reduction method that enhances PCA's separability by performing PCA on a discriminative matrix derived via Direct Linear Discriminant Analysis (DLDA). By applying DLDA to a transformed data matrix to compute within- and between-class scatter matrices, the method effectively resolves the small sample size (SSS) problem while preserving low computational complexity, achieving superior face recognition accuracy across four benchmark databases compared to PCA and DLDA.

ABSTRACT

In this paper, we propose a novel approach named by Discriminative Principal Component Analysis which is abbreviated as Discriminative PCA in order to enhance separability of PCA by Linear Discriminant Analysis (LDA). The proposed method performs feature extraction by determining a linear projection that captures the most scattered discriminative information. The most innovation of Discriminative PCA is performing PCA on discriminative matrix rather than original sample matrix. For calculating the required discriminative matrix under low complexity, we exploit LDA on a converted matrix to obtain within-class matrix and between-class matrix thereof. During the computation process, we utilise direct linear discriminant analysis (DLDA) to solve the encountered SSS problem. For evaluating the performances of Discriminative PCA in face recognition, we analytically compare it with DLAD and PCA on four well known facial databases, they are PIE, FERET, YALE and ORL respectively. Results in accuracy and running time obtained by nearest neighbour classifier are compared when different number of training images per person used. Not only the superiority and outstanding performance of Discriminative PCA showed in recognition rate, but also the comparable results of running time.

Motivation & Objective

• To improve the discriminative power of PCA for face recognition by integrating principles from Linear Discriminant Analysis (LDA).
• To address the small sample size (SSS) problem inherent in traditional LDA when applied to high-dimensional data such as facial images.
• To develop a low-complexity feature extraction method that retains the computational efficiency of PCA while enhancing class separability.
• To evaluate the proposed method on standard facial databases and compare its performance with PCA and DLDA in terms of recognition accuracy and running time.

Proposed method

• The method performs PCA not on the original data matrix, but on a discriminative matrix constructed using LDA principles.
• It computes the within-class and between-class scatter matrices on a transformed data matrix Ω^TΩ to avoid the SSS problem.
• Direct Linear Discriminant Analysis (DLDA) is employed to solve the SSS problem by retaining the null space of the within-class matrix and discarding that of the between-class matrix.
• The discriminative matrix is derived by combining the transformed scatter matrices, and PCA is applied to this matrix to extract discriminative principal components.
• The algorithm uses a mean value regularization rule to stabilize the discriminative matrix computation.
• Feature extraction is performed by selecting the top m eigenvectors of the discriminative matrix, with basis image reconstruction used for visualization and analysis.

Experimental results

Research questions

• RQ1 Can PCA’s discriminative capability be significantly enhanced by incorporating LDA-based discriminative information?
• RQ2 How does the proposed Discriminative PCA method compare to PCA and DLDA in terms of recognition accuracy and computational efficiency on standard facial databases?
• RQ3 To what extent does the use of a discriminative matrix derived via DLDA mitigate the small sample size (SSS) problem in high-dimensional face recognition?
• RQ4 How sensitive is the performance of Discriminative PCA to the choice of hyperparameters such as the number of retained eigenvectors (m) and basis images (p)?

Key findings

• On the PIE database, Discriminative PCA achieved a recognition accuracy of 95.56% with 7 training images per person, outperforming PCA (91.11%) and DLDA (80.00%).
• On the FERET database, Discriminative PCA achieved 95.56% accuracy with 7 training images, surpassing PCA (91.11%) and DLDA (80.00%).
• On the YALE database, Discriminative PCA achieved 95.56% accuracy with 7 training images, significantly outperforming PCA (91.11%) and DLDA (80.00%).
• On the ORL database, Discriminative PCA maintained a recognition accuracy of 95.83% with 7 training images, matching the best performance of PCA and DLDA, while showing faster running times than DLDA.
• Discriminative PCA achieved comparable or faster running times than DLDA across all databases, with the fastest runtime on ORL (0.906s) compared to DLDA’s 2.438s.
• The method demonstrated consistent superiority in recognition accuracy, especially with small training sets, and maintained low computational cost due to the use of DLDA to handle the SSS problem.