[Paper Review] Principal Component Analysis: A Natural Approach to Data Exploration

Felipe L. Gewers, Gustavo R. Ferreira+5 more|arXiv (Cornell University)|Apr 7, 2018

Spectroscopy and Chemometric Analyses62 references105 citations

TL;DR

The paper presents PCA as a practical, intuitive tool for data exploration and dimensionality reduction, discusses standardization effects, and surveys PCA applications across diverse fields, plus an experimental assessment of variance preservation.

ABSTRACT

Principal component analysis (PCA) is often used for analyzing data in the most diverse areas. In this work, we report an integrated approach to several theoretical and practical aspects of PCA. We start by providing, in an intuitive and accessible manner, the basic principles underlying PCA and its applications. Next, we present a systematic, though no exclusive, survey of some representative works illustrating the potential of PCA applications to a wide range of areas. An experimental investigation of the ability of PCA for variance explanation and dimensionality reduction is also developed, which confirms the efficacy of PCA and also shows that standardizing or not the original data can have important effects on the obtained results. Overall, we believe the several covered issues can assist researchers from the most diverse areas in using and interpreting PCA.

Motivation & Objective

• Explain PCA fundamentals and its geometric interpretation as rotation to maximize variance along principal axes.
• Survey PCA applications across biology, medicine, neuroscience, psychology, engineering, and more to illustrate versatility.
• Investigate how preprocessing choices, especially standardization, affect PCA outcomes and variance preservation.

Proposed method

• Formulate PCA as Y = W X where W contains eigenvectors of the covariance matrix of centered data.
• Compute the covariance matrix K = Cov(X) and its eigenvalues/eigenvectors to construct W.
• Explain variance preservation through S and Sc, and define the explained variance ratio G = (Sc/S) × 100%.
• Discuss criteria for choosing the number of components M based on retained variance (e.g., 70%), plus references to probabilistic and Bayesian approaches for dimensionality estimation.
• Analyze practical considerations like standardization vs. not standardizing, and properties of PCA axes (direction ambiguity, rotation, and maximum variance demonstration).

Experimental results

Research questions

• RQ1 How does PCA decorrelate data and maximize variance along principal axes?
• RQ2 What is the impact of standardizing versus not standardizing data on PCA results and variance preservation?
• RQ3 How many principal components are needed to retain a desirable amount of total variance across different datasets?
• RQ4 In which contexts should PCA be used as a preprocessing step to aid more complex projections or analyses?
• RQ5 What are the practical limitations and interpretational aspects of PCA axes and loadings across diverse domains?

Key findings

• PCA decorrelates original variables and concentrates variance in the first few axes, enabling effective dimensionality reduction.
• Standardization can significantly affect PCA outcomes; inappropriate standardization may amplify noise or meaningless variation.
• The total variance is preserved under PCA rotation, and the retained variance Sc determines the chosen number of components M via G = (Sc/S) × 100%.
• The direction of PCA axes is not unique (axes can flip sign without changing results).
• PCA can be used prior to more expensive projections to simplify data and improve statistical representation.
• Empirical evidence suggests substantial variance preservation is achievable for many real-world datasets.