[Paper Review] Huber Principal Component Analysis for Large-dimensional Factor Models

Yong He, Lingxiao Li+2 more|arXiv (Cornell University)|Mar 6, 2023

Matrix Theory and Algorithms4 citations

TL;DR

This paper proposes Huber Principal Component Analysis (HPCA) for large-dimensional factor models to achieve robust estimation under heavy-tailed financial and macroeconomic data. By minimizing Huber loss instead of squared error, HPCA maintains the convergence rate of conventional PCA under bounded second moments and provides consistent estimation of factor loadings, scores, and the number of factors via rank minimization, with theoretical guarantees and empirical validation in portfolio selection.

ABSTRACT

Factor models have been widely used in economics and finance. However, the heavy-tailed nature of macroeconomic and financial data is often neglected in the existing literature. To address this issue and achieve robustness, we propose an approach to estimate factor loadings and scores by minimizing the Huber loss function, which is motivated by the equivalence of conventional Principal Component Analysis (PCA) and the constrained least squares method in the factor model. We provide two algorithms that use different penalty forms. The first algorithm, which we refer to as Huber PCA, minimizes the $\ell_2$-norm-type Huber loss and performs PCA on the weighted sample covariance matrix. The second algorithm involves an element-wise type Huber loss minimization, which can be solved by an iterative Huber regression algorithm. Our study examines the theoretical minimizer of the element-wise Huber loss function and demonstrates that it has the same convergence rate as conventional PCA when the idiosyncratic errors have bounded second moments. We also derive their asymptotic distributions under mild conditions. Moreover, we suggest a consistent model selection criterion that relies on rank minimization to estimate the number of factors robustly. We showcase the benefits of Huber PCA through extensive numerical experiments and a real financial portfolio selection example. An R package named ``HDRFA" has been developed to implement the proposed robust factor analysis.

Motivation & Objective

• To address the poor performance of conventional PCA in large-dimensional factor models when idiosyncratic errors are heavy-tailed.
• To develop a robust alternative to PCA that maintains statistical efficiency under heavy-tailed distributions without requiring elliptical or moment restrictions on errors.
• To establish theoretical properties—convergence rates and asymptotic distributions—of Huber-based estimators under mild moment conditions.
• To propose a consistent model selection criterion based on rank minimization for robustly estimating the number of factors.
• To demonstrate the method’s effectiveness through numerical experiments and a real-world financial portfolio selection application.

Proposed method

• Proposes Huber PCA (HPCA) by minimizing an ℓ₂-norm-type Huber loss function, equivalent to performing PCA on a weighted sample covariance matrix.
• Introduces a second algorithm based on element-wise Huber loss minimization, solvable via iterative Huber regression.
• Establishes theoretical equivalence between conventional PCA and constrained least squares, motivating the use of Huber loss as a robust alternative.
• Derives convergence rates and asymptotic distributions for both estimators under mild moment conditions on idiosyncratic errors.
• Proposes a rank minimization-based model selection criterion to consistently estimate the number of factors.
• Develops and releases an R package, HDRFA, for implementing the proposed robust factor analysis methods.

Experimental results

Research questions

• RQ1 Can Huber loss be used to robustly estimate factor loadings and scores in large-dimensional factor models under heavy-tailed errors?
• RQ2 Does Huber PCA maintain the same convergence rate as conventional PCA when idiosyncratic errors have bounded second moments?
• RQ3 How do the asymptotic distributions of Huber-based estimators compare to those of classical PCA under mild moment assumptions?
• RQ4 Can a rank minimization criterion consistently estimate the number of factors in the presence of heavy-tailed errors?
• RQ5 How does Huber PCA perform in finite samples compared to classical PCA and other robust methods in financial data settings?

Key findings

• Huber PCA achieves the same convergence rate as classical PCA when idiosyncratic errors have bounded second moments, ensuring statistical efficiency under mild conditions.
• The asymptotic distribution of the Huber PCA estimator is derived and shown to be valid under mild moment assumptions, supporting inferential procedures.
• The proposed rank minimization criterion consistently estimates the number of factors, with probability of correct selection converging to one as sample size increases.
• Numerical experiments and a real financial portfolio example show that Huber PCA outperforms classical PCA in heavy-tailed settings, particularly in reducing estimation bias from outliers.
• The element-wise Huber loss minimization algorithm converges reliably and avoids local optima issues common in ℓ₁-based methods, offering better numerical stability.
• The R package HDRFA enables practical implementation of Huber PCA and the associated model selection, facilitating adoption in empirical finance and econometrics.