[论文解读] Factor selection by permutation

Edgar Dobriban|arXiv (Cornell University)|Oct 2, 2017

Statistical Methods and Inference参考文献 41被引用 3

一句话总结

本文为一种广泛使用的基于置换的主成分分析（PCA）和因子模型中成分数量选择方法——平行分析，提供了首个理论依据。它表明该方法通过在特征层面进行随机置换以保留噪声结构而破坏低秩信号，从而一致地识别出大成分，但无法检测到较小成分。

ABSTRACT

Researchers often have datasets measuring features $x_{ij}$ of samples, such as test scores of students. In factor analysis and PCA, these features are thought to be influenced by unobserved factors, such as skills. Can we determine how many components affect the data? This is an important problem, because it has a large impact on all downstream data analysis. Consequently, many approaches have been developed to address it. Parallel Analysis is a popular permutation method. It works by randomly scrambling each feature of the data. It selects components if their singular values are larger than those of the permuted data. Despite widespread use in leading textbooks and scientific publications, as well as empirical evidence for its accuracy, it currently has no theoretical justification. In this paper, we show that the parallel analysis permutation method consistently selects the large components in certain high-dimensional factor models. However, it does not select the smaller components. The intuition is that permutations keep the noise invariant, while destroying the low-rank signal. This provides justification for permutation methods in PCA and factor models under some conditions. Our work uncovers drawbacks of permutation methods, and paves the way to improvements.

研究动机与目标

• 为因子分析和主成分分析中平行分析的广泛应用提供理论依据。
• 研究基于置换的方法（如平行分析）在何种条件下能一致地选择出相关成分。
• 理解基于置换的方法在高维数据中检测较小、低秩成分时的局限性。
• 阐明为何置换能保留噪声但破坏信号，从而解释该方法在实践中取得成功的原因。

提出的方法

• 作者分析了高维因子模型，其中特征受未观测到的因子影响。
• 对数据矩阵实施特征层面的随机置换，以生成替代数据集。
• 通过比较原始数据与置换后数据的奇异值来确定成分选择。
• 理论分析聚焦于置换下奇异值的渐近行为，以区分信号与噪声。
• 该方法依赖于噪声结构在置换下保持不变，而信号被破坏。
• 分析建立了在何种条件下大成分能被一致选择，但小成分不能。

实验结果

研究问题

• RQ1 在何种条件下，平行分析能在高维因子模型中一致地选择出大成分的正确数量？
• RQ2 尽管缺乏理论依据，为何平行分析在实践中表现良好？
• RQ3 基于置换的方法在PCA和因子分析中检测较小成分时存在哪些局限性？
• RQ4 置换如何影响数据矩阵奇异值的变化，其与信号和噪声的关系如何？

主要发现

• 在特定条件下，平行分析能在高维因子模型中一致地选择出大成分。
• 由于小成分的奇异值低于置换数据所设定的阈值，该方法无法检测到小成分。
• 置换保留了噪声结构，同时破坏了低秩信号，这解释了该方法在实践中取得成功的原因。
• 理论依据揭示该方法对信号强度敏感，仅能识别主导成分。