[论文解读] Optimal Semi-supervised Estimation and Inference for High-dimensional Linear Regression

Siyi Deng, Yang Ning+2 more|arXiv (Cornell University)|Nov 28, 2020

Statistical Methods and Inference参考文献 47被引用 4

一句话总结

本文提出了一种用于高维线性回归的新型半监督估计器，利用标记数据和未标记数据，实现了比监督估计器更快的收敛速度。该文引入了一种高效估计器和一种安全估计器用于推断，确保在模型误设或条件均值函数估计不一致的情况下，其效率或性能仍优于监督方法。

ABSTRACT

There are many scenarios such as the electronic health records where the outcome is much more difficult to collect than the covariates. In this paper, we consider the linear regression problem with such a data structure under the high dimensionality. Our goal is to investigate when and how the unlabeled data can be exploited to improve the estimation and inference of the regression parameters in linear models, especially in light of the fact that such linear models may be misspecified in data analysis. In particular, we address the following two important questions. (1) Can we use the labeled data as well as the unlabeled data to construct a semi-supervised estimator such that its convergence rate is faster than the supervised estimators? (2) Can we construct confidence intervals or hypothesis tests that are guaranteed to be more efficient or powerful than the supervised estimators? To address the first question, we establish the minimax lower bound for parameter estimation in the semi-supervised setting. We show that the upper bound from the supervised estimators that only use the labeled data cannot attain this lower bound. We close this gap by proposing a new semi-supervised estimator which attains the lower bound. To address the second question, based on our proposed semi-supervised estimator, we propose two additional estimators for semi-supervised inference, the efficient estimator and the safe estimator. The former is fully efficient if the unknown conditional mean function is estimated consistently, but may not be more efficient than the supervised approach otherwise. The latter usually does not aim to provide fully efficient inference, but is guaranteed to be no worse than the supervised approach, no matter whether the linear model is correctly specified or the conditional mean function is consistently estimated.

研究动机与目标

• 解决当结果数据（标签）的获取成本远高于协变量时，高维线性回归所面临的挑战。
• 探究未标记数据是否能够使参数估计的收敛速度超越监督方法所能达到的水平。
• 开发推断程序——置信区间与假设检验——其效率或统计功效被保证优于对应的监督方法。

提出的方法

• 在半监督设置下建立参数估计的极小极大下界，以定义理论性能的极限。
• 提出一种新型半监督估计器，能够达到该极小极大下界，从而弥合理论最优性与监督估计器性能之间的差距。
• 引入一种高效估计器，当条件均值函数被一致估计时可实现完全效率，但否则可能无法改进。
• 开发一种安全估计器，确保其性能在任何情况下均不劣于监督方法，无论模型是否正确或估计是否一致。
• 以所提出的估计器为基础，构建具有更高效率或更强鲁棒性的置信区间与假设检验。

实验结果

研究问题

• RQ1 在高维线性回归中，是否可以利用未标记数据构建一种收敛速度优于监督估计器的半监督估计器？
• RQ2 是否可能基于半监督推断构建置信区间或假设检验，其效率或统计功效被保证优于监督方法？
• RQ3 模型误设或条件均值函数估计不一致如何影响半监督推断方法的性能？

主要发现

• 所提出的半监督估计器达到了参数估计的极小极大下界，证明其在半监督设置下为最优。
• 仅使用标记数据的监督估计器无法达到该极小极大下界，表明存在根本性的性能差距。
• 当条件均值函数被一致估计时，高效估计器可实现完全效率，但否则可能无法优于监督方法。
• 安全估计器在推断效率方面被保证不劣于监督方法，无论模型是否正确或估计是否一致。
• 所提出的推断方法在理论上保证了相较于监督方法的性能提升，或至少保持相当，即使在模型误设的情况下亦成立。