[논문 리뷰] Does data interpolation contradict statistical optimality?

Mikhail Belkin, Alexander Rakhlin외 1명|arXiv (Cornell University)|2018. 06. 25.

Advanced Statistical Methods and Models참고 문헌 10인용 수 76

한 줄 요약

본 논문은 보간 추정기가 Hölder 매끄러움 하에서 비모수 회귀 및 제곱 손실 예측에 대해 minimax-optimal 속도를 달성할 수 있음을 보여주며, 보간이 통계적 성능을 해친다는 믿음에 도전한다.

ABSTRACT

We show that learning methods interpolating the training data can achieve optimal rates for the problems of nonparametric regression and prediction with square loss.

연구 동기 및 목표

• Interpolation이 현대 학습 설정에서 좋은 out-of-sample 성능을 얻을 수 있다는 퍼즐을 동기화한다.
• Interolating estimators can attain minimax optimal rates for nonparametric regression.
• Establish finite-sample risk bounds for a class of singular-kernel Nadaraya-Watson estimators.
• Show that interpolation does not preclude optimality in excess loss under standard assumptions.

제안 방법

• Use a singular kernel K(u) = ||u||^{-a} I{||u|| ≤ 1} and variants to construct an interpolating estimator f_n.
• Analyze the Nadaraya-Watson estimator with bandwidth h and derive risk bounds for f_n(X) under Holder smoothness f ∈ Σ(β,L).
• Provide pointwise and integrated MSE bounds and prove they achieve the minimax rate n^{-2β/(2β+d)} under β ∈ (0,2].
• Decompose error into bias and variance and bound each term under assumptions (A1)-(A2) and density regularity.
• Balance bias-variance terms by choosing h = n^{-1/(2β+d)} to obtain the main rate.
• Discuss extensions to other singular kernels and to under-specified models where the regression function lies in the Hölder class.

실험 결과

연구 질문

• RQ1 Can an interpolating estimator achieve minimax-optimal rates for nonparametric regression under Hölder smoothness?
• RQ2 Do interpolating rules yield optimal excess loss in prediction with square loss when the regression function belongs to a Hölder class?
• RQ3 What conditions on the kernel, bandwidth, and density ensure optimal rates for interpolating estimators?
• RQ4 How do bias and variance behave for singular-kernel interpolants, and how should they be balanced?

주요 결과

• An interpolating estimator can achieve the classical minimax rate n^{-2β/(2β+d)} for estimating f in L2(P_X) when f ∈ Σ(β,L) with β ∈ (0,2].
• Using a singular kernel with appropriate bandwidth yields finite-sample risk bounds that match minimax rates for β ∈ (0,2].
• For β ∈ (1,2], the rate holds under an additional assumption p ∈ Σ(β−1,L_p) on the density, with p bounded away from zero on its support.
• The integrated MSE E||f_n − f||^2_{L2(P_X)} is bounded by C n^{-2β/(2β+d)} under the stated conditions.
• The interpolating estimator f_n is improper (its smoothness depends on n), yet it achieves optimal excess loss when the model is well-specified with f ∈ Σ(β,L).
• Numerical illustrations indicate the interpolating kernel can produce sharp fits locally while remaining compatible with optimal rates.