[Paper Review] Evaluating Model Performance Under Worst-case Subpopulations

Mike Li, Mittal, Daksh+2 more|arXiv (Cornell University)|Jul 1, 2024

Bayesian Modeling and Causal Inference6 citations

TL;DR

The paper defines and estimates worst-case subpopulation performance over all subpopulations of a given size, providing a debiased, scalable evaluation method with finite-sample guarantees.

ABSTRACT

The performance of ML models degrades when the training population is different from that seen under operation. Towards assessing distributional robustness, we study the worst-case performance of a model over all subpopulations of a given size, defined with respect to core attributes Z. This notion of robustness can consider arbitrary (continuous) attributes Z, and automatically accounts for complex intersectionality in disadvantaged groups. We develop a scalable yet principled two-stage estimation procedure that can evaluate the robustness of state-of-the-art models. We prove that our procedure enjoys several finite-sample convergence guarantees, including dimension-free convergence. Instead of overly conservative notions based on Rademacher complexities, our evaluation error depends on the dimension of Z only through the out-of-sample error in estimating the performance conditional on Z. On real datasets, we demonstrate that our method certifies the robustness of a model and prevents deployment of unreliable models.

Motivation & Objective

• Motivate robust evaluation of ML models under distribution shifts across arbitrary Z-defined subpopulations.
• Define worst-case subpopulation performance W_alpha* and its certificate alpha* for deployment safety.
• Develop a scalable two-stage estimation procedure to approximate conditional risks and the tail-risk objective.
• Provide finite-sample and asymptotic guarantees that are dimension-free with respect to Z, enabling deep networks to be used for mu estimation.

Proposed method

• Formulate worst-case subpopulation performance W_alpha* as a tail-average (CVaR) of the conditional risk mu*(Z).
• Apply a dual reformulation to express W_alpha* as a minimization over a scalar eta and a plus-term; relate to a (1-alpha)-quantile of mu*(Z).
• Estimate the conditional risk mu*(Z) by solving a regression-type problem over a model class H on auxiliary data.
• Use a debiased (augmented) estimator that corrects first-order errors from estimating mu*(Z) with respect to the final W_alpha* calculation.
• Employ cross-fitting to combine multiple folds and obtain a robust, data-efficient estimator with a central limit theorem.
• Provide procedures to estimate the robustness certificate alpha* and its confidence.

Experimental results

Research questions

• RQ1 How to quantify and certify worst-case subpopulation performance across arbitrary subpopulation definitions Z?
• RQ2 Can we construct a scalable, debiased estimator for the tail-risk objective W_alpha* that achieves favorable finite-sample guarantees?
• RQ3 What are the convergence rates and how do they depend on the complexity of the conditional-risk model class H?
• RQ4 How can we certify robustness through a threshold alpha* indicating minimum subpopulation size for acceptable performance?

Key findings

• A debiased, two-stage estimator achieves convergence at rate O_p(sqrt(Comp_n(H)/n)) for the worst-case subpopulation performance.
• The method admits a dimension-free concentration bound where the bound depends on the out-of-sample error in estimating mu*(Z).
• The central limit theorem shows sqrt(n)-rates for the debiased estimator even when mû converges slowly.
• The approach connects worst-case subpopulation performance to conditional value-at-risk and coherent risk measures.
• The methodology supports practical robustness certificates alpha* and enables model evaluation with deep networks for mu estimation.