[Paper Review] Predict-then-Calibrate: A New Perspective of Robust Contextual LP

Chunlin Sun, Linyu Liu+1 more|arXiv (Cornell University)|May 25, 2023

Risk and Portfolio OptimizationDecision Sciences3 citations

TL;DR

This paper introduces the predict-then-calibrate paradigm for robust contextual linear programming, decoupling prediction from uncertainty calibration to enable flexible use of off-the-shelf ML models and provide theoretical robustness guarantees. It achieves improved generalization bounds and convergence rates for both risk-sensitive and distributionally robust optimization, with empirical validation showing performance gains from better prediction or calibration components.

ABSTRACT

Contextual optimization, also known as predict-then-optimize or prescriptive analytics, considers an optimization problem with the presence of covariates (context or side information). The goal is to learn a prediction model (from the training data) that predicts the objective function from the covariates, and then in the test phase, solve the optimization problem with the covariates but without the observation of the objective function. In this paper, we consider a risk-sensitive version of the problem and propose a generic algorithm design paradigm called predict-then-calibrate. The idea is to first develop a prediction model without concern for the downstream risk profile or robustness guarantee, and then utilize calibration (or recalibration) methods to quantify the uncertainty of the prediction. While the existing methods suffer from either a restricted choice of the prediction model or strong assumptions on the underlying data, we show the disentangling of the prediction model and the calibration/uncertainty quantification has several advantages. First, it imposes no restriction on the prediction model and thus fully unleashes the potential of off-the-shelf machine learning methods. Second, the derivation of the risk and robustness guarantee can be made independent of the choice of the prediction model through a data-splitting idea. Third, our paradigm of predict-then-calibrate applies to both (risk-sensitive) robust and (risk-neutral) distributionally robust optimization (DRO) formulations. Theoretically, it gives new generalization bounds for the contextual LP problem and sheds light on the existing results of DRO for contextual LP. Numerical experiments further reinforce the advantage of the predict-then-calibrate paradigm in that an improvement on either the prediction model or the calibration model will lead to a better final performance.

Motivation & Objective

• To address the limitations of existing end-to-end or tightly coupled methods in contextual linear programming under risk-sensitive and robust optimization settings.
• To decouple prediction and uncertainty calibration, allowing unrestricted use of machine learning models and independent theoretical analysis of robustness.
• To derive generalization bounds for robust and distributionally robust contextual LP using nonparametric regression tools.
• To provide theoretical and empirical evidence that improvements in either prediction or calibration directly enhance final decision quality.
• To unify risk-sensitive and distributionally robust formulations under a single, flexible algorithmic paradigm.

Proposed method

• Propose a two-stage paradigm: first train a prediction model on covariates without regard to downstream robustness, then apply calibration to quantify prediction uncertainty.
• Use data splitting to ensure theoretical guarantees on risk and robustness are independent of the choice of prediction model.
• Apply uncertainty calibration methods to estimate the distribution of objective function coefficients given covariates, enabling robust optimization.
• Derive generalization bounds for the robust contextual LP problem using nonparametric regression techniques and covering number arguments.
• Design algorithms that output decisions with provable convergence to optimal solutions under increasing data size T.
• Establish convergence rates of order $ T^{-s/(2s+2d)} \log T $ for the optimality gap, with explicit dependence on model and data parameters.

Experimental results

Research questions

• RQ1 Can we decouple prediction and uncertainty calibration in contextual linear programming to improve robustness without restricting the choice of prediction model?
• RQ2 How can we derive theoretical robustness and generalization guarantees independent of the prediction model's architecture?
• RQ3 What is the convergence rate of the optimality gap under the predict-then-calibrate framework for robust contextual LP?
• RQ4 How does the predict-then-calibrate paradigm compare to end-to-end or integrated approaches in terms of performance and flexibility?
• RQ5 Can the same framework be applied to both risk-sensitive and distributionally robust optimization formulations?

Key findings

• The predict-then-calibrate framework achieves a convergence rate of $ O(T^{-s/(2s+2d)} \log T) $ for the optimality gap, with explicit dependence on data dimension and smoothness parameters.
• Theoretical guarantees on robustness and generalization are independent of the prediction model due to data splitting, enabling use of any off-the-shelf ML model.
• Empirical results confirm that improvements in either the prediction model or the calibration method lead to better final decision performance.
• The framework provides new generalization bounds for distributionally robust contextual LP using tools from nonparametric regression.
• The method achieves strong coverage and robustness without imposing structural constraints on the prediction model or strong parametric assumptions on the data.
• The convergence rate is derived under minimal assumptions, with explicit constants depending on model complexity, data range, and smoothness.