[Paper Review] Statistical inference and hypotheses testing of risk averse stochastic programs

Vincent Guigues, Volker Krätschmer+1 more|arXiv (Cornell University)|Mar 23, 2016

Risk and Portfolio Optimization20 references3 citations

TL;DR

This paper develops statistical inference and hypothesis testing methods for risk-averse stochastic programs using Sample Average Approximation (SAA). It establishes Central Limit Theorem-type results for optimal values and solutions under law-invariant coherent risk measures, enabling asymptotic and nonasymptotic tests to compare optimal values across multiple stochastic programs. The key contribution is demonstrating that asymptotic tests significantly reduce type II error compared to nonasymptotic methods, especially for small to moderate sample sizes.

ABSTRACT

We study statistical properties of the optimal value and optimal solutions of the Sample Average Approximation of risk averse stochastic problems. Central Limit Theorem type results are derived for the optimal value and optimal solutions when the stochastic program is expressed in terms of a law invariant coherent risk measure. The obtained results are applied to hypotheses testing problems aiming at comparing the optimal values of several risk averse convex stochastic programs on the basis of samples of the underlying random vectors. We also consider non-asymptotic tests based on confidence intervals on the optimal values of the stochastic programs obtained using the Stochastic Mirror Descent algorithm. Numerical simulations show how to use our developments to choose among different distributions and show the superiority of the asymptotic tests on a class of risk averse stochastic programs.

Motivation & Objective

• To develop statistical inference methods for optimal values and solutions of risk-averse stochastic programs formulated with law-invariant coherent risk measures.
• To enable hypothesis testing for comparing optimal values across multiple risk-averse stochastic programs using sample data.
• To evaluate the performance of asymptotic versus nonasymptotic tests in terms of type II error rates under varying sample sizes and problem dimensions.
• To provide practical tools for selecting among candidate solutions or distributions based on risk measure rankings using statistical confidence intervals.

Proposed method

• Derives Central Limit Theorem-type asymptotic distributions for the optimal value and optimal solutions of the Sample Average Approximation (SAA) problem under law-invariant coherent risk measures.
• Applies the SAA method to approximate the true stochastic program using i.i.d. samples of the underlying random vector, with convergence properties analyzed under regularity conditions.
• Develops asymptotic tests for hypotheses comparing optimal values across K risk-averse convex stochastic programs, including equality, inequality, and ordered constraints.
• Proposes nonasymptotic tests based on confidence intervals derived from the Stochastic Mirror Descent algorithm to assess optimal value differences with finite samples.
• Uses numerical simulations to compare the empirical performance of asymptotic and nonasymptotic tests in terms of type II error and confidence interval width.
• Employs a benchmark with a sample size of $10^6$ to estimate true optimal values for validating test accuracy.

Experimental results

Research questions

• RQ1 Can asymptotic statistical inference be reliably applied to the optimal value and solution of risk-averse stochastic programs using SAA?
• RQ2 How do asymptotic and nonasymptotic hypothesis tests compare in terms of type II error rates for small to moderate sample sizes?
• RQ3 To what extent does the normal approximation of the SAA optimal value hold for risk-averse problems with sample sizes as low as N=20?
• RQ4 Can statistical tests be effectively used to select among different candidate distributions or solutions based on their risk measure values?
• RQ5 How do the confidence intervals from asymptotic and nonasymptotic methods compare in width and proximity to the true optimal value?

Key findings

• The asymptotic test achieves zero type II error for sample sizes above $N=100$ in all tested comparisons, while nonasymptotic tests maintain 100% type II error up to $N=10^5$ for some cases.
• For $N=20$, the asymptotic confidence interval for the optimal value is already close to the true value, with coverage probability approaching the nominal level.
• The asymptotic test outperforms the nonasymptotic test in reducing type II error, even for small sample sizes, due to tighter and more accurate confidence bounds.
• The Gaussian approximation of the SAA optimal value distribution is accurate for $N \geq 20$ and problem sizes up to $n=10,000$.
• The asymptotic confidence intervals are narrower and closer to the true optimal value than nonasymptotic intervals, explaining their superior power in detecting true alternatives.
• For $N=10^5$, the asymptotic test achieves zero type II error in all comparisons, while the nonasymptotic test still shows 100% type II error for some cases, indicating a significant performance gap.