[Paper Review] Autocorrelated Optimize-via-Estimate: Predict-then-Optimize versus Finite-sample Optimal

Zichun Wang, Gar Goei Loke+1 more|arXiv (Cornell University)|Feb 2, 2026

Advanced Bandit Algorithms Research0 citations

TL;DR

The paper introduces A-OVE, an autocorrelated Optimize-via-Estimate method for VARMA time-series uncertainties and shows it achieves low regret relative to an oracle, outperforming PTO/ETO ML benchmarks in portfolio optimization with trading costs.

ABSTRACT

Models that directly optimize for out-of-sample performance in the finite-sample regime have emerged as a promising alternative to traditional estimate-then-optimize approaches in data-driven optimization. In this work, we compare their performance in the context of autocorrelated uncertainties, specifically, under a Vector Autoregressive Moving Average VARMA(p,q) process. We propose an autocorrelated Optimize-via-Estimate (A-OVE) model that obtains an out-of-sample optimal solution as a function of sufficient statistics, and propose a recursive form for computing its sufficient statistics. We evaluate these models on a portfolio optimization problem with trading costs. A-OVE achieves low regret relative to a perfect information oracle, outperforming predict-then-optimize machine learning benchmarks. Notably, machine learning models with higher accuracy can have poorer decision quality, echoing the growing literature in data-driven optimization. Performance is retained under small mis-specification.

Motivation & Objective

• Motivate data-driven optimization under finite-sample uncertainty with autocorrelated time-series.
• Propose A-OVE: an autocorrelated Optimize-via-Estimate model for VARMA(p,q) uncertainty.
• Derive a recursive approach to compute sufficient statistics via likelihood (Fisher-Neyman decomposition).
• Apply A-OVE to portfolio optimization with trading costs and compare to PTO/ETO baselines.
• Demonstrate robustness to small model mis-specification and validate on real-world data.

Proposed method

• Define data-driven optimization frameworks (PTO, ETO, FPtP, OVE).
• Extend OVE to autocorrelated VARMA(p,q) uncertainties; derive Fisher-Neyman decomposition for L(Y;ξ).
• Show that A-OVE decisions can be computed as a function of sufficient statistics, via a likelihood-based formulation.
• Provide corollaries giving closed-form A-OVE solution using weighted suffstats: xOVE = (D1̄ + D2̄)^{-1}(μ0 D1̄ e + D2̄ x0).
• Develop Algorithm 1 to approximate A-OVE by Monte Carlo integration overξ with prior u.
• Specialize the portfolio problem with trading costs, leading to quadratic cost form and explicit PTO/ETO/FPtP/A-OVE solutions.

Experimental results

Research questions

• RQ1 Can A-OVE achieve lower out-of-sample regret than PTO/ETO benchmarks under VARMA(p,q) uncertainty?
• RQ2 How can sufficient statistics for VARMA be computed and used to obtain tractable A-OVE solutions?
• RQ3 Is A-OVE robust to moderate model mis-specification and how does it compare on synthetic and real data?
• RQ4 Do ML-based PTO/ETO approaches necessarily translate predictive accuracy into better decision quality in autocorrelated settings?
• RQ5 How does A-OVE perform in portfolio optimization with trading costs against oracle performance?

Key findings

• A-OVE achieves low regret relative to a perfect-information oracle across dimensions (n=2,5,10 assets).
• A-OVE consistently outperforms PTO/ETO ML benchmarks in relative regret, including neural nets and tree ensembles.
• Machine-learning models with higher predictive accuracy do not always yield better decision quality; A-OVE aligns learning with downstream optimization.
• A-OVE shows robustness under small model mis-specification and remains competitive as problem size grows.
• Parametric ETO performs well in well-specified settings but can be unstable under mis-specification, where A-OVE remains favorable.
• On real-world data, A-OVE attains consistently low relative regret comparable to or better than strongest prediction-based methods.