[论文解读] Holistic Robust Data-Driven Decisions

A. Bennouna, Bart P. G. Van Parys+1 more|arXiv (Cornell University)|Jul 19, 2022

Advanced Bandit Algorithms Research被引用 6

一句话总结

本文提出了一种新颖的分布鲁棒优化公式——整体鲁棒（Holistic Robust, HR）数据驱动决策方法，可同时防范三种过拟合来源：统计误差、数据噪声和数据误设。通过结合Kullback-Leibler与Lévy-Prokhorov歧义集，HR公式在投资组合选择中实现了更优的风险-收益权衡，其在实证评估中优于标准基准方法。

ABSTRACT

The design of data-driven formulations for machine learning and decision-making with good out-of-sample performance is a key challenge. The observation that good in-sample performance does not guarantee good out-of-sample performance is generally known as overfitting. Practical overfitting can typically not be attributed to a single cause but is caused by several factors simultaneously. We consider here three overfitting sources: (i) statistical error as a result of working with finite sample data, (ii) data noise, which occurs when the data points are measured only with finite precision, and finally, (iii) data misspecification in which a small fraction of all data may be wholly corrupted. Although existing data-driven formulations may be robust against one of these three sources in isolation, they do not provide holistic protection against all overfitting sources simultaneously. We design a novel data-driven formulation that guarantees such holistic protection and is computationally viable. Our distributionally robust optimization formulation can be interpreted as a novel combination of a Kullback-Leibler and Lévy-Prokhorov robust optimization formulation. In the context of classification and regression problems, we show that several popular regularized and robust formulations naturally reduce to a particular case of our proposed novel formulation. Finally, we apply the proposed HR formulation to two real-life applications and study it alongside several benchmarks: (1) training neural networks on healthcare data, where we analyze various robustness and generalization properties in the presence of noise, labeling errors, and scarce data, (2) a portfolio selection problem with real stock data, and analyze the risk/return tradeoff under the natural severe distribution shift of the application.

研究动机与目标

• 解决现有数据驱动公式仅能针对单一过拟合来源提供鲁棒性的局限性。
• 开发一种统一的公式，可同时防范统计误差、数据噪声和数据误设。
• 在实现整体鲁棒性的同时，确保计算上的可行性。
• 在实际应用中证明所提公式的优越性，超越标准基准。
• 表明流行的正则化与鲁棒公式均为HR框架的特例。

提出的方法

• 提出一种新颖的歧义集，结合Kullback-Leibler与Lévy-Prokhorov度量，以建模分布不确定性。
• 设计一种分布鲁棒优化公式，确保对有限样本量、测量精度和数据污染的不变性。
• 推导出适用于分类与回归问题的计算上可行的重表述形式。
• 建立理论联系，表明常见正则化与鲁棒公式（如Tikhonov、Lasso、鲁棒M估计器）均为HR公式的特例。
• 将HR公式应用于基于历史股票数据的真实世界投资组合选择问题。
• 采用带有HR歧义集的期望风险最小化，推导出具有更好泛化性能的决策。

实验结果

研究问题

• RQ1 单一数据驱动公式能否同时防范统计误差、数据噪声和数据误设？
• RQ2 与标准SAA和ERM方法相比，所提出的HR公式在泛化性能方面表现如何？
• RQ3 HR公式在实际决策场景中的计算可行性与可扩展性如何？
• RQ4 现有鲁棒与正则化公式在多大程度上可作为HR框架的特例？
• RQ5 与基准相比，HR公式在真实金融数据中是否能实现更优的风险-收益权衡？

主要发现

• 所提出的HR公式可同时对三种过拟合来源——统计误差、数据噪声和数据误设——提供整体保护。
• 与标准基准相比，HR公式在真实投资组合选择问题中显著改善了风险-收益权衡。
• HR公式将多种流行的正则化与鲁棒优化方法作为特例统一于单一鲁棒框架之下。
• 在歧义集中结合Kullback-Leibler与Lévy-Prokhorov度量，可实现强于任一单独度量的分布鲁棒性。
• 基于真实股票数据的实证结果表明，HR公式推导出的决策在泛化性能上优于SAA与ERM方法。
• HR公式在计算上可行且具备可扩展性，可实现于数据驱动决策的实际部署中。