[Paper Review] An Online Method for Distributionally Deep Robust Optimization

Qi Qi, Zhishuai Guo+3 more|arXiv (Cornell University)|Jun 17, 2020

Stochastic Gradient Optimization Techniques44 references5 citations

TL;DR

This paper proposes a duality-free online stochastic method for distributionally robust optimization (DRO) in deep learning, transforming the min-max formulation into a minimization problem to avoid high-dimensional dual variables. Under a Polyak-Łojasiewicz (PL) condition, it achieves optimal sample complexity without requiring large batch sizes, enabling efficient online learning with improved robustness.

ABSTRACT

In this paper, we propose a practical online method for solving a distributionally robust optimization (DRO) for deep learning, which has important applications in machine learning for improving the robustness of neural networks. In the literature, most methods for solving DRO are based on stochastic primal-dual methods. However, primal-dual methods for deep DRO suffer from several drawbacks: (1) manipulating a high-dimensional dual variable corresponding to the size of data is time expensive; (2) they are not friendly to online learning where data is coming sequentially. To address these issues, we transform the min-max formulation into a minimization formulation and propose a practical duality-free online stochastic method for solving deep DRO with KL divergence regularization. The proposed online stochastic method resembles the practical stochastic Nesterovs method in several perspectives that are widely used for learning deep neural networks. Under a Polyak-Lojasiewicz (PL) condition, we prove that the proposed method can enjoy an optimal sample complexity without any requirements on large batch size. Of independent interest, the proposed method can be also used for solving a family of stochastic compositional problems.

Motivation & Objective

• Address the limitations of existing stochastic primal-dual methods for deep DRO, which suffer from high computational cost due to large dual variables.
• Overcome the incompatibility of primal-dual methods with online learning settings where data arrives sequentially.
• Develop a practical online optimization method for deep DRO that is efficient and scalable for sequential data.
• Achieve optimal sample complexity under the Polyak-Łojasiewicz (PL) condition without relying on large batch sizes.
• Enable application of the method to a broader class of stochastic compositional problems beyond DRO.

Proposed method

• Transform the min-max formulation of DRO into a minimization problem to eliminate the need for high-dimensional dual variables.
• Propose a duality-free online stochastic optimization method using KL divergence regularization to model distributional robustness.
• Adapt the structure of the stochastic Nesterov method to the DRO setting, leveraging its favorable convergence properties.
• Integrate online learning by processing data sequentially, making the method suitable for streaming data environments.
• Use a stochastic approximation framework to update model parameters based on mini-batches of incoming data.
• Leverage the Polyak-Łojasiewicz (PL) condition to establish convergence guarantees with optimal sample complexity.

Experimental results

Research questions

• RQ1 Can a duality-free online method be designed for deep DRO that avoids the computational burden of high-dimensional dual variables?
• RQ2 Does the proposed method maintain strong convergence guarantees in online settings with sequential data?
• RQ3 Can the method achieve optimal sample complexity without requiring large batch sizes under standard assumptions?
• RQ4 To what extent can the method be generalized to other stochastic compositional optimization problems?
• RQ5 How does the method compare in practice to existing primal-dual approaches in terms of convergence speed and robustness?

Key findings

• The proposed duality-free method eliminates the need to maintain and update high-dimensional dual variables, significantly reducing computational overhead.
• The method achieves optimal sample complexity under the Polyak-Łojasiewicz (PL) condition, even without large batch sizes.
• The method is compatible with online learning, enabling efficient training on sequentially arriving data streams.
• The convergence behavior of the method resembles that of the stochastic Nesterov method, benefiting from similar acceleration properties.
• The approach can be extended to solve a family of stochastic compositional problems beyond distributionally robust deep learning.