[Paper Review] Learning to Optimize: A Primer and A Benchmark

Tianlong Chen, Xiaohan Chen+5 more|arXiv (Cornell University)|Mar 23, 2021

Machine Learning and Data Classification240 references92 citations

TL;DR

A comprehensive survey and benchmark of Learning to Optimize (L2O) methods for continuous optimization, outlining taxonomies, architectures (model-free and model-based), training strategies, and reproducible Open-L2O benchmarks.

ABSTRACT

Learning to optimize (L2O) is an emerging approach that leverages machine learning to develop optimization methods, aiming at reducing the laborious iterations of hand engineering. It automates the design of an optimization method based on its performance on a set of training problems. This data-driven procedure generates methods that can efficiently solve problems similar to those in the training. In sharp contrast, the typical and traditional designs of optimization methods are theory-driven, so they obtain performance guarantees over the classes of problems specified by the theory. The difference makes L2O suitable for repeatedly solving a certain type of optimization problems over a specific distribution of data, while it typically fails on out-of-distribution problems. The practicality of L2O depends on the type of target optimization, the chosen architecture of the method to learn, and the training procedure. This new paradigm has motivated a community of researchers to explore L2O and report their findings. This article is poised to be the first comprehensive survey and benchmark of L2O for continuous optimization. We set up taxonomies, categorize existing works and research directions, present insights, and identify open challenges. We also benchmarked many existing L2O approaches on a few but representative optimization problems. For reproducible research and fair benchmarking purposes, we released our software implementation and data in the package Open-L2O at https://github.com/VITA-Group/Open-L2O.

Motivation & Objective

• Define the Learning to Optimize (L2O) paradigm and contrast it with classic theory-driven optimization methods.
• Survey model-free and model-based L2O approaches and their training mechanisms.
• Provide benchmark experiments and reproducible Open-L2O resources for fair comparison.

Proposed method

• Categorize L2O approaches into model-free and model-based, detailing architectures (e.g., LSTM, RNNs, MLPs) and update rule formulations.
• Describe the unrolling-based training objective that optimizes a weighted sum of objective values over a horizon T (Eq. 1).
• Discuss design choices: input features, base optimizer integration, and training strategies to improve generalization and stability.
• Summarize key techniques to address truncation bias, memory limitations, and generalization (e.g., random scaling, hierarchical RNNs, off-policy imitation).
• Present alternative implementations (reinforcement learning, neural symbolic methods) and non-gradient optimization tasks (black-box, minimax).
• Provide Open-L2O benchmark suite for reproducible evaluation.

Experimental results

Research questions

• RQ1 How can L2O optimizers be formulated and trained to outperform traditional hand-crafted optimizers on similar task distributions?
• RQ2 What are the main model-based and model-free approaches in L2O, and how do their training objectives and architectures affect performance and generalization?
• RQ3 What training techniques and architectural designs mitigate issues like truncation bias and memory bottlenecks in L2O?
• RQ4 How do model-free L2O methods perform on non-standard optimization tasks such as black-box, minimax, or swarm-based problems?
• RQ5 What resources and benchmarks (Open-L2O) are available to enable reproducible evaluation of L2O methods?

Key findings

• L2O methods can converge faster than classic optimizers on unseen optimizees from the same task distribution under suitable training.
• Model-free L2O primarily uses RNNs/LSTMs to learn update rules and can achieve efficiency gains but faces generalization and stability challenges for longer horizons.
• Model-based L2O leverages existing optimization structures as starting points to reduce search space and improve robustness.
• Training techniques (e.g., truncation management, random scaling, hierarchical architectures) help mitigate instability and enhance generalization.
• Open-L2O provides reproducible software and data to benchmark L2O methods across representative optimization problems.