[Paper Review] Learning to Perform Local Rewriting for Combinatorial Optimization

Xinyun Chen, Yuandong Tian|arXiv (Cornell University)|Sep 30, 2018

Constraint Satisfaction and Optimization54 references155 citations

TL;DR

NeuRewriter learns a policy to iteratively rewrite local parts of a current solution, improving combinatorial optimization tasks without solving from scratch. It outperforms strong baselines across expression simplification, online job scheduling, and vehicle routing.

ABSTRACT

Search-based methods for hard combinatorial optimization are often guided by heuristics. Tuning heuristics in various conditions and situations is often time-consuming. In this paper, we propose NeuRewriter that learns a policy to pick heuristics and rewrite the local components of the current solution to iteratively improve it until convergence. The policy factorizes into a region-picking and a rule-picking component, each parameterized by a neural network trained with actor-critic methods in reinforcement learning. NeuRewriter captures the general structure of combinatorial problems and shows strong performance in three versatile tasks: expression simplification, online job scheduling and vehicle routing problems. NeuRewriter outperforms the expression simplification component in Z3; outperforms DeepRM and Google OR-tools in online job scheduling; and outperforms recent neural baselines and Google OR-tools in vehicle routing problems.

Motivation & Objective

• Motivate reducing manual heuristics tuning by learning a policy-driven local rewriting framework.
• Develop NeuRewriter to iteratively improve a given solution through region- and rule-based rewrites.
• Demonstrate transferability and robustness of the approach across multiple domains.

Proposed method

• Two-part policy: region-picking to select a solution region and rule-picking to choose a rewriting action.
• Policy training via actor-critic reinforcement learning with a Q-function as the region scorer.
• Reward r = c(s_t) - c(s_{t+1}) to encourage cumulative improvement.
• Neural networks parameterize the region-picking Q and the rule-picking policy over domain-specific state representations.
• Domains include expression simplification (Halide parse trees), online job scheduling (dependency graphs), and vehicle routing (routes).
• A unified rewriting pipeline applies a chosen rule to a selected region to obtain the next state and repeats until convergence.

Experimental results

Research questions

• RQ1 Can a learned local rewriting policy outperform hand-tuned heuristics and full-solution neural predictors across diverse combinatorial problems?
• RQ2 Does a region-picking plus rule-picking decomposition generalize across problem domains and distributions?
• RQ3 How does NeuRewriter perform in terms of solution quality and runtime compared to traditional solvers and neural baselines?
• RQ4 What ablations reveal about the contributions of region choice versus rewriting rules?

Key findings

• NeuRewriter reduces expression length and parse-tree size by about 52% and 59% on average in the expression simplification domain.
• It outperforms Z3-simplify, Halide-rule, and heuristic search, and is faster than Z3-ctx-solver-simplify in the reported experiments.
• In online job scheduling, NeuRewriter outperforms Google OR-tools and DeepRM, especially in more complex settings with heterogeneous resources.
• For vehicle routing, NeuRewriter beats recent neural baselines and OR-tools, approaching the offline optimal bound on VRP with 20 nodes.
• Ablations show the approach is robust to distribution shifts and can generalize to longer expressions and different workload configurations.