[Paper Review] Attention, Learn to Solve Routing Problems!

Wouter Kool, Herke van Hoof+1 more|UvA-DARE (University of Amsterdam)|Mar 22, 2018

Vehicle Routing Optimization Methods191 citations

TL;DR

The paper presents an attention-based encoder-decoder model trained with REINFORCE using a greedy rollout baseline to learn heuristics for routing problems, achieving strong results on TSP, VRP variants, OP, PCTSP, and SPCTSP with a single set of hyperparameters.

ABSTRACT

The recently presented idea to learn heuristics for combinatorial optimization problems is promising as it can save costly development. However, to push this idea towards practical implementation, we need better models and better ways of training. We contribute in both directions: we propose a model based on attention layers with benefits over the Pointer Network and we show how to train this model using REINFORCE with a simple baseline based on a deterministic greedy rollout, which we find is more efficient than using a value function. We significantly improve over recent learned heuristics for the Travelling Salesman Problem (TSP), getting close to optimal results for problems up to 100 nodes. With the same hyperparameters, we learn strong heuristics for two variants of the Vehicle Routing Problem (VRP), the Orienteering Problem (OP) and (a stochastic variant of) the Prize Collecting TSP (PCTSP), outperforming a wide range of baselines and getting results close to highly optimized and specialized algorithms.

Motivation & Objective

• Motivate learning heuristics for combinatorial routing problems to reduce reliance on hand-crafted solvers.
• Propose an attention-based encoder-decoder architecture that improves over prior learned heuristics for routing tasks.
• Show training effectiveness using REINFORCE with a simple greedy rollout baseline.
• Demonstrate broad applicability by solving TSP, VRP variants, Orienteering Problem, Prize Collecting TSP, and stochastic variants with competitive performance.

Proposed method

• Use an attention-based encoder to produce node embeddings from graph-structured inputs.
• Employ a Transformer-like decoder that outputs a permutation (tour) step-by-step with a masking strategy for feasibility.
• Train with REINFORCE using a rollout baseline based on deterministic greedy solutions from the current best policy to reduce gradient variance.
• Adopt a lightweight, efficient decoding strategy with context nodes and single-head attention during logit computation.
• Utilize batch training with fixed hyperparameters across multiple routing problems to demonstrate generalization of the approach.

Experimental results

Research questions

• RQ1 Can an attention-based model learn effective routing heuristics across diverse routing problems with a single hyperparameter set?
• RQ2 Does REINFORCE training with a greedy rollout baseline yield more stable and efficient learning than value-function based critics for routing tasks?
• RQ3 How close can learned heuristics get to specialized or exact solvers on TSP, VRP variants, OP, and PCTSP (including stochastic variants) at practical sizes?
• RQ4 To what extent do the proposed architectural choices (encoder/decoder, masking, context node) impact performance and scalability?
• RQ5 Is the learned policy robust across problem sizes (n up to 100) and various problem types without problem-specific tuning?

Key findings

• The Attention Model achieves substantially improved results over prior learned heuristics for TSP up to 100 nodes, approaching optimal for some instances.
• With the same hyperparameters, the model learns strong heuristics for VRP variants, Orienteering Problem, and stochastic PCTSP, outperforming diverse baselines.
• REINFORCE with a simple greedy rollout baseline provides efficient training and competitive performance compared to actor-critic methods.
• The approach generalizes across multiple routing problems, suggesting the feasibility of a single learned heuristic family for practical routing tasks.
• Greedy decoding and sampling strategies offer flexible trade-offs between solution quality and runtime, often outperforming several non-learned baselines in practice.