[Paper Review] Large Neighborhood Search based on Neural Construction Heuristics

Jonas K. Falkner, Daniela Thyssens+1 more|arXiv (Cornell University)|May 2, 2022

Vehicle Routing Optimization Methods4 citations

TL;DR

This paper proposes a Large Neighborhood Search (LNS) framework that uses a graph neural network-based neural construction heuristic (NCH) as a repair operator for the vehicle routing problem with time windows (VRPTW). Trained via reinforcement learning without labels, the NCH enables efficient, differentiable solution reconstruction; the method achieves near-optimal performance, reaching 1.1× the best-known solution on 600-node instances, and demonstrates significant speedup on GPU hardware.

ABSTRACT

We propose a Large Neighborhood Search (LNS) approach utilizing a learned construction heuristic based on neural networks as repair operator to solve the vehicle routing problem with time windows (VRPTW). Our method uses graph neural networks to encode the problem and auto-regressively decodes a solution and is trained with reinforcement learning on the construction task without requiring any labels for supervision. The neural repair operator is combined with a local search routine, heuristic destruction operators and a selection procedure applied to a small population to arrive at a sophisticated solution approach. The key idea is to use the learned model to re-construct the partially destructed solution and to introduce randomness via the destruction heuristics (or the stochastic policy itself) to effectively explore a large neighborhood.

Motivation & Objective

• To develop a scalable and differentiable repair operator for Large Neighborhood Search (LNS) in vehicle routing problems.
• To address the limitations of standard attention-based models in large-scale VRPTW by replacing self-attention with localized graph neural networks.
• To enable effective neighborhood exploration in LNS by combining learned repair with heuristic destruction operators and stochasticity.
• To evaluate the method in a real-world benchmark setting, specifically the DIMACS VRPTW Challenge, to assess competitiveness against hand-crafted solvers.
• To demonstrate the feasibility and performance potential of end-to-end learned construction heuristics in complex combinatorial optimization.

Proposed method

• The method employs a graph neural network (GNN) to encode static spatial neighborhoods of each node, pre-computed and fixed during decoding to reduce complexity from O(N²) to O(Nk).
• Dynamic node embeddings are computed at each decoding step using a GNN over the current route's local neighborhood, capturing route-specific context for action selection.
• The final node embedding is the sum of static and dynamic embeddings: $\tilde{x}_i^{(t)} = \tilde{x}_i^{\text{stat}} + \tilde{x}_i^{\text{dyn}}$, enabling context-aware, autoregressive solution construction.
• A policy network based on attention over the combined embeddings selects the next node to add to the route, using either greedy selection or softmax sampling.
• The neural repair operator is integrated into an LNS framework with heuristic destruction operators (e.g., random, waiting-time-based, route removal) to explore diverse neighborhoods.
• Local search via Guided Local Search (GLS) is applied post-repair to further improve solutions, with time limits scaled by instance size.

Experimental results

Research questions

• RQ1 Can a neural construction heuristic trained via reinforcement learning serve as an effective and generalizable repair operator in Large Neighborhood Search for VRPTW?
• RQ2 How does replacing self-attention with localized GNNs impact scalability and performance on large VRPTW instances?
• RQ3 To what extent can a learned repair operator combined with heuristic destruction improve solution quality compared to traditional LNS with greedy heuristics?
• RQ4 How does the method perform on out-of-distribution instances and larger problem sizes (e.g., 600 nodes) compared to state-of-the-art solvers?
• RQ5 What is the computational advantage of using a GPU-optimized GNN-based NCH in a single-threaded CPU evaluation setting?

Key findings

• The proposed solver achieved an average performance gap of 1.1× the best-known solution (BKS) on 600-node VRPTW instances, with median gaps under 10% across all benchmark types.
• On the Solomon and Homberger & Gehring benchmarks, the method reached 9.99% gap on average for R2, C2, and RC2 instances with 600 nodes, indicating strong scalability.
• The model demonstrated a 15–20× speedup on a consumer-grade GPU (NVIDIA GeForce 1080 Ti) compared to single-threaded CPU inference, highlighting its GPU-native efficiency.
• The use of pre-computed static GNN embeddings reduced computational complexity from O(N²) to O(Nk), enabling feasible inference on large-scale instances.
• The method outperformed baseline LNS with standard heuristics, showing that a learned repair operator can significantly enhance neighborhood exploration and solution quality.
• Despite being in an experimental stage, the approach achieved competitive results against highly tuned, hand-crafted solvers, suggesting strong potential for future improvement through end-to-end fine-tuning.