[Paper Review] Learning to Iteratively Solve Routing Problems with Dual-Aspect Collaborative Transformer

Yining Ma, Jingwen Li+5 more|arXiv (Cornell University)|Oct 6, 2021

Vehicle Routing Optimization Methods49 citations

TL;DR

Introduces Dual-Aspect Collaborative Transformer (DACT) for solving routing problems by learning separate node and positional embeddings, with cyclic positional encoding, trained via PPO with curriculum learning; achieves strong generalization and improved performance over prior Transformer-based methods on TSP and CVRP.

ABSTRACT

Recently, Transformer has become a prevailing deep architecture for solving vehicle routing problems (VRPs). However, it is less effective in learning improvement models for VRP because its positional encoding (PE) method is not suitable in representing VRP solutions. This paper presents a novel Dual-Aspect Collaborative Transformer (DACT) to learn embeddings for the node and positional features separately, instead of fusing them together as done in existing ones, so as to avoid potential noises and incompatible correlations. Moreover, the positional features are embedded through a novel cyclic positional encoding (CPE) method to allow Transformer to effectively capture the circularity and symmetry of VRP solutions (i.e., cyclic sequences). We train DACT using Proximal Policy Optimization and design a curriculum learning strategy for better sample efficiency. We apply DACT to solve the traveling salesman problem (TSP) and capacitated vehicle routing problem (CVRP). Results show that our DACT outperforms existing Transformer based improvement models, and exhibits much better generalization performance across different problem sizes on synthetic and benchmark instances, respectively.

Motivation & Objective

• Motivate improved neural solvers for VRPs by addressing limitations of existing Transformer encoders and positional encodings.
• Propose a dual-aspect representation to separately encode node and positional information to reduce noise and improve learning.
• Develop a cyclic positional encoding (CPE) to capture circularity and symmetry in VRP solutions.
• Train the model with Proximal Policy Optimization and curriculum learning to enhance sample efficiency and convergence.
• Demonstrate generalization and performance gains of the proposed method on TSP and CVRP across multiple sizes.

Proposed method

• Introduce Dual-Aspect Collaborative Transformer (DACT) with separate node feature embeddings (NFEs) and positional feature embeddings (PFEs).
• Use a DAC encoder with self-attention per aspect plus cross-aspect referential attention to allow cross-utility between NFEs and PFEs.
• Embed PFEs with Cyclic Positional Encoding (CPE) based on cyclic Gray codes to reflect cyclicity and adjacency in VRP solutions.
• In the decoder, aggregate proposals from both aspects via Max-pooling and a Multi-Head Compatibility (MHC) layer, followed by a Feed-Forward Aggregation (FFA) to output action likelihoods.
• Train with Proximal Policy Optimization (PPO) and employ a curriculum learning strategy that starts from lower-quality solutions and gradually increases initial solution quality to improve sample efficiency.

Experimental results

Research questions

• RQ1 Can a dual-aspect embedding strategy improve the representation of VRP solutions over unified embeddings in standard Transformers?
• RQ2 Does cyclic positional encoding better capture the cyclic nature of VRP solutions and improve generalization across problem sizes?
• RQ3 How does curriculum learning affect sample efficiency and convergence in PPO-based training for neural VRP solvers?
• RQ4 What are the generalization capabilities of DACT on TSP and CVRP across different instance sizes and benchmark datasets?
• RQ5 How does DACT compare to state-of-the-art neural and traditional solvers in terms of solution quality and inference time?

Key findings

• DACT outperforms existing Transformer-based improvement models with fewer parameters.
• CPE enables better encoding of cyclic structure and adjacency, improving generalization across problem sizes.
• Dual-aspect representation consistently improves performance over single-aspect baselines on TSP50/100.
• Curriculum learning improves sample efficiency and stability of PPO training, leading to faster convergence.
• DACT achieves competitive or superior gaps on TSP and CVRP benchmarks, and shows strong generalization on TSPLIB and CVRPLIB datasets.