[Paper Review] Sym-NCO: Leveraging Symmetricity for Neural Combinatorial Optimization

Min-Su Kim, Junyoung Park+1 more|arXiv (Cornell University)|May 26, 2022

Vehicle Routing Optimization Methods18 citations

TL;DR

This paper proposes Sym-NCO, a regularization-based training scheme that leverages universal symmetricities—such as rotational and reflectional invariance—in combinatorial optimization problems to improve deep reinforcement learning (DRL)-based solvers. By incorporating symmetricity-aware baselines in the REINFORCE algorithm, Sym-NCO enhances generalization and convergence, achieving state-of-the-art performance on TSP, CVRP, PCTSP, and OP, with a 240× speedup over ILS on PCTSP while outperforming existing DRL-NCO methods and conventional solvers.

ABSTRACT

Deep reinforcement learning (DRL)-based combinatorial optimization (CO) methods (i.e., DRL-NCO) have shown significant merit over the conventional CO solvers as DRL-NCO is capable of learning CO solvers less relying on problem-specific expert domain knowledge (heuristic method) and supervised labeled data (supervised learning method). This paper presents a novel training scheme, Sym-NCO, which is a regularizer-based training scheme that leverages universal symmetricities in various CO problems and solutions. Leveraging symmetricities such as rotational and reflectional invariance can greatly improve the generalization capability of DRL-NCO because it allows the learned solver to exploit the commonly shared symmetricities in the same CO problem class. Our experimental results verify that our Sym-NCO greatly improves the performance of DRL-NCO methods in four CO tasks, including the traveling salesman problem (TSP), capacitated vehicle routing problem (CVRP), prize collecting TSP (PCTSP), and orienteering problem (OP), without utilizing problem-specific expert domain knowledge. Remarkably, Sym-NCO outperformed not only the existing DRL-NCO methods but also a competitive conventional solver, the iterative local search (ILS), in PCTSP at 240 faster speed. Our source code is available at https://github.com/alstn12088/Sym-NCO.

Motivation & Objective

• To close the performance gap between DRL-based and conventional combinatorial optimization solvers without relying on problem-specific heuristics.
• To improve generalization and training efficiency of DRL-NCO models by exploiting universal symmetricities inherent in Euclidean combinatorial optimization problems.
• To develop a general, architecture-agnostic training regularization scheme that enhances existing DRL-NCO models without requiring architectural redesign.
• To validate that symmetricity acts as a strong inductive bias that compactifies the training space and promotes invariant representations.
• To demonstrate the superiority of symmetricity regularization over equivariant neural networks in CO tasks, even when the latter are theoretically guaranteed to be equivariant.

Proposed method

• Introduces a symmetricity-aware baseline in the REINFORCE algorithm, where multiple rotated/reflected problem instances are sampled and their average return is used as a baseline.
• Employs a symmetricity regularization loss, $\mathcal{L}_{\text{inv}}$, to enforce invariant representations in the encoder's hidden states by maximizing cosine similarity between representations of symmetrically transformed inputs.
• Applies rotation augmentation to generate multiple problem variants during training, enabling the policy to learn symmetric solutions through shared optimization objectives.
• Uses a projection head $g(\cdot)$ to project hidden representations for symmetricity regularization, preserving the model's expressive capacity while enforcing invariance.
• Integrates the symmetricity regularization into existing DRL-NCO frameworks (e.g., POMO) without modifying the neural architecture, enabling plug-and-play performance gains.
• Employs a multi-task training objective combining standard reinforcement learning loss and symmetricity regularization to jointly optimize for solution quality and symmetricity.

Experimental results

Research questions

• RQ1 Can symmetricity regularization improve the generalization and convergence of DRL-NCO models across diverse combinatorial optimization problems?
• RQ2 Does leveraging universal symmetricities such as rotational and reflectional invariance lead to better performance than problem-specific heuristics or equivariant architectures?
• RQ3 Is symmetricity regularization more effective than architectural equivariance (e.g., EGNN) in CO tasks with fully connected, coordinate-based graphs?
• RQ4 To what extent does symmetricity regularization reduce the effective size of the training space and improve sample efficiency?
• RQ5 Can symmetricity regularization be applied universally to existing DRL-NCO models without architectural changes, and does it maintain or improve performance?

Key findings

• Sym-NCO significantly improves the performance of existing DRL-NCO models across four CO tasks: TSP, CVRP, PCTSP, and OP, without requiring problem-specific heuristics.
• On the PCTSP benchmark, Sym-NCO outperformed the conventional iterative local search (ILS) solver while being 240× faster, demonstrating superior speed-accuracy trade-offs.
• The symmetricity regularization loss $\mathcal{L}_{\text{inv}}$ increased cosine similarity between representations of symmetrically transformed inputs, confirming effective invariant representation learning.
• Ablation studies showed that applying symmetricity regularization to the encoder's hidden states ($h$) degraded performance, confirming that the projection head $g(\cdot)$ is essential to preserve expressivity while enforcing invariance.
• EGNN, despite being theoretically equivariant, underperformed Sym-NCO and failed to converge, indicating that equivariance alone is insufficient—symmetricity regularization is more effective in this context.
• Sym-NCO achieved Pareto-optimal performance across all benchmark datasets, outperforming all baselines in solution quality within the same time budget.