[论文解读] DeepACO: Neural-enhanced Ant Systems for Combinatorial Optimization

Haoran Ye, Jiarui Wang+3 more|arXiv (Cornell University)|Sep 25, 2023

Metaheuristic Optimization Algorithms Research被引用 23

一句话总结

DeepACO 是一个通用的神经强化学习增强的蚁群优化框架，通过深度强化学习学习问题特定的启发式度量，并用它们来引导解的构建与局部搜索，覆盖八个COPs，优于传统的ACO并与问题特定的NCO方法相匹配。

ABSTRACT

Ant Colony Optimization (ACO) is a meta-heuristic algorithm that has been successfully applied to various Combinatorial Optimization Problems (COPs). Traditionally, customizing ACO for a specific problem requires the expert design of knowledge-driven heuristics. In this paper, we propose DeepACO, a generic framework that leverages deep reinforcement learning to automate heuristic designs. DeepACO serves to strengthen the heuristic measures of existing ACO algorithms and dispense with laborious manual design in future ACO applications. As a neural-enhanced meta-heuristic, DeepACO consistently outperforms its ACO counterparts on eight COPs using a single neural architecture and a single set of hyperparameters. As a Neural Combinatorial Optimization method, DeepACO performs better than or on par with problem-specific methods on canonical routing problems. Our code is publicly available at https://github.com/henry-yeh/DeepACO.

研究动机与目标

• Motivate automation of heuristic design in ACO to reduce manual engineering.
• Develop a generic neural module that learns instance-specific heuristics transferable across COPs.
• Integrate learned heuristics with ACO construction and local search to improve solutions.
• Provide extensions balancing exploration and exploitation in heatmap-based NCO methods.

提出的方法

• Introduce a graph neural network (GNN) to map COP instances to heuristic measures for all solution components.
• bias solution construction with learned heuristics in an ACO framework and use standard pheromone updates.
• Train the heuristic learner across instances using a REINFORCE-based objective that combines direct construction quality and NLS-refined solutions (Eq. 4).
• Interleave local search with neural-guided perturbation (NLS) to escape local optima and improve exploration.
• Present three extensions to enhance exploration: multihead decoder with KL divergence loss, top-k entropy loss, and imitation loss.
• Demonstrate applicability across eight COPs (routing, assignment, scheduling, subset) and robustness to hyperparameters.

实验结果

研究问题

• RQ1 Can a single neural architecture learn effective problem-specific heuristics to guide ACO across diverse COPs?
• RQ2 Does neural enhancement improve both traditional ACO and neural combinatorial optimization (NCO) performance across routing, scheduling, and subset problems?
• RQ3 Do extensions (multihead, entropy, imitation) improve exploration without sacrificing exploitation?
• RQ4 How does DeepACO compare to problem-specific NCO methods on canonical routing problems and to ACO baselines on eight COPs?
• RQ5 Is the learned heuristic space compact enough to avoid requiring expert-crafted heuristics for new COPs?

主要发现

• DeepACO consistently outperforms baseline ACO variants across eight COPs with a single neural architecture and hyperparameters.
• DeepACO competes with or surpasses problem-specific NCO methods on canonical routing problems.
• Three extension designs (multihead, top-k entropy loss, imitation loss) improve exploration and performance on smaller COPs.
• DeepACO shows robustness to hyperparameters and generalizes across instance scales and distributions.
• Neural-guided perturbation and integrated LS enhance solution quality when combined with learned heuristics.