[论文解读] Rethinking LLM-Driven Heuristic Design: Generating Efficient and Specialized Solvers via Dynamics-Aware Optimization

Rongzheng Wang, Yihong Huang+7 more|arXiv (Cornell University)|Jan 14, 2026

Advanced Multi-Objective Optimization Algorithms被引用 0

一句话总结

引入DASH，一种面向动态的框架，通过协同优化求解器搜索机制与运行时调度，并结合基于配置文件的热启动以应对分布偏移，从而生成高效且专业化的求解器。

ABSTRACT

Large Language Models (LLMs) have advanced the field of Combinatorial Optimization through automated heuristic generation. Instead of relying on manual design, this LLM-Driven Heuristic Design (LHD) process leverages LLMs to iteratively generate and refine solvers to achieve high performance. However, existing LHD frameworks face two critical limitations: (1) Endpoint-only evaluation, which ranks solvers solely by final quality, ignoring the convergence process and runtime efficiency; (2) High adaptation costs, where distribution shifts necessitate re-adaptation to generate specialized solvers for new instance groups. To address these issues, we propose Dynamics-Aware Solver Heuristics (DASH), a framework that co-optimizes solver search mechanisms and runtime schedules guided by a convergence-aware metric, thereby identifying efficient and high-performance solvers. Furthermore, to mitigate expensive re-adaptation, DASH incorporates Profiled Library Retrieval (PLR). PLR efficiently archives specialized solvers concurrently with the evolutionary process, enabling cost-effective warm-starts for heterogeneous distributions. Experiments on four combinatorial optimization problems demonstrate that DASH improves runtime efficiency by over 3 times, while surpassing the solution quality of state-of-the-art baselines across diverse problem scales. Furthermore, by enabling profile-based warm starts, DASH maintains superior accuracy under different distributions while cutting LLM adaptation costs by over 90%.

研究动机与目标

• Motivate the limitations of endpoint-only evaluation and high adaptation costs in LLM-driven heuristic design for combinatorial optimization.
• Propose a dynamics-aware framework (DASH) that co-optimizes search mechanisms and runtime schedules guided by a convergence-aware metric.
• Introduce Profiled Library Retrieval (PLR) to archive and reuse specialized solvers for cost-effective warm starts across distributions.
• Validate DASH on multiple combinatorial problems and solver backbones, showing improvements in runtime efficiency and solution quality.
• Provide a trajectory-centric evaluation tool (tLDR) to quantify convergence dynamics and guide co-evolution.

提出的方法

• Model solver execution as a time-evolving trajectory with z(k+1)=F(z(k); x, θ, σ) and cumulative time τ.
• Define a trajectory-based Lyapunov potential V(τ) via the incumbent gap to target optimality and log-space progress ell(τ) for scale-consistent comparison.
• Introduce the Trajectory-aware Lyapunov Decay Rate (tLDR) kappa(T) = (2/T)(ell(0) - J(T)) to measure fast and sustained convergence.
• Decompose optimization into three layers: Mechanism Discovery Layer (MDL) for θ updates, Mechanism Consolidation Layer (MCL) for code refactoring, and Schedule Shaping Layer (SSL) for σ updates.
• Employ two-stage SSL: Compression to reduce runtime slack, then Enhancement to better allocate budget; acceptance guided by ell(T) and tLDR.
• Incorporate Profiled Library Retrieval (PLR) to maintain a global population and group-specific archives for fast, profile-aware warm starts at test time.

实验结果

研究问题

• RQ1 Can dynamics-aware evaluation (trajectory-based) improve the quality and efficiency of LLM-driven heuristic design compared with endpoint-only methods?
• RQ2 Do co-evolving solver mechanisms and runtime schedules yield better performance than evolving either component alone?
• RQ3 Does Profiled Library Retrieval (PLR) improve generalization and reduce adaptation costs across distribution shifts?
• RQ4 How well does the approach transfer across different solver backbones and problem domains within combinatorial optimization?

主要发现

• DASH achieves over 3x runtime efficiency improvements while surpassing state-of-the-art baselines in solution quality across four combinatorial problems.
• tLDR-guided co-evolution prefers solvers with fast and stable anytime improvement, avoiding endpoint-only trade-offs.
• PLR enables profile-based warm starts, maintaining accuracy under distribution shifts and cutting LLM adaptation costs by over 90%.
• DASH transfers across solver backbones (GLS, ILS, LKH) and remains robust under distribution shifts without excessive token usage.
• Ablation shows SSL compression drives most runtime reductions, and removing tLDR degrades both runtime and quality, while PLR contributes to both metrics.