[Paper Review] Graph Learning for Combinatorial Optimization: A Survey of State-of-the-Art

Yün Peng, Byron Choi+1 more|arXiv (Cornell University)|Aug 26, 2020

Advanced Graph Neural Networks88 references6 citations

TL;DR

This survey provides a comprehensive overview of graph learning methods for solving combinatorial optimization (CO) problems, categorizing recent approaches into two-stage frameworks: graph representation learning (via graph embedding or end-to-end learning) followed by solution prediction using non-autoregressive or autoregressive methods. The key contribution is a systematic taxonomy of state-of-the-art techniques, highlighting their mechanisms, limitations, and future research directions in generalization, task-specific design, and integration with traditional heuristics.

ABSTRACT

Graphs have been widely used to represent complex data in many applications. Efficient and effective analysis of graphs is important for graph-based applications. However, most graph analysis tasks are combinatorial optimization (CO) problems, which are NP-hard. Recent studies have focused a lot on the potential of using machine learning (ML) to solve graph-based CO problems. Most recent methods follow the two-stage framework. The first stage is graph representation learning, which embeds the graphs into low-dimension vectors. The second stage uses ML to solve the CO problems using the embeddings of the graphs learned in the first stage. The works for the first stage can be classified into two categories, graph embedding (GE) methods and end-to-end (E2E) learning methods. For GE methods, learning graph embedding has its own objective, which may not rely on the CO problems to be solved. The CO problems are solved by independent downstream tasks. For E2E learning methods, the learning of graph embeddings does not have its own objective and is an intermediate step of the learning procedure of solving the CO problems. The works for the second stage can also be classified into two categories, non-autoregressive methods and autoregressive methods. Non-autoregressive methods predict a solution for a CO problem in one shot. A non-autoregressive method predicts a matrix that denotes the probability of each node/edge being a part of a solution of the CO problem. The solution can be computed from the matrix. Autoregressive methods iteratively extend a partial solution step by step. At each step, an autoregressive method predicts a node/edge conditioned to current partial solution, which is used to its extension. In this survey, we provide a thorough overview of recent studies of the graph learning-based CO methods. The survey ends with several remarks on future research directions.

Motivation & Objective

• To provide a systematic review of recent machine learning-based approaches for solving NP-hard combinatorial optimization problems on graphs.
• To categorize and analyze the two-stage framework used in most current graph learning-based CO methods: graph representation learning followed by solution prediction.
• To identify key challenges in generalization, loss function design, and model architecture adaptation for diverse CO problems.
• To highlight open research problems such as encoding global graph structure, integrating traditional heuristics, and scaling to large graph sets.

Proposed method

• Classifies graph representation learning into two types: graph embedding methods (with independent objectives) and end-to-end learning methods (jointly optimized with CO tasks).
• Categorizes solution prediction into non-autoregressive methods, which predict a solution matrix in one step using probability matrices, and autoregressive methods, which iteratively extend partial solutions.
• Reviews key techniques including SkipGram-based methods, autoencoder-based methods, and GNN-based representations for graph encoding.
• Analyzes the use of heuristic search and reinforcement learning in autoregressive methods to guide partial solution extension.
• Examines loss functions such as cross-entropy and their limitations in capturing CO problem objectives, especially in problems like TSP where small changes drastically alter solution quality.
• Proposes future methodological directions, including designing problem-specific loss functions and integrating traditional heuristic operations (e.g., 2-opt, farthest insertion) into learning pipelines.

Experimental results

Research questions

• RQ1 How do graph representation learning techniques differ in their objectives and applicability to downstream combinatorial optimization tasks?
• RQ2 What are the key differences and trade-offs between non-autoregressive and autoregressive solution prediction methods in graph-based CO?
• RQ3 Why do current learning-based CO methods often fail to generalize to larger or unseen graphs compared to traditional heuristics?
• RQ4 How can traditional heuristic algorithms be effectively integrated into machine learning pipelines for CO problems to improve performance?
• RQ5 What are the most promising directions for improving the generalization and scalability of graph learning-based CO methods?

Key findings

• Non-autoregressive methods predict solution matrices in one step, with solutions derived via heuristics like beam search, offering fast inference but potentially suboptimal results.
• Autoregressive methods iteratively extend partial solutions using reinforcement learning or heuristic search, enabling better solution quality at the cost of higher inference time.
• End-to-end learning methods learn graph embeddings specifically for the CO task, while graph embedding methods use fixed, pre-trained representations, leading to different performance trade-offs.
• Current methods often underperform traditional heuristics on large or unseen graphs, indicating a critical need for improved generalization in learning pipelines.
• Integrating traditional heuristics—such as 2-opt or farthest-first insertion—into learning models can significantly improve solution quality, as demonstrated in TSP applications.
• Designing problem-specific loss functions is essential, as standard losses like cross-entropy may fail to reflect the true objective of CO problems like TSP, where small structural changes cause large objective shifts.