[Paper Review] Hybrid Models for Learning to Branch

Prateek Gupta, Maxime Gasse+4 more|arXiv (Cornell University)|Jun 26, 2020

Topic Modeling7 references48 citations

TL;DR

The paper proposes a CPU-friendly hybrid architecture that combines a root-node Graph Neural Network with fast MLP-based branching to speed up MILP solving, achieving up to 26% faster runtimes without GPUs compared to state-of-the-art CPU methods.

ABSTRACT

A recent Graph Neural Network (GNN) approach for learning to branch has been shown to successfully reduce the running time of branch-and-bound algorithms for Mixed Integer Linear Programming (MILP). While the GNN relies on a GPU for inference, MILP solvers are purely CPU-based. This severely limits its application as many practitioners may not have access to high-end GPUs. In this work, we ask two key questions. First, in a more realistic setting where only a CPU is available, is the GNN model still competitive? Second, can we devise an alternate computationally inexpensive model that retains the predictive power of the GNN architecture? We answer the first question in the negative, and address the second question by proposing a new hybrid architecture for efficient branching on CPU machines. The proposed architecture combines the expressive power of GNNs with computationally inexpensive multi-layer perceptrons (MLP) for branching. We evaluate our methods on four classes of MILP problems, and show that they lead to up to 26% reduction in solver running time compared to state-of-the-art methods without a GPU, while extrapolating to harder problems than it was trained on. The code for this project is publicly available at https://github.com/pg2455/Hybrid-learn2branch.

Motivation & Objective

• Motivate practical branching improvements when GPUs are unavailable.
• Develop a hybrid architecture that preserves GNN benefits while using inexpensive computation at most tree nodes.
• Investigate training protocols to improve generalization and performance in CPU-only settings.
• Evaluate on multiple MILP problem classes to demonstrate time savings and scalability.

Proposed method

• Introduce a hybrid architecture that uses a GNN at the root node to extract structural information and an MLP at remaining nodes to predict branching candidates.
• Combine root GNN representations with cheap node features via multiple fusion strategies (CONCAT, FiLM, HyperSVM, HyperSVM-FiLM).
• Train using imitation learning with strong branching as the expert, via cross-entropy loss; explore end-to-end training and pretraining variants.
• Enhance training with knowledge distillation from a pre-trained GNN, auxiliary tasks (ED or MHE) to diversify representations, and depth-weighted loss schemes.
• Propose and compare several data-extraction schemes with different computational costs to cover the spectrum from pure GNN to pure MLP.
• Evaluate on four MILP families (Capacitated Facility Location, Combinatorial Auctions, Set Covering, Independent Set) and analyze time, nodes, and solution quality.

Experimental results

Research questions

• RQ1 Is a CPU-only hybrid model competitive with GPU-accelerated GNNs for learning to branch in MILP?
• RQ2 Can a hybrid architecture (root GNN plus fast MLP) match or surpass pure MLP or pure GNN performance on CPU hardware?
• RQ3 Do training protocols like end-to-end training, knowledge distillation, auxiliary tasks, and depth-weighting improve generalization and time efficiency?
• RQ4 Which problem classes benefit most from the proposed hybrid approach and how does extrapolation to larger instances behave?
• RQ5 What are the trade-offs between different fusion strategies (CONCAT, FiLM, HyperSVM variants) in terms of accuracy and runtime?

Key findings

• Hybrid models achieve up to 26% reduction in solver running time on CPU compared to state-of-the-art CPU baselines.
• GNN-only approaches are not competitive on CPU without GPU inference, highlighting the value of the hybrid design.
• FiLM and CONCAT fusion strategies generally outperform other hybrids, while HyperSVM-based hybrids are cheaper but less accurate.
• Training protocols (end-to-end, knowledge distillation, auxiliary tasks, depth-based loss weighting) yield measurable accuracy gains and improved B&B performance, with KD often providing better generalization.
• FiLM-based hybrid models match or exceed non-ML baselines more consistently across problem families, and can extrapolate to harder instances beyond training distribution.