[Paper Review] AutoPhase: Compiler Phase-Ordering for High Level Synthesis with Deep Reinforcement Learning

Ameer Haj-Ali, Qijing Huang+5 more|arXiv (Cornell University)|Jan 15, 2019

Evolutionary Algorithms and Applications24 references4 citations

TL;DR

This paper proposes AutoPhase, a deep reinforcement learning framework that optimizes compiler phase ordering in High-Level Synthesis (HLS) to improve circuit performance. By using DQN and Policy Gradient algorithms on LLVM IR features, AutoPhase achieves 16% better performance than -O3 with 1-2 orders of magnitude faster search, demonstrating significant efficiency and effectiveness gains over traditional methods.

ABSTRACT

The performance of the code generated by a compiler depends on the order in which the optimization passes are applied. In high-level synthesis, the quality of the generated circuit relates directly to the code generated by the front-end compiler. Choosing a good order--often referred to as the phase-ordering problem--is an NP-hard problem. In this paper, we evaluate a new technique to address the phase-ordering problem: deep reinforcement learning. We implement a framework in the context of the LLVM compiler to optimize the ordering for HLS programs and compare the performance of deep reinforcement learning to state-of-the-art algorithms that address the phase-ordering problem. Overall, our framework runs one to two orders of magnitude faster than these algorithms, and achieves a 16% improvement in circuit performance over the -O3 compiler flag.

Motivation & Objective

• To address the NP-hard phase-ordering problem in High-Level Synthesis (HLS), where optimization pass order critically affects circuit performance.
• To explore whether deep reinforcement learning can outperform traditional heuristic and genetic algorithms in finding optimal phase sequences.
• To develop a framework that learns efficient phase-ordering policies with minimal retraining for new programs.
• To evaluate the trade-off between performance improvement and training/inference efficiency in the context of HLS optimization.

Proposed method

• Extracted 56 static features from LLVM IR, including basic block count, branch count, and instruction type frequencies, to represent program states.
• Represented states either as raw feature vectors or histograms of previously applied optimization passes.
• Trained deep reinforcement learning agents using DQN and Policy Gradient algorithms to learn phase-ordering policies that maximize circuit performance.
• Used the number of clock cycles reported by LegUp's HLS profiler as the reward signal for policy optimization.
• Evaluated the framework on 12 HLS benchmarks from CHstone and LegUp examples with varying sequence lengths (3, 12, 24) to assess scalability.
• Compared the RL-based approach against random search, greedy algorithms, genetic algorithms, and the -O3 compiler flag.

Experimental results

Research questions

• RQ1 Can deep reinforcement learning effectively learn optimal compiler phase-ordering sequences for HLS workloads?
• RQ2 How does the performance of RL-based phase ordering compare to state-of-the-art algorithms like genetic algorithms and greedy heuristics in terms of circuit speedup?
• RQ3 What is the trade-off between training time and optimization quality when using RL for phase ordering in HLS?
• RQ4 Can a trained RL agent generalize across different HLS programs with minimal retraining?
• RQ5 Does increasing the number of applied optimization passes beyond a certain point yield diminishing returns in performance improvement?

Key findings

• AutoPhase achieved a 16% improvement in circuit performance over the -O3 compiler flag, demonstrating significant optimization gains.
• The framework runs one to two orders of magnitude faster than genetic algorithms and other state-of-the-art methods.
• Increasing the sequence length from 12 to 24 passes yielded little to no additional performance improvement, indicating diminishing returns.
• Reinforcement learning methods matched or exceeded the performance of genetic algorithms while being significantly faster to train and execute.
• A trained RL agent showed potential for generalization, enabling fast optimization of new programs with minimal retraining.