[Paper Review] AutoPhase: Juggling HLS Phase Orderings in Random Forests with Deep Reinforcement Learning

Ameer Haj-Ali, Qijing Huang+5 more|arXiv (Cornell University)|Mar 15, 2020

Software Engineering Research16 citations

TL;DR

AutoPhase proposes a deep reinforcement learning framework that automatically determines optimal phase orderings in High-Level Synthesis (HLS) compilation using random forests to guide search space reduction. It achieves 28% better performance than -O3 optimization and generalizes effectively across diverse benchmarks after training on only 100 synthetic programs.

ABSTRACT

The performance of the code a compiler generates depends on the order in which it applies the optimization passes. Choosing a good order--often referred to as the phase-ordering problem, is an NP-hard problem. As a result, existing solutions rely on a variety of heuristics. In this paper, we evaluate a new technique to address the phase-ordering problem: deep reinforcement learning. To this end, we implement AutoPhase: a framework that takes a program and uses deep reinforcement learning to find a sequence of compilation passes that minimizes its execution time. Without loss of generality, we construct this framework in the context of the LLVM compiler toolchain and target high-level synthesis programs. We use random forests to quantify the correlation between the effectiveness of a given pass and the program's features. This helps us reduce the search space by avoiding phase orderings that are unlikely to improve the performance of a given program. We compare the performance of AutoPhase to state-of-the-art algorithms that address the phase-ordering problem. In our evaluation, we show that AutoPhase improves circuit performance by 28% when compared to using the -O3 compiler flag, and achieves competitive results compared to the state-of-the-art solutions, while requiring fewer samples. Furthermore, unlike existing state-of-the-art solutions, our deep reinforcement learning solution shows promising result in generalizing to real benchmarks and 12,874 different randomly generated programs, after training on a hundred randomly generated programs.

Motivation & Objective

• To address the NP-hard phase-ordering problem in High-Level Synthesis (HLS) compilers, which significantly impacts generated code performance.
• To reduce reliance on heuristic-based phase ordering strategies that often fail to generalize across diverse programs.
• To develop a learning-based approach that dynamically selects optimal pass sequences based on program-specific features.
• To improve performance over existing state-of-the-art phase-ordering techniques while minimizing the number of required training samples.
• To evaluate generalization capability across real benchmarks and a large set of randomly generated programs after limited training.

Proposed method

• Leverages deep reinforcement learning to learn optimal phase ordering policies for HLS compilation in the LLVM toolchain.
• Employs random forests to model the correlation between program features and pass effectiveness, enabling efficient search space pruning.
• Uses program-specific features to predict which compilation passes are likely to improve performance, reducing the number of candidate phase orderings.
• Trains the reinforcement learning agent on a small set of 100 randomly generated programs to generalize to 12,874 additional synthetic and real benchmarks.
• Integrates the learned policy into the compilation pipeline to minimize execution time through adaptive phase sequencing.

Experimental results

Research questions

• RQ1 Can deep reinforcement learning effectively learn optimal phase orderings in HLS compilation without relying on hand-crafted heuristics?
• RQ2 How does the integration of random forests for feature-based pass effectiveness prediction improve search efficiency in phase-ordering?
• RQ3 To what extent can a model trained on a small set of synthetic programs generalize to real-world and diverse benchmark programs?
• RQ4 How does AutoPhase compare in performance and sample efficiency to existing state-of-the-art phase-ordering techniques?
• RQ5 Can the learned policy achieve consistent performance gains across varied program types and workloads?

Key findings

• AutoPhase improves circuit performance by 28% compared to the default -O3 compiler optimization flag.
• The framework achieves competitive performance relative to state-of-the-art phase-ordering solutions while requiring fewer training samples.
• After being trained on only 100 randomly generated programs, AutoPhase generalizes effectively to 12,874 distinct programs, including real benchmarks.
• The use of random forests to predict pass effectiveness significantly reduces the search space, enhancing training efficiency.
• The deep reinforcement learning policy demonstrates strong generalization, outperforming heuristic-based approaches in diverse program scenarios.