[Paper Review] A Static Analysis-based Cross-Architecture Performance Prediction Using Machine Learning

Newsha Ardalani, Urmish Thakker+2 more|arXiv (Cornell University)|Jun 18, 2019

Parallel Computing and Optimization Techniques26 references4 citations

TL;DR

This paper proposes Static XAPP, a machine learning framework that predicts cross-architecture GPU speedup using only static analysis of single-threaded CPU source code. By extracting 10 program properties via static analysis and training a random forest classifier on discretized speedup thresholds, the method achieves 94% average accuracy in binary classification across different cutoffs, demonstrating that static features alone can enable accurate performance prediction without dynamic execution or human input.

ABSTRACT

Porting code from CPU to GPU is costly and time-consuming; Unless much time is invested in development and optimization, it is not obvious, a priori, how much speed-up is achievable or how much room is left for improvement. Knowing the potential speed-up a priori can be very useful: It can save hundreds of engineering hours, help programmers with prioritization and algorithm selection. We aim to address this problem using machine learning in a supervised setting, using solely the single-threaded source code of the program, without having to run or profile the code. We propose a static analysis-based cross-architecture performance prediction framework (Static XAPP) which relies solely on program properties collected using static analysis of the CPU source code and predicts whether the potential speed-up is above or below a given threshold. We offer preliminary results that show we can achieve 94% accuracy in binary classification, in average, across different thresholds

Motivation & Objective

• To address the high cost and uncertainty of porting CPU code to GPU by enabling early prediction of potential speedup.
• To eliminate reliance on dynamic binary instrumentation or manual feature estimation, which are slow and error-prone.
• To develop a framework that uses only static analysis of CPU source code to predict whether a program will achieve high or low speedup on GPU.
• To enable practical integration into IDEs and device placement optimization by providing fast, automated performance predictions.
• To demonstrate that static program properties are sufficiently explanatory for cross-architecture performance prediction, challenging the assumption that dynamic execution is necessary.

Proposed method

• The framework collects 10 program properties—such as memory access patterns, loop structures, and data dependencies—using static analysis on single-threaded CPU code.
• Feature values are discretized into two to three levels using equal frequency binning to improve robustness and reduce noise.
• The output speedup is discretized into two classes—low or high—based on a user-defined cutoff threshold, enabling binary classification.
• A random forest classifier with 1000 trees is trained on the labeled dataset, using random subsets of features and training examples per tree.
• The model is evaluated using leave-one-out cross-validation (LOOCV) across multiple GPU platforms and cutoff values to assess stability and generalization.
• The dataset is constructed from benchmark suites (Lonestar, Rodinia, NAS) and augmented with custom microbenchmarks to include negative examples (poor GPU candidates).

Experimental results

Research questions

• RQ1 Can static analysis of CPU source code alone predict whether a program will achieve high or low speedup on GPU?
• RQ2 How accurate is a machine learning model trained on static features in predicting cross-architecture performance across different GPU platforms?
• RQ3 How does model accuracy vary with different speedup cutoff thresholds and platform-specific performance characteristics?
• RQ4 Can the framework maintain high accuracy and stability when the number of speedup classes increases beyond binary classification?
• RQ5 Is the framework robust to dataset bias and small sample sizes, given the limited number of high-quality GPU-optimized kernels?

Key findings

• The framework achieves an average cross-validation accuracy of 94% in binary classification across different speedup cutoffs and GPU platforms.
• On Platform 1 (GTX 750, Maxwell), accuracy ranges from 79% (minimum) to 97% (maximum), with an average of 86% across cutoffs.
• On Platform 2 (GTX 660 Ti, Kepler), accuracy ranges from 76.5% (minimum) to 89% (maximum), with an average of 83% across cutoffs.
• The model remains robust to variations in cutoff values and platform-specific performance distributions, indicating strong generalization.
• Multi-class classification accuracy decreases with more speedup bins, as expected due to reduced data per class, but the model still performs reliably across 2 to 5 classes.
• The study demonstrates that static program properties are sufficiently explanatory for performance prediction, challenging the necessity of dynamic execution or human input.