[Paper Review] Ithemal: Accurate, Portable and Fast Basic Block Throughput Estimation using Deep Neural Networks

Charith Mendis, Alex Renda+2 more|arXiv (Cornell University)|Aug 21, 2018

Parallel Computing and Optimization Techniques44 references49 citations

TL;DR

Ithemal uses a hierarchical multiscale LSTM to predict basic-block throughput from assembly sequences, achieving higher accuracy than hand-written models (IACA LLVM-mca) while remaining fast and portable across microarchitectures.

ABSTRACT

Predicting the number of clock cycles a processor takes to execute a block of assembly instructions in steady state (the throughput) is important for both compiler designers and performance engineers. Building an analytical model to do so is especially complicated in modern x86-64 Complex Instruction Set Computer (CISC) machines with sophisticated processor microarchitectures in that it is tedious, error prone, and must be performed from scratch for each processor generation. In this paper we present Ithemal, the first tool which learns to predict the throughput of a set of instructions. Ithemal uses a hierarchical LSTM--based approach to predict throughput based on the opcodes and operands of instructions in a basic block. We show that Ithemal is more accurate than state-of-the-art hand-written tools currently used in compiler backends and static machine code analyzers. In particular, our model has less than half the error of state-of-the-art analytical models (LLVM's llvm-mca and Intel's IACA). Ithemal is also able to predict these throughput values just as fast as the aforementioned tools, and is easily ported across a variety of processor microarchitectures with minimal developer effort.

Motivation & Objective

• Motivate the need for accurate throughput estimation for compiler optimization and performance engineering.
• Propose a data-driven approach to predict basic block throughput without hand-crafted microarchitectural models.
• Develop a hierarchical multiscale RNN architecture to learn instruction embeddings and predict throughput.
• Demonstrate portability across Haswell, Ivy Bridge, and Skylake microarchitectures.
• Provide an open-source implementation for the community.

Proposed method

• Canonicalize the input assembly into a tokenized, structured representation of instructions and operands.
• Embed tokens and per-instruction representations using a token layer and an instruction-level LSTM to produce per-instruction embeddings.
• Use a hierarchical RNN to combine instruction embeddings into a basic-block embedding and predict throughput via a final linear layer.
• Train the model end-to-end on a large labeled dataset of basic blocks with ground-truth throughputs measured on target CPUs.
• Evaluate against hand-written models IACA and llvm-mca in terms of accuracy and speed across multiple microarchitectures.
• Explore architecture variants including DAG-RNN and token-level RNN to assess performance benefits.

Experimental results

Research questions

• RQ1 Can a data-driven model learn to predict basic-block throughput from assembly sequences more accurately than hand-crafted analytical models?
• RQ2 Does a hierarchical multiscale RNN better capture intra-block dependencies and microarchitectural effects than flat or graph-based architectures?
• RQ3 Is the learned model portable across different x86-64 microarchitectures without architecture-specific reengineering?
• RQ4 Can the predictor achieve real-time speed comparable to existing tools while maintaining higher accuracy?

Key findings

• Ithemal achieves higher prediction accuracy than IACA and llvm-mca across Haswell, Ivy Bridge, and Skylake (average error around 0.079–0.089 for Haswell/Ivy Bridge/Skylake respectively).
• The model shows stronger Spearman and Pearson correlations with ground truth than the baselines, improving rank utility for optimizers.
• Throughput estimation speed is comparable to LLVM-mca and IACA and far faster than empirical measurement, enabling drop-in replacement for throughput estimation in tooling.
• The approach is portable across microarchitectures: retraining on a new CPU requires no architecture-specific redesign, still outperforming hand-written models.
• Architectural variants (DAG-RNN, token-level RNN) indicate the hierarchical LSTM architecture provides the best trade-off between accuracy and modeling effort.