[Paper Review] SCALE-Sim: Systolic CNN Accelerator Simulator

Ananda Samajdar, Yuhao Zhu+3 more|arXiv (Cornell University)|Oct 16, 2018

Energy Harvesting in Wireless NetworksEngineering33 references194 citations

TL;DR

SCALE-Sim is an open-source, cycle-accurate simulator for systolic-array CNN accelerators, enabling exploration of dataflow, array shape, memory sizing, and system integration impacts on performance and energy.

ABSTRACT

Systolic Arrays are one of the most popular compute substrates within Deep Learning accelerators today, as they provide extremely high efficiency for running dense matrix multiplications. However, the research community lacks tools to insights on both the design trade-offs and efficient mapping strategies for systolic-array based accelerators. We introduce Systolic CNN Accelerator Simulator (SCALE-Sim), which is a configurable systolic array based cycle accurate DNN accelerator simulator. SCALE-Sim exposes various micro-architectural features as well as system integration parameters to the designer to enable comprehensive design space exploration. This is the first systolic-array simulator tuned for running DNNs to the best of our knowledge. Using SCALE-Sim, we conduct a suite of case studies and demonstrate the effect of bandwidth, data flow and aspect ratio on the overall runtime and energy of Deep Learning kernels across vision, speech, text, and games. We believe that these insights will be highly beneficial to architects and ML practitioners.

Motivation & Objective

• Identify key design parameters of systolic-array CNN accelerators and their interactions.
• Provide a cycle-accurate, open-source tool for rapid design space exploration.
• Demonstrate how dataflow, memory sizing, array shape, and system integration affect performance and energy across CNN workloads.

Proposed method

• Model the compute as a systolic 2D array of MAC units capable of matrix-matrix, matrix-vector, and vector-vector operations.
• Support three dataflows (Output Stationary, Weight Stationary, Input Stationary) and capture their impact on reuse and bandwidth.
• Implement a parameterizable on-chip memory hierarchy with three logical partitions (IFMAP, filter, OFMAP) and double-buffered memories to hide latency.
• Model system integration via a slave interface to a host processor, generating SRAM/DRAM traffic and enabling DRAM bandwidth estimation.
• Generate cycle-accurate traffic traces and summary metrics from layer-wise topology CSVs and architecture configs; validate against RTL for OS dataflow.

Experimental results

Research questions

• RQ1 How do dataflow choices (OS/WS/IS) interact with array size and workload hyper-parameters to affect performance and energy in systolic CNN accelerators?
• RQ2 What are the memory sizing requirements (scratchpad limits) to achieve stall-free operation and favorable energy characteristics across CNN workloads?
• RQ3 How does the shape of the array (aspect ratio) influence performance for common DNN workloads under different dataflows?
• RQ4 What are the trade-offs between scaling up (bigger arrays) vs scaling out (more arrays) under fixed compute budgets?
• RQ5 Can a single dataflow be effectively used across diverse network topologies, or is dataflow customization essential for efficiency?

Key findings

• OS dataflow frequently yields best performance across examined workloads, but stall-free implementation and hardware cost must be considered.
• IS and WS can require fewer SRAM banks for square arrays and show varying performance depending on workload and array size; smaller arrays may favor IS.
• Larger on-chip scratchpad memories reduce off-chip bandwidth needs and energy, with diminishing returns beyond a workload-dependent knee point.
• Array shape and dataflow interact in complex ways; for some networks, tall arrays perform poorly unless dataflow is adjusted, while square arrays often perform well overall.
• Scaling up vs scaling out yields different DRAM bandwidth and performance implications that depend on workload and dataflow, highlighting non-trivial trade-offs in accelerator scaling.