[Paper Review] MPU: Towards Bandwidth-abundant SIMT Processor via Near-bank Computing

Xinfeng Xie, Peng Gu+4 more|arXiv (Cornell University)|Mar 11, 2021

Parallel Computing and Optimization Techniques64 references4 citations

TL;DR

MPU proposes the first SIMT processor based on 3D-stacking near-bank computing, using a hybrid pipeline to offload instructions to near-bank units while minimizing TSV traffic. It achieves 3.46× speedup and 2.57× energy reduction over NVIDIA Tesla V100 on data-intensive workloads via architectural optimizations and an end-to-end CUDA-compatible compilation flow.

ABSTRACT

With the growing number of data-intensive workloads, GPU, which is the state-of-the-art single-instruction-multiple-thread (SIMT) processor, is hindered by the memory bandwidth wall. To alleviate this bottleneck, previously proposed 3D-stacking near-bank computing accelerators benefit from abundant bank-internal bandwidth by bringing computations closer to the DRAM banks. However, these accelerators are specialized for certain application domains with simple architecture data paths and customized software mapping schemes. For general purpose scenarios, lightweight hardware designs for diverse data paths, architectural supports for the SIMT programming model, and end-to-end software optimizations remain challenging. To address these issues, we propose MPU (Memory-centric Processing Unit), the first SIMT processor based on 3D-stacking near-bank computing architecture. First, to realize diverse data paths with small overheads while leveraging bank-level bandwidth, MPU adopts a hybrid pipeline with the capability of offloading instructions to near-bank compute-logic. Second, we explore two architectural supports for the SIMT programming model, including a near-bank shared memory design and a multiple activated row-buffers enhancement. Third, we present an end-to-end compilation flow for MPU to support CUDA programs. To fully utilize MPU's hybrid pipeline, we develop a backend optimization for the instruction offloading decision. The evaluation results of MPU demonstrate 3.46x speedup and 2.57x energy reduction compared with an NVIDIA Tesla V100 GPU on a set of representative data-intensive workloads.

Motivation & Objective

• Address the memory bandwidth wall in GPU workloads by leveraging near-bank computing in 3D-stacked DRAM.
• Enable general-purpose, programmable execution of data-intensive workloads on near-bank accelerators, overcoming limitations of prior domain-specific designs.
• Support the SIMT programming model with architectural features like near-bank shared memory and multiple activated row-buffers.
• Develop an end-to-end compilation flow for CUDA programs to enable seamless porting and optimization on the MPU architecture.
• Minimize TSV traffic and area overhead by offloading only lightweight instructions to DRAM dies while retaining complex control logic on the base die.

Proposed method

• Design a hybrid SIMT pipeline where instruction fetch, decode, and issue occur on the base logic die, while selected instructions are offloaded to near-bank units (NBUs) based on compiler hints or runtime policies.
• Implement an instruction offload engine to make dynamic decisions on which instructions to execute near the bank, reducing register movement across TSVs.
• Introduce a near-bank shared memory to reduce off-chip data movement and improve inter-thread communication efficiency.
• Enhance bandwidth utilization via multiple activated row-buffers, maintaining high DRAM bank-internal bandwidth even under dynamic warp scheduling.
• Develop a backend compiler optimization that statically analyzes CUDA kernels to annotate instruction locations (near-bank vs. base die) for optimal offloading.
• Use a separate memory space for MPU, enabling end-to-end kernel execution without shared memory coherence or host access contention issues.

Experimental results

Research questions

• RQ1 Can a general-purpose SIMT processor be efficiently implemented using 3D-stacked near-bank computing while minimizing area and TSV overhead?
• RQ2 How can the SIMT programming model be efficiently supported in a near-bank architecture, particularly for shared memory and warp scheduling?
• RQ3 What is the performance and energy impact of offloading specific instruction types to near-bank units while preserving program correctness?
• RQ4 To what extent can end-to-end software support, including a CUDA-compatible compilation flow, unlock the potential of near-bank computing for data-intensive workloads?
• RQ5 How does MPU’s hybrid pipeline and architectural optimizations compare to existing 3D-stacking GPU and near-data platforms in terms of bandwidth utilization and efficiency?

Key findings

• MPU achieves a 3.46× speedup over NVIDIA Tesla V100 GPU on a set of representative data-intensive workloads, demonstrating significant performance gains.
• The energy efficiency of MPU improves by 2.57× compared to the V100, primarily due to reduced data movement and efficient use of bank-internal bandwidth.
• The hybrid pipeline design reduces TSV traffic by offloading only lightweight instructions to DRAM dies, minimizing area and energy overhead.
• The near-bank shared memory reduces off-chip data movement, improving inter-thread communication efficiency and overall bandwidth utilization.
• The multiple activated row-buffers design maintains high DRAM bandwidth even under dynamic warp scheduling, preventing row-buffer thrashing.
• The end-to-end CUDA compilation flow enables transparent porting of existing kernels with backend optimizations that improve instruction offloading decisions, enhancing performance.