[Paper Review] Exploiting the DRAM Microarchitecture to Increase Memory-Level Parallelism

Yoongu Kim, Vivek Seshadri+3 more|arXiv (Cornell University)|May 4, 2018

Parallel Computing and Optimization Techniques46 references3 citations

TL;DR

This paper proposes Subarray-Level Parallelism (SALP) mechanisms—SALP-1, SALP-2, and MASA—to exploit independent subarrays within a DRAM bank and reduce serialization of memory requests. By overlapping activation, access, and precharge latencies across subarrays, the techniques improve memory-level parallelism with minimal area overhead (<0.15%), achieving up to 20% performance gains in multi-core systems and 17% in single-core systems across diverse workloads.

ABSTRACT

This paper summarizes the idea of Subarray-Level Parallelism (SALP) in DRAM, which was published in ISCA 2012, and examines the work's significance and future potential. Modern DRAMs have multiple banks to serve multiple memory requests in parallel. However, when two requests go to the same bank, they have to be served serially, exacerbating the high latency of on-chip memory. Adding more banks to the system to mitigate this problem incurs high system cost. Our goal in this work is to achieve the benefits of increasing the number of banks with a low-cost approach. To this end, we propose three new mechanisms, SALP-1, SALP-2, and MASA (Multitude of Activated Subarrays), to reduce the serialization of different requests that go to the same bank. The key observation exploited by our mechanisms is that a modern DRAM bank is implemented as a collection of subarrays that operate largely independently while sharing few global peripheral structures. Our three proposed mechanisms mitigate the negative impact of bank serialization by overlapping different components of the bank access latencies of multiple requests that go to different subarrays within the same bank. SALP-1 requires no changes to the existing DRAM structure, and needs to only reinterpret some of the existing DRAM timing parameters. SALP-2 and MASA require only modest changes (&lt; 0.15% area overhead) to the DRAM peripheral structures, which are much less design constrained than the DRAM core. Our evaluations show that SALP-1, SALP-2 and MASA significantly improve performance for both single-core systems (7%/13%/17%) and multi-core systems (15%/16%/20%), averaged across a wide range of workloads. We also demonstrate that our mechanisms can be combined with application-aware memory request scheduling in multicore systems to further improve performance and fairness.

Motivation & Objective

• Address the performance degradation caused by DRAM bank conflicts, where multiple requests to the same bank are serialized, increasing latency and reducing bandwidth.
• Overcome the limitations of simply increasing the number of banks, which incurs high system cost and diminishing returns due to physical and design constraints.
• Leverage the inherent subarray-level parallelism in modern DRAM banks—where each subarray has its own local row buffer—to enable concurrent access to different subarrays within the same bank.
• Improve memory-level parallelism and row buffer locality by allowing multiple subarrays to be activated and accessed simultaneously, reducing write recovery penalties and row buffer thrashing.
• Develop low-overhead, scalable mechanisms that can be integrated into existing DRAM architectures without major redesign, enabling practical deployment.

Proposed method

• Propose SALP-1, a mechanism requiring no hardware changes that reinterprets existing DRAM timing parameters to enable subarray-level parallelism.
• Introduce SALP-2, which makes minor modifications to DRAM peripheral structures (e.g., row decoder and precharge logic) to allow overlapping of subarray access latencies.
• Design MASA (Multitude of Activated Subarrays), a more advanced scheme that enables multiple subarrays within a single bank to be activated and accessed in parallel, significantly increasing row buffer utilization.
• Exploit the fact that activation and precharge operations are largely local to subarrays, allowing these operations to be overlapped across subarrays even within the same bank.
• Minimize area overhead by focusing on peripheral structures rather than the memory core, ensuring low cost (<0.15% area increase for SALP-2 and MASA).
• Combine SALP with application-aware memory scheduling in multi-core systems to further enhance performance and fairness.

Experimental results

Research questions

• RQ1 Can subarray-level parallelism be exploited to reduce serialization of memory requests within a single DRAM bank without increasing the number of banks?
• RQ2 What are the minimal architectural changes required to enable concurrent access to different subarrays within the same bank, while maintaining compatibility and low area cost?
• RQ3 To what extent can overlapping subarray access latencies improve memory-level parallelism and reduce write recovery penalties and row buffer thrashing?
• RQ4 How does subarray-level parallelism affect performance across diverse workloads in both single-core and multi-core systems?
• RQ5 Can SALP be effectively combined with software-level memory scheduling to further improve system performance and fairness in shared memory environments?

Key findings

• SALP-1, SALP-2, and MASA improve single-core system performance by 7%, 13%, and 17% on average across diverse workloads, respectively.
• In multi-core systems, the same mechanisms yield 15%, 16%, and 20% average performance improvements, respectively, demonstrating strong scalability.
• The mechanisms reduce the impact of bank conflicts by enabling concurrent access to different subarrays within the same bank, effectively increasing effective bandwidth and reducing latency.
• MASA enables multiple row buffers to remain active simultaneously across subarrays, significantly improving row buffer hit rates and reducing row buffer thrashing.
• The techniques improve memory energy efficiency by reducing the number of redundant row activations and precharges due to better locality and reduced serialization.
• SALP is compatible with future memory technologies that use hierarchical bank structures, and has been adopted and validated by industry (e.g., Samsung and Intel).