[Paper Review] Moving Processing to Data: On the Influence of Processing in Memory on Data Management

Tobias Vinçon, Andreas Koch+1 more|arXiv (Cornell University)|May 12, 2019

Parallel Computing and Optimization Techniques95 references4 citations

TL;DR

This paper investigates Processing-in-Memory (PIM) as a paradigm to reduce data movement by executing compute operations directly within or near memory storage, leveraging 3D-stacked DRAM and embedded processing units. It demonstrates that PIM can significantly improve performance, scalability, and energy efficiency—especially for data-intensive workloads like graph processing, neural networks, and analytical queries—by minimizing CPU and memory bandwidth bottlenecks.

ABSTRACT

Near-Data Processing refers to an architectural hardware and software paradigm, based on the co-location of storage and compute units. Ideally, it will allow to execute application-defined data- or compute-intensive operations in-situ, i.e. within (or close to) the physical data storage. Thus, Near-Data Processing seeks to minimize expensive data movement, improving performance, scalability, and resource-efficiency. Processing-in-Memory is a sub-class of Near-Data processing that targets data processing directly within memory (DRAM) chips. The effective use of Near-Data Processing mandates new architectures, algorithms, interfaces, and development toolchains.

Motivation & Objective

• Address the growing performance bottleneck caused by excessive data movement between storage, memory, and CPU in modern data-intensive workloads.
• Investigate how Processing-in-Memory (PIM) can mitigate the von Neumann bottleneck by co-locating computation with data in memory.
• Evaluate the potential of PIM to improve scalability and energy efficiency in database and analytics workloads with irregular access patterns and high data volumes.
• Examine the architectural and algorithmic changes required to support PIM in data management systems.
• Identify and analyze use cases where PIM offers significant performance and efficiency gains, such as graph processing, neural network training, and OLAP workloads.

Proposed method

• Propose a shift from the traditional data-to-code model to a code-to-data or in-situ processing model, where computation occurs within or near memory storage.
• Utilize 3D-stacked DRAM (HBM) and embedded processing units (e.g., FPGAs, CPUs) in storage devices to enable high-bandwidth, low-latency in-memory computation.
• Design specialized PIM hardware accelerators such as LiM (Logic-in-Memory) and TESSERACT for sparse matrix and graph processing, exploiting data locality and sparsity.
• Introduce the Data Rearrangement Engine (DRE) to dynamically restructure in-memory data layouts for improved cache efficiency and reduced memory bandwidth waste.
• Extend programming models like OpenCL to support heterogeneous PIM systems, enabling efficient task scheduling across CPU and PIM units.
• Simulate and evaluate PIM workloads using real-world workloads such as PageRank, generalized sparse matrix multiplication, and deep learning training to quantify performance and energy gains.

Experimental results

Research questions

• RQ1 To what extent can Processing-in-Memory reduce data movement and improve performance in data-intensive workloads?
• RQ2 How do PIM architectures handle irregular access patterns and low data locality common in analytical and machine learning workloads?
• RQ3 What are the key architectural and software interface challenges in integrating PIM into modern database and data management systems?
• RQ4 Can PIM-based accelerators achieve significant performance and energy efficiency improvements over traditional CPU-based execution for graph processing and neural network training?
• RQ5 How can programming models and abstractions be extended to effectively exploit PIM hardware while maintaining portability and maintainability?

Key findings

• PIM-based systems demonstrated more than two orders of magnitude improvement in performance and energy efficiency for sparse matrix-matrix multiplication compared to traditional multi-threaded CPU implementations.
• The Data Rearrangement Engine (DRE) reduced memory bandwidth waste by dynamically restructuring in-memory data layouts, improving cache efficiency in graph processing workloads like PageRank.
• TESSERACT, a PIM accelerator for graph processing, achieved high memory bandwidth utilization and reduced data movement by allowing user-specified access hints and efficient memory partitioning.
• Tetris, a scalable neural network accelerator using 3D-stacked memory, reduced contention on buses and improved data flow by offloading parts of the computation to DRAM, enhancing throughput and reducing energy consumption.
• The integration of PIM into data management systems requires new database operators, pipelining models, and atomicity semantics, which are still underdeveloped but show strong potential.
• Despite promising results, most existing PIM research remains workload-specific and lacks broad support for general-purpose database operations, indicating a need for standardized PIM primitives and interfaces in data management systems.