[Paper Review] Benchmarking a New Paradigm: An Experimental Analysis of a Real Processing-in-Memory Architecture

Juan Gómez-Luna, Izzat El Hajj+4 more|arXiv (Cornell University)|May 9, 2021

Parallel Computing and Optimization Techniques234 references50 citations

TL;DR

This paper provides the first comprehensive experimental characterization of the real-world UPMEM PIM architecture and introduces the PrIM benchmark suite, comparing PIM performance and energy to CPU and GPU baselines.

ABSTRACT

Many modern workloads, such as neural networks, databases, and graph processing, are fundamentally memory-bound. For such workloads, the data movement between main memory and CPU cores imposes a significant overhead in terms of both latency and energy. A major reason is that this communication happens through a narrow bus with high latency and limited bandwidth, and the low data reuse in memory-bound workloads is insufficient to amortize the cost of main memory access. Fundamentally addressing this data movement bottleneck requires a paradigm where the memory system assumes an active role in computing by integrating processing capabilities. This paradigm is known as processing-in-memory (PIM). Recent research explores different forms of PIM architectures, motivated by the emergence of new 3D-stacked memory technologies that integrate memory with a logic layer where processing elements can be easily placed. Past works evaluate these architectures in simulation or, at best, with simplified hardware prototypes. In contrast, the UPMEM company has designed and manufactured the first publicly-available real-world PIM architecture. This paper provides the first comprehensive analysis of the first publicly-available real-world PIM architecture. We make two key contributions. First, we conduct an experimental characterization of the UPMEM-based PIM system using microbenchmarks to assess various architecture limits such as compute throughput and memory bandwidth, yielding new insights. Second, we present PrIM, a benchmark suite of 16 workloads from different application domains (e.g., linear algebra, databases, graph processing, neural networks, bioinformatics).

Motivation & Objective

• Motivate the PIM paradigm to address data movement bottlenecks in memory-bound workloads such as neural networks, databases, and graph processing.
• Characterize the first publicly available real-world PIM system (UPMEM) through microbenchmarks and a new benchmark suite.
• Provide actionable programming and hardware design insights to improve future PIM architectures and software stacks.

Proposed method

• Develop microbenchmarks to measure DPU compute throughput across arithmetic operations and data types.
• Measure sustained bandwidth for two memory spaces (MRAM and WRAM) under various access patterns.
• Assess sustained host-to-MRAM bandwidth for host-CPU to DPU interactions and inter-DPU data transfers via the host.
• Create PrIM, a benchmark suite of 16 memory-bound workloads spanning multiple domains (linear algebra, databases, graph processing, neural nets, bioinformatics, image processing).
• Evaluate performance and energy of PrIM on two real UPMEM systems (640 and 2,556 DPUs) and compare against CPU and GPU baselines.
• Make microbenchmarks and PrIM openly available for community use.

Experimental results

Research questions

• RQ1 Which workloads and data patterns are most suitable for the UPMEM PIM architecture?
• RQ2 What are the primary bottlenecks and performance limits of real-world PIM hardware (compute vs. memory-bound constraints)?
• RQ3 How does inter-DPU communication (via the host CPU) impact scaling and energy efficiency?
• RQ4 How do PrIM workloads perform in terms of speedup and energy when compared to CPU and GPU counterparts?
• RQ5 What programming and architectural recommendations emerge for future PIM systems based on the characterization results?

Key findings

• The UPMEM PIM architecture is fundamentally compute-bound; complex arithmetic can saturate the pipeline before memory bandwidth becomes the limiter.
• Inter-DPU communication scales poorly because there is no direct DPU-to-DPU channel; all synchronization goes through the host via the memory bus.
• PrIM workloads outperform modern CPUs on average for 13 of 16 benchmarks without heavy inter-DPU synchronization or floating-point work.
• The 2,556-DPU PIM system outperforms the GPU on average for 10 benchmarks under streaming memory accesses and light inter-DPU synchronization, with a 2.54x speedup; the 640-DPU system shows a smaller gap but still closes for some workloads.
• Energy consumption trends align with performance, with PIM delivering large energy savings over CPU and GPU for many workloads.
• Compared to CPU/GPU, PIM demonstrates significant advantages for memory-bound workloads, and authors provide recommendations to guide software and hardware design for future PIM systems.