[Paper Review] Near-Data Processing for Differentiable Machine Learning Models

Hyeokjun Choe, Se-Il Lee+5 more|arXiv (Cornell University)|Oct 6, 2016

Advanced Data Storage Technologies29 references3 citations

TL;DR

This paper proposes ISP-ML, a full-fledged simulator for in-storage processing (ISP) on multi-channel SSDs, to evaluate near-data processing (NDP) for accelerating stochastic gradient descent (SGD) in differentiable machine learning models. It demonstrates that ISP-based SGD outperforms conventional in-host processing by reducing data movement and leveraging parallelism across NAND channels, with performance gains validated through simulation of synchronous, Downpour, and elastic averaging SGD variants on a scaled MNIST dataset.

ABSTRACT

Near-data processing (NDP) refers to augmenting memory or storage with processing power. Despite its potential for acceleration computing and reducing power requirements, only limited progress has been made in popularizing NDP for various reasons. Recently, two major changes have occurred that have ignited renewed interest and caused a resurgence of NDP. The first is the success of machine learning (ML), which often demands a great deal of computation for training, requiring frequent transfers of big data. The second is the popularity of NAND flash-based solid-state drives (SSDs) containing multicore processors that can accommodate extra computation for data processing. In this paper, we evaluate the potential of NDP for ML using a new SSD platform that allows us to simulate instorage processing (ISP) of ML workloads. Our platform (named ISP-ML) is a full-fledged simulator of a realistic multi-channel SSD that can execute various ML algorithms using data stored in the SSD. To conduct a thorough performance analysis and an in-depth comparison with alternative techniques, we focus on a specific algorithm: stochastic gradient descent (SGD), which is the de facto standard for training differentiable models such as logistic regression and neural networks. We implement and compare three SGD variants (synchronous, Downpour, and elastic averaging) using ISP-ML, exploiting the multiple NAND channels to parallelize SGD. In addition, we compare the performance of ISP and that of conventional in-host processing, revealing the advantages of ISP. Based on the advantages and limitations identified through our experiments, we further discuss directions for future research on ISP for accelerating ML.

Motivation & Objective

• To evaluate the potential of near-data processing (NDP) for accelerating machine learning workloads, particularly stochastic gradient descent (SGD), by leveraging processing capabilities within SSDs.
• To design and implement a realistic, full-fledged system-level simulator (ISP-ML) capable of emulating multi-channel SSDs with in-storage processing functionality.
• To compare the performance of in-storage processing (ISP) with conventional in-host processing (IHP) for training differentiable models using SGD variants.
• To identify performance bottlenecks and opportunities for future optimization in ISP-based machine learning workloads.
• To explore system-level cooperation between host and SSD for hybrid execution of high-level and low-level ML tasks.

Proposed method

• Developed ISP-ML, a system-level simulator of a realistic multi-channel NAND flash SSD that supports both storage operations and in-storage processing using a general-purpose ARM processor.
• Implemented three parallel SGD variants—synchronous, Downpour, and elastic averaging SGD—within the ISP-ML framework to exploit data-level parallelism across SSD channels.
• Used IO trace extraction from host execution to simulate I/O times in the baseline SSD, enabling fair relative performance comparison between ISP and IHP under consistent non-I/O conditions.
• Assumed negligible impact of storage type on non-I/O time, allowing focus on I/O and processing overhead differences between ISP and IHP.
• Simulated a 10× scaled version of the MNIST dataset to evaluate performance on larger workloads while maintaining realism in data access patterns.
• Explored architectural trade-offs by analyzing memory usage distribution across DRAM buffer, cache controller, and channel controllers in the ISP-ML framework.

Experimental results

Research questions

• RQ1 Can in-storage processing (ISP) on multi-channel SSDs effectively accelerate training of differentiable machine learning models, particularly those using stochastic gradient descent (SGD)?
• RQ2 How does the performance of ISP-based SGD compare to conventional in-host processing (IHP) in terms of execution time and data movement reduction?
• RQ3 What are the relative advantages and limitations of different parallel SGD variants (synchronous, Downpour, elastic averaging) when executed in an ISP environment?
• RQ4 How can system-level cooperation between host and SSD be designed to optimize performance in hybrid ML workloads?
• RQ5 What architectural optimizations—such as metadata pre-computation, data shuffling, or variable page sizes—can further improve ISP performance for ML workloads?

Key findings

• ISP-ML successfully simulates realistic multi-channel SSDs with in-storage processing capabilities, enabling detailed evaluation of NDP for machine learning workloads.
• The performance comparison shows that ISP-based SGD significantly reduces data movement overhead compared to in-host processing, leading to measurable execution time improvements.
• Among the three SGD variants, elastic averaging SGD (EASGD) demonstrated favorable convergence behavior and performance characteristics in the ISP environment due to its robustness to asynchronous updates.
• The simulation results indicate that the idle processing capacity of SSD controllers can be effectively utilized for ML computation, especially when data is partitioned across multiple NAND channels.
• The study identifies key challenges such as memory bottlenecks in channel controllers and data fragmentation when example sizes exceed NAND page sizes, suggesting the need for architectural optimizations.
• Future work shows promise in combining IHP and ISP, where the host handles high-level model logic and the SSD performs low-level data-dependent operations, enhancing overall system efficiency.