[Paper Review] Compiling Neural Networks for a Computational Memory Accelerator

Kornilios Kourtis, Martino Dazzi+4 more|arXiv (Cornell University)|Mar 5, 2020

Adversarial Robustness in Machine Learning45 references4 citations

TL;DR

This paper presents a compiler stack for a computational memory (CM) accelerator that enables efficient deep learning inference by compiling neural networks into pipelined dataflow execution across multi-core CM hardware. Using polyhedral compilation techniques, the work generates control logic to enforce data dependencies between layers, ensuring correct pipelined execution while minimizing reconfiguration overhead on analog crossbar arrays.

ABSTRACT

Computational memory (CM) is a promising approach for accelerating inference on neural networks (NN) by using enhanced memories that, in addition to storing data, allow computations on them. One of the main challenges of this approach is defining a hardware/software interface that allows a compiler to map NN models for efficient execution on the underlying CM accelerator. This is a non-trivial task because efficiency dictates that the CM accelerator is explicitly programmed as a dataflow engine where the execution of the different NN layers form a pipeline. In this paper, we present our work towards a software stack for executing ML models on such a multi-core CM accelerator. We describe an architecture for the hardware and software, and focus on the problem of implementing the appropriate control logic so that data dependencies are respected. We propose a solution to the latter that is based on polyhedral compilation.

Motivation & Objective

• To design a co-designed software stack and hardware architecture for computational memory (CM) accelerators targeting efficient deep learning inference at the edge.
• To address the challenge of generating correct control logic that respects data dependencies when executing neural network layers in a pipelined fashion across multiple CM cores.
• To enable transparent, high-performance execution of neural networks on CM accelerators by co-designing the compiler and hardware interface.
• To overcome the limitations of traditional compilers, which lack support for managing inter-core dataflow dependencies in dataflow-based accelerators.

Proposed method

• The authors propose a multi-core CM accelerator where each core contains a crossbar array for analog matrix-vector multiplication and a lightweight digital processing unit (DPU) for non-parallelizable operations.
• They model the NN execution as a dataflow pipeline, with each layer mapped to a separate CM core, and use polyhedral compilation to represent and analyze data dependencies between iterations.
• The control logic is generated via a sequence of ISL-based operations: computing write-to-read dependencies (K), determining the last valid write for each read (L), and deriving the maximum safe execution iteration for each output location (S).
• The key relation S is computed using lexmax over composed relations (K ∘ D′), ensuring that only the latest write iteration is retained for each read iteration, thus enabling safe pipelining.
• The compiler stack includes a prototype compiler (cmnnc) and a simulator to model the CM hardware, enabling end-to-end compilation and verification.
• The approach uses ISL (Integer Set Library) to express and manipulate iteration spaces, relations, and dependencies, enabling precise static analysis of dataflow constraints.

Experimental results

Research questions

• RQ1 How can a compiler generate correct and efficient control logic to enforce data dependencies between neural network layers executed on a multi-core computational memory accelerator?
• RQ2 What is the most effective way to represent and analyze dataflow dependencies in a pipelined, dataflow-based accelerator architecture?
• RQ3 Can polyhedral compilation techniques be adapted to generate control state machines for dataflow accelerators that are not natively supported by existing ML compilers?
• RQ4 How can the hardware and software stack be co-designed to minimize reconfiguration overhead and maximize inference throughput on analog crossbar-based CM accelerators?

Key findings

• The proposed polyhedral-based control generation method successfully computes the safe execution order for pipelined neural network layers, ensuring that all data dependencies are respected across CM cores.
• The method enables efficient, single-configuration inference on PCM-based CM accelerators, avoiding the high reconfiguration cost of reprogramming crossbars after each layer.
• The use of ISL and lexmax operations allows precise static analysis of write-read dependencies, resulting in minimal and correct control logic for state machines.
• The approach is generalizable to other dataflow architectures, not limited to CM, and provides a foundation for co-designing future accelerators with their software stacks.
• The prototype compiler (cmnnc) and simulator demonstrate end-to-end compilation and execution of neural networks on the CM accelerator, validating the feasibility of the approach.