[Paper Review] Arrow: A RISC-V Vector Accelerator for Machine Learning Inference

Imad Al Assir, Mohamad El Iskandarani+2 more|arXiv (Cornell University)|Jul 15, 2021

Advanced Memory and Neural Computing15 references18 citations

TL;DR

Arrow is a configurable RISC-V vector accelerator targeting edge machine learning inference, implementing a subset of the RISC-V v0.9 vector ISA. It achieves 2–78× speedup and 20–99% energy reduction over scalar RISC-V processors on FPGA-based benchmarks, demonstrating strong performance and efficiency for data-parallel ML workloads.

ABSTRACT

In this paper we present Arrow, a configurable hardware accelerator architecture that implements a subset of the RISC-V v0.9 vector ISA extension aimed at edge machine learning inference. Our experimental results show that an Arrow co-processor can execute a suite of vector and matrix benchmarks fundamental to machine learning inference 2 - 78x faster than a scalar RISC processor while consuming 20% - 99% less energy when implemented in a Xilinx XC7A200T-1SBG484C FPGA.

Motivation & Objective

• Address the growing demand for efficient, domain-specific accelerators in edge machine learning inference.
• Leverage the RISC-V vector ISA (v0.9) to enable customizable, high-throughput processing for data-parallel workloads.
• Design a hardware accelerator that achieves high performance and low energy consumption on resource-constrained edge devices.
• Evaluate the accelerator's performance and energy efficiency on representative ML inference kernels using FPGA prototyping.
• Demonstrate the viability of RISC-V vector extensions for practical, efficient edge AI acceleration.

Proposed method

• Implement a configurable vector accelerator based on the RISC-V v0.9 vector ISA extension.
• Design a pipelined datapath with multiple vector lanes and a banked vector register file to support wide data-parallel operations.
• Integrate vector load/store units with strided memory access support to improve data locality and reduce pointer management overhead.
• Use a custom memory interface and on-chip block RAM for efficient data movement and low-latency access.
• Implement vector-length and configuration registers to enable runtime flexibility and dynamic vector size adaptation.
• Deploy the design on a Xilinx XC7A200T FPGA for performance and energy evaluation using real ML inference kernels.

Experimental results

Research questions

• RQ1 How does a RISC-V vector accelerator compare to a scalar RISC-V processor in terms of performance and energy efficiency for machine learning inference workloads?
• RQ2 What performance and energy gains can be achieved by exploiting data-level parallelism via vectorization in edge AI applications?
• RQ3 How do different data size profiles affect the speedup and energy efficiency of vectorized kernels on the Arrow accelerator?
• RQ4 What are the primary performance bottlenecks in vectorizing complex ML kernels like 2D convolution and matrix max pooling?
• RQ5 To what extent can vectorization reduce execution time and energy consumption in memory-bound and compute-bound ML kernels?

Key findings

• Arrow achieves 2–78× speedup over a scalar RISC-V processor on vector and matrix benchmarks, with the highest speedup observed on large data profiles.
• Matrix multiplication shows up to 78× speedup, while vector addition and multiplication achieve 25–78× speedup depending on data size.
• 2D convolution achieves only 1.4–1.9× speedup due to high overhead from scalar pointer arithmetic, despite using optimized vector reduction.
• Energy consumption is reduced by 96–99% for vector benchmarks, 80–99% for matrix benchmarks, and 20–43% for 2D convolution, primarily due to reduced execution time.
• Performance scales with vector size, as vector setup overhead (e.g., setting vector length) has less impact on longer vectors.
• The results indicate that further optimization of memory access patterns—particularly through strided vector operations—can significantly improve performance on irregular kernels like 2D convolution and matrix max pooling.