[Paper Review] Integrating NVIDIA Deep Learning Accelerator (NVDLA) with RISC-V SoC on FireSim

Farzad Farshchi, Qijing Huang+1 more|arXiv (Cornell University)|Mar 5, 2019

CCD and CMOS Imaging Sensors10 references4 citations

TL;DR

This paper presents a cloud-based, cycle-exact simulation of the NVDLA DNN accelerator integrated into a RISC-V SoC using FireSim on Amazon FPGAs, enabling cost-effective, high-fidelity performance analysis. The key contribution is demonstrating that NVDLA achieves 7.5 fps on YOLOv3 inference, with up to 1.56x speedup via last-level cache sharing and revealing significant memory interference (up to 2.5x slowdown) when CPU and NVDLA share memory subsystems.

ABSTRACT

NVDLA is an open-source deep neural network (DNN) accelerator which has received a lot of attention by the community since its introduction by Nvidia. It is a full-featured hardware IP and can serve as a good reference for conducting research and development of SoCs with integrated accelerators. However, an expensive FPGA board is required to do experiments with this IP in a real SoC. Moreover, since NVDLA is clocked at a lower frequency on an FPGA, it would be hard to do accurate performance analysis with such a setup. To overcome these limitations, we integrate NVDLA into a real RISC-V SoC on the Amazon cloud FPGA using FireSim, a cycle-exact FPGA-accelerated simulator. We then evaluate the performance of NVDLA by running YOLOv3 object-detection algorithm. Our results show that NVDLA can sustain 7.5 fps when running YOLOv3. We further analyze the performance by showing that sharing the last-level cache with NVDLA can result in up to 1.56x speedup. We then identify that sharing the memory system with the accelerator can result in unpredictable execution time for the real-time tasks running on this platform. We believe this is an important issue that must be addressed in order for on-chip DNN accelerators to be incorporated in real-time embedded systems.

Motivation & Objective

• To overcome the high cost and performance limitations of physical FPGA-based NVDLA integration for research.
• To enable accurate, cycle-exact performance evaluation of NVDLA in a real RISC-V SoC environment.
• To investigate the impact of shared memory and cache resources on NVDLA performance in multicore systems.
• To identify critical interference issues in shared memory systems that affect real-time DNN inference in embedded systems.
• To provide a publicly available, open-source simulation platform for DNN accelerator research on RISC-V.

Proposed method

• Integrates NVDLA into a FireSim-optimized RISC-V Rocket Chip SoC design using Verilog and FireSim's hardware-software co-simulation framework.
• Uses FireSim's FPGA-accelerated, cycle-accurate simulation on Amazon cloud FPGAs to model a realistic memory subsystem including L2 cache and DRAM.
• Decouples the target SoC from the host FPGA's DRAM controller and replaces it with a configurable memory model to enable precise performance measurement.
• Configures shared last-level cache (LLC) and varies cache parameters (size, block size) to evaluate their impact on NVDLA performance.
• Co-schedules YOLOv3 inference on NVDLA with synthetic memory-intensive workloads (BwWrite) on CPU cores to measure interference effects.
• Measures and normalizes NVDLA execution time under varying memory contention levels (L1, LLC, DRAM working set sizes) and core counts (1–4).

Experimental results

Research questions

• RQ1 How does the performance of NVDLA on YOLOv3 inference compare when running in a simulated, cycle-accurate environment versus real FPGA?
• RQ2 What is the performance impact of sharing the last-level cache between the CPU and NVDLA on DNN inference throughput?
• RQ3 How does memory contention between CPU workloads and NVDLA affect the predictability and latency of DNN inference in real-time systems?
• RQ4 What role does cache block size play in optimizing NVDLA performance through spatial locality exploitation?
• RQ5 To what extent does shared memory access between CPU and accelerator lead to unpredictable execution times, especially in safety-critical embedded applications?

Key findings

• NVDLA achieves 7.5 fps when running YOLOv3 object detection on the FireSim-simulated RISC-V SoC, demonstrating viable inference performance for embedded applications.
• Sharing the last-level cache with NVDLA results in up to 1.56x speedup, with the maximum benefit observed at 128-byte cache block sizes due to better spatial locality capture.
• The performance gain from LLC sharing is sensitive to block size, with 1.51x speedup at 128-byte blocks and 4096 KiB cache capacity.
• Memory interference from co-located CPU workloads causes up to 2.5x slowdown in NVDLA execution when the CPU’s working set fits into DRAM and four BwWrite tasks are active.
• Interference is minimal when CPU workloads fit within L1 cache, but increases significantly when they access shared LLC or DRAM, highlighting a critical bottleneck for real-time systems.
• The results indicate a strong need for QoS mechanisms—such as memory bandwidth partitioning or priority scheduling—to ensure predictable performance in real-time embedded systems using on-chip accelerators.