[Paper Review] An OpenCL(TM) Deep Learning Accelerator on Arria 10

Utku Aydonat, Shane O’Connell+3 more|arXiv (Cornell University)|Jan 13, 2017

Advanced Neural Network Applications13 references17 citations

TL;DR

This paper presents an OpenCL-based Deep Learning Accelerator (DLA) for Intel's Arria 10 FPGA that achieves 1020 images per second on AlexNet with 23 img/s/W efficiency by minimizing external memory bandwidth through on-chip stream-buffer caching and leveraging the Winograd transform to reduce FLOPs. The DLA delivers 10x higher throughput and 8.4x more GFLOPS than prior FPGA implementations, matching the energy efficiency of NVIDIA's TitanX GPU.

ABSTRACT

Convolutional neural nets (CNNs) have become a practical means to perform vision tasks, particularly in the area of image classification. FPGAs are well known to be able to perform convolutions efficiently, however, most recent efforts to run CNNs on FPGAs have shown limited advantages over other devices such as GPUs. Previous approaches on FPGAs have often been memory bound due to the limited external memory bandwidth on the FPGA device. We show a novel architecture written in OpenCL(TM), which we refer to as a Deep Learning Accelerator (DLA), that maximizes data reuse and minimizes external memory bandwidth. Furthermore, we show how we can use the Winograd transform to significantly boost the performance of the FPGA. As a result, when running our DLA on Intel's Arria 10 device we can achieve a performance of 1020 img/s, or 23 img/s/W when running the AlexNet CNN benchmark. This comes to 1382 GFLOPs and is 10x faster with 8.4x more GFLOPS and 5.8x better efficiency than the state-of-the-art on FPGAs. Additionally, 23 img/s/W is competitive against the best publicly known implementation of AlexNet on nVidia's TitanX GPU.

Motivation & Objective

• To overcome the memory bandwidth bottleneck in FPGA-based CNN inference by reducing external memory access.
• To improve energy efficiency and GFLOPS performance of FPGA-accelerated CNNs compared to existing solutions.
• To enable full-architecture deployment of AlexNet on FPGA by accelerating all layers, including fully-connected layers.
• To demonstrate competitive performance per watt against high-end GPUs like the TitanX.
• To provide a scalable, analytically optimized design methodology for FPGA-based DLA architectures.

Proposed method

• The DLA uses on-chip stream buffers to cache intermediate feature maps, reducing external memory bandwidth by an order of magnitude.
• It employs batching in fully-connected layers to further minimize off-chip memory access.
• The architecture leverages vectorized data processing with 60%+ DSP utilization to maximize hardware efficiency.
• The Winograd transform is applied to convolutions to reduce the number of multiply-accumulate operations.
• A design space exploration model is used to analytically optimize resource usage and throughput for a given FPGA and CNN.
• The entire system is implemented in OpenCL for portability and high-level programmability on Arria 10 FPGAs.

Experimental results

Research questions

• RQ1 Can on-chip caching of feature maps significantly reduce external memory bandwidth in FPGA-based CNN accelerators?
• RQ2 To what extent can the Winograd transform reduce computational complexity in FPGA-accelerated convolutions?
• RQ3 Can a fully pipelined, multi-layer CNN accelerator achieve performance and energy efficiency competitive with high-end GPUs?
• RQ4 How does the combination of memory optimization and algorithmic transformation improve GFLOPS and throughput on FPGAs?
• RQ5 What is the optimal configuration of resource utilization and vectorization for maximizing throughput on Arria 10 FPGAs?

Key findings

• The DLA achieves 1020 images per second on the AlexNet benchmark when running at 303 MHz on the Arria 10 1150 FPGA.
• The system delivers 23 img/s per watt, matching the energy efficiency of NVIDIA’s TitanX GPU.
• The DLA achieves 1382 GFLOPS, representing 10x higher throughput and 8.4x more GFLOPS than the prior state-of-the-art FPGA implementation.
• External memory bandwidth is reduced by an order of magnitude through on-chip stream-buffer caching of feature maps.
• The Winograd transform reduces the number of multiply-accumulate operations in convolutions, enabling higher computational efficiency.
• The design achieves over 60% DSP utilization, significantly improving hardware resource efficiency compared to prior FPGA CNN accelerators.