[Paper Review] Field-Programmable Gate Array Architecture for Deep Learning: Survey & Future Directions

Andrew Boutros, Aman Arora+1 more|arXiv (Cornell University)|Apr 15, 2024

Embedded Systems Design Techniques4 citations

TL;DR

This paper surveys FPGA architectures optimized for deep learning inference, proposing enhanced fabric blocks, tensor-specific accelerators, and hybrid chiplet integration to achieve high performance and energy efficiency. It demonstrates that FPGA-based accelerators can outperform GPUs in latency and energy efficiency while maintaining reconfigurability for evolving DL workloads.

ABSTRACT

Deep learning (DL) is becoming the cornerstone of numerous applications both in datacenters and at the edge. Specialized hardware is often necessary to meet the performance requirements of state-of-the-art DL models, but the rapid pace of change in DL models and the wide variety of systems integrating DL make it impossible to create custom computer chips for all but the largest markets. Field-programmable gate arrays (FPGAs) present a unique blend of reprogrammability and direct hardware execution that make them suitable for accelerating DL inference. They offer the ability to customize processing pipelines and memory hierarchies to achieve lower latency and higher energy efficiency compared to general-purpose CPUs and GPUs, at a fraction of the development time and cost of custom chips. Their diverse high-speed IOs also enable directly interfacing the FPGA to the network and/or a variety of external sensors, making them suitable for both datacenter and edge use cases. As DL has become an ever more important workload, FPGA architectures are evolving to enable higher DL performance. In this article, we survey both academic and industrial FPGA architecture enhancements for DL. First, we give a brief introduction on the basics of FPGA architecture and how its components lead to strengths and weaknesses for DL applications. Next, we discuss different styles of DL inference accelerators on FPGA, ranging from model-specific dataflow styles to software-programmable overlay styles. We survey DL-specific enhancements to traditional FPGA building blocks such as logic blocks, arithmetic circuitry, and on-chip memories, as well as new in-fabric DL-specialized blocks for accelerating tensor computations. Finally, we discuss hybrid devices that combine processors and coarse-grained accelerator blocks with FPGA-like interconnect and networks-on-chip, and highlight promising future research directions.

Motivation & Objective

• Address the growing computational demands of deep learning models in datacenters and edge devices.
• Overcome limitations of general-purpose CPUs and GPUs in energy efficiency and latency for DL inference.
• Enable customizable, reconfigurable hardware acceleration for diverse and evolving DL workloads using FPGAs.
• Improve developer productivity and performance through software-programmable overlays and hardware-software co-design.
• Explore future FPGA-based accelerator designs integrating ASIC chiplets and 3D stacking for enhanced performance and scalability.

Proposed method

• Survey academic and industrial FPGA architecture enhancements for DL, focusing on logic blocks, DSPs, block RAMs, and new tensor-specific hardware blocks.
• Analyze different DL inference accelerator styles: model-specific dataflow and software-programmable overlays for performance and productivity.
• Introduce RAD-Sim and RAD-Gen tools for automated evaluation of architecture exploration tradeoffs in area, performance, and power.
• Examine hybrid FPGA architectures integrating coarse-grained accelerator blocks and chiplets via interposers or 3D stacking.
• Propose 3D stacked RADs with FPGA fabric on top of ASIC base dies for enhanced memory bandwidth and application-specific acceleration.
• Evaluate system-level integration using networks-on-chip (NoCs) and programmable interconnects to enable scalable, high-bandwidth communication.

Experimental results

Research questions

• RQ1 How can FPGA architectures be enhanced to better support the computational and memory demands of modern deep learning models?
• RQ2 What are the most effective design styles for FPGA-based DL accelerators in terms of performance, energy efficiency, and developer productivity?
• RQ3 How do FPGA-specific architectural enhancements—such as custom logic blocks, DSPs, and on-chip memories—impact DL inference performance?
• RQ4 What role do hybrid architectures with integrated chiplets and 3D stacking play in advancing FPGA-based DL acceleration?
• RQ5 What are the key challenges and opportunities in designing reconfigurable accelerators for evolving deep learning workloads?

Key findings

• FPGA-based accelerators achieved 16× lower latency and 34× higher energy efficiency than the largest same-generation NVIDIA GPU for memory-bound DL models.
• Integration of a TensorRAM chiplet with a Stratix 10 FPGA reduced latency and improved energy efficiency by leveraging specialized on-chip memory and computation.
• Hybrid FPGA-chiplet architectures using interposers or 3D stacking enable scalable, high-bandwidth systems that combine FPGA flexibility with ASIC performance.
• RAD-Sim and RAD-Gen enable automated exploration of FPGA architecture tradeoffs, significantly reducing design time for custom accelerator variants.
• Future FPGA-based accelerators will likely combine fine-grained reconfigurability with coarse-grained accelerator blocks and advanced packaging for optimal performance.
• The design space of reconfigurable accelerators (RADs) is vast, involving fabric optimization, new accelerator blocks, and advanced interconnects like NoCs and 2D/3D integration.