[Paper Review] FOS: A Modular FPGA Operating System for Dynamic Workloads

Anuj Vaishnav, Khoa Dang Pham+2 more|arXiv (Cornell University)|Jan 26, 2020

Embedded Systems Design Techniques27 references4 citations

TL;DR

FOS is a modular FPGA operating system that enables dynamic, multi-tenant acceleration by decoupling hardware and software components through a modular compilation flow and resource-elastic runtime. It reduces compilation time by up to 2.34x and update latency by over 100x, while supporting heterogeneous accelerators (RTL, C, C++, OpenCL) and transparently scheduling resources in time and space for single- and multi-tenant workloads with minimal overhead.

ABSTRACT

With FPGAs now being deployed in the cloud and at the edge, there is a need for scalable design methods which can incorporate the heterogeneity present in the hardware and software components of FPGA systems. Moreover, these FPGA systems need to be maintainable and adaptable to changing workloads while improving accessibility for the application developers. However, current FPGA systems fail to achieve modularity and support for multi-tenancy due to dependencies between system components and lack of standardised abstraction layers. To solve this, we introduce a modular FPGA operating system -- FOS, which adopts a modular FPGA development flow to allow each system component to be changed and be agnostic to the heterogeneity of EDA tool versions, hardware and software layers. Further, to dynamically maximise the utilisation transparently from the users, FOS employs resource-elastic scheduling to arbitrate the FPGA resources in both time and spatial domain for any type of accelerators. Our evaluation on different FPGA boards shows that FOS can provide performance improvements in both single-tenant and multi-tenant environments while substantially reducing the development time and, at the same time, improving flexibility.

Motivation & Objective

• To address the lack of modularity, maintainability, and multi-tenancy in current FPGA systems due to tight coupling between hardware, EDA tools, and software components.
• To improve developer productivity and accessibility by abstracting FPGA complexity through high-level, language-agnostic APIs for C, C++, OpenCL, and RTL.
• To enable dynamic resource allocation across multiple accelerators and users via a runtime system that supports time- and space-domain multiplexing.
• To reduce compilation and update overheads in FPGA systems by decoupling shell and module compilation and enabling module relocation.
• To support heterogeneous FPGA environments across different boards and EDA tool versions without requiring full system recompilation.

Proposed method

• FOS employs a modular FPGA development flow that isolates system components (hardware and software) into independently compilable modules with well-defined interfaces.
• It introduces a decoupled compilation model where shell and accelerator modules are compiled separately, enabling reuse and relocation across different FPGA platforms.
• The runtime system implements resource-elastic scheduling, dynamically allocating FPGA resources in both time (time-multiplexing) and space (spatial partitioning) domains.
• A unified user interface abstracts accelerator access across multiple languages (C, C++, OpenCL, RTL), enabling application-centric programming.
• The system supports dynamic accelerator switching and replication at runtime, allowing adaptation to changing workloads.
• An open-source software stack with high-level APIs enables application developers to use FOS in static, single-user dynamic, or multi-tenant dynamic acceleration modes.

Experimental results

Research questions

• RQ1 How can FPGA systems achieve modularity and maintainability across heterogeneous EDA tools, hardware platforms, and software stacks?
• RQ2 Can a modular FPGA OS reduce compilation and update overheads in FPGA systems while supporting dynamic workloads?
• RQ3 How effectively can a resource-elastic runtime system schedule diverse accelerators (RTL, C, C++, OpenCL) in time and space domains for single- and multi-tenant workloads?
• RQ4 To what extent does FOS improve developer productivity and accessibility for non-FPGA experts in heterogeneous CPU-FPGA systems?
• RQ5 What performance and scalability benefits does FOS provide in multi-tenant environments with concurrent, heterogeneous workloads?

Key findings

• FOS reduces compilation time by up to 2.34x and update latency by over 100x compared to monolithic FPGA flows.
• The system achieves near-linear performance scaling with up to 3 parallel partial reconfigurable regions on the Ultra-96 platform, after which performance stagnates due to time-multiplexing overhead.
• Execution latencies for Mandelbrot, Black-Scholes, and Sobel applications improve almost linearly with increasing request counts until the number of physical regions is reached.
• In multi-tenant scenarios, FOS achieves a 46% performance improvement over a baseline 1-Mandel × 1-Sobel configuration by dynamically reallocating resources, even with conflicting memory access patterns.
• The system demonstrates that greedy resource requests from individual applications can still lead to near-optimal system performance due to intelligent runtime scheduling.
• Overhead from the FOS runtime and abstraction layers is minimal and outweighed by performance gains from dynamic resource utilization and reduced recompilation.