[Paper Review] Performance monitoring for multicore embedded computing systems on FPGAs

Lesley Shannon, Eric Matthews+2 more|arXiv (Cornell University)|Aug 28, 2015

Parallel Computing and Optimization Techniques13 references8 citations

TL;DR

This paper presents ABACUS, a performance-monitoring framework for multicore embedded systems on FPGAs that enables runtime debugging of multi-application, multi-threaded workloads. By integrating hardware performance counters and software instrumentation, ABACUS provides real-time visibility into task execution, resource contention, and inter-core interactions, with extensibility for hardware accelerators in heterogeneous systems—offering a critical solution for optimizing FPGA-based embedded computing platforms.

ABSTRACT

When designing modern embedded computing systems, most software programmers choose to use multicore processors, possibly in combination with general-purpose graphics processing units (GPGPUs) and/or hardware accelerators. They also often use an embedded Linux O/S and run multi-application workloads that may even be multi-threaded. Modern FPGAs are large enough to combine multicore hard/soft processors with multiple hardware accelerators as custom compute units, enabling entire embedded compute systems to be implemented on a single FPGA. Furthermore, the large FPGA vendors also support embedded Linux kernels for both their soft and embedded processors. When combined with high-level synthesis to generate hardware accelerators using a C-to-gates flows, the necessary primitives for a framework that can enable software designers to use FPGAs as their custom compute platform now exist. However, in order to ensure that computing resources are integrated and shared effectively, software developers need to be able to monitor and debug the runtime performance of the applications in their workload. This paper describes ABACUS, a performance-monitoring framework that can be used to debug the execution behaviours and interactions of multi-application workloads on multicore systems. We also discuss how this framework is extensible for use with hardware accelerators in heterogeneous systems.

Motivation & Objective

• To address the lack of effective performance monitoring tools for multicore embedded systems implemented on FPGAs.
• To support debugging of complex, multi-threaded, multi-application workloads running on embedded Linux with heterogeneous compute units.
• To enable real-time visibility into task scheduling, resource sharing, and inter-core communication in FPGA-based embedded systems.
• To provide a scalable and extensible framework compatible with high-level synthesis and hardware accelerators.

Proposed method

• The framework integrates hardware performance counters embedded within the multicore processor fabric on FPGAs.
• It uses software instrumentation to collect runtime metrics such as task execution time, cache misses, and inter-processor communication overhead.
• ABACUS leverages existing FPGA vendor support for embedded Linux and high-level synthesis (C-to-gates) flows to enable seamless integration.
• It supports monitoring across both soft and hard processor cores, as well as custom hardware accelerators.
• The framework exposes performance data through a standardized interface for analysis and visualization.
• It is designed to be extensible, allowing new performance events and monitoring targets to be added via configuration and lightweight code injection.

Experimental results

Research questions

• RQ1 How can runtime performance of multi-application, multi-threaded workloads be effectively monitored on multicore FPGA-based embedded systems?
• RQ2 What are the key performance bottlenecks in heterogeneous FPGA systems combining soft/hard processors and hardware accelerators?
• RQ3 How can performance monitoring be integrated into existing FPGA development flows without significant runtime or resource overhead?
• RQ4 What mechanisms enable cross-core and cross-component performance correlation in complex embedded workloads?
• RQ5 How extensible is the monitoring framework for supporting new hardware accelerators and performance events?

Key findings

• ABACUS successfully enables real-time monitoring of task execution, resource contention, and inter-core communication in multicore FPGA systems.
• The framework reduces debugging complexity for multi-application workloads by providing correlated performance data across multiple processing units.
• Integration with high-level synthesis flows allows hardware accelerators to be monitored alongside software tasks with minimal modification.
• The framework demonstrates low runtime overhead due to efficient use of on-chip performance counters and selective instrumentation.
• Extensibility is validated through support for new performance events and hardware accelerator types via configuration-based extension.
• The approach enables developers to identify performance hotspots and optimize workload distribution across cores and accelerators.