[Paper Review] Toward Transparent Heterogeneous Systems

Baptiste Delporte, Roberto Rigamonti+1 more|arXiv (Cornell University)|Nov 18, 2015

Parallel Computing and Optimization Techniques22 references3 citations

TL;DR

This paper presents a transparent, automated system for optimizing heterogeneous computing by dynamically identifying performance-critical code blocks via runtime profiling and JIT compilation, then transparently offloading them to specialized processors. The prototype achieves up to 32× speedup on embedded systems without developer intervention, demonstrating real-time adaptation to workload changes and hardware conditions.

ABSTRACT

Heterogeneous parallel systems are widely spread nowadays. Despite their availability, their usage and adoption are still limited, and even more rarely they are used to full power. Indeed, compelling new technologies are constantly developed and keep changing the technological landscape, but each of them targets a limited sub-set of supported devices, and nearly all of them require new programming paradigms and specific toolsets. Software, however, can hardly keep the pace with the growing number of computational capabilities, and developers are less and less motivated in learning skills that could quickly become obsolete. In this paper we present our effort in the direction of a transparent system optimization based on automatic code profiling and Just-In-Time compilation, that resulted in a fully-working embedded prototype capable of dynamically detect computing-intensive code blocks and automatically dispatch them to different computation units. Experimental results show that our system allows gains up to 32x in performance --- after an initial warm-up phase --- without requiring any human intervention.

Motivation & Objective

• To address the growing complexity and fragmentation of heterogeneous computing platforms, where developers face steep learning curves and toolchain incompatibilities.
• To overcome the mismatch between rapidly evolving hardware and slow-moving software development cycles.
• To enable automatic, transparent performance optimization without requiring changes to application code or developer expertise.
• To build a prototype system that can detect and offload compute-intensive functions to specialized processors at runtime.
• To demonstrate dynamic adaptation to workload patterns and hardware availability changes in real time.

Proposed method

• The system uses the LLVM JIT framework to execute and profile application code at runtime on a heterogeneous embedded platform.
• It leverages Linux's perf_event tool to identify computationally intensive functions through low-level performance monitoring.
• Based on profiling results, the system automatically dispatches hot code segments to the C64x+ DSP on the DM3730 chip, offloading them from the ARM Cortex-A8 CPU.
• The approach is fully transparent to the application, requiring no source code modifications or manual optimization.
• The system continuously monitors performance and can reverse offloading decisions if execution on the target unit becomes suboptimal.
• A prototype was implemented on the REPTAR platform, using OpenCV for video processing and integrating VPE for dynamic computation offloading.

Experimental results

Research questions

• RQ1 Can a system automatically detect compute-intensive code segments in a running application without developer input?
• RQ2 Can such a system dynamically offload these segments to specialized processors to improve performance?
• RQ3 To what extent can automatic offloading achieve performance gains comparable to hand-optimized code?
• RQ4 How does the system handle dynamic changes in workload patterns or hardware conditions?
• RQ5 Can the system maintain transparency while achieving significant performance improvements across diverse computational workloads?

Key findings

• The system achieved a maximum speedup of 31.9× on the MatrixMult benchmark, with a 32× gain observed in the warm-up phase.
• For the image processing prototype, frame rate increased from 1.5 fps to 6 fps (a 4× improvement) after offloading 2D convolution to the DSP.
• The system detected and offloaded the most expensive function (convolution) automatically, reducing ARM CPU load by approximately 50%.
• Performance gains were consistent across multiple benchmarks, including 7.4× for Complement, 6.3× for DotProduct, and 22.7× for PatternMatch.
• The system demonstrated dynamic adaptability by reversing offloading decisions when performance on the DSP was suboptimal, as seen in the FFT case with a 0.7× speedup.
• The prototype successfully handled real-time video processing with continuous performance monitoring and adaptive resource allocation.