[Paper Review] SQUASH: Simple QoS-Aware High-Performance Memory Scheduler for Heterogeneous Systems with Hardware Accelerators

Hiroyuki Usui, Lavanya Subramanian+2 more|arXiv (Cornell University)|May 27, 2015

Parallel Computing and Optimization Techniques37 references8 citations

TL;DR

SQUASH is a QoS-aware memory scheduler for heterogeneous systems with CPUs and hardware accelerators (HWAs), using distributed priority, application-aware prioritization, and deadline-based scheduling to ensure all HWAs meet their deadlines while improving CPU performance by up to 24.0% over prior work. It dynamically adjusts scheduling based on memory access patterns and worst-case latency estimates for short-deadline HWAs.

ABSTRACT

Modern SoCs integrate multiple CPU cores and Hardware Accelerators (HWAs) that share the same main memory system, causing interference among memory requests from different agents. The result of this interference, if not controlled well, is missed deadlines for HWAs and low CPU performance. State-of-the-art mechanisms designed for CPU-GPU systems strive to meet a target frame rate for GPUs by prioritizing the GPU close to the time when it has to complete a frame. We observe two major problems when such an approach is adapted to a heterogeneous CPU-HWA system. First, HWAs miss deadlines because they are prioritized only close to their deadlines. Second, such an approach does not consider the diverse memory access characteristics of different applications running on CPUs and HWAs, leading to low performance for latency-sensitive CPU applications and deadline misses for some HWAs, including GPUs. In this paper, we propose a Simple Quality of service Aware memory Scheduler for Heterogeneous systems (SQUASH), that overcomes these problems using three key ideas, with the goal of meeting deadlines of HWAs while providing high CPU performance. First, SQUASH prioritizes a HWA when it is not on track to meet its deadline any time during a deadline period. Second, SQUASH prioritizes HWAs over memory-intensive CPU applications based on the observation that the performance of memory-intensive applications is not sensitive to memory latency. Third, SQUASH treats short-deadline HWAs differently as they are more likely to miss their deadlines and schedules their requests based on worst-case memory access time estimates. Extensive evaluations across a wide variety of different workloads and systems show that SQUASH achieves significantly better CPU performance than the best previous scheduler while always meeting the deadlines for all HWAs, including GPUs, thereby largely improving frame rates.

Motivation & Objective

• To address the limitations of existing memory schedulers that fail to balance HWA deadline compliance with high CPU performance in heterogeneous systems.
• To identify that prioritizing HWAs only near deadlines leads to deadline misses and suboptimal performance.
• To design a scheduler that considers diverse memory access characteristics of CPU applications and HWAs to improve overall system efficiency.
• To ensure all HWAs, including GPUs, consistently meet their deadlines while minimizing interference with latency-sensitive CPU workloads.
• To achieve high system performance without sacrificing real-time guarantees for hardware accelerators in shared memory SoCs.

Proposed method

• SQUASH uses distributed priority by continuously monitoring HWA progress and elevating priority whenever a HWA falls behind its deadline trajectory, rather than reserving priority for the final phase.
• It prioritizes HWAs over memory-intensive CPU applications, as these are less sensitive to memory latency, thereby accelerating HWA progress without significantly impacting CPU performance.
• For short-deadline HWAs, SQUASH schedules a short burst of high priority based on worst-case memory access time estimates to ensure timely completion.
• The scheduler employs a SchedulingUnit of 1000 cycles and a SwitchingUnit of 500 cycles, selected empirically to balance performance and fairness.
• It classifies HWA urgency dynamically based on real-time progress tracking and adjusts priority probabilities accordingly.
• SQUASH integrates with existing memory management techniques such as source throttling and bank partitioning, serving as a complementary scheduling substrate.

Experimental results

Research questions

• RQ1 Can a memory scheduler effectively meet HWA deadlines while maintaining high CPU performance in heterogeneous systems?
• RQ2 Does prioritizing HWAs only near their deadlines lead to deadline misses and suboptimal performance?
• RQ3 Can application-aware prioritization—based on memory access characteristics—improve both HWA deadline compliance and CPU performance?
• RQ4 How does scheduling based on worst-case memory access time estimates affect the performance of short-deadline HWAs?
• RQ5 Can a simple, non-predictive scheduling mechanism outperform complex, deadline-aware schedulers in real-world workloads?

Key findings

• SQUASH achieves a maximum 24.0% improvement in CPU performance over the best prior scheduler while always meeting all HWA deadlines.
• The deadline-met ratio for all HWAs, including GPUs, is 100%, demonstrating consistent and reliable QoS compliance.
• SQUASH reduces the number of times HWAs are prioritized over latency-sensitive CPU applications by intelligently scheduling only when necessary.
• The use of distributed priority prevents last-minute deadline misses by proactively addressing performance deviations early in the deadline period.
• SQUASH outperforms the state-of-the-art CPU-GPU scheduler [17] in both system performance and deadline-met ratio across diverse workloads.
• The optimal configuration uses a SchedulingUnit of 1000 cycles and a SwitchingUnit of 500 cycles, balancing performance and fairness.