[Paper Review] Polly's Polyhedral Scheduling in the Presence of Reductions

Johannes Doerfert, Kevin Streit+2 more|arXiv (Cornell University)|May 28, 2015

Parallel Computing and Optimization Techniques32 references15 citations

TL;DR

This paper presents a reduction-aware polyhedral scheduling framework for the Polly optimizer in LLVM, enabling detection and optimization of associative and commutative reductions to exploit parallelism. By modeling reductions via privatization and relaxing memory dependences using associativity and commutativity, the approach achieves speedups up to 2.21× on the BiCG benchmark by effectively scheduling reductions in multidimensional loop nests.

ABSTRACT

The polyhedral model provides a powerful mathematical abstraction to enable effective optimization of loop nests with respect to a given optimization goal, e.g., exploiting parallelism. Unexploited reduction properties are a frequent reason for polyhedral optimizers to assume parallelism prohibiting dependences. To our knowledge, no polyhedral loop optimizer available in any production compiler provides support for reductions. In this paper, we show that leveraging the parallelism of reductions can lead to a significant performance increase. We give a precise, dependence based, definition of reductions and discuss ways to extend polyhedral optimization to exploit the associativity and commutativity of reduction computations. We have implemented a reduction-enabled scheduling approach in the Polly polyhedral optimizer and evaluate it on the standard Polybench 3.2 benchmark suite. We were able to detect and model all 52 arithmetic reductions and achieve speedups up to 2.21$ imes$ on a quad core machine by exploiting the multidimensional reduction in the BiCG benchmark.

Motivation & Objective

• To address the lack of reduction support in production polyhedral optimizers, which often block parallelism due to loop-carried dependences.
• To enable polyhedral optimizers to detect and exploit reductions by leveraging associativity and commutativity properties.
• To extend the polyhedral model with a sound dependence-based mechanism for modeling reductions and their parallelization.
• To integrate reduction-aware scheduling into the Polly optimizer without requiring preprocessing to SARE form.
• To evaluate the performance impact of reduction-aware optimization on real-world benchmarks.

Proposed method

• Proposes a dependence-based algorithm to detect reductions using memory and value-based dependence information.
• Introduces a model to relax memory dependences by exploiting the associativity and commutativity of reduction operations.
• Employs privatization as a fix-up technique to enable vectorization and parallelization of reductions, introducing temporary arrays per vector lane or thread.
• Designs a scheduling framework that preserves reduction integrity by ensuring no access to the reduction location occurs between the first and last instance of the reduction statement.
• Integrates the approach into the Polly polyhedral optimizer within the LLVM compiler infrastructure.
• Uses a three-phase approach: reduction detection, dependence relaxation, and transformation scheduling, all within the polyhedral model.

Experimental results

Research questions

• RQ1 How can reductions be reliably detected in the polyhedral model using existing dependence analysis?
• RQ2 What is a sound and practical way to relax memory dependences caused by reductions while preserving correctness?
• RQ3 How can privatization be integrated into polyhedral scheduling to enable vectorization and parallelization of reductions?
• RQ4 What performance gains can be achieved by exploiting reduction parallelism in real-world benchmarks?
• RQ5 How does reduction-aware scheduling compare to traditional polyhedral optimization in the absence of reduction support?

Key findings

• The proposed reduction detection algorithm successfully identifies all 52 arithmetic reductions in the PolyBench 3.2 benchmark suite.
• The integration of reduction-aware scheduling into Polly enables the detection and optimization of multidimensional reductions, such as in the BiCG benchmark.
• Speedups of up to 2.21× were achieved on a quad-core machine by exploiting parallelism in the BiCG benchmark through reduction-aware scheduling.
• The approach supports both vectorization and parallelization of reductions without requiring preprocessing to SARE form, enabling more flexible optimization.
• The overhead of privatization is manageable, and performance gains are significant when the execution environment matches the optimization strategy.
• The framework demonstrates that reduction-aware scheduling must be integrated into the core scheduling process, not treated as a post-processing step.