[Paper Review] Applying Type Oriented Programming to the PGAS Memory Model

Nicholas Edward Brown|arXiv (Cornell University)|Sep 26, 2020

Parallel Computing and Optimization Techniques3 citations

TL;DR

This paper proposes type-oriented programming to enhance the PGAS memory model by encoding parallelism semantics—such as data distribution, communication patterns, and memory layout—directly in types, enabling high-level, composable, and tunable parallel code. Using the Mesham language, the authors demonstrate that type-driven compilation achieves competitive FFT performance versus FFTW3 while allowing non-expert programmers to easily tune code for scalability and performance through type annotations alone.

ABSTRACT

The Partitioned Global Address Space memory model has been popularised by a number of languages and applications. However this abstraction can often result in the programmer having to rely on some in built choices and with this implicit parallelism, with little assistance by the programmer, the scalability and performance of the code heavily depends on the compiler and choice of application. We propose an approach, type oriented programming, where all aspects of parallelism are encoded via types and the type system. The type information associated by the programmer will determine, for instance, how an array is allocated, partitioned and distributed. With this rich, high level of information the compiler can generate an efficient target executable. If the programmer wishes to omit detailed type information then the compiler will rely on well documented and safe default behaviour which can be tuned at a later date with the addition of types. The type oriented parallel programming language Mesham, which follows the PGAS memory model, is presented. We illustrate how, if so wished, with the use of types one can tune all parameters and options associated with this PGAS model in a clean and consistent manner without rewriting large portions of code. An FFT case study is presented and considered both in terms of programmability and performance - the latter we demonstrate by a comparison with an existing FFT solver.

Motivation & Objective

• To address the tension between programmer simplicity and performance control in PGAS-based parallel programming.
• To enable non-expert HPC users to express and tune low-level parallelism decisions without rewriting code.
• To reduce the complexity of porting and optimizing parallel codes across architectures by encoding parallelism semantics in types.
• To demonstrate that rich type information can guide the compiler to generate efficient, scalable code with minimal programmer effort.
• To evaluate the approach via a 2D FFT case study, comparing programmability and performance against established frameworks like FFTW3.

Proposed method

• Introduce a new type system where data types encode parallelism semantics such as data distribution, memory layout, and communication patterns.
• Design the Mesham language to embed PGAS semantics through user-defined types, with default behaviors for backward compatibility.
• Use type annotations at variable declarations to guide the compiler in generating optimized, architecture-aware parallel code.
• Implement a runtime abstraction layer that maps high-level types to low-level communication (e.g., MPI) and execution models (e.g., threading).
• Allow dynamic type coercion and function parameter passing with full type tracking to preserve data distribution semantics across function calls.
• Apply the type system to tune data decomposition strategies (e.g., block vs. cyclic) without modifying algorithmic logic.

Experimental results

Research questions

• RQ1 Can type annotations in a high-level language effectively encode and control low-level parallelism decisions in PGAS?
• RQ2 To what extent can type-oriented programming improve code maintainability and portability across heterogeneous architectures?
• RQ3 How does performance of a type-driven PGAS implementation compare to hand-optimized, mature libraries like FFTW3?
• RQ4 Can non-expert programmers achieve high performance through type-based tuning without deep HPC expertise?
• RQ5 What is the impact of type-based code tuning on communication overhead and scalability in data-parallel kernels?

Key findings

• The Mesham implementation achieved competitive performance with FFTW3 in 2D FFT benchmarks, demonstrating that type-driven communication optimization can match or exceed hand-tuned solutions.
• Changing data decomposition strategies required only type modifications, not algorithmic refactoring, significantly reducing tuning effort.
• The compiler successfully generated efficient, scalable code using only type information, with no need for explicit parallelism directives.
• The runtime system maintained full type and state tracking, eliminating unnecessary data redistribution during function calls.
• Performance degradation in Mesham beyond the optimal number of processes was observed, suggesting further tuning is needed for strong scaling.
• The approach enables seamless porting to heterogeneous systems via a pluggable runtime abstraction layer, with no source code changes required.