[Paper Review] 300x Faster Matlab using MatlabMPI

Jeremy Kepner, Stanley C. Ahalt|arXiv (Cornell University)|Jul 18, 2002

Parallel Computing and Optimization Techniques3 citations

TL;DR

This paper presents MatlabMPI, a lightweight, pure-Matlab implementation of the MPI standard that enables high-performance parallel computing in Matlab using standard file I/O. By leveraging Matlab's built-in file operations, it achieves up to 300x speedup on 304 CPUs with only 70 lines of code, demonstrating unprecedented software price performance (0.85 GFLOPS per SLOC).

ABSTRACT

The true costs of high performance computing are currently dominated by software. Addressing these costs requires shifting to high productivity languages such as Matlab. MatlabMPI is a Matlab implementation of the Message Passing Interface (MPI) standard and allows any Matlab program to exploit multiple processors. MatlabMPI currently implements the basic six functions that are the core of the MPI point-to-point communications standard. The key technical innovation of MatlabMPI is that it implements the widely used MPI ``look and feel'' on top of standard Matlab file I/O, resulting in an extremely compact (~250 lines of code) and ``pure'' implementation which runs anywhere Matlab runs, and on any heterogeneous combination of computers. The performance has been tested on both shared and distributed memory parallel computers (e.g. Sun, SGI, HP, IBM and Linux). MatlabMPI can match the bandwidth of C based MPI at large message sizes. A test image filtering application using MatlabMPI achieved a speedup of ~300 using 304 CPUs and ~15% of the theoretical peak (450 Gigaflops) on an IBM SP2 at the Maui High Performance Computing Center. In addition, this entire parallel benchmark application was implemented in 70 software-lines-of-code (SLOC) yielding 0.85 Gigaflop/SLOC or 4.4 CPUs/SLOC, which are the highest values of these software price performance metrics ever achieved for any application. The MatlabMPI software will be available for download.

Motivation & Objective

• To address the high software cost of high-performance computing by enabling high-productivity Matlab code to run efficiently on parallel systems.
• To overcome the lack of widely adopted, portable, and performant parallelization solutions for Matlab.
• To develop a minimal, portable, and high-performance MPI interface for Matlab that runs on any system supporting Matlab.
• To demonstrate that high-performance parallel computing in Matlab can be achieved with minimal code and maximum productivity.

Proposed method

• Implementing MPI point-to-point communication using standard Matlab file I/O to avoid low-level C/Fortran dependencies.
• Using file locking mechanisms to coordinate message sending and receiving between processes.
• Mapping MPI functions (e.g., MPI_Send, MPI_Recv) to file write/read operations with metadata for message identification.
• Enabling distributed execution via the MPI_Run command, which launches parallel Matlab scripts across multiple machines.
• Designing a compact, ~250-line implementation that runs on any heterogeneous cluster where Matlab is available.
• Using a ring-based communication pattern in the image filtering benchmark to manage neighbor data exchange efficiently.

Experimental results

Research questions

• RQ1 Can a pure-Matlab implementation of MPI achieve performance comparable to C-based MPI for large message sizes?
• RQ2 Can a minimal, portable MPI interface for Matlab deliver significant speedups on large-scale parallel systems?
• RQ3 What is the achievable software price performance (e.g., GFLOPS per SLOC) when using a high-level language like Matlab with a lightweight parallel layer?
• RQ4 How does the performance of MatlabMPI scale across a large number of processors in a real HPC environment?
• RQ5 Can a high-productivity language like Matlab achieve near-peak performance on a supercomputer when paired with an efficient communication layer?

Key findings

• MatlabMPI achieved a 300x speedup using 304 CPUs on an IBM SP2, reaching 15% of the theoretical peak performance (450 GFLOPS).
• The image filtering benchmark achieved 0.85 GFLOPS per SLOC, the highest reported value for any application at the time.
• MatlabMPI matched the bandwidth of C-based MPI at large message sizes, demonstrating strong performance portability.
• The entire parallel application was implemented in just 70 lines of code, a 100% increase over the 35-line serial version, yet still highly compact.
• The implementation is fully portable and runs on any system with Matlab, using only standard file I/O and no external libraries.
• The approach enables high productivity parallel computing with minimal coding overhead, setting a new benchmark for software price performance.