[Paper Review] FastFlow: Efficient Parallel Streaming Applications on Multi-core

Marco Aldinucci, Massimo Torquati+1 more|ArXiv.org|Sep 7, 2009

Distributed systems and fault tolerance34 references20 citations

TL;DR

FastFlow is a low-level, lock-free, fence-free programming framework for efficient parallel streaming on multi-core systems, using optimized MPMC (Multiple-Producer-Multiple-Consumer) queues to minimize synchronization overhead. It achieves up to 226% higher performance than Cilk and 96% higher than TBB on the Smith-Waterman sequence alignment benchmark, demonstrating superior efficiency for fine-grained streaming workloads.

ABSTRACT

Shared memory multiprocessors come back to popularity thanks to rapid spreading of commodity multi-core architectures. As ever, shared memory programs are fairly easy to write and quite hard to optimise; providing multi-core programmers with optimising tools and programming frameworks is a nowadays challenge. Few efforts have been done to support effective streaming applications on these architectures. In this paper we introduce FastFlow, a low-level programming framework based on lock-free queues explicitly designed to support high-level languages for streaming applications. We compare FastFlow with state-of-the-art programming frameworks such as Cilk, OpenMP, and Intel TBB. We experimentally demonstrate that FastFlow is always more efficient than all of them in a set of micro-benchmarks and on a real world application; the speedup edge of FastFlow over other solutions might be bold for fine grain tasks, as an example +35% on OpenMP, +226% on Cilk, +96% on TBB for the alignment of protein P01111 against UniProt DB using Smith-Waterman algorithm.

Motivation & Objective

• Address the performance bottleneck in shared-memory streaming applications caused by memory fences and atomic operations in traditional frameworks.
• Design a low-level framework that supports efficient, high-throughput communication in arbitrary streaming networks, including cyclic graphs.
• Enable efficient parallelization of legacy, hand-optimized code—such as the x86/SSE2 vectorized Smith-Waterman algorithm—using a high-performance communication layer.
• Demonstrate that FastFlow outperforms state-of-the-art frameworks (Cilk, OpenMP, TBB) in both micro-benchmarks and real-world streaming applications.
• Position FastFlow as a foundation for future high-level skeletal streaming frameworks on multi-core architectures.

Proposed method

• Implement lock-free, fence-free MPMC queues as the core communication primitive to eliminate synchronization overhead from mutex and atomic operations.
• Use thread-specific identifiers (via TSS or library calls) to enable efficient, low-overhead task routing in streaming networks.
• Structure the streaming pipeline as a farm pattern: Emitter generates sequence pairs, Workers process them using vectorized Smith-Waterman, and Collector aggregates results.
• Map each parallel entity (Emitter, Worker, Collector) directly to a thread, ensuring minimal runtime overhead and predictable load distribution.
• Use the Cell Updates Per Second (CUPS) metric to standardize performance comparison across different query and database sizes.
• Benchmark all implementations using the same sequential, hand-optimized x86/SSE2 Smith-Waterman code to ensure fair comparison.

Experimental results

Research questions

• RQ1 Can a lock-free, fence-free MPMC queue design achieve significantly lower communication overhead than traditional mutex-based or atomic-based synchronization in streaming workloads?
• RQ2 How does FastFlow’s performance compare to Cilk, OpenMP, TBB, and the hand-tuned SWPS3 version on the Smith-Waterman sequence alignment algorithm?
• RQ3 Does FastFlow’s low-level communication abstraction enable better performance than higher-level frameworks, especially for fine-grained, data-parallel streaming tasks?
• RQ4 Why does TBB show lower-than-expected performance on the Smith-Waterman benchmark despite strong micro-benchmark results?
• RQ5 Can FastFlow efficiently parallelize legacy, hand-optimized sequential code without requiring algorithmic changes?

Key findings

• FastFlow achieves up to 35% higher performance than OpenMP and 226% higher than Cilk on the Smith-Waterman alignment of protein P01111 against UniProt DB.
• For short query sequences, FastFlow’s performance advantage is most pronounced due to lower communication overhead relative to computation time.
• TBB’s performance on the Smith-Waterman benchmark was unexpectedly lower than expected, suggesting uncharacterized overheads in its task scheduler or queue management.
• FastFlow outperforms the state-of-the-art hand-tuned SWPS3 implementation on short sequences, demonstrating its ability to efficiently parallelize legacy code.
• The framework consistently outperforms all evaluated frameworks across both synthetic micro-benchmarks and real-world bioinformatics workloads.
• FastFlow’s design enables efficient support for arbitrary streaming network topologies, including cyclic graphs, due to its low-latency, lock-free communication channels.