[Paper Review] Optimal Low-Latency Network Topologies for Cluster Performance Enhancement

Yuefan Deng, Meng Guo+4 more|arXiv (Cornell University)|Apr 1, 2019

Interconnection Networks and Systems49 references4 citations

TL;DR

This paper proposes that minimizing mean path length in regular network topologies significantly enhances cluster performance. By simulating 256-node clusters with topologies like Dragonfly, torus, and Chvatal, the authors demonstrate that optimal low-latency designs improve performance by several folds in communication-intensive workloads, proving topology alone can reclaim performance from identical hardware.

ABSTRACT

We propose that clusters interconnected with network topologies having minimal mean path length will increase their overall performance for a variety of applications. We approach our heuristic by constructing clusters of up to 36 nodes having Dragonfly, torus, ring, Chvatal, Wagner, Bidiakis and several other topologies with minimal mean path lengths and by simulating the performance of 256-node clusters with the same network topologies. The optimal (or sub-optimal) low-latency network topologies are found by minimizing the mean path length of regular graphs. The selected topologies are benchmarked using ping-pong messaging, the MPI collective communications, and the standard parallel applications including effective bandwidth, FFTE, Graph 500 and NAS parallel benchmarks. We established strong correlations between the clusters' performances and the network topologies, especially the mean path lengths, for a wide range of applications. In communication-intensive benchmarks, clusters with optimal network topologies out-perform those with mainstream topologies by several folds. It is striking that a mere adjustment of the network topology suffices to reclaim performance from the same computing hardware.

Motivation & Objective

• To identify network topologies that minimize mean path length in regular graphs for improved cluster performance.
• To evaluate the impact of network topology on communication-intensive HPC workloads in clusters.
• To demonstrate that topology optimization can significantly enhance performance without changing hardware.
• To benchmark various topologies—Dragonfly, torus, ring, Chvatal, Wagner, Bidiakis—using standard HPC applications.
• To establish a strong correlation between mean path length and overall cluster performance across diverse workloads.

Proposed method

• Constructing clusters of up to 36 nodes with various regular topologies to analyze mean path length.
• Simulating 256-node clusters using the same topologies to assess scalability and performance.
• Minimizing mean path length as the primary heuristic for selecting optimal or sub-optimal topologies.
• Benchmarking topologies using ping-pong messaging, MPI collective operations, and standard HPC workloads (e.g., NAS Parallel Benchmarks, Graph 500, FFTE).
• Analyzing performance correlations with network topology, especially mean path length, across diverse applications.
• Using simulation-based evaluation to compare performance across topologies under identical hardware conditions.

Experimental results

Research questions

• RQ1 Which network topologies minimize mean path length in regular graphs for cluster interconnects?
• RQ2 How does mean path length correlate with performance in communication-intensive HPC applications?
• RQ3 To what extent can topology optimization improve cluster performance without hardware changes?
• RQ4 How do Dragonfly, torus, and other topologies compare in terms of latency and performance under real HPC workloads?
• RQ5 Can sub-optimal topologies be identified and replaced with optimal ones to achieve significant performance gains?

Key findings

• Clusters with optimal low-latency topologies outperform those with mainstream topologies by several folds in communication-intensive benchmarks.
• A strong correlation was established between mean path length and overall cluster performance across a wide range of applications.
• The same computing hardware achieved significantly higher performance when paired with an optimal network topology.
• Topologies such as Dragonfly and torus showed superior performance when optimized for minimal mean path length.
• The ping-pong and MPI collective communication benchmarks confirmed that reduced mean path length directly improves latency and bandwidth.
• The study demonstrates that network topology is a critical, underutilized lever for performance enhancement in HPC clusters.