[Paper Review] Myths and Legends in High-Performance Computing

Satoshi Matsuoka, Jens Domke+3 more|arXiv (Cornell University)|Jan 6, 2023

Scientific Computing and Data Management4 citations

TL;DR

This paper identifies and analyzes 12 prevalent myths and legends in the high-performance computing (HPC) community—ranging from quantum computing supremacy to deep learning replacing traditional HPC—using community polls and expert analysis. It argues these myths reflect the community's collective uncertainty amid shifting technological paradigms, advocating for them as catalysts for strategic research and investment discussions rather than literal truths.

ABSTRACT

In this thought-provoking article, we discuss certain myths and legends that are folklore among members of the high-performance computing community. We gathered these myths from conversations at conferences and meetings, product advertisements, papers, and other communications such as tweets, blogs, and news articles within and beyond our community. We believe they represent the zeitgeist of the current era of massive change, driven by the end of many scaling laws such as Dennard scaling and Moore's law. While some laws end, new directions are emerging, such as algorithmic scaling or novel architecture research. Nevertheless, these myths are rarely based on scientific facts, but rather on some evidence or argumentation. In fact, we believe that this is the very reason for the existence of many myths and why they cannot be answered clearly. While it feels like there should be clear answers for each, some may remain endless philosophical debates, such as whether Beethoven was better than Mozart. We would like to see our collection of myths as a discussion of possible new directions for research and industry investment.

Motivation & Objective

• To identify and document widely circulated myths and legends within the HPC community that shape discourse despite lacking scientific consensus.
• To analyze the underlying beliefs and economic, technical, and cultural drivers behind these myths, especially amid the end of traditional scaling laws like Moore’s and Dennard scaling.
• To stimulate critical discussion on future research and investment directions by framing myths not as falsehoods but as philosophical and strategic catalysts.
• To use large-scale community polls to quantify sentiment across key HPC debates, providing empirical grounding for anecdotal observations.
• To guide industry and government investment by distinguishing between plausible technological trajectories and speculative hype, particularly in quantum computing, AI, and cloud migration.

Proposed method

• Gathered myths from diverse sources: conferences, social media (e.g., Twitter), product ads, blogs, news, and academic papers.
• Collected community feedback via 33 structured polls across 12 myths, with questions on timelines, feasibility, and technological shifts.
• Analyzed poll results using sentiment classification (disagree to believe), visualized across myths to reveal community divides.
• Used expert reasoning and technical literature to assess the scientific plausibility of each myth, citing limitations in quantum computing, deep learning generalization, and cloud economics.
• Applied resource estimation techniques (e.g., Beverland et al., 2022) to evaluate quantum algorithm potential.
• Contrasted historical trends (e.g., end of Dennard scaling) with emerging paradigms like algorithmic scaling and reconfigurable hardware.

Experimental results

Research questions

• RQ1 What are the most persistent myths shaping current HPC discourse, and what societal or technical forces sustain them?
• RQ2 To what extent do myths like quantum computing replacing HPC or deep learning replacing simulation reflect real technological trajectories or speculative hype?
• RQ3 How do economic factors—such as CAPEX vs. OPEX—shape the debate between cloud-based and on-premise HPC infrastructures?
• RQ4 What role do community sentiment and belief structures play in guiding future HPC research and investment decisions?
• RQ5 Which myths are most divisive, and what evidence or benchmarks could resolve them definitively?

Key findings

• Community sentiment on HPC myths is deeply divided: 794 respondents disagreed with myths overall, while 682 believed in them, indicating significant polarization.
• The belief in reconfigurable hardware (e.g., FPGAs) for 100x speedups is controversial—while most do not believe in their HPC impact, a majority supports investing time and resources to explore them.
• Fewer than half of the community believe that more than half of HPC cycles will be spent in the cloud within the next decade, suggesting skepticism about cloud subsumption.
• A significant minority (682/2557) believe in the long-term viability of Fortran, defying repeated predictions of its demise, indicating enduring relevance in HPC.
• Poll results show strong skepticism toward quantum computing achieving commercial profitability within the next decade, though interest in foundational algorithms remains high.
• Despite impressive results in weather modeling and data compression, deep learning is seen as a high-risk, high-reward alternative to traditional simulation, with accuracy-speed trade-offs remaining unresolved.