[Paper Review] Monte Cimone: Paving the Road for the First Generation of RISC-V High-Performance Computers

Andrea Bartolini, Federico Ficarelli+10 more|arXiv (Cornell University)|May 7, 2022

Parallel Computing and Optimization Techniques4 citations

TL;DR

Monte Cimone is the first fully operational RISC-V high-performance computing (HPC) cluster prototype, built around SiFive's U740 64-bit RISC-V SoC, demonstrating full hardware-software integration with a complete HPC stack including SLURM, NAS, Spack, and ExaMon monitoring. It proves that the first generation of RISC-V HPC systems is feasible today, achieving stable execution of HPL and STREAM benchmarks with measurable power and thermal characteristics, despite limitations in double-precision performance and InfiniBand RDMA support.

ABSTRACT

The new open and royalty-free RISC-V ISA is attracting interest across the whole computing continuum, from microcontrollers to supercomputers. High-performance RISC-V processors and accelerators have been announced, but RISC-V-based HPC systems will need a holistic co-design effort, spanning memory, storage hierarchy interconnects and full software stack. In this paper, we describe Monte Cimone, a fully-operational multi-blade computer prototype and hardware-software test-bed based on U740, a double-precision capable multi-core, 64-bit RISC-V SoC. Monte Cimone does not aim to achieve strong floating-point performance, but it was built with the purpose of "priming the pipe" and exploring the challenges of integrating a multi-node RISC-V cluster capable of providing an HPC production stack including interconnect, storage and power monitoring infrastructure on RISC-V hardware. We present the results of our hardware/software integration effort, which demonstrate a remarkable level of software and hardware readiness and maturity - showing that the first generation of RISC-V HPC machines may not be so far in the future.

Motivation & Objective

• To demonstrate that a complete RISC-V-based HPC system is feasible with current-generation components, dispelling the myth that the RISC-V software and hardware stack is too immature for production HPC use.
• To integrate and validate a full HPC software stack—including job scheduler, filesystem, compilers, libraries, and monitoring tools—on RISC-V hardware.
• To characterize power consumption and thermal behavior of the U740 SoC and cluster configuration under real HPC workloads.
• To identify and resolve hardware-level bottlenecks such as thermal runaway and suboptimal airflow in multi-blade RISC-V systems.
• To lay the foundation for future RISC-V HPC systems by establishing a production-ready test-bed for co-design and optimization.

Proposed method

• Built a multi-blade RISC-V cluster with eight nodes, each based on the SiFive U740 SoC featuring four RV64GCB cores, 16GB DDR4, 1TB NVMe, and PCIe expansion, connected via InfiniBand.
• Integrated a full HPC software stack using Spack, including SLURM job scheduler, NAS filesystem, LDAP, compilers, scientific libraries, and HPC benchmarks.
• Deployed the ExaMon ODA (Open Data Acquisition) framework for real-time monitoring of power, temperature, and system activity across nodes.
• Measured power consumption across core, DDR, and PCIe subsystems using 1ms-resolution sampling during idle and CPU-intensive workloads.
• Conducted HPL and STREAM benchmarking using the Spack-installed toolchain to evaluate performance and efficiency.
• Identified thermal instability in central nodes due to poor airflow and resolved it by modifying the enclosure to increase vertical spacing and improve cooling.

Experimental results

Research questions

• RQ1 Can a fully operational, multi-node RISC-V HPC cluster be built and operated with current-generation components and software stacks?
• RQ2 To what extent is the RISC-V software stack mature enough to support real HPC workloads, including job scheduling, file systems, and scientific computing?
• RQ3 What are the power and thermal characteristics of the U740 SoC and cluster configuration under representative HPC workloads?
• RQ4 Can hardware-level issues such as thermal runaway be detected and mitigated in early-stage RISC-V HPC prototypes?
• RQ5 How does the performance of RISC-V HPC systems compare to existing top-tier systems in terms of HPL and STREAM benchmarks?

Key findings

• The Monte Cimone cluster achieved stable execution of HPL and STREAM benchmarks using a Spack-installed toolchain, demonstrating functional readiness of the RISC-V HPC software stack.
• Idle power consumption of the U740 SoC was measured at 4.81W, with 64% attributed to core power (32% leakage, 51% dynamic and clock tree), 13% to DDR, and 23% to PCIe, PLL, and IO subsystems.
• Under CPU-intensive workloads, total power increased to 5.935W, indicating a 23% increase in power draw.
• Thermal runaway occurred on node 7 during HPL execution, reaching 107°C due to poor airflow in the 1U enclosure, which was resolved by modifying the chassis to improve cooling.
• After enclosure modification, the hottest node temperature dropped from 71°C to 39°C, confirming that thermal management is critical and addressable in RISC-V HPC systems.
• InfiniBand network adapters were recognized by the system, but RDMA support was not yet functional, highlighting a key software-hardware integration gap to be addressed in future work.