[Paper Review] A Vision on the Status and Evolution of HEP Physics Software Tools

Philippe Canal, D. Elvira+3 more|arXiv (Cornell University)|Jul 29, 2013

Particle physics theoretical and experimental studies1 references3 citations

TL;DR

This paper proposes a major architectural overhaul of Geant4 to enable efficient, scalable, and reproducible simulation on modern parallel and heterogeneous computing systems. By rethinking event and particle processing through data-parallel track bundles and intelligent dispatching across CPUs and GPUs, the authors aim to drastically improve performance and enable Exabyte-scale HEP simulations while preserving physics validation and reproducibility.

ABSTRACT

This paper represents the vision of the members of the Fermilab Scientific Computing Division's Computational Physics Department (SCD-CPD) on the status and the evolution of various HEP software tools such as the Geant4 detector simulation toolkit, the Pythia and GENIE physics generators, and the ROOT data analysis framework. The goal of this paper is to contribute ideas to the Snowmass 2013 process toward the composition of a unified document on the current status and potential evolution of the physics software tools which are essential to HEP.

Motivation & Objective

• Address the growing computational demands of high-energy physics (HEP) simulations, which exceed the capabilities of current Geant4's event- and particle-by-particle processing model.
• Enable efficient utilization of emerging leadership-class computing platforms, including multi-core CPUs and GPU-accelerated clusters, to meet future simulation workloads.
• Improve simulation performance and scalability by re-architecting Geant4 around data-parallel processing of track bundles rather than sequential event-by-event execution.
• Ensure physics validation and reproducibility are maintained despite the introduction of non-deterministic parallelism and complex I/O patterns.
• Reduce human effort in managing increasingly large, globally distributed simulation datasets by optimizing data organization and I/O workflows.

Proposed method

• Re-engineer Geant4's processing model to operate on vectors of particles (track bundles) across event boundaries, enabling bulk processing for improved cache and memory locality.
• Implement a track dispatcher that schedules bundles to optimal processing units—such as specific CPU cores or GPU clusters—based on geometry and computational requirements.
• Introduce hardware-aware scheduling that assigns track bundles to processors with associated geometrical elements to maximize on-chip cache utilization.
• Decouple data organization and I/O management from the simulation core, potentially offloading these tasks to specialized run-time services for better scalability.
• Design random number stream management strategies to preserve reproducibility across parallel and non-deterministic execution environments.
• Leverage real-world Geant4 program traces to prototype and validate the new processing architecture.

Experimental results

Research questions

• RQ1 How can Geant4 be re-architected to efficiently exploit modern multi-core and GPU-accelerated computing systems without sacrificing physics accuracy?
• RQ2 What are the performance bottlenecks in current Geant4’s sequential processing model when scaling to Exabyte-scale simulation data?
• RQ3 How can reproducibility and physics validation be preserved in a non-deterministic, parallel execution environment?
• RQ4 What role should external I/O run-time services play in managing data organization and access for large-scale simulations?
• RQ5 What are the most effective abstractions for organizing and scheduling track bundles to maximize hardware utilization and minimize latency?

Key findings

• A shift from event- and particle-by-particle processing to data-parallel track bundle processing offers significant potential for performance gains on modern hardware architectures.
• The proposed architecture enables better utilization of CPU and GPU caches by co-locating tracks with their relevant geometrical elements and processing them together.
• Reproducibility can be maintained through careful random number stream management and seed recording, even in non-deterministic parallel execution.
• I/O performance is expected to improve by offloading data collection and organization to specialized run-time services, reducing contention and bottlenecks.
• Prototype validation using real Geant4 program traces confirms the feasibility of the new processing model and suggests near-term performance improvements on individual processors.
• The transformation of Geant4 is essential not only for future experiments but also for enabling current HEP collaborations to fully utilize existing leadership-class computing resources.