[Paper Review] Track Reconstruction in the ALICE TPC using GPUs for LHC Run 3

D. Røhr, S. Gorbunov+2 more|arXiv (Cornell University)|Nov 28, 2018

Advanced Data Storage Technologies4 citations

TL;DR

This paper presents a GPU-accelerated TPC track reconstruction algorithm for ALICE in LHC Run 3, leveraging a hybrid Cellular Automaton and Kalman filter approach with shared CPU/GPU code via OpenCL and CUDA. It achieves offline-quality resolution and efficiency while meeting stringent 50 kHz continuous readout requirements, with benchmarks showing ~20 seconds processing time per 23 ms data frame on an NVIDIA GTX 1080 GPU.

ABSTRACT

In LHC Run 3, ALICE will increase the data taking rate significantly to continuous readout of 50 kHz minimum bias Pb-Pb collisions. The reconstruction strategy of the online offline computing upgrade foresees a first synchronous online reconstruction stage during data taking enabling detector calibration, and a posterior calibrated asynchronous reconstruction stage. We present a tracking algorithm for the Time Projection Chamber (TPC), the main tracking detector of ALICE. The reconstruction must yield results comparable to current offline reconstruction and meet the time constraints like in the current High Level Trigger (HLT), processing 50 times as many collisions per second as today. It is derived from the current online tracking in the HLT, which is based on a Cellular automaton and the Kalman filter, and we integrate missing features from offline tracking for improved resolution. The continuous TPC readout and overlapping collisions pose new challenges: conversion to spatial coordinates and the application of time- and location dependent calibration must happen in between of track seeding and track fitting while the TPC occupancy increases five-fold. The huge data volume requires a data reduction factor of 20, which imposes additional requirements: the momentum range must be extended to identify low-pt looping tracks and a special refit in uncalibrated coordinates improves the track model entropy encoding. Our TPC track finding leverages the potential of hardware accelerators via the OpenCL and CUDA APIs in a shared source code for CPUs, GPUs, and both reconstruction stages. Porting more reconstruction steps like the remainder of the TPC reconstruction and tracking for other detectors will shift the computing balance from traditional processors to GPUs.

Motivation & Objective

• Address the challenge of reconstructing tracks in the ALICE TPC at 50 kHz continuous Pb–Pb collision rates, five times higher than Run 2.
• Achieve offline reconstruction quality in terms of resolution and efficiency within online processing time constraints.
• Enable real-time calibration and data compression by performing reconstruction synchronously during data taking.
• Overcome the absence of primary vertex information and time/position-dependent distortions in continuous readout mode.
• Minimize data volume by achieving a 20× compression factor through advanced track merging and entropy encoding.

Proposed method

• Adapt the Run 2 High-Level Trigger (HLT) tracking pipeline—based on Cellular Automaton seeding and Kalman filtering—for GPU execution using OpenCL and CUDA.
• Introduce on-the-fly transformation of TPC hits from native pad, row, and time coordinates to spatial (x, y, z) coordinates after vertex time estimation.
• Integrate offline tracking features such as looped track merging and central electrode crossing propagation to improve resolution.
• Implement a refit in uncalibrated coordinates to enhance track model entropy encoding for data compression.
• Design a shared-source software stack to run the same reconstruction code on CPUs and GPUs, enabling portability and performance portability.
• Plan to extend GPU acceleration to additional reconstruction stages, including ITS and TRD tracking, and data compression, minimizing host-GPU data transfers.

Experimental results

Research questions

• RQ1 How can track reconstruction in the ALICE TPC achieve offline-level resolution and efficiency under 50 kHz continuous data rates?
• RQ2 What modifications to the existing HLT tracking pipeline are necessary to support continuous readout without primary vertex assignment?
• RQ3 Can on-the-fly coordinate transformation and real-time calibration be integrated into the tracking pipeline without degrading performance?
• RQ4 What is the achievable data compression factor while preserving physics-sensitive track information for downstream analysis?
• RQ5 To what extent can GPU acceleration be extended beyond TPC tracking to cover the full reconstruction chain in the O2 computing framework?

Key findings

• The GPU-optimized TPC tracking algorithm achieves offline-quality resolution and efficiency, with track finding performance equal to or better than current offline reconstruction.
• Benchmark results show a processing time of approximately 20 seconds per 23 ms data frame on a single NVIDIA GTX 1080 GPU, meeting the real-time processing requirement.
• The algorithm successfully handles the absence of absolute z-coordinates and time/location-dependent distortions by deferring coordinate transformation until after seeding.
• The implementation supports a 20× data compression factor through advanced track merging and entropy encoding, crucial for managing the increased data volume.
• The shared CPU/GPU codebase enables seamless deployment across heterogeneous hardware, with plans to extend GPU acceleration to ITS, TRD, and data compression stages.
• A projected GPU capacity of ~1000 cards is required for full TPC tracking, with 1500 GPU-equipped servers planned for the O2 computing farm, leaving sufficient headroom for additional tasks.