[Paper Review] Accelerated Charged Particle Tracking with Graph Neural Networks on FPGAs

Aneesh Heintz, Vesal Razavimaleki+19 more|arXiv (Cornell University)|Nov 30, 2020

Parallel Computing and Optimization Techniques30 references31 citations

TL;DR

The paper presents two FPGA implementations (OpenCL coprocessing and hls4ml-based) of a graph neural network for charged particle tracking, achieving substantial speedups over CPU and enabling potential use in FPGA-based L1 triggers at the LHC.

ABSTRACT

We develop and study FPGA implementations of algorithms for charged particle tracking based on graph neural networks. The two complementary FPGA designs are based on OpenCL, a framework for writing programs that execute across heterogeneous platforms, and hls4ml, a high-level-synthesis-based compiler for neural network to firmware conversion. We evaluate and compare the resource usage, latency, and tracking performance of our implementations based on a benchmark dataset. We find a considerable speedup over CPU-based execution is possible, potentially enabling such algorithms to be used effectively in future computing workflows and the FPGA-based Level-1 trigger at the CERN Large Hadron Collider.

Motivation & Objective

• Motivate accelerated tracking for high-energy physics using heterogeneous hardware to meet tight latency and data throughput demands.
• Adapt and implement a graph neural network for segment classification on FPGAs.
• Evaluate resource usage, latency, and physics performance on benchmark TrackML data.
• Demonstrate potential integration of FPGA-based tracking in online trigger workflows at the LHC.

Proposed method

• Two FPGA-targeted GNN implementations of an interaction-network (IN) model for segment classification on graph-embedded detector hits.
• OpenCL implementation uses CPU-FPGA coprocessing with FPGA-accelerated matrix multiplications and input graph padding to uniform sizes.
• hls4ml implementation translates the neural network to FPGA firmware with pipelining, streaming inputs, and configurable reuse factors to control latency and parallelism.
• Edge and node blocks are composed of small multilayer perceptrons with ReLU activations and a sigmoid output for edge classification.
• Inputs include node features (r, phi, z) and edge features (Delta r, Delta phi, Delta z, Delta R) for one model; another variant uses only basic edge features.
• Performance metrics include resource usage, latency, and ROC-AUC (AUC) as a function of bit precision and model size.

Experimental results

Research questions

• RQ1 Can GNN-based segment classification for track reconstruction be efficiently implemented on FPGA hardware using OpenCL and hls4ml?
• RQ2 What are the resource, latency, and physics-performance trade-offs when comparing OpenCL coprocessing and hls4ml FPGA implementations?
• RQ3 How does model precision and reuse factor affect latency and ROC performance in FPGA implementations?
• RQ4 What is the potential speedup over CPU-based inference for these FPGA implementations in TrackML-like datasets?
• RQ5 Are these FPGA approaches viable for integration into LHC Level-1 trigger systems with sub-microsecond requirements?

Key findings

• OpenCL FPGA implementation achieves latency in the range of 10 ms to 1 s for full-event graphs, including data transfer and I/O.
• hls4ml implementation targets ultra-low latency, with FPGA execution latency of ~650 ns to 1 μs for smaller, sectorized graphs.
• CPU-based inference on the same models is significantly slower, e.g., ~27 ms in graph_nets TensorFlow implementation for pT>2 GeV graphs and ~86 ms (approx.) for pT>1 GeV in PyTorch, illustrating substantial speedups for FPGA implementations.
• OpenCL resource usage declines with lower data precision (8, 16, 32-bit), and latency scales with minimum pT and event size, showing flexible data-size handling under coprocessing.
• The hls4ml model reproduces full FP32 performance with around 12 total bits in fixed-point representation and latency in the 650 ns–1 μs range; higher reuse factors increase latency but reduce resource usage.
• Compared to CPU-only workflows, the FPGA approaches offer notable speedups, with ongoing work to optimize resource usage and further reduce latency for OpenCL-based workflows.