[Paper Review] Autoencoder-based Anomaly Detection System for Online Data Quality Monitoring of the CMS Electromagnetic Calorimeter

The CMS ECAL Collaboration|arXiv (Cornell University)|Sep 18, 2023

Anomaly Detection Techniques and ApplicationsComputer Science21 references3 citations

TL;DR

This paper presents a semi-supervised autoencoder-based anomaly detection system for real-time online data quality monitoring in the CMS electromagnetic calorimeter (ECAL), leveraging 2D detector response images with spatial and time-dependent corrections to improve anomaly detection performance. The system achieves a 10× improvement in detection accuracy and successfully identifies degrading channels missed by conventional DQM, reducing false alarms in LHC Run 3 operations.

ABSTRACT

The CMS detector is a general-purpose apparatus that detects high-energy collisions produced at the LHC. Online Data Quality Monitoring of the CMS electromagnetic calorimeter is a vital operational tool that allows detector experts to quickly identify, localize, and diagnose a broad range of detector issues that could affect the quality of physics data. A real-time autoencoder-based anomaly detection system using semi-supervised machine learning is presented enabling the detection of anomalies in the CMS electromagnetic calorimeter data. A novel method is introduced which maximizes the anomaly detection performance by exploiting the time-dependent evolution of anomalies as well as spatial variations in the detector response. The autoencoder-based system is able to efficiently detect anomalies, while maintaining a very low false discovery rate. The performance of the system is validated with anomalies found in 2018 and 2022 LHC collision data. Additionally, the first results from deploying the autoencoder-based system in the CMS online Data Quality Monitoring workflow during the beginning of Run 3 of the LHC are presented, showing its ability to detect issues missed by the existing system.

Motivation & Objective

• To develop an automated, real-time anomaly detection system for the CMS ECAL to enhance data quality monitoring during high-energy proton-proton collisions.
• To address limitations of traditional cut-based DQM systems in detecting subtle, evolving, or transient anomalies due to aging electronics and increasing collision rates.
• To improve anomaly detection performance by modeling spatial variations in detector response and time-dependent anomaly evolution.
• To deploy the system operationally in the CMS online DQM workflow during LHC Run 3 for real-time diagnostics.
• To reduce false discovery rates while maintaining high sensitivity to anomalies across diverse detector conditions.

Proposed method

• The system processes ECAL occupancy histograms as 2D images and feeds them into a deep autoencoder trained in a semi-supervised manner on normal data.
• Spatial response corrections are applied to account for intrinsic non-uniformities in detector response across the ECAL geometry.
• Time-dependent corrections are introduced to model the evolution of anomalies over time, such as degrading or transient channels.
• Anomaly scores are computed as the reconstruction error of the autoencoder, with thresholds set using validation data with injected fake anomalies.
• The system uses a single threshold per ECAL region (barrel/endcap) to flag anomalies at the tower level, enabling localization.
• Post-processing includes frequency-based masking of persistently flagged towers to proactively identify degrading channels.

Figure 1: Schematic view of the CMS detector and its various subdetectors.

Experimental results

Research questions

• RQ1 Can a semi-supervised autoencoder effectively detect anomalies in real-time ECAL data with low false positive rates?
• RQ2 How does incorporating spatial and temporal corrections improve anomaly detection performance compared to baseline methods?
• RQ3 Can the system detect anomalies that are missed by existing cut-based DQM systems, particularly transient or slowly degrading channels?
• RQ4 To what extent does the system maintain robustness across varying pile-up conditions and detector aging effects?
• RQ5 How effective is the system in localizing anomalies at the individual tower level during real LHC operations?

Key findings

• The autoencoder-based system achieved a 10× improvement in anomaly detection performance over baseline methods by integrating spatial and time corrections.
• The system successfully detected degrading ECAL towers with persistent zero occupancy, including cases missed by the existing DQM system during Run 3.
• In validation with real anomalies from 2018 and 2022, the system detected anomalies of various shapes, sizes, and locations with high sensitivity at the tower level.
• The anomaly tagging threshold was set at a 99% estimated tagging rate, ensuring high recall while minimizing false positives.
• The deployment in Run 3 demonstrated the system's ability to identify transient anomalies, such as Tower 2’s intermittent failure, which were not consistently flagged by conventional DQM.
• The system complements existing DQM workflows, enabling faster and more accurate diagnosis of detector issues by ECAL experts.

Figure 2: Schematic view of the ECAL showing the cylindrical barrel closed by the two endcap regions with one half endcap displayed.