[Paper Review] PyRCA: A Library for Metric-based Root Cause Analysis

Chenghao Liu, Wenzhuo Yang+4 more|arXiv (Cornell University)|Jun 20, 2023

Data Quality and ManagementDecision Sciences3 citations

TL;DR

PyRCA is an open-source Python library that provides an end-to-end framework for metric-based root cause analysis (RCA) in AIOps, integrating causal discovery, root cause scoring, and interactive visualization. It supports multiple RCA models via a unified interface, enhances results with expert knowledge, and includes a GUI dashboard for intuitive model tuning and validation, achieving state-of-the-art performance on benchmark datasets.

ABSTRACT

We introduce PyRCA, an open-source Python machine learning library of Root Cause Analysis (RCA) for Artificial Intelligence for IT Operations (AIOps). It provides a holistic framework to uncover the complicated metric causal dependencies and automatically locate root causes of incidents. It offers a unified interface for multiple commonly used RCA models, encompassing both graph construction and scoring tasks. This library aims to provide IT operations staff, data scientists, and researchers a one-step solution to rapid model development, model evaluation and deployment to online applications. In particular, our library includes various causal discovery methods to support causal graph construction, and multiple types of root cause scoring methods inspired by Bayesian analysis, graph analysis and causal analysis, etc. Our GUI dashboard offers practitioners an intuitive point-and-click interface, empowering them to easily inject expert knowledge through human interaction. With the ability to visualize causal graphs and the root cause of incidents, practitioners can quickly gain insights and improve their workflow efficiency. This technical report introduces PyRCA's architecture and major functionalities, while also presenting benchmark performance numbers in comparison to various baseline models. Additionally, we demonstrate PyRCA's capabilities through several example use cases.

Motivation & Objective

• To address the lack of a unified, open-source toolkit for end-to-end root cause analysis (RCA) in cloud and IT operations environments.
• To provide a standardized, extensible framework for loading metric data, constructing causal graphs, and scoring root causes across diverse RCA models.
• To improve RCA accuracy and usability by integrating expert knowledge through an interactive GUI dashboard.
• To enable rapid model development, evaluation, and deployment for both industrial and academic users.
• To support extensibility by allowing users to implement new RCA models through a simple class inheritance interface.

Proposed method

• PyRCA uses a unified interface to integrate multiple causal discovery algorithms, including PC and GES, for constructing causal graphs from time-series metric data.
• It implements various root cause scoring models inspired by Bayesian analysis, graph analysis, and causal inference, such as hypothesis testing (HT), random walk, and RCD.
• The library supports data loading via pandas.DataFrame and includes a built-in stats-based anomaly detector for identifying metric spikes.
• An interactive GUI dashboard enables users to upload data, visualize metrics, refine causal graphs with domain constraints (e.g., root/leaf nodes, required/forbidden links), and compare models visually.
• Users can inject expert knowledge via YAML-formatted domain knowledge files to correct or guide causal graph construction, improving robustness on noisy data.
• The framework supports model benchmarking with quantitative metrics (e.g., F1-score) and qualitative visualization of RCA results.

Experimental results

Research questions

• RQ1 How can a unified, open-source library streamline the development, evaluation, and deployment of RCA models in real-world AIOps pipelines?
• RQ2 What is the performance of different causal discovery algorithms (e.g., PC vs. GES) in constructing accurate causal graphs from metric data?
• RQ3 How does incorporating expert-provided domain knowledge improve the accuracy of causal graph construction and root cause localization?
• RQ4 To what extent does hypothesis testing (HT) outperform other root cause scoring models in benchmark scenarios?
• RQ5 Can an interactive GUI dashboard enhance usability and model validation without requiring coding?

Key findings

• The hypothesis testing (HT) method achieved the highest F1-score of 0.85 ± 0.12 on the simulated dataset, significantly outperforming other baselines.
• PC algorithm outperformed GES in causal graph construction on the simulated dataset, achieving a higher F1-score for graph accuracy.
• Using a true causal graph as input improved two-phase RCA models’ performance, highlighting the importance of accurate graph construction.
• The integration of expert knowledge through the GUI dashboard enabled users to correct erroneous links in the causal graph, improving model reliability on noisy data.
• The PyRCA GUI dashboard allowed non-coding users to interactively refine causal graphs and compare model outputs, enhancing workflow efficiency.
• The library demonstrated strong extensibility, enabling developers to add new RCA models by inheriting a base class with minimal code.