[Paper Review] A Survey on Automated Log Analysis for Reliability Engineering

Shilin He, Pinjia He+4 more|arXiv (Cornell University)|Sep 15, 2020

Software System Performance and Reliability171 references35 citations

TL;DR

A comprehensive survey of automated log analysis for reliability engineering, covering logging practices, log compression, log parsing, log mining, datasets, toolkits, and future directions.

ABSTRACT

Logs are semi-structured text generated by logging statements in software source code. In recent decades, software logs have become imperative in the reliability assurance mechanism of many software systems because they are often the only data available that record software runtime information. As modern software is evolving into a large scale, the volume of logs has increased rapidly. To enable effective and efficient usage of modern software logs in reliability engineering, a number of studies have been conducted on automated log analysis. This survey presents a detailed overview of automated log analysis research, including how to automate and assist the writing of logging statements, how to compress logs, how to parse logs into structured event templates, and how to employ logs to detect anomalies, predict failures, and facilitate diagnosis. Additionally, we survey work that releases open-source toolkits and datasets. Based on the discussion of the recent advances, we present several promising future directions toward real-world and next-generation automated log analysis.

Motivation & Objective

• Motivate the study of software reliability and the pivotal role of logs as runtime data.
• Synthesize 158 papers (1997–2020) across logging, compression, parsing, mining, and empirical studies.
• Highlight open-source toolkits and datasets and identify practical challenges and industry practices.

Proposed method

• Systematically collected literature from top venues using targeted keywords related to logs and reliability.
• Classified surveyed works into five focus areas: logging, log compression, log parsing, log mining, and empirical studies.
• Analyzed trends, open-source resources, and practical challenges to map state-of-the-art and gaps.

Experimental results

Research questions

• RQ1 What are the main research directions in automated log analysis for reliability engineering?
• RQ2 What techniques (static analysis, machine learning, empirical studies) dominate each direction?
• RQ3 What open-source tools and datasets are available to practitioners and researchers?
• RQ4 What future directions address real-world deployment challenges in logging and log analysis?

Key findings

• Log mining receives the largest research share, driven by its decomposed sub-tasks such as anomaly detection, failure prediction, and diagnosis.
• Logging practices and maintenance are actively studied, including where-to-log, what-to-log, and how-to-log, with attention to diagnosability, performance, and maintainability.
• Multiple log compression approaches exist (bucket-based, dictionary-based, statistics-based), each with trade-offs in generality, scalability, and heuristics.
• Several tools and datasets have been released to support research and practice, enabling reproducibility and industrial evaluation.
• There is a gap between academic advances and real-world deployment, with future directions emphasizing real-world applicability and next-generation log analysis.