[Paper Review] SemParser: A Semantic Parser for Log Analysis

Yintong Huo, Yuxin Su+2 more|arXiv (Cornell University)|Dec 23, 2021

Software System Performance and Reliability4 citations

TL;DR

SemParser is a novel semantic parser for log analysis that addresses the limitations of syntax-based log parsers by incorporating explicit and implicit semantics from log messages. It uses a two-stage approach—end-to-end semantics mining followed by joint parsing with contextual knowledge—achieving an average F1 score of 0.985 in semantic mining and improving downstream anomaly detection and failure identification models by up to 11.7% and 8.65%, respectively.

ABSTRACT

Logs, being run-time information automatically generated by software, record system events and activities with their timestamps. Before obtaining more insights into the run-time status of the software, a fundamental step of log analysis, called log parsing, is employed to extract structured templates and parameters from the semi-structured raw log messages. However, current log parsers are all syntax-based and regard each message as a character string, ignoring the semantic information included in parameters and templates. Thus, we propose the semantic-based parser SemParser to unlock the critical bottleneck of mining semantics from log messages. It contains two steps, an end-to-end semantic miner and a joint parser. Specifically, the first step aims to identify explicit semantics inside a single log, and the second step is responsible for jointly inferring implicit semantics and computing structural outputs based on the contextual knowledge base. To analyze the effectiveness of our semantic parser, we first demonstrate that it can derive rich semantics from log messages collected from six widely-applied systems with an average F1 score of 0.985. Then, we conduct two representative downstream tasks, showing that current downstream models improve their performance with appropriately extracted semantics by 1.2%-11.7% and 8.65% on two anomaly detection datasets and a failure identification dataset, respectively. We believe these findings provide insights into semantically understanding log messages for the log analysis community.

Motivation & Objective

• To address the limitations of syntax-based log parsers that ignore semantic meaning in parameters and templates.
• To identify informative tokens, extract intra-message semantics, and infer inter-message relationships in log messages.
• To enhance downstream log analysis tasks such as anomaly detection and failure identification through richer semantic representations.
• To develop a scalable, generalizable parser that overcomes the closed-world assumption of prior NER-based approaches.
• To demonstrate that semantic understanding significantly improves the performance of existing log analysis models.

Proposed method

• A two-phase architecture: first, an end-to-end semantics miner identifies explicit semantic roles within individual log messages.
• Second, a joint parser infers implicit semantics using a contextual knowledge base derived from historical logs.
• The model leverages domain-specific knowledge to resolve parameter meanings (e.g., mapping '949e1227' to 'cell ID').
• It integrates contextual dependencies across logs to infer semantics not explicitly stated in a single message.
• The framework is trained end-to-end and does not rely on handcrafted rules or pre-defined entity types.
• It uses deep learning components to encode log messages and jointly optimize for semantic labeling and structural parsing.

Experimental results

Research questions

• RQ1 Can a semantic parser effectively extract explicit semantics from individual log messages across diverse software systems?
• RQ2 To what extent can contextual knowledge from historical logs improve the inference of implicit semantics in log parameters?
• RQ3 How does semantic-aware parsing enhance the performance of downstream log analysis tasks like anomaly detection and failure diagnosis?
• RQ4 Can the proposed parser generalize across different systems without requiring system-specific rule engineering?
• RQ5 Does semantic parsing outperform syntax-based parsers in terms of both semantic accuracy and downstream task effectiveness?

Key findings

• SemParser achieves an average F1 score of 0.985 in semantic mining across six widely used software systems, demonstrating high accuracy in identifying explicit semantics.
• In anomaly detection, models using SemParser’s output improved performance by 1.2% to 11.7% compared to syntax-based parsers.
• For failure identification, the downstream model improved by 8.65% when using SemParser-extracted semantics.
• The joint parsing mechanism successfully infers implicit semantics—such as parameter types (e.g., cell ID)—using contextual knowledge from prior logs.
• The approach outperforms rule-based and heuristic parsers by eliminating the need for manual rule creation and reducing dependency on frequency-based heuristics.
• The results confirm that semantic understanding significantly enhances the quality of log representations for downstream AI-driven analysis.