[Paper Review] SAIC: Identifying Configuration Files for System Configuration Management

Zhen Huang, David Lie|arXiv (Cornell University)|Nov 6, 2017

Distributed systems and fault tolerance46 references3 citations

TL;DR

SAIC is a statistical framework that automatically identifies configuration files in system administration by analyzing file content similarity across versions, leveraging the insight that configuration files change slowly over time. It reduces recovery effort by filtering non-configuration files and achieving 90% configuration file detection with a 60% false positive rate, enabling versioning systems to eliminate up to 7GB of redundant file versions across two user traces.

ABSTRACT

Systems can become misconfigured for a variety of reasons such as operator errors or buggy patches. When a misconfiguration is discovered, usually the first order of business is to restore availability, often by undoing the misconfiguration. To simplify this task, we propose the Statistical Analysis for Identifying Configuration Files (SAIC), which analyzes how the contents of a file changes over time to automatically determine which files contain configuration state. In this way, SAIC reduces the number of files a user must manually examine during recovery and allows versioning file systems to make more efficient use of their versioning storage. The two key insights that enable SAIC to identify configuration files are that configuration state must persist across executions of an application and that configuration state changes at a slower rate than other types of application state. SAIC applies these insights through a set of filters, which eliminate non-persistent files from consideration, and a novel similarity metric, which measures how similar a file's versions are to each other. Together, these two mechanisms enable SAIC to identify all 72 configuration files out of 2363 versioned files from 6 common applications in two user traces, while mistaking only 33 non-configuration files as configuration files, which allows a versioning file system to eliminate roughly 66% of non-configuration file versions from its logs, thus reducing the number of file versions that a user must try to recover from a misconfiguration.

Motivation & Objective

• To address the challenge of identifying configuration files in misconfigured systems, where manual identification is error-prone and time-consuming.
• To reduce the storage and recovery overhead of versioning file systems by filtering out non-configuration files.
• To enable automated, heuristic-based detection of configuration files without requiring prior knowledge of application structure or configuration formats.
• To guide versioning systems in selectively retaining only relevant file versions, improving efficiency and usability.

Proposed method

• SAIC applies three filters to eliminate non-persistent, temporary, and user data files before analysis, reducing the candidate set of files.
• It uses a novel similarity metric based on Rabin fingerprints to measure how consistently file content changes across versions, with high similarity indicating configuration state.
• The system computes file similarity over time using a sampling method that reduces computational cost while preserving accuracy.
• It employs a trigger point function to determine optimal measurement times, ensuring sufficient temporal coverage with minimal measurements per file.
• The method dynamically adjusts similarity thresholds based on available storage, allowing trade-offs between detection rate and false positives.
• SAIC operates as a black-box analysis tool, requiring no instrumentation or prior knowledge of the application.

Experimental results

Research questions

• RQ1 How can configuration files be automatically identified in arbitrary applications without prior knowledge of their structure or configuration formats?
• RQ2 What characteristics of file content changes over time distinguish configuration files from other types of application state?
• RQ3 To what extent can statistical similarity in file versions be used to accurately identify configuration files across diverse applications?
• RQ4 How can versioning systems reduce storage overhead and recovery complexity by selectively retaining only configuration-related file versions?

Key findings

• SAIC successfully identified 72 out of 72 configuration files (100% recall) from 2363 versioned files across six common applications in two user traces.
• The system achieved 90% configuration file detection with a false positive rate of 60%, eliminating 713,129 non-configuration file versions and saving 7GB of storage.
• With a similarity threshold of 80%, SAIC correctly identified 90% of configuration files while removing 60% of non-configuration files from versioning logs.
• The sampling and trigger point mechanisms reduced computational cost with negligible impact on accuracy, enabling continuous operation.
• The method reduced the number of files a user must examine during rollback by one to two orders of magnitude.
• The approach is effective across diverse applications, even those that store configuration in non-standard, non-descriptive files.