[论文解读] Empirical Characterization of Logging Smells in Machine Learning Code

Patrick Loic Foalem, Léuson Da Silva+3 more|arXiv (Cornell University)|Mar 24, 2026

Scientific Computing and Data Management被引用 0

一句话总结

简而言之：本文在444个开源ML仓库中实证识别出12个与ML相关的日志记录“臭味”，通过从业者调查进行验证，并发布一个包含2,448个臭味实例的带标签数据集。

ABSTRACT

Logging plays a central role in ensuring reproducibility, observability, and reliability in machine learning (ML) systems. While logging is generally considered a good engineering practice, poorly designed logging can negatively affect experiment tracking, security, debugging, and system performance. In this paper, we present an empirical study of logging smells in ML projects and propose a taxonomy of ML-specific logging smell types. We conducted a large-scale analysis of 444 ML repositories and manually labeled 2,448 instances of logging smells. Based on this analysis, we identified 12 categories of logging smells spanning security, metric management, configuration, verbosity, and context-related issues. Our results show that logging smells are widespread in ML systems and vary in frequency and manifestation across projects. To assess practical relevance, we conducted a survey with 27 ML practitioners. Most respondents agreed with the identified smells and reported that several types, including Logging Sensitive Data, Metric Overwrite, Missing Hyperparameter Logging, and Log Without Context, have a strong impact on reproducibility, maintainability, and trustworthiness. Other smells, such as Heavy Data Logging and Print-based Logging, were perceived as more context-dependent. We publicly release our labeled dataset to support future research. Our findings highlight logging quality as a critical and underexplored aspect of ML system engineering and open opportunities for automated detection and repair of logging issues.

研究动机与目标

• 通过展示日志记录对ML可观测性至关重要但在ML代码库中常设计不良来驱动研究。
• 通过大规模标注构建面向ML特定日志记录臭味的数据驱动分类法。
• 与ML从业者验证该分类法，以评估相关性和严重性。
• 提供带标签数据集的复现包以支持未来研究。

提出的方法

• 使用Foalem等人（2024年）现有的ML日志实践数据集，在444个活跃的ML仓库的Python文件中识别日志语句。
• 聚类并过滤与日志相关的代码，排除纯配置语句，得到4,528个函数中的15,911条日志语句。
• 采用人机循环、结合LLM辅助编码（GPT-5-mini）与人工验证的迭代分类法构建12种臭味。
• 将2,448个函数级样本拆分为阶段以进行种子生成、迭代细化、达到饱和与可重复性评估（评审者间的一致性κ = 0.817）。
• 生成结构化的JSON数据集，记录使用场景、片段和元数据，并发布以用于复现。

实验结果

研究问题

• RQ1 RQ1：开源基于ML的系统中存在哪些日志臭味？
• RQ2 RQ2：ML从业者在实践中如何感知和体验这些日志臭味？

主要发现

• 识别出12种ML特定日志臭味，涵盖安全、指标管理、配置、冗长性和与上下文相关的问题。
• 臭味广泛存在，且在不同项目中出现频率和表现形式各异。
• 从业者的回答显示与分类法高度一致，并且对影响安全、可重复性和实验正确性的臭味有更高的相关性和严重性认知。
• 某些臭味（如Heavy Data Logging、Print-based Logging）具有情境依赖性，反映性能与可观测性之间的权衡。
• 公开发布的2,448个日志臭味实例数据集支持自动检测和AI辅助修复研究。