[论文解读] Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial

Venkat Pavan Nemani, Luca Biggio+7 more|arXiv (Cornell University)|May 7, 2023

Probabilistic and Robust Engineering Design参考文献 371被引用 8

一句话总结

论文综述了用于机器学习的不确定性量化（UQ）方法，聚焦于神经网络，并在电池寿命预测和涡扇发动机剩余使用寿命两个案例研究中进行演示。

ABSTRACT

On top of machine learning models, uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making by enabling sound risk assessment and management. The safety and reliability improvement of ML models empowered by UQ has the potential to significantly facilitate the broad adoption of ML solutions in high-stakes decision settings, such as healthcare, manufacturing, and aviation, to name a few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems. Toward this goal, we start with a comprehensive classification of uncertainty types, sources, and causes pertaining to UQ of ML models. Next, we provide a tutorial-style description of several state-of-the-art UQ methods: Gaussian process regression, Bayesian neural network, neural network ensemble, and deterministic UQ methods focusing on spectral-normalized neural Gaussian process. Established upon the mathematical formulations, we subsequently examine the soundness of these UQ methods quantitatively and qualitatively (by a toy regression example) to examine their strengths and shortcomings from different dimensions. Then, we review quantitative metrics commonly used to assess the quality of predictive uncertainty in classification and regression problems. Afterward, we discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics. Two case studies with source codes available on GitHub are used to demonstrate these UQ methods and compare their performance in the life prediction of lithium-ion batteries at the early stage and the remaining useful life prediction of turbofan engines.

研究动机与目标

• 在工程背景下对ML模型中的不确定性类型、来源和原因进行分类。
• 给出现代前沿的适用于神经网络的UQ方法。
• 通过示例与讨论评估UQ方法的可信度。
• 通过案例研究和可用的源代码，展示在工程设计与健康预测中的实际应用。

提出的方法

• 描述并比较高斯过程回归、贝叶斯神经网络、神经网络集成以及确定性UQ方法（光谱归一化神经高斯过程）。
• 给出将UQ方法扎根于数学的公式化表达。
• 通过一个 toy 回归示例分析UQ方法的优点与局限性。
• 回顾在分类和回归任务中评估预测不确定性的定量指标。
• 讨论UQ在工程设计与健康预测中的作用，结合两个案例研究与GitHub源代码。

实验结果

研究问题

• RQ1 在与工程设计和健康预测相关的ML模型中，不确定性的类型、来源和原因有哪些？
• RQ2 高斯过程、贝叶斯神经网络、集成方法以及确定性UQ方法在不确定性质量和鲁棒性方面有何比较？
• RQ3 在高风险领域中，哪些指标最能捕捉分类和回归任务的预测不确定性？
• RQ4 如何如案例研究所示将UQ方法有效应用于实际的PHM与设计问题？

主要发现

• 教程系统性地考察了主要UQ方法的数学基础及实际优缺点。
• 定量与定性评估展示了不同UQ方法在不同条件下的表现。
• 两个应用性案例研究展示了UQ方法在锂离子电池剩余寿命预测与涡扇发动机剩余可用寿命方面的应用。
• 案例研究的源代码已在GitHub提供，便于复制与比较。