[Paper Review] Semi-supervised Learning with Deep Generative Models for Asset Failure Prediction

A. S. Yoon, Taehoon Lee+6 more|arXiv (Cornell University)|Sep 4, 2017

Machine Fault Diagnosis TechniquesEngineering21 references59 citations

TL;DR

The paper introduces a semi-supervised framework using a variational autoencoder-based non-linear embedding with an RNN reliability model to predict remaining useful life when health labels are scarce. It shows strong gains over supervised and self-learning baselines, especially with very little labeled data.

ABSTRACT

This work presents a novel semi-supervised learning approach for data-driven modeling of asset failures when health status is only partially known in historical data. We combine a generative model parameterized by deep neural networks with non-linear embedding technique. It allows us to build prognostic models with the limited amount of health status information for the precise prediction of future asset reliability. The proposed method is evaluated on a publicly available dataset for remaining useful life (RUL) estimation, which shows significant improvement even when a fraction of the data with known health status is as sparse as 1% of the total. Our study suggests that the non-linear embedding based on a deep generative model can efficiently regularize a complex model with deep architectures while achieving high prediction accuracy that is far less sensitive to the availability of health status information.

Motivation & Objective

• Motivate data-driven prognostics when labeled health status is limited in historical data.
• Develop a deep generative model-based embedding to leverage unlabeled data.
• Train a reliability model on non-linear embeddings to predict remaining useful life.
• Evaluate the approach on NASA Turbofan (C-MAPSS) datasets across varying labeled fractions.

Proposed method

• Use a variational autoencoder (VAE) to learn a non-linear embedding from both labeled and unlabeled data.
• Parameterize the VAE with an RNN-based encoder for temporal data and a modified objective with a weighted KL and reconstruction term.
• Extract the embedding z from a kth layer of the VAE encoder to avoid stochastic posterior sampling in the predictor.
• Train an RNN-based reliability model (GRU/LSTM) on (z, y) to predict RUL.
• Compare semi-supervised VAE-SSL to supervised learning (SL) and self-learning (Self-SSL) under different labeled fractions.

Experimental results

Research questions

• RQ1 Does VAE-based non-linear embedding improve RUL prediction when labeled data are scarce?
• RQ2 How does VAE-SSL performance compare to supervised and self-learning baselines across varying fractions of labeled data?
• RQ3 What is the impact of using latent embeddings from a VAE encoder versus posterior samples on prediction accuracy?

Key findings

• VAE-SSL achieves substantial improvements over SL and Self-SSL as the labeled fraction decreases (e.g., MAE, MSE, Score, and R^2 improve with VAE-SSL at low labels).
• With only 1% labeled data, VAE-SSL maintains high predictive accuracy, outperforming SL and Self-SSL by notable margins.
• Across metrics, VAE-SSL shows up to 3.6x improvement in Score and significant gains in MAE/MSE compared to baselines at low labeling rates.
• The results on the NASA Turbofan (C-MAPSS) FD001 dataset demonstrate robustness of the embedding-based approach to label scarcity.