[Paper Review] Semi-Supervised Learning of Bearing Anomaly Detection via Deep Variational Autoencoders

Shen Zhang, Fei Ye+2 more|arXiv (Cornell University)|Dec 2, 2019

Machine Fault Diagnosis Techniques27 references30 citations

TL;DR

This paper proposes a semi-supervised bearing anomaly detection method using deep variational autoencoders (VAEs) to leverage limited labeled data and abundant unlabeled vibration signals. By jointly training a VAE with a classifier, the model learns robust representations and achieves up to 30% accuracy improvement over supervised and baseline semi-supervised methods on CWRU and IMS datasets, especially when only 4–15% of data are labeled.

ABSTRACT

Most of the data-driven approaches applied to bearing fault diagnosis up to date are established in the supervised learning paradigm, which usually requires a large set of labeled data collected a priori. In practical applications, however, obtaining accurate labels based on real-time bearing conditions can be far more challenging than simply collecting a huge amount of unlabeled data using various sensors. In this paper, we thus propose a semi-supervised learning approach for bearing anomaly detection using variational autoencoder (VAE) based deep generative models, which allows for effective utilization of dataset when only a small subset of data have labels. Finally, a series of experiments is performed using both the Case Western Reserve University (CWRU) bearing dataset and the University of Cincinnati's Center for Intelligent Maintenance Systems (IMS) dataset. The experimental results demonstrate that the proposed semi-supervised learning scheme greatly outperforms two mainstream semi-supervised learning approaches and a baseline supervised convolutional neural network approach, with the overall accuracy improvement ranging between 3% to 30% using different proportions of labeled samples.

Motivation & Objective

• Address the challenge of limited labeled bearing fault data in real-world industrial settings, where collecting large volumes of unlabeled data is easier than obtaining precise fault labels.
• Overcome the limitations of supervised learning, which requires extensive labeled data for reliable fault classification.
• Develop a semi-supervised deep generative model that effectively utilizes both small labeled and large unlabeled datasets to improve anomaly detection performance.
• Validate the method on real-world bearing datasets (CWRU and IMS) with naturally evolving faults, where labeling ambiguity is common at early fault stages.
• Demonstrate that incorporating unlabeled data through a VAE-based semi-supervised framework enhances classifier generalization and robustness against noisy or miss-labeled data.

Proposed method

• Propose a deep variational autoencoder (VAE) framework with a joint encoder-decoder structure that learns hierarchical latent representations from raw bearing vibration signals.
• Integrate a classifier head into the VAE architecture to enable end-to-end semi-supervised training, where labeled data supervise classification and unlabeled data guide representation learning.
• Use the reparameterization trick in VAEs to enable backpropagation through stochastic latent variables, allowing optimization of the variational lower bound (ELBO) objective.
• Train the model using a combination of reconstruction loss (for autoencoding) and classification loss (for labeled data), enabling joint optimization of representation and prediction.
• Apply two variants: VAE M1 (standard VAE) and VAE M2 (with additional regularization and improved latent space modeling), to evaluate performance under varying label scarcity.
• Utilize data from the CWRU and IMS bearing datasets, applying signal preprocessing and feature extraction before feeding into the VAE model.

Experimental results

Research questions

• RQ1 Can a deep VAE-based semi-supervised model achieve superior anomaly detection performance compared to supervised and unsupervised baselines when only a small fraction of bearing data are labeled?
• RQ2 How does the inclusion of unlabeled data affect the generalization and robustness of the classifier, especially in cases with ambiguous or noisy labels?
• RQ3 Does the proposed VAE-based semi-supervised approach outperform existing state-of-the-art semi-supervised methods like graph-based models or deep ladder networks on bearing fault detection tasks?
• RQ4 To what extent does the model’s performance degrade when labels are inaccurately assigned, particularly during early fault stages with subtle features?
• RQ5 How does the performance of the VAE M2 variant compare to VAE M1 and other models (e.g., CNN, autoencoder) across different label proportions on real-world datasets?

Key findings

• On the CWRU dataset, the proposed VAE M2 model achieved a 30% accuracy improvement over the baseline supervised CNN when only 4% of the data were labeled.
• With just 10 labeled samples (0.4% of total), VAE M2 reached 23.71% accuracy, outperforming PCA+SVM (17.10%) and autoencoder (27.72%) by over 6 percentage points.
• On the IMS dataset, VAE M2 achieved 90.87% accuracy with 2,000 labeled samples (25% of total), significantly outperforming the CNN (86.62%) and PCA+SVM (78.50%) under the same conditions.
• The model demonstrated consistent performance gains across label proportions: improvements ranged from 3% to 30% over baselines, with lower variance in predictions.
• When labeled data increased from 4,000 to 8,000, the CNN’s accuracy dropped by over 6%, while VAE M2 only lost 4%, indicating greater robustness to label noise and misclassification.
• The results suggest that semi-supervised VAEs can effectively mitigate performance degradation caused by inaccurate labeling, especially in early fault detection where labels are ambiguous.