[Paper Review] Robust Variational Autoencoders for Outlier Detection and Repair of Mixed-Type Data

Simão Eduardo, Alfredo Nazábal+2 more|arXiv (Cornell University)|Jul 15, 2019

Anomaly Detection Techniques and Applications5 references4 citations

TL;DR

This paper proposes the Robust Variational Autoencoder (RVAE), a deep generative model that jointly detects and repairs cell-level outliers in mixed-type tabular data by modeling each feature with a two-component mixture: one for clean data and one for outliers. By down-weighting anomalous cells during training and using a hybrid inference scheme, RVAE achieves superior performance in outlier detection and repair while remaining robust to hyperparameter choices.

ABSTRACT

We focus on the problem of unsupervised cell outlier detection and repair in mixed-type tabular data. Traditional methods are concerned only with detecting which rows in the dataset are outliers. However, identifying which cells are corrupted in a specific row is an important problem in practice, and the very first step towards repairing them. We introduce the Robust Variational Autoencoder (RVAE), a deep generative model that learns the joint distribution of the clean data while identifying the outlier cells, allowing their imputation (repair). RVAE explicitly learns the probability of each cell being an outlier, balancing different likelihood models in the row outlier score, making the method suitable for outlier detection in mixed-type datasets. We show experimentally that not only RVAE performs better than several state-of-the-art methods in cell outlier detection and repair for tabular data, but also that is robust against the initial hyper-parameter selection.

Motivation & Objective

• Address the gap in unsupervised outlier detection that focuses only on row-level anomalies, not cell-level corruption.
• Develop a method that identifies which specific cells in a row are anomalous, enabling precise data repair.
• Handle mixed-type tabular data (continuous and categorical features) with a unified, probabilistically consistent outlier scoring mechanism.
• Improve robustness of deep generative models to outliers by down-weighting their influence during training.
• Provide a principled, commensurate outlier score across different feature types without requiring labeled data or extensive hyperparameter tuning.

Proposed method

• Introduce a two-component mixture model per feature: one component models clean data, and another isolates outlier cells.
• Use a robust divergence (α-divergence) to down-weight the contribution of anomalous cells during model optimization.
• Implement a hybrid inference scheme combining amortized and exact variational updates to improve posterior estimation.
• Model continuous features using a Gaussian likelihood and categorical features using a softmax likelihood within the VAE framework.
• Learn the probability of each cell being an outlier via the mixture component allocation, enabling cell-level detection.
• Construct a row-level outlier score by combining likelihoods across features using a balanced, commensurate scoring mechanism.

Experimental results

Research questions

• RQ1 Can a deep generative model detect and repair individual outlier cells in mixed-type tabular data without relying on labeled data?
• RQ2 How can a unified, comparable outlier score be constructed across continuous and categorical features in a deep generative model?
• RQ3 Does incorporating robust inference during training improve both outlier detection and data repair performance compared to standard VAEs?
• RQ4 To what extent is the model's performance robust to hyperparameter selection, especially in the absence of labeled validation sets?
• RQ5 Can a hybrid inference strategy combining amortized and exact updates outperform standard amortized inference in outlier detection tasks?

Key findings

• RVAE outperforms several state-of-the-art methods in cell-level outlier detection and repair on mixed-type tabular data, particularly in scenarios with high noise or complex feature types.
• The model achieves consistent repair performance across different values of the α hyperparameter, with degradation only near extreme values (α ≈ 0 or 1), indicating robustness to hyperparameter choice.
• The hybrid inference scheme significantly improves posterior estimation and model performance compared to standard amortized inference.
• The proposed row outlier score, based on a balanced combination of likelihoods across mixed-type features, provides a commensurate and effective measure for detecting anomalous rows.
• RVAE reduces overfitting to outliers by down-weighting their influence during training, enabling more accurate estimation of clean data distributions.
• The method successfully identifies cell-level outliers even in high-dimensional datasets with mixed feature types, where traditional VAEs fail due to non-comparable likelihoods across feature types.