[Paper Review] Semi-supervised Learning with Missing Values Imputation

Buliao Huang, Yunhui Zhu+2 more|arXiv (Cornell University)|Jun 3, 2021

Machine Learning and Data Classification44 references4 citations

TL;DR

This paper proposes Semi-supervised Conditional Normalizing Flow (SSCFlow), a joint imputation and classification framework that leverages label information to improve missing value estimation and classification accuracy. By modeling the conditional distribution of incomplete instances using a semi-supervised normalizing flow and iteratively refining corrupted imputations via a denoising autoencoder, SSCFlow achieves state-of-the-art performance on multiple real-world datasets with missing values.

ABSTRACT

Incomplete instances with various missing attributes in many real-world applications have brought challenges to the classification tasks. Missing values imputation methods are often employed to replace the missing values with substitute values. However, this process often separates the imputation and classification, which may lead to inferior performance since label information are often ignored during imputation. Moreover, traditional methods may rely on improper assumptions to initialize the missing values, whereas the unreliability of such initialization might lead to inferior performance. To address these problems, a novel semi-supervised conditional normalizing flow (SSCFlow) is proposed in this paper. SSCFlow explicitly utilizes the label information to facilitate the imputation and classification simultaneously by estimating the conditional distribution of incomplete instances with a novel semi-supervised normalizing flow. Moreover, SSCFlow treats the initialized missing values as corrupted initial imputation and iteratively reconstructs their latent representations with an overcomplete denoising autoencoder to approximate their true conditional distribution. Experiments on real-world datasets demonstrate the robustness and effectiveness of the proposed algorithm.

Motivation & Objective

• Address the limitation of traditional two-step imputation-classification pipelines that ignore label information during imputation.
• Overcome the poor performance caused by unreliable initialization of missing values in existing methods.
• Simultaneously improve both imputation and classification by jointly optimizing conditional distribution estimation.
• Enable effective learning from incomplete data with both labeled and unlabeled instances by incorporating label priors into the generative process.
• Develop a robust framework that treats missing values as corrupted features and reconstructs them using semi-supervised latent space modeling.

Proposed method

• Proposes a semi-supervised conditional normalizing flow (SSCFlow) that estimates the conditional distribution of incomplete instances given their labels.
• Integrates observed labels as additional features in the normalizing flow to guide imputation and classification jointly.
• Treats unobserved labels as missing values and reconstructs them as part of the end-to-end training process.
• Uses an overcomplete denoising autoencoder to iteratively reconstruct corrupted full features and approximate the true conditional distribution of missing values.
• Employs invertible transformations in the normalizing flow to enable exact likelihood estimation and efficient sampling from the conditional distribution.
• Shares parameters between imputation and classification heads to enable feature and weight sharing, improving generalization and consistency.

(a) imputation with estimated data distribution

Experimental results

Research questions

• RQ1 Can joint imputation and classification using conditional distribution estimation outperform traditional two-step approaches that separate the tasks?
• RQ2 How does incorporating label information during imputation affect the quality of missing value estimation and downstream classification accuracy?
• RQ3 To what extent does iterative reconstruction of corrupted imputations improve performance compared to single-step imputation?
• RQ4 Does the use of a semi-supervised conditional normalizing flow lead to better decision boundary separation and interpretability in latent space?
• RQ5 How do different components of SSCFlow (e.g., IDM, IT, LS) contribute to overall performance, and which are most critical?

Key findings

• SSCFlow achieves the highest classification accuracy (ACC) on all tested datasets compared to MCFlow and other state-of-the-art methods, with a significant improvement on the HTRU2 dataset (0.9686 ± 0.5218 vs. 0.9602 ± 0.0020).
• On the breast cancer dataset, SSCFlow achieves 98.37% ± 0.0082 accuracy, substantially outperforming the second-best method (93.85% ± 0.0135).
• The latent space representations generated by SSCFlow show clearer class separability and more natural clustering boundaries, as visualized via MDS, indicating improved interpretability.
• Ablation studies confirm that all components of SSCFlow (IDM, IT, LS) contribute to performance, with SSCFlow-LS performing worst in imputation, showing that label influence must affect the generative process, not just the loss.
• XGBoost, which directly classifies incomplete data without imputation, performs the worst on most datasets, supporting the claim that ignoring missing value distribution degrades performance.
• The model demonstrates robustness across diverse real-world datasets, including banknote, sensorless, and wifi localization, with consistent improvements in both imputation and classification.

(b) imputation with estimated conditional distribution