[Paper Review] Quality control, data cleaning, imputation

Dawei Liu, Hanne Oberman+3 more|arXiv (Cornell University)|Oct 29, 2021

Statistical Methods and Inference101 references4 citations

TL;DR

This paper provides a comprehensive guide to quality control, data cleaning, and imputation in real-world data (RWD), emphasizing methods to handle missing data while minimizing bias. It evaluates statistical and machine learning imputation techniques, highlights challenges like informative missingness and repeated observations, and presents alternatives to imputation, offering researchers practical, evidence-based strategies for robust data analysis in RWD studies.

ABSTRACT

This chapter addresses important steps during the quality assurance and control of RWD, with particular emphasis on the identification and handling of missing values. A gentle introduction is provided on common statistical and machine learning methods for imputation. We discuss the main strengths and weaknesses of each method, and compare their performance in a literature review. We motivate why the imputation of RWD may require additional efforts to avoid bias, and highlight recent advances that account for informative missingness and repeated observations. Finally, we introduce alternative methods to address incomplete data without the need for imputation.

Motivation & Objective

• To address critical challenges in quality assurance and control of real-world data (RWD), particularly around missing data.
• To evaluate and compare statistical and machine learning methods for imputing missing values in RWD.
• To highlight the risks of bias in RWD imputation due to informative missingness and repeated observations.
• To present alternative approaches to imputation that avoid assumptions about missing data mechanisms.
• To support researchers in selecting appropriate, robust methods for data preprocessing in RWD studies.

Proposed method

• The paper introduces a structured framework for quality control and data cleaning in RWD, focusing on identifying and addressing data quality issues.
• It reviews common imputation techniques, including multiple imputation by chained equations (MICE), regression imputation, and machine learning-based methods such as random forests and k-nearest neighbors.
• The authors assess each method’s strengths and weaknesses, particularly regarding assumptions, scalability, and performance under different missing data mechanisms.
• The paper emphasizes the importance of modeling informative missingness—where missingness is related to unobserved values—using advanced techniques like pattern-mixture models and selection models.
• It discusses methods to handle repeated or clustered observations in RWD, such as mixed-effects models and marginal models, to preserve data structure during imputation.
• The paper also presents non-imputation alternatives, such as complete case analysis with sensitivity analysis and inverse probability weighting, to avoid potential biases from imputation.

Experimental results

Research questions

• RQ1 Which imputation methods are most effective for real-world data with complex missing data mechanisms?
• RQ2 How does informative missingness affect the validity of imputation results in RWD, and how can it be properly modeled?
• RQ3 What are the performance trade-offs between traditional statistical imputation and modern machine learning-based imputation in RWD?
• RQ4 How can repeated or clustered observations in RWD be appropriately handled during data cleaning and imputation?
• RQ5 In what scenarios are non-imputation approaches preferable to imputation for incomplete RWD?

Key findings

• Multiple imputation by chained equations (MICE) and machine learning-based imputation (e.g., random forests) perform well under ignorable missingness but may introduce bias under informative missingness if not properly modeled.
• Methods that explicitly model the missing data mechanism—such as pattern-mixture models—show improved robustness in the presence of informative missingness.
• Imputation in RWD requires careful consideration of data structure, including repeated measurements and clustering, to avoid violating assumptions of independence.
• The use of inverse probability weighting and complete case analysis with sensitivity analysis can be viable alternatives to imputation when imputation assumptions are questionable.
• Machine learning imputation methods, while flexible, require careful validation and are sensitive to hyperparameter tuning and data sparsity.
• The paper concludes that no single imputation method universally outperforms others; method choice must be guided by data structure, missing data mechanism, and study objectives.