[Paper Review] A Versatile Estimation Procedure Without Estimating the Nonignorable Missingness Mechanism

Jiwei Zhao, Yanyuan Ma|arXiv (Cornell University)|Jul 8, 2019

Advanced Causal Inference Techniques27 references4 citations

TL;DR

This paper proposes a semiparametric estimation procedure for regression models with nonignorable missing data that bypasses modeling the missingness mechanism entirely. By treating the missingness mechanism as a nuisance parameter and using empirical and nonparametric techniques to estimate integrals involving the covariate distribution, the method achieves asymptotic normality and robustness without requiring correct specification of the missingness model, offering a versatile and practical solution for missing data under nonignorable mechanisms.

ABSTRACT

We consider the estimation problem in a regression setting where the outcome variable is subject to nonignorable missingness and identifiability is ensured by the shadow variable approach. We propose a versatile estimation procedure where modeling of missingness mechanism is completely bypassed. We show that our estimator is easy to implement and we derive the asymptotic theory of the proposed estimator. We also investigate some alternative estimators under different scenarios. Comprehensive simulation studies are conducted to demonstrate the finite sample performance of the method. We apply the estimator to a children's mental health study to illustrate its usefulness.

Motivation & Objective

• To address the challenge of nonignorable missing data in regression models where the missingness depends on unobserved outcomes.
• To develop a method that avoids modeling or estimating the missingness mechanism, which is often difficult and prone to misspecification.
• To ensure identifiability through the shadow variable approach while maintaining estimation efficiency and robustness.
• To establish a theoretically grounded, asymptotically normal estimator that is easy to implement in practice.

Proposed method

• The method treats the missingness mechanism as a nuisance parameter in a semiparametric model and uses semiparametric theory to project it out.
• It estimates integrals involving the covariate density through empirical expectation when the integral is a marginal expectation.
• For conditional expectations, it employs nonparametric regression techniques such as kernel smoothing to estimate the required integrals.
• The estimator is constructed using efficient score functions derived under the shadow variable framework, ensuring asymptotic efficiency.
• The approach relies on the shadow variable to ensure model identifiability without requiring parametric assumptions on the missingness mechanism.
• Asymptotic theory is established using bilinear operators and semiparametric techniques, proving the estimator's asymptotic normality and efficiency.

Experimental results

Research questions

• RQ1 Can a robust and efficient estimator be developed for nonignorable missing data without modeling the missingness mechanism?
• RQ2 How does the performance of the proposed estimator compare to existing methods when the missingness mechanism is misspecified?
• RQ3 What is the asymptotic behavior of the estimator under correct and incorrect working models for the missingness mechanism?
• RQ4 Can the method maintain good finite-sample performance across diverse data-generating mechanisms and distributional assumptions?
• RQ5 How does the proposed method compare to oracle and parametric estimators in terms of bias, standard error, and coverage probability?

Key findings

• The proposed estimator achieves asymptotic normality and efficiency without requiring correct modeling of the missingness mechanism, as confirmed by theoretical derivation.
• In finite samples, the estimator shows low bias and good coverage probability (around 95%) across various scenarios, even when the working model for the missingness mechanism is misspecified.
• The method outperforms traditional parametric and EM-based approaches in terms of robustness when the missingness mechanism is incorrectly modeled.
• Empirical standard errors are well-estimated, with coverage probabilities close to nominal levels (e.g., 95%) in simulation studies.
• In the children’s mental health study, the proposed estimator yielded more precise and significant estimates for key predictors (e.g., father’s education) compared to complete-case and parametric EM methods.
• The estimator remains stable and efficient even under model misspecification of the conditional density of the shadow variable, demonstrating strong robustness.