[Paper Review] Parameter Estimation from Censored Samples using the Expectation-Maximization Algorithm

Chanseok Park, Seong Beom Lee|arXiv (Cornell University)|Mar 17, 2012

Statistical Distribution Estimation and Applications14 references3 citations

TL;DR

This paper proposes using the Expectation-Maximization (EM) and Monte Carlo EM (MCEM) algorithms for maximum likelihood estimation from right-censored data, particularly for normal, Laplace, and Rayleigh distributions. The method effectively handles missing data due to censoring by iteratively imputing censored values and updating parameters, yielding accurate and stable estimates that converge to MLEs regardless of initial values.

ABSTRACT

This paper deals with parameter estimation when the data are randomly right censored. The maximum likelihood estimates from censored samples are obtained by using the expectation-maximization (EM) and Monte Carlo EM (MCEM) algorithms. We introduce the concept of the EM and MCEM algorithms and develop parameter estimation methods for a variety of distributions such as normal, Laplace and Rayleigh distributions. These proposed methods are illustrated with three examples.

Motivation & Objective

• Address the challenge of parameter estimation in censored lifetime data, especially when standard numerical optimization fails due to sensitivity to initial values.
• Overcome limitations of existing methods like best linear unbiased estimates (BLUE) and approximate MLEs, which are restricted to specific censoring types and do not converge to true MLEs.
• Develop a generalizable framework using EM and MCEM algorithms applicable to Type-I (including random) and Type-II censoring for multiple distributions.
• Demonstrate robustness and convergence of the proposed algorithms across different starting values and distributional assumptions.
• Provide a computationally feasible and accurate alternative to direct likelihood maximization, especially for complex or high-dimensional models.

Proposed method

• Formulate the observed-data likelihood for right-censored samples using the (w_i, δ_i) representation, where w_i = min(x_i, R_i) and δ_i indicates whether failure occurred before censoring.
• Apply the EM algorithm by treating censored observations as missing data, iteratively computing the expected complete-data log-likelihood and maximizing it to update parameter estimates.
• For distributions where the E-step involves intractable integrals (e.g., normal, Laplace), use the MCEM variant with Monte Carlo sampling to approximate the expectation step.
• Implement simulation-based E-step via inverse transform sampling for truncated normal and Laplace distributions, enabling generation of censored values under current parameter estimates.
• Use iterative updates: for normal distribution, update mean and variance via conditional expectations; for Laplace, update location and scale using empirical moments; for Rayleigh, update scale parameter via MCEM with importance sampling.
• Ensure convergence by monitoring log-likelihood and parameter stability across iterations, with convergence achieved in 10–15 iterations for EM and 5–10 for MCEM.

Experimental results

Research questions

• RQ1 Can the EM and MCEM algorithms provide consistent and accurate maximum likelihood estimates for censored data when direct optimization fails due to poor starting values or flat likelihoods?
• RQ2 How does the performance of EM/MCEM compare to existing methods like BLUE or approximate MLE in terms of convergence and accuracy for Type-I and Type-II censored samples?
• RQ3 To what extent is the MCEM algorithm robust to the choice of initial parameter values across different distributions (normal, Laplace, Rayleigh)?
• RQ4 Can the MCEM algorithm handle complex likelihoods with intractable integrals in the E-step by using Monte Carlo approximation?
• RQ5 What is the convergence behavior and computational efficiency of EM and MCEM for censored data from various parametric distributions?

Key findings

• The EM and MCEM algorithms successfully estimate parameters for censored normal, Laplace, and Rayleigh distributions with high accuracy, converging to MLEs regardless of initial values.
• For the censored normal sample (N=10, r=7), the MLE via EM/MCEM yielded μ̂ = 1.742 and σ̂ = 0.079, correcting an earlier reported MLE error in Gupta (1952).
• In the Laplace example (N=20, r=18), MCEM produced μ̂ = 49.766 and σ̂ = 4.688, which are closer to the true MLE than the BLUE (μ̂ = 49.561, σ̂ = 4.813).
• For the Rayleigh distribution (N=20, r=15), MCEM converged to β̂ = 6.132–6.133 with different starting values, closely matching the MLE of 6.134.
• The MCEM algorithm demonstrated fast convergence (5–15 iterations) and insensitivity to initial values, with results stable to three decimal places.
• The simulation-based E-step using inverse transform sampling enabled accurate approximation of censored data imputations, validating the MCEM approach for non-exponential family distributions.