[Paper Review] General Semiparametric Shared Frailty Model Estimation and Simulation with frailtySurv

John V. Monaco, Malka Gorfine+1 more|PubMed|Feb 21, 2017

Frailty in Older Adults3 citations

TL;DR

This paper introduces the R package frailtySurv, which enables semiparametric estimation and simulation of shared frailty models with flexible frailty distributions—including gamma, log-normal, inverse Gaussian, and power variance function. It employs pseudo-full-likelihood estimation to produce consistent, asymptotically normal parameter estimates, enabling valid inference via Wald tests and confidence intervals, with demonstrated accuracy and robustness across simulations and real-world datasets from ophthalmology and hard drive failure studies.

ABSTRACT

The R package <b>frailtySurv</b> for simulating and fitting semi-parametric shared frailty models is introduced. Package <b>frailtySurv</b> implements semi-parametric consistent estimators for a variety of frailty distributions, including gamma, log-normal, inverse Gaussian and power variance function, and provides consistent estimators of the standard errors of the parameters' estimators. The parameters' estimators are asymptotically normally distributed, and therefore statistical inference based on the results of this package, such as hypothesis testing and confidence intervals, can be performed using the normal distribution. Extensive simulations demonstrate the flexibility and correct implementation of the estimator. Two case studies performed with publicly available datasets demonstrate applicability of the package. In the Diabetic Retinopathy Study, the onset of blindness is clustered by patient, and in a large hard drive failure dataset, failure times are thought to be clustered by the hard drive manufacturer and model.

Motivation & Objective

• To develop a flexible, general-purpose R package for fitting semiparametric shared frailty models with multiple frailty distributions.
• To provide consistent, asymptotically normal estimators of regression and frailty parameters, enabling valid statistical inference via normal-based tests and confidence intervals.
• To support simulation of clustered survival data under various frailty structures for methodological validation and study design.
• To address limitations in existing R packages, particularly the lack of consistent estimators in penalized partial likelihood approaches.
• To enable application of shared frailty models to real-world clustered survival data, such as patient-level eye outcomes and hard drive failure times.

Proposed method

• Employs pseudo-full-likelihood (PFL) estimation to jointly maximize the likelihood of regression coefficients and frailty distribution parameters, ensuring consistency and asymptotic normality.
• Uses numerical integration for frailty distributions without analytic Laplace transforms (e.g., log-normal, inverse Gaussian), with adaptive quadrature for accuracy.
• Implements a double-nested optimization loop: outer loop for parameter estimation and inner loop for numerical integration of frailty effects.
• Applies Rcpp to compile performance-critical C++ code, caches expensive computations, and supports parallelization to improve runtime efficiency.
• Estimates standard errors via observed information matrix, enabling Wald-type inference with normal distributional approximations.
• Supports multiple frailty distributions: gamma, log-normal, inverse Gaussian, and power variance function, with flexible parameterization.

Experimental results

Research questions

• RQ1 Can a general-purpose R package be developed to estimate semiparametric shared frailty models with multiple frailty distributions and consistent standard errors?
• RQ2 Does the pseudo-full-likelihood approach produce estimators that are consistent and asymptotically normal, even when penalized partial likelihood is inconsistent?
• RQ3 How does the performance of the proposed method compare to existing approaches in terms of accuracy, speed, and robustness across diverse data structures?
• RQ4 To what extent does misspecification of the frailty distribution affect the bias and efficiency of regression coefficient estimates?
• RQ5 Can the package reliably simulate clustered survival data with known frailty structures for method validation and study planning?

Key findings

• The frailtySurv package successfully implements consistent, asymptotically normal estimators for regression and frailty parameters across gamma, log-normal, inverse Gaussian, and power variance function frailty distributions.
• Simulations confirm that the pseudo-full-likelihood method produces accurate estimates with correct coverage for 95% confidence intervals, even under complex frailty structures.
• The method achieves valid statistical inference via normal-based Wald tests, with p-values correctly reflecting significance levels in simulation studies.
• Runtime complexity is O(n) for data generation and O(n²) for model fitting, with performance tunable via convergence criteria for numerical integration and outer loop.
• Relaxing convergence thresholds for numerical integration reduces runtime with minimal loss in accuracy, enabling a speed-accuracy trade-off.
• In real data applications, the model identified significant predictors of hard drive failure (pending sector count, reallocated sector count), with hazard ratios of 11 and 2, respectively, underlining its practical utility.