[Paper Review] Estimation methods for estimands using the treatment policy strategy; a simulation study based on the PIONEER 1 Trial

James Bell, Thomas Drury+7 more|arXiv (Cornell University)|Feb 20, 2024

Health Systems, Economic Evaluations, Quality of Life5 citations

TL;DR

The paper introduces likelihood-based estimation methods for the treatment policy strategy in intercurrent events, compares them to retrieved dropout, reference-based imputation, and simple models, using a PIONEER 1 trial simulation, and discusses variance inflation and bias trade-offs.

ABSTRACT

Estimands using the treatment policy strategy for addressing intercurrent events are common in Phase III clinical trials. One estimation approach for this strategy is retrieved dropout whereby observed data following an intercurrent event are used to multiply impute missing data. However, such methods have had issues with variance inflation and model fitting due to data sparsity. This paper introduces likelihood-based versions of these approaches, investigating and comparing their statistical properties to the existing retrieved dropout approaches, simpler analysis models and reference-based multiple imputation. We use a simulation based upon the data from the PIONEER 1 Phase III clinical trial in Type II diabetics to present complex and relevant estimation challenges. The likelihood-based methods display similar statistical properties to their multiple imputation equivalents, but all retrieved dropout approaches suffer from high variance. Retrieved dropout approaches appear less biased than reference-based approaches, resulting in a bias-variance trade-off, but we conclude that the large degree of variance inflation is often more problematic than the bias. Therefore, only the simpler retrieved dropout models appear appropriate as a primary analysis in a clinical trial, and only where it is believed most data following intercurrent events will be observed. The jump-to-reference approach may represent a more promising estimation approach for symptomatic treatments due to its relatively high power and ability to fit in the presence of much missing data, despite its strong assumptions and tendency towards conservative bias. More research is needed to further develop how to estimate the treatment effect for a treatment policy strategy.

Motivation & Objective

• Motivate evaluation of estimation approaches for the treatment policy strategy in phase III trials with intercurrent events.
• Develop likelihood-based counterparts to retrieved dropout methods.
• Compare statistical properties of likelihood-based methods with retrieved dropout, reference-based multiple imputation, and simple analysis models.
• Assess performance in a simulation based on PIONEER 1 Type II diabetes trial data.

Proposed method

• Propose likelihood-based versions of retrieved dropout approaches for the treatment policy estimand.
• Compare with existing retrieved dropout methods, simpler analysis models, and reference-based multiple imputation.
• Use a simulation study grounded in PIONEER 1 trial data to evaluate estimation properties.
• Assess variance inflation, bias, and power across approaches.
• Discuss practical implications for primary analysis in clinical trials with substantial missing post-intercurrent-event data.

Experimental results

Research questions

• RQ1 How do likelihood-based estimators for the treatment policy strategy perform in terms of variance and bias compared with retrieved dropout and reference-based imputation?
• RQ2 What are the trade-offs between bias and variance for retrieved dropout versus jump-to-reference approaches under substantial missingness?
• RQ3 Which estimation approach offers the best balance of power and robustness for symptomatic treatments under the treatment policy framework?
• RQ4 Can jump-to-reference provide a viable alternative under missing data but with its own assumptions and bias tendencies?

Key findings

• Likelihood-based retrieval approaches show similar statistical properties to their multiple imputation equivalents.
• Retrieved dropout methods exhibit high variance, often more problematic than bias.
• Retrieved dropout methods are less biased than reference-based approaches, indicating a bias-variance trade-off.
• The large variance inflation in some methods suggests they may not be suitable for primary analysis in many trials.
• Jump-to-reference may offer higher power and tolerance to missing data, but comes with strong assumptions and conservative bias.
• Overall, simpler retrieved dropout models are suggested as primary analyses where most data after intercurrent events are observed.