[Paper Review] Guidelines for estimating causal effects in pragmatic randomized trials

Eleanor J. Murray, Sonja A. Swanson+1 more|arXiv (Cornell University)|Nov 14, 2019

Advanced Causal Inference Techniques73 references21 citations

TL;DR

This paper provides comprehensive methodological guidelines for estimating both intention-to-treat and per-protocol causal effects in pragmatic randomized trials, addressing challenges from non-adherence and real-world implementation. It proposes targeted statistical approaches to improve causal inference validity and decision-making relevance in diverse clinical settings.

ABSTRACT

Pragmatic randomized trials are designed to provide evidence for clinical decision-making rather than regulatory approval. Common features of these trials include the inclusion of heterogeneous or diverse patient populations in a wide range of care settings, the use of active treatment strategies as comparators, unblinded treatment assignment, and the study of long-term, clinically relevant outcomes. These features can greatly increase the usefulness of the trial results for patients, clinicians, and other stakeholders. However, these features also introduce an increased risk of non-adherence, which reduces the value of the intention-to-treat effect as a patient-centered measure of causal effect. In these settings, the per-protocol effect provides useful complementary information for decision making. Unfortunately, there is little guidance for valid estimation of the per-protocol effect. Here, we present our full guidelines for analyses of pragmatic trials that will result in more informative causal inferences for both the intention-to-treat effect and the per-protocol effect.

Motivation & Objective

• Address the growing need for reliable causal effect estimation in pragmatic randomized trials that reflect real-world clinical practice.
• Identify limitations of intention-to-treat analysis when non-adherence is high in pragmatic trials with diverse populations and unblinded designs.
• Provide methodological guidance for valid estimation of the per-protocol effect as a complementary measure to intention-to-treat.
• Enhance the utility of trial results for patients, clinicians, and health system stakeholders by improving causal inference in complex, real-world settings.
• Fill a critical gap in statistical guidance for analyzing pragmatic trials where traditional assumptions may not hold.

Proposed method

• Propose a two-stage approach: first estimate the intention-to-treat effect using standard randomization-based methods.
• Apply marginal structural models or inverse probability weighting to adjust for time-varying confounders in intention-to-treat analysis.
• Define the per-protocol effect as the causal effect among individuals who adhere to the assigned treatment protocol.
• Use structural nested models or g-computation to estimate the per-protocol effect under specific consistency and positivity assumptions.
• Introduce sensitivity analysis frameworks to assess the robustness of per-protocol estimates to unmeasured confounding.
• Emphasize the importance of pre-specifying analysis plans and clearly defining the target population and treatment protocol.

Experimental results

Research questions

• RQ1 How can intention-to-treat effects be validly estimated in pragmatic trials with high non-adherence and unblinded treatment assignment?
• RQ2 What are the key assumptions required to identify the per-protocol causal effect in pragmatic trial settings?
• RQ3 How can per-protocol effects be estimated without introducing selection bias due to non-adherence?
• RQ4 What statistical methods are most appropriate for adjusting for time-varying confounders in pragmatic trial analyses?
• RQ5 How can researchers ensure that per-protocol estimates are interpretable and useful for clinical decision-making?

Key findings

• The per-protocol effect provides valuable complementary information to the intention-to-treat effect in pragmatic trials where non-adherence is common.
• Proper estimation of the per-protocol effect requires strong assumptions about consistency, positivity, and the absence of unmeasured confounding.
• Marginal structural models and g-computation are effective tools for estimating both intention-to-treat and per-protocol effects in complex, real-world settings.
• Sensitivity analyses are essential to evaluate the robustness of per-protocol estimates to violations of unmeasured confounding assumptions.
• Pre-specification of analysis plans and clear definition of the treatment protocol are critical for valid and transparent causal inference.
• The guidelines enhance the interpretability and clinical relevance of pragmatic trial results by supporting more nuanced causal inference.