[Paper Review] A Causal Roadmap for Generating High-Quality Real-World Evidence

Lauren Eyler Dang, Susan Gruber+24 more|arXiv (Cornell University)|May 11, 2023

Health Systems, Economic Evaluations, Quality of Life10 citations

TL;DR

The paper proposes an explicit, iterative Causal Roadmap to pre-specify analytic study designs, transparently evaluate causal assumptions, and compare design/analysis choices to generate high-quality real-world evidence (RWE) for regulatory and clinical use.

ABSTRACT

Increasing emphasis on the use of real-world evidence (RWE) to support clinical policy and regulatory decision-making has led to a proliferation of guidance, advice, and frameworks from regulatory agencies, academia, professional societies, and industry. A broad spectrum of studies use real-world data (RWD) to produce RWE, ranging from randomized controlled trials with outcomes assessed using RWD to fully observational studies. Yet many RWE study proposals lack sufficient detail to evaluate adequacy, and many analyses of RWD suffer from implausible assumptions, other methodological flaws, or inappropriate interpretations. The Causal Roadmap is an explicit, itemized, iterative process that guides investigators to pre-specify analytic study designs; it addresses a wide range of guidance within a single framework. By requiring transparent evaluation of causal assumptions and facilitating objective comparisons of design and analysis choices based on pre-specified criteria, the Roadmap can help investigators to evaluate the quality of evidence that a given study is likely to produce, specify a study to generate high-quality RWE, and communicate effectively with regulatory agencies and other stakeholders. This paper aims to disseminate and extend the Causal Roadmap framework for use by clinical and translational researchers, with companion papers demonstrating application of the Causal Roadmap for specific use cases.

Motivation & Objective

• Motivate the need for high-quality real-world evidence (RWE) in clinical policy and regulatory decision-making.
• Provide a unified, explicit framework (the Causal Roadmap) that guides pre-specification and evaluation of RWE study designs.
• Enable transparent assessment of causal assumptions and objective comparison across design/analysis options.
• Facilitate communication with regulators and stakeholders by standardizing the RWE generation process.

Proposed method

• Present an explicit, itemized, iterative process called the Causal Roadmap.
• Require pre-specification of analytic study designs within a single framework.
• Promote transparent evaluation of causal assumptions and criteria for design/analysis choices.
• Support objective comparisons of competing designs/analyses based on pre-specified criteria.
• Aim to disseminate and extend the framework for use in clinical and translational research, with companion use-case papers.

Experimental results

Research questions

• RQ1 How can a structured, iterative roadmap improve pre-specification and transparency in RWE studies?
• RQ2 What causal assumptions must be evaluated to ensure high-quality RWE, and how can they be transparently assessed?
• RQ3 How can the Causal Roadmap facilitate objective comparisons of study designs and analyses for RWE?
• RQ4 In what ways can the framework improve communication with regulatory agencies and stakeholders?

Key findings

• The Causal Roadmap provides an explicit, itemized process to guide RWE study design and analysis.
• The framework emphasizes transparent evaluation of causal assumptions.
• It facilitates objective, pre-specified comparisons of design and analysis choices.
• The Roadmap aims to improve communication with regulatory bodies and other stakeholders.
• The paper positions the Roadmap as extensible, with companion papers demonstrating practical use cases.