[Paper Review] Target estimands for population-adjusted indirect comparisons

Antonio Remiro‐Azócar|arXiv (Cornell University)|Dec 15, 2021

Advanced Causal Inference Techniques4 citations

TL;DR

This paper argues that marginal estimands—representing population-level treatment effects—are essential for health technology assessment (HTA) reimbursement decisions, advocating for multilevel network meta-regression (ML-NMR) over pairwise methods like MAIC or STC. ML-NMR enables flexible targeting of marginal effects in any desired population using real-world data, overcoming limitations of existing methods that are restricted to comparator study-specific populations.

ABSTRACT

Disagreement remains on what the target estimand should be for population-adjusted indirect treatment comparisons. This debate is of central importance for policy-makers and applied practitioners in health technology assessment. Misunderstandings are based on properties inherent to estimators, not estimands, and on generalizing conclusions based on linear regression to non-linear models. Estimators of marginal estimands need not be unadjusted and may be covariate-adjusted. The population-level interpretation of conditional estimates follows from collapsibility and does not necessarily hold for the underlying conditional estimands. For non-collapsible effect measures, neither conditional estimates nor estimands have a population-level interpretation. Estimators of marginal effects tend to be more precise and efficient than estimators of conditional effects where the measure of effect is non-collapsible. In any case, such comparisons are inconsequential for estimators targeting distinct estimands. Statistical efficiency should not drive the choice of the estimand. On the other hand, the estimand, selected on the basis of relevance to decision-making, should drive the choice of the most efficient estimator. Health technology assessment agencies make reimbursement decisions at the population level. Therefore, marginal estimands are required. Current pairwise population adjustment methods such as matching-adjusted indirect comparison are restricted to target marginal estimands that are specific to the comparator study sample. These may not be relevant for decision-making. Multilevel network meta-regression (ML-NMR) can potentially target marginal estimands in any population of interest. Such population could be characterized by decision-makers using increasingly available "real-world" data sources. Therefore, ML-NMR presents new directions and abundant opportunities for evidence synthesis.

Motivation & Objective

• To resolve ongoing debate on whether marginal or conditional estimands should guide population-adjusted indirect comparisons in health technology assessment (HTA).
• To clarify that estimands, not estimators, should drive methodological choice, with relevance to decision-making as the primary criterion.
• To demonstrate that current pairwise methods like MAIC and STC are limited in targeting generalizable population-level estimands.
• To highlight the potential of multilevel network meta-regression (ML-NMR) to estimate marginal treatment effects in any target population using real-world data.
• To support the use of outcome modeling-based methods like ML-NMR for improved precision and flexibility in evidence synthesis.

Proposed method

• Proposes that marginal estimands—representing average treatment effects across a target population—should be the primary inferential goal in HTA.
• Argues that estimators of marginal effects can be covariate-adjusted and need not be unadjusted, countering misconceptions about weighting methods.
• Emphasizes that collapsibility does not generally hold for non-collapsible effect measures (e.g., risk ratios), so conditional estimates do not inherently have population-level interpretations.
• Introduces multilevel network meta-regression (ML-NMR) as a method capable of estimating both conditional and marginal effects, with marginal effects obtainable via standardization of conditional estimates.
• Uses real-world data sources (e.g., electronic health records, large observational databases) to define external target populations for inference.
• Highlights that ML-NMR extends beyond pairwise comparisons to handle larger treatment networks, unlike traditional MAIC or STC.

Experimental results

Research questions

• RQ1 What is the appropriate target estimand for population-adjusted indirect comparisons in health technology assessment?
• RQ2 Why do current pairwise population adjustment methods like MAIC and STC fail to support generalizable population-level inference?
• RQ3 Can multilevel network meta-regression (ML-NMR) target marginal treatment effects in arbitrary populations of interest?
• RQ4 How do marginal and conditional estimands differ in interpretation and validity, especially under non-collapsible effect measures?
• RQ5 What are the implications of using estimators that are efficient but target different estimands than the one relevant to decision-making?

Key findings

• Marginal estimands are required for HTA reimbursement decisions because agencies make population-level decisions.
• Current pairwise methods such as MAIC and STC are restricted to targeting marginal effects specific to the comparator study sample, which may not be representative of the decision-making population.
• ML-NMR can target marginal estimands in any population of interest, enabling inference that is externally valid and aligned with decision-making needs.
• Outcome modeling-based methods like ML-NMR are generally more precise and efficient than weighting-based methods (e.g., MAIC) for estimating marginal effects, especially when overlap is poor.
• Conditional estimates from non-collapsible measures (e.g., risk ratios) do not have a valid population-level interpretation, even if they are collapsible in expectation.
• ML-NMR supports both conditional and marginal inference and can standardize conditional estimates into marginal effects, offering greater flexibility than pairwise methods.