[Paper Review] The Statistical Performance of Matching-Adjusted Indirect Comparisons

David Cheng, Rajeev Ayyagari+1 more|arXiv (Cornell University)|Oct 14, 2019

Statistical Methods and Inference36 references4 citations

TL;DR

This paper rigorously evaluates the statistical performance of matching-adjusted indirect comparison (MAIC), a method that uses individual patient data (IPD) from one study and aggregate data (AGD) from another to adjust for baseline covariate imbalances in indirect treatment comparisons. It establishes identification assumptions, derives large-sample properties, and compares standard error estimators through simulations, demonstrating MAIC's robustness under correct model specification and highlighting the importance of appropriate variance estimation for valid inference in health technology assessments.

ABSTRACT

Indirect comparisons of treatment-specific outcomes across separate studies often inform decision-making in the absence of head-to-head randomized comparisons. Differences in baseline characteristics between study populations may introduce confounding bias in such comparisons. Matching-adjusted indirect comparison (MAIC) (Signorovitch et al., 2010) has been used to adjust for differences in observed baseline covariates when the individual patient-level data (IPD) are available for only one study and aggregate data (AGD) are available for the other study. The approach weights outcomes from the IPD using estimates of trial selection odds that balance baseline covariates between the IPD and AGD. With the increasing use of MAIC, there is a need for formal assessments of its statistical properties. In this paper we formulate identification assumptions for causal estimands that justify MAIC estimators. We then examine large sample properties and evaluate strategies for estimating standard errors without the full IPD from both studies. The finite-sample bias of MAIC and the performance of confidence intervals based on different standard error estimators are evaluated through simulations. The method is illustrated through an example comparing placebo arm and natural history outcomes in Duchenne muscular dystrophy.

Motivation & Objective

• To formally assess the statistical properties of matching-adjusted indirect comparison (MAIC) in the context of indirect treatment comparisons using mixed IPD and AGD.
• To identify the causal identification assumptions that justify the use of MAIC estimators when baseline covariates differ between study populations.
• To evaluate the finite-sample bias and coverage properties of confidence intervals derived from different standard error estimators in MAIC, especially when full IPD is unavailable from both studies.
• To provide methodological guidance for researchers and decision-makers on the reliable application of MAIC in health technology assessments and regulatory evaluations.

Proposed method

• MAIC uses inverse probability weighting based on estimated trial selection odds to balance baseline covariates between individual patient-level data (IPD) and aggregate data (AGD) populations.
• The method models the propensity of patients in the IPD study to be selected into the target population using covariates, adjusting outcome estimates accordingly.
• Key components include the estimation of weights via exponential family models (e.g., logistic regression) to balance covariates between the IPD and AGD groups.
• The paper derives large-sample variance estimators for MAIC using influence functions and evaluates alternative standard error estimators under various model specifications.
• It employs simulation studies to assess finite-sample bias and coverage of confidence intervals under correct and misspecified models.
• The method is illustrated using a real-world example comparing placebo and natural history outcomes in Duchenne muscular dystrophy.

Experimental results

Research questions

• RQ1 Under what identification assumptions is the MAIC estimator valid for estimating causal treatment effects in indirect comparisons?
• RQ2 How does finite-sample bias in MAIC vary under different model specifications and sample sizes?
• RQ3 Which standard error estimator provides the most accurate confidence interval coverage in MAIC when full IPD is not available from both studies?
• RQ4 How does model misspecification affect the performance of MAIC estimators in terms of bias and precision?
• RQ5 What are the large-sample properties of the MAIC estimator under correct model assumptions?

Key findings

• MAIC estimators are asymptotically unbiased under correct model specification and the assumption that the outcome model is correctly specified.
• Finite-sample bias in MAIC is generally small when the model for the selection odds is correctly specified, but can increase under model misspecification.
• The delta method-based standard error estimator provides the most accurate confidence interval coverage in large samples when the model is correctly specified.
• Alternative standard error estimators, such as the sandwich estimator, show improved performance over naive estimators in finite samples, particularly when covariate balance is imperfect.
• The simulation results demonstrate that MAIC maintains good statistical properties when the key identifying assumptions—especially conditional exchangeability and positivity—are met.
• In the Duchenne muscular dystrophy case study, MAIC successfully adjusted for baseline differences and provided a plausible indirect comparison between placebo and natural history outcomes.