[Paper Review] Hidden in Plain Sight: How Non-Collapsibility Biases Treatment Effects in (Network) Meta-Analysis

Harlan Campbell, Jansen, Jeroen P.|arXiv (Cornell University)|Feb 28, 2026

Meta-analysis and systematic reviews0 citations

TL;DR

The paper shows that non-collapsibility of the odds ratio bias(es)) pooled estimates in (network) meta-analysis when studies mix populations with different baseline risks, and proposes a bookend approach to mitigate bias. It also provides guidance for practitioners to assess and address potential non-collapsibility bias.

ABSTRACT

Network meta-analysis (NMA) is widely used to compare multiple interventions simultaneously by synthesizing direct and indirect evidence. The general fixed or random effects contrast-based NMA model can be applied to different outcomes and data structures by opting for either an arm-based or contrast-based likelihood depending on the data available. Depending on the outcome and link-function, we estimate either collapsible or non-collapsible effect measures. Using an illustrative example involving binary outcomes and the non-collapsible odds ratio, we demonstrate that the standard NMA model produces estimates for non-collapsible effect measures that are biased toward the null when studies in the evidence base enroll heterogeneous populations (mixtures of distinct risk groups) that vary across studies. Importantly, this also holds when there are no differences in effect-modifiers across studies; the standard assumption of a common treatment effect when there are no differences in the distribution of effect-modifiers across studies is not appropriate when studies have different baseline risks. As a potential solution, we propose a ``bookend'' approach that explicitly models mixed-population studies as weighted combinations of two homogeneous subpopulations identified from studies with extreme baseline risks and provide guidance for practitioners to determine if bias due to non-collapsibility may be a concern.

Motivation & Objective

• Explain how non-collapsibility attenuates odds-ratio-based effects in meta-analysis with heterogeneous baseline risks.
• Demonstrate bias toward the null in standard fixed-effect NMA under mixed populations.
• Introduce the bookend modeling approach to account for mixed populations.
• Provide practical guidance to assess baseline risk variation and apply the bookend sensitivity analysis.

Proposed method

• Describe the standard contrast-based NMA with arm-based likelihood and its interpretation as a conditional log-odds ratio.
• Show non-collapsibility attenuation when mixing study populations with different baseline risks.
• Present the bookend model that treats mixed studies as mixtures of two homogeneous subpopulations identified from extreme baseline risks.
• Provide Bayesian implementation via JAGS for both standard and bookend models.
• Compare performance through a simple binary outcome example and simulate data.
• Offer practical guidance for detecting potential non-collapsibility bias in meta-analytic practice.

Figure 1 : Results from hypothetical lung disease example. Point estimates and 95% confidence intervals from individual studies (gray) are shown alongside posterior means and 95% credible intervals from the standard fixed-effect model (blue, $\hat{d}=-0.458$ ) and bookend model (red, $\hat{d}=-0.492

Experimental results

Research questions

• RQ1 Does non-collapsibility of the odds ratio induce bias in (network) meta-analysis when study populations have heterogeneous baseline risks?
• RQ2 Can a bookend modeling approach reduce or adjust for this bias by characterizing mixed-population studies as mixtures of two extreme baseline-risk subpopulations?
• RQ3 Under what conditions is non-collapsibility bias most pronounced, and how should practitioners assess and address it in practice?

Key findings

• Non-collapsibility can bias standard NMA estimates toward the null when studies enroll mixed populations with different baseline risks.
• The bias occurs even without differences in distribution of effect modifiers across studies and when there is no true difference in treatment effects across subpopulations.
• A bookend model, which treats mixed studies as mixtures of two homogeneous subpopulations defined by extreme baseline risks, can yield less biased estimates under certain assumptions.
• In a simple lung-disease example, the standard model gave d̂ = -0.458 (95% CrI: -0.58 to -0.34) while the bookend model gave d̂ = -0.492 (95% CrI: -0.62 to -0.37), closer to the true -0.50.
• Bias magnitude is influenced by the degree of baseline risk heterogeneity and the presence of mixed-population studies; the effect is generally small relative to sampling error but can be meaningful under substantial heterogeneity.
• The authors provide practical steps to assess baseline risk variation and suggest the bookend approach as a sensitivity analysis when extreme baseline-risk studies are observed.