[Paper Review] Mask mandates and COVID-19: A Re-analysis of the Boston school mask study

Tracy Beth Høeg, Ambarish Chandra+3 more|arXiv (Cornell University)|Jul 22, 2023

COVID-19 and healthcare impactsMedicine3 citations

TL;DR

This re-analysis challenges the claim that school mask mandates reduced COVID-19 cases in Massachusetts during the 2021–2022 academic year. Using state-level data from the CDC and Massachusetts DOE, the authors find no significant reduction in case rates in districts maintaining mask mandates compared to those that lifted them, with case rates actually increasing slightly in mandated districts; prior community immunity was found to explain up to 66% of case rate variation.

ABSTRACT

Background: A recent epidemiological analysis of staggered policy implementation reported a 29.4% reduction in COVID-19 cases by maintaining school mask mandates in the greater Boston area during the first half of 2022. The robustness of their results and the appropriateness of methodology are explored. Methods: Using data from the Massachusetts Department of Elementary and Secondary Education and the Centers for Disease Control and Prevention, we re-analyze differences in COVID-19 incidence in school districts that did and did not lift mask mandates using the same districts as the original study and expanded the analysis to the entire state of Massachusetts. We present changes in case rates and differences in prior immunity in areas with different mask lifting policies. Results: The Boston and Chelsea districts, which maintained mask mandates, were outliers in terms of size, demographics, and testing. We failed to find a notable change in student cases in mask mandate districts compared with the 70 districts in the original study (-0.08/1000; p=0.98) and found a slight increase compared with a statewide control group +3.63/1000 (p=0.291). Results were similar for students and staff combined. Districts that dropped mask mandates first experienced the largest decreases in cases (22% drop vs 12% in the masked districts). There was a moderate to strong relationship (R2 = 0.35-0.66; p-values <0.001) between prior community infection burden and district case rates in Spring 2022, with prior immunity alone explaining as much as two-thirds of the variation in case rates in Spring of 2022. Conclusions: We fail to find any consistent notable negative relationship between school mask mandates and infection rates in the Greater Boston Area or state of Massachusetts during the 2021-2022 academic year.

Motivation & Objective

• To reassess the validity and methodology of a prior study claiming a 29.4% reduction in COVID-19 cases due to school mask mandates in Boston during early 2022.
• To evaluate whether mask mandates in Massachusetts school districts were associated with lower infection rates during the 2021–2022 academic year.
• To examine the role of prior community immunity in explaining variations in school district case rates during spring 2022.
• To compare case trends between districts that maintained mask mandates and those that lifted them, using both local and statewide control groups.
• To determine whether demographic, size, or testing differences in Boston and Chelsea districts influenced the original study’s conclusions.

Proposed method

• Re-analyzed school-level COVID-19 case data from the Massachusetts Department of Elementary and Secondary Education and the CDC for the 2021–2022 academic year.
• Used the same 70 districts as the original study but expanded analysis to include all school districts in Massachusetts.
• Applied difference-in-differences and time-series comparisons between districts that maintained mask mandates and those that lifted them.
• Quantified the relationship between prior community infection burden (as a proxy for immunity) and subsequent case rates in schools using R-squared and p-values.
• Conducted subgroup analyses for students and staff combined, and assessed statistical significance using p-values.
• Used supplementary figures and tables to validate results and assess model robustness.

Experimental results

Research questions

• RQ1 Did school mask mandates in Massachusetts lead to a significant reduction in student and staff COVID-19 case rates during spring 2022?
• RQ2 How did case rates in districts that maintained mask mandates compare to those that lifted mandates, both locally and statewide?
• RQ3 To what extent did prior community immunity explain variations in school district case rates during spring 2022?
• RQ4 Were Boston and Chelsea districts, which maintained mandates, representative of broader Massachusetts trends, or were they outliers?
• RQ5 What was the statistical significance of the observed differences in case rates between mandated and non-mandated districts?

Key findings

• No significant reduction in student case rates was observed in districts that maintained mask mandates compared to the original study’s 70 districts (change: -0.08/1000; p=0.98).
• A slight increase in case rates was observed in mandated districts compared to a statewide control group (+3.63/1000; p=0.291).
• Districts that lifted mask mandates first experienced a 22% drop in cases, compared to a 12% drop in districts that maintained mandates.
• Prior community infection burden showed a moderate to strong relationship with school case rates (R² = 0.35–0.66; p < 0.001).
• Prior immunity alone explained as much as 66% of the variation in case rates across districts in spring 2022.
• Boston and Chelsea districts were identified as statistical outliers in size, demographics, and testing frequency, suggesting potential bias in the original study’s findings.