[Paper Review] Guidance note on best statistical practices for TOAR analyses

Kai‐Lan Chang, Martin G. Schultz+2 more|arXiv (Cornell University)|Apr 27, 2023

Risk and Safety Analysis47 citations

TL;DR

This guidance note prescribes best statistical practices for TOAR trend analyses, emphasizing quantile regression, uncertainty quantification, data preprocessing, and change point methods to ensure consistent reporting across TOAR publications.

ABSTRACT

The aim of this guidance note is to provide recommendations on best statistical practices and to ensure consistent communication of statistical analysis and associated uncertainty across TOAR publications. The scope includes approaches for reporting trends, a discussion of strengths and weaknesses of commonly used techniques, and calibrated language for the communication of uncertainty. The focus of this guidance note is placed on trend analysis, which is expected to be the main statistical topic of interest across many TOAR-II focus working groups, but some of the recommendations and principles provided below are also valid for other applications. Recommendations are highlighted and numbered from R1 to R9.

Motivation & Objective

• Define the scope and purpose of trend analysis for TOAR analyses, including how to quantify change and its uncertainty.
• Advise on when to use linear versus nonlinear trend methods and how to incorporate covariates and change points.
• Recommend a standardized statistical framework (quantile regression) for trend estimation and uncertainty communication.
• Provide data preparation guidelines (seasonal adjustment/deseasonalization) to ensure valid trend estimates.
• Outline how to assess and report uncertainty and reliability of trends.
• Promote calibrated language for communicating trend uncertainty across TOAR outputs.

Proposed method

• Recommend quantile regression (QR) as the standard trend analysis method for TOAR due to its ability to capture distributional changes and incorporate covariates.
• Compare linear trend techniques (GLS, Sen-Theil, QR) and discuss robustness, autocorrelation handling, and applicability to different data characteristics.
• Advise on default quantile reporting based on sample size, with cautions for extreme quantiles and suggestion to use GEV/threshold models for extremes.
• Describe data preparation steps including seasonal adjustments and deseasonalization to prevent inflated uncertainties.
• Explain how to quantify trend uncertainty using confidence intervals, standard errors, and methods that account for autocorrelation (e.g., moving block bootstrap, prewhitening).
• Advise on incorporating change point analysis (piecewise linear trends) to handle genuine shifts in trends and to validate data structure with visual and statistical checks.

Table 1: A brief comparison of statistical capabilities of three commonly applied techniques.

Experimental results

Research questions

• RQ1 What are the recommended statistical practices for reporting trends and their uncertainties in TOAR analyses?
• RQ2 How should linear and nonlinear trends be treated and communicated, and when should piecewise linear trends be used with change point analysis?
• RQ3 Why is quantile regression preferred for TOAR trend analyses, and what are its limitations for small samples or extreme quantiles?
• RQ4 How should data preparation (seasonality, deseasonalization) be conducted to ensure valid trend estimates?
• RQ5 How should uncertainty and reliability of trend estimates be calibrated and communicated across TOAR publications?
• RQ6 When and how should change point detection be applied to trend analyses, and how should results be interpreted?

Key findings

• Quantile regression is recommended as the standard method for TOAR trend analyses due to its ability to capture heterogeneous percentile trends and to enable covariate attribution and change point analysis.
• Linear trend methods (GLS, Sen-Theil) have complementary roles, but QR provides broader applicability, especially under heteroskedasticity and autocorrelation.
• Uncertainty in trend estimates should be quantified (e.g., 95% confidence intervals) and non-IID residuals should be accounted for using methods like block bootstrap or robust SEs; ignoring autocorrelation can lead to underestimation of uncertainty.
• Seasonal and diurnal cycles should be modeled or deseasonalized prior to QR to avoid inflated uncertainty and biased trend estimates.
• Change point analysis should be used to identify and interpret genuine shifts in trends, with caution about data quality and instrument changes; piecewise linear trends can provide interpretable attributions of trend changes.
• Calibrated, gradated language (avoiding “statistically significant”) should be used to report trend reliability, based on p-values or SNR.

Figure 1: A demonstration of the difference between a range of mean/median trend methods (upper panel) and percentile trends derived from QR (lower panel), based on surface ozone anomalies measured at Mace Head, Ireland. Loess (locally estimated scatterplot smoothing, gray curve) fit is added to sho