[Paper Review] Testing normality using the summary statistics with application to meta-analysis

Dehui Luo, Xiang Wan+2 more|arXiv (Cornell University)|Jan 29, 2018

Meta-analysis and systematic reviews43 references3 citations

TL;DR

This paper proposes three novel test statistics to assess normality (symmetry) in clinical data using only summary statistics—sample size, median, min/max, and interquartile range—before applying mean and standard deviation estimation methods in meta-analysis. The tests are designed to detect skewed data, improving the reliability of effect size calculations like Cohen’s d and Hedges’ g by excluding non-normal studies prior to transformation, thereby reducing estimation bias in evidence-based medicine.

ABSTRACT

As the most important tool to provide high-level evidence-based medicine, researchers can statistically summarize and combine data from multiple studies by conducting meta-analysis. In meta-analysis, mean differences are frequently used effect size measurements to deal with continuous data, such as the Cohen's d statistic and Hedges' g statistic values. To calculate the mean difference based effect sizes, the sample mean and standard deviation are two essential summary measures. However, many of the clinical reports tend not to directly record the sample mean and standard deviation. Instead, the sample size, median, minimum and maximum values and/or the first and third quartiles are reported. As a result, researchers have to transform the reported information to the sample mean and standard deviation for further compute the effect size. Since most of the popular transformation methods were developed upon the normality assumption of the underlying data, it is necessary to perform a pre-test before transforming the summary statistics. In this article, we had introduced test statistics for three popular scenarios in meta-analysis. We suggests medical researchers to perform a normality test of the selected studies before using them to conduct further analysis. Moreover, we applied three different case studies to demonstrate the usage of the newly proposed test statistics. The real data case studies indicate that the new test statistics are easy to apply in practice and by following the recommended path to conduct the meta-analysis, researchers can obtain more reliable conclusions.

Motivation & Objective

• To address the risk of bias in meta-analysis when transforming non-normal summary statistics (e.g., median, IQR) into mean and standard deviation.
• To develop a pre-test for normality that uses only commonly reported summary statistics: sample size, median, min/max, and interquartile range.
• To improve the reliability of effect size estimation in meta-analysis by filtering out studies with skewed underlying data before transformation.
• To provide a practical, statistically sound procedure for researchers to assess data symmetry before applying existing estimation methods like those by Wan et al. (2014) and Luo et al. (2017).

Proposed method

• Proposes three test statistics—T₁, T₂, and T₃—based on different combinations of summary measures: min/max (T₁), first/third quartiles (T₂), and both (T₃).
• Derives coefficient functions τ(n), φ(n), and κ(n) that adjust for sample size and expected normal order statistics using the inverse normal CDF Φ⁻¹.
• Uses the standardized deviation from symmetry (e.g., (a + b - 2m)/(b - a)) as a testable measure of skewness, scaled by sample-size-dependent coefficients.
• Applies asymptotic theory to ensure the test statistics follow a standard normal distribution under the null hypothesis of symmetry.
• Integrates the tests into a recommended workflow: pre-test symmetry → exclude skewed studies → estimate mean and SD → compute effect sizes.
• Validates the method via simulation studies and three real-world meta-analyses, demonstrating improved accuracy and consistency in effect size estimation.

Experimental results

Research questions

• RQ1 Can a normality test be reliably constructed using only summary statistics (sample size, median, min/max, IQR) without access to raw data?
• RQ2 How effective are the proposed test statistics in detecting skewed distributions in clinical data when only summary measures are available?
• RQ3 Does excluding studies identified as non-normal by the proposed test lead to more accurate and reliable meta-analytic effect size estimates?
• RQ4 How do the proposed tests compare in power and performance across different types of skewed distributions in meta-analysis settings?

Key findings

• The proposed test statistics (T₁, T₂, T₃) demonstrated statistical power close to one in simulation studies, indicating strong ability to detect non-normal (skewed) data.
• In the real-world case study on statin therapy and plasma lipid levels, the symmetry test led to a reversal in conclusion regarding total cholesterol and LDL-C levels compared to the original meta-analysis.
• For the BNP and COPD meta-analysis, the test identified several studies (e.g., Study 5 with median 50 and Q3 51) as likely right-skewed, suggesting that transformation methods assuming symmetry may introduce error.
• The forest plots from the three real data applications showed that excluding skewed studies altered the pooled effect size estimates, particularly for MMP-3 and TIMP-I, indicating that symmetry testing affects clinical conclusions.
• The test statistics are robust and easy to implement in practice, with closed-form expressions and no need for raw data, making them suitable for routine use in meta-analysis workflows.
• The recommended procedure—conducting a symmetry test before transformation—leads to more faithful and reliable meta-analytic results, especially when using estimators like Wan et al. (2014) and Luo et al. (2017).