[Paper Review] Detecting the skewness of data from the five-number summary and its application in meta-analysis

Jiandong Shi, Dehui Luo+5 more|arXiv (Cornell University)|Oct 12, 2020

Meta-analysis and systematic reviews35 citations

TL;DR

Proposes skewness tests based on the five-number summary and sample size to decide how to handle studies in meta-analysis, with exact and approximate critical values and a supporting flow chart.

ABSTRACT

For clinical studies with continuous outcomes, when the data are potentially skewed, researchers may choose to report the whole or part of the five-number summary (the sample median, the first and third quartiles, and the minimum and maximum values) rather than the sample mean and standard deviation. In the recent literature, it is often suggested to transform the five-number summary back to the sample mean and standard deviation, which can be subsequently used in a meta-analysis. However, if a study contains skewed data, this transformation and hence the conclusions from the meta-analysis are unreliable. Therefore, we introduce a novel method for detecting the skewness of data using only the five-number summary and the sample size, and meanwhile propose a new flow chart to handle the skewed studies in a different manner. We further show by simulations that our skewness tests are able to control the type I error rates and provide good statistical power, followed by a simulated meta-analysis and a real data example that illustrate the usefulness of our new method in meta-analysis and evidence-based medicine.

Motivation & Objective

• Motivated by meta-analytic practice where studies report only the five-number summary for skewed data.
• Develop a statistical test to detect skewness using the five-number summary under three reporting scenarios.
• Provide a decision flow chart to guide handling of skewed studies in meta-analysis.

Proposed method

• Define skewness levels via differences between key quintiles (e.g., a, m, b or q1, m, q3) and derive test statistics for each reporting scenario.
• Develop Wald-type and exact/approximate null distributions for the test statistics under normality.
• Provide critical values for finite samples and give practical approximations for easy implementation.
• Offer an online calculator implementing the flow chart and tests.
• Demonstrate performance through simulations and a simulated meta-analysis.

Experimental results

Research questions

• RQ1 Can skewness be detected using only the five-number summary and sample size under three common reporting scenarios?
• RQ2 How do different skewness tests perform in terms of type I error control and power for small to moderate sample sizes?
• RQ3 How should meta-analytic practice incorporate skewness testing to improve inference when five-number summaries are used?
• RQ4 What is the impact of applying the flow-chart-guided decisions on meta-analysis results compared to traditional methods?

Key findings

• Proposed skewness tests control type I error rates for exact and approximated critical values across scenarios, with asymptotic values showing limitations for small samples.
• Tests are generally powerful for detecting skewness across unimodal skewed alternatives, with performance varying by scenario.
• A flow chart enables three options for skewed studies: exclude, transform, or conduct subgroup analysis, improving meta-analytic reliability.
• In simulated meta-analysis, including skewness-filtered studies (option iii) yields unbiased effect estimates and competitive confidence interval lengths compared to the ideal case.
• An online calculator demonstrates practical implementation of the skewness tests and flow chart.