[论文解读] If your P value looks too good to be true, it probably is: Communicating reproducibility and variability in cell biology

Samuel J. Lord, Katrina Velle+2 more|arXiv (Cornell University)|Nov 8, 2019

Gene Regulatory Network Analysis参考文献 28被引用 89

一句话总结

本文警告将细胞计数视为独立样本会产生误导性的 P 值，并提供实用教程，帮助传达细胞水平变异性和实验可重复性，包括对 P 值的替代方法。

ABSTRACT

The cell biology literature is littered with erroneously tiny P values, often the result of evaluating individual cells as independent samples. Because readers use P values and error bars to infer whether a reported difference would likely recur if the experiment were repeated, the sample size N used for statistical tests should actually be the number of times an experiment is performed, not the number of cells (or subcellular structures) analyzed across all experiments. P values calculated using the number of cells do not reflect the reproducibility of the result and are thus highly misleading. To help authors avoid this mistake, we provide examples and practical tutorials for creating figures that communicate both the cell-level variability and the experimental reproducibility.

研究动机与目标

• Highlight the misuse of P values when cells are treated as independent samples.
• Demonstrate how cell-level variability differs from experimental reproducibility.
• Provide practical tutorials for creating figures that communicate variability and reproducibility.
• Advise on estimation statistics and alternative plots to P values in cell biology.

提出的方法

• Analyze how treating individual cells as independent samples inflates sample size and misleads P values.
• Provide examples illustrating the difference between cell-level variability and experimental reproducibility.
• Offer step-by-step tutorials for making plots in R, Python, and Excel to communicate variability and reproducibility.
• Discuss estimation statistics and forest plots as alternatives to relying solely on P values.

实验结果

研究问题

• RQ1 How does counting cells as independent samples affect P value interpretation in cell biology experiments?
• RQ2 How can researchers effectively communicate cell-level variability and experimental reproducibility?
• RQ3 What plotting and statistical approaches (e.g., forest plots, estimation statistics) better convey reproducibility than traditional P values?

主要发现

• P values based on the number of cells, not experimental repetitions, can be highly misleading regarding reproducibility.
• The paper provides examples and practical tutorials to create figures that show both cell-level variability and experimental reproducibility.
• Estimation statistics and forest plots are proposed as reasonable alternatives or complements to P values.
• The authors clarify the benefits of P values while acknowledging their flaws and offer guidance to improve communication in figures.