[Paper Review] Does modelling need a Reformation? Ideas for a new grammar of modelling

Andrea Saltelli|arXiv (Cornell University)|Dec 18, 2017

Philosophy and History of Science1 citations

TL;DR

The paper argues that mathematical modelling suffers from a crisis comparable to those in medicine and psychology, necessitating a 'Reformation' to establish a new grammar of modelling. It proposes integrating uncertainty and sensitivity analysis, sensitivity auditing, quantitative storytelling, and ethics of quantification to build a more rigorous, transparent, and ethically grounded foundation for modelling practice.

ABSTRACT

The quality of mathematical modelling is looked at from the perspective of science's own quality control arrangement and recent crises. It is argued that the crisis in the quality of modelling is at least as serious as that which has come to light in fields such as medicine, economics, psychology, and nutrition. In the context of the nascent sociology of quantification, the linkages between big data, algorithms, mathematical and statistical modelling (use and misuse of p-values) are evident. Looking at existing proposals for best practices the suggestion is put forward that the field needs a thorough Reformation, leading to a new grammar for modelling. Quantitative methodologies such as uncertainty and sensitivity analysis can form the bedrock on which the new grammar is built, while incorporating important normative and ethical elements. To this effect we introduce sensitivity auditing, quantitative storytelling, and ethics of quantification.

Motivation & Objective

• To address the growing crisis in the quality of mathematical modelling across scientific disciplines.
• To examine the parallels between modelling failures and similar crises in medicine, psychology, and nutrition.
• To propose a comprehensive reform of modelling practices through a new 'grammar' grounded in quantitative rigor and ethical principles.
• To integrate uncertainty and sensitivity analysis as core components of responsible modelling.
• To introduce novel frameworks—sensitivity auditing and quantitative storytelling—for improving transparency and interpretability in modelling.

Proposed method

• Analyzing existing best practices in modelling and identifying systemic weaknesses in quality control mechanisms.
• Drawing on the sociology of quantification to examine the role of big data, algorithms, and p-value misuse in undermining model reliability.
• Proposing sensitivity auditing as a systematic method to evaluate model robustness under varying assumptions and inputs.
• Introducing quantitative storytelling as a narrative framework to enhance interpretability and communication of model results.
• Embedding normative and ethical considerations into modelling through the ethics of quantification framework.
• Using uncertainty and sensitivity analysis as foundational techniques to strengthen model transparency and defensibility.

Experimental results

Research questions

• RQ1 How do systemic flaws in modelling practices compare to those in other scientific fields such as medicine and psychology?
• RQ2 What mechanisms can ensure greater transparency and accountability in mathematical modelling?
• RQ3 How can sensitivity auditing be operationalized to assess model reliability across diverse contexts?
• RQ4 In what ways can quantitative storytelling improve the communication and interpretation of complex models?
• RQ5 How can ethical principles be systematically integrated into the design and evaluation of mathematical models?

Key findings

• The crisis in modelling quality is as severe as those observed in medicine, psychology, and nutrition, indicating a systemic failure in quality control.
• The misuse of p-values and overreliance on big data and algorithms have contributed significantly to model unreliability.
• Sensitivity auditing provides a structured approach to evaluate model behavior under uncertainty and varying assumptions.
• Quantitative storytelling enhances model interpretability by framing results within coherent, context-aware narratives.
• Ethics of quantification offers a normative framework to guide responsible model development and deployment.
• Integrating uncertainty and sensitivity analysis into modelling workflows strengthens defensibility and transparency.