[Paper Review] Statistical modeling: the three cultures

Adel Daoud, Devdatt Dubhashi|arXiv (Cornell University)|Dec 8, 2020

Data Analysis with R4 citations

TL;DR

This paper introduces the Hybrid Modeling Culture (HMC), a fusion of the traditional Data Modeling Culture (DMC) and Algorithmic Modeling Culture (AMC), where prediction and inference are integrated to improve causal inference. By combining machine learning for prediction with statistical inference, HMC dissolves the traditional boundary between prediction and explanation, enhancing reliability and validity in scientific modeling across natural and social sciences.

ABSTRACT

Two decades ago, Leo Breiman identified two cultures for statistical modeling. The data modeling culture (DMC) refers to practices aiming to conduct statistical inference on one or several quantities of interest. The algorithmic modeling culture (AMC) refers to practices defining a machine-learning (ML) procedure that generates accurate predictions about an event of interest. Breiman argued that statisticians should give more attention to AMC than to DMC, because of the strengths of ML in adapting to data. While twenty years later, DMC has lost some of its dominant role in statistics because of the data-science revolution, we observe that this culture is still the leading practice in the natural and social sciences. DMC is the modus operandi because of the influence of the established scientific method, called the hypothetico-deductive scientific method. Despite the incompatibilities of AMC with this scientific method, among some research groups, AMC and DMC cultures mix intensely. We argue that this mixing has formed a fertile spawning pool for a mutated culture that we called the hybrid modeling culture (HMC) where prediction and inference have fused into new procedures where they reinforce one another. This article identifies key characteristics of HMC, thereby facilitating the scientific endeavor and fueling the evolution of statistical cultures towards better practices. By better, we mean increasingly reliable, valid, and efficient statistical practices in analyzing causal relationships. In combining inference and prediction, the result of HMC is that the distinction between prediction and inference, taken to its limit, melts away. We qualify our melting-away argument by describing three HMC practices, where each practice captures an aspect of the scientific cycle, namely, ML for causal inference, ML for data acquisition, and ML for theory prediction.

Motivation & Objective

• To address the growing tension between traditional statistical inference (DMC) and modern machine learning prediction (AMC) in scientific research.
• To identify the limitations of treating prediction and inference as separate paradigms in data science and social sciences.
• To propose a new synthesis—Hybrid Modeling Culture (HMC)—that unifies prediction and inference for more robust scientific modeling.
• To demonstrate how HMC supports the full scientific cycle, including causal inference, data acquisition, and theory prediction.
• To advocate for a shift toward more reliable, valid, and efficient statistical practices through integrated modeling approaches.

Proposed method

• Proposes a conceptual framework identifying three statistical cultures: Data Modeling Culture (DMC), Algorithmic Modeling Culture (AMC), and a newly defined Hybrid Modeling Culture (HMC).
• Analyzes the historical and philosophical roots of DMC, rooted in the hypothetico-deductive scientific method, and its dominance in natural and social sciences.
• Examines the rise of AMC, driven by machine learning’s predictive power and adaptability to complex data, and its incompatibility with traditional scientific inference.
• Introduces HMC as a synthesis where prediction and inference are no longer separate but mutually reinforcing, especially in causal modeling.
• Illustrates HMC through three practical applications: using ML for causal inference, ML for data acquisition, and ML for theory prediction.
• Argues that HMC dissolves the rigid distinction between prediction and inference by embedding inferential goals within predictive modeling frameworks.

Experimental results

Research questions

• RQ1 How can machine learning prediction and statistical inference be meaningfully integrated in scientific research?
• RQ2 What are the key characteristics of a hybrid modeling culture that unifies prediction and inference?
• RQ3 In what ways can HMC improve the reliability and validity of causal inference in data science?
• RQ4 How does HMC support the full scientific cycle, including theory development and data collection?
• RQ5 What are the practical implications of dissolving the traditional boundary between prediction and inference in statistical modeling?

Key findings

• The Hybrid Modeling Culture (HMC) emerges from the fusion of Data Modeling Culture (DMC) and Algorithmic Modeling Culture (AMC), creating a new paradigm for statistical modeling.
• HMC enables prediction and inference to reinforce each other, leading to more robust and reliable scientific conclusions.
• The distinction between prediction and inference, when taken to its limit, dissolves in HMC, as both functions are embedded in unified modeling procedures.
• HMC supports causal inference by using machine learning models not just for prediction, but to estimate causal effects with inferential validity.
• ML techniques in HMC are applied to data acquisition, improving sampling strategies and data quality in scientific experiments.
• HMC facilitates theory prediction by using predictive models to generate testable hypotheses, thus closing the loop in the scientific method.