[Paper Review] Uncertainty analysis and composite hypothesis under the likelihood paradigm

André Chalom, Paulo Inácio Prado|arXiv (Cornell University)|Aug 13, 2015

Probabilistic and Robust Engineering Design14 references3 citations

TL;DR

This paper proposes a likelihood-based methodology, PLUE (Profile Likelihood Uncertainty Estimation), to integrate model calibration and uncertainty analysis by profiling likelihood surfaces across parameter combinations. It enables intuitive, data-informed uncertainty estimation that respects the information in observed data, offering a coherent alternative to disconnected calibration and sensitivity analyses in biological modeling.

ABSTRACT

The correct use and interpretation of models depends on several steps, two of which being the calibration by parameter estimation and the analysis of uncertainty. In the biological literature, these steps are seldom discussed together, but they can be seen as fitting pieces of the same puzzle. In particular, analytical procedures for uncertainty estimation may be masking a high degree of uncertainty coming from a model with a stable structure, but insufficient data. Under a likelihoodist approach, the problem of uncertainty estimation is closely related to the problem of composite hypothesis. In this paper, we present a brief historical background on the statistical school of Likelihoodism, and examine the complex relations between the law of likelihood and the problem of composite hypothesis, together with the existing proposals for coping with it. Then, we propose a new integrative methodology for the uncertainty estimation of models using the information in the collected data. We argue that this methodology is intuitively appealing under a likelihood paradigm.

Motivation & Objective

• To address the fragmented treatment of model calibration and uncertainty analysis in biological modeling.
• To unify parameter estimation and uncertainty quantification under the likelihood paradigm.
• To develop a method that respects data quality and quantity while avoiding assumptions that mask underlying uncertainty.
• To provide a coherent, intuitive framework for uncertainty estimation that supports experimental design and parameter prioritization.

Proposed method

• Formulate a biological model as an R function that maps parameter combinations to model outputs.
• Define a likelihood function based on observed data, using statistical models (e.g., binomial) to represent parameter uncertainty.
• Use Monte Carlo sampling (via Metropolis-Hastings or MCMC) to generate samples from the likelihood distribution.
• Apply the biological model to each sampled parameter set to propagate uncertainty into model outputs.
• Profile the likelihood of each model output using the PLUE function, which combines sampling and likelihood-based weighting.
• Visualize results via profile plots, scatterplots, and PRCC (Partial Rank Correlation Coefficients) to assess sensitivity and uncertainty.

Experimental results

Research questions

• RQ1 How can uncertainty estimation in biological models be unified with parameter calibration under the likelihood paradigm?
• RQ2 What are the limitations of traditional sensitivity analysis when it ignores data quality and quantity?
• RQ3 How can likelihood profiling improve the interpretation of uncertainty in models with sparse or noisy data?
• RQ4 Can a single integrated framework reduce the risk of artifacts from assumptions in model calibration and uncertainty analysis?
• RQ5 How does the PLUE method compare to Latin Hypercube Sampling in capturing data-informed uncertainty?

Key findings

• The PLUE method provides a unified framework that integrates model calibration and uncertainty analysis by leveraging likelihood surfaces derived from observed data.
• Uncertainty estimates derived via PLUE are more reflective of data quality and quantity than traditional methods that assume uniform or arbitrary parameter ranges.
• The method enables identification of key parameters driving uncertainty, supporting targeted data collection and experimental design.
• Profile plots generated by PLUE offer a visual representation of uncertainty that is grounded in the likelihood of the observed data, not arbitrary sampling.
• Sensitivity analyses using PLUE, such as scatterplots and PRCC, reveal parameter influences that are consistent with data-driven likelihood, improving interpretability.
• The implementation in R using the pse and bbmle packages enables reproducible, modular, and extensible uncertainty analysis for ecological and biological models.