[Paper Review] Analysis of sloppiness in model simulations: unveiling parameter uncertainty when mathematical models are fitted to data

Gloria M. Monsalve-Bravo, Brodie Lawson+8 more|arXiv (Cornell University)|Mar 29, 2022

Spectroscopy and Quantum Chemical Studies94 references42 citations

TL;DR

This paper introduces a Bayesian-informed approach to analyze model sloppiness, identifying stiff eigenparameters—key parameter combinations that strongly influence model outputs—by integrating prior knowledge and data. It extends dimensionality reduction techniques (posterior covariance and likelihood-informed subspace) to distinguish data-driven from prior-informed parameter sensitivities, revealing hidden control mechanisms in complex models across systems biology, ecology, and cardiac electrophysiology.

ABSTRACT

This work introduces a comprehensive approach to assess the sensitivity of model outputs to changes in parameter values, constrained by the combination of prior beliefs and data. This novel approach identifies stiff parameter combinations strongly affecting the quality of the model-data fit while simultaneously revealing which of these key parameter combinations are informed primarily by the data or are also substantively influenced by the priors. We focus on the very common context in complex systems where the amount and quality of data are low compared to the number of model parameters to be collectively estimated, and showcase the benefits of this technique for applications in biochemistry, ecology, and cardiac electrophysiology. We also show how stiff parameter combinations, once identified, uncover controlling mechanisms underlying the system being modeled and inform which of the model parameters need to be prioritized in future experiments for improved parameter inference from collective model-data fitting.

Motivation & Objective

• To address the challenge of high-dimensional parameter uncertainty in complex models fitted to limited data.
• To identify critical parameter combinations (stiff eigenparameters) that control model outputs despite individual parameter ambiguity.
• To distinguish whether parameter sensitivities arise primarily from data or prior beliefs, enhancing interpretability of model inference.
• To provide a robust, uncertainty-informed method for sensitivity analysis that goes beyond local curvature at best-fit values.
• To enable better experimental design by identifying which parameters should be prioritized for future data collection.

Proposed method

• Uses Bayesian inference to generate posterior samples, incorporating both data and prior beliefs to characterize parameter uncertainty.
• Applies posterior covariance matrix (P) to assess data informativity on parameter combinations, assuming multivariate normality of log-transformed parameters.
• Employs likelihood-informed subspace (LIS) method (G) to isolate data-driven sensitivities by excluding prior influence, using Hessian approximations.
• Adapts dimensionality reduction techniques from Cui et al. (2016) to separate prior and data contributions to parameter sensitivity.
• Compares results from P and G matrices to identify stiff eigenparameters—combinations tightly constrained by data—while accounting for prior information.
• Validates methods across three domains: biochemical pathways, ecological population dynamics, and cardiac action potential models, using synthetic and real data.

Experimental results

Research questions

• RQ1 How can parameter sensitivities be disentangled into data-driven and prior-informed components in complex models with limited data?
• RQ2 What parameter combinations (eigenparameters) most strongly influence model outputs, and which of these are constrained by data rather than priors?
• RQ3 How do different Bayesian dimensionality reduction methods (posterior covariance vs. LIS) compare in identifying stiff eigenparameters across diverse modeling contexts?
• RQ4 Can the proposed method reveal hidden control mechanisms in model behavior that are obscured when analyzing individual parameters?
• RQ5 To what extent can this framework guide future experimental design by identifying which parameters should be measured to improve model inference?

Key findings

• The posterior covariance method (P) successfully identifies stiff eigenparameters under the assumption of approximately multivariate normal posterior distributions for log-transformed parameters.
• The likelihood-informed subspace (LIS) method (G) isolates data-driven sensitivities and outperforms standard methods when the likelihood surface is complex or non-Gaussian.
• In all test cases—biochemical, ecological, and cardiac models—stiff eigenparameters were found to be combinations of parameters, not individual parameters, that control model output.
• The method revealed that prior beliefs can significantly influence parameter sensitivities, especially when data are sparse, and that these effects can be explicitly separated from data-informed constraints.
• The approach enables identification of 'control knobs'—parameter combinations that must be measured with high precision to improve model predictions.
• The framework supports better experimental design by highlighting which parameters should be prioritized for data collection to reduce uncertainty in model outputs.