[Paper Review] Flexible model selection for mechanistic network models

Sixing Chen, Antonietta Mira+1 more|arXiv (Cornell University)|Apr 1, 2018

Markov Chains and Monte Carlo Methods45 references4 citations

TL;DR

This paper proposes a likelihood-free model selection framework for mechanistic network models using simulator-based inference with the Super Learner to select among competing models based on network summary statistics. By combining Approximate Bayesian Computation (ABC) with ensemble learning, the method enables robust, uncertainty-quantified model selection even when likelihoods are intractable, demonstrating strong performance on a yeast protein-protein interaction network.

ABSTRACT

Network models are applied across many domains where data can be represented as a network. Two prominent paradigms for modeling networks are statistical models (probabilistic models for the observed network) and mechanistic models (models for network growth and/or evolution). Mechanistic models are better suited for incorporating domain knowledge, to study effects of interventions (such as changes to specific mechanisms) and to forward simulate, but they typically have intractable likelihoods. As such, and in a stark contrast to statistical models, there is a relative dearth of research on model selection for such models despite the otherwise large body of extant work. In this paper, we propose a simulator-based procedure for mechanistic network model selection that borrows aspects from Approximate Bayesian Computation (ABC) along with a means to quantify the uncertainty in the selected model. To select the most suitable network model, we consider and assess the performance of several learning algorithms, most notably the so-called Super Learner, which makes our framework less sensitive to the choice of a particular learning algorithm. Our approach takes advantage of the ease to forward simulate from mechanistic network models to circumvent their intractable likelihoods. The overall process is flexible and widely applicable. Our simulation results demonstrate the approach's ability to accurately discriminate between competing mechanistic models. Finally, we showcase our approach with a protein-protein interaction network model from the literature for yeast (Saccharomyces cerevisiae).

Motivation & Objective

• To address the lack of model selection methods for mechanistic network models with intractable likelihoods.
• To develop a flexible, simulator-based approach that leverages forward simulation to bypass intractable likelihoods.
• To incorporate uncertainty quantification in model selection using ensemble learning and ABC principles.
• To reduce sensitivity to the choice of individual learning algorithms through the use of the Super Learner framework.
• To demonstrate the method’s effectiveness on a real-world biological network—yeast protein-protein interaction data.

Proposed method

• The method uses forward simulation from candidate mechanistic network models to generate synthetic network data.
• It applies the Super Learner algorithm to classify networks based on summary statistics, selecting the model that best predicts the observed network.
• Summary statistics are selected using domain knowledge to reflect differences between candidate models.
• Model calibration is performed by matching key network characteristics (e.g., degree distribution, clustering) between simulated and observed networks.
• The approach avoids reliance on sufficient statistics by using machine learning to learn optimal discriminative features from simulated data.
• Uncertainty in model selection is quantified via the Super Learner’s ensemble prediction performance and cross-validation.

Experimental results

Research questions

• RQ1 Can a simulator-based, likelihood-free method accurately select the most appropriate mechanistic network model from a set of candidates?
• RQ2 How does the performance of the proposed method compare to traditional ABC-based model selection in terms of accuracy and robustness?
• RQ3 To what extent does the Super Learner framework reduce sensitivity to the choice of individual classification algorithms in model selection?
• RQ4 How well can the method distinguish between mechanistic models with subtle differences in generative rules?
• RQ5 Can the method be effectively applied to real biological network data, such as the yeast protein-protein interaction network?

Key findings

• The proposed method successfully discriminates between competing mechanistic network models with high accuracy in simulation studies.
• The use of the Super Learner significantly improves model selection robustness by combining multiple learning algorithms without requiring prior selection of the best one.
• The method effectively handles intractable likelihoods by relying on forward simulation and summary statistics, avoiding the need for analytical likelihood computation.
• Model calibration based on matching key network characteristics (e.g., degree distribution, transitivity) ensures that simulated networks are comparable to the observed data.
• The approach enables uncertainty quantification in model selection, providing confidence in the selected model.
• The method was successfully applied to a real-world yeast protein-protein interaction network, demonstrating its practical utility in systems biology.