[Paper Review] Skeptical combination of experimental results using JAGS/rjags with application to the K$^{\pm}$ mass determination

G. D’Agostini|arXiv (Cornell University)|Jan 9, 2020

Statistical Mechanics and Entropy11 references4 citations

TL;DR

This paper presents a skeptical combination of experimental results using Bayesian hierarchical modeling via JAGS/rjags to address apparent discrepancies in measurements of the charged kaon mass. By employing Markov Chain Monte Carlo (MCMC) sampling instead of closed-form solutions, it provides a robust, probabilistic estimate of the mass as 493.677 ± 0.013 MeV, revealing a previously unrecognized bias in the PDG's averaging procedure due to sequential application of the χ²/ν scaling rule.

ABSTRACT

The question of how to combine experimental results that `appear' to be in mutual disagreement, treated in detail years ago in a previous paper, is revisited. The first novelty of the present note is the explicit use of graphical models, in order to make the deterministic and probabilistic links between the variables of interest more evident. Then, instead of aiming for results in closed formulae, the integrals of interest are evaluated by {\em Markov Chain Monte Carlo} (MCMC) sampling, with the algorithms (typically Gibbs Sampler) implemented in the package JAGS ("Just Another Gibbs Sampler"). For convenience, the JAGS functions are called from R scripts, thus gaining the advantage given by the rich collection of mathematical, statistical and graphical functions included in the R installation. The results of the previous paper are thus easily re-obtained and the method is applied to the determination of the charged kaon mass. This note, based on lectures to PhD students and young researchers has been written with a didactic touch, and the relevant JAGS/rjags code is provided. (A curious bias arising from the sequential application of the $\sqrt{χ^2/ν}$ scaling prescription to 'apparently' discrepant results, found here, will be discussed in more detail in a separate paper.)

Motivation & Objective

• To develop a transparent, reproducible method for combining experimental results that appear to be in mutual disagreement, particularly when standard averaging may be misleading.
• To demonstrate the use of graphical models and MCMC sampling via JAGS/rjags for skeptical combination of results, emphasizing interpretability and computational accessibility.
• To re-evaluate the charged kaon mass using a Bayesian hierarchical model, avoiding assumptions of Gaussianity and closed-form solutions.
• To identify and quantify a previously unrecognized bias in the PDG’s averaging procedure arising from sequential application of the χ²/ν scaling rule.
• To provide didactic, reusable JAGS/rjags code for researchers to apply the method to other experimental discrepancies.

Proposed method

• Modeling each experimental result as a normal likelihood with mean $ d_i $ and standard deviation $ s_i $, using a hierarchical prior on the true mass $ \mu $.
• Employing a Jeffrey’s prior for the precision $ \tau_i = 1/\sigma_i^2 $, and assigning a common precision $ \tau $ to all results to allow for inconsistency.
• Using MCMC via JAGS to sample from the joint posterior distribution of $ \mu $, $ \tau $, and individual precision parameters $ \tau_i $.
• Implementing the model in R using the rjags package, enabling integration with R’s statistical and graphical functions for analysis and visualization.
• Evaluating the posterior distribution of the mass through Monte Carlo sampling, avoiding parametric approximations and preserving non-Gaussian features.
• Assessing model convergence and correlations between parameters using trace plots, autocorrelation, and correlation matrices of MCMC chains.

Experimental results

Research questions

• RQ1 How can experimental results that appear to be in conflict be combined in a statistically rigorous and interpretable way?
• RQ2 What are the consequences of applying the χ²/ν scaling rule sequentially across multiple results in a combination procedure?
• RQ3 Can MCMC-based Bayesian modeling with graphical models improve the transparency and robustness of result combination compared to closed-form weighted averages?
• RQ4 To what extent does the PDG’s reported value for the charged kaon mass reflect a systematic bias due to its averaging methodology?
• RQ5 How do the posterior distributions of the mass and precision parameters reveal non-Gaussian features and model uncertainty?

Key findings

• The skeptical combination yields a final estimate of the charged kaon mass as $ 493.677 \pm 0.013 $ MeV, in practical agreement with the PDG’s reported value.
• The posterior distribution of the mass is significantly non-Gaussian, with a thick tail on the lower side, indicating high uncertainty and asymmetry.
• The 95% credible interval is $[493.650, 493.697]$ MeV, and the 50% interval is $[493.668, 493.687]$ MeV, showing strong skewness.
• A previously unrecognized bias of +13 keV was identified in the PDG’s result, caused by the sequential application of the $ \sqrt{\chi^2/\nu} $ scaling rule to already-scaled uncertainties.
• The result of the MCMC-based analysis is robust and reproducible, with clear diagnostics showing good convergence and low autocorrelation in the chains.
• The posterior correlation between the mass $ \mu $ and the precision of the most discrepant result ($ r_9 $) is -0.61, indicating strong influence of this outlier on the final estimate.