[Paper Review] The Probabilities of Orbital-Companion Models for Stellar Radial Velocity Data

Fengji Hou, Jonathan Goodman+1 more|arXiv (Cornell University)|Jan 23, 2014

Stellar, planetary, and galactic studies1 references3 citations

TL;DR

This paper introduces a geometric-path Monte Carlo method to efficiently compute the fully marginalized likelihood (FML), a critical component in Bayesian model selection for radial velocity data. The method uses a path-based sampling approach with adaptive step sizes and variance estimation, enabling fast, accurate FML estimation with reliable uncertainty quantification; applied to HIP 88048 and Gliese 581, it finds the 2-companion model favored for HIP 88048 and the 5-planet model nearly indistinguishable from the 6-planet model for Gliese 581, both highly sensitive to prior choices.

ABSTRACT

The fully marginalized likelihood, or Bayesian evidence, is of great importance in probabilistic data analysis, because it is involved in calculating the posterior probability of a model or re-weighting a mixture of models conditioned on data. It is, however, extremely challenging to compute. This paper presents a geometric-path Monte Carlo method, inspired by multi-canonical Monte Carlo to evaluate the fully marginalized likelihood. We show that the algorithm is very fast and easy to implement and produces a justified uncertainty estimate on the fully marginalized likelihood. The algorithm performs efficiently on a trial problem and multi-companion model fitting for radial velocity data. For the trial problem, the algorithm returns the correct fully marginalized likelihood, and the estimated uncertainty is also consistent with the standard deviation of results from multiple runs. We apply the algorithm to the problem of fitting radial velocity data from HIP 88048 ($ν$ Oph) and Gliese 581. We evaluate the fully marginalized likelihood of 1, 2, 3, and 4-companion models given data from HIP 88048 and various choices of prior distributions. We consider prior distributions with three different minimum radial velocity amplitude $K_{\mathrm{min}}$. Under all three priors, the 2-companion model has the largest marginalized likelihood, but the detailed values depend strongly on $K_{\mathrm{min}}$. We also evaluate the fully marginalized likelihood of 3, 4, 5, and 6-planet model given data from Gliese 581 and find that the fully marginalized likelihood of the 5-planet model is too close to that of the 6-planet model for us to confidently decide between them.

Motivation & Objective

• To develop a fast and reliable method for computing the fully marginalized likelihood (FML), essential for Bayesian model selection in exoplanet detection from radial velocity data.
• To address the computational challenge of high-dimensional marginalization over orbital parameters in multi-companion systems.
• To provide a method with explicit uncertainty estimates for the FML, enabling robust model comparison.
• To evaluate the FML for 1–4 companion models in HIP 88048 and 3–6 planet models in Gliese 581 under varying prior distributions.
• To investigate how sensitive model selection is to prior assumptions, particularly the minimum radial velocity amplitude $K_{\mathrm{min}}$.

Proposed method

• The method uses a geometric path $ Z_\beta = \int g(\bm{\theta})^{1-\beta} (L(\bm{\theta})\pi(\bm{\theta}))^\beta \, d\bm{\theta} $, interpolating between a Gaussian approximation $ g(\bm{\theta}) $ of the posterior and the true integrand at $ \beta = 1 $.
• It employs Markov chain Monte Carlo (MCMC) sampling along the path, with $ \beta $ incremented in steps $ \Delta\beta $, where step size is adaptively chosen using estimated auto-correlation times.
• The algorithm estimates the FML at $ \beta = 1 $ by integrating along the path, starting from $ \beta = 0 $, where $ Z_0 $ is analytically tractable (as in BIC).
• Variance estimation and error bars are computed directly from MCMC chains, providing explicit uncertainty estimates that are validated against repeated independent runs.
• The method is inspired by multi-canonical Monte Carlo and thermodynamic integration, but differs by using a Gaussian approximation as the reference and avoiding path sampling.
• The approach is computationally efficient, completing 4-companion model evaluations for HIP 88048 in under a day on a single-core workstation.

Experimental results

Research questions

• RQ1 What is the fully marginalized likelihood of 1–4 companion models for radial velocity data from HIP 88048 under different prior distributions?
• RQ2 How does the choice of minimum radial velocity amplitude $ K_{\mathrm{min}} $ affect model selection via the FML?
• RQ3 What is the FML for 3–6 planet models in Gliese 581, and how distinguishable are the 5- and 6-planet models?
• RQ4 Can the geometric-path Monte Carlo method provide an unbiased estimate of the FML with reliable uncertainty quantification?
• RQ5 Why do Bayesian FML results sometimes favor fewer planets than suggested by frequentist significance tests, as seen in the 54-day signal in HIP 88048?

Key findings

• The geometric-path Monte Carlo method produces nearly unbiased estimates of the FML, with uncertainty estimates that match the standard deviation from multiple independent runs.
• For HIP 88048, the 2-companion model has the highest marginalized likelihood under all tested priors, though the exact values depend strongly on $ K_{\mathrm{min}} $.
• For Gliese 581, the 5-planet and 6-planet models have FMLs that are too close to confidently distinguish, indicating high model degeneracy.
• The method computes the FML for 4-companion models in HIP 88048 in under a day on a single-core workstation, demonstrating high computational efficiency.
• A 54-day signal in HIP 88048 appears significant in frequentist tests but receives low Bayesian evidence, suggesting the prior may suppress such signals despite strong posterior peaks.
• The algorithm's performance is constrained by MCMC mixing; poor sampling efficiency increases auto-correlation time $ \tau $, which affects step size and convergence.