[Paper Review] CosmicFish Validation Notes V1.0

Marco Raveri, M. Martinelli+2 more|arXiv (Cornell University)|Jul 4, 2016

Research Data Management Practices3 citations

TL;DR

This paper validates the CosmicFish code, a cosmological forecast tool using Fisher matrix methods, across multiple experiments including CMB, supernovae, weak lensing, galaxy clustering, and redshift drift. It demonstrates strong agreement with literature results—especially in Planck 2015, CMBpol, and DES forecasts—confirming the code's reliability for future cosmological survey predictions.

ABSTRACT

These notes show and comment the examples that have been used to validate the CosmicFish code. We compare the results obtained with the code to several other results available in literature finding an overall good level of agreement. We will update this set of notes when relevant modifications to the CosmicFish code will be released or other validation examples are worked out. The CosmicFish code and the package to produce all the validation results presented here are publicly available at http://cosmicfish.github.io. The present version is based on CosmicFish Jun16.

Motivation & Objective

• To validate the accuracy of the CosmicFish code in forecasting cosmological parameter constraints across multiple future experiments.
• To ensure the code's Fisher matrix predictions align with established results from MCMC analyses and published literature.
• To provide a public, reproducible validation package for transparency and ongoing code improvement.
• To assess the code's performance under varying experimental specifications, including foreground modeling and sky fraction effects.
• To confirm consistency in bounds on standard parameters (e.g., Ωbh², ns, τ) and extensions (e.g., Neff, w0, wa, ∑mν) across diverse cosmological probes.

Proposed method

• Uses Fisher matrix forecasts to compute parameter constraints across multiple cosmological probes: CMB, redshift drift, supernovae, weak lensing, and galaxy clustering.
• Compares CosmicFish results to published MCMC results from Galli et al. (2010), Ade et al. (2015), Conley et al. (2011), Zablocki et al. (2014), and Martinelli et al. (2012).
• Employs fiducial parameter values and 68% credible intervals from literature to benchmark CosmicFish's forecasted error bars.
• Adjusts sky fraction (fsky) and excludes low-ℓ polarization (lowP) in Planck 2015 forecasts to account for missing foreground modeling.
• Incorporates E-ELT redshift drift specifications and Planck Blue Book CMB forecasts in combined forecasts to test joint constraining power.
• Reproduces results using publicly available code and validation packages hosted at http://cosmicfish.github.io.

Experimental results

Research questions

• RQ1 How accurately does CosmicFish reproduce published Fisher matrix forecasts for Planck and CMBpol experiments?
• RQ2 To what extent do CosmicFish forecasts for Planck 2015 (TT, TE, EE) match the actual Planck 2015 results, especially regarding polarization and low-ℓ effects?
• RQ3 How well does CosmicFish predict constraints when combining redshift drift (E-ELT) with CMB forecasts, compared to MCMC results?
• RQ4 Can CosmicFish accurately forecast constraints from supernovae surveys (e.g., SNLS, SDSS) and match the non-Gaussian posterior shapes observed in Conley et al. (2011)?
• RQ5 How do CosmicFish forecasts for DES and DETF Stage IV weak lensing experiments compare to the optimistic and pessimistic forecasts in Albrecht et al. (2006)?

Key findings

• CosmicFish forecasts for Planck Blue Book and CMBpol show excellent agreement with Galli et al. (2010), with 68% C.L. bounds on Neff differing by less than 0.05 in both cases.
• For Planck 2015 (TT only), CosmicFish recovers bounds on Ωbh², Ωch², and ns within 0.0002, 0.002, and 0.005 respectively, matching Ade et al. (2015) to within 10%.
• Including polarization (TT+TE+EE) in Planck 2015 forecasts, CosmicFish achieves 68% C.L. bounds on τ and As at 0.02 and 0.04, respectively, compatible with Planck 2015 after adjusting fsky to 0.01.
• In combined Planck + E-ELT redshift drift forecasts, CosmicFish recovers Ωm and w0 bounds of 0.01 and 0.05 respectively, consistent with Martinelli et al. (2012), though slightly looser due to missing HST prior.
• For supernovae, CosmicFish reproduces a 68% C.L. bound of 0.1 on Ωm and 0.3 on w0, matching Conley et al. (2011), though non-Gaussian posterior features are not fully captured.
• In DES and DETF Stage IV forecasts, CosmicFish matches optimistic weak lensing bounds (e.g., ΩΛ: 0.005, w0: 0.05) but shows less degradation in the pessimistic case due to missing systematic effects.