[Paper Review] Beta-coalescents when sample size is large

Jonathan A Chetwynd-Diggle, Bjarki Eldon|arXiv (Cornell University)|Jan 13, 2026

Evolution and Genetic Dynamics0 citations

TL;DR

The paper analyzes Beta-coalescents under large sample sizes, deriving coalescent rates and scaling regimes where time is measured in generations, and showing how an effective population size can vary with N and a parameter ψ(N).

ABSTRACT

Sweepstakes reproduction refers to a highly skewed individual recruitment success without involving natural selection and may apply to individuals in broadcast spawning populations characterised by Type III survivorship. We consider an extension of the model of sweepstakes reproduction for a haploid panmictic population of constant size $N$; the extension also works as an alternative to the Wright-Fisher model. Our model incorporates an upper bound on the random number of potential offspring (juveniles) produced by a given individual. Depending on how the bound behaves relative to the total population size, we obtain the Kingman coalescent, an incomplete Beta-coalescent, or the (complete) Beta-coalescent. We argue that applying such an upper bound is biologically reasonable. Moreover, we estimate the error of the coalescent approximation. The error estimates reveal that convergence can be slow, and small sample size can be sufficient to invalidate convergence, for example if the stated bound is of the form $N/\log N$. We use simulations to investigate the effect of increasing sample size on the site-frequency spectrum. When the limit is a Beta-coalescent, the site frequency spectrum will be as predicted by the limiting tree even though the full coalescent tree may deviate from the limiting one. When in the domain of attraction of the Kingman coalescent the effect of increasing sample size depends on the effective population size as has been noted in the case of the Wright-Fisher model. Conditioning on the population ancestry (the random ancestral relations of the entire population at all times) may have little effect on the site-frequency spectrum for the models considered here (as evidenced by simulation results).

Motivation & Objective

• Motivate studying coalescent models under large sample sizes and identify sources of error in coalescent approximations.
• Derive the general form of the coalescent rate λ_{n,k} for merging k out of n blocks and analyze scaling with N and ψ(N).
• Explore how choices of ψ(N) affect time scales and errors, and relate to Kingman and Beta-coalescents.
• Characterize the range of effective population sizes implied by different scaling regimes (1<α<2).

Proposed method

• Present the general coalescent rate for merging k out of n blocks: λ_{n,k}=c 1{k=2}+c′ ∫_0^1 1{0<x≤γ} x^{k-2}(1−x)^{n−k} Λ_{+}(dx) with Λ+ non-atomic at 0.
• Analyze time scaling that is proportional to N generations when ψ1,N=O(1) and ψ1,N=N/logN.
• Discuss how choosing ψ(N) and ignoring error terms affect the accuracy of the coalescent approximation.
• Use propositions CNversion0 and Lambda-convergence to extract sources of error and connections to Kingman coalescence.
• Relate the limiting process to Kingman coalescent in certain regimes while highlighting effective size changes.
• Describe how the effective size can vary as cN to c′N^{α−1}(log N)^{2−α} for 1<α<2.

Experimental results

Research questions

• RQ1 What is the form of the coalescent rate λ_{n,k} for merging k out of n blocks under large sample sizes?
• RQ2 How do different ψ(N) scalings affect the time scale of the coalescent and the size of error terms?
• RQ3 Under what conditions does the process resemble Kingman or Beta-coalescents, and how is the effective population size characterized?
• RQ4 What range of effective sizes emerges when 1<α<2 and how do logarithmic factors modify this range?

Key findings

• The coalescent rate for merging k out of n blocks has a general form involving an integral against Λ+ with an indicator on x≤γ.
• A new time-scaling regime appears, proportional to N generations when ψ1,N=O(1) and ψ1,N=N/logN.
• Errors in the coalescent approximation arise from incorrect upper-bound models for ψ(N) and from ignored error terms.
• Even when in the Kingman attraction domain (ψ(N)=N/logN), time is effectively measured similarly to Beta-coalescents due to the scaling.
• The effective size can range from cN to c′N^{α−1}(log N)^{2−α} for 1<α<2, reflecting substantial variability depending on scaling.
• Propositions CNversion0 and Lambda-convergence illuminate sources of approximation error and connections to limiting coalescents.