[Paper Review] Interbreeding conditions for explaining Neandertal DNA in living humans: the nonneutral case

A. Neves|arXiv (Cornell University)|May 30, 2011

Pleistocene-Era Hominins and Archaeology17 references3 citations

TL;DR

This paper extends a prior neutral model of human-Neandertal interbreeding by introducing fitness differences, showing that even a 1% fitness advantage for anatomically modern humans leads to Neandertal extinction too rapidly to explain archaeological evidence of prolonged coexistence in the Middle East. The nonneutral model predicts higher interbreeding rates are needed to achieve the observed 1–4% Neandertal DNA in non-Africans, but such rapid extinction contradicts the long alternation of Neandertals and modern humans in Israeli caves, suggesting the neutral model remains more plausible for explaining both genetic and archeological data.

ABSTRACT

We consider here an extension of a previous work by Neves and Serva, still unpublished, which estimates the amount of interbreeding between anatomically modern Africans and Neandertals necessary for explaining the experimental fact that 1 to 4% of the DNA in non-African living humans is of Neandertal origin. In that work we considered that Africans and Neandertals had the same fitness (neutral hypothesis) and Neandertal extinction was thus an event of fortune. In this work we consider that Africans had larger fitnesses. We show results for four values for the fitness difference: 1%, 5%, 10% and 20% and compare them with the corresponding neutral results. Some technical differences with respect to the neutral case appear. We conclude that even with 1% fitness difference Neandertals extinction comes up in too small a time, so the neutral model looks more suitable for explaining the known data on occupation of some caves in Israel for a very long time, alternately by Africans and Neandertals.

Motivation & Objective

• To investigate whether a nonneutral model—where modern humans have higher fitness than Neandertals—can explain the 1–4% Neandertal DNA in non-African populations.
• To assess whether fitness differences between modern humans and Neandertals are compatible with the archaeological record of prolonged coexistence in the Middle East.
• To compare the predictions of the nonneutral model with those of the previously studied neutral model, particularly regarding required interbreeding rates and extinction timing.
• To evaluate the validity of the deterministic approximation used in the nonneutral model under varying population sizes and fitness differences.

Proposed method

• The study employs a deterministic, mean-field model of two subpopulations (modern humans and Neandertals) in a constant-sized population of size $ N $, with non-overlapping generations.
• It introduces a fitness difference $ h $, where modern humans have a relative fitness advantage over Neandertals, and models the fraction of Neandertal alleles in the human population over time.
• The model uses a scaled interbreeding parameter $ \beta = \alpha / N $, where $ \alpha $ is the per-generation interbreeding rate, to ensure $ N $-independence of results.
• The probability distribution of Neandertal DNA contribution is derived under the nonneutral assumption, and compared with the neutral case ($ h = 0 $) using numerical simulations and analytical approximations.
• Extinction times are calculated deterministically as a function of initial human fraction and fitness difference $ h $, with generation length assumed to be 20 years.
• The model's validity is assessed by checking the condition $ \alpha \ll 1 / \log_2 N $, ensuring the mean-field approximation holds.

Experimental results

Research questions

• RQ1 How does introducing a fitness advantage for modern humans affect the required interbreeding rate to achieve 1–4% Neandertal DNA in non-Africans?
• RQ2 Can the nonneutral model explain the long-term alternation of Neandertals and modern humans in Middle Eastern caves, as seen in the archaeological record?
• RQ3 How do extinction times for Neandertals vary with fitness differences, and are they consistent with the 130,000-year occupation alternation observed at sites like Skhul and Qafzeh?
• RQ4 To what extent does the deterministic approximation used in the nonneutral model remain valid for realistic population sizes and interbreeding rates?
• RQ5 Is the neutral model still more suitable than the nonneutral model for reconciling genetic data with archaeological evidence of coexistence?

Key findings

• Even with a 1% fitness advantage for modern humans ($ h = 0.01 $), Neandertal extinction occurs in approximately 600 to 1,000 years, which is too rapid to match the 130,000-year alternation of Neandertals and modern humans in Israeli caves.
• For $ h = 0.01 $, the required interbreeding parameter $ \beta $ is approximately $ 9.3 \times 10^{-4} $, and the maximum of the probability density occurs at $ \beta_{\mathrm{max}} \approx 5.0 \times 10^{-4} $, indicating high interbreeding rates are needed.
• As fitness difference $ h $ increases, the required $ \beta $ increases, and extinction times decrease, with $ h = 0.2 $ leading to extinction in 600–1,000 years and $ h = 0.01 $ in 11,000–20,000 years.
• The model shows that the deterministic approximation breaks down for $ \beta \gg 10^{-4} $ when $ N = 1000 $, suggesting caution in interpreting results for larger $ \beta $, though smaller $ N $ could improve accuracy.
• Despite the nonneutral model's mathematical simplicity, the neutral model remains more consistent with the archaeological record of prolonged coexistence, suggesting it is better suited for explaining both genetic and archeological data.