[Paper Review] How Evolutionary Dynamics Affects Network Reciprocity in Prisoner's Dilemma

Giulio Cimini, Ángel Sánchez|arXiv (Cornell University)|Jan 1, 2015

Evolutionary Game Theory and Cooperation53 references16 citations

TL;DR

This paper investigates how different evolutionary dynamics affect network reciprocity in the Prisoner's Dilemma game, using agent-based simulations across diverse network structures and strategy-update rules. It finds that network reciprocity—where spatial structure sustains cooperation—only emerges when players use payoff-comparison-based strategies; when strategies are updated based on imitation or best response without considering neighbors' payoffs, cooperation levels remain unchanged across network types, explaining experimental results where network structure had no effect.

ABSTRACT

Cooperation lies at the foundations of human societies, yet why people cooperate remains a conundrum. The issue, known as network reciprocity, of whether population structure can foster cooperative behavior in social dilemmas has been addressed by many, but theoretical studies have yielded contradictory results so far—as the problem is very sensitive to how players adapt their strategy. However, recent experiments with the prisoner's dilemma game played on different networks and in a specific range of payoffs suggest that humans, at least for those experimental setups, do not consider neighbors' payoffs when making their decisions, and that the network structure does not influence the final outcome. In this work we carry out an extensive analysis of different evolutionary dynamics, taking into account most of the alternatives that have been proposed so far to implement players' strategy updating process. In this manner we show that the absence of network reciprocity is a general feature of the dynamics (among those we consider) that do not take neighbors' payoffs into account. Our results, together with experimental evidence, hint at how to properly model real people's behavior.

Motivation & Objective

• To resolve the contradiction between theoretical predictions of network reciprocity and experimental findings showing no network effect in human cooperation.
• To investigate how various evolutionary dynamics—especially those not based on payoff comparison—affect the emergence of cooperation in structured populations.
• To identify which strategy-update mechanisms are consistent with empirical human behavior in social dilemmas.
• To determine whether the absence of network reciprocity in human experiments is a general feature of non-payoff-comparison dynamics.

Proposed method

• An agent-based model simulates individuals on complex networks, each playing an iterated Prisoner’s Dilemma with neighbors.
• Players update their cooperation probability p(t) every τ rounds using one of ten distinct evolutionary dynamics, including imitation, Fermi rule, death-birth, best response, and reinforcement learning.
• Network structures include well-mixed, Erdős–Rényi random, scale-free, regular lattices, and two real-world networks (email and PGP).
• Payoff parameters are set with R=1, P=0, T∈(1,2), and S∈(−1,0], covering weak and strong social dilemmas.
• The system is simulated over 10,000 rounds, with results averaged over 10 independent realizations to ensure statistical robustness.
• Key observables are the fraction of cooperators c(t) and the stationary distribution P(p) of cooperation probabilities.

Experimental results

Research questions

• RQ1 Does network reciprocity persist when players do not use payoff-comparison-based strategy updates?
• RQ2 Which evolutionary dynamics lead to network reciprocity, and which do not?
• RQ3 How do the results of this model compare to recent human experiments on large-scale networks?
• RQ4 Is the absence of network effects in human experiments a general consequence of non-payoff-comparison strategy updates?
• RQ5 Can reinforcement learning and best-response dynamics explain the lack of network influence observed in experiments?

Key findings

• Network reciprocity is absent when evolutionary dynamics do not consider neighbors’ payoffs, such as in unconditional imitation, voter model, and best-response rules.
• For all non-payoff-comparison dynamics, the final level of cooperation c remains nearly identical across all network types, including well-mixed and structured networks.
• Only payoff-comparison-based dynamics—such as the Fermi rule and proportional imitation—show significant network reciprocity, with higher cooperation on regular lattices and scale-free networks.
• The stationary distribution P(p) of cooperation probabilities shows bimodal peaks under payoff-comparison rules, indicating stable clusters of cooperators and defectors, while non-comparison rules yield unimodal, broad distributions centered near p=0.5.
• Reinforcement learning with low learning rate λ=10−2 produces results similar to non-payoff-comparison dynamics, showing no network effect.
• The experimental observation that network structure does not affect cooperation outcomes is explained by the fact that human players do not use payoff-comparison in strategy updates, aligning with the model’s non-comparison dynamics.