Game Theory · Chapter 14
Evolutionary Game Theory
Animals don’t solve for equilibria, yet a stag rarely fights to the death and a lizard population can cycle like a game of rock–paper–scissors. Evolutionary game theory explains how: strategies that earn more leave more copies, and over many generations the population settles into patterns that look like rational play. Here you’ll release a flock of hawks and doves and watch them find a balance, grow a living kaleidoscope of cooperators and defectors on a grid, and follow three lizard types chasing each other around a triangle. The same ideas describe how people and firms learn by copying what works.
Worlds
Population
Live numbers
Missions
What you’re seeing
The contest. Animals meet in pairs over a prize worth V, such as food or a nest site. A hawk always fights; a dove displays, and retreats if attacked. Two hawks fight until one is hurt: each wins half the time, so each expects (V − C)/2, where C is the cost of losing. A hawk takes everything from a dove. Two doves share, V/2 each.
Selection. A strategy that earns more than the population average spreads; one that earns less shrinks. In the “very large” mode, the share of hawks follows the replicator equation exactly. In the “200 birds” mode, each bird meets a few random others each round, then sometimes compares itself with a random bird and copies it if that bird did better. Chance makes the share jitter, but it follows the same path on average.
The payoff chart. The red line is what a hawk earns, the blue line what a dove earns, for each share of hawks. Where red is above blue, hawks spread (arrows point right); where blue is above, doves spread. Where they cross, the population stops changing.
Evolutionarily stable strategies
In 1973 John Maynard Smith and George Price asked why animal contests are so often “limited wars”, with displays instead of fights to the death. Their answer used a new idea: an evolutionarily stable strategy (ESS), a strategy that, once everyone uses it, can’t be invaded by a small group of mutants playing something else. Write u(a, b) for the payoff of strategy a against b.
The first condition says (σ, σ) is a Nash equilibrium of the game played against yourself, so every ESS is a Nash equilibrium. The second is a tie-breaker that makes it stricter: not every Nash equilibrium is an ESS.
Hawk–dove. If V ≥ C, hawk is the ESS: fighting is worth it even against other hawks. If V < C, neither pure strategy is stable. Among doves, a hawk wins every contest (V > V/2). Among hawks, a dove avoids the costly fights (0 > (V − C)/2). The stable state is a mix in which the two earn the same:
With V = 2 and C = 4, the ESS has x* = 2/4 = 50% hawks. At that mix a hawk earns 0.5 × (2 − 4)/2 + 0.5 × 2 = 0.5, and a dove earns 0.5 × 2/2 = 0.5. Equal, as they must be. But a world of doves would give everyone V/2 = 1, twice as much. The stable outcome wastes half the prize on fighting, much like the prisoner’s dilemma in Chapter 2. A population of doves would be better off, but it can’t keep hawks out.
The replicator equation
Peter Taylor and Leo Jonker wrote down the dynamics in 1978, and Peter Schuster and Karl Sigmund named them in 1983. If xi is the share of strategy i and A the payoff matrix, each share grows at a rate equal to how much better than average it does:
For hawks and doves it becomes dx/dt = x(1 − x)(V − Cx)/2. It has three rest points, where nothing changes: all doves (x = 0), all hawks (x = 1) and the mix x* = V/C. A rest point is stable if the population returns after a small push. Here the slope of dx/dt at x* is −x*(1 − x*)C/2 < 0, so pushes die out; at 0 and 1 the slope is positive, so a few mutants grow.
With V = 2 and C = 4, the slope at x* = 0.5 is −0.5 × 0.5 × 4/2 = −0.5. A small gap from the ESS shrinks like e−0.5t: it halves every ln 2/0.5 ≈ 1.4 time units. With C = 12 (and x* = 1/6) the slope is −(1/6)(5/6)(6) ≈ −0.83, so the population snaps back faster.
How do these dynamics relate to Nash equilibrium? Results often called the folk theorem of evolutionary game theory (Josef Hofbauer and Karl Sigmund collect them in their 1998 book) say, roughly: every Nash equilibrium is a rest point; every stable rest point is a Nash equilibrium; if a path that starts with every strategy present converges, it converges to a Nash equilibrium; and every ESS is asymptotically stable. The converse fails: some rest points (like all doves) are not equilibria, and some Nash equilibria are unstable.
Learning in games. The same mathematics describes people. The “200 birds” mode uses imitation: copy a random other player with a probability proportional to how much better they did. Karl Schlag showed in 1998 that this rule produces the replicator dynamics on average. Other learning rules, like fictitious play (George Brown, 1951), where each player best-responds to the average of what others have done so far, lead to equilibria in many games but can cycle in others; Drew Fudenberg and David Levine’s The Theory of Learning in Games (1998) surveys them.
Selection compares strategies with each other, not with what would be best for the group. In hawk–dove with costly fights, an all-dove population earns the most, yet any hawk born into it thrives. The stable mix wastes part of the prize on fighting. Evolution finds equilibria, and equilibria can be inefficient.
Cooperation on a grid
In a well-mixed population, the prisoner’s dilemma ends with everyone defecting: defectors always earn more than the cooperators they meet. In 1992 Martin Nowak and Robert May asked what happens if players interact only with their neighbors. Each cell of a grid is a cooperator or a defector. Each plays the dilemma with its 8 neighbors and itself (two cooperators get 1, a defector who meets a cooperator gets the temptation b while the cooperator gets 0, two defectors get 0), and then copies the strategy of the highest scorer around it.
Clusters of cooperators protect each other: a cooperator inside a cluster earns a lot from its cooperative neighbors, and at the edges, cooperators and defectors trade ground. Nowak and May found that the two strategies can coexist forever over a wide range of b, in patterns that never settle. Starting from a single defector with b between 1.8 and 2, the grid grows a symmetric kaleidoscope. In our runs on a 100 × 100 grid starting with 10% defectors, the average share of cooperators was 98% at b = 1.1, 86% at 1.5, about 31% at 1.85 and 1.95, and 4% at 2.2.
Rock–paper–scissors in nature
Some games have no stable resting point at all. In rock–paper–scissors, every strategy is beaten by another. Barry Sinervo and Curt Lively reported in 1996 that males of the side-blotched lizard (Uta stansburiana) in a California population play this game with their throat colors. Orange-throated males are aggressive and hold large territories with many females. Blue-throated males guard a smaller territory and their mates closely. Yellow-throated males hold no territory and sneak matings, looking much like females. Orange beats blue by taking over territory; yellow beats orange, whose large territories can’t be watched everywhere; blue beats yellow by guarding its mates. They reported that the frequencies of the three types cycled over the years of the study.
With a win worth exactly what a loss costs, the replicator equation has a hidden conserved quantity: the product x1x2x3 never changes, so every path is a closed loop around the center. When wins count for more, paths spiral in to the even mix (1/3, 1/3, 1/3); when losses hurt more, they spiral out toward the edges, where one type nearly vanishes before it comes back. A similar three-way game has been demonstrated in the lab with strains of E. coli bacteria (Benjamin Kerr and colleagues, 2002), where the three strains coexisted when they lived on a plate and interacted locally, but not when they were mixed in a flask.
A little history
John Maynard Smith (1920–2004) trained as an aircraft engineer before becoming a biologist. With George Price he published “The Logic of Animal Conflict” in Nature in 1973, introducing the evolutionarily stable strategy. His book Evolution and the Theory of Games (1982) used the hawk–dove game to explain it and brought the field to a wide audience.
George Price (1922–1975) was an American chemist and science writer who moved to London and taught himself population genetics; the Price equation (1970) is still a cornerstone of evolutionary theory.
Peter Taylor and Leo Jonker introduced the replicator dynamics in Mathematical Biosciences in 1978. Martin Nowak and Robert May published “Evolutionary Games and Spatial Chaos” in Nature in 1992.
Where it shows up
- Animal behavior. Ritualized contests, territorial displays and alternative mating strategies like the lizards’ are classic applications.
- Microbes. Bacteria that produce costly shared resources face a public-goods dilemma; “cheater” strains that don’t contribute can invade, much like hawks among doves.
- Business. Firms copy practices that seem to work for their rivals, an imitation dynamic. Aggressive pricing can spread through an industry the way hawks spread, even when every firm would earn more if all were less aggressive.
- Social norms. Conventions such as which side of the road to drive on are stable states of coordination games: once most people follow one, it pays everyone to follow it.
Practice
1. With V = 6 and C = 10, what share of hawks is evolutionarily stable, and what does everyone earn there?
Show the answer
x* = 6/10 = 60%. A dove earns (1 − 0.6) × 6/2 = 1.2, and so does a hawk: 0.6 × (6 − 10)/2 + 0.4 × 6 = −1.2 + 2.4 = 1.2. An all-dove world would pay 3.
2. Show that “always dove” is not an ESS when V > 0.
Show the answer
Against doves, a dove earns V/2 but a hawk earns V, so u(H, D) > u(D, D). The first ESS condition fails: a few hawks would spread.
3. With V = 10 and C = 4, find the ESS and compare it with an all-dove world.
Show the answer
Hawk is a strict best reply to itself: (10 − 4)/2 = 3 > 0, so all hawks is the ESS. Everyone earns 3, against 5 in an all-dove world. The game is a prisoner’s dilemma.
4. Derive dx/dt = x(1 − x)(V − Cx)/2 from the replicator equation, and show that x* = V/C is stable.
Show the answer
With two strategies, dx/dt = x(πH − π̄) and π̄ = xπH + (1 − x)πD, so dx/dt = x(1 − x)(πH − πD). Here πH − πD = x(V − C)/2 + (1 − x)V − (1 − x)V/2 = (V − Cx)/2. The derivative of the right side at x* is x*(1 − x*)(−C/2) < 0, so small deviations shrink.
5. In the coordination game where matching on A pays 2, matching on B pays 1 and mismatching pays 0, which strategies are ESS? Is the mixed equilibrium an ESS?
Show the answer
Both pure strategies are strict Nash equilibria (2 > 0 and 1 > 0), so both are ESS. The mixed equilibrium plays A with probability 1/3 (where 2p = 1 − p). It isn’t an ESS: a mutant playing pure A ties against it and does better against itself than the mixed strategy does against A. Under the replicator dynamics, 1/3 is the unstable boundary between the two basins: start with more than a third playing A and everyone ends up on A.
6. In rock–paper–scissors with win = 1 and loss = −1, show that x1x2x3 stays constant under the replicator equation.
Show the answer
d/dt ln(x1x2x3) = Σi [(Ax)i − x⊤Ax] = Σi(Ax)i − 3x⊤Ax. The game is zero-sum, so A is antisymmetric and x⊤Ax = 0; and each column of A adds to 1 − 1 = 0, so Σi(Ax)i = 0. Our test found the product changed by about 1 part in 1012 over 200 time units.
7. (Harder) On the Nowak–May grid, a single defector sits among cooperators. For which temptations b do its 8 neighbors become defectors in the next generation?
Show the answer
The defector meets 8 cooperators (and itself, a defector): score 8b. A neighboring cooperator meets 7 cooperators, itself, and the defector: score 8. A cooperator two steps away sees 9 cooperators: score 9. The neighbor copies the defector only if 8b beats every other score in its block, including the 9 of the cooperators beside it: 8b > 9, so b > 9/8 = 1.125. Our simulation agrees: at b = 1.12 the defector stays alone, at 1.13 it becomes a 3 × 3 block.
How the interactive works
Hawks and doves. The “very large” mode integrates dx/dt = x(1 − x)(V − Cx)/2 with the fourth-order Runge–Kutta method. The birds on screen are colored to match the share. In the “200 birds” mode, each round every bird meets 4 random others; then a quarter of the birds each look at one random bird and copy it with probability equal to the payoff difference divided by the largest possible difference. The birds’ flight is decoration.
The grid wraps around at the edges and updates every cell at once. Rock–paper–scissors integrates the replicator equation with Runge–Kutta in small steps, renormalizing the shares to add to 1.
What’s simplified. Real animals don’t reproduce in lockstep, payoffs aren’t fixed numbers, and strategies in nature are rarely pure. The lizard picture shows the mathematics of a rock–paper–scissors cycle, not fitted field data.
How it was checked (Node.js tests, run before the page was built):
- Hawks and doves earn the same at x = V/C for (2, 4), (4, 10) and (3, 12). From five starting shares between 1% and 99%, the replicator equation reached 0.5 for V = 2, C = 4 with an error below 10−14; with V = 6 > C = 4 hawks reached 100%. Rest points 0 and 1 are unstable, V/C stable.
- The mix V/C passed both ESS conditions against 100 mutant mixes.
- 400 imitating animals with V = 2, C = 4 averaged 50.2% hawks over 2,500 rounds.
- Grid: one defector at the center of a 99 × 99 grid with b = 1.85 stayed perfectly symmetric for 30 generations; the threshold for a lone defector to spread is between 1.12 and 1.13, matching 9/8.
- Rock–paper–scissors: with win = loss, x1x2x3 drifted by 1.3 × 10−12 (relative) over 200 time units; with win 1.3 the shares reached 1/3 each; with win 0.7 the smallest share fell below 10−9.
Check yourself
With V = 3 and C = 12, what share of hawks is stable?
Show the answer
V/C = 3/12 = 25%.
What does it mean for a strategy to be evolutionarily stable?
Show the answer
An ESS resists invasion: mutants either do worse against the residents, or tie there and do worse against each other.
In the replicator equation, a strategy’s share grows when:
Show the answer
Growth rate = own payoff minus average payoff. A strategy with a negative payoff can still grow if others do even worse.
Why can cooperators survive in Nowak and May’s grid but not in a well-mixed population?
Show the answer
Local interaction lets cooperators enjoy each other’s company; in a mixed population, defectors meet cooperators at random and always earn more.
In rock–paper–scissors with wins worth exactly as much as losses cost, what does the population do?
Show the answer
The product of the three shares is conserved, so paths are closed loops.
Sources
- M. J. Osborne, An Introduction to Game Theory (Oxford University Press, 2004), chapter 13, “Evolutionary Equilibrium”.
- J. Maynard Smith and G. R. Price, “The Logic of Animal Conflict”, Nature 246, 15–18 (1973); J. Maynard Smith, Evolution and the Theory of Games (Cambridge University Press, 1982).
- P. D. Taylor and L. B. Jonker, “Evolutionarily Stable Strategies and Game Dynamics”, Mathematical Biosciences 40(1–2), 145–156 (1978); J. Hofbauer and K. Sigmund, Evolutionary Games and Population Dynamics (Cambridge University Press, 1998).
- K. H. Schlag, “Why Imitate, and If So, How?”, Journal of Economic Theory 78(1), 130–156 (1998); D. Fudenberg and D. K. Levine, The Theory of Learning in Games (MIT Press, 1998).
- M. A. Nowak and R. M. May, “Evolutionary Games and Spatial Chaos”, Nature 359, 826–829 (1992); Yale University, Spatial Prisoner’s Dilemma (Fractal Geometry course pages): the rules and the color scheme.
- B. Sinervo and C. M. Lively, “The Rock–Paper–Scissors Game and the Evolution of Alternative Male Strategies”, Nature 380, 240–243 (1996).
- B. Kerr, M. A. Riley, M. W. Feldman and B. J. M. Bohannan, “Local Dispersal Promotes Biodiversity in a Real-Life Game of Rock–Paper–Scissors”, Nature 418, 171–174 (2002).
- MIT OpenCourseWare, 14.12 Economic Applications of Game Theory.
- Wikipedia (secondary): Evolutionarily stable strategy, Replicator equation, Common side-blotched lizard, George R. Price.