6. Evolutionary game theory

So far we considered games in which individual players make decisions, and the payoff to each player depends on the decisions made by all.

As we saw, a key question in game theory is to reason about the behavior we should expect to see when players take part in a given game.

In other words, players simultaneously reason about what the other players may do.

In this Section we explore the notion of evolutionary game theory, which shows that the basic ideas of game theory can be applied even to situations in which no individual is overtly reasoning, or even making explicit decisions.

Rather, game-theoretic analysis will be applied to settings in which individuals can exhibit different forms of behavior (including those that may not be the result of conscious choices).

We will consider which forms of behavior have the ability to persist in the population, and which forms of behavior have a tendency to be driven out by others.

As its name suggests, this approach has been applied most widely in the area of evolutionary biology, the domain in which the idea was first articulated by the British theoretical evolutionary biologist John Maynard Smith and American population geneticist George R. Price.

Evolutionary biology is based on the idea that an organism’s genes largely determine its observable characteristics, and hence its fitness in a given environment.

Organisms that are more fit will tend to produce more offspring, causing genes that provide greater fitness to increase their representation in the population.

In this way, fitter genes tend to win over time, because they provide higher rates of reproduction.

The key insight of evolutionary game theory is that many behaviours involve the interaction of multiple organisms in a population, and the success of any one of these organisms depends on how its behaviour interacts with that of others.

So the fitness of an individual organism cannot be measured in isolation; rather it has to be evaluated in the context of the full population in which it lives.

This opens the door to a natural game-theoretic analogy: an organism’s genetically-determined characteristics and behaviors are like its strategy in a game, its fitness is like its payoff, and this payoff depends on the strategies (characteristics) of the organisms with which it interacts.

Game-theoretic ideas like equilibrium will prove to be a useful way to make predictions about the results of evolution on a population.

For an example, let us consider a particular species of beetle, and suppose that each beetle’s fitness in a given environment is determined largely by the extent to which it can find food and use the nutrients from the food effectively.

Now, suppose a particular mutation is introduced into the population, causing beetles with the mutation to grow a significantly larger body size. Thus, we now have two distinct kinds of beetles in the population — small ones and large ones.

It is actually difficult for the large beetles to maintain the metabolic requirements of their larger body size — it requires diverting more nutrients from the food they eat — and so this has a negative effect on fitness.

If this were the full story, we would conclude that the large-body-size mutation is fitness- decreasing, and so it will likely be driven out of the population over time, through multiple generations. But in fact, there is more to the story, as we will now see.

The beetles in this population compete with each other for food, when they come upon a food source, there is crowding among the beetles as they each try to get as much of the food as they can.

And, not surprisingly, the beetles with large body sizes are more effective at claiming a bigger share of the food.

Let’s assume for simplicity that food competition in this population involves two beetles interacting with each other at any given point in time.

When two beetles compete for some food, we have the following possible outcomes.

  • When beetles of the same size compete, they get equal shares of the food.
  • When a large beetle competes with a small beetle, the large beetle gets the majority of the food.
  • In all cases, large beetles experience less of a fitness benefit from a given quantity of food, since some of it is diverted into maintaining their expensive metabolism.

Thus, the fitness that each beetle gets from a given food-related interaction can be thought of as a numerical payoff in a two-player game between a first beetle and a second beetle, as follows.

The first beetle plays one of the two strategies Small or Large, depending on its body size, and the second beetle plays one of these two strategies as well.

Based on the two strategies used, the payoffs to the beetles are described as follows

Small Large
Small \((5,5)\) \((1,8)\)
Large \((8,1)\) \((3,3)\)

This payoff matrix is a nice way to summarize what happens when two beetles meet, but compared with the game considered in section 2, there is something fundamentally different in what is being described here.

The beetles in this game are not asking themselves, “What do I want my body size to be in this interaction?”

Rather, each is genetically hard-wired to play one of these two strategies through its whole lifetime. Given this important difference, the idea of choosing strategies — which was central to our formulation of game theory — is missing from the biological side of the analogy.

As a result, in place of the idea of Nash equilibrium — which was based fundamentally on the relative benefit of changing one’s own personal strategy — we will need to think about strategy changes that operate over longer time scales, taking place as shifts in a population under evolutionary forces.

We now consider a population of individuals engaged in pairwise contests.

Every single contest is represented as a game which involves two players, each has \(n\) pure strategies and the payoffs are summarized by the \(n\times n\) payoff matrix \(A\).

Note that we now assume that both players have the strategic choices, so we are looking at the game with payoff matrices \(A\) for player 1 and \(A^T\) for player 2.

This is not the same convention that we used for zero-sum games!

The players in the population are matched randomly and every game is independent of the others. The total payoff to the individual is taken as an average payoff of all the games the individual plays.

Before we move on we need to clarify how we represent the population of players.

The structure of the population is described by the density of individuals playing a particular strategy.

For any pure or mixed strategy \(\underline{x}=(x_1,\ldots,x_n)\) we let \(\delta_\underline{x}\) denote the population where a randomly selected individual plays strategy \(\underline{x}\) with probability 1.

We write \[\sum\limits_{i=1}^nx_i\delta_i=\underline{x}^T.\] The row vector \(\underline{x}\) denotes the mixed strategy (a given individual plays strategy \(i\) with probability \(x_i\)) whereas the column vector \(\underline{x}^T\) denotes the population where a randomly selected individual plays strategy \(i\) with probability \(x_i\).

Let’s consider an individual playing strategy \(\underline{\sigma}\). It plays \(k\) contests against randomly selected individuals from a population described by \(\underline{x}^T\). Then the expected total payoff after \(k\) contests is given by \[k \sum\limits_{i=1}^n x_i \mathbb E_1(\underline{\sigma},i) =k\sum\limits_{i=1}^nx_i\sum\limits_{j=1}^n \sigma_ja_{ji}\] and consequently the expected average payoff from one game is \[\mathcal{E}(\underline{\sigma},\underline{x}^T)=\sum\limits_{i=1}^nx_i\sum\limits_{j=1}^n \sigma_j a_{ji}=\underline{\sigma}A \underline{x}^T=\mathbb E_1(\underline{\sigma},\underline{x})=\sum\limits_{i=1}^nx_i\mathbb E_1(\underline{\sigma},i)\] Here \(\mathcal{E}(\underline{\sigma},\underline{x}^T)\) describes the fitness of an individual playing strategy \(\underline{\sigma}\) in a population represented by \(\underline{x}^T\).

But which strategy should we expect to find in a population?

We have already studied the Nash equilibrium which is the traditional solution concept in game theory. It is assumed that players are aware of the structure of the game and consciously try to predict the moves of their opponents and to maximize their own payoffs.

In addition, it is presumed that all the players know this. These assumptions are then used to explain why players choose Nash equilibrium strategies.

However, now the situation is different. Players do not choose a strategy among a number of alternatives, it is presumed that the players’ strategies are biologically encoded.

Individuals have no control over their strategy and need not be aware of the game. They reproduce and are subject to the forces of natural selection, with the payoffs of the game representing reproductive success (biological fitness).

It is imagined that alternative strategies of the game occasionally occur, via processes like mutation.

We consider a strategy to be an evolutionary stable strategy (ESS) if it is resistant to all alternative strategies. In other words, a population playing the ESS cannot be invaded by a different strategy.

Definition 6.1

A mixed strategy \(\underline{x}\) is an evolutionarily stable strategy (ESS) if for every \(\underline{y}\neq \underline{x}\) there is an \(\varepsilon_\underline{y}>0\) such that whenever \(0<\varepsilon<\varepsilon_\underline{y}\) we have \[\mathcal{E}(\underline{x}, (1-\varepsilon)\delta_\underline{x}+\varepsilon \delta_\underline{y})>\mathcal{E}(\underline{y}; (1-\varepsilon)\delta_\underline{x}+\varepsilon \delta_\underline{y}).\] By our earlier calculations this inequality is equivalent to \[\mathbb E_1(\underline{x}, (1-\varepsilon)\underline{x}+\varepsilon \underline{y})> \mathbb E_1(\underline{y}, (1-\varepsilon)\underline{x}+\varepsilon \underline{y}).\]

What does this definition mean?

We are assuming that our total population is almost entirely made up of individuals who play strategy \(\underline{x}\) and a tiny number of mutants (or invaders) who play strategy \(\underline{y}\).

We use \(\epsilon\) (which we assume is small) to represent the proportion of the population who play strategy \(\underline{y}\).

We then require that the expected fitness of a player playing strategy \(\underline{x}\) against a randomly chosen member of this population is greater than the expected fitness of a player playing strategy \(\underline{y}\).

That is, that a mutant performs more poorly than a non-mutant in this population.

Thus in an ESS we expect that a small number of mutants will die out in competition with the rest, and so the population will remain stable with strategy \(\underline{x}\).