Nash equilibrium

A strategy profile is a set of strategies, one for each player. Informally, a strategy profile is a Nash equilibrium if no player can do better by unilaterally changing his strategy. To see what this means, imagine that each player is told the strategies of the others. Suppose then that each player asks himself: "Knowing the strategies of the other players, and treating the strategies of the other players as set in stone, can I benefit by changing my strategy?"

If any player could answer "Yes", then that set of strategies is not a Nash equilibrium. But if every player prefers not to switch (or is indifferent between switching and not) then the strategy profile is a Nash equilibrium. Thus, each strategy in a Nash equilibrium is a best response to the other players' strategies in that equilibrium.[14]

Formally, let ${\displaystyle S_{i}}$ be the set of all possible strategies for player ${\displaystyle i}$, where ${\displaystyle i=1,\ldots N}$. Let ${\displaystyle s^{*}=(s_{i}^{*},s_{-i}^{*})}$ be a strategy profile, a set consisting of one strategy for each player, where ${\displaystyle s_{-i}^{*}}$ denotes the ${\displaystyle N-1}$ strategies of all the players except ${\displaystyle i}$. Let ${\displaystyle u_{i}(s_{i},s_{-i}^{*})}$ be player i's payoff as a function of the strategies. The strategy profile ${\displaystyle s^{*}}$ is a Nash equilibrium if[15]

${\displaystyle u_{i}(s_{i}^{*},s_{-i}^{*})\geq u_{i}(s_{i},s_{-i}^{*})\;\;{\rm {for\;all}}\;\;s_{i}\in S_{i}}$

A game can have more than one Nash equilibrium. Even if the equilibrium is unique, it might be weak: a player might be indifferent among several strategies given the other players' choices. It is unique and called a strict Nash equilibrium if the inequality is strict so one strategy is the unique best response:

${\displaystyle u_{i}(s_{i}^{*},s_{-i}^{*})>u_{i}(s_{i},s_{-i}^{*})\;\;{\rm {for\;all}}\;\;s_{i}\in S_{i}}$

Note that the strategy set ${\displaystyle S_{i}}$ can be different for different players, and its elements can be a variety of mathematical objects. Most simply, a player might choose between two strategies, e.g. ${\displaystyle S_{i}=\{Yes,No\}.}$ Or, the strategy set might be a finite set of conditional strategies responding to other players, e.g. ${\displaystyle S_{i}=\{Yes|p=Low,No|p=High\}.}$ Or, it might be an infinite set, a continuum or unbounded, e.g. ${\displaystyle S_{i}=\{Price\}}$ such that ${\displaystyle Price}$ is a non-negative real number. Nash's existence proofs assume a finite strategy set, but the concept of Nash equilibrium does not require it.

The Nash equilibrium may sometimes appear non-rational in a third-person perspective. This is because a Nash equilibrium is not necessarily Pareto optimal.

Nash equilibrium may also have non-rational consequences in sequential games because players may "threaten" each other with threats they would not actually carry out. For such games the subgame perfect Nash equilibrium may be more meaningful as a tool of analysis.

Suppose that in the Nash equilibrium, each player asks themselves: "Knowing the strategies of the other players, and treating the strategies of the other players as set in stone, would I suffer a loss by changing my strategy?"

If every player's answer is "Yes", then the equilibrium is classified as a strict Nash equilibrium.[16]

If instead, for some player, there is exact equality between the strategy in Nash equilibrium and some other strategy that gives exactly the same payout (i.e. this player is indifferent between switching and not), then the equilibrium is classified as a weak Nash equilibrium.

A game can have a pure-strategy or a mixed-strategy Nash equilibrium. (In the latter a pure strategy is chosen stochastically with a fixed probability).

Nash proved that if mixed strategies (where a player chooses probabilities of using various pure strategies) are allowed, then every game with a finite number of players in which each player can choose from finitely many pure strategies has at least one Nash equilibrium, which might be a pure strategy for each player or might be a probability distribution over strategies for each player.

Nash equilibria need not exist if the set of choices is infinite and non-compact. An example is a game where two players simultaneously name a number and the player naming the larger number wins. Another example is where each of two players chooses a real number strictly less than 5 and the winner is whoever has the biggest number; no biggest number strictly less than 5 exists (if the number could equal 5, the Nash equilibrium would have both players choosing 5 and tying the game). However, a Nash equilibrium exists if the set of choices is compact with each player's payoff continuous in the strategies of all the players.[17]

The coordination game is a classic (symmetric) two player, two strategy game, with an example payoff matrix shown to the right. The players should thus coordinate, both adopting strategy A, to receive the highest payoff; i.e., 4. If both players chose strategy B though, there is still a Nash equilibrium. Although each player is awarded less than optimal payoff, neither player has incentive to change strategy due to a reduction in the immediate payoff (from 2 to 1).

A famous example of this type of game was called the stag hunt; in the game two players may choose to hunt a stag or a rabbit, the former providing more meat (4 utility units) than the latter (1 utility unit). The caveat is that the stag must be cooperatively hunted, so if one player attempts to hunt the stag, while the other hunts the rabbit, she/he will fail in hunting (0 utility units), whereas if they both hunt it they will split the payoff (2, 2). The game hence exhibits two equilibria at (stag, stag) and (rabbit, rabbit) and hence the players' optimal strategy depend on their expectation on what the other player may do. If one hunter trusts that the other will hunt the stag, they should hunt the stag; however if they suspect that the other will hunt the rabbit, they should hunt the rabbit. This game was used as an analogy for social cooperation, since much of the benefit that people gain in society depends upon people cooperating and implicitly trusting one another to act in a manner corresponding with cooperation.

Another example of a coordination game is the setting where two technologies are available to two firms with comparable products, and they have to elect a strategy to become the market standard. If both firms agree on the chosen technology, high sales are expected for both firms. If the firms do not agree on the standard technology, few sales result. Both strategies are Nash equilibria of the game.

Driving on a road against an oncoming car, and having to choose either to swerve on the left or to swerve on the right of the road, is also a coordination game. For example, with payoffs 10 meaning no crash and 0 meaning a crash, the coordination game can be defined with the following payoff matrix:

In this case there are two pure-strategy Nash equilibria, when both choose to either drive on the left or on the right. If we admit mixed strategies (where a pure strategy is chosen at random, subject to some fixed probability), then there are three Nash equilibria for the same case: two we have seen from the pure-strategy form, where the probabilities are (0%, 100%) for player one, (0%, 100%) for player two; and (100%, 0%) for player one, (100%, 0%) for player two respectively. We add another where the probabilities for each player are (50%, 50%).

Sample network graph. Values on edges are the travel time experienced by a 'car' traveling down that edge. x is the number of cars traveling via that edge.

An application of Nash equilibria is in determining the expected flow of traffic in a network. Consider the graph on the right. If we assume that there are x "cars" traveling from A to D, what is the expected distribution of traffic in the network?

This situation can be modeled as a "game" where every traveler has a choice of 3 strategies, where each strategy is a route from A to D (either ABD, ABCD, or ACD). The "payoff" of each strategy is the travel time of each route. In the graph on the right, a car travelling via ABD experiences travel time of (1+x/100)+2, where x is the number of cars traveling on edge AB. Thus, payoffs for any given strategy depend on the choices of the other players, as is usual. However, the goal, in this case, is to minimize travel time, not maximize it. Equilibrium will occur when the time on all paths is exactly the same. When that happens, no single driver has any incentive to switch routes, since it can only add to their travel time. For the graph on the right, if, for example, 100 cars are travelling from A to D, then equilibrium will occur when 25 drivers travel via ABD, 50 via ABCD, and 25 via ACD. Every driver now has a total travel time of 3.75 (to see this, note that a total of 75 cars take the AB edge, and likewise, 75 cars take the CD edge).

Notice that this distribution is not, actually, socially optimal. If the 100 cars agreed that 50 travel via ABD and the other 50 through ACD, then travel time for any single car would actually be 3.5, which is less than 3.75. This is also the Nash equilibrium if the path between B and C is removed, which means that adding another possible route can decrease the efficiency of the system, a phenomenon known as Braess's paradox.

This can be illustrated by a two-player game in which both players simultaneously choose an integer from 0 to 3 and they both win the smaller of the two numbers in points. In addition, if one player chooses a larger number than the other, then they have to give up two points to the other.

This game has a unique pure-strategy Nash equilibrium: both players choosing 0 (highlighted in light red). Any other strategy can be improved by a player switching their number to one less than that of the other player. In the adjacent table, if the game begins at the green square, it is in player 1's interest to move to the purple square and it is in player 2's interest to move to the blue square. Although it would not fit the definition of a competition game, if the game is modified so that the two players win the named amount if they both choose the same number, and otherwise win nothing, then there are 4 Nash equilibria: (0,0), (1,1), (2,2), and (3,3).

There is an easy numerical way to identify Nash equilibria on a payoff matrix. It is especially helpful in two-person games where players have more than two strategies. In this case formal analysis may become too long. This rule does not apply to the case where mixed (stochastic) strategies are of interest. The rule goes as follows: if the first payoff number, in the payoff pair of the cell, is the maximum of the column of the cell and if the second number is the maximum of the row of the cell - then the cell represents a Nash equilibrium.

We can apply this rule to a 3×3 matrix:

Using the rule, we can very quickly (much faster than with formal analysis) see that the Nash equilibria cells are (B,A), (A,B), and (C,C). Indeed, for cell (B,A) 40 is the maximum of the first column and 25 is the maximum of the second row. For (A,B) 25 is the maximum of the second column and 40 is the maximum of the first row. Same for cell (C,C). For other cells, either one or both of the duplet members are not the maximum of the corresponding rows and columns.

This said, the actual mechanics of finding equilibrium cells is obvious: find the maximum of a column and check if the second member of the pair is the maximum of the row. If these conditions are met, the cell represents a Nash equilibrium. Check all columns this way to find all NE cells. An N×N matrix may have between 0 and N×N pure-strategy Nash equilibria.

Examples of game theory problems in which these conditions are not met:

1. The first condition is not met if the game does not correctly describe the quantities a player wishes to maximize. In this case there is no particular reason for that player to adopt an equilibrium strategy. For instance, the prisoner's dilemma is not a dilemma if either player is happy to be jailed indefinitely.
2. Intentional or accidental imperfection in execution. For example, a computer capable of flawless logical play facing a second flawless computer will result in equilibrium. Introduction of imperfection will lead to its disruption either through loss to the player who makes the mistake, or through negation of the common knowledge criterion leading to possible victory for the player. (An example would be a player suddenly putting the car into reverse in the game of chicken, ensuring a no-loss no-win scenario).
3. In many cases, the third condition is not met because, even though the equilibrium must exist, it is unknown due to the complexity of the game, for instance in Chinese chess.[21] Or, if known, it may not be known to all players, as when playing tic-tac-toe with a small child who desperately wants to win (meeting the other criteria).
4. The criterion of common knowledge may not be met even if all players do, in fact, meet all the other criteria. Players wrongly distrusting each other's rationality may adopt counter-strategies to expected irrational play on their opponents’ behalf. This is a major consideration in "chicken" or an arms race, for example.

In his Ph.D. dissertation, John Nash proposed two interpretations of his equilibrium concept, with the objective of showing how equilibrium points can be connected with observable phenomenon.

(...) One interpretation is rationalistic: if we assume that players are rational, know the full structure of the game, the game is played just once, and there is just one Nash equilibrium, then players will play according to that equilibrium.

This idea was formalized by Aumann, R. and A. Brandenburger, 1995, Epistemic Conditions for Nash Equilibrium, Econometrica, 63, 1161-1180 who interpreted each player's mixed strategy as a conjecture about the behaviour of other players and have shown that if the game and the rationality of players is mutually known and these conjectures are commonly know, then the conjectures must be a Nash equilibrium (a common prior assumption is needed for this result in general, but not in the case of two players. In this case, the conjectures need only be mutually known).

A second interpretation, that Nash referred to by the mass action interpretation, is less demanding on players:

[i]t is unnecessary to assume that the participants have full knowledge of the total structure of the game, or the ability and inclination to go through any complex reasoning processes. What is assumed is that there is a population of participants for each position in the game, which will be played throughout time by participants drawn at random from the different populations. If there is a stable average frequency with which each pure strategy is employed by the average member of the appropriate population, then this stable average frequency constitutes a mixed strategy Nash equilibrium.

For a formal result along these lines, see Kuhn, H. and et al., 1996, "The Work of John Nash in Game Theory," Journal of Economic Theory, 69, 153–185.

Due to the limited conditions in which NE can actually be observed, they are rarely treated as a guide to day-to-day behaviour, or observed in practice in human negotiations. However, as a theoretical concept in economics and evolutionary biology, the NE has explanatory power. The payoff in economics is utility (or sometimes money), and in evolutionary biology is gene transmission; both are the fundamental bottom line of survival. Researchers who apply games theory in these fields claim that strategies failing to maximize these for whatever reason will be competed out of the market or environment, which are ascribed the ability to test all strategies. This conclusion is drawn from the "stability" theory above. In these situations the assumption that the strategy observed is actually a NE has often been borne out by research.[22]

Nash's original proof (in his thesis) used Brouwer's fixed-point theorem (e.g., see below for a variant). We give a simpler proof via the Kakutani fixed-point theorem, following Nash's 1950 paper (he credits David Gale with the observation that such a simplification is possible).

To prove the existence of a Nash equilibrium, let ${\displaystyle r_{i}(\sigma _{-i})}$ be the best response of player i to the strategies of all other players.

${\displaystyle r_{i}(\sigma _{-i})=\mathop {\underset {\sigma _{i}}{\operatorname {arg\,max} }} u_{i}(\sigma _{i},\sigma _{-i})}$

Here, ${\displaystyle \sigma \in \Sigma }$, where ${\displaystyle \Sigma =\Sigma _{i}\times \Sigma _{-i}}$, is a mixed-strategy profile in the set of all mixed strategies and ${\displaystyle u_{i}}$ is the payoff function for player i. Define a set-valued function ${\displaystyle r\colon \Sigma \rightarrow 2^{\Sigma }}$ such that ${\displaystyle r=r_{i}(\sigma _{-i})\times r_{-i}(\sigma _{i})}$. The existence of a Nash equilibrium is equivalent to ${\displaystyle r}$ having a fixed point.

Kakutani's fixed point theorem guarantees the existence of a fixed point if the following four conditions are satisfied.

1. ${\displaystyle \Sigma }$ is compact, convex, and nonempty.
2. ${\displaystyle r(\sigma )}$ is nonempty.
3. ${\displaystyle r(\sigma )}$ is upper hemicontinuous
4. ${\displaystyle r(\sigma )}$ is convex.

Condition 1. is satisfied from the fact that ${\displaystyle \Sigma }$ is a simplex and thus compact. Convexity follows from players' ability to mix strategies. ${\displaystyle \Sigma }$ is nonempty as long as players have strategies.

Condition 2. and 3. are satisfied by way of Berge's maximum theorem. Because ${\displaystyle u_{i}}$ is continuous and compact, ${\displaystyle r(\sigma _{i})}$ is non-empty and upper hemicontinuous.

Condition 4. is satisfied as a result of mixed strategies. Suppose ${\displaystyle \sigma _{i},\sigma '_{i}\in r(\sigma _{-i})}$, then ${\displaystyle \lambda \sigma _{i}+(1-\lambda )\sigma '_{i}\in r(\sigma _{-i})}$. i.e. if two strategies maximize payoffs, then a mix between the two strategies will yield the same payoff.

Therefore, there exists a fixed point in ${\displaystyle r}$ and a Nash equilibrium.[23]

When Nash made this point to John von Neumann in 1949, von Neumann famously dismissed it with the words, "That's trivial, you know. That's just a fixed-point theorem." (See Nasar, 1998, p. 94.)

We have a game ${\displaystyle G=(N,A,u)}$ where ${\displaystyle N}$ is the number of players and ${\displaystyle A=A_{1}\times \cdots \times A_{N}}$ is the action set for the players. All of the action sets ${\displaystyle A_{i}}$ are finite. Let ${\displaystyle \Delta =\Delta _{1}\times \cdots \times \Delta _{N}}$ denote the set of mixed strategies for the players. The finiteness of the ${\displaystyle A_{i}}$s ensures the compactness of ${\displaystyle \Delta }$.

We can now define the gain functions. For a mixed strategy ${\displaystyle \sigma \in \Delta }$, we let the gain for player ${\displaystyle i}$ on action ${\displaystyle a\in A_{i}}$ be

${\displaystyle {\text{Gain}}_{i}(\sigma ,a)=\max\{0,u_{i}(a,\sigma _{-i})-u_{i}(\sigma _{i},\sigma _{-i})\}.}$

The gain function represents the benefit a player gets by unilaterally changing their strategy. We now define ${\displaystyle g=(g_{1},\dotsc ,g_{N})}$ where

${\displaystyle g_{i}(\sigma )(a)=\sigma _{i}(a)+{\text{Gain}}_{i}(\sigma ,a)}$

for ${\displaystyle \sigma \in \Delta ,a\in A_{i}}$. We see that

${\displaystyle \sum _{a\in A_{i}}g_{i}(\sigma )(a)=\sum _{a\in A_{i}}\sigma _{i}(a)+{\text{Gain}}_{i}(\sigma ,a)=1+\sum _{a\in A_{i}}{\text{Gain}}_{i}(\sigma ,a)>0.}$

Next we define:

${\displaystyle {\begin{cases}f=(f_{1},\cdots ,f_{N}):\Delta \to \Delta \\f_{i}(\sigma )(a)={\frac {g_{i}(\sigma )(a)}{\sum _{b\in A_{i}}g_{i}(\sigma )(b)}}&a\in A_{i}\end{cases}}}$

It is easy to see that each ${\displaystyle f_{i}}$ is a valid mixed strategy in ${\displaystyle \Delta _{i}}$. It is also easy to check that each ${\displaystyle f_{i}}$ is a continuous function of ${\displaystyle \sigma }$, and hence ${\displaystyle f}$ is a continuous function. As the cross product of a finite number of compact convex sets, ${\displaystyle \Delta }$ is also compact and convex. Applying the Brouwer fixed point theorem to ${\displaystyle f}$ and ${\displaystyle \Delta }$ we conclude that ${\displaystyle f}$ has a fixed point in ${\displaystyle \Delta }$, call it ${\displaystyle \sigma ^{*}}$. We claim that ${\displaystyle \sigma ^{*}}$ is a Nash equilibrium in ${\displaystyle G}$. For this purpose, it suffices to show that

${\displaystyle \forall i\in \{1,\cdots ,N\},\forall a\in A_{i}:\quad {\text{Gain}}_{i}(\sigma ^{*},a)=0.}$

This simply states that each player gains no benefit by unilaterally changing their strategy, which is exactly the necessary condition for a Nash equilibrium.

Now assume that the gains are not all zero. Therefore, ${\displaystyle \exists i\in \{1,\cdots ,N\},}$ and ${\displaystyle a\in A_{i}}$ such that ${\displaystyle {\text{Gain}}_{i}(\sigma ^{*},a)>0}$. Note then that

${\displaystyle \sum _{a\in A_{i}}g_{i}(\sigma ^{*},a)=1+\sum _{a\in A_{i}}{\text{Gain}}_{i}(\sigma ^{*},a)>1.}$

So let

${\displaystyle C=\sum _{a\in A_{i}}g_{i}(\sigma ^{*},a).}$

Also we shall denote ${\displaystyle {\text{Gain}}(i,\cdot )}$ as the gain vector indexed by actions in ${\displaystyle A_{i}}$. Since ${\displaystyle \sigma ^{*}}$ is the fixed point we have:

${\displaystyle {\begin{aligned}\sigma ^{*}=f(\sigma ^{*})&\Rightarrow \sigma _{i}^{*}=f_{i}(\sigma ^{*})\\&\Rightarrow \sigma _{i}^{*}={\frac {g_{i}(\sigma ^{*})}{\sum _{a\in A_{i}}g_{i}(\sigma ^{*})(a)}}\\[6pt]&\Rightarrow \sigma _{i}^{*}={\frac {1}{C}}\left(\sigma _{i}^{*}+{\text{Gain}}_{i}(\sigma ^{*},\cdot )\right)\\[6pt]&\Rightarrow C\sigma _{i}^{*}=\sigma _{i}^{*}+{\text{Gain}}_{i}(\sigma ^{*},\cdot )\\&\Rightarrow \left(C-1\right)\sigma _{i}^{*}={\text{Gain}}_{i}(\sigma ^{*},\cdot )\\&\Rightarrow \sigma _{i}^{*}=\left({\frac {1}{C-1}}\right){\text{Gain}}_{i}(\sigma ^{*},\cdot ).\end{aligned}}}$

Since ${\displaystyle C>1}$ we have that ${\displaystyle \sigma _{i}^{*}}$ is some positive scaling of the vector ${\displaystyle {\text{Gain}}_{i}(\sigma ^{*},\cdot )}$. Now we claim that

${\displaystyle \forall a\in A_{i}:\quad \sigma _{i}^{*}(a)(u_{i}(a_{i},\sigma _{-i}^{*})-u_{i}(\sigma _{i}^{*},\sigma _{-i}^{*}))=\sigma _{i}^{*}(a){\text{Gain}}_{i}(\sigma ^{*},a)}$

To see this, we first note that if ${\displaystyle {\text{Gain}}_{i}(\sigma ^{*},a)>0}$ then this is true by definition of the gain function. Now assume that ${\displaystyle {\text{Gain}}_{i}(\sigma ^{*},a)=0}$. By our previous statements we have that

${\displaystyle \sigma _{i}^{*}(a)=\left({\frac {1}{C-1}}\right){\text{Gain}}_{i}(\sigma ^{*},a)=0}$

and so the left term is zero, giving us that the entire expression is ${\displaystyle 0}$ as needed.

So we finally have that

${\displaystyle {\begin{aligned}0&=u_{i}(\sigma _{i}^{*},\sigma _{-i}^{*})-u_{i}(\sigma _{i}^{*},\sigma _{-i}^{*})\\&=\left(\sum _{a\in A_{i}}\sigma _{i}^{*}(a)u_{i}(a_{i},\sigma _{-i}^{*})\right)-u_{i}(\sigma _{i}^{*},\sigma _{-i}^{*})\\&=\sum _{a\in A_{i}}\sigma _{i}^{*}(a)(u_{i}(a_{i},\sigma _{-i}^{*})-u_{i}(\sigma _{i}^{*},\sigma _{-i}^{*}))\\&=\sum _{a\in A_{i}}\sigma _{i}^{*}(a){\text{Gain}}_{i}(\sigma ^{*},a)&&{\text{ by the previous statements }}\\&=\sum _{a\in A_{i}}\left(C-1\right)\sigma _{i}^{*}(a)^{2}>0\end{aligned}}}$

where the last inequality follows since ${\displaystyle \sigma _{i}^{*}}$ is a non-zero vector. But this is a clear contradiction, so all the gains must indeed be zero. Therefore, ${\displaystyle \sigma ^{*}}$ is a Nash equilibrium for ${\displaystyle G}$ as needed.

In the matching pennies game, player A loses a point to B if A and B play the same strategy and wins a point from B if they play different strategies. To compute the mixed-strategy Nash equilibrium, assign A the probability p of playing H and (1−p) of playing T, and assign B the probability q of playing H and (1−q) of playing T.

E[payoff for A playing H] = (−1)q + (+1)(1−q) = 1−2q

E[payoff for A playing T] = (+1)q + (−1)(1−q) = 2q−1

E[payoff for A playing H] = E[payoff for A playing T] ⇒ 1−2q = 2q−1 ⇒ q = 1/2

E[payoff for B playing H] = (+1)p + (−1)(1−p) = 2p−1

E[payoff for B playing T] = (−1)p + (+1)(1−p) = 1−2p

E[payoff for B playing H] = E[payoff for B playing T] ⇒ 2p−1 = 1−2p ⇒ p = 1/2

Thus a mixed-strategy Nash equilibrium, in this game, is for each player to randomly choose H or T with p = 1/2 and q = 1/2.