Gambling and information theory

A bit is the amount of entropy in a bettable event with two possible outcomes and even odds. Obviously we could double our money if we knew beforehand for certain what the outcome of that event would be. Kelly's insight was that no matter how complicated the betting scenario is, we can use an optimum betting strategy, called the Kelly criterion, to make our money grow exponentially with whatever side information we are able to obtain. The value of this "illicit" side information is measured as mutual information relative to the outcome of the betable event:

${\displaystyle {\begin{aligned}I(X;Y)&=\mathbb {E} _{Y}\{D_{\mathrm {KL} }{\big (}P(X|Y)\|P(X|I){\big )}\}\\&=\mathbb {E} _{Y}\{D_{\mathrm {KL} }{\big (}P(X|{\textrm {side}}\ {\textrm {information}}\ Y)\|P(X|{\textrm {stated}}\ {\textrm {odds}}\ I){\big )}\},\end{aligned}}}$

where Y is the side information, X is the outcome of the betable event, and I is the state of the bookmaker's knowledge. This is the average Kullback–Leibler divergence, or information gain, of the a posteriori probability distribution of X given the value of Y relative to the a priori distribution, or stated odds, on X. Notice that the expectation is taken over Y rather than X: we need to evaluate how accurate, in the long term, our side information Y is before we start betting real money on X. This is a straightforward application of Bayesian inference. Note that the side information Y might affect not just our knowledge of the event X but also the event itself. For example, Y might be a horse that had too many oats or not enough water. The same mathematics applies in this case, because from the bookmaker's point of view, the occasional race fixing is already taken into account when he makes his odds.

The nature of side information is extremely finicky. We have already seen that it can affect the actual event as well as our knowledge of the outcome. Suppose we have an informer, who tells us that a certain horse is going to win. We certainly do not want to bet all our money on that horse just upon a rumor: that informer may be betting on another horse, and may be spreading rumors just so he can get better odds himself. Instead, as we have indicated, we need to evaluate our side information in the long term to see how it correlates with the outcomes of the races. This way we can determine exactly how reliable our informer is, and place our bets precisely to maximize the expected logarithm of our capital according to the Kelly criterion. Even if our informer is lying to us, we can still profit from his lies if we can find some reverse correlation between his tips and the actual race results.

Doubling rate in gambling on a horse race is [3]

${\displaystyle W(b,p)=\mathbb {E} [\log _{2}S(X)]=\sum _{i=1}^{m}p_{i}\log _{2}b_{i}o_{i}}$

where there are ${\displaystyle m}$ horses, the probability of the ${\displaystyle i}$th horse winning being ${\displaystyle p_{i}}$, the proportion of wealth bet on the horse being ${\displaystyle b_{i}}$, and the odds (payoff) being ${\displaystyle o_{i}}$ (e.g., ${\displaystyle o_{i}=2}$ if the ${\displaystyle i}$th horse winning pays double the amount bet). This quantity is maximized by proportional (Kelly) gambling:

${\displaystyle b=p\,}$

for which

${\displaystyle \max _{b}W(b,p)=\sum _{i}p_{i}\log _{2}o_{i}-H(p)\,}$

where ${\displaystyle H(p)}$ is information entropy.

An important but simple relation exists between the amount of side information a gambler obtains and the expected exponential growth of his capital (Kelly):

${\displaystyle \mathbb {E} \log K_{t}=\log K_{0}+\sum _{i=1}^{t}H_{i}}$

for an optimal betting strategy, where ${\displaystyle K_{0}}$ is the initial capital, ${\displaystyle K_{t}}$ is the capital after the tth bet, and ${\displaystyle H_{i}}$ is the amount of side information obtained concerning the ith bet (in particular, the mutual information relative to the outcome of each betable event). This equation applies in the absence of any transaction costs or minimum bets. When these constraints apply (as they invariably do in real life), another important gambling concept comes into play: the gambler (or unscrupulous investor) must face a certain probability of ultimate ruin, which is known as the gambler's ruin scenario. Note that even food, clothing, and shelter can be considered fixed transaction costs and thus contribute to the gambler's probability of ultimate ruin.

This equation was the first application of Shannon's theory of information outside its prevailing paradigm of data communications (Pierce).