The "Balls in the Hat" Game

Ranjan Bhaduri writes:

allaboutalpha.com: Welcome to AllAboutAlpha.com: By: Ranjan Bhaduri, special to AllAboutAlpha.com: The word “liquidity” gets bandied about quite a lot, but it is surprising how many portfolio managers take a naïve approach to liquidity. It is well known that one should be compensated for investing in less liquid instruments (liquidity premium), but how much? What is the value of liquidity? It is dangerous in merely trust one’s intuition on the value of liquidity. Consider the following one-person game: 

The “Balls in the Hat Game”

The game consists of a hat that contains 6 black balls and 4 white balls. The player picks balls from the hat and gains $1 for each white ball, and loses $1 for each black ball.  The selection is done without replacement. At the end of each pick, the player may choose to stop or continue. The player has the right to refuse to play (i.e. not pick any balls at all). Given these rules, and a hat containing 6 black balls and 4 white balls, would you play? (Why?)

Mathematically one can prove that there is a POSITIVE expected value (of 1/15) in playing this game, so one SHOULD play! The ability to stop any time is analogous to perfect liquidity (i.e. being able to pull out of an investment at any time without the action having an impact on the value of the investment). This value of liquidity helps overcome the imbalance between the black and white balls, and thus makes this game profitable. This is interesting from a behavioral finance point of view, since it seems to suggest that humans are wired such that they will tend to underestimate the value of liquidity.

The mathematics behind calculating the value of liquidity can be complex as there can be subtle nuances. Niall Whelan of Scotia Capital and I wrote a pair of papers coming out which tackles the above game in the asymptotic case (i.e. hats of infinite size) and connect the value of liquidity to option pricing. Niall is one of the best quants north of the South Pole and much of these papers were hammered out in an all-night bus ride that we were forced to take from NY to Toronto (our flight from La Guardia got cancelled but we both needed to be back in Toronto in the morning for important meetings)...

I suspect that this is wrong. What is going on here, I think, is not so much liquidity as mean reversion: if you sample with replacement the effect goes away. The reason that it makes sense to play even though one might at first glance think the odds are unfavorable is that if you lose in the early stages the chances of winning in the later stages go up--that the ability to keep playing provides a degree of insurance in the cases in which things break badly. (Of course, "liquidity" does play a role: if you could never stop playing that would be offset by the fact that if you won in the early stages the odds would then move away from you.)

In order to see what is going on here, let us write down the value function: V(m,n) is the expected value of playing the game (and dropping out at the optimal point) when there are m white balls and n black balls in the hat. To begin with:

V(1,0) = 1; if there is one white ball left in the hat, the value of the game is 1--you play, and collect the white ball.

V(0,1) = 0; if there is one black ball left in the hat, the value of the game is 0--you don't play at all.

What is the value of V(1,1)? If you do play, then half the time you will draw a black ball--and be down 1--but then you will be playing the game V(1,0), which is worth one, so if you draw a black ball next, you are even. And if you do play, then half the time you will draw a white ball--and be up one--and then you will be playing the game V(0,1), and so you stop and that is worth zero. The value of V(1,1) is therefore:

V(1,1) = (1/2)(-1+V(1,0)) + (1/2)(1 + V(0,1)) = (1/2)0 + 1/2(1) = 1/2

This is interesting: you might initially think that this is a fair game--there is, after all, one white and one black ball, so you have a 50-50 chance of being up after the first draw. But it is rigged in your favor.

More generally:

V(m,n) = max[0, (m/(m+n))(1+V(m-1,n)) + (n/(m+n))*(-1+V(m,n-1))]

http://spreadsheets.google.com/pub?key=p_zylRhg4tozt6QuTHcKPVw&output=html&gid=0&single=true

Comments

[Roy]

I don't see how this game is connected to liquidity, but I'm pretty sure what's happening isn't mean reversion. You are getting a positive return because you are permitted to make state dependent decisions. In particular, you get to stop if you ever get ahead. Even in problem instances where m and n are large and m>n, my guess (without working out the markov decision process details), is that you stop early.

[Roy]

I had a typo above, m>n should have been mn and m and n are large, you probably are likely to play to the end if you fall behind early. I guess one could interpret this as mean reversion in the sense that if you play a long time you end up with m-n. On the other hand, you really have a dynamic program on a finite horizon and the notion of a steady state mean is not very well defined.

[James]

I'm not an economist, but it seems to me there's a problem with the spreadsheet; in the case of n >> m (many more black balls than white), shouldn't the value actually be negative?

[No, when the value turns negative, you simply stop playing and leave the table.]

I can see how the expected value can be positive in the case of 6 black balls and 4 white if the player can stop at their choice and has perfect knowledge of the contents of the hat; but I'm sure not playing if there are 9 black balls and 1 white, because if I don't draw the white within the first two draws (19/90 chance?), then I'm definitely losing.

[Doctor Jay]

Your spreadsheet looks right, but I think you dropped a sign in the formula. Since the payoff if you draw a black ball is -1, the second term ought to be n/(n+m)(V(m,n-1)-1) instead of n/(n+m)(V(m,n-1)+1) shouldn't it?

[Roy]

James,

The expected payoff never goes negative even for n >> m because you get to choose simply to not play when the odds are truly against you.

[Manta1976]

What is the closed form of the formula for V(m,n)?

[Old Ari]

What's the name of the quant at the south pole, what his/her opinion?

[Paul J. Reber]

It's liquidity *and* mean reversion -- liquidity concerns here are contributing to the risk profile of the investment (game).

The true underlying value of this instrument is n-m. With replacement or over a long enough horizon (large m,n) the experienced value converges on the true value (reversion to the mean).

But like with an investment, if it's perfectly liquid, when you buy you not only gain the value of the difference between the purchase price and the underlying value, you also benefit from volatility in that if you can detect when the current traded value has randomly ticked maximally higher than the underlying value, you can sell.

If the instrument has restricted liquidity, then you can't count on this and your expected gain from investing is smaller than it would be (thus you lose the liquidity premium).

Bhaduri's point is that this premium is bigger than you'd think. Enough that you could play a -2 EV game (6 black, 4 white) and still expect to come out a slight winner by riding the volatility and selling high.

But if the math is this non-intuitive for a simple perfect knowledge game, I shudder to think of what it would take to calculate and incorporate the liquidity premium into real investments.

[Michael Carroll]

Did I miss something?

Brad said: "I suspect this is wrong..." What exactly is Brad saying is wrong?

[Felix Salmon]

I find this interesting because calculating the EV is clearly easier than actually working out the optimal strategy for the (4,6) case. Has anybody done that?

[Keep playing until you get to (0,1) or (1,2) or (1,3) or (2,4) or (3,5) or (3,6). At those points the game is no longer worthwhile.]

[Darren]

Yes, I was wondering what the rule of thumb would be as well.

another interesting observation - the expected value of the game with 3 black balls and 2 white ones is not the same as 6/4.

Here is a python routine that solves the problem (then I looked at what Brad wrote and realized I'd duplicated his solution)

def EV(black, white):
if black == 0:
ret = float(white)
elif white == 0:
ret = 0 # stop playing for sure if there are no white balls left
else:
pBlack = float(black/(black+white))
pWhite = float(white/(black+white))
ret = max(0, pBlack*(-1+EV(black-1, white)) + pWhite*(1+EV(black, white-1)))
return ret

print EV(6.0, 4.0)

[Brian Weatherson]

I think it's very hard to spin this as a case where liquidity is the driving factor. It seems the data is much better explained by the changes in probability as we're playing the game.

Compare the game where we start with 4 white balls and 6 black balls to one where we start with 40 white and 60 black.

Now in one sense, in the 40/60 game you have *much more* liquidity, in that you have many more chances to get out of the game. And the starting odds are the same. But the 40/60 game should not be played. That's simply because the odds don't move in your favour too often.

I'd also be a little surprised to see a lot of real-world situations that behave quite like this. The thing about draws without replacement from a known urn, is that success on one round is evidence that you *won't* succeed in the next round. And conversely, loss in one round is evidence of success in the next. So it's a situation where, to a first approximation, counter-induction is a sensible learning procedure. There aren't a lot of real-world situations like this. I'm pretty sure there are a lot fewer real world situations that are like this than there are real world situations where liquidity is valuable.

[AP]

Paul Reber is bang on.

[James B. Shearer]

Actually I think this game illustrates the value of the bankruptcy put. Abandoning the game when there are too many black balls corresponds to declaring bankruptcy and walking away from your debts. Continuing in the 4 white 6 black game corresponds to continuing to operate an insolvent company in hopes of being able to strip some of the remaining assets before leaving the debts.

[Roy]

This situation is quite similar to the famous "Ballot Box" problem where candidate A got a votes, candidate B got b votes where a>b. When the votes are counted, what is the probability P(a,b) that A will always be strictly ahead of B in the count. The answer is P(a,b)=(a-b)/(a+b). With a=n black balls and b=m white balls, the probability that we are always at a loss (assuming we play to the end) is P(n,m)=(n-m)/(n+m). On the other hand, the probability that we eventually reach break even or better is
1-P(n,m)=2n/(m+n).

That says with m=4 white and n=6 black, we will reach break-even at some point with probability 8/10.

Interestingly, this is the same answer if m=40 and n=60. To me, this suggests that you might very well play the m=40, n=60 game.

[gorobei]

Darren,

Your python solution is buggy (but you knew that because you had to pass 6.0 and 4.0 as args.)

float(x/y) != x/float(y).

Fix that, and you can remove all those nasty float() hacks.

Also, memoize EV(b,w) and the you can easily explore the problem space up to EV( 100,100.) Yes, python's small stack limit is ridiculous.

[Darren]

Thanks, gorobei. I wrote it at work when I should have been doing something else... didn't have time to do it right:) Thanks for the memoization technique, I was not familiar with it (though it's pretty obvious).

I think in this game that "liquidity" is the ability to leave the game.
"Volatility" is simulated by the fairly large change in expected value that comes from a draw due to the small number of balls in a hat. As Paul Reber points out, even an instrument (the hat) with an underlying value of -2 can be profitable to own for a while if it is volatile and liquid. (take away the volatility with a hat with 60/40 balls and the game isn't worth playing anymore)

[Tonsure Wimple]

A major cause of the current liquidity crisis is demonstrated by this example: everyone uses the same mathematical model.

[wood turtle]

Brad DeLong, I think you need to go to the math department soon.