Probability brainteaser: expected value of these two games

Let $E(h,n)$ be the expected winnings (under the optimal stopping strategy) after flipping the coin $n$ times and obtaining $h$ heads.

If you flip again, your expected earnings will be $\dfrac{1}{2}E(h,n+1)+\dfrac{1}{2}E(h+1,n+1)$.

If you don't flip again, your expected earnings will be $\dfrac{h}{n}$.

Therefore, you flip again iff $\dfrac{1}{2}E(h,n+1)+\dfrac{1}{2}E(h+1,n+1) > \dfrac{h}{n}$, and your expected earnings satisfy $E(h,n) = \max\left\{\dfrac{h}{n},\dfrac{1}{2}E(h,n+1)+\dfrac{1}{2}E(h+1,n+1)\right\}$.

Trivally, if you flip $100$ times and get $h$ heads, your winnings are $E(h,100) = \dfrac{h}{100}$.

Using a computer, we can use the above recursion to get $E(0,0) \approx 0.783894497678384$.

As it turns out, stopping after $h > t$ is pretty close to the optimal strategy. After bashing out the above recursion, the actual optimal strategy can be phrased like "After the $n$-th flip, stop if you have gotten at least $h(n)$ heads", where $h(n)$ is given by this table:

$\begin{matrix} n&\ 1&\ \ 2&\ \ 3&\ \ 4&\ \ 5&\ \ 6&\ \ 7&\ \ 8&\ \ 9&\ 10\\ h(n)&\ 1&\ \ 2&\ \ 2&\ \ 3&\ \ 4&\ \ 4&\ \ 5&\ \ 5&\ \ 6&\ 6\end{matrix}$

$\begin{matrix} n&11&12&13&14&15&16&17&18&19&20\\ h(n)&7&8&8&9&9&10&10&11&11&12\end{matrix}$

$\begin{matrix} n&21&22&23&24&25&26&27&28&29&30\\ h(n)&12&13&13&14&14&15&15&16&16&17\end{matrix}$

$\begin{matrix} n&31&32&33&34&35&36&37&38&39&40\\ h(n)&17&18&18&19&19&20&20&21&21&22\end{matrix}$

$\begin{matrix} n&41&42&43&44&45&46&47&48&49&50\\ h(n)&22&23&23&24&24&25&25&26&26&27\end{matrix}$

$\begin{matrix} n&51&52&53&54&55&56&57&58&59&60\\ h(n)&27&28&28&29&29&30&30&31&31&32\end{matrix}$

$\begin{matrix} n&61&62&63&64&65&66&67&68&69&70\\ h(n)&32&33&33&34&34&35&35&36&36&37\end{matrix}$

$\begin{matrix} n&71&72&73&74&75&76&77&78&79&80\\ h(n)&37&38&38&39&39&40&40&41&41&42\end{matrix}$

$\begin{matrix} n&81&82&83&84&85&86&87&88&89&90\\ h(n)&42&43&43&43&44&44&45&45&46&46\end{matrix}$

$\begin{matrix} n&91&92&93&94&95&96&97&98&99&100\\ h(n)&47&47&48&48&48&49&49&50&50&0\end{matrix}$


For the pinball game, your winnings on each ball are independent of each other. So, you should either play the game all day long, or not at all depending on whether your expected earnings is positive or not. This is an easy calculation, so I'll let you do it.


For a trivial estimate of the coin flip game, you can't win more than $1$ (all heads), so that is an upper bound.

Less trivially, we need to define our strategy. Suppose you have flipped $h$ heads and $t$ tails. Let $V(h,t)$ be the value of the game at this point. Clearly $V(h,t) \ge \frac h{h+t}$ because we can stop now and get that. If we flip again, the alternative is $\frac 12(V(h+1,t)+V(h,t+1))$ We also have $V(1,0)=1$ because we can't do better and stop, while $V(0,1)=\frac 12(V(1,1)+V(0,2))$ because we can't do worse and should flip again.

I suggest that you should stop any time $h \gt t$ If you decide to flip once and stop either way, you start with $\frac h{h+t}$ and trade that for $\frac {2h+1}{2(h+t+1)}$ which is less. Repeating has you in the same boat-some handwaving here.

One strategy is as follows: You flip the first time. Heads you get $1$. Tails you will keep flipping until the law of large numbers kicks in and you have (about) half heads and get $\frac 12$. The value of the game is $\frac 34$ You can do a little better by throwing when you are at exactly half heads-if you win you are ahead, if you lose you can keep flipping and (with probability $1$) get back to at least even. The value then must be a little more than $3/4$

For the pinball case, the logic is the same. You assess whether the next ball improves your payoff and play it or not. I think you can again show that you don't need to worry about long term effects-just ask if this ball improves my payoff, but I haven't proved that at all.

Tags:

Probability