Expected Value Proof - Law of Total Expectation.

This is the Law of Total Expectation. The proof is as follows:

$$ \begin{align} E[E[X|Y]] &= E \left[ \sum_{x} x \cdot P(X = x | Y) \right] \\ &= \sum_y \left[ \sum_{x} x \cdot P(X = x | Y = y) \right] P(Y = y)\\ &= \sum_x x \sum_y P(X = x | Y = y) \cdot P(Y = y) \\ &= \sum_x x \sum_y P(X = x \, \text{and} \, Y = y) \\ &= \sum_x x \cdot P(X = x)\\ &= E[X] \end{align} $$

The non-discrete is a bit tougher I think. The Wikipedia link should help you out here.

The continuous case is similar. Let us specify the Law of Total Expectation (also called Tower Property) more precisely:

$$E_Y(E_X[X|Y])=E_X[X]$$

where $E_Y$ is the expectation w.r.t. $Y$ and $E_X$ w.r.t. $X$. Let $S_X=supp(X)$ and $S_Y=supp(Y)$ $$E_Y(E_X[X|Y])=\int_{S_Y} E_X[X|Y]\; f(y)\; dy$$ $$E_Y(E_X[X|Y])=\int_{S_Y}\int_{S_X}x\; \frac{f(x,y)}{f(y)}dx\; f(y)\; dy$$ $f(y)$ is constant w.r.t. $X$. Thus, $$E_Y(E_X[X|Y])=\int_{S_Y}\int_{S_X}x\; f(x,y)\;dx\; \frac{f(y)}{f(y)}\; dy$$ $$E_Y(E_X[X|Y])=\int_{S_Y}\int_{S_X}x\; f(x,y)\;dx\; dy$$ By Fubini's theorem $$E_Y(E_X[X|Y])=\int_{S_X}\int_{S_Y}x\; f(x,y)\;dy\; dx$$ Now $X$ is constant w.r.t. $Y$. $$E_Y(E_X[X|Y])=\int_{S_X}x\; \int_{S_Y} f(x,y)\;dy\; dx$$

The inner integral is the marginalisation from $f(x,y)$ to $f(x)$. Thus, $$E_Y(E_X[X|Y])=\int_{S_X}x\; f(x)\; dx$$ This is just the definition of the expectation. Thus, $$E_Y(E_X[X|Y])=E_X[X]$$

I hope this helps. This proof also exists in a measure theoretic context which is more general. The Wikipedia article is quite good.