Conditional Expectation

For two r.v.s $X, Y$ , we have

E [Y] = E [E [Y ∣ X]]

This is because

E [Y] = \int_{- \infty}^{\infty} y f_{Y} (y) d y = \int_{- \infty}^{\infty} y \int_{- \infty}^{\infty} f (x, y) d x d y = \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} y f_{Y ∣ X} f (y ∣ x) f_{X} (x) d x d y = \int_{- \infty}^{\infty} f_{X} (x) \int_{- \infty}^{\infty} y f_{Y ∣ X} (y ∣ x) d x d y = \int_{- \infty}^{\infty} f_{X} (x) E [Y ∣ X] d x = E [E [Y ∣ X]]

For discrete $X$ , this gives

E [Y] = i \sum P (X_{i}) E [Y ∣ X_{i}]

This can be seen as the law of total expectation.

Conditional Variance

For two r.v.s $X, Y$ , we have

Var [Y] = Var [E [Y ∣ X]] + E [Var [Y ∣ X]]

Tip

The variance of a r.v. is the sum of variance of conditional expectation and the expectation of conditional variance

This is because

Var [E [Y ∣ X]] = E [E^{2} [Y∣X]] - E^{2} [E [Y ∣ X]] = E [E^{2} [Y ∣ X]] - E^{2} [Y]

and

E [Var [Y ∣ X]] = E [E [Y^{2} ∣ X] - E^{2} [Y ∣ X]] = E [Y^{2}] - E^{2} [E [Y ∣ X]]

Adding the two formulas above gets the result.