Mathematical Expectation

Definition

The expected value of a discrete r.v. $X$ whose distinct possible valances are $x_1,x_2,\dots$ is defined by

$$\mathrm{E}[X]=\sum _{j=1}^{\infty }{x}_{j}P(X=x_j)$$

If the support is finite, then this is replaced by a finite sum, and we can also write

$$\mathrm{E}[X]=\sum _x\underset{\text{value}}{\underbrace{x}}\underset{\text{PMF at }x}{\underbrace{P(X=x)}}$$

The expectation is undefined is ${\sum }_{j=1}^{\infty }\left|x_j\right|P(X=x_j)$ diverges.

Tip

If $X$ and $Y$ are discrete r.v.s with the same distribution (need not to be independent from each other), then $\mathrm{E}[X]=\mathrm{E}[Y]$

Another way to calculate expectation for a non-negative r.v. $X$ is

$$\mathrm{E}[X]={\int }_0^{+\infty }P(X>t)\mathrm{d}t={\int }_0^{+\infty }\left[1-F_X(t)\right]\mathrm{d}t={\int }_0^{+\infty }G(t)\mathrm{d}t$$

where $G(x)$ is the Survival Function of $X$

Linearity of Expectation

For any r.v.s $X,Y$ and any constant $c$,

$$\begin{array}{rl}\mathrm{E}[X+Y]& =\mathrm{E}[X]+\mathrm{E}[Y]\\ \mathrm{E}[cX]& =c\mathrm{E}[X]\end{array}$$

Intuitive Proof

Recall $X$ is a function which assigns a real number to every outcome $s$ in the sample space. The r.v. $X$ may assign the same value to multiple sample outcomes. When this happens, out definition of expectation groups all these outcomes together, and whose weight, $P(X=x)$, is the total weight of the constituent outcomes. If $X(s)$ is the value that $X$ assigns to outcome $s$, we can take the weighted average

$$\mathrm{E}[X]=\sum _{s}X(s)P(\left\{s\right\})$$

where $P(\left\{s\right\})$ is the weight of outcome $s$. So we have

$$\begin{array}{rl}\mathrm{E}[X]+\mathrm{E}[Y]& =\sum _{s}X(s)P(\left\{s\right\})+\sum _{s}Y(s)P(\left\{s\right\})\\ & =\sum _s(X+Y)(s)P(\left\{s\right\})\\ & =\mathrm{E}(X+Y)\end{array}$$

Expectation of the Product of Independent Random Variables

For independent r.v.s $X_1,\dots ,X_n$, we have

$$\mathrm{E}[X_1\cdots X_n]=\mathrm{E}[X_1]\cdots \mathrm{E}[X_n]$$

Expectation of Functions of Random Variables

See Law of the Unconscious Statistician (LOTUS)

Conditional Expectation

See Conditional Expectation