Jensen's Inequality

Definition

Jensen's Inequality is a fundamental result in probability theory and statistical mathematics that describes the relationship between the expected value of a convex (or concave) function and the function of an expected value.

Mathematical Statement

For a convex function $f(x)$, Jensen's Inequality states that:

$$f(\mathbb{E}[X])\le \mathbb{E}[f(X)]$$

Where:

• $\mathbb{E}[X]$ represents the expected value of a r.v. $X$
• $f(x)$ is a convex function

Simple Example

Consider the convex function $f(x)=x^2$:

• Let $X$ take values 1 and 3 with equal probability
• $\mathbb{E}[X]=\frac{1+3}{2}=2$
• $f(\mathbb{E}[X])=2^2=4$
• $\mathbb{E}[f(X)]=\frac{1^2+3^2}{2}=\frac{1+9}{2}=5$

Here, $f(\mathbb{E}[X])=4\le \mathbb{E}[f(X)]=5$, demonstrating Jensen's Inequality.