Statistics

Simple Random Sample

Let the distribution of the population is $F(x,\theta )$, where $\theta \in \Theta$ is some parameters. If $X_1,X_2,\dots ,X_n$ i.i.d. $\sim F(x,\theta )$, then we call $X_1,\dots ,X(n)$ simple random sample (SRS)

The joint distribution of $(X_1,\dots ,X_n)$ is

$$\prod _{i=1}^{n}F(x_i,\theta ),(x_1,\dots ,x_n{)}^T\in {\mathbb{R}}^n$$

Sample Mean

$$\bar{X}=\frac{1}{n}\sum _{i=1}^n{X}_i$$

Sample Variance

$$S^2=\frac{1}{n-1}\sum _{i=1}^n{\left(X_i-\bar{X}\right)}^2$$

Here $n-1$ can be explained in two ways

1. Since $(X_1-\bar{X})+\cdots +(X_n-\bar{X})=0$, so these variables just have $n-1$ degrees of freedom
2. This is a unbiased estimation, where we have $\mathrm{E}[S^2]=\mathrm{Var}(X)$

Sample Moments

Raw Moments

$${\xi }_k=\frac{1}{n}\sum _{i=1}^n{X}_i^k,k=1,2,\dots$$

Central Moments

$${\eta }_k=\frac{1}{n-1}\sum _{i=1}^n(X_i-\bar{X}{)}^k,k=2,3,\dots$$

Empirical Cumulative Distribution Function (ECDF)

$$F_n(x)=\frac{1}{n}\sum _{i=1}^n{I}_{\left\{X_i\le x\right\}},x\in \mathbb{R}$$

Order Statistics

Order Statistic

Let $X_1,X_2,\dots ,X_n$ be a simple random sample drawn from a population $X$. When these samples are arranged in ascending order, we obtain

$$X_{(1)}\le X_{(2)}\le \dots \le X_{(n)}$$

where $X_{(i)}$ is called the $i$-th order statistic.

For continuous random variables, the PDF of the $k$-th order statistic $X_{(k)}$ can be expressed as:

$$f_{X_{(k)}}(x)=\frac{n!}{(k-1)!(n-k)!}[F(x){]}^{k-1}[1-F(x){]}^{n-k}f(x)$$

where $F(x)$ is the CDF of the population

This can be concluded from Multinomial Distribution.

Standard Error

See Standard Error