Interval Estimation

Confidence Intervals

Definition

A confidence interval (CI) gives a range of values derived from sample statistics that is likely to contain the true population parameter. For example, a 95% confidence interval implies that if we were to take many samples and build intervals from each, approximately 95% of those intervals would contain the true parameter.

Let $X_1,X_2,\dots ,X_n$ i.i.d. $\sim f(x;\theta )$, $\theta \in \Theta$. The parameter to be estimated is $g(\theta )$. Denote ${\hat{g}}_L={\hat{g}}_L(X_1,X_2,\dots ,X_n)$, ${\hat{g}}_U={\hat{g}}_U(X_1,X_2,\dots ,X_n)$, which satisfy

$$P\left({\hat{g}}_L\le g(\theta )\le {\hat{g}}_U\right)=1-\alpha ,\forall \theta \in \Theta$$

We say $[{\hat{g}}_L,{\hat{g}}_u]$ is a $1-\alpha$ confidence interval of parameter $g(\theta )$.

Pivot Method

1. Find a Pivotal Quantity A pivotal quantity $Q$ must satisfy two key properties

• It must be a function of both the data and the unknown parameter $\theta$
• Its probability distribution must not depend on any unknown parameters For Normal data with unknown mean $\mu$ and known variance ${\sigma }^2$, a common pivot is

$$Q=\frac{\bar{X}-\mu }{\sigma /\sqrt{n}}\sim \mathcal{N}(0,1)$$

For Normal data with both $\mu ,{\sigma }^2$ unknown, we can have

$$Q_{\mu }=\frac{\bar{X}-\mu }{S/\sqrt{n}}\sim t_{n-1},Q_{{\sigma }^2}=\frac{(n-1)S^2}{{\sigma }^2}\sim {\chi }_{n-1}^2$$

where $t_{n-1}$ is the Student's t-distribution with $n-1$ degree of freedom, ${\chi }_{n-1}^2$ is the Chi-Squared Distribution with $n-1$ degree of freedom. 2. Determine Critical Values Find values $c_1$ and $c_2$ such that

$$P(c_1\le Q\le c_2)=1-\alpha$$

If $Q\sim \mathcal{N}(0,1)$, then we can use $c_1:\Phi (c_1)=\frac{\alpha }{2}$ and $c_2:\Phi (c_2)=1-\frac{\alpha }{2}$ 3. Solve for Parameter Solve the CI for $\theta$ from $c_1\le Q\le c_2$, the CI for $Q$.

Normal Approximation Method (from large sample data)

1. Determine Sample Mean. Calculate the average $\bar{x}$ of the sample data
2. Find Z-Score. Depending on your desired confidence level (e.g., 90%, 95%, or 99%), find the corresponding z-score from a z-table
   • For For 90% confidence ($\alpha =0.1$), $z\approx 1.645$
   • For For 95% confidence $(\alpha =0.05)$, $z\approx 1.960$
   • For For 99% confidence $(\alpha =0.01)$, $z\approx 2.576$ That is $z:\Phi (z)=1-\frac{\alpha }{2}$
3. Calculate Standard Error (the standard deviation of the sample). That is

$$\mathrm{SE}=\frac{\sigma }{\sqrt{n}}$$

4. Get the Normal Distribution

$$\bar{X}\sim \mathcal{N}\left(\mathrm{E}[X],{\mathrm{SE}}^2\right)⟹\frac{\bar{X}-\mathrm{E}[X]}{\mathrm{SE}}\sim \mathcal{N}(0,1)$$

1. Solve for the Parameter. Solve the interval of the parameter from

$$-z\le \frac{\bar{X}-\mathrm{E}[X]}{\mathrm{SE}}\le z$$

Note

If not required, do not use this method in exams