Continuous Distributions — Uniform, Exponential, Normal, t and Chi-squared

Continuous uniform

$X \sim U(a, b)$ has constant density on $[a, b]$:

$$f(x) = \frac{1}{b - a}, \quad F(x) = \frac{x - a}{b - a}\ (a \le x \le b), \quad E[X] = \frac{a + b}{2}, \quad \operatorname{Var}(X) = \frac{(b - a)^2}{12}.$$

Probabilities are just length ratios. $U(0,1)$ is the raw material of simulation (inverse transform sampling).

Exponential

$X \sim \text{Exp}(\lambda)$ models waiting times between events of a Poisson process with rate $\lambda$:

$$f(x) = \lambda e^{-\lambda x},\quad F(x) = 1 - e^{-\lambda x},\quad P(X > x) = e^{-\lambda x}\ (x \ge 0),$$ $$E[X] = \frac{1}{\lambda},\quad \operatorname{Var}(X) = \frac{1}{\lambda^2},\quad \text{median} = \frac{\ln 2}{\lambda}.$$

• Memoryless: $P(X > s + t \mid X > s) = P(X > t)$. A component that has survived 100 hours is "as good as new".
• Minimum of exponentials: if $X_i \sim \text{Exp}(\lambda_i)$ independent, $\min_i X_i \sim \text{Exp}(\sum\lambda_i)$, and the probability that $X_j$ is the minimum is $\lambda_j / \sum_i\lambda_i$.
• Sum of $n$ i.i.d. Exp($\lambda$) is Gamma($n, \lambda$) (the Erlang distribution).

Normal (Gaussian)

$X \sim N(\mu, \sigma^2)$:

$$f(x) = \frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{(x - \mu)^2}{2\sigma^2}\right).$$

Bell-shaped, symmetric about $\mu$; mean = median = mode $= \mu$; inflection points at $\mu \pm \sigma$.

Standard normal and z-scores

$Z = \dfrac{X - \mu}{\sigma} \sim N(0, 1)$, with CDF $\Phi(z)$. Then $P(X \le x) = \Phi\left(\frac{x - \mu}{\sigma}\right)$, and by symmetry $\Phi(-z) = 1 - \Phi(z)$.

Properties you will use

• Linear transformations stay normal: $aX + b \sim N(a\mu + b, a^2\sigma^2)$.
• Independent sums stay normal: $X + Y \sim N(\mu_X + \mu_Y, \sigma_X^2 + \sigma_Y^2)$; for differences the variances still add.
• The sample mean of $n$ i.i.d. $N(\mu, \sigma^2)$ values is exactly $N(\mu, \sigma^2/n)$.
• MGF: $e^{\mu t + \sigma^2 t^2/2}$; moments of $Z$: $E[Z^2] = 1$, $E[Z^4] = 3$.

Normal approximation to the binomial

If $np \ge 5$ and $n(1-p) \ge 5$ (some texts say 10), $\text{Bin}(n,p) \approx N(np, np(1-p))$. Use the continuity correction: $P(X \le k) \approx \Phi\left(\frac{k + 0.5 - np}{\sqrt{np(1-p)}}\right)$.

Chi-squared distribution

If $Z_1, \dots, Z_k$ are i.i.d. $N(0,1)$, then

$$Q = \sum_{i=1}^k Z_i^2 \sim \chi^2_k, \qquad E[Q] = k,\ \operatorname{Var}(Q) = 2k.$$

• Supported on $[0, \infty)$, right-skewed; becomes more symmetric as $k$ grows.
• Independent chi-squareds add: $\chi^2_a + \chi^2_b = \chi^2_{a+b}$.
• $\chi^2_2$ is exactly $\text{Exp}(1/2)$.
• Sample variance: for normal data, $\dfrac{(n-1)s^2}{\sigma^2} \sim \chi^2_{n-1}$, independent of $\bar X$.

It powers the chi-squared tests for goodness of fit and independence.

Student's t distribution

If $Z \sim N(0,1)$ and $V \sim \chi^2_\nu$ are independent,

$$T = \frac{Z}{\sqrt{V/\nu}} \sim t_\nu, \qquad E[T] = 0\ (\nu > 1),\quad \operatorname{Var}(T) = \frac{\nu}{\nu - 2}\ (\nu > 2).$$

The key application: for normal data with unknown $\sigma$,

$$\frac{\bar X - \mu}{s/\sqrt n} \sim t_{n-1}.$$

Replacing $\sigma$ by the estimate $s$ adds uncertainty, which shows up as heavier tails. $t_1$ is the Cauchy distribution (no mean); as $\nu \to \infty$, $t_\nu \to N(0,1)$. Critical values exceed the normal ones, e.g. $t_{0.025, 10} = 2.228$ vs $z_{0.025} = 1.96$.

Choosing the right distribution

Quick revision