Discrete Distributions — Uniform, Bernoulli, Binomial, Poisson and Friends

The discrete uniform, Bernoulli, binomial, Poisson, geometric, negative binomial and hypergeometric distributions — when each arises, their PMFs, means, variances and MGFs, the Poisson approximation to the binomial, memorylessness, and how to recognise which model a word problem is describing.

Each discrete distribution is a story. Recognising the story in a word problem is most of the work; the formulas then follow.

Summary table

Distribution	Story	PMF $P(X=k)$	Mean	Variance
Discrete uniform on $\{a,\dots,b\}$	each of $n = b-a+1$ values equally likely	$1/n$	$\frac{a+b}{2}$	$\frac{n^2-1}{12}$
Bernoulli( $p$ )	one trial, success = 1	$p^k(1-p)^{1-k}$ , $k\in\{0,1\}$	$p$	$p(1-p)$
Binomial( $n,p$ )	successes in $n$ independent trials	$\binom nk p^k (1-p)^{n-k}$	$np$	$np(1-p)$
Poisson( $\lambda$ )	count of rare events in a fixed interval	$e^{-\lambda}\frac{\lambda^k}{k!}$	$\lambda$	$\lambda$
Geometric( $p$ )	trials until the first success, $k = 1,2,\dots$	$(1-p)^{k-1}p$	$1/p$	$\frac{1-p}{p^2}$
Negative binomial( $r,p$ )	trials until the $r$ -th success	$\binom{k-1}{r-1}p^r(1-p)^{k-r}$	$r/p$	$\frac{r(1-p)}{p^2}$
Hypergeometric( $N,K,n$ )	successes in $n$ draws without replacement from $N$ items, $K$ of them successes	$\frac{\binom Kk\binom{N-K}{n-k}}{\binom Nn}$	$n\frac KN$	$n\frac KN\frac{N-K}{N}\frac{N-n}{N-1}$

Bernoulli and binomial

A Bernoulli RV is an indicator of success. A binomial is a sum of $n$ independent Bernoulli( $p$ ) indicators, which immediately gives $E = np$ and $\operatorname{Var} = np(1-p)$ by linearity and independence.

The PMF peaks at the mode $\lfloor (n+1)p \rfloor$ .
$P(X \ge 1) = 1 - (1-p)^n$ — the "at least one success" formula.
MGF: $(1 - p + pe^t)^n$ .

Poisson

The Poisson models counts of events that occur independently at a constant average rate: arrivals per minute, typos per page, defects per metre. If events occur at rate $r$ per unit time, the count in time $t$ is Poisson( $rt$ ).

$\sum_k e^{-\lambda}\lambda^k/k! = 1$ because $\sum \lambda^k/k! = e^\lambda$ .
Mean = variance = $\lambda$ (a quick check for over-dispersed data, where variance > mean).
Recurrence: $P(k+1) = P(k)\cdot\frac{\lambda}{k+1}$ — handy for computing several terms.
MGF: $e^{\lambda(e^t - 1)}$ .

Poisson approximation to the binomial

If $n$ is large, $p$ is small and $np = \lambda$ is moderate (rule of thumb $n \ge 20$ , $p \le 0.05$ ), then $\text{Bin}(n, p) \approx \text{Poisson}(np)$ . This is a limit theorem: $\binom nk p^k(1-p)^{n-k} \to e^{-\lambda}\lambda^k/k!$ .

Geometric and memorylessness

$P(X > k) = (1-p)^k$ (the first $k$ trials all fail). Hence

P(X > s + t \mid X > s) = \frac{(1-p)^{s+t}}{(1-p)^s} = (1-p)^t = P(X > t).

Past failures do not make success "due" — the gambler's fallacy. The geometric is the only memoryless discrete distribution (the exponential is the only memoryless continuous one).

Hypergeometric vs binomial

Drawing without replacement gives a hypergeometric; with replacement gives a binomial with $p = K/N$ . The variances differ by the finite population correction $\frac{N-n}{N-1}$ ; when $N \gg n$ the two are nearly identical.

Discrete Distributions — Uniform, Bernoulli, Binomial, Poisson and Friends

Summary table

Bernoulli and binomial

Poisson

Poisson approximation to the binomial

Geometric and memorylessness

Hypergeometric vs binomial

Which distribution is it?

Quick revision

Practice questions