The normal distribution

Bells of every centre and width slide and squeeze onto one bell. Watch the shaded bands fill to the same shares.

One bell, stretched and shifted

Measure the heights of many adults, the scores on a large exam or the noise on a radio, and the histograms often take the same shape: a hump in the middle, falling away evenly on both sides. That shape is the normal distribution, or bell curve. Its density, for a centre $\mu$ and a width $\sigma$, is

I write it $\mathcal{N}(\mu,\sigma^2)$. A variable $X$ that follows it has mean $\mu_X=\mu$ and standard deviation $\sigma_X=\sigma$. Here a bare $\sigma$ is the bell’s standard deviation; signal processing pages use the same letter for something else.

The clip showed the one fact that makes the bell easy to use: every bell is the same bell, moved and stretched. Subtract the centre and divide by the width:

This is standardising. $Z$ measures how many standard deviations $X$ lies from its mean, and it follows the standard normal law $\mathcal{N}(0,1)$, with mean 0 and standard deviation 1. A value of $Z$ is called a $z$-score.

Areas under the one bell

Since every bell is the standard bell in other units, one set of areas serves them all. The distribution function of $Z$ has its own letter:

$\Phi$ has no formula in everyday functions. Software computes it: norm.cdf in SciPy, or the normal cumulative function of any spreadsheet. Its curve reads like this:

The bell is symmetric, so $\Phi(-z)=1-\Phi(z)$: the area below $-1$ is $1-0.8413=0.1587$. The area within $k$ standard deviations of the mean is

Three values are worth knowing by heart:

And the edge that holds 95 % is 1.96 standard deviations, to two decimals: $2\Phi(1.96)-1=0.95$.

After the clip, drag the band’s edge and watch the area. Most of it is gathered quickly: one standard deviation already holds two thirds, and past three there is almost nothing left.

Not every shape is a bell

These areas belong to the bell. Other shapes give other areas, so check the shape before you use them.

Take the exponential waiting time $\mathrm{Exp}(1)$ of Continuous random variables, with mean 1 and standard deviation 1. Within one standard deviation of the mean, from 0 to 2, lies $1-e^{-2}=0.8647$ of it, not 0.6827. Within two, from $-1$ to 3, lies $1-e^{-3}=0.9502$, not 0.9545.

What holds for every shape is much weaker. Whatever the law, at least $1-1/k^2$ of it lies within $k$ standard deviations of the mean: at least 75 % within two. The bell gives 95.45 %. This guarantee, Chebyshev’s inequality, is proved in The law of large numbers.

Sums of bells

Add two independent bell-shaped variables and the sum is a bell too. Their means add, and their variances add:

where $X_1\sim\mathcal{N}(\mu_1,\sigma_1^2)$ and $X_2\sim\mathcal{N}(\mu_2,\sigma_2^2)$, and $\sim$ reads “follows”. The standard deviations do not add: they combine like the sides of a right triangle, $\sqrt{\sigma_1^2+\sigma_2^2}$. Why variances add for independent variables, and why the sum stays a bell, is the subject of Sums of random variables.

Worked example

1. Heights. Adult heights in a population follow $\mathcal{N}(170,8^2)$ cm. What share is taller than 186 cm? Standardise: $z=(186-170)/8=2$. So

about 2 people in 100.

What share is between 160 and 178 cm? The edges standardise to $(160-170)/8=-1.25$ and $(178-170)/8=1$:

2. A journey. Your drive takes $\mathcal{N}(30,4^2)$ minutes and the walk after it $\mathcal{N}(20,3^2)$ minutes, independently. The whole journey is $\mathcal{N}(50,\,16+9)=\mathcal{N}(50,5^2)$: a standard deviation of 5 minutes, not $4+3=7$. The chance it takes more than an hour is $\Pr\{Z>(60-50)/5\}=1-\Phi(2)=0.0228$.

And the chance it takes between 45 and 55 minutes, one standard deviation either side, is 0.6827.

Where you’ll meet this

Measurement error and noise. Errors that come from many small, independent causes added together pile into a bell, the reason given by The central limit theorem. Thermal noise in electronics is the classic case, as in Random variables for signals in the signal processing course.

Test scores and heights. Large exams are often scaled so that scores form a bell; heights of adults of one sex in one population are close to one. “In the top 2.3 %” is a statement about $1-\Phi(2)$.

Tolerances. A factory that keeps a part’s size within three standard deviations of the target expects about 3 parts in 1000 outside, if the sizes are bell-shaped.

Reference card