Variance and standard deviation

Mark a die’s six faces on a line. From each face, draw a square out to the mean. Watch what their average square is.

Spread as squares

Two dice can have the same mean and still feel different. A fair die gives 1 to 6, centred on 3.5. A die marked 3, 3, 4, 4, 3, 4 also averages 3.5, but its rolls never stray far. The mean tells you where the values sit; it says nothing about how far they wander.

To measure the wandering, start with each value’s distance from the mean, its deviation $x-\mu_X$. For the fair die the deviations are $-2.5,-1.5,-0.5,0.5,1.5,2.5$. Their average is 0: the mean sits in the middle, so the minus side cancels the plus side. Averaging plain deviations always gives 0.

So square them first, as the picture did. A square is never negative, and it grows fast with distance. The squares are $6.25,2.25,0.25$, each twice, and their average is

That average square is the variance:

The variance is in squared units: for a die, “faces squared”. To get back to the units of $X$, take the square root. That is the standard deviation, $\sigma_X=\sqrt{\mathrm{Var}\{X\}}$, the side of the average square in the picture. For the die, $\sigma_X=\sqrt{35/12}=1.7078$.

I write the variance $\sigma_X^2$ too. In the statistics pages a $\sigma$ is a standard deviation; signal processing pages use a bare $\sigma$ for the decay rate of the Laplace variable.

After the clip, you answered the question about moving a dot. Now drag one dot twice as far from the mean and watch its square: it becomes four times as big. Twice the distance makes four times the square, which is why the variance listens most to the values far from the mean.

A shortcut

Expand the square inside the mean and use linearity from Expected value:

The variance is the mean of the square minus the square of the mean: exactly the gap between $\mathbb{E}\{X^2\}$ and $\mathbb{E}\{X\}^2$ that the last page found. For the die, $91/6-12.25=35/12$, the same as before.

Changing units

Shift every value by $b$ and the mean shifts too, so every deviation stays the same: the spread does not change. Multiply every value by $a$ and every deviation is multiplied by $a$, so every square is multiplied by $a^2$:

So if $\mathrm{Var}\{X\}=4$, then $\mathrm{Var}\{3X+2\}=9\times4=36$.

Spread of the named shapes

Bernoulli. For a variable that is 1 with chance $p$ and 0 otherwise, $X^2=X$, so $\mathbb{E}\{X^2\}=p$ and

It is largest at $p=\tfrac12$, a fair coin, where it is $0.25$.

Binomial. A binomial count is a sum of $n$ independent Bernoulli variables. For independent variables the variances add, as Sums of random variables shows, so

The ten-row board of Discrete random variables at $p=\tfrac12$ has variance $10\times0.25=2.5$ and standard deviation $1.5811$.

This also explains the typical miss of What probability measures. The share of $n$ darts inside a shape is $X/n$, with $X$ binomial. By the scaling rule its variance is $np(1-p)/n^2=p(1-p)/n$, and its standard deviation is the typical miss, $\sqrt{p(1-p)/n}$.

Beyond spread

The mean and the variance describe a centre and a width. Two more words describe a shape. Skewness measures lopsidedness: a long tail on one side, like incomes, where a few values sit far above the rest. Kurtosis measures heavy tails: how often values land very far from the mean compared with a bell, as with large market crashes.

Worked example

1. The die, both ways. By the definition: deviations $\pm2.5,\pm1.5,\pm0.5$, squares $6.25,2.25,0.25$ twice each, mean $17.5/6=2.9167$. By the shortcut: $91/6-3.5^2=15.1667-12.25=2.9167$. Standard deviation $1.7078$.

2. A biased coin. A Bernoulli variable with $p=0.3$ has variance $0.3\times0.7=0.21$ and standard deviation $\sqrt{0.21}=0.4583$.

3. Units. Daily temperatures in a city have a standard deviation of 5 °C. In degrees Fahrenheit, $F=1.8\,C+32$. The $+32$ moves the mean only; the $1.8$ stretches the spread, so the standard deviation is $1.8\times5=9$ °F.

Try it yourself. A fair coin scored 0 or 1 has variance $\tfrac12\times\tfrac12=0.25$.

Where you’ll meet this

Noise power. Remove a signal’s mean and its average power is its variance. The noise in a sensor or a radio is quoted by its standard deviation, often called its RMS value, as in Random variables for signals in the signal processing course.

Measurement error. A scale whose readings have a standard deviation of 0.2 g scatters about 0.2 g around the true weight on a typical reading. Instruments are sold by this number.

Tolerances. A part machined to 10 mm with a standard deviation of 0.01 mm fits; one with 0.1 mm may not. Factories track the standard deviation of their parts as closely as the mean.

Risk. In finance the standard deviation of a return is a common first measure of risk: two investments with the same mean return can carry very different spreads.

Reference card