Continuous random variables

Draw random waiting times and sort them into bins. Watch the bars settle onto a smooth curve as the draws pile up.

Bars that become a curve

A die has six values you can list. A waiting time does not: the bus can come after 2 minutes, 2.1, 2.13, any number at all. A random variable that can take any value in a range is continuous. No list of values with their chances can describe it, so I need something else. The picture shows what.

Each bar’s height is its count divided by (number of draws × bin width). That makes a bar’s area, height times width, equal to its count divided by the number of draws: the share of draws in that bin. All the bars together have area 1.

Now make the draws many and the bins narrow. The bar tops settle onto a curve, the same curve whatever the bin width. That curve is the density of $X$, written $f_X(x)$. Here it is $f_X(x)=e^{-x}$ for $x\ge0$.

A bar shows the curve’s average over its width. The first bar, from 0 to 0.5, averages $0.787$; the first bar of width 0.125 averages $0.940$; the curve itself starts at 1. Narrower bins follow the curve more closely.

Probability is area

Bar areas are shares, so in the limit the probability that $X$ lands between $a$ and $b$ is the area under the density there:

Every draw lands somewhere, so the total area is 1, and a density is never negative.

A single exact value is a strip of width zero, so its area is zero: $\Pr\{X=1\}=0$. That does not mean a wait of exactly 1 minute is impossible: such a wait can happen, but the chance of hitting 1.000… exactly is zero. For the same reason, $<$ and $\le$ give the same answer here.

The distribution function

The distribution function is the same as for discrete variables, the chance of landing at or below $x$:

It climbs smoothly from 0 to 1. Its slope is the density, $f_X(x)=F_X'(x)$: where the density is high, $F_X$ climbs fast.

For the curve above, $F_X(x)=1-e^{-x}$ for $x\ge0$. The chance of landing above $x$ is then $1-F_X(x)=e^{-x}$. This is called the survival function, because for lifetimes it is the chance of lasting past $x$.

The median is the value with half the area on each side, where $F_X(x)=\tfrac12$. Here $1-e^{-x}=\tfrac12$ gives $x=\ln2=0.6931$.

Means and variances by integrals

The mean and the variance work as in Expected value and Variance and standard deviation, with the sum over values replaced by an integral over the density:

The uniform law. A variable spread evenly between $a$ and $b$ has the flat density $1/(b-a)$ there and 0 elsewhere. I write it $\mathrm{U}(a,b)$. By symmetry its mean is the middle, $(a+b)/2$. Its variance, the integral of the squared distance from the middle over a width $w=b-a$, is

The exponential law. The waiting times in the picture follow the exponential law with rate $\lambda$, $\mathrm{Exp}(\lambda)$:

The picture used $\lambda=1$. Here $\lambda$ is a rate, events per unit of time; in the linear algebra course the same letter is an eigenvalue. Integrating by parts,

A second integration by parts gives $\mathbb{E}\{X^2\}=2/\lambda^2$, so $\mathrm{Var}\{X\}=2/\lambda^2-1/\lambda^2=1/\lambda^2$. The standard deviation equals the mean, $1/\lambda$. Its distribution function is $F_X(x)=1-e^{-\lambda x}$.

Worked example

1. Areas under $e^{-x}$. For $X\sim\mathrm{Exp}(1)$, read $\sim$ as “follows”:

2. Waiting for a bus. A bus comes every 10 minutes and you arrive at a random moment, so your wait is $\mathrm{U}(0,10)$ minutes. The chance of waiting more than 7 minutes is the area from 7 to 10, $3\times\tfrac1{10}=0.3$. The mean wait is 5 minutes, the variance $10^2/12=8.333$ and the standard deviation $2.887$ minutes.

3. A light bulb. A bulb’s lifetime $T$ is exponential with mean 1000 hours, so $\lambda=1/1000$ per hour. The chance it lasts past 1500 hours is $e^{-1500/1000}=e^{-1.5}=0.2231$.

Now suppose it has already lasted 1000 hours. The chance it lasts past 1500 is, by Conditional probability,

the same as the chance that a new bulb lasts 500 hours. An exponential lifetime does not age: what is left of it does not depend on how long it has run. This is called being memoryless, and it is why the exponential law fits failures that strike at random, not wear-out.

Try it yourself. For $\mathrm{Exp}(1)$, is $\Pr\{X=1\}$ equal to $f_X(1)=0.3679$? No: it is 0. Only areas are probabilities.

Where you’ll meet this

Waiting times. The time to the next phone call, the next radioactive decay or the next customer is often exponential: events that strike at random, at a steady rate. Random arrivals builds on it.

Lifetimes. Reliability engineers write lifetimes with densities and quote survival functions: the chance a part still works after a given time.

Measurement noise. The noise in a sensor reading takes any value in a range, so it is described by a density, most often the bell of The normal distribution.

Reference card