What probability measures

Throw darts at random at a square that holds a circle. Watch the share that lands inside wobble, then settle.

Darts and a share of area

Here is the game. A dart lands somewhere on the square, and every spot is as likely as every other. You cannot say where the next dart will land. But you can ask what share of many darts will land inside the circle.

Watch the line under the square. After the first dart the share is either 0 or 1. After ten darts it can still be anywhere from about 0.2 to 0.8. After a thousand it sits close to the dotted line. In the clip, 6 of the first 10 darts landed inside, a share of 0.600, and 504 of 1000, a share of 0.504, beside the exact 0.5027.

The dotted line is at 0.5027, the circle’s area. The square has area 1, so the circle covers 0.5027 of it. In the long run, darts that land evenly over the square land inside the circle in the same share as its area.

That long-run share is what a probability measures. The probability that a dart lands inside the circle is 0.5027: over many, many darts, that is the share that lands inside.

After the clip, drag the radius to 0.5, the largest circle that fits. The dotted line moves to $\pi/4=0.7854$. Darts can measure any shape this way, and this one gives you $\pi$: four times the share inside. The clip’s 1000 darts put 795 inside, so they estimate $\pi$ as $4\times0.795=3.18$; more darts would do better.

How large is the wobble? The share after $n$ darts misses the true probability $p$ by about

on a typical run. I show where this comes from in The law of large numbers; here it is enough to read it. For the circle of radius 0.4 it gives a typical miss of 0.158 after 10 darts, 0.050 after 100 and 0.016 after 1000.

A hundred times as many darts make the miss ten times smaller. So the 7 darts in 10 above are only $(0.7-0.5)/0.158=1.3$ typical misses away from the share of 0.5. That is ordinary.

Outcomes, events and the sample space

Let me give the parts of this game their names. Each throw ends in one outcome: one spot on the square. For a die, the outcomes are the six faces. The set of all possible outcomes is the sample space, and I write it $S$. For the die, $S=\{1,2,3,4,5,6\}$; for the darts, $S$ is the whole square.

An event is a set of outcomes, a question you can answer yes or no once the throw is done. “The die shows an even number” is the event $\{2,4,6\}$. “The dart lands inside the circle” is the set of spots inside the circle. I name events with capital letters, $A$, $B$, and write $\Pr\{A\}$ for the probability of $A$.

Events combine like regions of the square:

• $A\cup B$, read ”$A$ or $B$”, is every outcome in $A$, in $B$, or in both. This is the union.
• $A\cap B$, read ”$A$ and $B$”, is every outcome in both. This is the intersection.
• $A^\mathrm{c}$, read “not $A$”, is every outcome outside $A$. This is the complement.

Two events that cannot happen together, with no outcome in common, are called disjoint. “The die shows 1” and “the die shows an even number” are disjoint. Their intersection is the empty set, $\varnothing$.

Three rules, and two that follow

Think of $\Pr\{A\}$ as the long-run share of throws that land in $A$. Three facts about shares are plain:

1. A share is never negative: $\Pr\{A\}\ge0$.
2. Every throw lands somewhere in $S$, so $\Pr\{S\}=1$.
3. If $A$ and $B$ are disjoint, a throw lands in at most one of them, so their shares add: $\Pr\{A\cup B\}=\Pr\{A\}+\Pr\{B\}$.

These three rules are the whole foundation. Everything else on this page follows from them.

The complement rule. $A$ and $A^\mathrm{c}$ are disjoint and together make up $S$. Rules 2 and 3 give $\Pr\{A\}+\Pr\{A^\mathrm{c}\}=1$, so

The chance a dart misses the circle of radius 0.4 is $1-0.5027=0.4973$.

The addition rule. When $A$ and $B$ overlap, adding their shares counts the overlap twice: look at the first panel of the figure. Take it away once:

To see that it follows from rule 3, split $A\cup B$ into three disjoint pieces: $A$ without $B$, the overlap, and $B$ without $A$. Then add the pieces.

Equally likely outcomes. When $S$ has a finite number of outcomes and each is as likely as the others, a probability is a count. I write $\lvert A\rvert$ for the number of outcomes in $A$:

A fair die gives $\Pr\{\text{even}\}=3/6=1/2$. The darts are the same idea with area in place of a count: the probability of landing in a shape is its area divided by the area of the square.

Worked example

Roll two fair dice, one after the other. Each outcome is a pair, (first die, second die), and all 36 pairs are equally likely, so $\lvert S\rvert=36$. Let $A$ be “the sum is 7” and $B$ be “at least one die shows a six”. What is $\Pr\{A\cup B\}$, the chance of a sum of 7 or a six?

Count each event. $A$ holds (1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1): 6 outcomes. $B$ is the last column and the last row of the grid, 6 + 6 cells with (6, 6) counted twice: 11 outcomes. Their intersection holds only (1, 6) and (6, 1): 2 outcomes.

Check $B$ by its complement. “No six on either die” leaves 5 faces for each die, $5\times5=25$ pairs. So $\Pr\{B\}=1-25/36=11/36$, the same as the count.

Add, and take the overlap away once.

Count the cells in the figure that are shaded or ringed, or both: 15. The rule and the picture agree.

Try it yourself.

• A circle of radius 0.3 in the unit square: the chance a dart lands inside is $\pi\times0.09=0.283$.
• Throw 500 darts at a shape of area 0.2. Expect about $500\times0.2=100$ inside, give or take about $\sqrt{500\times0.2\times0.8}=8.9$, so about 9 darts.

Where you’ll meet this

Reliability. An engineer who says a part fails with probability 0.001 in its first year means that, of many such parts, about one in a thousand fail in their first year. The complement rule gives the other side: 0.999 survive.

Weather forecasts. “A 70 % chance of rain” means that on days like this one, it rained on about 70 of every 100. It does not promise rain tomorrow, and a dry day does not make the forecast wrong.

Games and estimates by darts. Card and dice games are built on counts of equally likely outcomes, like the dice grid above. And throwing random points to measure an area, as the instrument does, is a working method: it is called Monte Carlo estimation, and it measures areas and volumes too awkward for a formula. Its error shrinks like the typical miss above, ten times smaller for a hundred times as many points.

Random noise in a signal is the same idea, a fresh random number at every moment; Random variables for signals in the signal processing course follows that thread. Here, the next page, Counting, finds $\lvert A\rvert$ and $\lvert S\rvert$ when there are far too many outcomes to list.

Reference card