Conditional probability

Of 100 people, 40 own a bike. Keep only the 50 who live in the city, and count the bikes again.

Knowing shrinks the world

Before the clip you guessed. Among everyone, 40 people in 100 own a bike, a share of 0.40. Among the 50 city people, 30 own a bike, a share of 0.60. Knowing that someone lives in the city changed the answer.

What happened in the picture is the whole idea. Once you know a fact, every person it rules out leaves the picture. The world shrinks to the people who are left, and the new probability is a share of that smaller world.

The probability of $A$ once you know that $B$ happened is called the conditional probability of $A$ given $B$. I write it $\Pr\{A\mid B\}$, and read the bar as “given”. With $A$ “owns a bike” and $B$ “lives in the city”, the picture counted

Divide the top and the bottom by the 100 people in the whole grid, and the counts become probabilities:

This is the definition, and it works whenever $\Pr\{B\}$ is not 0. The top is the part of $A$ that survives the shrinking. The bottom is the size of the new world.

After the clip, switch the fact to “owns a bike”. Now the 60 people without a bike fade. Of the 40 bike owners, 30 live in the city: $\Pr\{B\mid A\}=30/40=0.75$.

The multiplication rule

Turn the definition around and you get a way to find the chance that two things both happen:

In words: first $B$ must happen, then, in the world where it did, $A$ must happen too. This is the multiplication rule.

Deal two cards from a shuffled deck of 52. What is the chance both are aces? The first card is an ace with chance $4/52$. Given that it was, 51 cards are left and 3 of them are aces, so the second is an ace with chance $3/51$:

The second card’s chance depends on the first, because the first card is not put back. The conditional probability carries that.

Trees and total probability

A factory has two machines. Machine 1 makes 60 % of the parts, and 2 % of its parts are defective. Machine 2 makes 40 %, and 5 % of its parts are defective. What share of all parts is defective?

Counts first. Of 1000 parts, 600 come from machine 1 and 2 % of those, 12, are defective. 400 come from machine 2 and 5 % of those, 20, are defective. So 32 of the 1000 are defective.

A probability tree draws this. Each branch carries a conditional probability, and following a path multiplies them, by the multiplication rule.

Let $D$ be “the part is defective”, $B_1$ “it came from machine 1” and $B_2$ “it came from machine 2”. Every part comes from exactly one machine, so $D$ splits into two disjoint pieces, one per machine. Adding the pieces gives

32 parts in 1000, as counted.

The same works for any number of pieces. Events $B_1,B_2,\dots$ that never overlap and together cover $S$ are a partition, and for any event $A$

This is the law of total probability: add up every path of the tree that ends in $A$.

Three doors

Here is a famous puzzle that uses everything on this page. A game show has three closed doors. Behind one is a car; behind the other two, goats. You pick a door. The host, who knows where the car is, opens one of the other two doors and shows a goat. Then you may stay with your door, or switch to the last closed door.

Most people say it makes no difference: two doors are left, so it is fifty-fifty. Make your guess, then watch 600 games.

In the clip, switching won 410 of the 600 games and staying won 190, close to the 400 and 200 that two thirds and one third would give. Switching wins about twice as often as staying. Here is why, with counts first.

Say you always pick door 1, and picture 600 games with the car behind each door exactly 200 times. When the car is behind door 1, the host can open either other door; say the host opens door 3 in half of those games, 100. When the car is behind door 2, the host must open door 3: all 200 games. When it is behind door 3, door 3 stays shut.

So in 300 games you see door 3 open with a goat, and the car is behind door 2 in 200 of them. Given what you saw, switching to door 2 wins 200 times in 300. The same holds when the host opens door 2.

In symbols, by total probability and then the definition:

Your first pick was right one time in three, and nothing the host does changes that. The host’s knowledge is what pushes the other two thirds onto the one door the host leaves closed.

Worked example

1. Which machine made it? In the factory above, you pick up a part and it is defective. What is the chance it came from machine 1?

Shrink the world to the 32 defective parts. Machine 1 made 12 of them, so

Machine 1 makes most of the parts, but only 37.5 % of the defective ones. Turning “defective given machine” around into “machine given defective” is the subject of Bayes’ theorem.

2. A host who does not know. Suppose the host opens one of the two other doors at random, and it happens to show a goat. Does switching still help?

Again you pick door 1. The car is at door 1 and the host opens door 3, with chance $\tfrac13\cdot\tfrac12=\tfrac16$; that door has a goat. The car is at door 2 and the host opens door 3, also with chance $\tfrac13\cdot\tfrac12=\tfrac16$; that door has a goat too.

The two paths that show a goat behind door 3 are equally likely, so stay and switch each win with chance $\tfrac12$. A random host who happens to show a goat tells you nothing about doors 1 and 2.

3. A queen, then not a queen. From a deck of 52, what is the chance the first card is a queen and the second is not? $\tfrac{4}{52}\cdot\tfrac{48}{51}=\tfrac{16}{221}=0.0724$.

Where you’ll meet this

Diagnosis and reliability. A doctor or an engineer rarely asks “how likely is a fault?” in general. They ask how likely it is given what they see: given this batch, this symptom, this alarm. Each new fact shrinks the world, as in the grid.

Sorting by evidence. A spam filter asks how likely a message is to be spam given the words in it. Getting the direction of the condition right is the subject of Bayes’ theorem.

Sampling without putting back. Dealing cards, drawing a sample of people for a survey, or testing parts from a batch without returning them: each draw changes the world the next one comes from, as with the two aces.

Games. The three doors show how much a little knowledge can be worth, and why it matters who knew what.

Reference card