Bayes’ theorem

A test finds 99 % of the ill and wrongly flags 5 % of the healthy. You test positive. How likely is it that you are ill?

Count the people first

Before the clip you guessed. Most people pick 99 %, because the test is right 99 % of the time for the ill. The picture says one in six.

Count with it. Of 10 000 people, 100 are ill and 9900 are healthy. The test flags 99 of the ill. It also flags 5 % of the healthy, and 5 % of 9900 is 495. So 594 people are flagged, and only 99 of them are ill:

Five in six of the flagged people are healthy. The test is good; the trouble is that the healthy are 99 times as many as the ill, so 5 % of them outnumber 99 % of the ill.

After the clip, drag how common the illness is. At 10 %, a positive test means ill about 69 times in 100; at 0.1 %, only about 2 times in 100. The rarer the illness, the more the false alarms from the healthy crowd the box.

Counting people like this, whole numbers out of a crowd instead of percentages, is called using natural frequencies. In studies, far more people get these questions right when they are asked in counts than in percentages. That is why this page counts first and writes the formula second.

From the box to Bayes’ formula

Name the events. Let $A$ be “the person is ill” and $B$ be “the test flags them”. The test’s makers tell you $\Pr\{B\mid A\}=0.99$ and $\Pr\{B\mid A^\mathrm{c}\}=0.05$. You want the other direction, $\Pr\{A\mid B\}$.

By the definition of Conditional probability, $\Pr\{A\mid B\}=\Pr\{A\cap B\}/\Pr\{B\}$. The top is the ill people who are flagged; the multiplication rule gives it as

99 people in 10 000. The bottom is everyone in the box, ill or healthy, which the law of total probability adds up:

594 people in 10 000.

Put the two together and you have Bayes’ theorem:

Each piece has a name, because it plays the same part in every use:

• $\Pr\{A\}$ is the prior: how likely the cause was before the evidence. Here, the base rate, 1 %.
• $\Pr\{B\mid A\}$ is the likelihood: how likely the evidence is if the cause is real. Here, 99 %.
• $\Pr\{A\mid B\}$ is the posterior: how likely the cause is after the evidence. Here, one in six.

The theorem turns the likelihood, which the test’s makers can measure, into the posterior, which the person tested wants to know.

Odds

There is a shorter way to do the same sum. The odds of an event compare its chance with the chance against it. Here 1 person in 100 is ill and 99 are not, so the odds of being ill are 1 to 99, written $1:99$.

Write Bayes’ theorem once for $A$ and once for $A^\mathrm{c}$, and divide one by the other. $\Pr\{B\}$ cancels, and what is left is

In words: the odds after the evidence are the odds before, times the likelihood ratio, how many times more likely the evidence is if the cause is real than if it is not.

For the test, the likelihood ratio is $0.99/0.05=19.8$. The odds go from $1:99$ to $19.8:99$, which is $1:5$: one ill person for every five healthy ones in the box, a chance of $1/6$.

The box said the same: 99 ill to 495 healthy is $1:5$.

Worked example

1. How common the illness is. Keep the same test and change the base rate. Counting 100 000 people makes every count a whole number:

The same test is nearly useless for a rare illness and very good for a common one. When half the people tested are ill, a positive result means ill 95 times in 100.

2. Testing again. You tested positive once; your chance of being ill is now one in six. You take a second, separate test of the same kind, and it is positive too. I assume the two tests make their mistakes separately: given that you are ill, or given that you are healthy, the first result tells nothing about the second.

Then the first posterior is the second test’s prior. The odds go from $1:5$ to $19.8:5=3.96$, a chance of $3.96/4.96=0.798$. A third positive gives odds $78.4$, a chance of $0.987$.

Watch the same thing as bars for one person.

Each positive test multiplies the odds by 19.8. Three in a row turn one in a hundred into nearly certain.

3. A spam filter. Of the mail you receive, 30 % is spam. The word “prize” is in 50 % of spam and in 5 % of other mail. A message contains “prize”. How likely is it to be spam?

Count 1000 messages: 300 are spam and 150 of those contain “prize”; 700 are not, and 35 of those contain it. So 185 messages contain “prize”, and 150 of them are spam:

Where you’ll meet this

Medical screening. Before a screening programme starts, its planners work out how many of the people it flags will be healthy. For a rare condition it is most of them, so a positive screen leads to a second, more careful test, not straight to treatment.

Spam filters. A filter starts from the share of mail that is spam, the prior, and multiplies its odds by a likelihood ratio for each word it sees, as in the third example, assuming the words make their mistakes separately.

Fault diagnosis and alarms. A warning light on a machine is a test with a base rate: if the fault is rare, most alarms are false. Engineers who set an alarm’s threshold trade missed faults against false alarms.

Reference card