Bayes' theorem is going to reappear almost immediately, in a place it might not look at first like it belongs: the very definition of information. It's worth having the statement fresh before that happens.
Start with a belief about how likely something is — that's the prior. New evidence comes in, and Bayes' theorem tells you exactly how to revise that belief into an updated one — the posterior — instead of guessing at the revision by feel. The likelihood answers a different question than the posterior does: not "how likely is A, given the evidence?" but "if A were true, how likely is this exact evidence?" Those two questions get confused constantly, and mixing them up is the single most common mistake in reasoning about evidence. The number on the bottom, the evidence term, is just a normalizing constant: the total chance of seeing B at all, averaged over every way A could have turned out (the law of total probability) — it's there so the posterior comes out as a genuine probability, between 0 and 1.
The one-sentence version: multiply the prior by the likelihood, then divide by however likely the evidence was overall. Whatever you believed before the evidence, times how well the evidence fits that belief, rescaled so the answer is still a legitimate probability.
Bayes' theorem is the rule for updating a probability once new evidence arrives. For two events A and B with P(B) \neq 0, P(A\mid B) = \frac{P(B\mid A)\,P(A)}{P(B)}. Four names attach to the four quantities in that equation. P(A) is the prior — the probability assigned to A before B is known. P(A\mid B) is the posterior — the updated probability of A once B is known to have happened. P(B\mid A) is the likelihood — how probable the evidence B is, assuming A is true. P(B), the evidence term in the denominator, is the overall probability of seeing B at all, regardless of whether A holds; when A has only finitely many mutually exclusive alternatives A_1,\ldots,A_n, the law of total probability expands it as P(B) = \sum_i P(B\mid A_i)P(A_i) — sum the likelihood times the prior over every way B could have happened.
The theorem itself is nothing more than conditional probability run in both directions: P(A\mid B)P(B) = P(A\text{ and }B) = P(B\mid A)P(A) (the joint probability of A and B, written two ways), and dividing both ends by P(B) isolates P(A\mid B).
Theorem
Bayes' Theorem
Recalled from College Algebra · Odds, Bayes' theorem, and binomial probability — first taught there.
For events A and B with P(B) \neq 0, P(A \mid B) = \frac{P(B \mid A)\,P(A)}{P(B)}.
The Conditional Probability formula, applied to P(A\mid B) and to P(B\mid A) separately, gives two ways to write the same joint probability: P(A\text{ and }B) = P(A\mid B)\,P(B), \qquad P(A\text{ and }B) = P(B\mid A)\,P(A). Both right-hand sides equal the same quantity, so they equal each other: P(A\mid B)\,P(B) = P(B\mid A)\,P(A). Dividing both sides by P(B) (nonzero by hypothesis) isolates P(A\mid B): P(A\mid B) = \frac{P(B\mid A)\,P(A)}{P(B)}.
The connection ahead, previewed without the notation yet: information theory measures how much learning that event B happened changes what you believe about event A. Bayes' theorem is precisely the tool that turns P(A) (belief before) into P(A\mid B) (belief after). Chapter 1 defines the information in that update as a function of the ratio P(A\mid B)/P(A) — how much the posterior moved relative to the prior — so reading Bayes' theorem fluently now means Chapter 1's definition will look like a direct continuation of something familiar, not a new idea appearing from nowhere.
Worked example
The ratio that's about to matter
-
1
A disease affects 1% of a population. A test is 95% accurate for both sick and healthy people (95% true positive rate, 95% true negative rate). Given a positive result, find P(\text{sick}\mid \text{positive}), and compare it to the prior P(\text{sick}).
-
2
Apply Bayes' theorem. P(\text{sick}\mid\text{pos}) = \dfrac{P(\text{pos}\mid\text{sick})P(\text{sick})}{P(\text{pos})}. The numerator is 0.95\times 0.01 = 0.0095. The denominator, by the law of total probability, is 0.95\times 0.01 + 0.05\times 0.99 = 0.0095 + 0.0495 = 0.059.
-
3\boxed{P(\text{sick}\mid\text{pos}) = \frac{0.0095}{0.059}\approx 0.161}
The posterior (16.1%) is over 16 times the prior (1%) — a positive test result carries real information, even though the final probability is still well under one-half.
Worked example
The evidence term with three alternatives, not two
-
1
A factory buys the same part from three suppliers: 50% from A, 30% from B, 20% from C. Each supplier's defect rate is different: 2% for A, 5% for B, 1% for C. A part is drawn at random and found defective. Find P(A\mid\text{defective}).
-
2
Build the evidence term first. With three alternatives instead of two, P(\text{defective}) needs the law of total probability summed over all three suppliers: P(\text{def}) = P(\text{def}\mid A)P(A)+P(\text{def}\mid B)P(B)+P(\text{def}\mid C)P(C) = (0.02)(0.5)+(0.05)(0.3)+(0.01)(0.2) = 0.01+0.015+0.002 = 0.027.
-
3\boxed{P(A\mid\text{defective}) = \frac{P(\text{def}\mid A)P(A)}{P(\text{def})} = \frac{0.01}{0.027} \approx 0.370}
Supplier A provides half the parts overall, but only 37% of the defective ones — a lower defect rate elsewhere (C) pulls the posterior below the prior even for a supplier as large as A.
The same numbers, as a tree
Click both 'sick' and 'healthy' branches' positive-test leaves — the sum of their counts is exactly the denominator, 0.059, from the worked example.
Loading interactive…
1,000 people, sorted by test result
Drag the prevalence slider down toward 0.1% — watch the posterior P(D|+) fall even further below what the test's own accuracy would suggest.
Loading interactive…
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
What this task checks
Structure checked automatically. Whether your ideas are convincing isn't graded, so reread with the checklist.
The compiler says
A model answer
Correct answer
Also accepted:
Why
Prior, likelihood, or posterior?
Sort each quantity in Bayes' theorem's formula by its role.
Loading interactive…
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
What this task checks
Structure checked automatically. Whether your ideas are convincing isn't graded, so reread with the checklist.
The compiler says
A model answer
Correct answer
Also accepted:
Why
Why
Where this goes next
Chapter 1 defines self-information from exactly this kind of before/after ratio, and Chapter 15 returns to Bayes' theorem directly when it discusses maximum-likelihood and maximum-entropy inference — the same update rule, reused as the foundation of statistical estimation.
Check your understanding (counts toward your progress)
In P(A\mid B) = \dfrac{P(B\mid A)P(A)}{P(B)}, which quantity is the posterior?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
P(A) is the prior — belief about A before observing B.
P(B\mid A) is the likelihood, not the posterior.
P(B) is the normalizing evidence term, not the posterior.
Why
P(A\mid B) — belief about A AFTER observing B — is the posterior, the quantity Bayes' theorem solves for.
A rare-event detector has excellent 99% sensitivity and specificity, but the event it detects occurs in only 1 in 10,000 cases. What does Bayes' theorem say about P(\text{event}\mid\text{flagged})?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
This is the base-rate trap — high accuracy doesn't survive a low enough prior.
The flag DOES move the probability, just not nearly as far as the detector's own accuracy suggests.
Bayes' theorem computes it exactly from the prior, sensitivity, and specificity, all given.
Why
With a 1-in-10,000 prior, false positives from the 99% of the population without the event vastly outnumber true positives, even at 99% specificity — the posterior stays well under 50%.
Compare how many people the 99% specificity flags falsely among the huge event-free population against how many the 99% sensitivity flags correctly among the tiny event-present population — which group is bigger?
A channel has a 2% error rate. A parity check flags an error with 99% probability when one occurred, and flags a false alarm 3% of the time when there was no error. Given the check flags an error, find P(\text{error} \mid \text{flagged}) to 3 decimal places.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
P(\text{error}\mid\text{flagged}) = \dfrac{0.99\times 0.02}{0.99\times 0.02 + 0.03\times 0.98} = \dfrac{0.0198}{0.0492}\approx 0.403.
Correct answer
Also accepted:
Why
Build the denominator first using the law of total probability — the false-flag rate applies to the 98% of channels with no error, not to the 2% that do.
Practice: Bayes' theorem (5 problems · doesn’t affect your progress)
A spam filter flags spam correctly 98% of the time and false-flags legitimate email 4% of the time; 5% of all email is spam. In this scenario, P(\text{flagged}\mid\text{spam})=0.98 plays which role in Bayes' theorem?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
The prior is P(\text{spam})=0.05, the belief before seeing whether it was flagged.
The posterior is P(\text{spam}\mid\text{flagged}), the quantity being solved for — not a quantity that's already given.
The evidence term is P(\text{flagged}) overall, not this conditional probability.
Why
P(\text{flagged}\mid\text{spam}) answers 'if it's spam, how likely is a flag' — that's the likelihood, exactly as defined in this section's technical box.
Likelihood
Using the spam-filter numbers (5% spam rate, 98% true-positive rate), compute the numerator of Bayes' theorem, P(\text{flagged}\mid\text{spam})\,P(\text{spam}).
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
0.98 \times 0.05 = 0.049.
Correct answer
Also accepted:
Why
Continuing the spam-filter example (5% spam, 98% true-positive rate, 4% false-positive rate), find P(\text{spam}\mid\text{flagged}) to three decimal places.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
The denominator is 0.98(0.05)+0.04(0.95) = 0.049+0.038=0.087, so P(\text{spam}\mid\text{flagged}) = 0.049/0.087 \approx 0.563.
Correct answer
Also accepted:
Why
Build the denominator with the law of total probability: the false-positive rate applies to the 95% of email that's legitimate, not to the 5% that's spam.
If the spam rate dropped from 5% to 1% but the filter's accuracy stayed the same, what happens to P(\text{spam}\mid\text{flagged})?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
How rare the flag itself is doesn't determine the direction — what matters is how the SHARE of flags that are true spam changes.
The accuracy numbers staying fixed doesn't freeze the posterior — a smaller prior still pulls it down, exactly the base-rate lesson from the earlier worked example.
A lower prior pulls the posterior down, but it doesn't zero it out completely — some genuine spam still gets correctly flagged.
Why
A lower prior means false positives from the now much larger legitimate-email pool make up a bigger share of every flag raised, pulling P(\text{spam}\mid\text{flagged}) down — the same base-rate effect the disease-test worked example showed.
It goes down, since spam becomes a smaller share of what gets flagged
Suppose a spam filter's true-positive rate and false-positive rate happen to be EQUAL (say both 50%). What happens to P(\text{spam}\mid\text{flagged})?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
The 50% figures are the filter's OWN rates, not the resulting posterior probability — they don't transfer directly to the answer.
Nothing about equal rates makes the formula undefined; it's a perfectly ordinary input to Bayes' theorem.
It doesn't shift at all in this case — read the ratio P(\text{flagged}\mid\text{spam})/P(\text{flagged}\mid\text{not spam}) carefully when the two rates are equal.
Why
When P(\text{flagged}\mid\text{spam})=P(\text{flagged}\mid\text{not spam}), Bayes' ratio collapses so the posterior equals the prior exactly — evidence that doesn't discriminate between the two hypotheses carries zero information, a preview of the mutual-information idea ahead.
It equals the prior P(\text{spam}) exactly — the flag carries no information