Every hypothesis test in this course ends with the same kind of sentence: "if this were true, a result like ours would happen only 3% of the time." That sentence only means something if probability is understood correctly. Probability must mean a long-run frequency. It measures how often something happens, over and over, not just once. It is not a one-time guess.
Quick review
Practice from earlier coursework, pulled in for this topic.
From Probability
Which of these is a THEORETICAL probability, not an empirical one?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
This is a count from an actual run of trials — that's the definition of empirical.
This is an observed relative frequency from real data, not a model-derived value.
This is an observed count from a specific practice session, not derived from a model.
Why
1/6 comes from a model (the assumption that each face is equally likely), not from counting actual outcomes — that makes it theoretical. Every other option reports an observed frequency from real trials.
Ask which option was computed from a model's assumptions versus counted from something that actually happened.
From Probability
A die is rolled 12 times and lands on 6 exactly 4 times. What is the EMPIRICAL probability of rolling a 6, based on this data?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
1/6 is the THEORETICAL probability for a fair die, not what this particular run of data showed.
That doesn't match the given count of 4 successes out of 12 trials.
That doesn't match the given data either.
Why
Empirical probability is observed frequency: 4 successes out of 12 trials, or 4/12 = 1/3.
Flip a fair coin 10 times and you might see 7 heads. That's not proof the coin is unfair. It's just small-sample noise. Flip it 10,000 times, and the fraction of heads will land close to 50%. Probability is what the frequency settles toward, not what one short run shows.
The theoretical probability of an event comes from a model, before data is gathered. The empirical probability is the observed relative frequency, \dfrac{\text{times it happened}}{\text{trials run}}, found by running many trials. As trials increase, the observed fraction settles toward it. This is the idea behind the p-value.
Watching frequency settle toward probability
Run increasingly many trials and watch the running relative frequency approach the true probability.
Loading interactive…
Worked example
Small-sample noise
-
1
A coin believed fair is flipped 10 times and lands heads 7 times. Does this disprove fairness?
-
2
The observed fraction is 7/10 = 0.70. With only 10 trials, a fair coin can easily produce results this far from 0.5 just by chance.
-
3\boxed{\text{No} - \text{10 trials is far too few to distinguish 0.70 from ordinary noise around 0.5}}
Loading interactive…
Worked example
Long-run convergence
-
1
The same coin is flipped 100,000 times and lands heads 49,830 times. What does this suggest, and why does 100,000 trials carry more weight than 10?
-
2
The observed fraction is 49{,}830/100{,}000 = 0.4983. That is extremely close to 0.5. With this many trials, chance alone almost never pushes that fraction this far from the true probability, unless the true probability were actually different.
-
3\boxed{\text{Strong evidence the coin is close to fair} - \text{large-}n\text{ frequency is a much sharper reading of the truth}}
Loading interactive…
Where this goes next
Hypothesis testing, chapters 12-13
A p-value later in this course is exactly this idea run in reverse. First assume a claim is true. Then ask how often a sample this extreme would occur, over and over. A p-value of 0.03 means "3% of the time, under repeated sampling." It does not mean "3% chance the claim is true." This section's frequency framing is what makes that distinction precise.
Check your understanding (counts toward your progress)
Out of 250 trials, an event occurred 70 times. What is the empirical probability, as a decimal?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
70/250 = 0.28.
Correct answer
Also accepted:
Why
Why does an empirical frequency from only 10 trials carry much less weight than one from 100,000 trials?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
With few trials, random fluctuation can easily push the observed frequency far from the true probability; with many trials, that fluctuation shrinks relative to the signal, and the frequency reliably settles near the truth.
A fair coin has landed heads 6 times in a row. What is the probability the next flip is heads?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
The gambler's fallacy — independent trials don't owe a correction for a streak.
A streak doesn't change a fair coin's probability on the next independent flip.
Why
Independent trials are unaffected by prior outcomes; the theoretical probability stays 0.5 regardless of the streak.