USMLE Step 1Biostatistics and epidemiologyEP 07 of 08

Type I and type II errors: alpha, beta, power and the p-value

A type I error rejects a true null hypothesis; a type II error misses a real effect. Learn how alpha, beta, power and the p-value fit together, and what raises power.

  • 5:16 video
  • 3 min read
  • 6 sources
  • Published 16 Nov 2026
Chapters (11)
By the end of this lesson

Define type I and type II errors, alpha, beta and power, and explain the factors that increase power.

The short version

  1. A type I error rejects a true null hypothesis (probability alpha); a type II error fails to reject a false one (probability beta).
  2. Power is 1 − beta: the chance of detecting a real effect of the size the study was designed for, usually set at 80%.
  3. A p-value below alpha is statistically significant; a p-value above alpha does not prove there is no effect.
  4. Power rises with a larger sample, a larger true effect, less variability in the outcome or a higher alpha.

A p-value of 0.08 does not mean the drug does not work

A trial compares a new drug with placebo and the p-value comes back at 0.08. The headline reads: the drug does not work. That is not what a p-value of 0.08 says.[1]

To see why, you need the vocabulary of type I and type II errors: the two ways a significance test can be wrong, and the probabilities that go with them, alpha, beta and power.

The null hypothesis and the two by two table

Every significance test starts by assuming the null hypothesis: there is no difference between the groups. The alternative hypothesis says there is one. The test asks whether the data give enough evidence to reject the null.[1]

Draw a two by two table, the same shape as the one used for sensitivity and specificity. The columns are the truth: the null is really true, or really false. The rows are your decision: reject it, or fail to reject it.[2][5]

Truth against decision
DecisionNull is really trueNull is really false
Reject the nullType I error (false positive): you claimed a difference that is not thereCorrect decision
Fail to reject the nullCorrect decisionType II error (false negative): you missed a difference that is there

Both errors come from chance in sampling.[1] The wording matters: the exam convention is to say "fail to reject the null", not "accept the null".[3]

Alpha, beta and power

  • Alpha is the probability of a type I error. It is the significance level, fixed before the study starts, usually 0.05.[1]
  • Beta is the probability of a type II error.[1]
  • Power is 1 − beta: the chance of detecting a real effect of the size the study was designed for, usually set at 80%.[1][2]
Power
Power = 1 − β

β = probability of a type II error; conventional β = 0.20, so power = 80% (sometimes 90%)

Alpha lines up with the false-positive rate and power with the true-positive rate of the diagnostic two by two table.[2]

What alpha and power mean across 100 trials

The percentages become concrete if you imagine repeating a study many times.

Two sets of 100 trials
  1. A drug that truly works, each trial with 80% power: about 80 detect the effect.
  2. The other 20 miss it: those misses are type II errors (20 / 100 = 0.20 = β).
  3. A drug that truly does nothing, alpha 0.05: about 5 of 100 still look significant by chance.
  4. Those 5 are type I errors (5 / 100 = 0.05 = α).

The p-value: what it is and what it is not

The p-value is the probability of getting results at least this extreme if the null hypothesis is true. If p is below alpha, we reject the null and call the result statistically significant.[1][3]

Back to the opening trial: 0.08 is above an alpha of 0.05, so we fail to reject the null. That does not prove the drug is useless. A small trial may simply have lacked the power to see a real effect, which would be a type II error.[1][3]

What increases statistical power

Picture two bell curves: the null on the left, the real effect on the right, and a cut-off line between them. The part of the left curve beyond the line is alpha. The part of the right curve short of the line is beta. The rest of the right curve is power. This is a schematic built from the definitions of alpha, beta and power.[1][2]

  1. A larger sample: the curves narrow, overlap less, and power rises.[2][6]
  2. A larger true effect: the right curve slides away from the left.[2][6]
  3. Less variability in the outcome: again the curves narrow.[2][6]
  4. A higher alpha: the cut-off moves toward the null, so more real effects are caught, at the price of more type I errors.[2][6]

Tightening alpha, for example from 0.05 to 0.01, lowers power at a fixed sample size.[2][6]

Practice question

Original · USMLE style
Your turnQB-MS-S1-BIOS-007-01

A randomized trial of 24 patients per group compares a new antihypertensive drug with placebo. Systolic blood pressure falls by 6 mm Hg more in the drug group, with a p-value of 0.16. The significance level was set at 0.05. A much larger trial later demonstrates that the drug does lower blood pressure.

Which of the following best describes the result of the small trial?
Pick one option. Your answers stay on this device.

Common questions

What is the difference between a type I and a type II error?

A type I error rejects a null hypothesis that is actually true, a false positive. A type II error fails to reject a null hypothesis that is actually false, a false negative.

What is statistical power?

Power is 1 − beta: the probability of detecting a real effect of the size the study was designed for. It is usually set at 80%.

Does a p-value above 0.05 mean there is no effect?

No. It means the evidence was insufficient. A small, underpowered study can miss a real effect, which is a type II error.

What does a p-value actually tell you?

It is the probability of getting results at least this extreme if the null hypothesis is true. It is not the probability that the null hypothesis is true.

How do you increase the power of a study?

Use a larger sample, study a larger true effect, reduce variability in the outcome, or accept a higher alpha, which brings more type I errors.

Transcript

Read the full narration, by chapter693 words

0:00The hook: p = 0.08 and "the drug does not work"

A trial compares a new drug with placebo, and the p-value comes back 0.08. The headline reads: the drug does not work. But that is not what a p-value of 0.08 says. Today: define type one and type two errors, alpha, beta and power, and see what makes a study more powerful.

0:22The null hypothesis and the two-by-two table

Every significance test starts by assuming the null hypothesis: there is no difference between the groups. Draw a two by two table, as in the sensitivity lesson. The columns are the truth: the null is really true, or really false. The rows are your decision: reject it, or fail to reject it.

0:43Type I and type II errors

Reject a null that is true: a type one error, a false positive. You claimed a difference that is not there. Fail to reject a null that is false: a type two error, a false negative. You missed a difference that is there. Both come from chance in sampling.

1:02Alpha, beta and power

Alpha is the probability of a type one error. It is the significance level, fixed before the study starts, usually 0.05. Beta is the probability of a type two error. Power is one minus beta: the chance of detecting a real effect of the size the study was designed for, usually set at 80%.

1:26100 trials: 80 detect, 20 miss, 5 false alarms

Picture a 100 trials of a drug that truly works, each with 80% power. About 80 detect the effect. About 20 miss it. Those misses are type two errors. Now a 100 trials of a drug that truly does nothing, with alpha at 0.05. About five will still look significant by chance. Those are type one errors.

1:52What a p-value is, and what it is not

The p-value is the probability of getting results at least this extreme if the null hypothesis is true. If p is below alpha, we reject the null and call the result statistically significant. It is not the probability that the null hypothesis is true. Back to our trial: 0.08 is above an alpha of 0.05, so we fail to reject the null.

2:18Low power, and what raises it: two bell curves, four levers

That does not prove the drug is useless. A small trial may simply have lacked the power to see a real effect. So what raises power? Picture two bell curves: the null on the left, the real effect on the right, a cut-off line between. Left curve beyond the line: alpha. Right curve short of the line: beta. The rest of the right curve: power. One: a larger sample. The curves narrow, overlap less, and power rises. Two: a larger true effect. The right curve slides away from the left. Three: less variability in the outcome. Again the curves narrow. Four: a higher alpha. The cut-off moves toward the null, so more real effects are caught, at the price of more type one errors.

3:13Practice question

Your turn. A trial of 24 patients per group compares a new antihypertensive with placebo. Systolic pressure falls by six millimetres of mercury more with the drug, with a p-value of 0.16. The significance level was 0.05. A much larger trial later demonstrates that the drug does lower blood pressure. Which best describes the small trial? A, type one error, probability alpha. B, type one error, probability one minus beta. C, type two error, probability alpha. D, type two error, probability beta. E, type two error, probability one minus beta. Pause here and decide.

4:05The options, wrong ones first

A is out: a type one error rejects a true null, and this trial rejected nothing. B is out: the same wrong error, and one minus beta is power. C is out: right error, but alpha belongs to type one. E is out: one minus beta is power, the opposite of missing an effect.

4:26The answer: type II error, probability beta

So the answer is D. The drug works, so the null was false. The small trial failed to reject it: a type two error, with probability beta.

4:37Recap and next lesson

Three things to keep. One: a type one error rejects a true null, probability alpha. A type two error misses a real effect, probability beta. Power is one minus beta. Two: p below alpha is significant. P above alpha does not prove there is no effect. Three: power rises with a larger sample, a larger effect, less variability or a higher alpha. Next, confidence intervals and statistical significance.

Sources

Numbers in the text, e.g. [1], point here
  1. Banerjee A, Chitnis UB, Jadhav SL, Bhawalkar JS, Chaudhury S. Hypothesis testing, type I and type II errors. Ind Psychiatry J. 2009;18(2):127-131. pmc.ncbi.nlm.nih.gov/articles/PMC2996198/
  2. Hickey GL, Grant SW, Dunning J, Siepe M. Statistical primer: sample size and power calculations, why, when and how? Eur J Cardiothorac Surg. 2018;54(1):4-9. pmc.ncbi.nlm.nih.gov/articles/PMC6005113/
  3. Sener A. Interpreting P values in clinical research: a practical guide for clinicians. Turk J Emerg Med. 2026;26(4):281-287. pmc.ncbi.nlm.nih.gov/articles/PMC13722342/
  4. Greenland S, Senn SJ, Rothman KJ, et al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 2016;31:337-350. pmc.ncbi.nlm.nih.gov/articles/PMC4877414/
  5. OpenStax Introductory Statistics 2e, sections 9.1 and 9.2. openstax.org/books/introductory-statistics-2e/pages/9-2-outcomes-and-the-type-i-and-type-ii-errors
  6. Wikipedia (background, checked against the sources above): Type I and type II errors; Statistical power. en.wikipedia.org/wiki/Type_I_and_type_II_errors · en.wikipedia.org/wiki/Statistical_power

The bell curves on screen are a schematic with no numeric axis. The hook and teaching numbers, 100 trials, 80% power and alpha 0.05, are teaching examples.

The practice question is original and its patient is synthetic. Every fact is checked against the sources above in independent accuracy reviews. Always follow your local guidelines and your own laboratory’s ranges.

Image credits. Illustrations: Servier Medical Art (smart.servier.com), CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/), adapted (recoloured, outlined, resized): adult man, tablets, blood pressure apparatus. The result slip, newspaper clipping, tables, dot grids, number line, bell curves, question card and all handwriting are drawn in code (original). No AI-generated teaching images.

Corrections. None so far. Spotted an error? Email info@medservatory.com with the lesson code MS-S1-BIOS-007.

For exam study only, not medical advice. Read the medical disclaimer.

  1. EP 01Sensitivity vs specificity: the 2×2 table, SnNout and SpPin5:34
  2. EP 02PPV, NPV and prevalence explained5:27
  3. EP 03Relative risk vs odds ratio: the 2x2 table explained5:15
  4. EP 04Number needed to treat: ARR, RRR and NNH from trial data5:00
  5. EP 05Study designs explained: case-control, cohort, cross-sectional and RCT5:22
  6. EP 06Types of bias: selection, recall, lead-time and confounding5:30
  7. EP 07Type I and type II errors: alpha, beta, power and the p-value5:16
  8. EP 08Confidence intervals and significance: is the null value inside?5:12