Confidence intervals and significance: is the null value inside?
A confidence interval is a range built to capture the true value. Check whether it includes the null value, and you can judge significance and precision at a glance.
Chapters (11)
Interpret confidence intervals for differences and ratios and decide significance from whether the interval includes the null value.
The short version
- A 95% interval for a mean or a difference is the estimate ± 1.96 standard errors; ratio intervals are calculated on the log scale.
- "95%" describes the method: about 95% of intervals from repeated studies would contain the true value. It is not a 95% probability for one interval.
- The null value is 0 for differences and 1 for ratios.
- A 95% interval that includes the null value is not significant at 0.05; one that excludes it is significant.
- A wide interval that includes the null is inconclusive, not proof of no effect.
Two trials, one estimate, different significance
Two trials test the same drug. Both report a relative risk of death of 0.75. The small trial's 95% confidence interval runs from 0.45 to 1.25. The big trial's runs from 0.64 to 0.88.
Same estimate, yet only one trial found a significant effect. This lesson is about confidence intervals and significance: how to tell which trial, without a p-value, by checking the interval against the null value. The two trial intervals here are teaching schematics, not real trials.
What a confidence interval is
A trial measures one sample, so its result is an estimate. A confidence interval is a range around that estimate, built to capture the true value. It shows how much uncertainty the data leave.[1][2]
Recall from the previous lesson: a p-value below alpha rejects the null hypothesis of no effect. A 95% interval matches a two-sided alpha of 0.05, the usual exam convention.[1]
Building a 95% interval: estimate ± 1.96 SE
SE of a mean = SD / √n, so it shrinks as the sample grows
The standard error of a mean is the standard deviation divided by the square root of the sample size. A larger sample gives a smaller standard error.[1][3]
- A drug lowers systolic pressure by 8 mm Hg more than placebo, with a standard error of 3 mm Hg
- Margin = 1.96 × 3 = 5.88
- Lower limit = 8 − 5.88 = 2.12, about 2.1
- Upper limit = 8 + 5.88 = 13.88, about 13.9
- 95% CI = 2.1 to 13.9 mm Hg, which excludes 0, so the result is significant
Ratios work the same way, but on a log scale, so their intervals are lopsided around the estimate. That is why the hook interval 0.45 to 1.25 sits unevenly around 0.75.[4]
What "95%" does and does not mean
Repeat the study many times, and about 95% of the intervals built this way would contain the true value. The 95% describes the method over repeated studies.[2][6]
The null value: 0 for differences, 1 for ratios
| Type of measure | Examples | Null value (no effect) |
|---|---|---|
| Difference | Mean difference, risk difference | 0 |
| Ratio | Relative risk, odds ratio | 1 |
A relative risk of 1 means the exposure or treatment does not affect the outcome.[1][4][6]
If a 95% interval includes the null value, the result is not significant at 0.05. If it excludes the null value, the result is significant.[1][2][6]
Our worked example, 2.1 to 13.9, excludes 0, so it is significant.
Back to the two trials
Both trials report ratios, so the null value is 1.
| Trial | Relative risk | 95% CI | Includes 1? | Significant at 0.05? |
|---|---|---|---|---|
| Small | 0.75 | 0.45 to 1.25 | Yes | No |
| Big | 0.75 | 0.64 to 0.88 | No | Yes |
The big trial is the significant one, and the interval told us without a p-value.[1][2]
Width shows precision, and the range shows what is plausible
Width shows precision. A bigger sample or less variability narrows the interval; a higher confidence level widens it.[1][2][3]
The interval also shows what is plausible. The small trial's interval spans a 55% drop in risk (0.45) to a 25% rise (1.25).[1]
Practice question
Original · USMLE styleA randomized trial of an exercise program versus usual care reports the effect on five outcomes, each with a 95% confidence interval.
Common questions
How do you tell if a confidence interval is statistically significant?
Check whether it includes the null value. A 95% interval that includes the null value is not significant at 0.05; one that excludes it is significant.
What is the null value for a confidence interval?
It is 0 for difference measures such as a mean difference or risk difference, and 1 for ratio measures such as relative risk or an odds ratio.
How is a 95% confidence interval calculated?
For a mean or a difference it is the estimate ± 1.96 standard errors. Ratio intervals are calculated on the log scale, so they are not symmetric around the estimate.
What does a 95% confidence interval actually mean?
If the study were repeated many times, about 95% of the intervals built this way would contain the true value. It is not a 95% probability for one reported interval.
Does a wide interval that includes the null mean there is no effect?
No. It is inconclusive: it often reflects a small study or high variability, and it does not show that the drug has no effect.
Transcript
Read the full narration, by chapter631 words
0:00The hook: two trials, one relative risk of 0.75, only one significant
Two trials test the same drug. Both report a relative risk of death of 0.75. The small trial's 95% confidence interval runs from 0.45 to 1.25. The big trial's runs from 0.64 to 0.88. Same estimate, yet only one trial found a significant effect. Which one, and how can you tell without a p-value? Today: interpret confidence intervals and decide significance from the null value. A trial measures one sample,
0:33What a confidence interval is, and alpha
so its result is an estimate. A confidence interval is a range around the estimate, built to capture the true value. Recall: a p-value below alpha rejects the null hypothesis of no effect. A 95% interval matches an alpha of 0.05.
0:54The formula: estimate ± 1.96 SE, and the standard error
For a mean or a difference, a 95% interval is the estimate, plus or minus 1.96 standard errors. A mean's standard error is the standard deviation divided by the square root of the sample size, so it shrinks as the sample grows.
1:11Worked example: 8 mm Hg, SE 3, 2.1 to 13.9
Example: a drug lowers systolic pressure by eight millimetres of mercury more than placebo, with a standard error of three. The margin is 1.96 times three, 5.88, so the interval runs from 2.1 to 13.9 mm Hg. Ratios work the same way, but on a log scale, so their intervals are lopsided around the estimate.
1:37What "95%" means (and what it does not)
What does 95% mean? Repeat the study many times, and about 95% of intervals would contain the true value. It is not a 95% probability for this one interval.
1:51The null value: 0 for differences, 1 for ratios
For a difference, such as a mean difference or risk difference, no effect is zero. For a ratio, such as relative risk or odds ratio, no effect is one. If a 95% interval includes the null value, the result is not significant at 0.05. If it excludes it, the result is significant. Our example runs from 2.1 to 13.9.
2:15Our example, and back to the two trials
It excludes zero, so it is significant. Back to the two trials. Both report ratios, so the null value is one. The small trial: 0.45 to 1.25. That includes one, so it is not significant. The big trial: 0.64 to 0.88. That excludes one, so it is significant. Width shows precision.
2:45Width = precision, and what the interval says is plausible
A bigger sample or less variability narrows it; a higher confidence level widens it. And the interval shows what is plausible. The small trial's interval spans a 55% drop in risk to a 25% rise. A wide interval that includes the null value often reflects a small study or high variability. It does not show that the drug has no effect.
3:13Practice question
Your turn. A trial of an exercise programme reports five outcomes, each with a 95% interval. Judging each interval on its own, which outcome is statistically significant at the 0.05 level? A: body weight, mean difference minus three kilograms, interval minus seven to plus one. B: hospital admission, relative risk 0.55, interval 0.25 to 1.2. C: systolic pressure, mean difference minus five millimetres of mercury, interval minus nine to minus one. D: falls, odds ratio 1.4, interval 0.9 to 2.2. E: heart attack, relative risk 1.1, interval 0.7 to 1.7. Pause here and decide.
4:12The answer and the options
A is out: the interval for this difference includes zero. B is out: it excludes zero, but for a ratio the null value is one, and this interval includes one. D and E are out: both intervals include one. So the answer is C. The null value for a difference is zero, and minus nine to minus one excludes it.
4:37Recap
Three things to keep. One: a 95% interval is the estimate plus or minus about two standard errors, with ratios calculated on the log scale. Two: the null value is zero for differences and one for ratios. An interval that includes it is not significant at 0.05. Three: a wide interval that includes the null is inconclusive, not proof of no effect. That wraps up this series on biostatistics.
Sources
Numbers in the text, e.g. [1], point here- Sener A. Interpreting P values in clinical research: a practical guide for clinicians. Turk J Emerg Med. 2026;26(4):281-287. pmc.ncbi.nlm.nih.gov/articles/PMC13722342/
- Greenland S, Senn SJ, Rothman KJ, et al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 2016;31:337-350. pmc.ncbi.nlm.nih.gov/articles/PMC4877414/
- Hickey GL, Grant SW, Dunning J, Siepe M. Statistical primer: sample size and power calculations, why, when and how? Eur J Cardiothorac Surg. 2018;54(1):4-9. pmc.ncbi.nlm.nih.gov/articles/PMC6005113/
- Cochrane Handbook for Systematic Reviews of Interventions, chapter 6: choosing effect measures and computing estimates of effect. training.cochrane.org/handbook/current/chapter-06
- Banerjee A, Chitnis UB, Jadhav SL, Bhawalkar JS, Chaudhury S. Hypothesis testing, type I and type II errors. Ind Psychiatry J. 2009;18(2):127-131. pmc.ncbi.nlm.nih.gov/articles/PMC2996198/
- Wikipedia (background, checked against the sources above): Confidence interval; Relative risk. en.wikipedia.org/wiki/Confidence_interval · en.wikipedia.org/wiki/Relative_risk
The hook intervals and the 20-repeat stack of intervals are teaching schematics, not from real trials. The hook and teaching numbers differ from the practice question's numbers on purpose. The width bars have no numeric axis.
The practice question is original and its patient is synthetic. Every fact is checked against the sources above in independent accuracy reviews. Always follow your local guidelines and your own laboratory’s ranges.
Image credits. Illustrations: Servier Medical Art (smart.servier.com), CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/), adapted (cropped, recoloured): adult man, adult woman, blood pressure apparatus. Icon: magnifying glass by Marcel Tisch via bioicons, CC0. The number lines, interval bars, tables, question card, recap card and all handwriting are drawn in code (original). No AI-generated teaching images.
Corrections. None so far. Spotted an error? Email info@medservatory.com with the lesson code MS-S1-BIOS-008.
For exam study only, not medical advice. Read the medical disclaimer.
In this series
Playlist on YouTube- EP 01Sensitivity vs specificity: the 2×2 table, SnNout and SpPin5:34
- EP 02PPV, NPV and prevalence explained5:27
- EP 03Relative risk vs odds ratio: the 2x2 table explained5:15
- EP 04Number needed to treat: ARR, RRR and NNH from trial data5:00
- EP 05Study designs explained: case-control, cohort, cross-sectional and RCT5:22
- EP 06Types of bias: selection, recall, lead-time and confounding5:30
- EP 07Type I and type II errors: alpha, beta, power and the p-value5:16
- EP 08Confidence intervals and significance: is the null value inside?5:12