Statistics · BEGINNER LESSON
Statistical significance
Statistical significance means a study result is unlikely to be seen if the null hypothesis is true, as measured mainly with a P value. It does not by itself tell you that the effect is large, important, or clinically meaningful.
What it is and what is measured
Statistical significance is a decision about how compatible the observed data are with a null hypothesis, which is the claim that there is no real effect or no difference. The most common measure is the P value, which estimates how surprising the observed result would be if the null hypothesis were true. A small P value means the data are less compatible with the null hypothesis, but it does not prove a real effect or show that the effect matters in practice.
[2] Statistically significant results from low-power analyses - PMC - NIH [3] The P Value: What It Is and What It Is Not - PMC
How it works
In hypothesis testing, a test statistic such as z, t, or chi-square is calculated from the sample. That statistic is compared with what would be expected under the null hypothesis. If the result crosses a chosen threshold, the null hypothesis is rejected and the result is called statistically significant. This threshold is a decision rule, not a direct measure of effect size. Sample size also matters: with more data, the same observed effect can produce a smaller P value.
[3] The P Value: What It Is and What It Is Not - PMC [2] Statistically significant results from low-power analyses - PMC - NIH
What statistical significance does not mean
Statistical significance is not the same as clinical significance, which asks whether an effect is meaningful for patients or practice. A small P value can come from a very small effect in a large sample. A non-significant result does not prove there is no effect; it may also reflect low statistical power, meaning the study had a limited chance to detect a real effect. Low-power analyses can exaggerate significant estimates, so effect size and confidence intervals should be read together with the P value.
[3] The P Value: What It Is and What It Is Not - PMC [2] Statistically significant results from low-power analyses - PMC - NIH
Limitations and careful use
The P value is often misunderstood. It is not the probability that the null hypothesis is true, and it is not the chance that the same result will repeat in another study. Statistical significance should be interpreted with the study design, sample size, effect size, confidence intervals, and the research question. Poor test choice or incorrect data structure can also lead to wrong conclusions even when the calculations are correct.
[3] The P Value: What It Is and What It Is Not - PMC [1] Clinical Significance
SEE THE IDEA
How statistical significance is judged
The sample gives a test statistic and a P value.
The starting assumption is that no real effect exists.
A chosen cutoff separates significant from not significant.
Read the result with effect size and confidence intervals.
Observed data are compared with the null hypothesis through a threshold-based test, then interpreted in context.
Statistical significance is a comparison between sample results and what would be expected if no real effect were present.
TRY IT YOURSELF
Test your understanding
Choose an answer, then check the explanation.
PUT IT TOGETHER
Match each concept to its meaning
Drag a label onto its description, or select a label and then select a description. This also works with a keyboard.
4 pairs to match
Sources and further reading
This general lesson was generated from public source material. It explains a concept; it does not interpret a private paper or provide medical advice.