Statistics · BEGINNER LESSON
P value
A P value is a conditional probability: assuming the null hypothesis is true, it is the chance of getting data at least as extreme as the data observed. It is a measure of evidence against the null hypothesis, not the probability that the null hypothesis is true.
What a P value is
A P value is a number used in hypothesis testing, a statistical method for judging whether sample data are unusual under a starting assumption called the null hypothesis, often meaning “no effect” or “no difference.” The P value answers one specific question: if the null hypothesis were true, how likely would we be to see data this extreme or more extreme by chance alone? Smaller P values mean the observed data are less compatible with the null hypothesis, so they provide stronger evidence against it. [0,1,2]
[1] The P Value: What It Is and What It Is Not - PMC [2] P Value and the Theory of Hypothesis Testing - PMC - NIH [3] The P Value: What Is It and What Does It Tell You? - PMC - NIH
How it is used
Researchers compare the P value with a chosen significance level, often 0.05. If the P value is below that cutoff, they may reject the null hypothesis in favor of an alternative hypothesis, which says there is some effect or difference. This is a decision rule, not a direct statement that the hypothesis is true or false. The Neyman-Pearson framework also uses Type I error, meaning false rejection of a true null hypothesis, and Type II error, meaning failing to reject a false null hypothesis. [0,1]
[1] The P Value: What It Is and What It Is Not - PMC [2] P Value and the Theory of Hypothesis Testing - PMC - NIH
What it does not mean
A P value is not the probability that the null hypothesis is true, not the probability that the alternative hypothesis is true, and not the probability that a similar study will give the same result. A small P value also does not tell you how big or important an effect is. Statistical significance means the observed result is unlikely under the null hypothesis; clinical significance asks whether the size of the effect matters in real practice. [0,1,2]
[1] The P Value: What It Is and What It Is Not - PMC [2] P Value and the Theory of Hypothesis Testing - PMC - NIH [3] The P Value: What Is It and What Does It Tell You? - PMC - NIH
Mechanism and limits
P values are shaped by both the observed effect and the sample size. A small study can miss a real effect and produce a non-significant P value, while a very large study can make a tiny effect look statistically significant. Because of this, P values should be read with effect sizes, confidence intervals (CIs), and study context. They also do not establish causation by themselves, and replication is important before accepting a claim. [0,1,2]
[1] The P Value: What It Is and What It Is Not - PMC [2] P Value and the Theory of Hypothesis Testing - PMC - NIH [3] The P Value: What Is It and What Does It Tell You? - PMC - NIH
SEE THE IDEA
How to read a P value
Start with the assumption of no effect or no difference.
Measure the result from the sample or experiment.
Calculate how extreme the data would be if the null hypothesis were true.
Compare with a cutoff, then also check effect size, CIs, and context.
Assumption -> observed data -> probability under that assumption -> statistical decision plus interpretation
A P value is one piece of evidence in a broader analysis; it is not the whole conclusion.
TRY IT YOURSELF
Test your understanding
Choose an answer, then check the explanation.
PUT IT TOGETHER
Match each concept to its meaning
Drag a label onto its description, or select a label and then select a description. This also works with a keyboard.
4 pairs to match
Sources and further reading
This general lesson was generated from public source material. It explains a concept; it does not interpret a private paper or provide medical advice.