statistical error control · BEGINNER LESSON
False discovery rate
False discovery rate (FDR) is the expected proportion of false positive findings among the findings declared significant. It is used when many hypothesis tests are run, so the main goal is to limit how many declared discoveries are actually wrong.
What it is
False discovery rate, or FDR, is a statistical rule for many hypothesis tests. A hypothesis test asks whether an observed result is likely to be real or could happen by chance; a false positive, also called a type I error, is when a test says “significant” even though the null hypothesis is actually true. FDR focuses on the proportion of these false positives among all results called discoveries, rather than trying to eliminate every false positive.
[1] The application conditions of false discovery rate control - PMC - NIH [2] Control procedures and estimators of the false discovery rate ... - NIH [3] Selective inference in complex research - PMC
Why it is used
FDR is meant for multiple testing, meaning the same question is checked many times, such as across many genes, markers, or comparison groups. When many tests are run, using the same p-value threshold for each one can produce too many false positives. FDR control gives a way to adjust p-values or decision rules so the overall set of declared discoveries stays within an acceptable error range.
[1] The application conditions of false discovery rate control - PMC - NIH [2] Control procedures and estimators of the false discovery rate ... - NIH [3] Selective inference in complex research - PMC
How it works
The idea is to rank or evaluate many p-values together and choose a cutoff that keeps the expected fraction of false discoveries low. One common procedure is the Benjamini-Hochberg method, which orders p-values and compares them with a moving threshold based on the number of tests. Some related methods, such as q-value approaches, estimate the error level for each p-value. These methods are designed for large studies where some true discoveries may exist and it is useful to find more of them without letting false positives dominate.
[2] Control procedures and estimators of the false discovery rate ... - NIH [3] Selective inference in complex research - PMC
Limits and when it does not fit
FDR is not the right tool when there is only a single hypothesis test, because multiplicity is absent. It is also only an error-control method, not proof that any one discovery is true. Its behavior depends on the testing setup, including whether p-values are independent or positively dependent, and some estimators need enough data to work reliably. In small studies, or when the goal is to avoid even one false positive, family-wise error rate methods may be preferred.
[1] The application conditions of false discovery rate control - PMC - NIH [2] Control procedures and estimators of the false discovery rate ... - NIH [3] Selective inference in complex research - PMC
SEE THE IDEA
How false discovery rate control is used in many tests
Run separate tests across genes, outcomes, or groups.
Each test produces a number that helps judge evidence against the null hypothesis.
A combined method ranks or adjusts the p-values to limit the expected share of false discoveries.
Results passing the rule are reported as significant.
Many tests produce p-values; the FDR rule is applied to those p-values; the output is a set of declared discoveries with limited expected false positives.
FDR controls the mix of wrong and right calls among the results that are reported as discoveries, especially in multiple-testing settings.
TRY IT YOURSELF
Test your understanding
Choose an answer, then check the explanation.
PUT IT TOGETHER
Match each concept to its meaning
Drag a label onto its description, or select a label and then select a description. This also works with a keyboard.
4 pairs to match
Sources and further reading
This general lesson was generated from public source material. It explains a concept; it does not interpret a private paper or provide medical advice.