False discovery rate
Benjamini-Hochberg controls the expected fraction of false findings among rejected hypotheses, under its dependence conditions.
Fit
When to use it, and when not to
Use it when
- You screen a defined family for discoveries that will need follow-up.
- You accept false-discovery rate control rather than protection against any false rejection.
Not when
- Any false rejection would undermine a confirmatory claim. Use family-wise error control such as Holm. Read: Holm adjustment
- Dependence is arbitrary or unknown. Obtain a justified dependence-robust procedure, or use Holm when family-wise control fits. Read: Holm adjustment
- You repeatedly inspect trial results. Use a planned sequential monitoring method. Read: Interim analyses and stopping rules
Before you start
What you need
- All p-values in the defined familyInclude nonsignificant tests and keep labels. Do not select the family using favorable results.
- The target false-discovery rateChoose the tolerated expected fraction before examining results. State why this target fits the scientific claims.
- Each true null has a valid p-value.How to check: Check models, sampling and missingness before correction. Adjustment does not remove confounding or biased tests.
- Tests are independent or meet positive regression dependence on true nulls (PRDS).How to check: PRDS is a specific joint-distribution condition. Positive pairwise correlations alone do not establish it; seek statistical advice if uncertain.
- The complete tested family is retained.How to check: Document exclusions and any filtering. Filtering related to null p-values can break the guarantee.
Plain language
How it works
The false discovery proportion is the fraction of rejected null hypotheses that are actually true; define it as zero when none are rejected. False discovery rate is its expectation across repeated studies. It is not the expected number of false rejections divided by expected rejections.
Sort m p-values. At each rank i, compare with i times the target divided by m. Find the largest passing rank; reject everything through it. No passing rank means no rejections. Adjusted p-values use reverse cumulative minima of m times each p-value divided by rank, capped at one.
A target of 0.05 means at most five in 100 discoveries on average under the assumptions. It does not bound the realized fraction in each study.
Worked example
A worked example
- Setting
- An invented exploratory screen has five tests with independent, valid p-values.
- Question
- Which null hypotheses can be rejected at target false-discovery rate 0.05?
- Approach
- Sorted p-values are 0.008, 0.012, 0.031, 0.032 and 0.200. Compare each with its rank times 0.05/5.
- Result
- The largest passing rank is 4: 0.032 is at most 0.040. Reject the first four, including rank 3 despite its failed individual comparison.
- What it means
- BH-adjusted p-values are 0.030, 0.030, 0.040, 0.040 and 0.200. The target is an expected fraction, not this study’s actual false-discovery fraction.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
Benjamini-Hochberg controls an expected fraction across repeated studies, not the probability that a particular finding is false. Its usual guarantee requires independent tests or the specified positive dependence condition, not arbitrary correlation. It does not control the chance of any false discovery.
- Stopping at the first failed rank comparison.Instead: Find the largest passing rank, then reject every hypothesis through that rank. This is step-up.
- Calling an adjusted p-value the probability a finding is false.Instead: The guarantee concerns the expected false-discovery fraction across repeated studies, not an individual hypothesis.
- Assuming every correlated dataset meets the guarantee.Instead: Justify independence or PRDS. If neither holds, choose a procedure with appropriate dependence protection.
Inside Mallard
How Mallard uses this
Mallard includes this method in Learn search and the Atlas. It has no automatic plan cue, dedicated calculator or execution route for this method.
analysis.false_discovery_rate, the id a plan cue records.How to report it
Report the full family, target rate, dependence justification, raw and BH-adjusted p-values, and rejected hypotheses. Report effect sizes and clearly labeled intervals; the adjustment does not turn ordinary intervals into simultaneous intervals. A reporting guideline is not a design manual: use it to make the study transparent, not as a substitute for careful design. Check the target journal's own requirements as well, which this page cannot know.
Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing (opens in a new tab) · Journal of the Royal Statistical Society Series B 57:289–300
- The control of the false discovery rate in multiple testing under dependency (opens in a new tab) · Annals of Statistics 29:1165–1188
- A practical guide to methods controlling false discoveries in computational biology. (opens in a new tab) · Genome Biol · doi 10.1186/s13059-019-1716-1
- False discovery rate control is a recommended alternative to Bonferroni-type adjustments in health studies. (opens in a new tab) · J Clin Epidemiol · doi 10.1016/j.jclinepi.2014.03.012