Exact binomial test
An exact binomial test compares one observed binary count with a fixed benchmark using binomial probabilities.
Fit
When to use it, and when not to
Use it when
- You have one success count out of a known number of independent trials.
- You need a test against a fixed proportion, including when a normal approximation is doubtful.
Not when
- Two sampled groups provide the risks. Use a two-group categorical test. Read: Chi-square test of independence
- Outcomes are paired within people. Use McNemar’s test. Read: McNemar’s test
- People share clinics or survey weights. Use methods that reflect the sampling design. Read: Complex survey analysis
Before you start
What you need
- Success count and totalDefine success and count every eligible observed trial once.
- Fixed null proportionJustify the benchmark before examining outcomes.
- Direction and false-positive thresholdPrespecify a one-sided or two-sided alternative and the two-sided probability rule.
- Interval methodUse a Wilson or Clopper-Pearson interval for the single proportion. Specify confidence level and method.
- Trials are independent, with the same success probability.How to check: Check sampling, repeated records and shared clusters.
- The total trials and benchmark suit the binomial question.How to check: A sampled benchmark or outcome-dependent stopping needs further analysis.
Plain language
How it works
Under the benchmark, the success count follows a binomial distribution. The test sums probabilities for counts at least as extreme under the chosen rule. A common two-sided rule sums probabilities no larger than that of the observed count. Discrete counts can make the test conservative. Report the single proportion with a Wilson or Clopper-Pearson interval.
The diagram shows interval precision, not the test’s rejection rule. Mallard’s benchmark sizing uses a normal approximation; verify exact-test power separately. Choosing a sample-size method helps you match the planning question to the right inputs.
Worked example
A worked example
- Setting
- A service observes 15 responses among 20 independent patients. Its fixed benchmark is 50%, or 50 in 100.
- Question
- Is the population response proportion different from the benchmark?
- Approach
- Prespecify a two-sided exact test. Describe the observed proportion with a 95% Wilson confidence interval.
- Result
- The observed proportion is 15 of 20, or 75 in 100. The 95% Wilson confidence interval is 53.1% to 88.8%.
- What it means
- The interval describes one proportion. A Wilson interval is not the inversion of the exact test; run the prespecified test separately.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
Exact describes the null calculation under the binomial model. It does not repair dependence, selection bias or uncertainty in a historical benchmark. Specify the tail rule in advance.
- Treating an exact p-value as proof of a useful effect.Instead: Report the observed proportion, its interval and clinical relevance.
- Choosing the favorable tail after seeing the count.Instead: Fix the alternative and stopping rule before observing outcomes.
Inside Mallard
How Mallard uses this
The method registry links its single-proportion benchmark test to this article. Mallard does not calculate an exact-binomial p-value. Its oneProportion sizing method returns a normal-approximation total and power; these are not exact-binomial results.
analysis.exact_binomial_test, the id a plan cue records.Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- Two-sided confidence intervals for the single proportion: comparison of seven methods (opens in a new tab) · Stat Med · doi 10.1002/(sici)1097-0258(19980430)17:8<857::aid-sim777>3.0.co;2-e
- R stats documentation: binom.test: Exact Binomial Test (opens in a new tab) · R documentation