Ask Mallard
Analysis method

Exact binomial test

4 min read

An exact binomial test compares one observed binary count with a fixed benchmark using binomial probabilities.

Fit

When to use it, and when not to

Use it when

  • You have one success count out of a known number of independent trials.
  • You need a test against a fixed proportion, including when a normal approximation is doubtful.

Not when

Before you start

What you need

Decide these first
  1. Success count and totalDefine success and count every eligible observed trial once.
  2. Fixed null proportionJustify the benchmark before examining outcomes.
  3. Direction and false-positive thresholdPrespecify a one-sided or two-sided alternative and the two-sided probability rule.
  4. Interval methodUse a Wilson or Clopper-Pearson interval for the single proportion. Specify confidence level and method.
It rests on these
  1. Trials are independent, with the same success probability.How to check: Check sampling, repeated records and shared clusters.
  2. The total trials and benchmark suit the binomial question.How to check: A sampled benchmark or outcome-dependent stopping needs further analysis.

Plain language

How it works

Under the benchmark, the success count follows a binomial distribution. The test sums probabilities for counts at least as extreme under the chosen rule. A common two-sided rule sums probabilities no larger than that of the observed count. Discrete counts can make the test conservative. Report the single proportion with a Wilson or Clopper-Pearson interval.

Planning and interval limits

The diagram shows interval precision, not the test’s rejection rule. Mallard’s benchmark sizing uses a normal approximation; verify exact-test power separately. Choosing a sample-size method helps you match the planning question to the right inputs.

Interval precision
The half-width of a 95 percent Wilson interval for a proportion near 50 percent shrinks as the sample grows, roughly halving each time the sample size quadruples: about 13.4 percentage points at 50, 9.6 at 100, 6.9 at 200, 4.9 at 400 and 3.5 at 800.Precision for a proportion near 50%: interval half-width against sample size25501002004008000 pts5 pts10 pts15 pts20 ptsSample size (log scale)Half-width of the 95% interval13.4 pts9.6 pts6.9 pts4.9 pts3.5 ptsWilson interval. The app's precision calculator uses the simpler Wald width: a little wider for smallsamples, 13.9 points against 13.4 at 50, and the same to one decimal from 200 on. Clustering, nonresponseand a proportion far from 50% all change the width; a planned width is a target under stated assumptions.
Worked arithmetic. The inputs are round illustrative numbers, not results from any study; the outputs are calculated from them.

Worked example

A worked example

Invented for illustration
Setting
A service observes 15 responses among 20 independent patients. Its fixed benchmark is 50%, or 50 in 100.
Question
Is the population response proportion different from the benchmark?
Approach
Prespecify a two-sided exact test. Describe the observed proportion with a 95% Wilson confidence interval.
Result
The observed proportion is 15 of 20, or 75 in 100. The 95% Wilson confidence interval is 53.1% to 88.8%.
What it means
The interval describes one proportion. A Wilson interval is not the inversion of the exact test; run the prespecified test separately.

In practice

Recent studies PubMed search · not vetted

Searching PubMed…

Watch out

Common mistakes

The one to remember

Exact describes the null calculation under the binomial model. It does not repair dependence, selection bias or uncertainty in a historical benchmark. Specify the tail rule in advance.

  • Treating an exact p-value as proof of a useful effect.Instead: Report the observed proportion, its interval and clinical relevance.
  • Choosing the favorable tail after seeing the count.Instead: Fix the alternative and stopping rule before observing outcomes.

Inside Mallard

How Mallard uses this

The method registry links its single-proportion benchmark test to this article. Mallard does not calculate an exact-binomial p-value. Its oneProportion sizing method returns a normal-approximation total and power; these are not exact-binomial results.

No Mallard code example points to this page.
Sample size: Mallard sizes the main comparison with one proportion. It does not size adjustment or any secondary analysis.
Atlas node analysis.exact_binomial_test, the id a plan cue records.

Related Learn pages

Related designs and methods

Evidence

Sources & further reading

  1. Two-sided confidence intervals for the single proportion: comparison of seven methods (opens in a new tab) · Stat Med · doi 10.1002/(sici)1097-0258(19980430)17:8<857::aid-sim777>3.0.co;2-e
  2. R stats documentation: binom.test: Exact Binomial Test (opens in a new tab) · R documentation
Prepared with AI assistance. PubMed titles, authors, years and DOIs checked on 2026-11-07. Record verified, confirm it supports this use. No independent professional approval is claimed. Newcombe supports the choice of a single-proportion interval. R documentation describes the probability-ordered two-sided exact test and its Clopper-Pearson interval. The software documentation is not in PubMed; its URL was not verified here.
Last source review: 2026-11-07
AI-generated educational content
Mallard uses reputable methodological and reporting sources, but this material may contain errors or omit important context. Verify important methodological decisions with the cited sources and, when appropriate, a qualified research methodologist or statistician.