Ask Mallard

The bootstrap

The bootstrap draws many samples with replacement from the observed data, recomputes the statistic in each, and uses the spread of those values as the sampling distribution.

What question does this help answer?

How uncertain an estimate is, when the estimate is a complicated function of the data and no closed-form standard error exists or the usual one relies on assumptions that plainly do not hold.

When it earns its place

  • A ratio, a median, a difference in medians, or any statistic without a tidy variance formula.
  • A quantity derived through several steps, such as a predicted probability from a model built by a multi-stage procedure.
  • Internal validation of a prediction model, where the whole model-building procedure is repeated in each resample to estimate optimism honestly.
  • A mediation or standardization estimate assembled from several model outputs.
  • Small samples where the normal approximation is doubtful, though the bootstrap is not magic in very small samples either.

Resample what the design sampled

This is the rule that gets broken. If clinics were sampled and patients within them, resample clinics. If the design is matched, resample the matched sets. If the data are a time series, an ordinary bootstrap destroys the dependence that matters. The resampling has to imitate how the data came to exist.

What resampling looks like

The bootstrap resamples what the design sampled. With independent people, people are resampled with replacement. With clustered data, whole clusters are resampled; resampling individuals instead would give intervals that are far too narrow.Independent people: resample peopleOriginal sample of peopleResampleResampleResampleDistribution of the statisticClustered data: resample whole clustersOriginal sample of clustersResampleclustersResampleclustersResampleclustersDistribution of the statisticEvery resample is drawn with replacement and is the same size as the original. The bootstrap cannot repaira biased sample.
Conceptual diagram. It shows the structure of the idea, not data from any study.

Which interval

The percentile interval is the simplest and is adequate when the statistic's distribution is roughly symmetric and unbiased. The bias-corrected and accelerated interval adjusts for skew and bias and is generally preferable. Report which was used and how many resamples were drawn; a few hundred is enough for a standard error, but intervals need thousands.

Why Mallard may recommend it

Mallard may name a bootstrap for a derived quantity, for internal validation of a prediction model, or for a confidence interval on something the usual formulas do not cover.

What to watch out for

The bootstrap resamples the data you have, so it cannot repair a biased sample, and it inherits every dependency in the design: with clustered data, resampling individuals rather than clusters produces intervals that are far too narrow. It is also not one method but several, and the plain percentile interval is the weakest of them; where the statistic is skewed, a bias-corrected and accelerated interval behaves considerably better.

Clinical example

Question: What is the confidence interval around the difference in median cost between two pathways? Analysis: A bias-corrected and accelerated bootstrap over 2000 resamples, resampling patients within the hospitals they were sampled from.

Code for this method
Loading…
Recent studies that used this
Searching PubMed…
Related Learn pages
Sources & further reading
  1. Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians (opens in a new tab) · Stat Med · doi 10.1002/(sici)1097-0258(20000515)19:9<1141::aid-sim479>3.0.co;2-f
  2. Variable selection - A review and recommendations for the practicing statistician (opens in a new tab) · Biom J · doi 10.1002/bimj.201700067
Last source review: 2026-09-06
AI-generated educational content
Mallard uses reputable methodological and reporting sources, but this material may contain errors or omit important context. Verify important methodological decisions with the cited sources and, when appropriate, a qualified research methodologist or statistician.