Simulation-based power and sample-size planning
When to use it
Use simulation when the exact design, correlation structure, missingness, model, or success rule cannot be represented faithfully by a validated standard calculation. Cluster trials, stepped-wedge designs, interrupted time series, mediation, and recurrent-event analyses often need study-specific simulation.
When not to use it
Do not run a generic simulator merely because the plan says ‘simulation’. If a validated analytic calculation fits the estimand and design, use that as the primary calculation and use simulation as a check when useful. Code that has not been given the actual assumptions has not estimated this study's power.
Build the scenario before running code
- State the primary estimand, clinically meaningful effect, success rule, significance level and target power.
- Generate the recruitment, allocation, timing, cluster sizes and within-unit dependence the planned study will actually have.
- Specify baseline outcome rates or variability, nuisance parameters, missingness and any departures from ideal model assumptions using sourced plausible ranges.
- Fit the prespecified analysis to each generated dataset, including the same covariates, contrasts and handling of missing outcomes intended for the real study.
- Evaluate several feasible sample sizes or design configurations, not just one favorable scenario.
What the result means
Estimated power is the fraction of attempted simulated studies meeting the prespecified success rule under a particular scenario. It is conditional on the data-generating assumptions; it is not a guarantee about the future study. Report power across plausible assumptions and select a feasible design that meets the target in the scenarios the team considers credible.
Make the run reproducible
Record the random seed, software and version, scenario inputs, attempted replicates, successful fits, failed or nonconvergent fits, the rule for handling failures, estimated power and Monte Carlo standard error. Choose enough replicates that simulation noise is small compared with the design decision. A large failure rate is a design warning, not a reason to discard those runs silently.
How Mallard's code fits
The public R and Python power-plan guidance is a scaffold for specifying and reporting a bespoke simulation, not an executed calculation or a recommended enrollment number. Replace its illustrative inputs, implement and validate the full data-generating and analysis models, then have the scenario choices reviewed before enrollment.
Clinical example
A stepped-wedge study has a fixed set of clinics and rollout periods. The team simulates outcomes with plausible between-clinic and within-clinic correlation, fits the planned time-adjusted model, and compares attainable power across cluster-period recruitment and effect assumptions. The number of clinics is a constraint, not an individual-level sample size invented from an unclustered formula.
What to watch out for
A generic simulation template is not a completed power calculation. Its assumptions must reproduce the intended design and analysis, and failed fits and Monte Carlo uncertainty must be reported rather than hidden.
- Simulation methods to estimate design power: an overview for applied research (opens in a new tab) · BMC Med Res Methodol · doi 10.1186/1471-2288-11-94
- Using simulation studies to evaluate statistical methods (opens in a new tab) · Stat Med · doi 10.1002/sim.8086
- The design of simulation studies in medical statistics (opens in a new tab) · Stat Med · doi 10.1002/sim.2673