Planning when the sample size is already fixed
What question does this help answer?
With the data already available, how precisely can the main quantity be estimated, which effects could plausibly be detected, and is the dataset large enough for the planned model at all?
When to use it
Use this approach for secondary analyses of existing trials, registries, electronic health records and biobanks, for chart reviews limited to a fixed period, and for any study where recruitment cannot be extended.
When not to use it
Do not treat a fixed dataset as a reason to skip planning. If a prospective study is still possible, a sample size justified before data collection remains the stronger plan. And do not calculate power after the analysis to explain a non-significant result.
What to plan instead
- Precision: state the expected width of the confidence interval for the primary estimate under plausible assumptions, and whether that width would inform the decision the study supports.
- Detectable effect: before analysis, state the smallest effect the available sample could detect with a stated power and significance level, and compare it with the smallest clinically important difference.
- Model capacity: for multivariable and prediction models, check whether the number of outcome events supports the number of parameters planned. Events-per-variable rules are a minimum floor, and developing a prediction model has its own sample-size criteria based on limiting overfitting and estimating risk precisely.
- Subgroups and interactions: state which the dataset can support, because an interaction needs far more data than a main effect.
- Usable sample: estimate how many records meet eligibility and contain the key variables, since exclusions and missing data shrink the fixed number further.
Why post hoc power is not the answer
Power calculated from the effect observed in the same data is a restatement of the p-value and adds no information. A non-significant result from a small dataset is interpreted through its confidence interval, which shows which effects remain compatible with the data.
If the dataset is not enough
Say so before analysis. Options include changing the primary question to one the data can answer, presenting the analysis as descriptive or exploratory, combining datasets whose populations are genuinely compatible, or reducing the number of model parameters in a prespecified way.
Clinical example
A registry holds about 400 patients with a rare cancer, of whom roughly 60 died within two years, and the team would like to model mortality with twelve candidate predictors. Before analysis they record that the number of deaths limits how many parameters can be estimated reliably, prespecify a smaller model, and report the expected precision of the main estimate instead of a power figure. The numbers are illustrative and no calculation is shown.
What to watch out for
When the dataset already exists, its size is a constraint to report, not a calculation to reverse-engineer. Power computed from the observed effect adds nothing beyond the confidence interval, and a statement about detectable effects is only honest if it was compared with the clinically important difference before the analysis rather than after.
Before using this in your workspace
Record the fixed sample and number of events available after eligibility and missing data, the precision or detectable effect stated before analysis with its assumptions, the clinically important difference used for comparison, the number of model parameters, and any change to the question that the data required.
- The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results (opens in a new tab) · Ann Intern Med · doi 10.7326/0003-4819-121-3-199408010-00008
- Calculating the sample size required for developing a clinical prediction model (opens in a new tab) · BMJ · doi 10.1136/bmj.m441
- Planning Study Size Based on Precision Rather Than Power (opens in a new tab) · Epidemiology · doi 10.1097/EDE.0000000000000876