Interim analyses and stopping rules
Why look at all
Because continuing to randomise patients to a treatment already shown to be harmful, or clearly beneficial, is not defensible, and because continuing a trial that cannot possibly answer its question wastes patients' participation. The ethical case for looking is strong. The statistical cost is real and must be paid for in advance.
How the error rate is protected
Group sequential designs set boundaries in advance that are harder to cross early and easier to cross late, spending the total false-positive allowance across the planned looks. O'Brien-Fleming boundaries are very stringent early, so the final analysis is barely affected. Pocock boundaries are constant, which makes early stopping more likely at the cost of a stricter final threshold. The alpha spending function generalises this and allows the number and timing of looks to change, provided the spending function itself was fixed in advance.
What stopping boundaries look like
What must be prespecified
- The number and approximate timing of interim analyses.
- The spending function or boundary shape, chosen before any data are seen.
- Whether there is a futility boundary as well, and whether it is binding.
- Who sees the unblinded data: an independent data monitoring committee, not the investigators.
- How the final estimate will be adjusted for the sequential design, which is the step most often omitted.
Stopping early overstates the effect
A trial crosses an efficacy boundary partly because the treatment works and partly because random variation happened to run in its favour at that moment. The published effect therefore carries that upward fluctuation. Empirically, trials stopped early report effects around a third larger than comparable trials that ran to completion, and the discrepancy is worst with few accrued events. Point estimates from a stopped trial should be adjusted, and read with that in mind even when they are.
Why Mallard may recommend it
Mallard may propose interim monitoring for a trial with a serious outcome or a long recruitment period, and will treat the boundary as part of the design rather than something to add later.
What to watch out for
Repeatedly testing accumulating data inflates the false-positive rate: with five unadjusted looks, the chance of crossing p below 0.05 at some point is well above 0.05 even when the treatment does nothing. Adjusted boundaries fix the error rate but not the estimate. Trials stopped early for benefit systematically OVERSTATE the effect, most severely when few events have accrued, and that bias is present whether or not a formal stopping rule was used.
Clinical example
Question: Should a mortality trial be able to stop early? Design: Two interim analyses with an O'Brien-Fleming spending function, reviewed by an independent monitoring committee, with the final estimate adjusted for the sequential design and reported as such.
- Interim analysis: the alpha spending function approach (opens in a new tab) · Stat Med · doi 10.1002/sim.4780131308
- Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis (opens in a new tab) · JAMA · doi 10.1001/jama.2010.310