Choosing a before-and-after design
Choose a design by the comparison and time pattern you can observe, not by whether data come from an audit.
Fit
When it matters, and when it does not
It matters when
- You must evaluate a service change without random allocation.
- You can check which time points and comparison services exist before choosing an analysis.
It matters less when
- When random allocation is feasible, consider a randomized trial. Read: Randomized controlled trial
- For process monitoring without a causal claim, consider run charts. Read: Control charts for process monitoring
Before you start
What to check in your study
- Observation scheduleCount repeated observations before and after the change.
- Concurrent comparisonFind a service exposed to common shocks but not the intervention.
- Intervention timeDefine implementation, any gradual start and other changes at that time.
- The comparison describes what would happen without the change.How to check: Check prior trends, case mix, seasonality and concurrent policies.
- Repeated observations can be correlated.How to check: Model serial correlation and account for service-level clustering where relevant.
- Data capture remains comparable over time.How to check: Describe missing at random assumptions for modeling; test departures when recording may change.
Plain language
How it works
An uncontrolled design compares outcomes before and after in one group. It cannot separate intervention effects from other changes. A controlled design compares that change with a concurrent untreated group. Its causal interpretation needs credible parallel trends without the intervention.
Interrupted time series uses repeated outcomes to estimate a change in level, slope or both. It projects the earlier pattern forward as a no-intervention comparison. Wagner and colleagues describe segmented regression; Bernal and colleagues explain seasonality, serial correlation and other threats. A controlled series can help separate shared shocks. More observations cannot remove a simultaneous local policy change.
Worked example
A worked example
- Setting
- A hospital changes a discharge process.
- Question
- Did the policy reduce readmissions beyond the background trend?
- Approach
- Compare the treated hospital with a concurrent hospital. Then inspect the repeated monthly series.
- Result
- Readmissions fall from 20 to 12 in 100 at the treated hospital and from 18 to 14 in 100 at the control. The change contrast is -4 percentage points.
- What it means
- If the treated pretrend instead predicts 10 in 100, an observed 12 is 2 points above that projection. The causal answer depends on the credible counterfactual.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
A before-and-after change alone does not establish causation. A concurrent control needs a credible comparison trend; interrupted time series needs enough repeated observations and a defensible counterfactual trend.
- Calling two observations a time series.Instead: Collect enough repeated points to estimate the time pattern.
- Assuming a concurrent control removes all bias.Instead: Check pretrends and changes affecting only one group.
- Interpreting a slope change as an immediate effect.Instead: Report the time horizon for the effect.
Inside Mallard
How Mallard uses this
Mallard includes this comparison in Learn search and the Atlas. Plans do not link to this comparison automatically.
concept.choosing_before_after_designs, the id a plan cue records.Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- Segmented regression analysis of interrupted time series studies in medication use research (opens in a new tab) · J Clin Pharm Ther · doi 10.1046/j.1365-2710.2002.00430.x
- Interrupted time series regression for the evaluation of public health interventions: a tutorial (opens in a new tab) · Int J Epidemiol · doi 10.1093/ije/dyw098