Ask Mallard
Study design

Uncontrolled before-and-after studies

3 min read

An uncontrolled before-and-after study compares an outcome in a period before a change with the same outcome after it, with no comparison group measured over the same time.

NHMRC EVIDENCE LEVEL · 2009Depends on how the study is designed · InterventionProvisional design classification. Study quality and certainty of findings require separate appraisal.
Why this level?The design family and its defining features have not been specified for this hierarchy. NHMRC 2009 source

A single series measured before and after is Level IV intervention evidence. A distinct historical comparison is III-3, while a controlled before-and-after design with concurrent controls is III-2. The broad pre/post label is insufficient to choose among them.

Where the NHMRC 2009 hierarchy names this design, it places it here:

Research questionLevelWhen
InterventionIII-2An observational comparison uses concurrent controls. Statistical adjustment does not change this design level.
InterventionIII-3The comparison uses historical controls or separate single-arm studies, without concurrent controls.
InterventionIVA single case series has post-test or pre-test/post-test outcomes, without a comparison group.

Fit

When to use it, and when not to

Use it when

  • The change is being made anyway, and the choice is this study or no evaluation at all.
  • You are auditing a change already made, and the data allow nothing stronger.
  • You will treat the finding as provisional, a sign that something changed rather than proof of why.

Not when

Before you start

What you need

Decide these first
  1. What changed, and exactly whenA clear start date separates the before period from the after period.
  2. The before and after periodsMake them the same length and cover the same seasons. Monthly values on each side tell you more than two averages.
  3. How the outcome is countedoutcomeTypeIt must be counted the same way in both periods. A new form or code can look like an effect.
  4. Why the change was made when it wasIf a bad result prompted it, some improvement would have come anyway.
  5. An outcome the change should not affectIf it moves too, something broader than the change was going on.
  6. A missing-data plan that starts from an assumptionDecide why outcome records may be missing before choosing a method. The assumption picks the method, not the share missing.
It rests on these
  1. Nothing else that affects the outcome changed between the two periods.How to check: This assumption is the whole design, so state it. List staffing, policy, coding and case-mix changes over the same time.
  2. The outcome was not already drifting up or down.How to check: Plot as many earlier points as the data allow. A trend already under way looks just like an effect.
  3. The before period was typical, not an unusually bad stretch.How to check: If the change followed a spike, expect a fall anyway. Look at a longer run of earlier data.
  4. The outcome is measured the same way before and after.How to check: Same definition, same data source and the same people collecting it.
  5. The patients are alike in both periods.How to check: Compare case mix across the periods. A shift in who is treated can look like an effect.
  6. Missing values follow the assumption you named.How to check: Records are often less complete in one period, for example before a new form. So missing completely at random (MCAR) is rarely plausible. Missing at random (MAR) means the gaps depend only on recorded factors, such as period, age and ward. Under MAR, multiple imputation can use those factors, but it is not automatically better than analyzing complete records. If bad outcomes go unrecorded more often, even given recorded factors, missing not at random (MNAR) is plausible. Then add a tipping-point analysis, which shifts the imputed values until the conclusion changes.

Plain language

How it works

The outcome is measured during a period before a change and again during a period after it, and the two are compared. No group goes without the change over the same time. So anything else that shifted between the periods looks the same as an effect of the change.

Three stories for the same change
Three panels with the same before and after values. The first shows the observed improvement. The second shows a trend that was already under way passing through both points. The third shows a group that started unusually high returning to its usual level.What was observedTimeBetter afterA trend already under wayTimeRegression to the meanTimeUsual levelStarted unusually highThe same before and after values fit all three stories. Without a concurrent comparison, repeated earliermeasurements, or both, the change cannot be attributed to the intervention.
Conceptual diagram. It shows the structure of the idea, not data from any study.
How to strengthen it

Three additions do most of the work. More time points before and after turn it into an interrupted time series, which can tell a sudden step from a trend already under way. A comparison site turns it into a controlled before-and-after study, analyzed as difference-in-differences. An outcome the change should not affect shows whether something broader was changing.

Worked example

A worked example

Invented for illustration
Setting
One medical ward starts a discharge checklist after a year of unusually high readmissions.
Question
Did the checklist reduce readmissions within 30 days?
Approach
Readmissions in the 12 months before the checklist are compared with the 12 months after it.
Result
Readmissions fell from 15 to 12 in every 100 discharges, a drop of 3 in 100.
What it means
The checklist may have helped, but a fall was likely anyway after an unusually bad year. Monthly rates from earlier years would show whether that year was unusual or readmissions were already falling. A ward without the checklist would show whether readmissions fell there too. Neither can rule out another change on this ward at the same time, so the drop is consistent with an effect of the checklist, not proof of one.

In practice

Recent studies PubMed search · not vetted

Searching PubMed…

Watch out

Common mistakes

The one to remember

Without a concurrent comparison group, any secular trend, seasonal pattern, regression to the mean, or concurrent change is indistinguishable from the intervention's effect. It is the weakest of the common evaluation designs and should be strengthened wherever possible with multiple pre-intervention time points, a control site, or both.

  • Reporting the drop as proof that the change worked.Instead: Say the result is consistent with an effect, and name what else could explain it.
  • Changing how the outcome is recorded when the change starts.Instead: Keep the definition, data source and collection the same in both periods.
  • Reading a small P value as evidence that the change caused the drop.Instead: A P value can say the drop is unlikely to be chance alone. It cannot rule out a trend or another change.

Inside Mallard

How Mallard uses this

When a design option in Mallard's brief is described as a before-and-after study, the option links to this design.

No Mallard code example points to this page.
Atlas node design.before_after, the id a plan cue records.

How to report it

No reporting guideline dedicated to the uncontrolled before-and-after study was found in a PubMed search on 2026-09-16. TREND is the closest fit, written for nonrandomized evaluations of an intervention rather than for this design in particular. Its items on the intervention, the comparison and the threats to validity apply; what it cannot supply is a concurrent control, so state what else changed over the same period. A reporting guideline is not a design manual: use it to make the study transparent, not as a substitute for careful design. Check the target journal's own requirements as well, which this page cannot know.

Part of the learning path Evaluate a change on your own ward (step 2 of 6).

Related Learn pages

Related designs and methods

Evidence

Sources & further reading

  1. Segmented Regression in an Interrupted Time Series Study Design (opens in a new tab) · Anesth Analg · doi 10.1213/ANE.0000000000005269
  2. Overview of clinical study designs (opens in a new tab) · Clin Exp Emerg Med · doi 10.15441/ceem.23.036
  3. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls (opens in a new tab) · BMJ · doi 10.1136/bmj.b2393
  4. Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement (opens in a new tab) · Am J Public Health · doi 10.2105/ajph.94.3.361
The first reference covers segmented regression for interrupted time series, which is the natural strengthening of this design and the method to reach for whenever more than one pre-intervention time point exists.
Last source review: 2026-11-05
AI-generated educational content
Mallard uses reputable methodological and reporting sources, but this material may contain errors or omit important context. Verify important methodological decisions with the cited sources and, when appropriate, a qualified research methodologist or statistician.