Ask Mallard
Core concept

Choosing a design to evaluate a QI change

4 min read

A quality improvement study tests a change in how care is delivered, and reports what changed, in which setting, and how strongly the evidence links the change to the results.

Fit

When it matters, and when it does not

It matters when

  • A service has a measured gap in care, and a change to close it is ready to test.
  • You can measure the outcome, the process and any harm repeatedly, before and after the change.
  • You want to learn whether the change worked here, and why, while adapting it as you go.

It matters less when

Before you start

What to check in your study

Decide these first
  1. The aim, with a number and a dateSuch as cutting discharge medicine errors from 30 to 15 in 100 by June. A clear aim makes progress measurable.
  2. Outcome, process and balancing measuresoutcomeTypeThe outcome that matters, whether the change was delivered, and any harm it caused elsewhere. Define each one's numerator and denominator.
  3. The evaluation designassignmentrepeatedMeasuresA single before and after describes a change. Many measurements over time, or a comparison ward, can show more.
  4. What else is changingStaffing, case mix, coding or other projects. Record them with dates, because each can explain a change.
  5. A missing-data plan that starts from an assumptionDecide why audits or records may be missing in each period before choosing a method. The assumption picks the method.
It rests on these
  1. The expected timing and shape of the change are stated in advance.How to check: Date the change and each adaptation. A design chosen after seeing the run chart can find a pattern in noise.
  2. Nothing else changed at the same time.How to check: Look for other projects, new staff, a change in patients and regression to the mean after a bad month.
  3. The analysis allows for repeated measurements and for wards.How to check: Points close in time are alike, and patients on one ward are alike. Few wards give little precision, however many patients.
  4. Missing audits and records follow the assumption you named.How to check: Audits are skipped more often on busy days, so missing completely at random (MCAR) is rarely plausible. Missing at random (MAR) means the gaps depend only on recorded factors, such as ward and day. Under MAR, multiple imputation can help, but it is not automatically better than analyzing the audits done. If audits are skipped when care went worst, missing not at random (MNAR) is plausible. Then add a tipping-point analysis, which shifts the missing results until the conclusion changes.

Plain language

How it works

Name the care gap, the change and the outcome before choosing a design. Include process measures, such as checklist use, and balancing measures, such as discharge delays. Record other changes that could explain the result.

Choose by the claim and the available comparison

One measurement before and after describes change but leaves its cause uncertain. Repeated points with a clear intervention date support an interrupted time series. A concurrent comparison can help separate shared changes. Random allocation strengthens the comparison when feasible. Use the linked pages for each design and chart.

Three stories for the same change
Three panels with the same before and after values. The first shows the observed improvement. The second shows a trend that was already under way passing through both points. The third shows a group that started unusually high returning to its usual level.What was observedTimeBetter afterA trend already under wayTimeRegression to the meanTimeUsual levelStarted unusually highThe same before and after values fit all three stories. Without a concurrent comparison, repeated earliermeasurements, or both, the change cannot be attributed to the intervention.
Conceptual diagram. It shows the structure of the idea, not data from any study.

Worked example

A worked example

Invented for illustration
Setting
A medical ward introduces a pharmacist discharge checklist, with weekly audits of discharge medicine errors.
Question
Did errors fall after the checklist started?
Approach
Twenty weeks of audits before the change and twenty after are plotted, with checklist use and discharge delays measured too. A pharmacy staffing change in week 25 is recorded.
Result
Errors fell from 30 in 100 discharges before the checklist to 18 in 100 after, a fall of 12 per 100.
What it means
Errors fell, but the staffing change could explain part of the fall. A comparison ward, or a longer series, would show more about the cause.

In practice

Recent studies PubMed search · not vetted

Searching PubMed…

Watch out

Common mistakes

The one to remember

An improvement after implementation does not by itself show that the intervention caused it. Secular trends, regression to the mean, changing denominators and concurrent changes need an explicit evaluation strategy.

  • Calling a fall after the change proof that the change worked.Instead: Secular trends, regression to the mean and other changes can all produce a fall. Choose an evaluation that can rule them out.
  • Reporting only the process measure.Instead: A checklist can be completed every time while errors stay the same. Report outcome, process and balancing measures together.
  • Using the label quality improvement to skip an ethics or oversight decision.Instead: Ask your institution which review applies, especially if you will publish or compare groups.

Inside Mallard

How Mallard uses this

Plans do not link to this concept yet. If your plan types its data as clustered, such as by ward, and is sized as if patients were independent, Mallard withholds the headline sample size.

No Mallard code example points to this page.
Atlas node concept.choosing_qi_design, the id a plan cue records.
Part of the learning path Evaluate a change on your own ward (step 1 of 6).

Related Learn pages

Related designs and methods

Evidence

Sources & further reading

  1. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process (opens in a new tab) · BMJ Qual Saf · doi 10.1136/bmjqs-2015-004411
  2. Interrupted time series regression for the evaluation of public health interventions: a tutorial (opens in a new tab) · Int J Epidemiol · doi 10.1093/ije/dyw098
  3. How to study improvement interventions: a brief overview of possible study types (opens in a new tab) · BMJ Qual Saf · doi 10.1136/bmjqs-2014-003620
  4. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls (opens in a new tab) · BMJ · doi 10.1136/bmj.b2393
Prepared with AI assistance; publication follows agent methodological review under the owner’s expansion request. Each source record was rechecked by PMID, with its title, journal and DOI matched against PubMed, on 2026-11-06; a record PubMed holds without a DOI was matched by title and journal. No independent professional or methodological approval is claimed. SQUIRE is reporting guidance spanning several improvement methods. It does not certify causal validity or determine research-oversight requirements. Source records rechecked against PubMed on 2026-11-07: record verified, confirm it supports this use. No independent professional or methodological approval is claimed.
Last source review: 2026-11-07
AI-generated educational content
Mallard uses reputable methodological and reporting sources, but this material may contain errors or omit important context. Verify important methodological decisions with the cited sources and, when appropriate, a qualified research methodologist or statistician.