Difference-in-differences
Why this level?
The design family and its defining features have not been specified for this hierarchy. NHMRC 2009 sourceNobody randomised the exposure, so the comparison leans on the parallel-trends assumption instead of on allocation. Where the comparison group is concurrent this is III-2; against a historical comparison it falls to III-3.
What question does this help answer?
“Did the outcome change more in the group exposed to a policy or program than it changed over the same period in a suitable comparison group?”
When to use it
DiD is useful for natural experiments, policy evaluations, and group-level interventions where randomized assignment is unavailable but treated and comparison groups can be observed before and after implementation.
When not to use it
Do not use a simple DiD merely because two groups and two time periods exist. The key causal requirement is that the comparison group provides a credible counterfactual for how the treated group would have changed without the intervention.
How it works in plain language
First estimate the pre-to-post change in the treated group. Then estimate the same change in the comparison group. The difference between those changes is the DiD estimate.
What the counterfactual looks like
Why Mallard may recommend it
Mallard may recommend DiD for hospital, clinic, state, employer, or policy changes that affect some groups but not comparable others.
Key assumptions and design requirements
- Parallel trends is plausible: absent treatment, groups would have had similar average outcome trends.
- Treatment timing and comparison-group construction are defensible.
- Anticipation and spillover are considered.
- Staggered adoption and treatment-effect heterogeneity are handled with modern methods when relevant.
- Standard errors reflect clustering and the assignment structure.
What to watch out for
A traditional two-way fixed-effects model can be biased under staggered treatment timing with heterogeneous effects. Pre-trend plots are useful but cannot prove the parallel-trends assumption.
Clinical example
Compare changes in screening rates at medical centers that launched a navigator program with changes at centers that had not yet launched it.
How to report it
No reporting guideline dedicated to difference-in-differences was found in a PubMed search on 2026-09-16. STROBE is the guideline to use, or TREND where the study evaluates a deliberate intervention. Neither asks for the parallel trends evidence, so present it yourself: the pre-period trends in both groups, and what else changed for one group and not the other. A reporting guideline is not a design manual: use it to make the study transparent, not as a substitute for careful design. Check the target journal's own requirements as well, which this page cannot know.
- A Tutorial on Applying the Difference-in-Differences Method to Health Data (opens in a new tab) · Current Epidemiology Reports · doi 10.1007/s40471-023-00327-x
- Difference-in-Differences for Health Policy and Practice: A Review of Modern Methods (opens in a new tab) · Stat Med · doi 10.1002/sim.70247
- Three Statistical Approaches for Assessment of Intervention Effects: A Primer for Practitioners (opens in a new tab) · Risk Manag Healthc Policy · doi 10.2147/RMHP.S275831
- Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies (opens in a new tab) · BMJ · doi 10.1136/bmj.39335.541782.AD
- Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement (opens in a new tab) · Am J Public Health · doi 10.2105/ajph.94.3.361