Process evaluation and fidelity
Process evaluation studies what was delivered, how people responded and how context shaped an intervention. Fidelity asks whether delivery preserved its intended core functions.
Why this level?
not one of the five research areas this hierarchy covers. NHMRC 2009 sourceProcess evaluation can combine quantitative and qualitative work to answer delivery and mechanism questions. Its methods and claims need appraisal; a treatment-effect hierarchy alone does not describe them.
Fit
When to use it, and when not to
Use it when
- You need to understand delivery, mechanisms or context within a trial or service change.
- The intervention has several parts, varies between sites, or needs adaptation during implementation.
Not when
- You only need an unbiased treatment effect estimate. Use an appropriate comparative design; process measures can accompany it. Read: Randomized controlled trial
- Your main question is whether an implementation strategy improves uptake. Use an implementation study with process evaluation alongside. Read: Hybrid effectiveness-implementation trial
Before you start
What you need
- The intervention theory and core functionsDescribe how activities should produce change. Separate essential functions from forms that can be adapted.
- Reach, dose and fidelity definitionsReach is who participates. Dose is how much is delivered or received. Define the denominator and quality criteria for each measure.
- People, sites and times to sampleInclude variation in delivery and experience. Seek nonparticipants and less successful sites, not only enthusiastic staff.
- How observations and interviews meet outcome dataPlan integration before outcomes are known. Explain when process findings will be examined relative to trial results.
- Process measures represent delivery and receipt.How to check: Combine logs, observation and interviews when useful. Staff checklists may measure completion rather than quality or participant understanding.
- High fidelity and better outcomes need not have a causal link.How to check: Motivation, staffing and case mix may affect both. A post-allocation fidelity comparison does not preserve randomization.
- Observations may share staff, sites or participants.How to check: Account for clustering in quantitative inference or state a justification. Describe how sampling and context shape qualitative findings.
- Gaps in process data may track delivery failure.How to check: Missing completely at random (MCAR) means gaps are unrelated to recorded and unrecorded values; this is rarely plausible. Missing at random (MAR) means gaps depend only on recorded factors. Under MAR, consider multiple imputation; it is not automatically better than a justified complete-case analysis. Missing not at random (MNAR) is plausible if unobserved values themselves influence whether records are absent. Under MNAR, test departures with a delta or tipping-point analysis that shifts the unobserved values.
Plain language
How it works
Start with a program theory: the activities, mechanisms and context expected to produce change. Observe delivery and receipt, record reach and dose, and ask people about their experiences. Fidelity can concern core content, delivery quality and participant understanding. A local adaptation may preserve a function while changing its form.
Mix quantitative process measures with qualitative accounts when the question requires both. Look for differences across sites, times and groups, including those not reached. Integrate findings with outcome results, keeping exploratory explanations labeled. Comparing outcomes by achieved fidelity can introduce confounding because fidelity occurs after allocation. A process evaluation complements an effect evaluation rather than proving its result.
Worked example
A worked example
- Setting
- An intervention has delivery records for 100 planned sessions across several clinics.
- Question
- How often were the prespecified core activities delivered?
- Approach
- Use observation to assess core activities and interviews to explain gaps. Keep this descriptive delivery measure separate from the trial effect.
- Result
- Core activities were delivered in 75 of 100 planned sessions: 75 in 100.
- What it means
- The 75 in 100 figure describes session fidelity. It does not mean that patients benefited or that the remaining sessions were equivalent.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
Fidelity describes delivery relative to the intervention intended; it is not proof of effectiveness. Associations between fidelity and outcomes can be confounded. Distinguish planned adaptation from loss of a core function.
- Calling attendance proof that an intervention was delivered as intended.Instead: Assess the core content, quality, receipt and enactment, as relevant to the intervention theory.
- Labeling every adaptation a fidelity failure.Instead: Ask whether the adaptation preserved the core function. Record what changed, why and in which context.
- Explaining a trial result only after selecting favorable process measures.Instead: Prespecify the theory and measures. Label later explanations as exploratory and examine contradictory accounts.
Inside Mallard
How Mallard uses this
Plans do not link to this design yet. Mallard has no dedicated sizing form for process evaluation or fidelity.
design.process_evaluation, the id a plan cue records.How to report it
Use the Medical Research Council guidance to report the program theory, implementation, mechanisms and context. State the core functions, adaptations, reach, dose, fidelity measures, sampling and how process findings were integrated with outcomes. Explain when analysts saw outcome results, and distinguish planned explanations from later hypotheses. EQUATOR can help locate further checklists for the quantitative and qualitative strands. The framework does not turn process associations into causal effects. A reporting guideline is not a design manual: use it to make the study transparent, not as a substitute for careful design. Check the target journal's own requirements as well, which this page cannot know.
Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- Process evaluation of complex interventions: Medical Research Council guidance (opens in a new tab) · BMJ · doi 10.1136/bmj.h1258
- Dancing for Parkinson's Disease Online: Clinical Trial Process Evaluation (opens in a new tab) · Healthcare (Basel) · doi 10.3390/healthcare11040604
- Breech specialist midwives and clinics in the OptiBreech Trial feasibility study: An implementation process evaluation (opens in a new tab) · Birth · doi 10.1111/birt.12685
- Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls (opens in a new tab) · BMJ · doi 10.1136/bmj.b2393