Sensitivity analyses
What question does this help answer?
“Would the conclusion change if a reasonable alternative assumption or analytic decision were used?”
When to use it
Sensitivity analyses are useful whenever results depend materially on assumptions about confounding, missing data, outcome definitions, exposure windows, model form, censoring, unmeasured bias, or analytic choices.
When not to use it
Do not use a large collection of post hoc analyses to search for a favorable result. The most important sensitivity analyses should be motivated by the study’s actual threats to validity and ideally prespecified.
How it works in plain language
Identify the assumptions most capable of changing the answer. Re-run the analysis under credible alternatives, then compare effect estimates, uncertainty, and conclusions.
Why Mallard may recommend it
Mallard may recommend sensitivity analyses when a plan has a known vulnerability — for example uncertain exposure timing, unmeasured confounding, alternative outcome definitions, or missing-not-at-random concerns.
Useful sensitivity-analysis families
- Alternative cohort or exposure definitions
- Alternative covariate adjustment sets
- Alternative functional forms or model specifications
- Negative-control or falsification analyses when appropriate
- Quantitative bias analysis or unmeasured-confounding analyses
- Missing-data scenarios
- Lag periods and washout windows
- Robustness to influential observations or sites
Two named methods: the E-value and the tipping point
The E-value, for unmeasured confounding, is the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need with both the exposure and the outcome to fully explain away the observed association, beyond the measured covariates. For a risk ratio RR above 1 it is RR + √(RR × (RR − 1)); a risk ratio below 1 is inverted first. Report it for the estimate and for the confidence limit closest to the null, whose E-value is 1 when the interval already includes no effect. It says nothing about what the confounder is, which is both its strength and its limit: whether a confounder that strong is plausible is a subject-matter judgement, and a bias analysis with stated parameters for a named confounder is the more specific alternative. Odds ratios and hazard ratios can be used directly only when the outcome is rare; otherwise they are first converted approximately to the risk ratio scale. A delta-adjusted tipping-point analysis, for missing outcomes, imputes the missing values under missing at random, then shifts them by a chosen amount (delta), usually in one arm, and repeats with larger shifts until the conclusion changes. The reader then judges whether a shift of that size is clinically plausible. It needs a credible missing-at-random analysis as its starting point, and the range of shifts stated in advance.
What to watch out for
“Results were robust” should mean the important scientific conclusion remained reasonably stable across prespecified plausible alternatives — not merely that every p-value stayed below 0.05.
Clinical example
A retrospective cohort analysis repeats the primary model using a stricter exposure definition, a 90-day washout, a different confounder set, and a quantitative unmeasured-confounding assessment.
- Principles of confounder selection (opens in a new tab) · Eur J Epidemiol · doi 10.1007/s10654-019-00494-6
- Methodological Tutorial Series for Epidemiological Studies: Confounder Selection and Sensitivity Analyses to Unmeasured Confounding From Epidemiological and Statistical Perspectives (opens in a new tab) · J Epidemiol · doi 10.2188/jea.JE20240082
- Sensitivity Analysis in Observational Research: Introducing the E-Value (opens in a new tab) · Ann Intern Med · doi 10.7326/M16-2607
- Sensitivity analysis for clinical trials with missing continuous outcome data using controlled multiple imputation: A practical guide (opens in a new tab) · Stat Med · doi 10.1002/sim.8569