Glossary

P-value

The probability of seeing a result at least as extreme as yours if the null hypothesis were true. It is a statement about the data under an assumption, not the probability that the hypothesis is true and not the probability you are wrong. A small p tells you the data sit awkwardly with the null; it says nothing about how big or how important the effect is. The 0.05 line is a convention, not a law of nature — the American Statistical Association's 2016 statement was written largely to say so.

Confidence interval

The range of effect sizes the data are reasonably compatible with. "95%" describes the procedure: intervals built this way capture the true value 95% of the time in the long run. Read the width as precision — a wide interval means the study cannot tell a small effect from a large one, whether or not it crosses the null. An interval is almost always more informative than a p-value, because it shows the size of what you might have.

Confounder

A common cause of both the exposure and the outcome, which makes them look related when the relationship is partly or wholly the third variable's doing. Adjusting for a true confounder removes that distortion. The test is causal, not statistical: a variable is not a confounder because it correlates with both, but because it causes both. This is why the choice belongs in a diagram drawn before the analysis rather than in a stepwise variable-selection routine.

Mediator

A variable on the causal path — the exposure causes it, and it in turn affects the outcome. Adjusting for a mediator when you want the total effect blocks part of the very pathway you are measuring and biases the estimate toward null. Mediators matter for a separate mediation analysis asking how much of the effect travels through that route, which is a different question with its own assumptions and its own, much larger, sample-size requirements.

Collider

A variable caused by two others. Adjusting for one — or selecting your sample on it — opens a spurious association between its causes where none existed. It is the counter-intuitive one: here adjustment creates bias rather than removing it. Restricting a study to hospitalised patients is the classic case, because admission is caused by many things at once.

Effect modification and interaction

The effect of the exposure genuinely differs across levels of another variable — the drug works better in younger patients, say. This is handled analytically with an interaction term and reported as separate estimates within strata; it is not something you adjust away. Effect modification is a finding about the world. Confounding is a distortion to remove. They are often confused because both involve a third variable.

Odds ratio

The odds of the outcome in one group divided by the odds in another. Odds are not risks: when the outcome is common the odds ratio is noticeably further from 1 than the risk ratio, and reading one as the other overstates the effect. Odds ratios are also non-collapsible — the adjusted and unadjusted values differ even with no confounding at all, purely from the mathematics. That property is why mediation with a binary outcome needs a counterfactual approach rather than the product of coefficients.

Risk ratio and risk difference

The risk ratio divides one group's risk by the other's; the risk difference subtracts. The ratio travels well between populations but hides how much is actually at stake — a doubling of a one-in-ten-thousand risk is still almost nothing. The difference carries the clinical weight and is what the number needed to treat is built from. Report both where you can; a paper giving only the ratio is a paper that has left out the part patients care about.

Hazard ratio

The ratio of instantaneous event rates between groups, averaged over follow-up. It is the natural output of a Cox model. It assumes proportional hazards — that the ratio holds steady over time — which is worth checking rather than assuming, with Schoenfeld residuals or by looking at whether the survival curves cross. When hazards are not proportional, a single hazard ratio is an average of something that changed, and it can be misleading.

Statistical power

The probability of detecting an effect of a given size if it is really there. Convention is 80%, which means accepting a one-in-five chance of missing a real effect. Power depends on the effect size you care about, the variability, the sample size and alpha — change any and it moves. Post-hoc power computed from your own observed effect adds nothing: it is a restatement of the p-value, not evidence about what you could have found.

Sample size

The number the design needs to answer its question at the stated power, given an assumed effect size and variability. Those assumptions carry the whole calculation, which is why they should come from pilot data or published work rather than from what produces a convenient answer. State them explicitly. A sample-size calculation is not a prediction — it is a conditional statement, and the condition is the part reviewers read.

Type I and Type II error

Type I is finding an effect that is not there; alpha (usually 0.05) is the rate you accept. Type II is missing one that is; beta is its rate, and power is 1 − beta. The two trade against each other at a fixed sample size — tighten alpha and you lose power. Which matters more is a judgment about consequences, not statistics: a missed harm signal and a false efficacy claim are not equally bad.

Multiple comparisons

Every additional test is another chance at a false positive; run twenty at alpha 0.05 and one spurious hit is the expectation, not bad luck. Bonferroni divides alpha by the number of tests — simple and conservative. False discovery rate methods control the share of claims that are wrong instead, which suits exploratory work better. The stronger protection is structural: name the primary outcome in advance and label everything else exploratory.

Intention to treat

Analyse participants in the group they were randomised to, whatever they actually received. It preserves randomisation, which is the only thing guaranteeing the groups are comparable, and it answers the question a clinician faces: what happens if I offer this. Per-protocol analysis answers a different question and breaks randomisation, so it is a secondary analysis with a caveat rather than a substitute.

Missing data

Dropping incomplete records assumes the missingness is unrelated to what you are studying, which is rarely true and usually untested. The standard vocabulary: MCAR, missing completely at random; MAR, explainable by variables you observed; MNAR, related to the missing value itself. Multiple imputation is valid under MAR and generally beats complete-case analysis. State which assumption you are making and why — that sentence is the plan, and its absence is what reviewers notice.

Clustering

Observations from the same clinic, ward, school or patient are more alike than observations picked at random. Treating them as independent understates the standard errors and produces confidence intervals that are too narrow and p-values that are too small. A random intercept or cluster-robust standard errors handles it. This is why an exposure measured at group level — a neighbourhood index, a hospital policy — with an individual-level outcome must account for clustering or justify not doing so.

Regression to the mean

Pick subjects because a measurement was extreme and their next measurement will tend to be less extreme, with no treatment involved at all. Any single-arm before-and-after study in patients selected for high values is partly measuring this. It is one of the strongest arguments for a control group, and one of the most common reasons an uncontrolled improvement turns out not to be real.

Selection bias

The people in the study differ systematically from the people the question is about, in a way that distorts the answer. It can enter through recruitment, through who consents, through who is retained, and through exclusion rules — including exclusions that look purely technical, such as dropping everyone whose exposure could not be linked or measured. Selection bias is not fixable in the analysis, which is why it is a design problem.

Measurement error and misclassification

Non-differential error — equally noisy in both groups — usually pulls an effect toward the null, so a null result in a poorly measured study is weak evidence of no effect. Differential error, where the mistakes differ by group, can bias in either direction and is the more dangerous kind. Recall bias in case-control studies is the standard example: cases think harder about exposures than controls do.

Events per variable

The old rule of ten outcome events per candidate predictor in a regression, from Peduzzi and colleagues in 1996. Treat it as a floor and not a target: current guidance derives the required sample size directly from the expected model performance and the number of predictors — Riley and colleagues in Statistics in Medicine (2019), and van Smeden and colleagues — rather than relying on a single ratio. A model built at the floor is a model that will not replicate.

Overfitting

The model has learned the peculiarities of your dataset rather than the pattern underneath, so it performs far better in the sample it was built on than it will anywhere else. More predictors, smaller samples and any selection done by looking at the outcome all make it worse. Internal validation by bootstrapping, and shrinkage of the coefficients, give an honest picture; a single split-sample estimate in a small dataset mostly adds noise.

Causal diagram (DAG)

A picture of what you believe causes what, drawn before the analysis. It is what turns "which covariates should I adjust for?" from a statistical question into an answerable one: adjust for the variables that block the backdoor paths, and do not adjust for mediators or colliders. The diagram makes assumptions arguable, which is its real value — it puts the reasoning on the page where a reviewer can disagree with it.

Randomisation

Assigning exposure by chance so that everything else, measured and unmeasured, is balanced on average between groups. It is the only design that handles confounders nobody has thought of. What makes it work in practice is allocation concealment — the person enrolling cannot know or guess the next assignment — which is distinct from blinding after assignment, and more often the thing that fails.

Propensity score

The modelled probability of receiving the exposure given measured covariates, used to match, weight or stratify so the compared groups look alike on those covariates. It is a way of handling many confounders at once, not a way of handling unmeasured ones — it can do nothing about what you did not record. Check overlap and report covariate balance after adjustment; a propensity model whose balance is never shown is a black box.

Sensitivity analysis

Repeating the analysis under a different defensible assumption to see whether the conclusion survives — a different missing-data model, a different outcome definition, a different set of covariates. It is not the same as a subgroup analysis. For unmeasured confounding, the E-value quantifies how strong a hidden confounder would have to be to explain the result away, which is a more useful thing to report than a reassurance that one probably does not exist.

Absolute vs relative effect

A relative measure without the baseline risk cannot be interpreted. "Halves the risk" means something very different at a baseline of 40% than at 0.04%. Report the absolute change alongside the ratio, and say what size of effect would matter clinically before you look at the data — that number is what the power calculation should have been built on, and stating it afterwards invites the suspicion that it was chosen to fit.

Prespecification

Naming the primary outcome, the primary analysis and the subgroups before seeing the data. It is what separates a confirmatory result from an exploratory one, and the distinction is about timing rather than statistics. Exploratory analyses are valuable and worth reporting — labeled as such. The problem is only ever a finding presented as though it had been the plan all along.

Normality and non-parametric tests

Most common tests assume the sampling distribution of the estimate is approximately normal, not that the raw data are — with a reasonable sample size the central limit theorem does much of the work. Non-parametric alternatives such as Wilcoxon or Mann–Whitney relax the assumption but test a slightly different hypothesis and give you no effect estimate. Testing for normality first and then choosing the test on the result is itself a form of data-dependent analysis.

Correlation and causation

An association is a pattern in data; a causal claim says what would happen if you intervened. Getting from one to the other takes a design that rules out the alternatives — confounding, reverse causation, selection, chance — not a stronger p-value or a bigger sample. Bradford Hill's 1965 considerations are the standard checklist for weighing the case, and they are considerations rather than criteria: he was explicit that none of them is required.

Sensitivity and specificity

Sensitivity is the share of people with the condition the test catches; specificity is the share without it that it correctly clears. Both are properties of the test. Predictive values are what a clinician actually wants — the chance the patient has the condition given the result — and those depend on prevalence, so a test that performs well in a specialist clinic can be near-useless as a screen. The AUC summarizes discrimination across every threshold and says nothing about calibration.

Calibration

Whether predicted probabilities match observed frequencies: of the patients given a 20% risk, do about 20% have the event. A model can discriminate well — ranking patients correctly — and still be badly calibrated, which makes its numbers unusable for decisions even though its AUC looks respectable. Report a calibration plot, not just the AUC.

Equivalence and non-inferiority

A non-significant result is not evidence of no difference; it is often just an underpowered study. Showing that two things are similar requires a design built for it, with a margin stated in advance for how much difference would still count as acceptable. That margin is a clinical judgment, and it should be justified before the trial rather than defended after it.

Survival analysis and censoring

Time-to-event methods use the follow-up of people who have not had the event yet rather than discarding them. Censoring is assumed to be uninformative — that leaving the study tells you nothing about the risk — and that assumption fails exactly when people drop out because they are getting worse. Kaplan–Meier describes; the log-rank test compares; Cox regression adjusts.

Competing risks

When another event makes the outcome of interest impossible — death before readmission, say — treating it as ordinary censoring overstates the cumulative incidence, sometimes badly. Cumulative incidence functions handle it properly. Cause-specific hazards answer questions about aetiology; the Fine–Gray subdistribution hazard answers questions about prediction and absolute risk. They are different questions, and which one you want should be decided before the model is fitted.

Bayesian analysis

Bayesian methods produce the probability of a hypothesis given the data, which is what most people mistakenly believe a p-value gives them. That requires a prior, stated openly and ideally examined with a sensitivity analysis. A credible interval says what a confidence interval is usually assumed to say: there is a 95% probability the value lies inside it, given the model and prior.

Reliability and agreement

Cohen's kappa for categorical ratings, the intraclass correlation for continuous ones, and Bland–Altman for whether two methods of measuring the same thing can be used interchangeably. Note that a correlation coefficient is the wrong tool for agreement: two instruments can correlate perfectly while one reads consistently ten units high.

Effect size measures

A standardised way of expressing how big a difference is, independent of the scale — Cohen's d for means, and the correlation coefficient or various R-squared analogues elsewhere. Cohen's small/medium/large labels were offered as a rough fallback for fields with no better yardstick, and he said so; in clinical work a difference in the original units, judged against what matters to a patient, is almost always more informative.

Estimand

The precise quantity a study is trying to estimate, written down before the analysis. It names the population, the treatments being compared, the outcome, how intercurrent events (death, dropout, treatment switching) are handled, and the summary measure. The point is that "does the drug work" is not a question statistics can answer until those five things are fixed. Most disagreements about a result are really disagreements about the estimand, and they are much easier to have before the data arrive than after.

Immortal time bias

A stretch of follow-up during which, by the way the groups were defined, a participant could not have had the outcome. Classic case: patients are called "treated" if they ever filled a prescription, but they had to survive long enough to fill it, so the treated group is guaranteed some event-free time the untreated group is not. The effect is to make treatment look protective when it may do nothing. The fix is to classify exposure as it changes over time, or to align follow-up so that time zero is the same decision point in both groups.

Non-collapsibility

An odds ratio can change when you add a covariate even if that covariate is not a confounder. That is a mathematical property of the odds ratio, not bias: the adjusted and unadjusted numbers answer different questions (the effect for an individual with given covariates, versus the effect averaged over the population). Risk ratios and risk differences do not behave this way. It matters because "the estimate moved when I adjusted" is routinely read as evidence of confounding, and for odds ratios it is not evidence of anything on its own.

The Table 2 fallacy

Reading every row of a multivariable model as if it were a causal effect. The model was built to estimate the effect of one exposure, and the adjustment set was chosen for that exposure; the coefficients on the covariates were not, and each would need its own adjustment set to mean anything causal. Only the exposure row estimates the effect of interest. Westreich and Greenland named this in 2013 and it remains one of the most common errors in published tables.

Person-time and incidence rate

When people are followed for different lengths of time, the denominator is not the number of people but the time they contributed. Someone followed two years contributes two person-years. The incidence rate is events divided by total person-time, which is why rates are quoted per 1,000 person-years rather than as a plain percentage. Using a headcount denominator when follow-up varies makes the group with longer follow-up look worse.

Intraclass correlation (ICC)

How much of the total variation sits between clusters rather than within them — how alike two patients from the same ward are, compared with two from different wards. It drives the design effect, which is how much a cluster-randomised study must inflate its sample size, and even an ICC of 0.02 can add substantially with large clusters. Careful: the same acronym is also used for a reliability coefficient describing agreement between raters. They are different statistics and context is the only thing that separates them.

Marginal standardisation and g-computation

Fit a model, then use it to predict the outcome for every person in the study as if everyone had been treated, and again as if nobody had, and average each set. The difference between those averages is a risk difference or risk ratio for the whole population. It gets you an absolute effect out of a logistic model without the non-collapsibility problem, and it handles covariates without asking the reader to interpret a coefficient. Confidence intervals usually need the bootstrap.

Instrumental variable

A variable that affects the outcome only by affecting the exposure, and is unrelated to the confounders. If one genuinely exists, it recovers a causal effect even with unmeasured confounding. The three assumptions are strong and mostly untestable, and a weak instrument produces estimates that are badly biased and falsely precise. Mendelian randomisation uses genetic variants as the instrument, on the argument that alleles are allocated at conception independently of lifestyle.

Multiple imputation

Missing values are filled in several times, each time with a different plausible draw, so the analysis is run on several complete datasets and the results combined by Rubin's rules. The multiple draws are the point: they carry the uncertainty about what was missing into the final interval, which single imputation throws away. It assumes the data are missing at random given what you have measured, and it is not automatically better than a complete-case analysis — the assumption is what decides.

The bootstrap

Take your sample, draw from it with replacement many times, recompute the statistic each time, and use the spread of those values as the sampling distribution. It gives confidence intervals for quantities that have no tidy formula — a median difference, a c-statistic, anything from g-computation. It is not magic: it assumes the sample represents the population, and it does poorly for quantities that depend on the extremes, like a maximum.

Penalised regression and shrinkage

Deliberately pulls coefficients toward zero to stop the model chasing noise. Ridge shrinks everything a little; lasso can shrink some coefficients exactly to zero and so selects variables; elastic net mixes them. The trade is a little bias for a lot less variance, which usually predicts better in a new sample. The catch for inference: the usual confidence intervals and p-values are not valid after penalisation, so it is a prediction tool rather than a route to an effect estimate.

Splines

A way of letting a continuous variable have a curved relationship with the outcome without guessing the shape. The range is split at a few knots and a smooth curve is fitted between them. It is almost always better than the common alternative of chopping a continuous variable into categories, which throws away information and creates artificial jumps at the cut points. Report it as a curve with a confidence band, not as a single coefficient.

Generalized estimating equations (GEE)

A way of analysing correlated data — repeated measures, or patients within clinics — that estimates the POPULATION-AVERAGE effect: what would happen on average across the whole population if everyone were treated. It gets the standard errors right even if the assumed correlation structure is wrong, provided there are enough clusters. Under about 30-40 clusters the standard errors are too small and need a small-sample correction.

Mixed-effects models

Handles clustered or repeated data by giving each cluster its own intercept (and sometimes slope) drawn from a distribution. The effect it estimates is CLUSTER-SPECIFIC: the effect for a given ward, holding that ward's own level fixed. For a nonlinear model like logistic regression, that is a genuinely different quantity from the population-average effect a GEE reports, and the population-average one is closer to the null. Which one answers the question is a choice to be stated, not a technicality.

Cluster-robust standard errors

Widens the standard errors to account for people within a cluster being more alike than people across clusters, without modelling the correlation. Reliable when there are many clusters; with few — under about 30, and badly under 15 — they are too narrow and the p-values too small. With few clusters use a small-sample correction (CR2, Satterthwaite, Kenward-Roger), analyse at the cluster level, or use randomisation inference.

Interim analysis and alpha spending

Looking at the data before the trial ends inflates the false-positive rate, because each look is another chance to cross the line. An alpha-spending function parcels out the total false-positive budget across the planned looks, so the boundaries are stricter early and the overall rate is preserved. Stopping early for benefit tends to overestimate the effect, which is why boundaries are conservative and why the final estimate needs adjustment.

Minimal clinically important difference (MCID)

The smallest change a patient would notice and care about. It is what makes a result meaningful rather than merely detectable: a study can be so large that a difference nobody would feel comes out statistically significant. Powering a study to the MCID rather than to whatever effect would be convenient is the honest way round. Note that MCIDs are estimated, vary by method and population, and are often quoted with more confidence than the underlying evidence supports.

Surrogate outcome

A measurement that stands in for the outcome you actually care about — a lab value instead of a heart attack. It shortens and shrinks trials, and it goes wrong when the treatment moves the surrogate without moving the real outcome, or moves it the wrong way. A surrogate is only trustworthy if the treatment's whole effect on the outcome runs through it, which is a high bar and has repeatedly failed in practice.

Composite outcome

Several events counted together as one — death, or reinfarction, or revascularisation. It raises the event count, which buys power. The problem is interpretation: the components are usually not equally important, and the common ones (which drive the result) are often the least serious. Always read the components separately, and be wary when the composite is significant and death alone is not.

Per-protocol analysis

Analyses only participants who followed the protocol. It answers what happens under perfect adherence, which is often the clinically interesting question, but it breaks randomisation — adherence is a choice, and the people who adhere differ from those who do not in ways nobody measured. It is reported alongside intention-to-treat, never instead of it, and modern practice prefers methods designed for adherence rather than simple exclusion.

Loss to follow-up and attrition

Participants who leave before the outcome is measured. It costs power, but the real damage is bias when leaving is related to the outcome — sicker patients dropping out makes a treatment look better than it is. What matters is not the percentage alone but whether it differs between groups and why. Report the reasons, compare the leavers with the stayers on baseline characteristics, and test what happens under different assumptions about them.

Generalisability and transportability

Whether a result holds outside the study that produced it. Generalisability asks about the wider population the sample came from; transportability asks about a different population entirely. Both turn on whether the things that modify the effect are distributed differently — a trial in 60-year-olds transports to 80-year-olds only if age does not change how the treatment works. A randomised trial has strong internal validity and that says nothing about either.

The ecological fallacy

Concluding something about individuals from data about groups. Countries with higher average fat intake may have higher cancer rates without any individual's fat intake mattering, because the association at group level can differ in size or even in direction from the one at individual level. Area-level exposures are legitimate to study; the fallacy is in the interpretation, not the data.

Reverse causation

The outcome caused the exposure rather than the other way round. Early disease reduces activity, so low activity looks like a cause of the disease it is actually a symptom of. It is the standard threat in cross-sectional data and in cohorts with short follow-up. The usual defences are establishing that the exposure came first, excluding events in the first years of follow-up, and asking whether the timing makes biological sense.

Likelihood ratios and pre-test probability

A likelihood ratio says how much a test result moves your belief. Positive LR is the chance of that result in someone with the disease divided by the chance in someone without; above 10 is a large move up, below 0.1 a large move down. Crucially the same test moves you from wherever you started, so the post-test probability depends on the pre-test probability — which is why a good test used in a low-prevalence population still produces mostly false positives.

Discrimination and the c-statistic

How well a model separates those who have the outcome from those who do not. The c-statistic is the probability that a randomly chosen case scores higher than a randomly chosen non-case: 0.5 is a coin flip, 1.0 is perfect. It says nothing about whether the predicted risks are numerically right — that is calibration, and a model can discriminate well while being systematically wrong about everyone's absolute risk.

Allocation concealment and blinding

Two different things that are often confused. Allocation concealment hides the UPCOMING assignment from whoever is recruiting, so they cannot steer patients into arms; it is possible in almost every randomised trial and its absence is a well-documented source of exaggerated effects. Blinding hides the assignment AFTER it happens, from patients, clinicians or assessors; it is sometimes impossible, and blinding the outcome assessor is often achievable even when blinding the patient is not.

Heterogeneity and I-squared

In a meta-analysis, whether the studies are estimating the same thing. I-squared is the percentage of variation across studies that is not due to chance — but it is a proportion, not an amount, and a large I-squared with tiny studies can matter less than a small one with huge studies. Tau-squared, the between-study variance, is on the scale of the effect and is usually the more informative number. Heterogeneity is a reason to investigate, not automatically to abandon pooling.

Publication bias and the funnel plot

Studies with striking results are more likely to be published, so a meta-analysis of the literature can be biased even if every individual study is sound. A funnel plot puts effect size against precision; symmetry is reassuring and asymmetry is suggestive, though asymmetry has other causes including genuine differences between small and large studies. Registries and protocols exist largely to make the unpublished findable.

Cost-effectiveness, QALYs and the ICER

A QALY is a year of life weighted by quality, so a year at half quality counts as half. The ICER is the extra cost divided by the extra QALYs of one option over another, and it is judged against a willingness-to-pay threshold. The arithmetic is straightforward; the contested parts are where the quality weights came from, what time horizon was used, and whose costs were counted.

Stepped wedge design

Every cluster eventually gets the intervention, but the order of rollout is randomised. It suits situations where withholding entirely is not acceptable, and every cluster acts partly as its own control. The cost is that it is entangled with time: because later periods have more intervention clusters, any secular trend can masquerade as an effect, so the analysis MUST include a term for time. Sizing needs simulation, not a two-group formula.

Crossover trial and washout

Each participant receives both treatments in a randomised order, so each is their own control, which is efficient for stable chronic conditions. Two things break it: carryover, where the first treatment is still acting during the second period, and period effects, where the condition itself changes over time. A washout gap addresses carryover but lengthens the trial, and a crossover is unsuitable for anything curable or progressive.

Standard error versus standard deviation

The standard deviation describes how spread out the individuals are. The standard error describes how precisely you have estimated something, and it shrinks as the sample grows. They answer different questions and are routinely mixed up in figures: error bars of one standard error look reassuringly tight and are not describing the patients at all. Say which you have plotted.

Interquartile range and skew

The middle half of the data, from the 25th to the 75th percentile. It is reported with the median when the distribution is skewed, because the mean is dragged by the tail and stops describing a typical patient — length of stay and cost are the standard examples. If mean and median differ substantially, the distribution is skewed and the median is usually the honest summary.

Degrees of freedom

Roughly, the number of independent pieces of information left after estimating what the model needed to estimate. It sets the reference distribution for t-tests and F-tests, so it decides how wide an interval is. It is why a small study penalises each extra covariate twice over: fewer degrees of freedom means a wider interval as well as more risk of overfitting.

Wald and profile-likelihood intervals

A Wald interval is the estimate plus or minus a multiple of its standard error, so it is symmetric on whatever scale it was built. That works well in large samples and badly near the boundary — with few events, a Wald interval on a proportion can run below zero. A profile-likelihood interval is built from the likelihood itself, is not forced to be symmetric, and behaves better with sparse data. For odds ratios and hazard ratios the profile version is preferred where the numbers are small.

Tipping-point and E-value analyses

Two ways of asking how fragile a conclusion is. A tipping-point analysis varies an untestable assumption — usually about the people with missing data — until the conclusion changes, and reports how far you had to go. An E-value asks how strongly an unmeasured confounder would have to be associated with both exposure and outcome to explain the whole result. Neither proves anything; both replace "we assumed no unmeasured confounding" with a number a reader can judge.

Restricted mean survival time (RMST)

The average event-free time over a fixed window — the area under the survival curve up to, say, five years. It is reported in months or years, which is directly interpretable, and unlike the hazard ratio it does not assume proportional hazards. When the survival curves cross or converge, a single hazard ratio is close to meaningless and RMST is the honest summary.

Kaplan-Meier and the log-rank test

The Kaplan-Meier curve estimates the proportion still event-free over time, handling censoring by shrinking the at-risk set as people leave. The log-rank test compares whole curves and is most powerful when hazards are proportional. Two cautions: always show the numbers at risk, because the right-hand tail is often built on very few people; and where a competing event such as death can occur, Kaplan-Meier treats it as censoring and overstates cumulative incidence.

Difference-in-differences

Compares the before-and-after change in a group that got the intervention with the change in one that did not, so anything affecting both groups equally cancels out. Everything rests on the parallel-trends assumption: that the two groups would have moved together without the intervention. That assumption is about a counterfactual and cannot be proven, but plotting several pre-intervention periods shows whether it is at least plausible.

Interrupted time series

Uses repeated measurements before and after a change to see whether the series shifts in level, in slope, or both. It is much stronger than a single before-and-after comparison because it can see the underlying trend. It needs enough points either side (a dozen each is a common rule of thumb), must account for autocorrelation and seasonality, and is far more convincing with a control series that experienced no intervention.

Target trial emulation

Write down the randomised trial you would run, then use observational data to imitate it as closely as possible — the same eligibility, the same time zero, the same treatment strategies, the same outcome. The discipline is the point: most classic biases in observational studies, immortal time chief among them, come from misaligning eligibility, treatment assignment and the start of follow-up, and the protocol makes that misalignment visible.