Equivalence analysis: two one-sided tests
Equivalence analysis asks whether a treatment difference is small enough in both directions, using limits chosen before outcomes are examined.
Fit
When to use it, and when not to
Use it when
- You need to rule out clinically important differences in either direction.
- You can defend both equivalence limits for a stated outcome, scale, population and time point.
Not when
- You only need to rule out unacceptable harm in one direction. Use noninferiority analysis. Read: Noninferiority analysis
- You want evidence that one treatment is better. Use a superiority analysis for the chosen outcome and effect. Read: The estimand
Before you start
What you need
- The effect, direction and analysis populationState the treatment-minus-control contrast and the scale on which limits apply.
- Two clinically justified equivalence limitsChoose them before results are known, using external evidence and clinical judgment.
- An estimator, standard error and one-sided significance levelThe confidence interval must use the same estimator, variance method and reference distribution as the tests.
- The limits separate acceptable differences from clinically important differences.How to check: Explain the justification and retain the prespecified limits after results are known.
- The estimator and uncertainty method are valid for the design.How to check: Use the appropriate t, normal or other model-based distribution; account for pairing, clustering and repeated measures.
- Study conduct can reveal a real difference if one exists.How to check: Poor adherence and insensitive measurement can make treatments look alike; examine analysis populations and missing outcomes.
- The missing-data strategy has a stated assumption.How to check: If missing at random is used, justify it and assess departures that could move either interval bound beyond a limit.
Plain language
How it works
Write the effect as treatment minus control. The null includes effects at or beyond either unacceptable limit. One test asks whether the effect exceeds the lower limit; the other asks whether it is less than the upper limit. Declare equivalence only if both reject at the chosen one-sided significance level. Do not halve that level: the claim requires both rejections.
With one-sided tests at 0.05, the corresponding two-sided confidence interval is 90%, and must lie strictly inside both limits. Use the same estimator and uncertainty method for tests and interval. Schuirmann establishes the two one-sided tests procedure and its interval connection. Westlake develops confidence-interval reasoning for bioequivalence. A failed equivalence test can be inconclusive; it does not establish an important difference.
Worked example
A worked example
- Setting
- A large trial estimates a treatment-minus-control mean difference of 1 point, with a standard error of 2 points.
- Question
- Can differences at or beyond -5 and 5 points be ruled out?
- Approach
- Use large-sample normal Wald tests, each at a one-sided significance level of 0.05, with limits fixed before analysis.
- Result
- The 90% Wald interval is -2.29 to 4.29 points. The larger one-sided p-value is 0.023; both tests reject at 0.05.
- What it means
- The interval lies strictly inside -5 to 5, supporting equivalence within those limits. It does not prove that the true effect is zero.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
A nonsignificant difference does not establish equivalence. Both one-sided tests must reject effects at or beyond the prespecified limits.
- Calling a large superiority p-value proof of equality.Instead: Test both limits directly and report the interval with its margin.
- Using a 95% interval for the usual one-sided 0.05 decision.Instead: The matching two-sided interval is 90%. Report a 95% interval additionally if useful, clearly labeled.
- Choosing a margin after seeing the confidence interval.Instead: Use an externally justified prespecified margin; changing it changes the claim.
Inside Mallard
How Mallard uses this
Plans do not link to this method yet.
analysis.equivalence, the id a plan cue records.Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability (opens in a new tab) · J Pharmacokinet Biopharm · doi 10.1007/BF01068419
- Symmetrical confidence intervals for bioequivalence trials (opens in a new tab) · Biometrics