Two-sample t-test
Use Welch’s two-sample t-test to compare mean outcomes in two independent groups without assuming equal spreads.
Fit
When to use it, and when not to
Use it when
- You compare a continuous outcome between two separate groups of people.
- The mean answers your question, and no other factors need adjustment.
Not when
- The same people supply both measurements. Use a paired t-test, covered in the tests-for-means overview. Read: Choosing a test for means
- You must adjust for age or baseline values. Use linear regression. Read: Linear regression
- Patients share clinics or have repeated visits. Use a mixed-effects model. Read: Mixed-effects models
Before you start
What you need
- Each group’s count, mean and standard deviationUse the standard deviation of individual values, not the standard error of the mean.
- The subtraction order and outcome unitsWrite group 1 minus group 2 before analysis. Keep that direction in the estimate and report.
- The population and measurement timeDefine who the mean describes and when each person’s outcome is measured.
- Patients contribute independently within and between groups.How to check: Check for repeated visits, matching and shared clinics.
- Small-sample inference needs roughly normal values within each group.How to check: Plot each group for heavy skew and extreme values. Larger samples can tolerate skew, but do not cure dependence.
- The groups may have different standard deviations.How to check: Use Welch by default. Do not select a pooled test after a variance pretest.
Plain language
How it works
Welch divides the difference in means by its standard error. That error combines each group’s variance and sample size separately. The degrees of freedom can be fractional. The null hypothesis is equal population means.
Welch’s method removes the pooled equal-variance assumption. Fagerland’s simulations show why a rank test need not answer a question about means. Fagerland and Sandvik show that performance also depends on skew, spread and group sizes.
Worked example
A worked example
- Setting
- Two independent groups have blood pressure measured at one planned visit.
- Question
- How far apart are their mean pressures?
- Approach
- Group 1: n = 40, mean = 128.4, SD = 14.2. Group 2: n = 42, mean = 121.1, SD = 16.8.
- Result
- The difference is 7.30 mmHg; t = 2.13, df = 78.9, p = 0.036. The calculator’s approximate interval is 0.58 to 14.02 mmHg.
- What it means
- The interval uses 1.96 standard errors, not a t critical value. Use a Welch t-based confidence interval for the final report.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
Welch’s test allows unequal spreads, but still needs independent observations. Report the mean difference and its interval, not just a p-value.
- Treating a small p-value as a large clinical benefit.Instead: Judge the mean difference against a clinically meaningful change.
- Calling a rank test a test of the same means.Instead: Rank tests answer a different question. Choose the target first.
Inside Mallard
How Mallard uses this
Mallard links this article to its Welch calculator. The calculator reports a t-based p-value and a normal-approximation interval. The method registry links this method to its published analysis article and code.
welch-two-sample-t-test).analysis.two_sample_t_test, the id a plan cue records.Work it out
Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- t-tests, non-parametric tests, and large studies--a paradox of statistical practice? (opens in a new tab) · BMC Med Res Methodol · doi 10.1186/1471-2288-12-78
- Performance of five two-sample location tests for skewed distributions with unequal variances (opens in a new tab) · Contemp Clin Trials · doi 10.1016/j.cct.2009.06.007