Ask Mallard
Analysis method

Two-sample t-test

4 min read

Use Welch’s two-sample t-test to compare mean outcomes in two independent groups without assuming equal spreads.

Fit

When to use it, and when not to

Use it when

  • You compare a continuous outcome between two separate groups of people.
  • The mean answers your question, and no other factors need adjustment.

Not when

Before you start

What you need

Decide these first
  1. Each group’s count, mean and standard deviationUse the standard deviation of individual values, not the standard error of the mean.
  2. The subtraction order and outcome unitsWrite group 1 minus group 2 before analysis. Keep that direction in the estimate and report.
  3. The population and measurement timeDefine who the mean describes and when each person’s outcome is measured.
It rests on these
  1. Patients contribute independently within and between groups.How to check: Check for repeated visits, matching and shared clinics.
  2. Small-sample inference needs roughly normal values within each group.How to check: Plot each group for heavy skew and extreme values. Larger samples can tolerate skew, but do not cure dependence.
  3. The groups may have different standard deviations.How to check: Use Welch by default. Do not select a pooled test after a variance pretest.

Plain language

How it works

Welch divides the difference in means by its standard error. That error combines each group’s variance and sample size separately. The degrees of freedom can be fractional. The null hypothesis is equal population means.

What the sources show

Welch’s method removes the pooled equal-variance assumption. Fagerland’s simulations show why a rank test need not answer a question about means. Fagerland and Sandvik show that performance also depends on skew, spread and group sizes.

The method at a glance
Two-sample t-test: Independent group summaries. Then Separate variance contributions. Then Mean difference and Welch t test.Two-sample t-testIndependent groupsummariesSeparate variancecontributionsMean difference and Welcht test
Conceptual diagram. It shows the structure of the idea, not data from any study.

Worked example

A worked example

Invented for illustration
Setting
Two independent groups have blood pressure measured at one planned visit.
Question
How far apart are their mean pressures?
Approach
Group 1: n = 40, mean = 128.4, SD = 14.2. Group 2: n = 42, mean = 121.1, SD = 16.8.
Result
The difference is 7.30 mmHg; t = 2.13, df = 78.9, p = 0.036. The calculator’s approximate interval is 0.58 to 14.02 mmHg.
What it means
The interval uses 1.96 standard errors, not a t critical value. Use a Welch t-based confidence interval for the final report.

In practice

Recent studies PubMed search · not vetted

Searching PubMed…

Watch out

Common mistakes

The one to remember

Welch’s test allows unequal spreads, but still needs independent observations. Report the mean difference and its interval, not just a p-value.

  • Treating a small p-value as a large clinical benefit.Instead: Judge the mean difference against a clinically meaningful change.
  • Calling a rank test a test of the same means.Instead: Rank tests answer a different question. Choose the target first.

Inside Mallard

How Mallard uses this

Mallard links this article to its Welch calculator. The calculator reports a t-based p-value and a normal-approximation interval. The method registry links this method to its published analysis article and code.

Code example in Mallard: Two-sample comparison of means with unequal variances (Welch) (welch-two-sample-t-test).
Sample size: Mallard sizes the main comparison with two means. It does not size adjustment or any secondary analysis.
Calculators: Independent t-test (Welch) (§2.3).
Atlas node analysis.two_sample_t_test, the id a plan cue records.
Code for this method
Loading…

Work it out

Related Learn pages

Related designs and methods

Evidence

Sources & further reading

  1. t-tests, non-parametric tests, and large studies--a paradox of statistical practice? (opens in a new tab) · BMC Med Res Methodol · doi 10.1186/1471-2288-12-78
  2. Performance of five two-sample location tests for skewed distributions with unequal variances (opens in a new tab) · Contemp Clin Trials · doi 10.1016/j.cct.2009.06.007
Prepared with AI assistance. Source records verified by PMID, title, authors, year and DOI on 2026-11-06. No independent professional approval is claimed. The two primary simulation studies support the distinction between means and ranks and the limits of robustness under skew and unequal variances.
Last source review: 2026-11-06
AI-generated educational content
Mallard uses reputable methodological and reporting sources, but this material may contain errors or omit important context. Verify important methodological decisions with the cited sources and, when appropriate, a qualified research methodologist or statistician.