t-tests and ANOVA
What question does this help answer?
Whether the average value of something differs between groups by more than sampling variation would plausibly produce.
They are special cases of regression
An independent-samples t-test is a linear regression with one binary predictor, and a one-way ANOVA is a linear regression with one categorical predictor. Nothing is lost by thinking of them that way, and a good deal is gained: regression accommodates covariates, unequal group sizes and clustering, while a t-test does not.
Which version
- Independent-samples t-test: two separate groups of people.
- Paired t-test: two measurements on the same person, analysed as one column of differences. Using the independent-samples version here throws away the pairing and loses power.
- Welch's t-test: two groups without assuming equal variances. It is the sensible default, since equal variance is rarely known and the cost of Welch when variances ARE equal is negligible.
- One-way ANOVA: three or more independent groups.
- Repeated-measures ANOVA: several measurements per person, though a mixed-effects model handles the same data with fewer restrictions and copes with missing visits.
What group means and their spread look like
Assumptions
Independence of observations, approximate normality of the residuals rather than of the raw data, and, for the classical versions, similar variances. Normality matters least in large samples, where the central limit theorem carries the mean; it matters most with small samples and heavy skew, which is exactly where people reach for a normality test and misread a non-significant result as reassurance.
Why Mallard may recommend it
Mallard may name a t-test or ANOVA when the outcome is continuous, the groups are independent and the comparison is simple, and it will size the study with the exact noncentral t rather than a normal approximation.
What to watch out for
A significant t-test says the means differ, not by how much or whether that matters; report the difference with its confidence interval, never the p-value alone. ANOVA's global F test says only that the groups are not all alike, so the pairwise comparisons that follow it need a multiplicity adjustment. And both assume observations are INDEPENDENT: repeated measurements on the same patient, or patients clustered within wards, break that assumption and understate the standard error.
After a significant ANOVA
The F test does not say which groups differ. Prespecify the comparisons that matter and adjust for them; running every pair and reporting the ones that reached significance is a multiplicity problem with a predictable false-positive rate.
Clinical example
Question: Does mean postoperative pain score differ between three analgesic regimens? Analysis: One-way ANOVA for the overall comparison, then the two prespecified pairwise contrasts with an adjustment, reporting mean differences with confidence intervals.
- T test as a parametric statistic (opens in a new tab) · Korean J Anesthesiol · doi 10.4097/kjae.2015.68.6.540
- Analysis of variance (ANOVA) comparing means of more than two groups (opens in a new tab) · Restor Dent Endod · doi 10.5395/rde.2014.39.1.74