Choosing a test for a table of counts
Choose a table test by whether observations are paired, whether counts support an approximation, and whether an ordered trend is the question.
Fit
When it matters, and when it does not
It matters when
- You have counts of people in categories and want to compare proportions.
- You need to distinguish independent groups from repeated binary outcomes.
It matters less when
- The entries are counts of repeated events per person, not people in categories. Consider count regression. Read: Zero-inflated Poisson regression
- You need covariate adjustment. Consider a regression model that matches the design. Read: Logistic regression
Before you start
What to check in your study
- What one count representsEach person enters one cell in an independent table. Repeated outcomes need their pairing preserved.
- Expected counts under the nullAssess expected cell counts, not just the smallest observed cell.
- Whether categories have a meaningful orderSpecify scores for a trend test before seeing the results.
- The test matches how people were sampled.How to check: Use McNemar for paired binary outcomes. Clustered samples need methods that allow dependence.
- A trend test targets a change across ordered scores.How to check: An overall chi-square test can detect patterns that a trend test misses, including a rise followed by a fall.
Plain language
How it works
Pearson chi-square tests association in an independent table using a large-sample approximation. Sparse expected counts can make that approximation unreliable. Fisher gives conditional exact inference for an independent table, commonly a two-by-two table.
Fisher can be conservative; exact means a calculation under a specified null model, not freedom from design assumptions. Other exact and mid-p options exist. State the version used.
McNemar compares paired binary margins through the discordant pairs. A Cochran-Armitage trend test targets a linear trend in proportions across assigned category scores. It needs independent observations and a binary outcome; it is not an all-purpose test for ordered tables.
Worked example
A worked example
- Setting
- The same patients receive two binary tests.
- Question
- Does the positive fraction differ between tests?
- Approach
- There are 12 pairs positive only on test A and 4 positive only on test B.
- Result
- Of 16 discordant pairs, 12 favor A: 75 in 100. The other 4 favor B.
- What it means
- McNemar compares the directions within discordant pairs. A Fisher test on an unpaired table would discard the paired design.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
Pairing comes before cell size. Fisher and ordinary chi-square do not replace McNemar for paired binary data.
- Using Fisher whenever a paired table has small cells.Instead: Keep McNemar and choose an appropriate small-sample version. Name whether it is exact, asymptotic or mid-p.
- Selecting a trend test because category labels can be numbered.Instead: Use a prespecified, meaningful order. Use an overall test when departures in any direction matter.
Inside Mallard
How Mallard uses this
Mallard lists Choosing a test for a table of counts under Core research concepts.
concept.choosing_counts_test, the id a plan cue records.Work it out
Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- Recommended tests for association in 2 x 2 tables (opens in a new tab) · Stat Med · doi 10.1002/sim.3531
- The McNemar test for binary matched-pairs data: mid-p and asymptotic are better than exact conditional (opens in a new tab) · BMC Med Res Methodol · doi 10.1186/1471-2288-13-91
- Differentiating the Cochran-Armitage Trend Test and Pearson's χ2 Test: Location and Dispersion (opens in a new tab) · Ann Hum Genet · doi 10.1111/ahg.12202