Choosing factor analysis or item response theory
Choose factor analysis for latent structure questions and item response theory for how item responses vary across a trait.
Fit
When it matters, and when it does not
It matters when
- You need to explore or test which unobserved traits account for item associations.
- You have a hypothesized factor structure and want to test its fit.
It matters less when
- Choose item response theory for item difficulty, trait-specific precision or adaptive item selection. Read: Measurement properties and the designs that test them
- For repeatability alone, use an agreement design rather than either structural model. Read: Agreement and reliability studies
Before you start
What to check in your study
- Response formatSpecify continuous, ordered or binary item responses and missingness.
- Measurement purposeChoose exploration, confirmation, scoring, item banking or group comparison.
- Sample coverageCheck that respondents cover the trait range and groups of interest.
- The latent dimensions, unobserved traits, are adequately specified.How to check: Check model fit and whether exploratory findings hold in new data.
- Items are locally independent conditional on the modeled traits.How to check: Inspect residual associations and consider linked item content.
- The intended score works comparably across groups.How to check: Test measurement invariance or differential item functioning before group comparisons.
Plain language
How it works
Exploratory factor analysis looks for a structure behind item associations. Confirmatory factor analysis tests a specified structure. Treat ordered responses with an appropriate categorical model rather than assuming equal distances between response levels.
Item response theory models the chance of an item response as a function of an unobserved trait. It can describe where items are most precise. Some categorical factor and item response models are reparameterizations of one another. Reise and colleagues compare their use for invariance. Morales and colleagues illustrate checking group comparability. Neither a good fit nor high internal consistency proves that the intended construct is measured.
Worked example
A worked example
- Setting
- A team develops a short symptom scale.
- Question
- Does good fit guarantee that the score works equally across languages?
- Approach
- They test the proposed factor structure, then compare item response patterns across language groups.
- Result
- Two of 10 items show differential item functioning, or 20 in 100 items. The other 8 have no detected difference in this sample.
- What it means
- The 2 flagged items may affect group comparisons. No detected difference in the others is not proof of invariance; assess precision, content and score impact.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
Factor analysis and item response theory are related latent-variable approaches, not competing proofs of validity. Check dimensionality, local dependence and invariance before interpreting a score or comparing groups.
- Treating exploratory factor discovery as confirmation.Instead: Test the structure in separate data or through a prespecified validation plan.
- Calling any item difficulty estimate invariant.Instead: Check model fit and comparability in the target groups.
- Using alpha alone to prove dimensionality.Instead: Examine the measurement model and item content.
Inside Mallard
How Mallard uses this
Mallard includes this comparison in Learn search and the Atlas. Plans do not link to this comparison automatically.
concept.choosing_factor_irt, the id a plan cue records.Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance (opens in a new tab) · Psychol Bull · doi 10.1037/0033-2909.114.3.552
- Evaluating the equivalence of health care ratings by whites and Hispanics (opens in a new tab) · Med Care · doi 10.1097/00005650-200005000-00008
- Item response theory and health outcomes measurement in the 21st century (opens in a new tab) · Med Care · doi 10.1097/00005650-200009002-00007