Item response theory
Item response theory links the chance of each item response to an unobserved trait, allowing measurement precision to vary across that trait.
Fit
When to use it, and when not to
Use it when
- You develop or evaluate an item bank or a scale with item-level responses.
- You want to know which items measure well at different trait levels.
Not when
- You only need a summary of a proposed factor structure. Use factor analysis for that structural question. Read: Factor analysis
- You need agreement between two raters. Use an agreement method. Read: Bland-Altman method comparison
Before you start
What you need
- The trait and intended useDefine what a higher score means and whose responses the scale should represent.
- Item responses and a suitable response modelBinary and ordered response formats need suitable models and enough responses in each category.
- How to set the trait scaleFix scale location and spread or use anchor items. Document linking across groups or forms.
- The chosen trait structure fits.How to check: Check whether one trait suffices or a multidimensional model is needed.
- Items are independent conditional on the modeled traits.How to check: Inspect residual item links, such as pairs sharing nearly identical wording.
- Item behavior supports the intended comparisons.How to check: Check differential item functioning: people at the same trait level may respond differently across groups.
Plain language
How it works
An item response curve gives the chance of a response at each trait level. Difficulty locates the curve; discrimination describes its steepness in models that estimate it.
Item information describes where an item contributes precision. It supports short forms or adaptive testing only after calibration and fit checks. Ordered responses use models with category thresholds.
Cella and colleagues describe the PROMIS item-bank program and its first large-scale testing. Those studies illustrate item-level modeling; they do not guarantee that a new bank works in every language or group.
Worked example
A worked example
- Setting
- A team models a binary response to a symptom item.
- Question
- What is its response probability when trait level equals item difficulty?
- Approach
- Use a logistic item model with discrimination 1, difficulty 0 and trait level 0, with no guessing term.
- Result
- The response probability is 50 in 100: an expected 50 of 100 people at that trait level endorse the item.
- What it means
- This is a model probability, not an observed count or the prevalence of a disorder.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
A fitted item response model needs checks of dimensionality, local dependence, fit and group differences. Similar raw totals need not mean equally precise or comparable scores.
- Assuming precision is constant across all scores.Instead: Examine item and test information across the trait range.
- Comparing groups without checking item behavior.Instead: Assess group differences at the same trait level before interpreting score differences.
Inside Mallard
How Mallard uses this
Plans do not link to this method yet. Mallard does not run this analysis.
analysis.item_response_theory, the id a plan cue records.Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years (opens in a new tab) · Med Care · doi 10.1097/01.mlr.0000258615.42478.55
- The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008 (opens in a new tab) · J Clin Epidemiol · doi 10.1016/j.jclinepi.2010.04.011
- Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS) (opens in a new tab) · Med Care · doi 10.1097/01.mlr.0000250483.85507.04
- Development and psychometric evaluation of item banks for memory and attention - supplements to the EORTC CAT Core instrument (opens in a new tab) · Health Qual Life Outcomes · doi 10.1186/s12955-023-02199-7