Measurement properties and the designs that test them
A measurement validation study shows what a score measures, in which people and for which use, and how consistently and accurately it measures it.
Fit
When it matters, and when it does not
It matters when
- You need a questionnaire or rating scale, and none has evidence for your patients, language and use.
- You are translating, shortening or adapting an existing instrument for a new group.
- You plan to compare scores between groups or over time, and need evidence that such comparisons are fair.
It matters less when
- A suitable instrument already has evidence for your patients and use. Choose it, and run your main study. Read: Survey and questionnaire research
- You only need to know whether two raters or two devices give the same answer. Use an agreement study. Read: Agreement and reliability studies
- You want a test's accuracy against a reference standard for a diagnosis. Use a diagnostic accuracy study. Read: Diagnostic accuracy study
Before you start
What to check in your study
- The construct, the people and the useSay what the score should capture, in whom, and which decision it will inform. Validity belongs to that use, not to the instrument.
- How content validity will be shownAsk patients and experts whether the items are relevant, complete and understood, using cognitive interviews, before any statistics.
- Which measurement properties you will studySuch as structure, internal consistency, test-retest reliability, measurement error, construct validity, responsiveness and measurement invariance. Each needs its own design.
- Hypotheses fixed in advanceFor construct validity and responsiveness, state the expected correlations or group differences, and their direction, before the data are in.
- A missing-data plan that starts from an assumptionDecide why items or whole questionnaires may be missing before choosing a method. The scoring rules say what to do with a few blank items.
- For test-retest reliability, the construct does not change between the two occasions.How to check: Choose an interval short enough for stability and long enough that people do not recall their first answers. Ask whether anything changed.
- Responsiveness is judged in people whose construct really changed.How to check: Use an anchor, such as a patient's own rating of change, and compare the score's change with what you predicted for each group.
- A score means the same thing in each group you will compare.How to check: Test measurement invariance across languages, sexes or ages before comparing their scores. Items can behave differently in different groups.
- Missing items and questionnaires are handled under a stated assumption.How to check: People who are tired, unwell or struggle with reading leave more items blank, so missing completely at random (MCAR) is rarely plausible. Missing at random (MAR) means the gaps depend only on recorded factors, such as other items and age. Under MAR, multiple imputation can help, but it is not automatically better than the instrument's own scoring rule. If people skip an item because their answer would be extreme, missing not at random (MNAR) is plausible. Then add a tipping-point analysis, which shifts the imputed answers until the conclusion changes.
Plain language
How it works
Start with the construct and the use the score will serve. Build or choose items with patients and experts, and check that people understand them. Then study each measurement property with its own design: structure, internal consistency, reliability over repeated measurement, measurement error, and validity against hypotheses set in advance.
Responsiveness is whether the score changes when the construct changes. Test it in people expected to change, against predictions made in advance. Measurement invariance is whether a score means the same thing in different groups, such as two languages. Without it, a difference between groups may come from the questionnaire rather than the people.
Worked example
A worked example
- Setting
- A fatigue questionnaire for adults with cancer is tested in 150 patients starting chemotherapy.
- Question
- Does the score change by more than its measurement error when patients say their fatigue got worse?
- Approach
- A test-retest study gave a smallest detectable change of 6 points. The team predicted, in advance, that patients reporting worse fatigue at six weeks would change by more than that.
- Result
- Among the 48 patients who reported worse fatigue, the mean score rose from 54 to 62, a change of 8 points.
- What it means
- The change of 8 points is larger than the 6 points the scale could detect, as predicted. That is one piece of evidence for responsiveness, not proof that the scale is valid.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
A high Cronbach alpha or correlation with another score does not establish validity. Evidence must support the intended construct, population, language and use.
- Treating a high Cronbach's alpha as proof that a scale is valid.Instead: Alpha rises with the number of items and does not show that they measure one thing. Validity needs content work and hypotheses about the score.
- Using a translated or shortened version as if it carried the original's evidence.Instead: Check content and measurement invariance for the new version, in the people who will use it.
- Dropping respondents with any missing item responses.Instead: Those left may be healthier or better readers, which is selection bias. Apply the scoring rule for blank items, or use multiple imputation.
Inside Mallard
How Mallard uses this
Plans do not link to this concept yet. The kappa and Bland-Altman calculators sit beside this concept. When a plan names factor analysis or item response theory, Mallard's audit rates its complexity very high.
concept.measurement_properties, the id a plan cue records.How to report it
COSMIN reporting guideline, in its 2025 version 2.0, covers studies of the measurement properties of patient-reported outcome measures, and GRRAS covers reliability and agreement studies of any scale, instrument or classification. GRRAS was written because such reports so often left out how the sample was selected, how the study was designed and how the analysis was done, so state all three for every property you assess. A reporting guideline is not a design manual: use it to make the study transparent, not as a substitute for careful design. Check the target journal's own requirements as well, which this page cannot know.
Work it out
Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study (opens in a new tab) · Qual Life Res · doi 10.1007/s11136-018-1829-0
- The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study (opens in a new tab) · Qual Life Res · doi 10.1007/s11136-010-9606-8
- The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes (opens in a new tab) · J Clin Epidemiol · doi 10.1016/j.jclinepi.2010.02.006
- Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls (opens in a new tab) · BMJ · doi 10.1136/bmj.b2393
- COSMIN reporting guideline for studies on measurement properties of patient-reported outcome measures: version 2.0 (opens in a new tab) · Qual Life Res · doi 10.1007/s11136-025-03950-x
- Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed (opens in a new tab) · J Clin Epidemiol · doi 10.1016/j.jclinepi.2010.03.002