Survey and questionnaire research
A questionnaire is a way to measure responses. Choose its instrument, sampling frame and timing to fit the question and the study design.
Fit
When it matters, and when it does not
It matters when
- The information you need is something people can report, such as symptoms, experiences or practices.
- You can define who should answer, and reach them through a list or another sampling frame.
- A suitable questionnaire exists, or you can develop and test one first.
It matters less when
- You need a measurement people cannot report reliably, such as a blood level. Measure it, in a cross-sectional study. Read: Cross-sectional study
- No questionnaire has evidence for your construct and people. Develop and validate one first. Read: Measurement properties and the designs that test them
- You want why and how in people's own words. Use a qualitative study. Read: Choosing a qualitative approach
Before you start
What to check in your study
- The target population and the sampling frameSay who the results should describe, and which list you will sample from. Name who the list misses.
- How people are chosen and invitedsamplingStructureclusteredA random sample, everyone on a list, or an open link. Only a probability sample supports an estimate for the whole population.
- The questionnaire, its version and its scoringChoose one with evidence for your construct, people and language. Pretest it, and keep to its scoring rules.
- How the response rate is definedState the denominator: everyone invited, or only those reached and eligible. Report both if they differ.
- A missing-data plan for nonresponse and blank itemsDecide why people or items may be missing before choosing weighting or imputation. The assumption picks the method.
- The sampling frame covers the people the results are meant to describe.How to check: Compare the frame with the target population. A clinic list misses people who stopped attending.
- Respondents understand each question as intended.How to check: Use cognitive interviews and a pilot before launch, in each language and mode you will use.
- The analysis follows the sampling design.How to check: Use the sampling weights, strata and clusters in the analysis. Ignoring them gives the wrong standard errors.
- Nonresponse and blank items are handled under a stated assumption.How to check: People who are busy, unwell or unhappy with care respond less often, so missing completely at random (MCAR) is rarely plausible. Missing at random (MAR) means response depends only on recorded factors, such as age and site. Under MAR, multiple imputation or weighting can help, but it is not automatically better than reporting respondents alone. If people do not respond because of the answer they would give, missing not at random (MNAR) is plausible. Then add a tipping-point analysis, which shifts nonrespondents' answers until the conclusion changes.
Plain language
How it works
Decide what the questions should measure and who should answer them. Check that the instrument fits their language and setting. Test how people understand the questions before use. A questionnaire is a data collection tool, not a study design.
A cross-sectional survey measures people at one time. Repeated cross-sectional surveys sample again; a panel follows the same people. Use the linked cross-sectional and prevalence pages for sampling and precision. Use the repeated-measures concept for within-person follow-up. Report coverage and nonresponse; a large response count does not remove bias.
Worked example
A worked example
- Setting
- A hospital samples 700 patients discharged last quarter, at random from its discharge list.
- Question
- How many patients understood their discharge instructions?
- Approach
- Each patient is sent a validated questionnaire in their own language, with two reminders. Respondents are compared with nonrespondents on age, ward and language.
- Result
- 420 of 700 patients responded, 60 in 100. Nonrespondents were older and more often needed an interpreter.
- What it means
- Respondents may have understood their instructions better than nonrespondents. Weighting by age and language adjusts for those differences, but not for any that were not recorded.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
A large number of responses does not repair a poor sampling frame, selective participation or an unsuitable questionnaire. Reliability alone does not establish validity, and weighting cannot guarantee that respondents represent nonrespondents.
- Treating a large number of responses from an open link as representative.Instead: Size does not repair selection. Describe whom the respondents represent, and estimate prevalence only from a probability sample.
- Writing new questions when a tested questionnaire exists.Instead: Search first. A new or changed item needs pretesting and its own evidence of validity.
- Reporting results without the response rate and its denominator.Instead: Give the number invited, reached, eligible and responding, so readers can judge nonresponse.
Inside Mallard
How Mallard uses this
Plans do not link to this concept yet. Mallard can size a survey by the precision of one proportion, allowing for clusters through their correlation. It makes no allowance for nonresponse, so inflate the number yourself.
concept.survey_questionnaire, the id a plan cue records.How to report it
CROSS is the guideline for survey studies, and CHERRIES covers surveys administered over the internet. They ask how respondents were sampled and recruited, how the response rate was defined or calculated, whether the questionnaire was pretested, and how non-response was addressed. A reporting guideline is not a design manual: use it to make the study transparent, not as a substitute for careful design. Check the target journal's own requirements as well, which this page cannot know.
Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- A Consensus-Based Checklist for Reporting of Survey Studies (CROSS) (opens in a new tab) · J Gen Intern Med · doi 10.1007/s11606-021-06737-1
- Developing questionnaires for educational research: AMEE Guide No. 87 (opens in a new tab) · Med Teach · doi 10.3109/0142159X.2014.889814
- Translation and Cross-Cultural Adaptation: A Critical Step in Multi-National Survey Studies (opens in a new tab) · J Korean Med Sci · doi 10.3346/jkms.2024.39.e336
- Response rates and nonresponse errors in surveys (opens in a new tab) · JAMA · doi 10.1001/jama.2012.3532
- Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls (opens in a new tab) · BMJ · doi 10.1136/bmj.b2393
- Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) (opens in a new tab) · J Med Internet Res · doi 10.2196/jmir.6.3.e34