Ask Mallard
Core concept

Survey and questionnaire research

4 min read

A questionnaire is a way to measure responses. Choose its instrument, sampling frame and timing to fit the question and the study design.

Fit

When it matters, and when it does not

It matters when

  • The information you need is something people can report, such as symptoms, experiences or practices.
  • You can define who should answer, and reach them through a list or another sampling frame.
  • A suitable questionnaire exists, or you can develop and test one first.

It matters less when

Before you start

What to check in your study

Decide these first
  1. The target population and the sampling frameSay who the results should describe, and which list you will sample from. Name who the list misses.
  2. How people are chosen and invitedsamplingStructureclusteredA random sample, everyone on a list, or an open link. Only a probability sample supports an estimate for the whole population.
  3. The questionnaire, its version and its scoringChoose one with evidence for your construct, people and language. Pretest it, and keep to its scoring rules.
  4. How the response rate is definedState the denominator: everyone invited, or only those reached and eligible. Report both if they differ.
  5. A missing-data plan for nonresponse and blank itemsDecide why people or items may be missing before choosing weighting or imputation. The assumption picks the method.
It rests on these
  1. The sampling frame covers the people the results are meant to describe.How to check: Compare the frame with the target population. A clinic list misses people who stopped attending.
  2. Respondents understand each question as intended.How to check: Use cognitive interviews and a pilot before launch, in each language and mode you will use.
  3. The analysis follows the sampling design.How to check: Use the sampling weights, strata and clusters in the analysis. Ignoring them gives the wrong standard errors.
  4. Nonresponse and blank items are handled under a stated assumption.How to check: People who are busy, unwell or unhappy with care respond less often, so missing completely at random (MCAR) is rarely plausible. Missing at random (MAR) means response depends only on recorded factors, such as age and site. Under MAR, multiple imputation or weighting can help, but it is not automatically better than reporting respondents alone. If people do not respond because of the answer they would give, missing not at random (MNAR) is plausible. Then add a tipping-point analysis, which shifts nonrespondents' answers until the conclusion changes.

Plain language

How it works

Decide what the questions should measure and who should answer them. Check that the instrument fits their language and setting. Test how people understand the questions before use. A questionnaire is a data collection tool, not a study design.

Choose the timing and design

A cross-sectional survey measures people at one time. Repeated cross-sectional surveys sample again; a panel follows the same people. Use the linked cross-sectional and prevalence pages for sampling and precision. Use the repeated-measures concept for within-person follow-up. Report coverage and nonresponse; a large response count does not remove bias.

Conceptual diagram
Questionnaire conduct begins with a defined construct and target group. A suitable tested instrument is chosen, language and mode are pretested, delivery is kept consistent, and responses are scored by the stated rules.Use and deliver a questionnaire consistentlyDefine the construct and target groupChoose a tested instrument and scoring rulesPretest language, mode and understandingDeliver consistently and record responseScore by the stated rulesA tested instrument still needs evidence for the intended language, mode and group. Response counts alonedo not establish representativeness.
Conceptual diagram. It shows the structure of the idea, not data from any study.

Worked example

A worked example

Invented for illustration
Setting
A hospital samples 700 patients discharged last quarter, at random from its discharge list.
Question
How many patients understood their discharge instructions?
Approach
Each patient is sent a validated questionnaire in their own language, with two reminders. Respondents are compared with nonrespondents on age, ward and language.
Result
420 of 700 patients responded, 60 in 100. Nonrespondents were older and more often needed an interpreter.
What it means
Respondents may have understood their instructions better than nonrespondents. Weighting by age and language adjusts for those differences, but not for any that were not recorded.

In practice

Recent studies PubMed search · not vetted

Searching PubMed…

Watch out

Common mistakes

The one to remember

A large number of responses does not repair a poor sampling frame, selective participation or an unsuitable questionnaire. Reliability alone does not establish validity, and weighting cannot guarantee that respondents represent nonrespondents.

  • Treating a large number of responses from an open link as representative.Instead: Size does not repair selection. Describe whom the respondents represent, and estimate prevalence only from a probability sample.
  • Writing new questions when a tested questionnaire exists.Instead: Search first. A new or changed item needs pretesting and its own evidence of validity.
  • Reporting results without the response rate and its denominator.Instead: Give the number invited, reached, eligible and responding, so readers can judge nonresponse.

Inside Mallard

How Mallard uses this

Plans do not link to this concept yet. Mallard can size a survey by the precision of one proportion, allowing for clusters through their correlation. It makes no allowance for nonresponse, so inflate the number yourself.

No Mallard code example points to this page.
Atlas node concept.survey_questionnaire, the id a plan cue records.

How to report it

CROSS is the guideline for survey studies, and CHERRIES covers surveys administered over the internet. They ask how respondents were sampled and recruited, how the response rate was defined or calculated, whether the questionnaire was pretested, and how non-response was addressed. A reporting guideline is not a design manual: use it to make the study transparent, not as a substitute for careful design. Check the target journal's own requirements as well, which this page cannot know.

Related Learn pages

Related designs and methods

Evidence

Sources & further reading

  1. A Consensus-Based Checklist for Reporting of Survey Studies (CROSS) (opens in a new tab) · J Gen Intern Med · doi 10.1007/s11606-021-06737-1
  2. Developing questionnaires for educational research: AMEE Guide No. 87 (opens in a new tab) · Med Teach · doi 10.3109/0142159X.2014.889814
  3. Translation and Cross-Cultural Adaptation: A Critical Step in Multi-National Survey Studies (opens in a new tab) · J Korean Med Sci · doi 10.3346/jkms.2024.39.e336
  4. Response rates and nonresponse errors in surveys (opens in a new tab) · JAMA · doi 10.1001/jama.2012.3532
  5. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls (opens in a new tab) · BMJ · doi 10.1136/bmj.b2393
  6. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) (opens in a new tab) · J Med Internet Res · doi 10.2196/jmir.6.3.e34
Prepared with AI assistance; publication follows agent methodological review under the owner’s expansion request. Each source record was rechecked by PMID, with its title, journal and DOI matched against PubMed, on 2026-11-06; a record PubMed holds without a DOI was matched by title and journal. No independent professional or methodological approval is claimed. CROSS supports reporting. The questionnaire guide focuses on educational research; instrument selection and validation must be justified for the clinical construct and intended respondents. Source records rechecked against PubMed on 2026-11-07: record verified, confirm it supports this use. No independent professional or methodological approval is claimed.
Last source review: 2026-11-07
AI-generated educational content
Mallard uses reputable methodological and reporting sources, but this material may contain errors or omit important context. Verify important methodological decisions with the cited sources and, when appropriate, a qualified research methodologist or statistician.