Explain the evaluation of quantitative data collection in pharmaceutical research
August 18, 2020
Explain the integration of artificial intelligence and bioinformatics?
August 27, 2020Designing and Constructing an interview schedule or questionnaire for health care industries
Summary
Most problems that surface when healthcare survey data are analyzed were created when the instrument was designed, and no statistical method can fully repair them afterwards. This guide shows hospital quality teams, CROs, pharmaceutical and medtech evidence teams, and healthcare researchers how to choose between a questionnaire and an interview schedule, write questions that map to a planned analysis, select response scales, estimate sample size, test validity and reliability, and pilot the instrument before fieldwork.Questionnaire vs Interview Schedule: Which Should You Use?
Both instruments collect standardized data. The difference lies in who records the answers and how much interaction takes place.
Feature | Questionnaire | Interview schedule |
Who records answers | The respondent | A trained interviewer (enumerator) |
Literacy required of respondent | Yes | No |
Typical data | Mainly quantitative, with optional open-ended items | Quantitative, qualitative or both |
Clarification | Not possible during completion | Interviewer can clarify within the approved protocol |
Cost per completed response | Lower | Higher (interviewer time, training, travel) |
Typical sample | Larger samples | Smaller or purposively selected samples |
Main risk | Non-response and misunderstood items | Interviewer effects and social desirability |
Common healthcare use | Patient-experience surveys, staff surveys | Community health surveys, low-literacy populations, qualitative studies |
Use a self-administered questionnaire when respondents can read the language comfortably and you need comparable answers from many people at low cost. Use an interview schedule when respondents may struggle to complete a form, when you need a high response rate from a hard-to-reach group, or when you need to explore experiences in depth.
The two are often combined. In a mixed-methods study, a questionnaire might measure patient satisfaction across several clinics, and follow-up semi-structured interviews could then explain why one clinic scored lower than the others.
How to Design a Questionnaire or Interview Schedule for Healthcare Research
A questionnaire is a structured set of questions that respondents usually complete themselves, on paper or online. An interview schedule is a structured set of questions that a trained interviewer reads out and records, word for word, in the same order for every participant. When interviewers work from a looser list of topics and probes, the instrument is more accurately called an interview guide, and the interview is semi-structured.
The choice matters more in healthcare than in most fields. Patients may have limited literacy, poor eyesight, cognitive impairment or a first language different from the instrument’s. Clinical topics are often sensitive. And healthcare findings frequently feed decisions about services, regulatory submissions or publications, where a reviewer will ask how the instrument was developed and tested (Boynton & Greenhalgh, 2004) [4].
Good healthcare survey design therefore starts long before the first question is written. This guide follows the decisions in the order a research team should make them: which instrument, which questions, which scales, how many participants, and how to show that the instrument measures what it claims to measure.
How to Design a Healthcare Research Questionnaire
Healthcare questionnaire development starts with the research objective and the analysis plan, not with a list of questions. Every item should exist because a specific analysis needs it (Rattray & Jones, 2007) [9].
Healthcare research questionnaire design process: from research objective to data collection
Define the Objective, Population and Outcome
A usable objective names the construct, the population and the setting. For example: “To measure patient satisfaction with outpatient services among adults attending urban primary care clinics.”
From that objective, the team can list the dimensions to measure (waiting time, communication, accessibility, staff interaction, perceived quality of care) and decide on the primary outcome. The primary outcome drives the sample-size calculation later, so it must be fixed before any questions are written.
The population also shapes wording and mode. An instrument for patients usually needs plainer language than one for physicians.
Map Every Question to a Variable and an Analysis
Before writing items, draft a data dictionary: one row per variable, with its name, the question that produces it, the response options, the coding (for example, 1 = Very dissatisfied to 5 = Very satisfied), the level of measurement and the planned analysis. This single step prevents the two most common design failures: questions nobody analyzes, and analyses for which no question exists.
The level of measurement that a question produces limits the statistics you can use afterwards:
Question type | Example | Level of measurement | Suitable summary | Typical comparison |
Yes/No | “Did you receive written discharge instructions?” | Binary | Percentage | Chi-square test or logistic regression |
Single choice | “Which clinic did you attend?” | Nominal | Counts and percentages | Chi-square test |
Single Likert item | “I was treated with respect” (5 points) | Ordinal | Median and distribution | Mann–Whitney U or ordinal regression |
Multi-item scale score | Sum of 6 satisfaction items | Usually treated as approximately interval | Mean and SD (check distribution) | t-test, ANOVA or linear regression |
Numeric | “How many minutes did you wait?” | Ratio | Median or mean | Depends on distribution |
Open-ended | “What could be improved?” | Text | Coded themes | Thematic analysis with a coding frame |
A practical consequence: if you want to model waiting time as a continuous outcome, ask for minutes. If you ask “Was your wait short, acceptable or long?”, you can only ever analyze three ordered categories.
Write Clear, Neutral Questions
Write in the respondents’ language, not the research team’s. Avoid:
- Double-barreled questions. “How satisfied are you with the speed and quality of care?” asks two things. A patient may be happy with quality and unhappy with speed. Split it into “How satisfied are you with the time you waited to receive care?” and “How satisfied are you with the quality of care you received?”
- Leading questions. “How helpful was our friendly nursing staff?” assumes the answer. Ask “How would you rate the help you received from the nursing staff?”
- Vague timeframes. “Recently” and “regularly” mean different things to different people. Specify a recall period, such as “in the past 4 weeks”.
- “Did you experience any adverse events?” will confuse many patients. Describe the experience instead.
- Hidden assumptions. “How often do you take your medication late?” assumes the respondent takes medication.
Order the Questions and Add Skip Logic
Start with easy, non-sensitive questions that build rapport, and place sensitive topics (income, sexual health, mental health, substance use) later. Group related items so respondents are not switching topics constantly.
Skip logic keeps the instrument short and the data clean. A participant who has never used a telehealth service should not see ten questions about their telehealth experience. Every skip must be reflected in the data dictionary, so analysts can distinguish “not applicable” from “missing”.
Selecting Appropriate Response Scales
The response scale should match the construct and the planned analysis.
A Likert item is a single statement with ordered response options, such as Strongly disagree | Disagree | Neither agree nor disagree | Agree | Strongly agree. A Likert scale is the sum or mean of several Likert items that measure the same construct. The distinction affects analysis. A single item is ordinal, so it is usually summarized with medians and compared with non-parametric tests or ordinal regression. A multi-item scale score has more possible values and is often treated as approximately interval, which allows means and linear models when its distribution supports it.
Other decisions to make deliberately:
- Number of points. Five or seven points are common. More points add little if respondents cannot distinguish between them.
- A neutral option lets genuinely neutral respondents answer honestly, but it can also attract people who have not thought about the question.
- “Not applicable” option. Without it, respondents who have no experience of the topic may choose the midpoint, which distorts the results.
- Labeled anchors. Label every point, or at least both ends, so respondents interpret the scale consistently.
Use, Adapt or Develop: Validated Instruments First
Before writing a new scale, search for an existing validated instrument. Established patient-reported outcome measures (PROMs) and patient-experience instruments already have published evidence of validity and reliability, and they let you compare your results with other studies. Developing a new measurement scale is justified only when no existing instrument measures your construct in your population.
Two cautions apply. First, many instruments are copyrighted and require permission or a license. Second, translating an instrument is not the same as using it. A translated instrument needs a formal cross-cultural adaptation process, typically forward translation, synthesis, back-translation, expert committee review and pretesting, before its scores can be treated as equivalent to the original (Beaton et al., 2000) [1]. This is especially relevant in multilingual settings, where one study may need the same instrument in several languages.
Clinical research questionnaire design carries an extra layer of scrutiny. Where results may support a regulatory labeling claim, the US Food and Drug Administration’s guidance on patient-reported outcome measures sets out what evidence of content validity and measurement properties is expected (U.S. Food and Drug Administration, 2009) [11]. The COSMIN initiative provides a widely used framework for evaluating the measurement properties of PROMs (Prinsen et al., 2018) [8] .
How to Design an Interview Schedule for Healthcare Research
Structured Interview Schedules
In a structured interview, the interviewer reads each question exactly as written, in the same order, and records the answer in predefined categories. This is the right choice for large community health surveys, for respondents who cannot complete a written form, and for any study where interviewer answers must be comparable across sites. Its weakness is that it cannot explore unexpected answers.
Semi-Structured Interview Guides
A semi-structured interview uses core questions and topics but lets the interviewer follow up. It suits qualitative research on experiences, barriers, motivations and decision-making.
For example, an interviewer might ask: “What challenges did you experience when accessing this service?” If the participant mentions transport, a suitable probe would be: “Can you tell me more about how transport affected your ability to attend appointments?”
Useful neutral probes include:
- “Can you tell me more about that?”
- “What happened next?”
- “How did that affect you?”
- “Can you explain what you mean by that?”
Probes should invite clarification without introducing the interviewer’s own assumptions.
Training and Recording
Interviewer effects are a real source of variation. Train interviewers on the objectives, exact wording, probing rules, consent procedures and how to record answers. For qualitative interviews, decide in advance how sessions will be recorded and transcribed, and develop a coding frame. If two people code the transcripts, measure their agreement (for example, with Cohen’s kappa) on a sample before coding the full dataset.
Response Bias in Healthcare Questionnaires
Good wording reduces bias, but some biases come from the topic, the mode or the respondent. Patient survey questionnaire design has to anticipate all of them.
- Recall bias. Patients remember recent and dramatic events better than routine ones. Shorter, specific recall periods help.
- Social desirability bias. Respondents under-report stigmatized behaviors and over-report healthy ones. This tends to be stronger in face-to-face interviews than in anonymous self-completion.
- Acquiescence bias. Some respondents tend to agree with any statement. Including a few reverse-worded items can reveal this pattern, although reverse wording can also confuse respondents, so pilot it.
- Mode effects. The same question can produce different answers on paper, online and by interview. Avoid mixing modes within a study unless you plan to test for mode differences.
- Non-response bias. If the patients who did not respond differ systematically from those who did, the results describe only the responders. Compare responders and non-responders on any data you have for both, such as age, sex or clinic.
Sample Size and Questionnaire Design
Sample size depends on the study objective and the primary outcome, not on the length of the questionnaire. There are three different calculations, and teams often confuse them.
- Estimating a proportion. Suppose the primary outcome is the proportion of patients who are satisfied, and you want to estimate it within ±5 percentage points with 95% confidence. With no prior estimate, assume 50%, which gives the largest sample:
n = 1.96² × 0.5 × 0.5 ÷ 0.05² = 384.2, rounded up to 385 completed responses.
If you expect 30% of invited patients not to respond, you need to invite 385 ÷ 0.70 = 550 patients. If the study will compare subgroups, such as three clinics, each subgroup needs enough responses for its own estimate, which increases the total considerably.
- Validating a new scale. Psychometric analyses such as factor analysis need their own sample, which depends on the number of items, the strength of the relationships between them and the analysis method. Simple rules of thumb (a fixed number of respondents per item) are widely quoted but are not a substitute for a justified calculation.
- Qualitative interviews. Interview studies do not use a power calculation. Sample adequacy is judged by information richness and saturation, the point at which new interviews add little new insight. A systematic review of empirical saturation studies found that saturation was often reached within 9 to 17 interviews, particularly in relatively homogeneous populations (Hennink & Kaiser, 2022) [5]. More heterogeneous populations or broader questions need more.
Validity and Reliability of Healthcare Research Instruments
A questionnaire can be clearly worded and easy to complete and still measure the wrong thing, or measure the right thing inconsistently. Validity and reliability need evidence, not assertion (Bolarinwa, 2015; Rosellini & Brown, 2021) [2,10].
Validity: Does It Measure the Right Thing?
- Content validity. Do the items cover all relevant aspects of the construct, and only those? This is usually assessed by an expert panel and by interviews with members of the target population.
- Construct validity. Do the scores behave as theory predicts? Evidence includes factor analysis (exploratory factor analysis to discover structure, confirmatory factor analysis to test a hypothesized structure) and expected correlations with related measures.
- Criterion validity. Do scores agree with an accepted reference measure or predict a relevant outcome?
Reliability: Does It Measure Consistently?
- Internal consistency. Cronbach’s alpha is the most reported statistic, but it is often misinterpreted. It assumes all items contribute equally to the construct, and a high alpha does not prove that a scale measures a single construct. McDonald’s omega is a more flexible alternative (McNeish, 2018) [7]. Very high alpha values can also signal redundant items that ask the same thing in slightly different words, rather than a better scale (Boyle, 1991) [3].
- Test–retest reliability. If the construct is stable, scores should be similar when the same respondents complete the instrument twice. This is usually assessed with the intraclass correlation coefficient (ICC). A commonly used guideline treats ICC values below 0.5 as poor, 0.5 to 0.75 as moderate, 0.75 to 0.9 as good and above 0.9 as excellent, and requires the ICC form and its confidence interval to be reported (Koo & Li, 2016) [6].
- Inter-rater reliability. When interviewers or coders classify answers, their agreement should be measured, for example with Cohen’s kappa for categorical codes.
Worked interpretation. Suppose a six-item “communication” subscale returns α = 0.96. Rather than reporting this as excellent, examine the item correlations. If two items read “The doctor explained things clearly” and “The doctor’s explanations were easy to understand”, they may be near-duplicates. Removing one can shorten the instrument with almost no loss of information.
Pilot Testing and Cognitive Interviewing
Pilot testing finds problems while they are still cheap to fix. Two kinds of testing are useful.
Cognitive interviewing asks a small number of people from the target population to think aloud while answering, or to explain afterwards how they understood each question. It shows whether respondents interpret terms the way the researchers intended. A word such as “procedure” may mean surgery to one patient and a blood test to another.
Field piloting runs the full instrument under real conditions to check:
- completion time and drop-out points
- whether any response option is missing (many “Other” answers are a warning sign)
- whether skip logic and mandatory fields work on every device
- whether interviewers follow the instructions consistently
- whether the exported data match the data dictionary
Document every finding and every change. A pilot is not a validation study: it tests feasibility, comprehension and administration. Formal validity and reliability testing needs its own design and sample.
Ethical and Data-Quality Considerations
Healthcare instruments often collect sensitive data about health conditions, treatment and personal circumstances. Design should address:
- informed consent, written in plain language
- privacy and confidentiality, including how responses are de-identified
- data minimization: collect only what the analysis plan needs
- careful wording and placement of sensitive questions
- secure collection, transfer and storage
- ethics committee or institutional review board approval where required
For digital surveys, test the instrument on every device and browser that respondents are likely to use, and confirm that validation rules, branching logic and data exports work. For interviews, define how missing, refused and “don’t know” answers will be recorded, because each one is analyzed differently.
Common Mistakes in Healthcare Questionnaire and Interview Design
- Writing questions before fixing the objective and primary outcome. The instrument collects a lot of data but cannot answer the main question.
- Designing questions without an analysis plan. The data cannot support the planned statistical tests, for example ordinal answers where a continuous outcome was needed.
- Building a new scale when a validated one exists. Results cannot be compared with other studies and must be validated from scratch.
- Translating without cross-cultural adaptation. Scores in different languages may not be equivalent.
- Reporting Cronbach’s alpha as proof of validity. Alpha is evidence about internal consistency only.
- Making the instrument too long. Respondent burden increases drop-out and careless answers.
- Skipping cognitive testing and piloting. Misunderstood items are found only after data collection, when it is too late.
- Inconsistent interviewing. Untrained interviewers probe differently and reduce comparability.
How Statswork Helps Healthcare Teams Build Research Instruments
The hardest part of instrument design is not writing questions. It is making sure every question produces data that the planned analysis can use, that the scales are valid and reliable for your population, and that reviewers, ethics committees and regulators will accept the evidence. Those decisions need statistical and psychometric expertise at the design stage, before any data are collected.
Statswork provides questionnaire design services for healthcare organizations, including hospitals, contract research organizations, pharmaceutical and medtech teams, and academic research groups. Our statisticians:
- translate study objectives into an item bank and a data dictionary linked to the statistical analysis plan
- select or adapt existing validated instruments and plan cross-cultural adaptation
- choose response scales and question formats that suit the planned analysis
- calculate sample sizes for estimation, comparison and psychometric validation
- analyze pilot data and deliver a validation report covering factor structure, internal consistency, test–retest reliability and recommended item revisions
If you are planning a patient survey, clinical study questionnaire or interview-based study, talk to the Statswork healthcare research consulting team before fieldwork begins, while design changes are still inexpensive.
Frequently Asked Question
What is questionnaire design in healthcare research?
Questionnaire design is the process of developing the questions, response options, instructions, order and skip logic of a research instrument so that it collects valid, reliable and analyzable data. It starts from the research objective and analysis plan and ends with pilot testing and, where needed, formal validation.
What is the difference between a questionnaire and an interview schedule?
A questionnaire is usually completed by the respondent. An interview schedule is read out and recorded by a trained interviewer, which makes it suitable for respondents with limited literacy and allows clarification within the study protocol. Questionnaires cost less per response and suit larger samples.
Should I develop a new questionnaire or use a validated one?
Use an existing validated instrument whenever one measures your construct in a similar population. It saves validation work and allows comparison with published studies. Develop a new instrument only when no suitable measure exists, and plan content validity, structural validity and reliability testing from the start.
How do I check the reliability of a healthcare questionnaire?
Assess internal consistency for multi-item scales (Cronbach’s alpha or McDonald’s omega), test–retest reliability for stable constructs (intraclass correlation coefficient), and inter-rater reliability when interviewers or coders classify answers (for example, Cohen’s kappa). Report confidence intervals, not just point estimates.
Why is pilot testing important for healthcare questionnaires?
Pilot testing shows unclear wording, missing response options, broken skip logic, excessive length and administration problems before the main data collection. Fixing these after fieldwork is usually impossible, because the affected data cannot be recollected.
References
- Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186–3191. https://doi.org/10.1097/00007632-200012150-00014
- Bolarinwa, O. A. (2015). Principles and methods of validity and reliability testing of questionnaires used in social and health science researches. Nigerian Postgraduate Medical Journal, 22(4), 195–201. https://doi.org/10.4103/1117-1936.173959
- Boyle, G. J. (1991). Does item homogeneity indicate internal consistency or item redundancy in psychometric scales? Personality and Individual Differences, 12(3), 291–294. https://doi.org/10.1016/0191-8869(91)90115-R
- Boynton, P. M., & Greenhalgh, T. (2004). Selecting, designing, and developing your questionnaire. BMJ, 328(7451), 1312–1315. https://doi.org/10.1136/bmj.328.7451.1312
- Hennink, M., & Kaiser, B. N. (2022). Sample sizes for saturation in qualitative research: A systematic review of empirical tests. Social Science & Medicine, 292, 114523. https://doi.org/10.1016/j.socscimed.2021.114523
- Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
- McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433. https://doi.org/10.1037/met0000144
- Prinsen, C. A. C., Mokkink, L. B., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., & Terwee, C. B. (2018). COSMIN guideline for systematic reviews of patient-reported outcome measures. Quality of Life Research, 27(5), 1147–1157. https://doi.org/10.1007/s11136-018-1798-3
- Rattray, J., & Jones, M. C. (2007). Essential elements of questionnaire design and development. Journal of Clinical Nursing, 16(2), 234–243. https://doi.org/10.1111/j.1365-2702.2006.01573.x
- Rosellini, A. J., & Brown, T. A. (2021). Developing and validating clinical questionnaires. Annual Review of Clinical Psychology, 17, 55–81. https://doi.org/10.1146/annurev-clinpsy-081219-115343
- S. Food and Drug Administration. (2009). Guidance for industry: Patient-reported outcome measures: Use in medical product development to support labeling claims. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-reported-outcome-measures-use-medical-product-development-support-labeling-claims
