Benefits & Limitations of Questionnaires
Benefits & Limitations of Questionnaires
Lessons from the United States and Britain, Policy Sciences Journal (1974 – out of
date)
Method strategy:
- Determine purpose of study/hypothesis to be investigated
- Appraisal of relevant literature
- Assess of suitable research methods
- Pilot study
- Define sample
- Collect and process data
- Analyse data
- Assemble results – relationship to hypothesis
- Write up findings, interpretation etc.
Pilot Work;
Pilot study to determine most suitable question sequence, question form and wording, style,
tone, colour of paper.
Test questions on guinea pigs (?) to avoid poor responses (unsuitable data inputs) due to
misunderstanding on part of the respondent.
Advantage of flexibility and generation of rapport between participants. Employ visual aids,
extend questions and clarify response. Limitation: Interviewer may infuse his bias on the
response, may lead the respondent or misinterpret his response.
Mailed Questionnaire:
Advantage of mailed questionnaire: Cheap, facility for a large sample and absence of
interviewer bias.
Limitations of mailed questionnaire: people with lower intelligence rarely respond in a
meaningful way
No single method of data collection is universally suitable: the nature and purpose of the
participatory exercise should significantly influence the decision.
Question Sequences:
Avoid beginning with abrupt questions which might generate a hostile response and set the
tone for the interview. Broad opening questioning, from which questions then become
increasing specific and detailed.
Open – free answer, advantageous in eliciting rich and fuller answers. Ideas expressed in
own language of respondent. Although easy to ask, difficult to answer presicely and
concisely, very difficult to analyse (unless a coding device is used)
Closed questions – likert scale, yes/no binary. Easy and quick to answer, require no
interpretation of speech, results are easy to tabulate and quantify. Main disadvantage:
leading of respondent into areas of thought that might not have occurred to him
spontaneously – what does this mean? Also, such questions might irritate the respondent if it
is felt the alternative choices fail to do justice to his own ideas. Mitigation: might be beneficial
to ask the same question in both open and closed form.
Interviewer will have to know the precise purpose of the questions to aboid ambiguity in the
wording. Should not be assumed that every respondent has the information sought.
Respondent may be reluctant to admit that, anxious , embarassed or wanting to please the
interviewer, may guess instead of asking for clarification.
Short questions, easy to understand, plain language. No leading questions (Such as?) or
loaded words or phrases
Reliability and Validity of Responses – ask Hiky which responses should I discard?
Limitations: Errors in the data input, high non-response rates, interviewer bias,
misunderstanding or reticence of respondent, errors in stats analysis and final interpretation.
Case studies of similar studies that were conducted and the limitations they faced in their
research method design