English for Academic and

Professional Purposes Quarter

2 – Module 5: Designing,
Testing, and Revising Survey
Questionnaires
 I. Introduction One of the ways to gather data
is through a questionnaire. However, as a
researcher, you must know the right questions
that you will use to get the informationyou
need. Designing a questionnaire is quite
challenging but fun. Do not give up if your first
attempt needs revision. Remember, practice
makes perfect. Keep pushing!
II. Objective: Designs, tests
and revises survey
questionnaires
 III. Vocabulary List: Before you
begin, here are some words that
you will encounter in the module:
⮚ Questionnaire – a written
document containing
questions and other types of
itemsdesigned to solicit
information appropriate to
analysis.
 ⮚ Survey Research - the collection
of information froma sampleof
individuals through their responses
to questions or statements
 ⮚ Respondent – a person who
provides data in a survey research.
 A questionnaire is a structured series of
questions designedtocollect primary data
from respondents. A well-
designedquestionnaire motivates
respondents to provide accurate and
complete information which is very helpful
in attaining thesurvey’sobjective.
(QuickMBA.com, n.d.)
DESIGNING A QUESTIONNAIRE Here are
the suggested steps on how to develop a
questionnaire: (Adapted from
QuickMBA.com)
 QUESTION WORDING BASIC
(Adapted from Filiberto, D. (2013)
with some notes from Baxter, L.
and Babbie, E. (2003))
 1. Write short and simple questions
Respondents are often unwilling to study
an item in order to understand it.
Assumethat respondents will answer the
questionnaire quickly. Therefore,
provide clear, short items that will not be
misinterpreted.
Example: Given the current trend of more
hits, more home runs, longer games
ingeneral, and more injuries in baseball
today, do you think that steroid use
should continue to be banned even though
it is not enforced?
*Problem: Long questions can be
confusing
 Better question: Steroid use has
both positive and negative effects
on baseball. Do you think that
steroid use should be banned?"
2. Avoid leading questions, wording
that influences respondents to
consider asubject in a weighted
manner, or injects a preference or
opinion. Example: Do you hate the
president of the Philippines?”
 Why is this leading? Because the question itself
includes an opinion word.
Who do you think of when you hear COVID -19?
a. China b. Pres. Duterte c. Chinese d. Lockdown
Why is this leading? Because it forces the
respondent to answer one of thesechoices, even if
none of them comes to mind.
What makes our product better than our
competitors’ products?
 3. Appropriately Open-Ended and Closed-
Ended Questions Use open-ended questions
when responses need to be elaborated by
therespondents for exhaustive and
comprehensive data gathering. They’re
moresuitedto exploratory research that
looks to describe a subject based on trends
and patterns.
Closed-Ended questions are popular because
they provide greater uniformity or
responses and are easily processed
compared to open-ended questions.
However, closed-ended questions the
response categories should be exhaustive
and mutuallyexclusive. In other words, all
possible options
Example: Why do you play sports?
1. Enjoyment 2. Health 3. Friends
4. Other-----
 4. Questions must be non-
threatening and attempt to evoke the
truth.
Example: Who do you think consume
more cigarettes: you or your
friends?
 When a respondent is concerned
about the consequences of
answering a question in a particular
manner, there is a good possibility
that the answer will not be truthful.
 5. Question Clarity
Avoid ambiguities and vague words
(e.g. usual, regular, normal) Example:
What is your number of serving of
eggs in a typical day?
Problem: How many eggs constitute a
serving? What does ‘a typical day’
mean?
 Better question: On days you eat eggs,
how many eggs do you usually
consume? Do you watch television
regularly? *Vague questions are
difficult to answer (what is the meaning
of "regularly"?) Better question: How
often do you watch Television?"
6. Don’t use double-barrelled questions
Ask one question at a time. Avoid asking
2 questions, imposing
unwarrantedassumptions, or hidden
contingencies. Whenever you use ‘and’
on a questionor astatement, check if it
is double-barrelled.
 Example: Do you find the classes you
took during your first semester in SHS
more demandingand interesting than
your JHS classes?
Yes No
 * How would someone respond if they felt their
SHS classes were more demandingbut also more
boring than their JHS classes? Or less
demanding but more interesting?Because the
question combines “demanding” and
“interesting,” there is no waytorespond yes to
one criterion but no to the other. Do you find the
classes you took during your first semester in
SHSmoredemanding than your JHS classes?
7. Clearly define the response scale
dimension or continuum.
When using a response scale, clearly
define the dimension or
continuumrespondents are to use in
their rating task
 Example: Response categories - Make them
logical and meaningful: NOT:
Many......Some.......A Few......Very Few.....None
DO a Bipolar or Unipolar rating scale:
Bipolar measures both direction and
intensityof an attitude: Unipolar scale
measures one concept with varying degrees
of intensity.
 8. Minimize presuppositions – an
assumption about the world whose truth
is takenfor granted. Answering a
question implies accepting its
presuppositions, a respondent may
beledto provide an answer even if its
presuppositions are false.
 Example:
Are you a DDS or a Dilawan? Problem:
presupposes that one of the alternatives is
true.
What are your usual hours of work?”
Problem: Does respondent have usual hours
of work?
Better Question: What are your usual hours
of work, or do you not have usual hours?
TYPE OF QUESTIONS AND ITS USAGE
 1. Open-Ended
Best Used for:
1.Breaking the ice in an interview
2.When respondent’s own words are
important
3.When the researcher does not know all
the possible answers
 Example: What changes do you
recommend for the school to do in
order to help students perform better?
 2. Closed-Ended
Best Used for:
1.Collecting rank ordered data
2.When all response choices are
known 3.When quantitative statistical
tool results are desired
Example: In which of the following do
you live?
A house
An apartment
A condo unit
Other forms closed-
ended questions:
 a. Likert-Scale
Best Used for: Assessing a
person’s opinion and feelings
about something
Example: Please circle the way you
feel about the following: 1 =
Disagree 5 = Agree
 b. Multiple Choice
Best Used for:
When there are finite number of options
Example:
Which of the following best describes your
current civil status? Single Married
Widowed Divorced
 c. Rating Scales
Best Used for:
Rate things in relation to other things
Example:
How likely would you recommend the current
strand you are enrolled in to your friend?
 d. Ranking Questions
Best Used for:
Ordering answer choices by way of
preference. This allows you to not
onlyunderstand how respondents feel about
each answer option, but it alsohelpsyou
understand each one’s relative popularity.
 Example:
Rank the following subjects in order of
preference – 1 being your favorite and 5
beingyour least favorite. English Social Sciences
Math Music, Arts, PE and HealthScience TLE
Filipino Edukasyon sa Pagpapakatao
ORDERING THE QUESTIONS (Adapted
from Contemporary Communication
Research by Smith,
 1. Adapt a general organizational pattern that
complements a survey’s research objectives.
Two general patterns:
o Funnel pattern – begins with broad questions
followed by progressively narrower or more
specific ones
o Inverted pattern – narrowly focused
questions are followed by more general
ones.
 2. Topically related questions should be
grouped together.
A researcher should group together
questions pertinent to a single
topicthenmove to another topic. It is easier
for the answer questions this way.
3. Easy-to-answer questions should be
placed first. Easy questions serve as
motivation.
 4. Questions should be ordered to avoid
establishing a response bias. *Response Bias – a
tendency of a respondent to answer all closed-
questions thesame way regardless of content.
Example: A respondent check “Somewhat agree”
TESTING AND REVISING THE
QUESTIONNAIRE
 Some Practical Tips on Testing a
Questionnaire: (Adapted from
tools4dev.org)
1. Find 5 to 10 people from your target
group
 3. Observe how they complete the survey.
*note their hesitations or where they made
mistakes in answering. This is anindication that
the survey questions and layout are not clear
enough and needsimprovement.
4. Make improvements based on the results.
Quick tips before you finish the module:
✔ All questionnaires need an introduction. Be
sure to have one.
✔ It is useful to begin every questionnaire with
basic instructions for completing it.
 ✔ The format of a questionnaire is as
important as the wording and ordering.
Besure that it is spread out and
uncluttered.
✔ Physical aspects such as page
layout, font type and size, questions
spacing, and the type of paper should be
considered. Always check on these.
 Questionnaire: Definition, Examples, Design and
Types By Saul McLeod, updated 2018
A questionnaire is a research instrument
consisting of a series of questions for the purpose
of gathering information from respondents.
Questionnaires can be thought of as a kind of
written interview. They can be carried out face to
face, by telephone, computer or post.
 Questionnaires provide a relatively
cheap, quick and efficient way of
obtaining large amounts of
information from a large sample of
people. Data can be collected relatively
quickly because the researcher would
not need to be present when the
questionnaires were completed.
 This is useful for large
populations when interviews
would be impractical. However,
a problem with questionnaires
is that respondents
may lie due to social desirability.
Most people want to present a
positive image of themselves and
so may lie or bend the truth to
look good, e.g., learners would
exaggerate revision duration.
Questionnaires can be an effective
means of measuring the behavior,
attitudes, preferences, opinions
and intentions of relatively large
numbers of subjects more cheaply
and quickly than other methods.
An important distinction is between
openended and closed questions.
Often a questionnaire uses both
open and closed questions to collect
data. This is beneficial as it means
both quantitative and qualitative
data can be obtained.
 Closed Questions
Closed questions structure the answer by only
allowing responses which fit into pre-decided
categories. Data that can be placed into a category is
called nominal data. The category can be restricted
to as few as two options, i.e., dichotomous (e.g. “yes”
or “no” “male” or “female”) or include quite complex
lists of alternatives from which the respondent can
choose (e.g., polytomous).
 Closed questions can also provide ordinal
data (which can be ranked). This often
involves using a continuous rating scale to
measure the strength of attitudes or
emotions. For example, strongly agree /
agree / neutral / disagree / strongly
disagree / unable to answer.
 Closed questions have been used to
research type A personality (e.g., Friedman
& Rosenman, 1974), and also to assess life
events which may cause stress (Holmes &
Rahe, 1967), and attachment (Fraley,
Waller, & Brennan, 2000).
 Strengths
• They can be economical. This means they
can provide large amounts of research data
for relatively low costs. Therefore, a large
sample size can be obtained which should be
representative of the population, which a
researcher can then generalize from.
 • The respondent provides information
which can be easily converted into
quantitative data (e.g., count the number
of 'yes' or 'no' answers), allowing
statistical analysis of the responses.
• The questions are standardized. All
respondents are asked exactly the same
questions in the same order. This means a
questionnaire can be replicated easily to
check for reliability. Therefore, a second
researcher can use the questionnaire to check
that the results are consistent.
 Limitations
• They lack detail. Because the
responses are fixed, there is less scope
for respondents to supply answers
which reflect their true feelings on a
topic.
 Open Questions
Open questions allow people to express what they
think in their own words. Openended questions
enable the respondent to answer in as much detail
as they like in their own words. For example: “Can
you tell me how happy you feel right now?” If you
want to gather more in-depth answers from your
respondents, then open questions will work better
 These give no pre-set answer options and
instead allow the respondents to put down
exactly what they like in their own words.
Open questions are often used for complex
questions that cannot be answered in a few
simple categories but require more detail
and discussion.
 Strengths
• Rich qualitative data is obtained
as open questions allow the
respondent to elaborate on their
answer. This means the research
can find out why a person holds a
certain attitude.
 Limitations
• Time-consuming to collect the data. It
takes longer for the respondent to
complete open questions. This is a
problem as a smaller sample size may be
obtained.
 • Time-consuming to analyze the data. It takes
longer for the researcher to analyze qualitative
data as they have to read the answers and try to
put them into categories by coding, which is often
subjective and difficult. However, Smith (1992) has
devoted an entire book to the issues of thematic
content analysis the includes 14 different scoring
systems for open-ended questions.
 • Not suitable for less educated
respondents as open questions
require superior writing skills and a
better ability to express one's feelings
verbally.
 Questionnaire Design
With some questionnaires suffering from a response rate as low as 5%, it is
essential that a questionnaire is well designed. There are a number of important
factors in questionnaire design.
The researcher should ensure that the answer to a question is
not influenced by previous questions.
Terminology
There should be a minimum of technical jargon. Questions
should be simple, to the point and easy to understand. The
language of a questionnaire should be appropriate to the
vocabulary of the group of people being studied. Use
statements which are interpreted in the same way by
members of different subpopulations of the population of
interest. For example, the researcher must change the
language of questions to match the social background of
respondents' age / educational level / social class / ethnicity
etc.
 Presentation
Make sure it looks professional, include
clear and concise instructions. If sent
through the post make sure the
envelope does not signify ‘junk mail.’
 Ethical Issues
The researcher must ensure that the information provided
by the respondent is kept confidential, e.g., name, address,
etc. This means questionnaires are good for researching
sensitive topics as respondents will be more honest when
they cannot be identified. Keeping the questionnaire
confidential should also reduce the likelihood of any
psychological harm, such as embarrassment. Participants
must provide informed consent prior to completing the
questionnaire, and must be aware that they have the right
to withdraw their information at any time during the
 Problems with Postal Questionnaires
The data might not be valid (i.e., truthful) as we
can never be sure that the right person actually
completed the postal questionnaire. Also, postal
questionnaires may not be representative of the
population they are studying?
• This is because some questionnaires may
be lost in the post reducing the sample size.
• The questionnaire may be completed by
someone who is not a member of the
research population.
• Those with strong views on the
questionnaire’s subject are more likely to
complete it than those with no interest in it.
 Benefits of a Pilot Study
A pilot study is a practice / small-scale
study conducted before the main study. It
allows the researcher to try out the study
with a few participants so that adjustments
can be made before the main study, so
saving time and money. It is important to
conduct a questionnaire pilot study for the
following reasons:
• Check that respondents understand the
terminology used in the questionnaire.
• Check that emotive questions have not
been used as they make people defensive
and could invalidate their answers.
• Check that leading questions have not
been used as they could bias the
respondent's answer.
 DESIGNING A SURVEY QUESTIONNAIRE
Questionnaire research design proceeds in
an orderly and specific manner. Each item
in the flow chart depends upon the
successful completion of all the previous
items. Therefore, it is important not to skip a
single step. Notice that there are two
feedback loops in the flow chart to allow
revisions to the process and questionnaire.
 Establishing Goals. The first step in any survey
is deciding what you want to learn. The goals of
the project determine whom you will survey and
what you will ask them. If your goals are
unclear, the results will probably be unclear.
Selecting Your Sample. There are two main
components in determining whom you will
interview. The first is deciding what kind of
people to interview. Researchers often call this
group the target population.
Selecting Your Sample. There are two
main components in determining whom
you will interview. The first is deciding
what kind of people to interview.
Researchers often call this group the
target population. If you are trying to
determine the likely success of a
product, the target population may be
less obvious.
Correctly determining the target population is
critical. If you do not interview the right
kinds of people, you will not successfully meet
your goals. The next thing to decide is how
many people you need to interview.
Statisticians know that a small,
representative sample will reflect the group
from which it is drawn. The larger the
sample, the more precisely it reflects the
target group.
Avoiding a Biased Sample. A biased
sample will produce biased results.
Totally excluding all bias is almost
impossible; however, if you
recognize bias exists, you can
intuitively discount some of the
answers Interviewing Methods.
Once you have decided on your
sample you must decide on your
method of data collection. Each
method has advantages and
disadvantages.
 Personal Interviews. An interview is called
personal when the Interviewer asks the
questions face-to-face with the Interviewee.
Personal interviews can take place in the
home, at a shopping mall, on the street,
outside a movie theatre or polling place, and
so on
 Telephone Surveys. Surveying by
telephone is the most popular
interviewing method in the USA. This is
made possible by nearly universal
coverage (96% of homes have a
telephone).
Computer Direct Interviews. These
are interviews in which the
Interviewees enter their own
answers directly into a computer
Email Surveys.
 Email surveys are both very
economical and very fast. More
people have email than have full
Internet access. This makes email a
better choice than a Web page survey
for some populations. Email surveys
are limited to simple questionnaires.
 Internet/Intranet (Web Page) Surveys.
Web surveys are rapidly gaining
popularity. They have major speed, cost,
and flexibility advantages, but also
significant sampling limitations. These
limitations make software selection
especially important and restrict the
groups you can study using this
technique.
 Questionnaire Design General Considerations
1. The first rule is to design the questionnaire
to fit the medium. People responding to mail or
Web surveys cannot easily ask “What exactly
do you mean by that?” if they do not
understand a question. Personal questions are
sometimes best handled by mail or computer,
where anonymity is most assured.
 2. KISS - keep it short and simple.

3. Start with an introduction or welcome

message. A good introduction or
welcome message will encourage people
to complete your questionnaire. When
practical, state who you are and why you
want the information in the survey.
 4. Allow a “Don't Know” or “Not
Applicable” response to all questions,
except to those in which you are certain
that all respondents will have a clear
answer
 Researchers use three basic types of
questions: multiple choice, numeric open
end and text open end. Examples of each
kind of question follow:
 Rating Scales and Agreement Scales are
two common types of questions that some
researchers treat as multiple choice
questions and others treat as numeric
open end questions. Examples of these
kinds of questions are:
 Question and Answer Choice Order
There are two broad issues to keep in mind
when considering question and answer
choice order. One is how the question and
answer choice order can encourage people
to complete your survey. The other issue is
how the order of questions or the order of
answer choices could affect the results of
your survey.
 Ideally, the early questions in a
survey should be easy and pleasant
to answer. These kinds of questions
encourage people to continue the
survey. Grouping together
questions on the same topic also
makes the questionnaire easier to
answer.
Whenever possible leave difficult
or sensitive questions until near
the end of your survey. If people
quit at that point anyway, at
least they will have answered
most of your questions.
 Other General Tips Keep the
questionnaire as short as possible.
More people will complete a
shorter questionnaire, regardless
of the interviewing method. If a
question is not necessary, do not
include it.
Start with a Title (e.g., Leisure Activities
Survey). Always include a short
introduction - who you are and why you
are doing the survey.
Reassure your respondent that his or
her responses will not be revealed to
your client, but only combined with
many others to learn about overall
attitudes.
Include a cover letter with all mail surveys. The
most effective cover letters and invitations
include the following elements: Ask the
recipient to take the survey. Explain why
taking it will improve some aspect of the
recipient's life (it will help improve a product,
make an organization better meet their needs,
make their opinions heard). Appeal to the
recipient's sense of altruism ("please help").
Ask the recipient again to take the survey.
 The overriding consideration in
questionnaire design is to make sure
your questions can accurately tell you
what you want to learn. The way you
phrase a question can change the
answers you get. Try to make sure the
wording does not favor one answer
choice over another.
 Pre-test the Questionnaire
The last step in questionnaire design is to test
a questionnaire with a small number of
interviews before conducting your main
interviews. Ideally, you should test the survey
on the same kinds of people you will include
in the main study. If that is not possible, at
least have a few people, other than the
question writer, try the questionnaire.
 This kind of test run can reveal
unanticipated problems with
question wording, instructions to
skip questions, etc. It can help you
see if the interviewees understand
your questions and give useful
answers.
 If you change any questions after a
pre-test, you should not combine the
results from the pre-test with the
results of post-test interviews.
Choosing sensible questions and
administering surveys with sensitivity
and common sense will improve the
quality of your results dramatically.
 5 COMMON SURVEY QUESTION
MISTAKES THAT WILL RUIN YOUR
DATA
 2. Avoid loaded questions
Loaded questions are questions written in
a way that forces the respondent into an
answer that doesn’t accurately reflect his
or her opinion or situation. This key survey
mistake will throw off your survey
respondents and is one of the leading
contributors to respondents abandoning
surveys.
 Bad Question: Where do you enjoy
drinking beer?
By answering this question, the
respondent is announcing that they
drink beer. However, many people
dislike beer or will not drink alcohol
and therefore can’t answer the
question truthfully.
Usually, loaded questions are best avoided by
pretesting your survey to make sure every
respondent has a way to answer honestly. In
the case of the example above, you may
choose to ask a preliminary question on
whether the respondent drinks beer and use
skip logic to let people who don’t drink beer
pass over the questions that don’t apply to
them.
 Bad Question: How useful will this textbook
be for students and young professionals in
the field?
Now the respondent is forced to give a single
answer for both parties. Instead break the
question into two; one measuring usefulness
for students and one measuring usefulness
for professionals.
 Good Questions: How useful
will this textbook be for
students? How useful will this
textbook be for young
professionals in the field?
 4. Absolutely do not use absolutes in
questions
Absolutes in questions force
respondents into a corner where they
can’t give useful feedback. These
questions usually have the options
Yes/No and include wording such as
“always,” “all,” “every,” “ever,” etc.
 Bad Question: Do you always eat
breakfast? (Yes/No) Read literally, the
example above would force almost any
respondent to answer “No.” Even then,
there would be some respondents who
would interpret the question as asking
whether they always eat a full
breakfast when they have a chance.
 The inflexibility of absolutes makes
questions too rigid to be used in a survey.
Instead, the question should have a variety
of options that people will feel more
comfortable choosing from. Good
Question: How many days a week do you
usually eat breakfast? (Every day/ 5-6
days/ 3-4 days/ 1-2 days/ I usually don’t
eat breakfast)
 5. Be clear by speaking your respondent’s
language
Regardless of who’s taking your survey, use clear,
concise, and uncomplicated language while trying
to avoid acronyms, technical terms or jargon that
may confuse your respondents. And make sure to
provide definitions or examples if you need to
include tricky terms or concepts. That way, you
can be certain that almost anybody can answer
your questions easily, and that they’ll be more
inclined to complete your survey.
Bad Question: Do you own a tablet PC?
Good Question: Do you own a tablet
PC? (e.g. iPad, Android tablet)
Bad Question: What was the state of
the cleanliness of the room?
Good Question: How clean was the
room?
 Generally, you should strive to write questions
using language that is easily understood. Certain
sample groups, however, may have a knowledge
base that can make the use of more difficult
terms and ideas a viable option.
Ask yourself if your respondents have a deep
understanding of certain events, terms, and
issues dealt with in the survey. The more you can
focus on writing good questions, as opposed to
explaining things in common terms, the better.
For example, if you are surveying patients
in a hospital, you’ll want to avoid using
medical jargon. However, if your survey
sample is made up of doctors, it makes
sense to ask more specialized questions
and use higher level medical vocabulary.
 By avoiding these five-common survey-
writing mistakes, your survey should run
like a well-oiled machine, your data will be
more accurate, and your respondents will
exit your survey feeling great because
they’ve shared honest and accurate
feedback. Triple win! So put your writing
cap on and get to creating those
questions.
THANK YOU
Post-Test
1. A questionnaire is defined by the following except:
a. to gather data from respondents
b. translates the research problem into questions that
will answered by respondents
c. wording, appearance and flow of questions fosters
cooperation and motivatethe respondents to answer
d. a tool designed to control data for a survey
2. A well-designed questionnaire needs to be as
short as possible. a. True b. False
3. What is the first step in designing a survey
questionnaire?
a. Pre-testing
b. Identifying the respondents
c. Identifying the research problem
d. Identifying the type of questions to be used
 4. Which of the following is considered a good
questionnaire item?
a. How long does it take you to walk and run around
the court?
b. If you were the President, what will you do to
combat the COVIDPandemic?
c. How much did you enjoy the reading the new
novel? d. Rate your experience in the service
provided.
 5. Study the questionnaire item below and choose the
correct statement that describes the question.
Do you go to the park when you are stressed from work?
a. The question assumes that the respondent gets stressed
at work.
b. It is a double-barreled question since work and the park
are two separateplaces.
c. It is an open-ended question and will be hard to quantify.
d. There is nothing wrong with the question
 6. If you want to know how the students feel
about the new rules and regulationsset by your
organization, what type of questions will you
most likely use?
a. Likert-Scale
b. Multiple Choice
c. Ranking
d. Open-ended
 7. “How much is your ‘baon’ every day?” If you
were to revise this question, how should it be?

a. Do your parents give you ‘baon’? Yes or No

b. I will provide options so that the respondent will

not feel embarrassedExample: Our parents does
not give us money as ‘baon’. Php 1.00 – 49.99 Php
50.00 – 99.99 Php 100.00 - above
 c. I will not include this question in my
questionnaire because it is not
necessary.
d. There is no need to revise this question
because it will give the informationI need.
 8. Participants in a pre-test should be:
a. Representative of the target population
under study
b. Friends and relatives
c. Other survey researchers
d. Individuals from outside the population
under study
9. In testing your questionnaire, why is it necessary
to observe where the respondents changed their
mind in answering?
a. It will show what items needs revision.
b. It will reveal some items that might be confusing
so they hesitated or changetheir minds.
c. It will help the researcher understand why the
items might be confusing.
d. All of the above
 10. Now that you have all the questions
ready, reviewed, and sequenced, whichof
the following do you still need to consider?
a. the type of paper where it will be printed
b. writing an introduction and instructions
c. the format of the question items
d. all of the above