Introduction to Survey Research

RENE N. ARGENAL
Strategic Research & Development, Inc.
What is a Survey Research?
 Survey Research is About Asking
Questions About…

 Behaviors
 Opinions/Attitudes
 Facts
 Beliefs
 There are lots of ways to ask these questions
 Questionnaire
 Telephones
 Interviews
 Mail
 Internet
 Characteristics of Surveys

 Information is collected from a group of

people in order to describe some aspects or
characteristics – abilities, opinion, attitude,
beliefs, knowledge, of the population of
which that group is a part
 Characteristics of Surveys

 The main way in which the information is

collected is through asking questions; the
answers to these questions by the members
of the group constitute the data of the study
 Information is collected from a sample rather
than from every member of the population
 Why Are Surveys Conducted?

 Interested in Individuals

 Interested in Aggregation of Views

 Interested in Recent & Current Events

 Interested in Big Groups/Big Data

 Surveys may be used for descriptive, explanatory,
and exploratory purposes. They are typically used
in studies that have the individual as the unit of
analysis.

 Although the method may be used for other units

of analysis, some individual persons must serve as
respondents or informants.
 Survey research is probably the best method available to
the social researcher who is interested in collecting
original data for describing a population too large to
observe directly.
 Careful probability sampling provides a group of
respondents whose characteristics may taken to reflect
those of the larger population
 Carefully constructed standardized questionnaires provide
data in the same form from all respondents
 Six Steps in Survey Research

 Research Design
 Select Sample
 Construct Questions
 Ask Questions
 Analyze Data
 Report Results
 Types of Surveys
A. Cross-Sectional Survey
- collects information from a sample that
been drawn from a predetermined
population. The info is collected at just
one point in time, although the time it
takes to collect all of the data may take
anywhere from a day to a few weeks or
more
 Types of Surveys
B. Longitudinal Survey
- collects information at different points
in time in order to study changes over time
3 Types
1. Trend Study – different samples from a
population whose members may change are
surveyed at different points in time.
 Types of Surveys
2. Cohort Study –samples a particular
population whose members do not change
over the course of the study

3.Panel Study – surveys the same sample of

individuals at different times during the course
of the survey
STOP
Selecting and Constructing Data
Collection Instruments
 Data Collection Strategies

 No one best way: decision depends on:

 What you need to know: numbers or stories
 Where the data reside: environment, files, people
 Resources and time available
 Complexity of the data to be collected
 Frequency of data collection
 Intended forms of data analysis
 Rules for Collecting Data

 Use multiple data collection methods

 Use available data, but need to know
 how the measures were defined
 how the data were collected and cleaned
 the extent of missing data
 how accuracy of the data was ensured
 Structured Approach

 All data collected in the same way

 Especially important for multi-site and
cluster evaluations so you can compare
 Important when you need to make
comparisons with alternate interventions
 Use Structured Approach
When:
 need to address extent questions
 have a large sample or population
 know what needs to be measured
 need to show results numerically
 need to make comparisons across different
sites or interventions
 Use Semi-structured Approach
when:
 conducting exploratory work
 seeking understanding, themes, and/or issues
 need narratives or stories
 want in-depth, rich, “backstage” information
 seek to understand results of data that are
unexpected
 Characteristics of Good
Measures
 Is the measure relevant?
 Is the measure credible?
 Is the measure valid?
 Is the measure reliable?
 Relevance

Does the measure Do not measure

capture what what is easy instead
matters? of what is needed
 Credibility

 Is the measure believable? Will it be viewed

as a reasonable and appropriate way to
capture the information sought?
 Internal Validity

How well does the Are waiting lists

measure capture a valid measure
what it is supposed to? of demand?
 Reliability

A measure’s precision How reliable are:

and stability- extent to -birth weights of
which the same result newborn infants?
Would be obtained
with repeated trials - speeds measured
by a stopwatch?
 Quantitative Approach

 Data in numerical form

 Data that can be precisely measured
 age, cost, length, height, area, volume, weight,
speed, time, and temperature
 Harder to develop
 Easier to analyze
 Qualitative Approach
 Data that deal with description
 Data that can be observed or self-reported, but not
always precisely measured
 Less structured, easier to develop
 Can provide “rich data” — detailed and widely
applicable
 Is challenging to analyze
 Is labor intensive to collect
 Usually generates longer reports
 Which Data?

If you: Then Use:

want to conduct statistical analysis
- want to be precise
- know what you want to measure Quantitative
- want to cover a large group
-want narrative or in-depth information
-are not sure what you are able to measure Qualitative
-do not need to quantify the results
Obtrusive vs. Unobtrusive Methods

Obtrusive Unobtrusive
data collection methods that data collection
directly obtain information from methods that do not
those being evaluated
collect information
e.g. interviews, surveys, directly from those
focus groups being evaluated
e.g., document analysis,
GoogleEarth, observation
at a distance, trash of the
stars
 How to Decide on Data
Collection Approach
 Choice depends on the situation
 Each technique is more appropriate in some
situations than others
 Caution: All techniques are subject to bias
 Triangulation to Increase
Accuracy of Data
 Triangulation of methods
 collection of same information using different
methods
 Triangulation of sources
 collection of same information from a variety of
sources
 Triangulation of evaluators
 collection of same information from more than
one evaluator
 Data Collection Tools
 Participatory Methods
 Records and Secondary Data
 Observation
 Surveys and Interviews
 Focus Groups
 Diaries, Journals, Self-reported Checklists
 Expert Judgment
 Delphi Technique
 Other Tools
 Tool 1: Participatory Methods

 Involve groups or communities heavily in

data collection
 Examples:
 community meetings
 mapping
 transect walks
 Community Meetings

 One of the most common participatory

methods
 Must be well organized
 agree on purpose
 establish ground rules
 who will speak
 time allotted for speakers

 format for questions and answers

SAMPLING
 Reasons for Drawing a Sample

 Less time consuming than a census

 Less costly to administer than a census
 Less cumbersome and more practical to
administer than a census of the targeted
population
 Types of Sampling Methods
Samples

Non-Probability Probability Samples

Samples
Simple
Random Stratified
Convenience Purposive
Cluster
Quota Snowball Systematic
Multistage
 Nonprobability Samples

 Convenience Sampling – respondents are

chosen at the convenience of the researcher. Is
also referred to as accidental sampling
 Purposive Sampling - the researcher uses his
or her own judgment about which respondents
to choose, and picks only those who best meet
the purposes of the study.

 Nonprobability Samples

 Quota Sampling
is the non-probability equivalent of
stratified sampling, with added requirement
that each stratum is generally represented in the
sample in the same proportion as in the entire
population.
 Nonprobability Samples

 Snowball Sampling
Stems from the analogy of a snowball
which begins small and becomes bigger &
bigger as it rolls down the hill
 Simple Random Samples

 Every individual or item from the frame has

an equal chance of being selected
 Samples obtained from table of random
numbers or computer random number
generators
 Systematic Samples

 Decide on sample size: n

 Divide frame of N individuals into groups of k
individuals: k=N/n
 Randomly select one individual from the 1 st
group
 Select every k-th individual thereafter
 Stratified Samples

 Population divided into two or more groups

according to some common characteristic
 Simple random sample selected from each
group
 The two or more samples are combined into
one
 Cluster Samples

 Population divided into several “clusters,” each

representative of the population
 Simple random sample selected from each
 The samples are combined into one
 Multistage Sampling
 A modification of the cluster sampling procedure
 After a cluster is sampled, random sampling of
population elements belonging to the cluster is done.
 It may be two-stage, or three-stage, or several stages
of sampling.
 It is also less costly than cluster sampling. This is
done when a frame is not available. Listing will be
done, however, on the sampled clusters.
 Advantages and Disadvantages
 Simple random sample and systematic sample
 Simple to use
 May not be a good representation of the population’s
underlying characteristics
 Stratified sample
 Ensures representation of individuals across the entire
population
 Cluster sample
 More cost effective
 Less efficient (need larger sample to acquire the same
level of precision
 Determination of Sample Size

 Approaches
1. To Estimate Average
2. To Estimate Proportion
3. To Estimate Proportion (Finite Population)
4. Slovin’s Formula (assumes at least 15%
variance)
Refer to attachments for examples
 Response Rate
 Response Rate is the percentage of surveys that are
returned.
 Overall response rate is one guide to the representativeness
of the sample respondents.

 A high response rate indicates there is a less chance of

significant response bias than a low response rate.

 A low response rate is a danger signal, because the non

respondents are likely to differ from the respondents in
ways other than just their willingness to participate in the
survey.
 Acceptable Response Rates

 According to Babbie,
 a response rate of 50% is adequate for analysis
and reporting
 a response rate of 60% is good
 a response rate of 70% is very good
 Strengths of Survey Research

 Only Way to Probe Opinion of a Very Large Group

 A Very Flexible Tool:

 Medical, Educational, Political, Business, etc.

 Facilitates Hypothesis Testing

 Encourages Standardization and Systematic

Comparison
 Weaknesses of Survey Research

 Standardization Comes at a Price

 Surveys Often Don’t Get at “Why”
 Respondents Might Not Have An Opinion
 Respondents May Not Actively Engage
 Respondents May Lie
 Design of Survey Research
 Choose an appropriate mode of response
 Reliable primary modes

 Personal interview

 Telephone interview

 Mail survey

 Less reliable self-selection modes (not appropriate for

making inferences about the population)

 Television survey

 Internet survey

 Printed survey on newspapers and magazines

 Product or service questionnaires

 Design of Survey Research

 Identify broad categories

 List complete and non-overlapping categories that

reflect the theme

 Formulate accurate questions
 Make questions clear and unambiguous. Use

universally-accepted definitions
 Test the survey
 Pilot test the survey on a small group of

participants to assess clarity and length

 Design of Survey Research

 Write a cover letter

 State the goal and purpose of the survey
 Explain the importance of a response
 Provide assurance of respondent’s anonymity
 Offer incentive gift for respondent participation
 Evaluating Survey Worthiness

 What is the purpose of the survey?

 Is the survey based on a probability sample?
 Coverage error – appropriate frame
 Nonresponse error – follow up
 Measurement error – good questions elicit good
responses
 Sampling error – always exists
 Types of Survey Errors

 Coverage error

 Non response error

 Sampling error

 Measurement error