Quantitative Research Approach Reviewer
Quantitative Research Approach Reviewer
• The basic intent of an experimental design is to The data collection may also involve creating a Web-
test the impact of a treatment (or an intervention) based or Internet survey and administering it online
on an outcome, controlling for all other factors (Nesbary, 2000; Sue & Ritter, 2007). Regardless of
that might influence that outcome the form of data collection, provide a rationale for the
• As one form of control, researchers randomly procedure, using arguments based on its strengths
assign individuals to groups. and weaknesses, costs, data availability, and
convenience.
• When one group receives a treatment and the other
group does not, the experimenter can isolate 3. THE POPULATION AND THE SAMPLE
whether it is the treatment and not other factors that Specify the characteristics of the population and the.
influence the outcome. sampling procedure.
2. COMPONENTS OF A SURVEY METHOD PLAN Here are essential aspects of the population and
Methodology sample to describe in a research plan:
Identify the population in the study.
The Survey Design - state the size of this population, if size can be
determined, and the means of identifying individuals
- In a proposal or plan, one of the first parts of the in the population.
method section can introduce readers to the basic - Identify whether the sampling design for this
purpose and rationale for survey research. population is single stage or multistage (called
- Begin the discussion by reviewing the purpose of a clustering).
survey and the rationale for its selection for the
proposed study. Cluster sampling is ideal when it is impossible or
impractical to compile a list of the elements
composing the population (Babbie, 2007).
- Single-stage Cluster Sampling
As the name suggests, sampling is done just once.
An example of single-stage cluster sampling - An
NGO wants to create a sample of girls across five
neighboring towns to provide education. Using
single-stage sampling, the NGO randomly selects
towns (clusters) to form a sample and extend help to
the girls deprived of education in those towns.
Double-stage sampling
- Instead of selecting all the elements of a cluster, only a handful of members are chosen from each group by
implementing systematic or simple random sampling.
- An example of two-stage cluster sampling - A business owner wants to explore the performance of his/her
plants that are spread across various parts of the U.S. The owner creates clusters of plants. So, he/she then
selects random samples from these clusters to conduct research.
Here are essential aspects of the population and - In addition, instruments are being increasingly
sample to describe in a research plan: designed for online surveys.
Identify the selection process for individuals To use an existing instrument, describe the
- selecting a random sample, in which each established validity and reliability of scores
individual in the population has an equal probability obtained from past use of the instrument.
of being selected (a systematic or probabilistic - This means reporting efforts by authors to establish
sample) validity-whether one can draw meaningful and useful
- Less desirable is a nonprobability sample (or inferences from scores on the instruments.
convenience sample), in which respondents are
chosen based on their convenience and Name the three forms of traditional forms of
availability (Babbie, 1990). validity
- With randomization, a representative sample Content validity- (do the items measure the content
from a population provides the ability to they were intended to measure?), Predictive or
generalize to a population. concurrent validity- (do scores predict a criterion
measure? Do results correlate with other results?)
Identify whether the study will involve Construct validity- (do items measure hypothetical
stratification of the population before selecting constructs or concepts?)
the sample.
Stratification means that specific characteristics of - In more recent studies, construct validity has also
individuals (e.g., both females and males) are included whether the scores serve a useful purpose
represented in the sample and the sample reflects and have positive consequences when they are used
the true proportion in the population of individuals in practice (Humbley & Zumbo, 1996).
with certain characteristics (Fowler, 2002).
- Stratification ensures their representation - Establishing the validity of the scores in a survey
Also identify the characteristics used in stratifying the helps to identify whether an instrument might be a
population (e.g., gender, education). good one to use in survey research.
- Within each stratum, identify whether the sample
contain individuals with the characteristic in the - whether scores resulting from past use of the
same proportion as the characteristic appears in instrument demonstrate reliability
the entire population
- Look for whether authors report measures of
Discuss the procedures for selecting the sample internal consistency (are the items' responses
from available lists. consistent across constructs?)
- The most rigorous method for selecting the sample - Test-retest correlations (are scores stable over
is to choose individuals using a random numbers time when the instrument is administered a second
table, a table available in many introductory statistics time?).
texts. - Also determine whether there was consistency in
test administration and scoring (were errors caused
Indicate the number of people in the sample and by carelessness in administration or scoring?
the procedures used to compute this number.
When one modifies an instrument or combines
4. INSTRUMENTATION instruments in a study, the original validity and
As part of rigorous data collection, the proposal reliability may not hold for the new instrument,
developer also provides detailed information about and it becomes important to reestablish validity
the actual survey instrument to be used in the and reliability during data analysis.
proposed study. Consider the following:
Include sample items from the instrument so that
Name the survey instrument readers can see the actual items used. In an
- Discuss whether it is an instrument designed for appendix to the proposal, attach sample items or
this research, a modified instrument, or an intact the entire instrument.
instrument developed by someone else.
- modified instrument, indicate whether the developer
has provided appropriate permission to use it.
- permission to use any part of other instruments
needs to be obtained.
- Indicate the major content sections in the Discuss plans for pilot testing or field testing the
instrument, such as the cover letter (Dillman, 1978, survey and provide a rationale for these plans
provides a useful list of items to include in cover - This testing is important to establish the content
letters), the items (e.g., demographics, attitudinal validity of an instrument and to improve questions,
items, behavioral items, factual items), and the format, and scales.
closing instructions. - Indicate the number of people who will test the
• Also mention the type of scales used to instrument and the plans to incorporate their
measure the items on the instrument, such as comments into final instrument revisions.
continuous scales (e.g., strongly agree to strongly
disagree) and categorical scales (e.g., yes/no, rank END
from highest to lowest importance).