Introduction to

Research Designs
 Assumptions in Scientific
Research
• Nature is orderly and regular
• To some extent, events are consistent and
predictable
• Events or conditions have one or more causes that
can be discovered
• This enables establishing cause and effect relationships
 The Scientific Method

The scientific method has been defined as a systematic,

empirical, controlled and critical examination of hypothetical
propositions about the association among natural phenomena.1,2

1. Kerlinger FN. Foundations of Behavioral Research. New York: Holt, Rinehart & Winston,
1973.
2. Portney LG, Watkins MP. 2000. Foundations of Clinical Research: Applications to Practice.
2nd Ed. Upper Saddle River, NJ: Prentice Hall Health.
 Properties of scientific method
Systematic
• Use of orderly procedures to ensure reliability
• Logical sequence is used from problem identification, through data collection, analysis, &
interpretation
Empirical
• Documentation of objective data through direct observation (or other systematic methods)
• Findings are grounded in the objective observation of phenomena rather than the personal
bias or subjective belief of the researcher
Control
• In order to understand how one phenomenon relates to another, factors are controlled that
are not directly related to the variables in question
• Investigators have confidence in their research outcomes to the extent that they control
extraneous influences
 Limitations

• Science is imperfect, especially when it is applied

to human behavior and performance
 Limitations

• Science is imperfect, especially when it is applied

to human behavior and performance
• Sources of uncertainty:
• Complexity and variability within nature
• The unique psychosocial and physiological capacities of
individuals
 Limitations

• Science is imperfect, especially when it is applied to

human behavior and performance
• Sources of uncertainty:
• Complexity and variability within nature
• The unique psychosocial and physiological capacities of
individuals
• Social science researchers must be acutely aware of
extraneous influences in order to interpret findings in a
meaningful way
 Types of Research

• Descriptive
• Case study
• Cross-sectional study
• Qualitative study

• Exploratory
• Cohort study
• Case control study

• Experimental
• True experimental designs
• Quasi-experimental designs
 Descriptive Research

• Descriptive: investigator attempts to describe a

group of individuals on a set of variables or
characteristics.
• Enables classification and understanding
• Methods: survey research, case study, qualitative,
developmental (natural history of something, patterns of
growth and change), normative, evaluation
 Store Type Quality Composite Example
(33 stores) Score (mean)

Convenience Stores -0.74

(70%)

Local Markets -0.38

(24%)

Supermarkets (6%) 6.5

Composite score = sum of scores for access to fresh fruit,

fresh vegetables, lean meats, low-fat milk,
tobacco products, alcohol. Chronbach’s alpha = .76

Source: Freedman & Bell, 2009

 Exploratory Research

• Investigator examines a phenomenon of interest

and explores its dimensions, including how it
relates to other factors.
• Proven relationships between the phenomenon and other
factors can lead to predictive models
• Correlational studies, cohort and case control, secondary
analysis, historical research
Freedman, Blake, & Liese, 2013
 *p<.05

Figure 2. Simplified Path Analytic Model 1 of Environmental Influence on FV Intake

Source: Liese et al., 2013.
Freedman et al., under review
 Experimental Research

• Provides a basis for comparing 2 or more conditions

• Controls or accounts for the effects of extraneous factors,
providing the highest degree of confidence in the validity
of outcomes
• Enables the researcher to draw meaningful conclusions
about observed differences
• Randomized controlled trials, single subject designs,
sequential clinical trials, evaluation research, quasi-
experimental research, meta-analysis
Individual-level Change in Fruit
and Vegetable Consumption
• Design: Longitudinal; no comparison group
• Sample: 45 diabetic patients at FQHC
• Intervention: FQHC-based farmers’ market + financial incentive
(up to $50)
• Outcome measure: F/V consumption measured with NCI screener
• Results:
• Dose-response relationship between improvement in F/V consumption and
use of market
• Improvers more likely to rely on financial incentive to purchase foods at
market

Source: Freedman et al., 2013

 Descriptive Exploratory Experimental

Continuum
Describe populations
Case Study
of Research
Find relationships Cause and Effect
Experimental randomized
controlled trial (RCT)

Quasi-experimental designs
----------------------------------------------Survey research--------------------------------
----------------------Qualitative research---------------- Sequential clinical trial
Correlational research Single subject designs
Evaluation research Evaluation research
------------------------Secondary analysis----------------- Meta-analysis
-----Cohort/Case-Control Study-----
Historical research

Based on: Portney LG, Watkins MP. 2000. Foundations of Clinical Research: Applications to Practice. 2 nd Ed. Upper Saddle River,
NJ: Prentice Hall Health., p. 13.
 Community-engaged research

• Philosophy versus method

• Who are the “knowers” of phenomenon?
• Participatory processes during some or all stages of
research
• Knowledge for action/change
• Can be used with any research approach
• Example: Community Visions Photovoice Project
http://www.youtube.com/watch?v=95IMZlKLs2c (~9 min)
 Quantitative/Qualitative

• Quantitative research involves measurement of outcomes

using numerical data under standardized conditions
• May be used along the continuum of research
• Qualitative research is concerned with narrative
information under less structured conditions that often
takes the research context into account
• Descriptive and exploratory research
• Purposes: describing conditions, exploring associations,
formulating theory, generating hypotheses
Choosing evaluation methods

20
 Descriptive Research

• Case study
• Cross-sectional study
• Qualitative study
 Case Study Design
• Often a description of a individual case’s condition or response
to an intervention
• can focus on a group, institution, school, community, family, etc.
• data may be qualitative, quantitative, or both
• Case series: observations of several similar cases are reported
 Case Study
Example
• In 1848, young railroad worker, Phineas Gage, was forcing gun powder into a rock with
a long iron rod when the gun powder exploded. The iron rod shot through his cheek and
out the top of his head, resulting in substantial damage to the frontal lobe of his brain.
Incredibly, he did not appear to be seriously injured. His memory and mental abilities
were intact, and he could speak and work. However, his personality was markedly
changed. Before the accident, he had been a kind and friendly person, but afterward he
became ill-tempered and dishonest.

• Phineas Gage’s injury served as a case study for the effects of frontal lobe damage. He
did not lose a specific mental ability, such as the ability to speak or follow directions.
However, his personality and moral sense were altered. It is now known that parts of the
cortex (called the association areas) are involved in general mental processes, and
damage to those areas can greatly change a person’s personality.
 Case Study Design
• Strengths
• Enables understanding of the totality of an individual’s (or organization, community)
experience
• The in-depth examination of a situation or ‘case’ can lead to discovery of relationships
that were not obvious before
• Useful for generating new hypotheses or for describing new phenomena
• Weaknesses
• No control group
• Prone to selection bias and confounding
• The interaction of environmental and personal characteristics make it weak in internal
validity
• Limited generalizability
 Cross-sectional Study
• Researcher studies a stratified group of subjects at one point in
time
• Draws conclusions by comparing the characteristics of the
stratified groups
• Well-suited to describing variables and their distribution patterns
• Can be used for examining associations; determination of which
variables are predictors and which are outcomes depends on the
hypothesis
• eg. Does lead paint ingestion cause hyperactivity or does hyperactivity
lead to lead paint ingestion?
 Cross-sectional Study

• Example:
What is the prevalence of chlamydia in women age 18-35 in
Cleveland, and is it associated with the use of oral contraceptives?
• Select a sample of 100 women attending an STD clinic in the
city of Cleveland
• Measure the predictor and outcome variables by taking a history
of oral contraceptive use and sending a cervical swab to the lab
for chlamydia culture
• A questionnaire may be used to gather information abut oral
contraceptive history
 Cross-sectional Study
• Strengths
• Fast and inexpensive
• No loss to follow-up (no follow-up)
• Ideal for studying prevalence
• Convenient for examining potential networks of causal links
• e.g., in analysis, examine age as a predictor of oral contraceptive use, and then examine
oral contraceptive use as a predictor for chlamydia infection

• Weaknesses:
• Difficult to establish a causal relationship from data collected in a cross-sectional
time-frame (Lack of a temporal relationship between predictor variables and outcome
variables - Does not establish sequence of events)
• Not practical for studying rare phenomena
 Qualitative Study

• Seeks to describe how individuals perceive their

own experiences within a social context
• Emphasizes in-depth, nuanced understanding of
human experience and interactions
• Methods include in-depth interviews, direct
observations, examining documents, focus groups
• Data are often participants’ own words and
narrative summaries of observed behavior
 Qualitative Study

Example
A researcher wants to understand how provision of
healthcare to undocumented persons affects the people
and institutions involved
• In 3 communities, information is gathered from
undocumented patients, FQHC primary care clinicians,
specialists, and hospital administrators
• Methods: in-depth interviews, key informant interviews,
participant observations, case studies, focus groups
 Qualitative Study
Strengths
• Data based on the participants’ own categories of meaning
• Useful for studying a limited number of cases in depth or describing complex phenomena
• Provides understanding and description of people’s personal experiences of phenomena
• Can describe in rich detail phenomena as they are embedded in local contexts
• The researcher can study dynamic processes (i.e., document sequential patterns/change)
Weaknesses
• Knowledge produced might not generalize to other people or other settings
• It is difficult to make quantitative predictions
• It might have lower credibility with some administrators and commissioners of programs
• Takes more time to collect and analyze the data when compared to quantitative research
• The results are more easily influenced by the researcher’s personal biases and idiosyncrasies
 Exploratory Research

• Cohort study

• Case control study

 Cohort Study

• A group of individuals who do not yet have the outcome of

interest are followed together over time to see who
develops the condition
• Participants are interviewed or observed to determine the
presence or absence of certain exposures, risks, or
characteristics
• May be simply descriptive
• May identify risk by comparing the incidence of specific
outcomes in exposed and not exposed participants
 Cohort Study

• Example
To determine whether exercise protects against coronary heart
disease (CHD).
1. Assemble the cohort: 16,936 Harvard alumni were enrolled
2. Measure predictor variables: Administer a questionnaire
about activity and other potential risk factors , collected
data from college records
3. 10 years later, sent a follow-up questionnaire about CHD
and collected data about CHD from death certificates
 Cohort Study

• Strengths
• Powerful strategy for defining incidence and investigating potential
causes of an outcome before it occurs
• Time sequence strengthens inference that the factor may cause the
outcome
• Weaknesses
• Expensive – many subjects must be studied to observe outcome of
interest
• Potential confounders: eg, cigarette smoking might confound the
association between exercise and CHD
 Case-Control Study

• Generally retrospective
• Identify groups with or without the condition
• Look backward in time to find differences in predictor
variables that may explain why the cases got the condition
and the controls did not
• Assumption is that differences in exposure histories should
explain why the cases have the condition
• Data collection via direct interview, mailed questionnaire,
chart review
 Case-Control Study
• Strengths
• Useful for studying rare conditions
• Short duration & relatively inexpensive
• High yield of information from relatively few participants
• Useful for generating hypotheses
• Weaknesses
• Increased susceptibility to bias:
• Separate sampling of cases and controls
• Retrospective measurement of predictor variables
• No way to estimate the excess risk of exposure
• Only one outcome can be studied
 Case-Control Study

• Example
Purpose: To determine whether there is an association between the
use of aspirin and the development of Reye’s syndrome in children.
1. Draw the sample of cases – 30 patients who have had Reye’s
syndrome
2. Draw the sample of controls – 60 patients from the much larger
population who have had minor viral illnesses without Reye’s
syndrome
3. Measure the predictor variable: ask patients in both groups
about their use of aspirin
 Experimental Research
• True experimental designs

• Quasi-experimental designs
 Efficacy vs. Effectiveness

• Efficacy: the benefit of an intervention compared

to a control or standard program under controlled,
randomized conditions
• Randomized controlled trial (RCT) design often used
• Effectiveness: the benefit of an intervention under
less controlled ‘real world’ conditions
• Quasi-experimental design often used
 Types of designs

1. One group posttest only design

P T2

P = Program or intervention
T2 = Posttest

40
 Types of designs

2. Before and After Design

One group pretest-post-test design

T1 P T2

T1 = Pretest (treatment group)

T2 = Posttest (treatment group)
P = Program or intervention
41
How much of the effect is due to the program?

Desired
Outcome T
Net
(Y) Effect

C
Gross
Effect

Pre Time (X) Post

42
 Types of designs
2. Comparison Group Design

T1 P T2

C1 C2

• T1 = Pretest (treatment group)

• T2 = Posttest (treatment group)
• P = Program or intervention
• C1 = Pretest (comparison group)
• C2 = Posttest (comparison group) 43
 Experimental Design
• True experimental design: Subjects are randomly assigned to at
least 2 comparison groups
• Purpose is to compare 2 or more groups that are formed by
random assignment
• The groups differ solely on the basis of what occurs between
measurements (ie, intervention)
• Changes from pretest to posttest can be reasonably attributed to the
intervention
• Most basic is the pretest-posttest control group design
(randomized controlled trial, RCT)
 Experimental Design
Example:
• Researchers conducted an RCT to study the effect of progressive
resistance exercises in depressed elders. They studied 35 volunteers
who had depression.
• Participants were randomly assigned to an exercise group, which met
three times per week for 10 weeks, or a control group which met 2
times per week for an interactive health education program.
• The outcome variables were: level of depression, functional status, and
quality of life, using standardized instruments.
• Pretest and posttest measures were taken for both groups and
differences were compared.
 Experimental Design
Strengths
• Controls the influence of confounding variables, providing more conclusive
answers
• Randomization eliminates bias due to pre-randomization confounding variables
• Blinding the interventions eliminates bias due to unintended interventions
Weaknesses
• Costly in time and money
• Many research questions are not suitable for experimental designs
• Usually reserved for more mature research questions that have already been
examined by descriptive studies
• Experiments tend to restrict the scope and narrow the study question
 Quasi-Experimental Design

• Quasi-Experimental designs do not use randomized

assignments for comparisons
 Quasi-Experimental Design

• Example:
• A study was designed to examine the effect of electrical
stimulation on passive range of motion of wrist extension
in 16 patients who suffered a stroke.
• Outcomes: effects of treatment on sensation, range of
motion, & hand strength.
• Patients were given pretest and posttest measurements
before and after a 4-week intervention program.
• Note: No randomization, and no comparison group
 Quasi-Experimental Design
Strengths
• Q-E designs are a reasonable alternative to RCT
• Useful where pre-selection and randomization of groups is difficult
• Saves time and resources vs. experimental designs
Weaknesses
• Nonequivalent groups may differ in many ways -- in addition to the differences between
treatment conditions, introducing bias
• Non-blinding allows the possibility of unintended interventions; blinding can be used in
some Q-E studies
• Must document participant characteristics extensively
• Potential biases of the sample must be acknowledged when reporting findings
• Causal inferences are weakened by the potential for biases vs. experimental designs
 Compared to what?

 Over time  Between groups

 Pre to post  Randomly composed
 Longitudinal  Naturally occurring (waitlist,
other programs)
 National norms/standards

Low Ability to High Ability to

Attribute Effect Attribute Effect

Post-test Pre & Post Nonequivalent Quasi-experiment Randomized

only test comparison (matched groups, experiment
group regression 50

discontinuity)
51
 Major sources

Portney LG, Watkins MP. 2000. Foundations of

Clinical Research: applications to practice. 2nd Ed.
Upper Saddle River, NJ: Prentice Hall Health
Hulley SB, Cummings SR. 1998. Designing Clinical
Research: an epidemiologic approach. Baltimore,
MD: Williams and Wilkins
Cook TD, Campbell DT. 1979. Quasi-
Experimentation: design & analysis issues for field
settings. Boston, MA: Houghton Mifflin Company