CHERRY ROSE L.

SORROSA
MA in Science Education (Biology)
EXPERIMENTAL VALIDITY

Experimental validity refers to the

manner in which variables influences
both the results of the research and
the generalizability to the population
at large.
THREATS TO INTERNAL AND
EXTERNAL VALIDITY:
Is the researcher’s conclusion correct?

Are the changes in the independent variable

indeed responsible for the observed variation
in the dependent variable?
Is it possible that the variation in the
dependent variable can be attributed to
other factors?
INTERNAL VALIDITY
Internal validity refers to a study’s ability to
determine if a causal relationship exists between
one or more independent variables and one or
more dependent variables.

 In other words, can we be reasonably sure that

the change (or lack of change) was caused by the
treatment? .
WHY IS INTERNAL VALIDITY IMPORTANT?
If the study shows a high degree of internal
validity, then we can conclude that we have
strong evidence of causality.

If a study has low internal validity, then we

can conclude that the study has little or no
evidence of causality.
1. HISTORY
History refers to any event outside of the research
study that can alter or effect subjects’
performance.

The events occurring between the first and

second measurements in addition to the
experimental variable which might affect the
measurement.
Using randomization procedures can often
minimize this risk, assuring that outside events
that occur in one group are also likely to occur in
the other.

Example: Researcher collects gross sales data

before and after a 5 day 50% off sale. During the
sale a hurricane occurs and results of the study
may be affected because of the hurricane, not the
sale.
2. MATURATION
Maturation refers to the natural physiological or
psychological changes that take place as we age.

The process of maturing which takes place in the

individual during the duration of the experiment
which is not a result of specific events but of
simply growing older, growing more tired, or
similar changes.
Example: Subjects become tired after completing a
training session, and their responses on the Posttest
are affected.

For instance, an episode of major depression typically

decreases significantly within a six-month period even
without treatment. Imagine we tested a new
medication designed to treat depression. If our
results showed that subjects who took this
medication showed a significant decrease in
depressive symptoms within six months, could we
truly say that the medication caused the decrease in
symptoms? Probably not, especially since maturation
alone would have shown similar results.
3. PRE-TESTING
The effect created on the second measurement by
having a measurement before the experiment.

Example: Subjects take a Pretest and think about

some of the items. On the Posttest they change to
answers they feel are more acceptable.
Experimental group learns from the pretest.
4. MEASURING INSTRUMENTS
Changes in instruments, calibration of instruments,
observers, or scorers may cause changes in the
measurements.

If the measurement device(s) used in your study

changes during the course of the study, changes in
scores may be related to the instrument rather than
the independent variable.
Example: Interviewers are very careful with their
first two or three interviews but on the 4th, 5th,
6th become fatigued and are less careful and
make errors.

For instance, if your pretest and posttest are

different, the change in scores may be a result of
the second test being easier than the first rather
than the teaching method employed. For this
reason, it is recommended that pre- and posttests
be identical or at least highly correlated.
5. STATISTICAL REGRESSION
It refers to the tendency for subjects who score
very high or very low to score more toward the
mean on subsequent testing.

Groups are chosen because of extreme scores of

measurements; those scores or measurements
tend to move toward the mean with repeated
measurements even without an experimental
variable.
Example: Managers who are performing poorly
are selected for training. Their average Posttest
scores will be higher than their Pretest scores
because of statistical regression, even if no
training were given.

If you get a 99% on a test, for instance, the odds

that your score will be lower the second time are
much greater than the odds of increasing your
score.
6. EXPERIMENTAL MORTALITY
The loss of subjects from comparison groups could greatly
affect the comparisons because of unique characteristics of
those subjects. Groups to be compared need to be the
same after as before the experiment.

Example: Over a 6 month experiment aimed to change

accounting practices, 12 accountants drop out of the
experimental group and none drop out of the control
group. Not only is there differential loss in the two groups,
but the 12 dropouts may be very different from those who
remained in the experimental group.
EXTERNAL VALIDITY
External validity refers to the generalizability of a
study.

In other words, can we be reasonably sure that

the results of our study consisting of a sample of
the population truly represents the entire
population?
Threats to external validity can result in
significant results within a sample group but an
inability for this to be generalized to the
population at large.

External Validity asks to what populations,

settings, treatment variables, and measurement
variables can this observed effect be generalized.

Threats to external validity compromise our

confidence in stating whether the study’s result
are applicable to other groups.
1. PRE-TESTING
Individuals who were pretested might be less or more
sensitive to the experimental variable or might have
"learned" from the pre-test making them
unrepresentative of the population who had not been
pre-tested.

Example: Prior to viewing a film on Environmental

Effects of Chemical, a group of subjects is given a 60
item antichemical test. Taking the Pretest may increase
the effect of the film. The film may not be effective for a
nonpretested group.
2. DIFFERENTIAL SELECTION
The selection of the subjects determines how the findings
can be generalized. Subjects selected from a small group
or one with particular characteristics would limit
generalizability. Randomly chosen subjects from the entire
population could be generalized to the entire population.

Example: Researcher, requesting permission to conduct

experiment, is turned down by 11 corporations, but the
12th corporation grant permission. The 12th corporation is
obviously different then the others because they accepted.
Thus subjects in the 12th corporation may be more
accepting or sensitive to the treatment.
3. EXPERIMENTAL PROCEDURES
The experimental procedures and arrangements
have a certain amount of effect on the subjects in
the experimental settings. Generalization to
persons not in the experimental setting may be
precluded.

Example: Department heads realize they are

being studied, try to guess what the experimenter
wants and respond accordingly rather than
respond to the treatment.