VALIDITY

Validity in experiments refers to the principle of studying the variables the researchers intend to
study. It is the first criterion for evaluating a measurement procedure and is important in
formulating definitions.

The researchers must demonstrate that the measurement procedure is actually measuring what it
claims to be measuring.

Historically, much of the work devoted to establishing validity has focused on developing
psychological tests, such as intelligence tests, personality tests, tests used to diagnose
psychological disorders, aptitude, achievement tests, and personnel selection tests (Passer, 2017).

Methods for Assessing the Validity of Measurement

● Face Validity: This is an unscientific form of validity demonstrated when a measurement

procedure superficially appears to measure what it claims to measure. It is the simplest
definition of validity.

It concerns a measurement procedure's superficial appearance or face value. Moreover, face

validity is based on subjective judgment and is difficult to quantify.

● Concurrent Validity: is demonstrated when scores obtained from a new measure are
directly related to scores obtained from an established measure of the same variable.

It establishes consistency between two different procedures for measuring the same
variable, suggesting that the two measurement procedures are measuring the same thing.

● Predictive Validity: is demonstrated when scores obtained from a measure accurately

predict behavior according to a theory. It is under the category of criterion validity, which
addresses the ability of a measure to predict an outcome.

● Construct Validity: requires that the scores obtained from a measurement procedure
behave exactly the same as the variable itself. It is based on many research studies that
use the same measurement procedure and grows gradually as each new study contributes
more evidence.

It is the most important aspect of validity as it deals with the transition from theory to
research application. Furthermore, construct validity is the broadest and most theoretically
based type of validity.
 ● Convergent Validity: is demonstrated by a strong relationship between the scores
obtained from two or more different methods of measuring the same content. The goal is
to demonstrate that different measurement procedures "converge" or join on the same
construct.

● Divergent Validity: is demonstrated by showing little or no relationship between the

measurements of two different constructs. The
goal is to differentiate between two conceptually distinct constructs by measuring both
constructs and then showing that there is little or no relationship between the two
measurements.

Evaluating the Experiment: Internal Validity

Internal Validity is the degree to which a researcher is able to state a causal relationship
between antecedent conditions and the subsequent observed behavior.
When researchers set up an experiment, they plan procedures to measure the effects of
various treatment levels.

The goal of an experiment is to assess the impact of the independent variable.

An experiment is internally valid if researchers can be sure that the changes in behavior
observed across the treatment conditions of the experiment were actually caused by the
independent variable (Campbell, 1957).

Logically, researchers cannot make any correct decisions about how well the findings of an
experiment would generalize if the experiment is not, first and foremost, internally valid.
Internal validity is one of the most important concepts in experimentation (Myers &
Hansen, 2012).

Extraneous Variables and Confounding

A typical research study concentrates on two variables and attempts to demonstrate a relationship
between them. Though most studies focus on two variables, countless other elements vary within
the study, meaning many additional variables are part of every research study.

Some of these extra variables are related to the individuals participating, and other variables
involve the study's environment. Additional variables that exist in a research study but are not
directly investigated are called extraneous variables, and every research study has thousands of
them (Gravetter & Forzano, 2018).
A confounding variable is an extraneous variable (usually unmonitored) that changes
systematically along with the two variables being studied. A confounding variable provides an
alternative explanation for the observed relationship between the two variables and threatens
internal validity (Gravetter & Forzano, 2018).

A confounding variable has two important characteristics:

1. First, an extraneous variable becomes a confounding variable only if it influences the

dependent variable. Something totally unrelated to the dependent variable is not a threat.
In most experiments, for example, the participants are wearing different types of shoes
(sneakers, flats, heels, loafers, or sandals); however, it is unlikely that the type of shoe
influences the dependent variable. Thus, it is not necessary to take any steps to control
the shoe variable.

1. Second, a confounding variable must vary systematically from the independent variable.
The concept of random versus systematic change is an important part of control
(Gravetter & Forzano, 2018).

A variable that changes randomly, with no relation to the independent variable, is not a
threat.

Basic Threats to Internal Validity

Psychologists identified seven kinds of extraneous variables that can threaten the internal validity
of experiments and quasi-experiments.

Suppose one or more of these threats are present. In that case, the experiment will lack internal
validity, and researchers can never be sure that the experimental manipulation really caused the
effects produced on the dependent variable.

● History. This refers to events that occur while a study is being conducted and are not a
part of the experimental manipulation or treatment. History is most often a problem when
a whole group of individuals is tested together in the same experimental condition.

Some outside events that occurred before their group testing session could influence the
responses of the entire group, and the effects produced by the event could be mistaken for
the effect of the IV.
● Maturation. This refers to ways that people naturally change over time, independent of
their participation in a study.

This includes changes in cognitive and physical capabilities that occur with aging,
fluctuations in alertness and fatigue accompanying biological rhythms, and normal
recovery from physical illness or psychological disorders.

Maturation also includes the general accrual of knowledge and skills as we gain more
experience over time.

● Testing. This concerns whether the act of measuring participants' responses affects how
they respond to subsequent measures. Moreover, a testing threat refers to effects on the
dependent variable produced by a previous administration of the same test or other
measuring instruments.

Changes in motivation or practice habits that occur in response to taking a performance

pretest also illustrate testing effects (Shadish et al., 2002).

● Instrumentation. This refers to changes that occur in a measuring instrument during the
course of data collection.

Whenever some feature of the measuring instrument itself changes during the experiment,
the researcher is faced with the possibility of a threat to internal validity. Hence, the
researcher's and/or the observers' measurement approach should remain consistent from
start to finish.

● Regression to the mean. Also called statistical regression, it is the concept that when
two variables are not perfectly correlated, more extreme scores on one variable will be
associated with less extreme scores on the other variable.

It can occur whenever subjects are assigned to conditions based on extreme scores on a
test. Statistically, extreme scores tend to have less test-retest reliability than moderate
scores - those closer to the mean or average (Myers & Hansen, 2012).

● Selection. This refers to situations in which, at the start of a study, participants in various
conditions already differ on a characteristic that can partly or fully account for the
eventual results.
 Also, a selection threat is present whenever the researcher does not assign subjects
randomly to the different conditions of an experiment. Random assignment is an important
feature of true experiments (Myers & Hansen, 2012).

● Attrition. Also called subject loss and subject mortality occurs when participants fail to
complete a study. Attrition can occur for many reasons. A participant may feel
uncomfortable and not wish to continue.

Some may move away, lose interest, and become too ill to continue or die before
completing the study.

Attrition is always undesirable, and it poses the greatest threat to internal validity when
participants who discontinue differ from those who complete the study in some attribute
that could account for the changes obtained on the dependent variable (Passer, 2017).

Figure 5. Some Potential Threats to Internal Validity

SOURCE POTENTIAL THREAT

History Events to which people are exposed while participating in a

study, but that are not part of the experimental manipulation
(e.g., the treatment or intervention) being examined.

Maturation Ways in which people naturally change over time, independent

of their participation in a study.

Testing The act of measuring individuals' responses may affect their

responses on subsequent measures.

Instrumentation Changes that occur in a measuring instrument during the course

of data collection.
Regression to the The statistical concept that more extreme scores on a variable are
Mean associated, overall, with less extreme scores on another variable.
When people selected for a treatment have an extreme pretest
mean score and a less extreme posttest mean score (e.g., overall,
high depression scores decrease; low performance scores
increase), this may partly or fully reflect statistical regression.

Attrition Also called subject loss; occurs when participants fail to

complete a study.

Selection Refers to the fact that, at the start of a study, participants in the
various conditions already differ on a characteristic that can
partly or fully account for the eventual results.

Guide Questions:

Identify whether the statement is True or False.

1. Simply by taking the same test twice, participants' mean score improves. This is an
instrumentation effect.

2. Construct validity is the most theoretical type of validity.

3. Even in a randomized experiment, the potential for attrition remains a concern.

ANSWERS to the Guide Questions

Experimental Method/ Research Hypothesis

Fill in the blanks to complete the thought of the sentence.

1. The hypothesis must be ________ meaning it must be possible to obtain research results
that are contrary to the hypothesis. (refutable)

2. Researchers perform an experiment in the absence of a compelling theory just to see what
happens. This is called a (what-if experiment) and the alternative
 3. A null hypothesis is represented by the symbol hypothesis by the symbol ____________.
(Ho and H1)

The Basics of Experimentation/Independent and Dependent Variables

Identify whether the underlined variable is Independent or Dependent.

1. How does stress affect the mental state of a human being? (Independent)

2. Promotion affects employees' motivation (Independent)

3. Higher Education typically leads to Higher Income (Dependent)

Experimental Method/ Research Hypothesis

Identify whether the statement is True or False.

1. Simply by taking the same test twice, participants' mean score improves. This is an
instrumentation effect. (False)

2. Construct validity is the most theoretical type of validity. (True)

3. Even in a randomized experiment, the potential for attrition remains a concern. (True)