Reliability and

Validity
Psy 104
 Reliability concerns the extent to which a
measurement of a phenomenon provides
stable and consist result (Carmines and
Zeller, 1979).

Reliability Reliability is also concerned with

repeatability (tekrar edilebilirlik).
(güvenirlik)
For example, a scale or test is said to be
reliable if repeat measurement made by it
under constant conditions will give the
same result (Moser and Kalton, 1989).
 Testing for reliability is important as
it refers to the consistency across
the parts of a measuring instrument
(Huck, 2007).

Reliability A scale is said to have high internal

consistency (iç tutarlık) reliability if
the items of a scale “hang together”
and measure the same construct
(Huck, 2007, Robinson, 2009).
RELIABILITY
The consistency of measurements

A RELIABLE TEST
Produces similar scores across
various conditions and situations,
including different evaluators and
testing environments.
 The most commonly used internal
consistency measure is the Cronbach
Alpha coefficient. It is viewed as the
most appropriate measure of reliability
when making use of Likert scales
(Whitley, 2002, Robinson, 2009).
Cronbach’s
Alpha No absolute rules exist for internal
consistencies, however, most agree on
a minimum internal consistency
coefficient of .70 (Whitley, 2002,
Robinson, 2009).
How do we account for an individual who
does not get exactly the same test score
every time he or she takes the test?

1. Test-taker’s temporary psychological or

physical state
2. Environmental factors
3. Test form
4. Multiple raters
RELIABILITY COEFFICIENTS
• The statistic for expressing
reliability.
• Expresses the degree of
consistency in the measurement
of test scores.
• Donoted by the letter r with two
identical subscripts (rxx)
Güvenirlik
• Ölçümünüzün tekrarlanabilirliği veya
tutarlılığı anlamına gelir.
• Test-tekrar test güvenilirliği: Zaman
içindeki tutarlılıkla ilgilidir.
• Değerlendiriciler arası güvenilirlik:
Değerlendiriciler arasındaki tutarlılıkla
ilgilidir.
• Paralel form güvenilirliği: Kuramsal
olarak eşdeğer ölçümlerdeki tutarlılık
ile ilgilidir.
• İç tutarlılık güvenilirliği: Ölçüm, benzer
işlevleri yerine getiren çok sayıda farklı
parçadan oluşturulmuşsa, tek tek
parçalar arasındaki tutarlılık.
TEST-RETEST RELIABILITY

Suggests that subjects tend to

obtain the same score when
tested at different times.
Split-Half Reliability

•Sometimes referred to as internal

consistency
•Indicates that subjects’ scores on some
trials consistently match their scores on
other trials
INTERRATER RELIABILITY
Involves having two raters independently
observe and record specified behaviors,
such as hitting, crying, yelling, and getting
out of the seat, during the same time period

TARGET BEHAVIOR
A specific behavior the observer is
looking to record
ALTERNATE FORMS RELIABILITY
Also known as equivalent forms reliability or parallel forms
reliability
Obtained by administering two equivalent tests to the same
group of examinees
Items are matched for difficulty on each test

It is necessary that the time frame between giving the two forms
be as short as possible
 OBTAINED SCORE
•The score you get when you administer a test
•Consists of two parts: the true score and the
error score

STANDARD ERROR of
MEASUREMENT (SEM)
Gives the margin or error that you should
expect in an individual test score because of
imperfect reliability of the test
Evaluating the Reliability Coefficients

• The test manual should indicate why a certain type of

reliability coefficient was reported.
• The manual should indicate the conditions under which the
data were obtained
• The manual should indicate the important characteristics of
the group used in gathering reliability information
FACTORS AFFECTING RELIABILITY

1. Test length
2. Test-retest interval
3. Variability of scores
4. Guessing
5. Variation within the test situation
Reliability
• For an exploratory or pilot study, it is suggested that reliability
should be equal to or above 0.60 (Straub et al., 2004).
• Hinton et al. (2004) have suggested four cut-off points for reliability,
which includes excellent reliability (0.90 and above), high
reliability (0.70-0.90), moderate reliability (0.50-0.70) and low
reliability (0.50 and below)(Hinton et al., 2004).
• Although reliability is important for study, it is not sufficient unless
combined with validity.
• In other words, for a test to be reliable, it also needs to be valid
(Wilson, 2010).
APA table
What Is The Relationship Between Reliability And
Validity?
• When details are valid, it needs to be reliable as well.
• If the scores on a test are wildly different every time the participants
take the test, then it is unlikely that the test will predict anything.
• Even if a test is reliable, it does not automatically mean it is valid.
• For example, we would not measure someone's strength as a
measure of their intelligence.
• The two are not related and would not create a valid conclusion.
 • Reliability is a necessary condition for
validity if you have a valid test, but it alone is
not a sufficient reason to call a test valid.
What Is The • Why is validity so important?
Relationship • As a body of research is built, the validity is
demonstrated in the relationship between
Between the test and the behavior it is intended to
Reliability measure.
• A valid test also ensures that results
And Validity? accurately reflect the dimension undergoing
assessment.
 • The concept of validity was formulated
by Kelly (1927, p. 14), who stated that a
test is valid if it measures what it claims
Validity to measure.
(geçerlik)
• For example, a test of intelligence
should measure intelligence and not
something else (such as memory).
Validity (geçerlik)
• Validity explains how well the collected data covers the actual
area of investigation (Ghauri and Gronhaug, 2005).
• Validity basically means “measure what is intended to be
measured” (Field, 2005).
Internal And External Validity In Research

• Internal validity refers to whether the effects observed in a study are due to
the manipulation of the independent variable and not some other factor.
• In other words, there is a causal relationship between the
independent and dependent variables.
• Internal validity can be improved by controlling extraneous variables,
using standardized instructions, counterbalancing, and eliminating
demand characteristics and investigator effects.
• External validity refers to the extent to which the results of a study can be
generalized to other settings (ecological validity), other people (population
validity), and over time (historical validity).
• External validity can be improved by setting experiments in a more natural
setting and using random sampling to select participants.