1

Outline
1. Ethics vs. Law
2. Ethical use of tests – issues:
 Competence
 Informed Consent
 Knowledge of Results
 Confidentiality
 Test security
 Divided Loyalties
 2
Outline
4. Professional Issues
 Theoretical issues
 Test adequacy
 Actuarial vs. clinical prediction
5. Social Issues
 Dehumanization
 Access to benefits of psychological testing
 3
Outline
 6. Two examples of bad tests
 Handwriting analysis
 TOEFL
 7. The CPA Code of Ethics (synopsis)
 Respect for the dignity of persons
 Responsible caring
 Integrity in relationships
 Responsibility to society
 4
Ethics vs. Law
 Ethics: what one  Ethics codes are not
should or should not produced by
do, according to democratically-elected
principles or norms of legislatures
conduct  Enforcement
mechanisms are
usually informal, may
be complex, even
unconscious
 5
Ethics vs. Law
 Law: what one must  Laws are created by
or must not do, democratically-elected
according to legal legislatures in
dictates democracies such as
Canada
 Laws come with
explicit penalties for
infractions and a
formal enforcement
system
 6
Ethical use of tests – issues:
1. Competence
2. Informed Consent
3. Knowledge of Results
4. Confidentiality
5. Test security
6. Divided Loyalties
 7
1. Competence

 develop competence in  competence is very

assessment concepts important if test result
and methods affects someone’s life
 recognize boundaries
of your competence
 8
2. Informed Consent

 Generally, the  psychologist must

client/subject must inform client about
voluntarily consent nature and purpose of
assessment in
understandable
language
 9
Informed consent - exceptions

 Sometimes, it is  Sometimes it is
acceptable to test necessary to test
without getting consent without getting
– e.g., you have given consent, even when
implied consent to be
tested by registering in consent is explicitly
this course, but I have refused by person to be
never sought your tested – e.g., when
explicit consent mandated by law
 10
3. Knowledge of Results
 must fully disclose  but do not do so in a
test results in way that reveals the
understandable content of test items,
language because that content is
proprietary
information (e.g.,
somebody owns it –
it’s not yours to
reveal)
 11
Constraints on client access to results
 PIPEDA Principal 4.9:  Section 9.3b says that
 Upon request, a
access can be denied if
person must be
informed of the it would mean
existence, use, and revealing confidential
disclosure of his or commercial
her personal
information and must information (unless
be given access if latter is severable).
requested.
 12
Constraints on client access to results
 Ontario Personal Health  CPA and leading Canadian
Information Act: access test publishers say that Test
rights do not apply to a Materials are not “personal
record that contains raw information” so should not
data from standardized be released.
psychological tests or
assessments, unless those
data are reasonably
severable.
 13
4. Confidentiality
 Test results are  Release of results
confidential should only be made
information to another qualified
professional after
client’s consent
 14
5. Test Security
 Test materials must be  Test items are not
kept secure revealed except in
training programs and
when mandated by
law, to protect test
integrity
 Test items are private
property
 15
6. Automated Scoring/Interpretation Systems
 Psychologist is still  Psychologists are
responsible for proper professionals – they
interpretation of test take personal
results responsibility for their
work
 16
7. Divided loyalties
 Who is the client?  What if these parties
 The person being have conflicting
tested, or the interests?
institution you work  How do you maintain
for? test security but also
explain an adverse
decision?
 17
Divided loyalties
 Suppose you are asked  Suppose Y is not
to test a child who common and no
might have disorder X resources are available
or Y in school to deal with
 Suppose X is very it
common and so the  If your testing
government provides suggests Y, will you
resources to help report Y – or X, so the
schools deal with it child gets some sort of
help?
 18
Professional Issues
1. Theoretical issues
2. Test adequacy
3. Actuarial vs. clinical prediction
 19
1. Theoretical Issues
 Are you measuring a stable  If so, differences in scores
characteristic of the person over time reflect
being tested? measurement error or
subject variables such as
fatigue
 20
1. Theoretical Issues
 Are you measuring a stable  In fact, we have little
characteristic of the person evidence that this is true,
being tested? so what is the value of your
test result – will it still be
true next year?
 Especially problematic for
personality tests
 21
1. Theoretical issues
 Are you measuring a stable  Suppose you test someone
characteristic of the person as part of a hiring process.
being tested?  If the test result says that
person does not have the
characteristic you’re
looking for, does that mean
they could never acquire
that characteristic?
 22
2. Adequacy of tests
 How do we measure  What should go into
test adequacy? an assessment of test
 (We’re supposed to be adequacy?
good at measuring
things!)
 23
2. Adequacy of tests
 So far, in considering  We haven’t asked: is
the relative merits of the best test available
various tests, we have good enough?
asked whether they  Society uses that
are psychometrically standard when testing
adequate becomes a legal issue
 But the answer is not
psychometric
 24
3. Actuarial vs. clinical judgment
 U.S. Supreme Court  Is it possible for us to
Justice Potter Stewart make good judgments
said in a 1964 case on a question when we
that he didn’t know cannot articulate the
how to define basis for the
pornography, but he judgment?
knew it when he saw
it.
 25
3. Actuarial vs. clinical judgment
 Actuarial judgment  Clinical judgment
occurs when we feed occurs when we have
test scores into a trained psychologist
statistical formulas to interpret test scores to
diagnose a diagnose a
psychological psychological
condition or predict condition or predict
future performance. future performance.
 26
3. Actuarial vs. clinical judgment
 In actuarial judgment,  In clinical judgment,
we cannot make the claim is that you
accurate predictions can determine “what
tailored to individuals caused what” in an
 Instead, our individual’s person’s
conclusion will be the life (Dawes, 1994)
same for every person  But clinical judgment
with a given set of test does not improve with
scores experience
 27
3. Actuarial vs. clinical prediction
 Meehl’s work in the  statistical formula
1950s: feed test scores does a better job of
into statistical formula prediction
to predict future
performance – or have
 still, someone has to
trained psychologist be legally responsible
interpret scores – that has to be a
trained professional
 28
Social issues
1. Dehumanization
2. Access to benefits of psychological testing
 29
1. Dehumanization
 Does computerized  humans are very
testing and analysis of complex – which
test results create a allows us to be
danger of minimizing individuals, different
human uniqueness? from each other
 but testing and
interpretation
generalize
 30
1. Dehumanization
 Is generalization a  If you think this is a
problem? problem, why do you
 Computer-scored and think so? What
administered evidence do you have
psychological tests that this hurts people?
might determine  What alternative
important aspects of approach would you
your life use?
 31
2. Access to psychological testing
 We’ve considered  WAIS-III kit costs
possibility of being $775 per person – for
hurt by tests – but tester to buy the kit.
what about possible Cost must be passed
benefits of tests? Who
gets those benefits? on to someone. Who
should that be?
 32
2. Access to psychological testing
 Psychological services  Is that just turf
are not covered by protection?
OHIP; Psychiatric  There is rivalry
services are between the
 Psychiatrists have professions – but we
opposed extending could make a stronger
OHIP coverage to case for billing OHIP
psychological testing if we used only
actuarial judgment
 33
Two examples of bad tests
1. Graphology (handwriting analysis)
2. TOEFL
 34
Graphology (handwriting analysis)
 Used in hiring  Simner & Goffin
decisions in business (2003): very little
 Used to make evidence of validity
assessments of for graphology
personality type and to  studies using non-
predict on-the-job autobiographical
performance writing are the most
useful and show the
least evidence of
validity.
 35
Graphology (handwriting analysis)
 Dean (1992) meta-analysis  Correlation drops to .09
of handwriting studies when using non-
showed an average autobiographical writing.
correlation of .16 between
graphological judgment
and job performance, using
autobiographical writing.
 36
Graphology (handwriting analysis)
 Simner & Goffin: both  Thus, GMA testing
general mental ability and interviewing are
testing and structured much more likely to
employment lead to sensible hiring
interviews correlate decisions.
around .50 with job
performance. When
used together, the two
correlate .63 with job
performance.
 37
Test of English as a Foreign Language

 Widely used to select  TOEFL manual

among students who presents a distorted
are not native speakers picture of validity
of English for studies
admission to Canadian  Simner (1998)
universities. presents CPA’s
recommendation –
don’t use TOEFL for
university admission
decisions
 38
TOEFL – Criterion Validity
 TOEFL manual leaves  TOEFL manual leaves
out studies that show out studies that show
low to moderate low to moderate
correlations between correlations between
TOEFL scores and TOEFL scores and
teachers’ ratings of oral interviews (.30
English language to .60)
proficiency (.34 to .
55)
 39
TOEFL – Criterion validity
 Generally, these  Simner: some criterion
studies assess validity tests for
concurrent validity, TOEFL involve the
not more relevant Michigan TELP, very
predictive validity similar to TOEFL – so
this is really parallel
forms reliability, not
validity evidence
 40
TOEFL – Construct validity
 ETS: Native speakers  Non-natives were
of English do much unselected examinees
better than non-natives from a range of
on TOEFL. backgrounds.
 Simner: in these  Natives were selected
studies, native from among entering
speakers are a more freshmen
highly-selected group.  Test anxiety
differences
 41
TOEFL – Construct validity
 Unpublished ETS  If native English
study: 5000 English speakers can succeed
speakers had an with scores < 600 why
average score of 590, can’t non-natives also
below the 600 cutoff succeed with scores
used for non-natives at below 600?
many Canadian
universities.
 42
Test of English as a Foreign Language
 Simner: Since 1995, 10  Taylor-Russell tables: r = .
Ontario universities have 20, now 10% accepted,
raised their TOEFL cutoff before 30%
from around 550 to 600.  Net gain in percentage
This means that only the likely to be successful is
top 10% of examinees are 4%, at a cost of rejecting
accepted, instead of top an additional 20%.
30%.
TOEFL: Simner’s 43

recommendations
 make admissions  consider the kinds and
decisions on the basis levels of English
of all relevant proficiency required in
information, not just different fields and
TOEFL scores levels of study
 do not use rigid cutoff  consider resources
scores available for improving
English language skills
 44
CPA Code of Ethics
 The following slides show the four basic principles
advanced in the CPA Code of Ethics. All of the
material on the next 15 slides is quoted directly from
that Code.

 There is significantly more information on each

principle at this web address: http://
www.cpa.ca/cpasite/userfiles/Documents/Canadian%20
Code%20of%20Ethics%20for%20Psycho.pdf

 (Note: scroll down if the page looks blank)

 45
CPA Principle I: Respect for the Dignity of Persons.

 In the course of their work as scientists, practitioners,

or scientist-practitioners, psychologists come into
contact with many different individuals and groups,
including: research participants; clients seeking help
with individual, family, organizational, industrial, or
community issues; students; trainees; supervisees;
employees; business partners; business competitors;
colleagues; employers; third party payers; and, the
general public…
 46
CPA Principle I: Respect for the Dignity of Persons.

 In these contacts, psychologists accept as

fundamental the principle of respect for the dignity
of persons; that is, the belief that each person should
be treated primarily as a person or an end in
him/herself, not as an object or a means to an end…
 47
CPA Principle II: Responsible Caring.
 A basic ethical expectation of any discipline is that its
activities will benefit members of society or, at least, do
no harm. Therefore, psychologists demonstrate an
active concern for the welfare of any individual, family,
group, or community with whom they relate in their
role as psychologists. This concern includes both those
directly involved and those indirectly involved in their
activities.
 48
CPA Principle II: Responsible Caring.
 However, as with Principle I, psychologists’ greatest
responsibility is to protect the welfare of those in the
most vulnerable position. Normally, persons directly
involved in their activities (e.g., research participants,
clients, students) are in such a position. Psychologists’
responsibility to those indirectly involved (e.g.,
employers, third party payers, the general public)
normally is secondary.
CPA Principle III: Integrity in 49

Relationships.
 Psychologists are expected to demonstrate the highest
integrity in all of their relationships. However, in rare
circumstances, values such as openness and
straightforwardness might need to be subordinated to
the values contained in the Principles of Respect for the
Dignity of Persons and Responsible Caring.
CPA Principle III: Integrity in 50

Relationships.
 The relationships formed by psychologists in the course of
their work embody explicit and implicit mutual expectations of
integrity that are vital to the advancement of scientific
knowledge and to the maintenance of public confidence in the
discipline of psychology. These expectations include: accuracy
and honesty; straightforwardness and openness; the
maximization of objectivity and minimization of bias; and,
avoidance of conflicts of interest. Psychologists have a
responsibility to meet these expectations and to encourage
reciprocity.
CPA Principle III: Integrity in 51

Relationships.
 In addition to accuracy, honesty, and the obvious
prohibitions of fraud or misrepresentation, meeting
expectations of integrity is enhanced by self-knowledge
and the use of critical analysis.
 CPA Principle IV: Responsibility to 52

Society.

 Psychology functions as a discipline within the

context of human society. Psychologists, both in their
work and as private citizens, have responsibilities to the
societies in which they live and work, such as the
neighborhood or city, and to the welfare of all human
beings in those societies.
 CPA Principle IV: Responsibility to 53

Society.

 Two of the legitimate expectations of psychology as a

science and a profession are that it will increase
knowledge and that it will conduct its affairs in such
ways that it will promote the welfare of all human
beings.
 CPA Principle IV: Responsibility to 54

Society.

 Freedom of enquiry and debate (including scientific

and academic freedom) is a foundation of psychological
education, science, and practice. In the context of
society, the above expectations imply that psychologists
will exercise this freedom through the use of activities
and methods that are consistent with ethical
requirements.
 CPA Principle IV: Responsibility to 55

Society.
 The above expectations also imply that psychologists
will do whatever they can to ensure that psychological
knowledge, when used in the development of social
structures and policies, will be used for beneficial
purposes, and that the discipline’s own structures and
policies will support those beneficial purposes. Within
the context of this document, social structures and
policies that have beneficial purposes are defined as
those that more readily support and reflect respect for
the dignity of persons, responsible caring, integrity in
relationships, and responsibility to society.
CPA Principle IV: Responsibility to 56

Society.
 If psychological knowledge or structures are used against
these purposes, psychologists have an ethical responsibility to
try to draw attention to and correct the misuse. Although this
is a collective responsibility, those psychologists having direct
involvement in the structures of the discipline, in social
development, or in the theoretical or research data base that is
being used (e.g., through research, expert testimony, or policy
advice) have the greatest responsibility to act. Other
psychologists must decide for themselves the most
appropriate and beneficial use of their time and talents to help
meet this collective responsibility.
 CPA Principle IV: Responsibility to 57

Society.

 When a person’s welfare appears to conflict with

benefits to society, it is often possible to find ways of
working for the benefit of society that do not violate
respect and responsible caring for the person. However,
if this is not possible, the dignity and well-being of a
person should not be sacrificed to a vision of the greater
good of society, and greater weight must be given to
respect and responsible caring for the person.
 58
CPA’s Caveat…
 Even with the above ordering of the principles,
psychologists will be faced with ethical dilemmas
that are difficult to resolve. In these circumstances,
psychologists are expected to engage in an ethical
decision-making process that is explicit enough to
bear public scrutiny. In some cases, resolution might
be a matter of personal conscience.
 59
CPA’s Caveat…
 However, decisions of personal conscience are also
expected to be the result of a decision-making
process that is based on a reasonably coherent set of
ethical principles and that can bear public scrutiny. If
the psychologist can demonstrate that every
reasonable effort was made to apply the ethical
principles of this Code and resolution of the conflict
has had to depend on the personal conscience of the
psychologist, such a psychologist would be deemed
to have followed this Code.