Scribd
Upload
257 views23 pages

Alternative Approaches To Evaluation

This document discusses alternative approaches to evaluation. It begins by defining evaluation and describing how formal and informal evaluations differ. It then discusses two major conceptual approaches to evaluation - objectives-oriented and management-oriented approaches. The objectives-oriented approach focuses on measuring goals and objectives, while the management-oriented approach aims to serve the informational needs of decision makers. The document also describes several models that fall under these approaches, including Tyler's objectives model, goal-free evaluation, and Stufflebeam's CIPP model. It concludes by discussing the pros and cons as well as uses of these different evaluation approaches.

Uploaded by

Dzulkifli Awang
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
257 views23 pages

Alternative Approaches To Evaluation

This document discusses alternative approaches to evaluation. It begins by defining evaluation and describing how formal and informal evaluations differ. It then discusses two major conceptual approaches to evaluation - objectives-oriented and management-oriented approaches. The objectives-oriented approach focuses on measuring goals and objectives, while the management-oriented approach aims to serve the informational needs of decision makers. The document also describes several models that fall under these approaches, including Tyler's objectives model, goal-free evaluation, and Stufflebeam's CIPP model. It concludes by discussing the pros and cons as well as uses of these different evaluation approaches.

Uploaded by

Dzulkifli Awang
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 23

Alternative Approaches to

Evaluation
Dzulkifli Awang, PhD
Universiti Teknologi Malaysia
(courtesy of Dr. Mary Schutten, Western Michigan University)

Review: Introduction
Define

Evaluation
How do formal/informal evaluation differ?
What are two uses of evaluation in
education?
What are the pros/cons of using an
external evaluator?

Alternative Approaches
Stakeholders:
Stakeholders

individuals and groups who


have a direct interest in, and may be
affected by, evaluation; should be involved
early, actively & continuously

Program:
Program

activities that are provided on a


continuing basis; typically what is evaluated

There

are a variety of alternative, often


conflicting, views of what evaluation is and
how it should be carried out

Why so many alternatives?


The

way one views evaluation directly impacts


the type of activities/methods used

Origins

of alternative models stem from


differences in:
Philosophical & ideological beliefs
Methodological preferences
Practical choices

Philosophical & Ideological


Beliefs
Epistemologies (philosophies of knowing)

Objectivism (social science base of


empiricism; replicate)
Subjectivism (experientially-based; tacit
knowledge)
Pros/Cons of each?
Principles for assigning value (parallel obj/subj)
Utilitarian:
Utilitarian focus on group gains (avg scores);
greatest good for the greatest number
Intuitionist-pluralist:
Intuitionist-pluralist value is individuallydetermined
Room for both or are these dichotomous?
Philosophical purists are rare (impractical?)
Choose the methods right for THAT evaluation
Understand assumptions/limitations of
different approaches

Methodological
Preferences

Quantitative (numerical)

Qualitative (non-numerical)
Evaluation

is a transdiscipline; crosses

paradigms
Law of the instrument fallacy
With hammer/nails, all appears to need hammering

Identify

what is useful in each evaluation


approach, use it wisely & avoid being
distracted by approaches designed to
deal w/ different needs

Practical Considerations
Evaluators

disagree whether/not intent of


evaluation is to render a value judgment
Decision-makers or evaluator render
judgment?

Evaluators

differ in views of evaluations political

role
Authority? Responsibility? These dictate eval
style
Influence

of evaluators prior experience

Who

should conduct the evaluation and nature


of expertise needed to do so

Desirability

(?) of having a wide variety of


evaluation approaches

Classification Schema for


Evaluation Approaches
Conceptual approaches to evaluation, NOT
techniques
Objectives-oriented:
Objectives-oriented focus on goals/objectives &
degree to which they are achieved
Management-oriented:
Management-oriented identifying and meeting
informational needs of decision makers
Consumer-oriented:
Consumer-oriented generate information to
guide product/service use by consumers
Expertise-oriented:
Expertise-oriented use of professional expertise
to judge quality of evaluation object
Participant-oriented:
Participant-oriented stakeholders centrally
involved in process

Objectives-oriented
Approach

Purposes of some activity are specified and

then evaluation focuses on the extent to which


these purposes are achieved
Ralph W. Tyler popularized this approach in
education (criterion ref test)
Tylerian models
Metfessel & Michaels paradigm (enlarged
vision of alternative instruments to collect
evaluation data)
Provuss Discrepancy Evaluation Model (agree
on stds, det if discrepancy exists btwn
perf/std, use discrepancy info to decide to
improve, maintain, terminate program)
Logic models
Determine long-term outcomes & backtrack
to today

Objectives-oriented Steps
Establish

broad goals or objectives tied


to mission statement
Classify the goals or objectives
Define objectives in behavioral terms
Find situations where achievement of
objectives can be shown
Select/develop measurement
techniques
Collect performance data
Compare data with behaviorally stated
objectives

Objectives-oriented
Pros/Cons
Strengths:
Strengths

simplicity, easy to understand, follow and


implement; produces information relevant to the
mission

Weakness:
Weakness

can lead to tunnel vision


Ignores outcomes not covered by objectives
Neglects the value of the objectives themselves
Neglects the context in which evaluation takes
place.

Goal Free Evaluation


This is the opposite of objectives-oriented
evaluation, but the two supplement one
another
Purposefully avoid awareness of goals;
should not be taken as given, goals should be
evaluated
Predetermined goals not allowed to narrow
focus of evaluation study
Focus on actual outcomes rather than
intended
Evaluator has limited contact with program
manager and staff
Increases likelihood of seeing unintended
outcomes

Management-oriented
Approach
Geared

to serve decision makers


Identifies decisions administrator must make
Collects data re: +/- of each decision
alternative
Success based on teamwork between
evaluators and decision makers
Systems approach to education in which
decisions are made about inputs, processes,
and outputs
Decision maker is always the audience to
whom evaluation is directed

CIPP Evaluation Model


(Stufflebeam)
Context

Evaluation: planning decisions


Needs to address? Existing programs?
Input Evaluation: structuring decisions
Available resources, alternative strategies?
Process Evaluation: implementing decisions
How well is plan being implemented?
Barriers to success? Revisions needed?
Product Evaluation: recycling decisions
Results? Needs reduced? What to do after
program has run its course?

CIPP Steps
Focusing

the Evaluation
Collection of Information
Organization of Information
Analysis of Information
Reporting of Information
Administration of Evaluation
(timeline, staffing, budget etc)

Context Evaluation
Objective:

define institutional context, target


population and assess their needs

Method:

system analysis, survey, hearings,


interviews, diagnostic tests, Delphi technique
(experts)

For

deciding upon the setting to be served, the


goals associated with meeting needs and
objectives for solving problems

Input Evaluation
Objective:

identify and assess system


capabilities, procedural designs for
implementing the strategies, budgets,
schedules

Method:

inventory human and material


resources, feasibility, economics via literature
review, visit exemplary programs

For

selecting sources of support, solution


strategies in order to structure change
activities, provide basis to judge
implementation

Process Evaluation
Objective:

identify or predict defects in the


process or procedural design, record/judge
procedural events

Method:

monitoring potential procedural


barriers, continual interaction with and
observation of the activities of the staff

For

implementing and refining the program


design and procedure (a.k.a., process
control)

Product Evaluation
Objective:

collect descriptions and judgments of


outcomes and relate them to CIP, interpret
worth/merit

Methods:

measure outcomes, collect


stakeholder information, analyses of data

For

deciding to continue, terminate, modify, or


refocus an activity and to document the effects
(whether intended or unintended)

Uses of Management-oriented
Approaches to Evaluation
CIPP

has been used in school districts, state


and federal government agencies
Useful guide for program improvement
Accountability
Figure 5.1 (p. 94)
Formative and summative aspects of CIPP

Management-oriented
Pros/Cons
Strengths:
Strengths

appealing to many who like


rational, orderly approaches, gives focus
to the evaluation, allows for formative and
summative evaluation

Weaknesses:
Weaknesses

preference given to top


management, can be costly and complex,
assumes important decisions can be
identified in advance of the evaluation

REVIEW/Qs
Why

are there so many alternative approaches


to evaluation?
What two conceptual approaches to evaluation
did we discuss tonight? What are their +/-?
Which, if either, of these approaches do you
think will work for your evaluation object?

Identify

your most likely evaluation object

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy