Available online at www.sciencedirect.

com

Procedia - Social and Behavioral Sciences 83 (2013) 30 – 34

2nd World Conference on Educational Technology Researches – WCETR2012

Students’ Rating: Is it a Measure of an Effective Teaching or Best

Gauge of Learning?

Shihab Jimaa
Khalifa University of Science, Technology, and Research, PO Box 573, Sharjah Campus, U. A. E.

Abstract

This paper discusses the issue of how students’ rating could reflect on the effectiveness of teaching or gauge students’ learning.
Student ratings of instruction are not the best gauge of learning. To boost student ratings we have created an evidence of their
learning. Ratings of overall teaching effectiveness are moderately correlated with independent measures of student learning and
achievement. At one time, it was considered contentious to administer student rating forms. Now such forms have become
commonplace because it makes sense to survey students to find out what they think about their experiences in the class over the
term. Also new suggestions that reflect on students’ learning and creative thinking were proposed here and consequently students
rating form has been updated.
© 2013 The Authors. Published by Elsevier Ltd. Open access under CC BY-NC-ND license.
Selection and peer
Selection and/or review under
peer-review underresponsibility
responsibility of Prof.
of Prof. Dr. Dr. Hafize
Hafize KeserKeser.
Ankara University, Turkey
Keywords: Students rating, effective teaching, students feedback, learning;

1. Introduction

Student ratings remain a debate in the teaching world including faculty, students, administrators, and researchers
[1]. Although students rating can provide helpful and valid feedback, they unfortunately are not an entirely accurate
measure of instruction. As students do not take much time out to really think about teaching, and may even not take
this form of questions seriously by making comments about the teacher. Therefore student ratings are not really the
ideal method to evaluate student learning. Hence student ratings could be boosted by creating evidence of their
learning. One guiding principle for this paper is that feedback on teaching is helpful for any faculty member.
Feedback from students, colleagues, and other sources lets faculty know how a class went and where changes might
be needed or attempted. Also feedback is integral in letting concerned parties know the progress of a class, areas of
strength, weaknesses and those that need improvement. This doesn’t just apply to instructors whose performance is
shaky or weak, in fact even the best teachers can benefit from student feedback. There is always place to improve
and getting information from students can prove very helpful. However many seem to be against relying entirely on
student feedback to evaluate teaching methods and effectiveness, said Linda B. Nilson [2].

In fact these ratings are very good for measuring customer satisfaction, no doubt about that, but really, under the
best of circumstances, there’s only a mild correlation between student learning and an instructor’s rating.

* Corresponding Author: Shibab Jimaa Tel: +23 234799

Email:shibabjimma@yahoo.com

1877-0428 © 2013 The Authors. Published by Elsevier Ltd. Open access under CC BY-NC-ND license.
Selection and/or peer-review under responsibility of Prof. Dr. Hafize Keser Ankara University, Turkey
doi:10.1016/j.sbspro.2013.06.006
 Shihab Jimaa / Procedia - Social and Behavioral Sciences 83 (2013) 30 – 34 31

Shihab Jimaa, Tel.: +971-6-597-8849

E-mail address: saj@kustar.ac.ae
The Students’ Evaluation of Educational Quality (SEEQ) feedback, with its proven reliability and validity,
provides a useful platform for beginning faculty development efforts [3]. SEEQ are increasingly being used in
universities throughout the world to assess effective teaching for purposes of improving teaching, and quality
assurance exercises. Many universities around the world are now using some forms of students’ questionnaire
feedback. It has proven to be a reliable, and a good foundation for developing faculty. It is also being used to assess
teaching quality. Some universities design their own questionnaires. In this paper the proposed questionnaire will be
named as Students’ Evaluation of Teaching Effectiveness (SETE) as the focus here is mainly on the teaching
effectiveness and its impact on the students’ learning .

Many seem to believe that faculty who is good in research must be good in teaching, and according to a large
meta-analysis [4] this is not the case as teaching and research productivity are not linked. University lecturers are
expected to be good teachers and good researchers, but receive little teacher training. Hence, it is not surprising that
many lecturers lack confidence about their teaching effectiveness and do not know how to use SETE feedback to
improve their teaching effectiveness. Relying on students rating only will not be effective. Rather constant
monitoring of teaching is required. Faculty and students need to realize that as useful and as effective as student
ratings maybe there are also difficulties in the SETE data. This paper explores how to evaluate more effectively the
student’s evaluations of teaching. Also this paper will explore surveys which have gathered feedback from both
graduate and undergraduate students regarding the impact of learning, teaching methods, and students creative
thinking on their ratings. The surveys will show how best to design a student questionnaire which will obtain the
best possible feedback from students. The paper will provide a background on the history of research regarding
student ratings, and also strategies for the best way to use the feedback to better teaching techniques.

Finally, a version of the students’ rating form that is capable of measuring teaching effectiveness and
students’ learning response to assessment environments, independent learning, and critical thinking will be
developed. Also a rating form will be composed which measures the effectiveness of teaching and the students'
response to environments where they are assessed.

2. Background History

At one time, it was considered controversial to administer student rating forms. Now such forms have become
conventional because it makes sense to survey students to find out what they think about their experiences in the
class over the term [6]. Student rating forms have become an integral part of the teaching process. It is now seen as
important to survey students to obtain feedback regarding their student experience in class. Research has supported
this and concludes that questionnaires are a reliable method.

Ratings of overall teaching effectiveness are reasonably associated with independent measures of student learning
and achievement [7], [8], and [10]. There is a clear correlation between independent measures of students learning
and effectiveness of teachings. Students who are taught by teachers who have been given a high rating are likely to
achieve better grades, and can utilize the course material effectively.

An instructor's rating for a given course tend to be relatively consistent over successive years [8] -[10]. Rating of
the instructor is usually constant for each course, it does not change much. The instructor usually gets similar
feedback over the years. Also student’s rating is not affected by any other factors related to the student such as
his/her age, year of study, intelligence, or GPA (grade point average).

Researchers are reporting the following relationships [10], [11]:

 Students tend to rate courses in their major fields and elective courses higher than required courses outside
their majors.
32 Shihab Jimaa / Procedia - Social and Behavioral Sciences 83 (2013) 30 – 34

 Faculty tends to receive more positive ratings than graduate student instructors.
 The gender of a student has little effect on ratings. The gender of an instructor, however, may have an
impact. Though some studies report no relationship between a professor's gender and student ratings, others
show that adhering to a gender-appropriate teaching style may be rewarded by higher evaluations.
 Ratings can be influenced by class size (very small classes tend to receive higher ratings), by discipline
(humanities instructors tend to receive higher ratings than engineering instructors), and by type (discussion
courses tend to receive higher ratings than lecture courses).
 Students' expectations affect their ratings: students who expect a course or teacher to be good generally find
their expectations confirmed.

3. Development of SETE

Student rating forms, also called student end-of-course questionnaires or student evaluation forms are
traditionally administered at the end of each term to solicit student evaluations of a course. Typically such end- of-
course information is used by managements to make comments and suggestions regarding the instructors’ teaching
experience and the course delivery [12].

The proposal of Students’ Evaluation of Teaching Effectiveness (SETE) is developed based on the Students’
Evaluation of Educational Quality (SEEQ) that was developed by Herbert Marsh [3]. Marsh and others have used
SEEQ data to explore the many issues that have characterized the past decades of student ratings research:
reliability, validity, and stability of results; sources of bias in the responses; the utility of ratings in administrative
decisions (summative evaluation); and the usefulness of the ratings for improving teaching (formative evaluation).
Over the course of these studies, as in any academic field, debates and disagreements have surfaced concerning
SEEQ. Some understanding of the development of SEEQ and these subsequent debates is needed for the most
reasoned and effective use of student rating's feedback.

The development of SEEQ begins with the question, “What is effective teaching?” Marsh and his colleagues [3],
[4], and [10] found considerable agreement in the idea that effective teaching is comprised of a definable set of
independent elements. These elements include specific behaviors, such as “expressive speaking” or “highlights
important points,” and the general characteristics perceived by students from these behaviors, such as “instructor
was dynamic and energetic” or “lecture facilitated note taking.” These elements are independent of one another in
the sense that a student’s judgment of a professor’s organizational skills might differ greatly from that student’s
judgment of the professor’s rapport with students. Marsh began by trying to identify the various attributes or factors
of good teaching and to define specific questions about those factors.

Many general factors emerged from extensive statistical studies of how students rated the importance of items for
evaluating teaching and how professors rated item’s usefulness as a feedback. These are Learning/Value,
Enthusiasm, Organization, Group Interaction, Individual Rapport, Breadth, Exams, Assignments, and Workload
[13]. But surely teaching is too complex to be summarized by these variables. Also there are some attributes that do
not appear in SEEQ, as different groups may emphasize different attributes (e.g., students might emphasize a
professor’s helpfulness, while professors might place more emphasis on encouraging self-guidance. The areas, and
even specific elements, of agreement among SETE and other studies suggest a strong consensus across the various
constituent groups in higher education as to the major variables describing effective teaching.

4. SETE Proposed form

In this section an updated SETE form is developed which is based on the form that has been developed as in [13].
The updated form examines students’ knowledge on the impact of their independent learning, creative thinking, and
instructor’s teaching effectiveness of their rating. The proposed form, shown in Table I, has nine categories: Learning,
 Shihab Jimaa / Procedia - Social and Behavioral Sciences 83 (2013) 30 – 34 33

Enthusiasm, Organization, group Interaction, Individual Rapport, Breadth, Examinations, Assignments, and an
Overall category. Students’ Evaluation of Teaching Effectiveness is a paper or electronic questionnaire, which
requires a written or selected response answer to a series of questions in order to evaluate the instruction of a given
course. They are a means to produce useful feedback which the instructor and the school can use to improve their
quality of instruction. The process of (i) gathering information about the impact of learning and of teaching practice
on student learning, (ii) analyzing and interpreting this information, and (iii) responding to and acting on the results, is
valuable for several reasons [14]. They are beneficial because instructors can review how others interpret their
teaching methods, thereby improving their instruction. The information can be also used by administrators, along with
other input, to make formative recommendations (e.g., identify areas where a faculty member needs to improve) [15].
Typically, these evaluations are combined with peer evaluations, chair evaluations, and results of student’s test scores
to create an overall picture of teaching performance. Questioning skills are essential to good teaching. Teachers often
use questions to ensure that students are attentive and engaged, and to assess students’ understanding. What is
important to note is that in addition to the intent of the question, the question itself matters.

5. Conclusion

The efficiency of student rating of teaching effectiveness is debatable; however what we can conclude is that it
has its benefits. Obtaining student feedback on their learning process, critical thinking, and independent learning
means that instructors can monitor their teaching process, and improve upon any areas that need improving or help
further explore techniques to increase student learning. Saying this however it should not be a sole measure of the
level of teaching. Despite that students’ rating is not the best gauge of learning and it should not be considered as the
sole measure of effective teaching, the developed form shown in Table I of the students’ evaluation of teaching
effectiveness (SETE) gives evidence of students’ learning and is a mean by which we can obtain information
regarding the extent of students learning, and the effectiveness of the particular teaching methods of their thought
process and independent learning.

References

[1] M. L. Lawall, “Students Rating Teaching”, Centre for Higher Education Research and Development, UTS, March 1998.
[2] Linda B. Nilson, “Teaching at Its Best: A Research-Based Resource for College Instructors, Jossey-Bass, 2010.
[3] Marsh, H. W. “ Students’ evaluations of university teaching: A multidimensional perspective”, In R. P. Perry & J C. Smart (Ed.), The
scholarship of teaching and learning in higher education: An evidence-based perspective”, pp.319-384., New York: Springer, 2007.
[4] Marsh, H. W., Rowe, K., & Martin, A., “ PhD students’ evaluations of research supervision: Issues, complexities and challenges in a
nationwide Australian experiment in benchmarking universities”, Journal of Higher Education, 73 (3), 313-348, 2002.
[5] Gibbs, G., Simpson, C. & Macdonald, R.” Improving student learning through changing assessment - a conceptual and practical
framework. European Association for Research into Learning and Instruction Conference, Padova, Italy, 2003.
[6] Davis, B. G. Sourcebook for Evaluating Teaching. Berkeley: Office of Educational Development, University of California, 1988.
[7] Abrami, R C., Apollonia, S., and Cohen, P. A. "Validity of Student Ratings of Instruction: What We Know and What We Do Not." Journal
of Educational Psychology, 1990, 82(2), 219-231.Y.
[8] Braskamp, L. A., Brandenburg, D. C., and Ory, J. C. Evaluating Teaching Effectiveness: A Practical Guide. Newbury Park, CA: Sage,
1984.
[9] McMillan, J. H., Wergin, J. E, Forsyth, D. R., and Brown, J. C. "Student Ratings of Instruction: A Summary of Literature." Instructional
Evaluation, 9(l), 2-9, 1986.
[10] Marsh, H. W. and Dunkin, M. J., “ Student Evaluations of University Teaching: A Multidimentional Perspective”, In J. C. Smart (ed.),
Higher Education: A Handbook of Theory and Research, vol. 8, New York, Agathon Press, 1992.
[11] Kierstead, D., D’Agostin, P., and Dill, W,m” Sex Role Sterotyping of College Professors: Bias in Students’ Rating of Instructors”, Journal
of Educational Psychology, 80(3), 342-344, 1988,
[12] Davis, B. G., “ Tools for Teaching”, Jossey-Bass Publisher: San Francisco, 1993.
[13] Lawall, M. L.,”Students Rating Teaching”, University Teaching Services, Centre for Higher Education Research and Development, The
University of Manitoba, 1998.
[14] Rahman, K., “Learning from your business lectures: using stepwise regression to understand course evaluation data”, Journal of American
Academy of Business, Cambridge, 19(2), 272–279, 2006.
[15] Dunegan, K. J., & Hrivnak, M. W., “Characteristics of mindless teaching evaluations and the moderating effects of image compatibility”,
Journal of Management Education, 27(3), 280–303, 2003.
34 Shihab Jimaa / Procedia - Social and Behavioral Sciences 83 (2013) 30 – 34

TABLE I. THE DEVELOPED FORM OF STUDENTS’ EVALUATION OF TEACHING EFFECTIVENESS

College:
Department:
Course:

Please tick one box only against each statement. sa a n d sd n/a

LEARNING
1 I benefited a great deal from the course content and understood the subject material of this course
2 I have found the course intellectually challenging and stimulating
3 My interest in the subject has increased as a consequence of this course
4 I believe I have attained the learning outcomes of the course
5 The laboratory work/assignments helped me attain the learning outcomes of the course
6 The course notes and text book helped me attain the learning outcomes.
ENTHUSIASM
7 Instructor spoke clearly and was enthusiastic about teaching the course
8 Instructor’s style of presentation held my interest during class
9 Instructor encouraged questions in the class
10 Instructor support critical thinking and independent learning
11 Instructor explained and helped in solving the tutorial questions.
12 Instructor asks questions that tap higher level thinking
13 Instructor recognized which students did not understand and reviewed as needed
ORGANIZATION
14 Instructor gave lectures which facilitated taking notes
15 Instructor’s explanations were clear
16 Instructor’s materials were well prepared and carefully explained
17 Instructor was available for help during his/her office hours
18 Instructors started and ended their lectures / tutorial on time
GROUP INTERACTION
19 Students were encouraged to participate in class discussions
20 Students were encouraged to share their ideas and knowledge with others
21 Students encouraged to work in groups
INDIVIDUAL RAPPORT
22 Instructor handled student discipline fairly
23 Instructor sufficiently available to students during office hours or after class
24 Instructor had a sincere interest in individual students
25 Instructor appeared to be genuinely concerned about students and their success in class
BREADTH
26 Instructor covered all the course syllabus in the time available
27 Instructor discussed all the objectives and learning outcomes and what expected from students at the
start of the course
28 Instructor adequately discussed current developments in the field
EXAMINATIONS
29 Examinations papers were clearly written and tested course content as stressed by the instructor
30 Methods of evaluating student work were fair and appropriate
31 Feedback on examinations / tests was timely and valuable
ASSIGNMENTS
32 Assignments, quizzes, and homework were adequate and contributed to appreciation and understanding
of subjects
33 Types of assignments were formative and further enhanced the learning outcomes of the course
OVERALL
34 As an overall rating, I would say I am very satisfied with the instructor
35 As an overall rating, I would say I am very satisfied with the outcomes of this course

sa → strongly agree a → agree n → neutral d → disagree sd → strongly disagree n/a → not applicable