HMEF5093

QUANTITATIVE
RESEARCH
METHODOLOGY
Prof Dr John Arul Phillips

Copyright © Open University Malaysia (OUM)

Project Directors: Prof Dato’ Dr Mansor Fadzil
Assoc Prof Dr Chung Han Tek
Open University Malaysia

Module Writer: Prof Dr John Arul Phillips

Moderator: Prof Dr Kuldip Kaur

Open University Malaysia

Adapted by: Assoc Prof Dr Chung Han Tek

Assoc Prof Dr Nagarajah Lee Hun Leong
Open University Malaysia

Developed by: Centre for Instructional Design and Technology

Open University Malaysia

Printed by: Meteor Doc Sdn Bhd

Lot 47-48, Jalan SR 1/9, Seksyen 9,
Jalan Serdang Raya, Taman Serdang Raya,
43300 Seri Kembangan, Selangor Darul Ehsan

First Edition, August 2009

Second Edition, April 2013 (rs)
Copyright © Open University Malaysia (OUM), April 2013, HMEF5093
All rights reserved. No part of this work may be reproduced in any form or by any means
without the written permission of the President, Open University Malaysia (OUM).

Copyright © Open University Malaysia (OUM)

Table of Contents
Course Guide ix-xvii
Course Assignment Guide xxi-xxiii

Topic 1 The Educational Research Process 1

1.1 Research Defined 2
1.2 Characteristics of Research 3
1.3 What is Educational Research? 4
1.4 The Research Process 5
1.4.1 Generate Research Ideas 6
1.4.2 Formulate a Research Problem 7
1.4.3 Develop Hypotheses or Research Questions 9
1.4.4 Design Study to Answer Research Questions or 10
Test Hypotheses
1.4.5 Collect Data 11
1.4.6 Analyse and Interpret Data 11
1.4.7 Communicate Results 12
1.5 Differences between Quantitative and Qualitative Research 12
in Education
Summary 16
Key Terms 17

Topic 2 Theory and Review of Literature 18

2.1 What is Theory? 19
2.2 Confirming or Disconfirming a Theory 20
2.3 What is the Review of Literature? 22
2.4 The Importance of the Review of Literature 23
2.5 Steps in Developing a Literature Review 24
2.6 Common Weaknesses 27
2.7 Sources 28
2.8 Evaluating Search Articles 30
Summary 33
Key Terms 34

Topic 3 Constructs, Variables and Data Types 35

3.1 Constructs and Variables 35
3.2 Types of Variables 39
3.3 Operational Definition of Variables 43

Copyright © Open University Malaysia (OUM)

iv X TABLE OF CONTENTS

3.4 Data Types 44

3.4.1 Nominal Data 45
3.4.2 Ordinal Data 45
3.4.3 Interval Data 47
3.4.4 Ratio Data 47
Summary 48
Key Terms 49

Topic 4 Experimental Research Methodology 50

4.1 The Experimental Method 51
4.2 Extraneous Variables to be Controlled to Enhance Internal 53
Validity of Experiments
4.2.1 Time Interval and Threats to Interval Validity 54
4.2.2 Other Threats to Interval Validity 55
4.2.3 Subject-Experimenter Effects to be Controlled 55
4.3 Random Assignment to Enhance Internal Validity 57
4.4 Other Techniques to Ensure Groups are Equivalent 59
4.4.1 Matching 59
4.4.2 Holding One or More Variables Constant 59
4.4.3 Including an Extraneous Variable in the 60
Research Design
4.4.4 Analysis of Covariance (ANCOVA) 61
4.5 Hypothesis Testing 61
4.5.1 Directional Hypothesis 62
4.5.2 Null Hypothesis 62
4.6 Test of Significance 63
Summary 65
Key Terms 66

Topic 5 Experimental Research Designs 67

5.1 Symbols Used in Experimental Research Designs 68
5.2 Weak Designs 68
5.2.1 One-Shot Design 68
5.2.2 One-Group Pre-Test and Post-Test Design 69
5.2.3 Non-Equivalent Post Test Only 70
5.3 True Experimental Designs 71
5.3.1 After-Only Research Design 72
5.3.2 Factorial Research Design 73
5.3.3 Before-After Research Design 79

Copyright © Open University Malaysia (OUM)

 TABLE OF CONTENTS W v

5.4 Quasi-Experimental Design 80

5.4.1 Non-Equivalent Control-group Design 80
5.4.2 Interrupted Time Series Design 81
5.5 Ethics in Experimental Research 83
Summary 85
Key Terms 86

Topic 6 Survey Research Methodology 87

6.1 What is a Survey? 89
6.2 Types of Survey 90
6.2.1 Cross-Sectional Survey 90
6.2.2 Longitudinal Survey 91
6.3 Sampling 91
6.4 Sampling Techniques 93
6.4.1 Probability Sampling Techniques 93
6.4.2 Non-Probability Sampling Techniques 98
6.5 Sample Size 100
6.6 The Process of Conducting a Survey 101
6.7 Data Collection Methods Using Surveys 106
6.8 Ethics in Conducting a Survey 107
Summary 109
Key Terms 110

Topic 7 Instrumentation 111

7.1 The Questionnaire 113
7.2 Question Response Format 114
7.2.1 Structured or Closed Questions 114
7.2.2 Types of Structured Questions 115
7.2.3 Unstructured Questions 117
7.3 Guidelines on Questionnaire Design 118
7.4 Pilot Testing the Questionnaire 120
7.5 Designing an Attitude Test 121
Summary 125
Key Terms 125

Copyright © Open University Malaysia (OUM)

vi X TABLE OF CONTENTS

Topic 8 Quantitative Data Analysis I: Introduction to Descriptive 126

Statistics
8.1 Data Presentation 127
8.2 Measures of Central Tendency 135
8.3 Measures of Dispersion 139
Summary 144
Key Terms 145
Case Scenario 145

Topic 9 Quantitative Data Analysis II: Introduction to Inferential Statistics 146

9.1 Confidence Intervals 147
9.2 Statistical Inference with Confidence Interval 169
9.3 Hypothesis Testing 151
9.4 Statistical Inference with Hypothesis Testing 154
9.5 Parametric Versus Non-Parametric Test 157
Summary 162
Key Terms 163

Topic 10 Writing the Research Report 164

10.1 What is a Research Report? 165
10.2 Preliminary Section 167
10.3 The Body of a Research Report 169
10.4 Supporting Materials 176
Summary 177
Key Terms 177

References 178
Appendix 1 183
Appendix 2 184

Copyright © Open University Malaysia (OUM)

COURSE GUIDE

Copyright © Open University Malaysia (OUM)

viii X COURSE GUIDE

Copyright © Open University Malaysia (OUM)

 COURSE GUIDE W ix

WELCOME TO HMEF5093
Welcome to HMEF5093 Quantitative Research Methodology, which is one of the
required courses for the Master of Education (MEd) programme, Option 2. The
course assumes no previous knowledge and experience in educational research
but you are encouraged to tap into your experiences as a teacher, instructor,
lecturer or trainer, and relate them to the concepts and principles discussed. This
is a three-credit course conducted over a semester of 14 weeks.

WHAT WILL YOU GET FROM DOING THIS COURSE?

Description of the Course
The course begins with a discussion of the research process followed by
techniques in writing a review of literature. The relevance of theory in
quantitative research and the characteristics of quantitative research will also be
discussed. Among the quantitative methods emphasised are the experimental
research method and the survey method. Various kinds of experimental research
designs are introduced together with the procedure for conducting a survey. The
design and development of instruments and issues relating to reliability and
validity are discussed. The application of basic statistical techniques in
quantitative research is also introduced. The final part of the course focuses on
the writing of the research proposal based on the quantitative research paradigm.

Aim of the Course

The objective of the course is to provide you with a foundation on the principles
and practices of quantitative research and its application in the design of a
research project. Do read Appendix 2 on the scope and depth of the research that
is required of the MEd programme (Option 2).

Copyright © Open University Malaysia (OUM)

x X COURSE GUIDE

Objectives of the Course

Upon completion of the course, you should be able to:

1. Define and explain the educational research process;

2. Identify the characteristics of quantitative research;
3. Explain the concepts of constructs, variables and data types;
4. Discuss the importance of theory in quantitative research methods;
5. Critique quantitative research in education;
6. Write a review of literature;
7. Describe various quantitative research methods;
8. Explain basic statistical techniques in quantitative research; and
9. Prepare a research report.

HOW CAN YOU GET THE MOST FROM THIS COURSE?

Learning Package
In this Learning Package you, are provided with THREE kinds of courses
materials:
1. The Course Guide, you are currently reading;
2. The Course Content (consisting of 10 topics); and
3. The Course Assessment Guide (which describes the assignments to be
submitted and the examinations you have to sit for).
Please ensure that you have all of these materials.

Course Topics
To enable you to achieve the NINE objectives of the course, HMEF 5093 is
divided into 10 topics. Specific learning outcomes are stated at the start of each
topic indicating what you should be able to achieve after completing the topic.

Copyright © Open University Malaysia (OUM)

 COURSE GUIDE W xi

Topic 1: The Educational Research Process

The topic examines what is research and discusses the characteristics
of research. It then goes on to explain what educational research is
and describes the steps involved in the research process. The topic
ends with a discussion of the differences between quantitative and
qualitative research in education.

Topic 2: Theory and Review of Literature

The first part of the topic defines what theory is and the role of theory
in educational research. The second part deals with the role of
literature review in research and guidelines in writing a review.

Topic 3: Constructs, Variables and Data Types

The topic explains the concepts of constructs and variables in
educational research. It also deliberates on the types of data that are
normally collected in educational research.

Topic 4: Experimental Research Methodology

The topic introduces the concept of the experiment in educational
research, emphasising the issue of internal validity, random
assignment, hypothesis testing and test of significance.

Topic 5: Experimental Research Designs

The topic presents different types of weak designs, true experiments
and quasi-experiments. Focus is on the reasons for using a particular
design and the ethics involved in experiments using human subjects.

Topic 6: Survey Research Methodology

The topic examines the different types of surveys and the procedure
in conducting a survey. Sampling and generalisation of findings are
discussed together with the methods of data collection. The ethics of
conducting surveys is also discussed.

Topic 7: Instrumentation
The topic is devoted towards explaining the design and development
of instruments such as the survey questionnaire, attitude scales and
use of commercially available instruments. The issue of reliability and
validity of instruments is discussed.

Topic 8: Quantitative Data Analysis I: Introduction to Descriptive Statistics

The topic introduces the concept of descriptive statistics. Descriptive
statistics is concerned with summarising data in order to describe the
characteristics of the sample. The commonly used methods in
describing data are; presentation using charts and graphs, measure of
Copyright © Open University Malaysia (OUM)
xii X COURSE GUIDE

central tendency, and measure of dispersion. These form of

descriptive analysis in also known as exploratory data analysis (EDA).
This Topic will not delve into Descriptive Statistics in detail. You will
learn more about it in HMEF 5113 Statistics for Educational Research.

Topic 9: Quantitative Data Analysis II: Introduction to Inferential Statistics

The topic introduces the concept of inferential statistics. Unlike
descriptive statistics presented in the earlier topic, the emphasis of
inferential statistics is NOT the samples but is about making
generalisation from sample data to the population. This topic
discusses two ways of making inference from sample to population
that is, either estimating the population parameter from sample
statistics or testing the hypothesis. However, as in Topic 9, this topic
will not delve into Inferential Statistics in detail. You will learn more
about it in HMEF 5113 Statistics for Educational Research.

Topic 10: Writing the Research Report

The topic outlines the different chapters and sections of a research
report and provide suggestions on how to go about writing the
document.

Organisation of Course Content

In distance learning, the module replaces the university lecturer. This is one of
the main advantages of distance learning where our specially designed module
allows you to study at your own pace, anywhere and at anytime. Think of it as
reading the lecture instead of listening to a lecturer. In the same way that a
lecturer might assign something for you to read or do, the module tells you what
to read, when to read and when to do the activities. Just as a lecturer might
ask you questions in class, your module provides exercises for you to do at
appropriate points.

To help you read and understand the individual topics, numerous realistic
examples support all definitions, concepts and theories. Diagrams and text are
combined into a visually appealing, easy-to-read module. Throughout the course
content, diagrams, illustrations, tables and charts are used to reinforce important
points and simplify the more complex concepts. The module has adopted the
following features in each topic:

Copyright © Open University Malaysia (OUM)

 COURSE GUIDE W xiii

X INTRODUCTION
Lists the headings and subheadings of each topic to provide an overview of the
contents of the topic and prepare you for the major concepts to be studied and
learned.

LEARNING OUTCOMES
This is a listing of what you should be able to do after successful
completion of a topic. In other words, whether you are be able to explain,
compare, evaluate, distinguish, list, describe, relate and so forth. You
should use these indicators to guide your study. When you have finished
a topic, you must go back and check whether you have achieved the
learning outcomes or be able to do what is required of you. If you make a
habit of doing this, you will improve your chances of understanding the
contents of the course.

SELF-CHECK

Questions are interspersed at strategic points in the topic to encourage

review of what you have just read and retention of recently learned
material. The answers to these questions are found in the paragraphs
before the questions. This is to test immediately whether you
understand the few paragraphs of text you have read. Working through
these tests will help you determine whether you understand the topic
and prepare you for the assignments and the examination.

ACTIVITY

These are situations drawn from research projects to show how

knowledge of the principles of research methodology may be applied to
real-world situations. The activities illustrate key points and concepts
dealt with in each topic.

Copyright © Open University Malaysia (OUM)

xiv X COURSE GUIDE

The main ideas of each topic are listed in brief sentences to provide a review of
the content. You should ensure that you understand every statement listed. If
you do not, go back to the topic and find out what you do not know.

Key Terms discussed in the topic are placed at end of each topic to make you
aware of the main ideas. If you are unable to explain these terms, you should go
back to the topic to clarify.

DISCUSSION QUESTIONS:
At the end of each topic a list of questions is presented that are best solved
through group interaction and discussion. You can answer the questions
yourself. But, you are encouraged to work with your course-mates and discuss
online and during the seminar sessions.

At the end of each topic a list of articles and topics of books is provided that is
directly related to the contents of the topic. As far as possible the articles and
books suggested for further reading will be available in OUMÊs Digital Library
which you can access and OUMÊs Library. Also, relevant internet resources are
made available to enhance your understanding of selected curriculum concepts
and principles as applied in real-world situations.

Copyright © Open University Malaysia (OUM)

 COURSE GUIDE W xv

WHAT SUPPORT WILL YOU GET IN STUDYING THIS

COURSE?
Seminars
There are 15 hours of seminars or face-to-face interaction supporting the course.
This consists of FIVE seminar sessions of 3 hours each. You will be notified of the
dates, times and location of these tutorials, together with the names and phone
number of your facilitator, as soon as you are allocated a seminar group.

myVLE Online Discussion

Besides the face-to-face seminar sessions, you have the support of online
discussions. You should interact with other students and your facilitator using
myVLE. Your contributions to the online discussion will greatly enhance your
understanding of course content, how to go about doing the assignment and
preparation for the examinations.

Facilitator
Your facilitator will mark your assignments and provide assistance to you during
the course. Do not hesitate to discuss during the seminar sessions or online if:
Ć You do not understand any part of the course content or the assigned
readings
Ć You have difficulty with the self-tests and activities
Ć You have a question or problem with the assignments.

HOW SHOULD YOU STUDY FOR THIS COURSE?

1. Time Commitment for Studying
You should plan to spend about 6ă8 hours per topic, reading the notes,
doing the self-tests and activities and referring to the suggested readings. You
must schedule your time to discuss online. It is often more convenient you
distribute the hours over a number of days rather than spend one whole
day per week on study. Some topics may require more work than others,
although on average, it is suggested that you spend approximately three days
per topic.

Copyright © Open University Malaysia (OUM)

xvi X COURSE GUIDE

2. Proposed Study Strategy

The following is a proposed strategy for working through the course. If you
run into any trouble, discuss it with your tutor either online or during the
tutorial sessions. Remember the tutor is there to help you.
(a) The most important step is to read the contents of this Course Guide
thoroughly.
(b) Organise a study schedule. Note the time you are expected to spend
on each topic, date for submission of the assignment, tutorials and
examination dates. These are stated in your Course Assessment Guide.
Put all this information in one place, such as your diary or a wall
calendar. Whatever method you choose to use, you should decide on
and jot down your own dates for working on each topic. You have some
flexibility as there are nine topics spread over a period of 14 weeks.
(c) Once you have created your own study schedule, make every effort to
„stick to it‰. The main reason students are unable to cope is because
they are behind in their course work.
(d) To begin reading a topic:
• Remember in distance learning much of your time will be spent
READING the course content. Study the course map and examine
the relationship of the topic to the other eight topics.
• Read the reports at the beginning and try to figure out how it is
connected to the topic
• Read the topic overview showing the headings and subheadings to
get a broad picture of the topic.
• Read topic learning outcomes (what is expected of you). Do you
already know some of the things to be discussed? What are the
things you do not know?
• Read the introduction (see how it is connected with the previous
topic)
• Work through the topic. (The contents of the topic have been
arranged to provide a sequence for you to follow)
• As you work through the topic you will be asked to do the self-test
at appropriate points in the topic. This is to find out if you
understand what you have just read.
• Do the activities (to see if you can apply the concepts learned to real-
world situations)

Copyright © Open University Malaysia (OUM)

 COURSE GUIDE  xvii

(e) When have completed the topic, review the learning outcomes to confirm
that you have achieved them and able to do what is required.
(f) If you are confident, you can proceed to the next topic. Proceed topic by
topic through the course and try to pace your study so that you keep
yourself on schedule.
(g) After completing all topics, review the course and prepare yourself for
the final examination. Check that you have achieved all topic leaning
outcomes and the course objectives (listed in this Course Guide)

FINAL REMARKS
One again, welcome to the course. To maximise your gain from this course you
should try at all times relate what you are studying with the real-world. Look at
the environment in your institution and ask yourself whether they provide
opportunities for research. Most of the ideas, concepts and principles you learn in
this course have practical applications. It is important to realise that much we do
in education and training has to be based on sound theoretical foundations. The
contents of this course provide the principles for doing research in education
whether it be in a school, college, university or training organisation.

We wish you success with the course and hope that you will find it interesting,
useful and relevant towards your development as a professional. We hope you
enjoy your experience with OUM and we would like to end with this saying,
„Education is a lifetime of learning, relearning and unlearning‰ by Alvin Toffler
(an adaptation).

Copyright © Open University Malaysia (OUM)

xviii X COURSE GUIDE

Copyright © Open University Malaysia (OUM)

COURSE ASSIGNMENT
GUIDE

Copyright © Open University Malaysia (OUM)

xx X COURSE ASSIGNMENT GUIDE

Copyright © Open University Malaysia (OUM)

 COURSE ASSIGNMENT GUIDE W xxi

INTRODUCTION
This guide explains the basis on which you will be assessed in this course during
the semester. It contains details of the facilitator-marked assignments, final
examination and participation required for the course.

One element in the assessment strategy of the course is that, all students should
have the same information as facilitators about the answers to be assessed.
Therefore, this guide also contains the marking criteria that facilitators will use in
assessing your work.

Please read through the whole guide at the beginning of the course.

ACADEMIC WRITING
(a) Plagiarism

(i) What is Plagiarism?

Any written assignment (essays, project, take-home examinations, etc)
submitted by a student must not be deceptive regarding the abilities,
knowledge or amount of work contributed by the student. There are
many ways that this rule can be violated. Among them are:

Paraphrases A closely reasoned argument of an author is paraphrased but

the student does not acknowledge doing so. (Clearly, all our
knowledge is derived from somewhere, but detailed arguments
from clearly identifiable sources must be acknowledged.)
Outright Large sections of the paper are simply copied from other sources
plagiarism and the copied parts are not acknowledged as quotations.
Other These often include essays written by other students or sold
sources by unscrupulous organisations. Quoting from such papers is
perfectly legitimate if quotation marks are used and the source
is cited.
Works by Taking credit deliberately or not deliberately for works
others produced by others without giving proper acknowledgement.
These works include photographs, charts, graphs, drawings,
statistics, video clips, audio clips, verbal exchanges, such as
interviews or lectures, performances on television and texts
printed on the Web.
Duplication The student submits the same essay for two or more courses.

Copyright © Open University Malaysia (OUM)

xxii X COURSE ASSIGNMENT GUIDE

(ii) How Can I Avoid Plagiarism?

• Insert quotation marks around Âcopy and pasteÊ clause, phrase,

sentence, paragraph and cite the original source.

• Paraphrase clause, phrase, sentence or paragraph in your own

words and cite your source

• Adhere to the APA (American Psychological Association) stylistic

format, whichever applicable, when citing a source and when
writing out the bibliography or reference page

• Attempt to write independently without being overly dependent

on information from anotherÊs original works

• Educate yourself on what may be considered as common

knowledge (no copyright necessary), public domain (copyright
has expired or not protected under copyright law), or copyright
(legally protected).

(b) Documenting Sources

Whenever you quote, paraphrase, summarize or otherwise refer to the
work of others, you are required to cite its original source documentation.
Offered here are some of the most commonly cited forms of material.

• Direct Citation Simply having a thinking skill is no assurance

that children will use it. In order for such skills to
become part of day-to-day behaviour, they must
be cultivated in an environment that values and
sustains them. „Just as childrenÊs musical skills
will likely lay fallow in an environment that
doesnÊt encourage music, learnerÊs thinking
skills tend to languish in a culture that doesnÊt
encourage thinking‰ (Tishman, Perkins and Jay,
1995, p. 5)

• Indirect Citation According to Wurman (1988), the new disease of

the 21st century will be information anxiety,
which has been defined as the ever-widening gap
between what one understands and what one
thinks one should understand.

Copyright © Open University Malaysia (OUM)

 COURSE ASSIGNMENT GUIDE W xxiii

(c) Referencing
All sources that you cite in your paper should be listed in the Reference
section at the end of your paper. HereÊs how you should do your Reference.

Journal Article DuFour, R. (2002). The learning-centred principal.

Educational Leadership, 59(8). 12ă15.
Online Journal Evnine, S. J. (2001). The universality of logic: On the
connection between rationality and logical ability
[Electronic version]. Mind, 110, 335ă367.
Webpage National Park Service. (2003, February 11). Abraham
Lincoln Birthplace National Historic Site. Retrieved
February 13, 2003, from http://www.nps.gov/abli/
Book Naisbitt, J., & Aburdence, M. (1989). Megatrends 2000.
London: Pan Books.
Article in a Nickerson, R. (1987). Why teach thinking? In J. B. Baron,
Book & R. J. Sternberg (Eds). Teaching thinking skills:
Theory and practice. New York: W. H. Freeman and
Company. 27ă37.
Printed Holden, S. (1998, May 16). Frank Sinatra dies at 82:
Newspaper Matchless stylist of pop. The New York Times,
pp. A1, A22ăA23.

ASSESSMENT
Please refer to myVLE.

TAN SRI DR ABDULLAH SANUSI (TSDAS) DIGITAL

LIBRARY
The TSDAS Digital Library has a wide range of print and online resources for
the use of its learners. This comprehensive digital library, which is accessible
through the OUM portal, provides access to more than 30 online databases
comprising e-journals, e-theses, e-books and more. Examples of databases
available are EBSCOhost, ProQuest, SpringerLink, Books24x7, InfoSci Books,
Emerald Management Plus and Ebrary Electronic Books. As an OUM learner,
you are encouraged to make full use of the resources available through this
library.

Copyright © Open University Malaysia (OUM)

xxiv X COURSE ASSIGNMENT GUIDE

Copyright © Open University Malaysia (OUM)

Topic X The Educational
1 Research Process

LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Define what research is;
2. Explain the characteristics of research;
3. Define what educational research is;
4. Describe the steps involved in the research process; and
5. Differentiate between quantitative and qualitative research.

X INTRODUCTION
The word „research‰ has been used in many different ways, sometimes rather
loosely, giving rise to confusion, and at times with the intention to deceive. This
is illustrated in the following circumstances:

(i) „Research has shown that four out of five dentists interviewed used
Plantoid Herbal Toothpaste.‰ ă The claim that the majority of dentists used
the particular brand of toothpaste is misleading as there is no mention of
the total number of dentists interviewed and whether they were
representative of all dentists in the country.

(ii) „Years of research have shown that SlimTex Capsules reduce oneÊs weight
in a matter of weeks.‰ ă Research on the use of various types of medicine
and supplements by humans tends to be inconclusive. In relation to the
advertisement on slimming pills, it would be difficult to make a claim that
the pills reduce weight as there are many other contributory factors such as
gender, level of health and diet.

(iii) „Azlan does research by browsing through consumer reports, catalogues,

brochures and market surveys before deciding what car to buy‰ and „Kong
Copyright © Open University Malaysia (OUM)
2 X TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS

Beng surfs the Internet to do research on the symptoms of HIV/AIDS‰. ă In

the strictest sense of the term, what Azlan and Kong Beng are doing is not
research. Looking up facts and writing them down are nothing more than
fact finding and fact transcribing. There is even the tendency to call the
written work a "research report", which is not accurate. It may be a report
but it is not a research report.

There needs to be a distinction between true research and the accumulation of

facts. Research is a way of thinking; it involves thinking what we want to study,
how we go about collecting data, analysing the data and deriving conclusions.

ACTIVITY 1.1

Identify the different ways in which the word „research‰ has been used
in the statements above.

1.1 RESEARCH DEFINED

According to the WebsterÊs dictionary, a research is a diligent scientific search or
inquiry to discover facts. Kerlinger (1973) defines research as „the systematic,
controlled, empirical and critical investigation of natural phenomena guided by
theory and hypotheses about the presumed relations among the phenomena‰.
You will notice certain keywords in these definitions of research:

• Scientific • Controlled
• Systematic • Theory and hypotheses
• Empirical and critical investigation • Presumed relations
• Discover facts • Natural phenomena

Research is thus a scientific (or systematic) process of gathering information

about the hypothesised relations between phenomena, such as to investigate
whether there is a relationship between a studentÊs attitude towards mathematics
and his or her performance in mathematics. The scientific method, popularised
by John Dewey in 1933, lists the following steps:

(a) Formulate a hypothesis (a tentative statement about the relation between

two or more theoretical constructs such as attitude and mathematics
performance);

Copyright © Open University Malaysia (OUM)

 TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS W 3

(b) Test the hypothesis (design a study to establish whether the relationship
between the constructs is as hypothesised);

(c) Collect data (e.g. on attitude towards mathematics and mathematics

performance); and

(d) Decide to accept or reject the hypothesis (e.g. correlation between attitude
towards mathematics and mathematics performance).

The purpose of using the scientific method is to enable the researcher to describe
(the relations between factors); to predict (given what is known, we can predict
what might happen); to control (when certain variables are manipulated, does it
lead to a particular condition), and to explain (can a theory be formulated to
explain the phenomena being investigated).

1.2 CHARACTERISTICS OF RESEARCH

Research is a way of thinking. To qualify as a research, it needs to have certain
characteristics such as the following:

(a) Research begins with a question in the mind of the researcher.

You need only look around, and everywhere you see phenomena which
will arouse your curiosity. For example, why are children in a school unable
to read? Why are girls performing better than boys? These are situations
you may have difficulty comprehending. By asking relevant questions, we
create an inquisitive environment which is the prerequisite for research.
Research arises from a question that is intelligently asked about a
phenomenon that the researcher observes and is puzzling him or her.

(b) Research requires a plan.

One does not discover the truth or explanation about a phenomenon
without serious and meticulous planning. Research is not looking
something up in the hope of coming across the solution to your problem.
Rather, it entails a definite plan, direction and design.

(c) Research demands a clear statement of the problem.

Successful research begins with a clear, simple statement of the problem.
The statement of the problem should be stated precisely and in a
grammatically correct manner. It should also set forth what it seeks to
discover and enable one to see what one is attempting to research.

Copyright © Open University Malaysia (OUM)

4 X TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS

(d) Research deals with the main problem through sub-problems.

Divide the main problem into appropriate sub-problems, which, when
resolved, will result in the solution of the main research problem.

(e) Research seeks direction through appropriate research questions and

hypotheses.
Express the sub-problems in the form of research questions or hypotheses.
A research question or a hypothesis which states the relationship between
variables to be investigated is a logical supposition, a reasonable guess or
an educated conjecture which may give direction to thinking with respect
to the problem, and, thus, aid in solving it.

(f) Research deals with facts and their meaning.

Having defined the problem and the sub-problems and developed the
research questions and hypotheses, the next step is to collect whatever facts
are pertinent to the problem. The data collected needs to be organised into a
form that is potentially meaningful.

SELF-CHECK 1.1
1. Define research in your own words.
2. Identify the steps in gaining knowledge through the use of scientific
method.
3. What are some characteristics of research?

1.3 WHAT IS EDUCATIONAL RESEARCH?

What is educational research? Generally, an educational research is defined as a
research that investigates the behaviour of students, teachers, administrators,
parents and other members of the community who interact with educational
institutions. The word „behaviour‰ is taken broadly to mean such phenomena as
learning, attitudes, aptitudes, abilities, interests, practices, processes and
emotions.

What is the purpose of educational research? Since education is fundamentally

the development of individuals, the central purpose of educational research is to
find ways to improve student learning. It has been argued that educational
research without this as its ultimate motivation and objective is not educational
research. Anyone who is engaged in a systematic search of ways to improve
student learning is doing educational research. For example, a classroom teacher
experimenting with alternative ways of explaining the laws of physics and a full-

Copyright © Open University Malaysia (OUM)

 TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS W 5

time researcher comparing the effectiveness of different reading methods in early

literacy programmes are both engaged in finding ways to improve student
learning, and, in this sense, are engaged in educational research. The leadership
styles of the school principal will influence teacher morale and job satisfaction,
which will affect how teachers play their role in the classroom. Thus, the
approach adopted by school principals and headmasters can influence student
learning and attitudes.

ACTIVITY 1.2

1. Do you agree with the statement „⁄the central purpose of

educational research is to find ways to improve student learning‰?
2. Give some examples of findings from educational research in your
area of interest.
3. A graduate student is conducting a research on the effects of ISO
implementation on the work performance of Education Department
staff. Is he conducting an educational research? Discuss.

1.4 THE RESEARCH PROCESS

As discussed earlier, research is a systematic process, which means there are definite
steps involved. Figure 1.1 lists the seven steps involved in the research process.

Figure 1.1: The seven steps of the research process

Copyright © Open University Malaysia (OUM)

6 X TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS

1.4.1 Generate Research Ideas

Many beginning researchers and graduate students face „the problem of finding
a problem". There are cases of graduate students who have completed all
coursework requirements and "get stuck" at the thesis stage, while some never
graduate. Hence, it is advisable that students start early and search for a suitable
problem which they can explore throughout their coursework. The word
„problem‰ often means there is dispute, controversy, debate or disagreement
that needs to be addressed and solved. For example, why do young learners have
more difficulty solving multiplication and division operations than addition and
subtraction operations in mathematics?

Where does one find research problems in education? They are all around you!
There are a host of research problems or unresolved issues everywhere. Look at
the 14-year-old who says, „I hate history‰. Do you know why? Do you want to
know why? In fact, whatever that arouses your interest for which there are as yet
no answers or are inconclusive have the potential of being a research problem.
Often, one starts with a rather general, diffuse and even confused notion of the
problem. Do not worry as this is the nature and complexity of the research
process. It is the first step that helps you towards becoming a mature researcher.
The following steps are to help you obtain a research problem:

(a) Step 1: Identify a Broad Problem in Your Area that is of interest to you and
related to your professional goals. You should build sizeable knowledge in
your area of interest for the long term. For example, you may be interested
in how young children learn mathematics, how to get teenagers interested
in science, how to enhance the teaching of moral education, how to improve
training in the corporate sector using e-learning, and the relationship
between headmastersÊ leadership style and teacher morale.

(b) Step 2: Systematic Programme of Reading within your broad area of study.
For example, if you are interested in why children have difficulty learning
mathematics, you can start with textbooks in the area. These textbooks
explain the basic concepts and facts related to the issue and may cite
research in the „references‰ section which you can further explore. Journals
such as the Review of Research in Education and the Review of Educational
Research provide valuable information about a particular field as they
review related literature. Next, read articles in the relevant journals in the
field. For example, if you are interested in reading about research, you may
want to read the Journal of Reading and the Reading Research Quarterly.
You can also look through the catalogue both in the library and online, and
identify the journals in your field.

Copyright © Open University Malaysia (OUM)

 TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS W 7

(c) Students often do not adequately refer to journals in the field but, instead,
tend to cite from popular sources such as newspapers, magazines and
speeches. Journals report empirical evidence on the field you are interested
in, reflect the current thinking about research and indicate the trend of
research efforts in the field. This will give you a grasp of leading edge
research in the United States, Britain, Europe and Australia and how you
can do the same in Malaysia.

(d) Step 3: Relate Your Research Problem to a Theory in the field. A theory is
an explanation of events, phenomena or behaviour. For example, if you are
interested in finding out whether providing children with multimedia
presentations on science concepts will enhance understanding, you may
want to explore the underlying theories of visual learning. Many
phenomena in education are explained by drawing upon theories from
cognitive psychology, sociology, psycholinguistics, management, computer
science and so forth. It should be remembered that theory provides the
direction of the research. We will discuss in more detail the role of theory in
Topic 2.

SELF-CHECK 1.2

1. How do you go about finding a research problem in education?

2. What other sources provide research problems in education?

1.4.2 Formulate a Research Problem

Upon having a broad idea of what you want to investigate, the next step is to
formulate the problem simply, clearly and completely. This is what we mean by
„statement of the problem‰. An adequate statement of the research problem is an
important step in the research process. Obviously, if you want to solve a
problem, you must know what the problem is. What is a good problem
statement? Although research problems differ greatly, and there is no one ‰right‰
way to state one, certain characteristics of problems and problem statements can
be learned and used to good advantage. Let us see an example.

Example: The relationship between attitudes towards science and academic

performance in science among primary school pupils.
This is an example of a research problem stating the relation between two factors
or variables (attitudes and academic performance) and an indication of
population. A problem is an interrogative sentence or statement that asks: What

Copyright © Open University Malaysia (OUM)

8 X TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS

relation exists between the variables? The answer to this question will be sought
by conducting the research.

Three Criteria of Good Problem Statements

1. The problem should express a relation between two or more variables

Is A related to B?
How are A and B related?
How is A related to B under condition C?
Is there a difference between A and B in terms of C?

2. The problem should be stated clearly and unambiguously, preferably in question

form. Instead of saying, "The problem is ....", or „The purpose of this study is....",
ask a question. Questions have the advantage of posing problems directly. The
purpose of a study, on the other hand, does not necessarily reflect the problem of
the study. For example, the question, „Is self-esteem related to academic
performance?‰ states the problem more effectively than the purpose of the study,
which is presented thus: to throw light on the relationship between academic
performance and self-esteem.

3. The problem should imply possibilities of empirical testing. A problem that does
not contain implications for testing its relationship or relations is not a scientific
problem. So, if you can measure the constructs of self-esteem and academic
performance, then the problem is considered a good problem.

One critical issue to consider when you look for a problem is problem
importance i.e. how important the problem is and why the problem is worth
your attention. As the problem limits the scope of your study by focusing on
some variables and not others, it also provides an opportunity for you to
demonstrate why these variables are important. You can use the following
questions as a guide to assess the importance of your problem:
• Is the problem of current interest?
• Will the problem continue in the future?
• Will gathering more information about the problem have any practical
application?
• Will gathering more information about the problem have any theoretical
importance?
• How big and influential is the population affected by the problem?
• Would research into the problem substantially revise or extend existing
knowledge?

Copyright © Open University Malaysia (OUM)

 TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS W 9

• Would research into the problem create or improve an instrument of some

utility?
• Would research findings lead to some useful change in best practices?
• Is there evidence or authoritative opinion from others to support the need for
this research?
Besides, you should evaluate if your literature review is adequate in solving your
research problem. If it cannot do so, then your research into the problem is
worthwhile.

SELF-CHECK 1.3
1. What is a good problem statement?
2. Why do graduate students have difficulty in stating research
problems?

1.4.3 Develop Research Questions Or Hypotheses

Your car will not start. You put forward the hypothesis that „the car does not
start because there is no petrol‰. You check the fuel gauge to either reject or
accept the hypothesis. If you find there is petrol, you reject the hypothesis.

Next, you hypothesise that „the car does not start because the spark plugs are
dirty‰. You check the spark plugs to determine whether they are dirty and accept
or reject the hypothesis accordingly.

Similarly, in educational research, you have to put forward research questions or

hypotheses that state a relationship between the variables or constructs you are
studying. After having established the research problem or area that you plan to
investigate, the next step is to break down the problem into sub-problems. These
sub-problems are expressed in research questions or hypotheses. A research
question or hypothesis is an "educated guess" or a hunch about possible
relationships or differences. The research question or hypothesis guides the
selection of appropriate research method, data collection techniques, data
analysis techniques (e.g. the statistical analysis to be used) and so forth.

Research questions and hypotheses are closely related statements in quantitative

research. For example, you may begin a study with a research question, „What is
the role of preschool childrenÊs perception of themselves in their process of
learning to read?‰ Your hypothesis might read, „There is a positive relationship
between preschool childrenÊs perception of themselves and their reading
achievement‰. However, hypotheses are not absolutely essential in all research
Copyright © Open University Malaysia (OUM)
10 X TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS

studies. For example, in exploratory research, where there is little accumulated

background information, hypotheses are often absent.

1.4.4 Design Study to Answer Research Questions or

Test Hypotheses
Having determined the hypotheses or research questions, the next step is to design
the study. We often hear graduate students say they want to do an experiment or
a survey. But they are unable to state with clarity and precision the hypotheses or
research questions they intend to answer. It is like putting the cart before the horse.
Hypotheses or research questions determine the design of a study. If you intend to
test the effectiveness of an educational phenomenon such as a teaching method or
a counselling technique, the logical choice would be to design an experiment. If
you intend to find out whether teachers are satisfied with their profession or how
they perceive their principals, a survey would be the appropriate research design.
If your intention is to study interracial mixing among students in the school
canteen, then a qualitative approach using the observation technique might be
more appropriate.

Hence, the decision on which methodology to use will depend on the research
problem and the research questions or hypotheses. It is not good practice to decide
on a methodology and then work on the research questions. You have to be able to
state clearly what you intend to study and then decide on an appropriate
methodology. If you are clear about your research problem and research questions,
you will find it easy to get assistance from your supervisor and other students in
designing a study to obtain answers to your research questions.

ACTIVITY 1.3
The following research problems are taken from research literature. Study
them carefully and construct one or two hypotheses or research questions
based on them.
(a) The influence of self-esteem and level of aspiration on academic
achievement.
(b) The effectiveness of graphic organisers in enhancing the
understanding of science text material learners.
(c) The effects of organisational climate in schools on teacher
satisfaction and morale.

Copyright © Open University Malaysia (OUM)

 TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS W 11

1.4.5 Collect Data

The research question determines the design of a study and method of data
collection. For example, one of your research questions is determined based on
whether there are differences in self-esteem between 16-year-old male and female
students in secondary schools. To answer this question, you have to collect data
on the self-esteem of students. This may be done by developing a self-esteem
instrument or using an available instrument and administering it to a sample of
secondary school students. The sample will have to be representative of 16-year-
old students in secondary schools to allow you to generalise the results obtained
to the population. Here, you are using quantitative data collection methods.

On the other hand, if you are interested in studying student-teacher interaction,

you may have to spend time in classrooms. You can use a structured observation
checklist or leave it open-ended and record all the processes that occur in the
classroom. Here, you are using qualitative data collection methods.

ACTIVITY 1.4
You stopped at a junction because the lights had just turned red. But,
on your left, a motorcyclist and a car beat the red light. You were
most annoyed. Being a good social scientist, you asked yourself why
people did that. If you were to convert the problem into a hypothesis,
which of the following would be the best hypothesis? Explain your
answer.
(a) Some people beat the red light more frequently than others.
(b) People who beat the red light are a danger to themselves and
other road users.
(c) Beating the red light is common in crowded cities.
(d) Road users are more likely to beat the red light when traffic is
light than when traffic is heavy.

1.4.6 Analyse and Interpret Data

Data collected from subjects such as students, teachers and school administrators
have to be analysed. If your study involves quantitative data, statistical
procedures will be used to analyse the data. The analysed data is usually
presented in tables and graphs. Based on the statistical analysis, the researcher
interprets the data in relation to the research questions or hypotheses. In the case

Copyright © Open University Malaysia (OUM)

12 X TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS

of qualitative data, information is coded and presented anecdotally. Instead of

numbers, data is presented in the form of words and sentences. Similarly, the
data is interpreted in relation to the research questions or objectives of the study.

1.4.7 Communicate Results

Data that has been analysed and interpreted has to be communicated to the
community of fellow researchers and practitioners. The results of a study are
most commonly communicated to interested parties through journals. There are
numerous journals in education reporting the findings of studies in the many
fields of education (e.g. early childhood, reading, second language learning,
educational psychology, adolescents and mathematics teaching). If you are a
graduate student, you will most probably be communicating the results of your
study in the form of a thesis, dissertation or even a research practicum. There is
an established format of presenting the findings of your study, which will be
discussed in detail in Topic 10.

ACTIVITY 1.5
1. Identify a research problem that you are interested in
investigating.
2. Formulate two research questions or hypotheses based on the
research problem.

1.5 DIFFERENCES BETWEEN QUANTITATIVE

AND QUALITATIVE RESEARCH IN
EDUCATION
Educational research is typically divided into two broad categories: quantitative
and qualitative research. You will find that the discussion so far has focused
more on the characteristics of research that are typical of the quantitative
research paradigm. What are the differences between the two research
paradigms?

(a) Quantitative and qualitative research stem from different philosophical

assumptions that determine how researchers approach problems and
collect and analyse data. Quantitative research originated in positivism,
which believes that knowledge can only come from positive affirmation of
theories through strict scientific methods. In other words, it typically
involves testing theories and hypotheses, and emphasises gathering and

Copyright © Open University Malaysia (OUM)

 TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS W 13

analysing data with objective techniques such as the statistical analysis of

numeric data as the best way to answer questions and to explain and
predict behaviour. The purpose of the research is to study relationship and
cause and effect. The result is research that is systematic and open to
replication by other researchers. Qualitative research, on the other hand, is
rooted in phenomenology, which believes that knowledge is discovered
through open and unbiased description of experiences in a natural setting.
The result of this type of research is a narrative report so comprehensive
that one can understand the social reality experienced by the participants of
the research. The purpose of the research is thus to understand social
phenomena in their natural settings. Furthermore, because researchers do
not know in advance how naturally occurring events will unfold or what
variables may be important, they do not begin a study with hypotheses. In
this sense, the research designs in qualitative research normally evolve
during the study. As quantitative researchers know exactly what variables
to investigate and how they should be investigated, the research designs
are more structured and predetermined before the study begins.

(b) Quantitative research adopts the deductive approach. The researcher begins
by formulating research questions and/or hypotheses based on an available
theory or theories. He/she then designs an empirical study to test the
hypotheses and/or answer the research questions. Next, he/she conducts
the study to gather relevant data. Based on the hypotheses tested and/or
answers to the research questions, the researcher confirms or revises the
theory or theories. On the contrary, qualitative research adopts the inductive
approach. Normally, such research is conducted because there is a lack of
theory or existing theories are unable to explain a phenomenon
convincingly. Due to this, no hypotheses are put forward to guide the
research. The qualitative researcher begins by observing the phenomenon
and continues to find patterns in the form of themes, categories, concepts
and typologies that emerge. Tentative hypotheses are introduced and
additional information is collected to explain the phenomenon. Thus,
quantitative research is theory-driven while qualitative research is data-
driven. Figure 1.2 summarises the general research procedures in the two
approaches.

Copyright © Open University Malaysia (OUM)

14 X TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS

Figure 1.2: General research procedures in deductive and inductive approaches

(c) In quantitative research, the researcher uses carefully designed instruments

such as questionnaires, rating scales, inventories and tests to collect data.
For example, to study the attitudes of secondary school students towards
learning Mathematics in English, he can use the questionnaire constructed
as an attitude scale. Likewise, to measure the studentsÊ performance in
Mathematics, the researcher can opt for self-devised or standardised tests.
In qualitative research, the researcher, however, is the main instrument in
data collection. Being so, the researcher is more responsive to the situation,
and he or she is able to adapt to the changing conditions. For example, in a
face-to-face interview, a common technique in data collection, the
researcher is more sensitive to reactions of participants and the data can be
immediately processed, and he or she is able to take whatever action to
check and confirm with the participants if there are any doubts or
uncertainties.
(d) Quantitative research focuses on the perspective of the researcher rather
than that of the participants in the research. This has been termed as etic or
the outsiderÊs perspective, as against the emic or insiderÊs perspective.
Table 1.1 illustrates the differences in the questions focused on in the two
perspectives.

Copyright © Open University Malaysia (OUM)

 TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS W 15

Table 1.1: Different Focus Questions in Quantitative and Qualitative Research

Etic Perspective (Quantitative) Emic Perspective (Qualitative)

1. What do I know about a problem that 1. What do my informants know about
will allow me to formulate and test a their culture that I can discover?
hypothesis?
2. What concepts can I use to test this 2. What concepts do my informants use
hypothesis? to classify their experiences?
3. How can I operationally define these 3. How do my informants define these
concepts? concepts?
4. What scientific theory can explain the 4. What folk theory do my informants use
data? to explain their experience?
5. How can I interpret the results and 5. How can I translate the cultural
report them in the language of my knowledge of my informants into a
colleagues? cultural description my colleagues will
understand?

ACTIVITY 1.6
Examine the two research studies on „reasons for discipline problems
in schools‰. Then, state which is quantitative research and which is
qualitative research. Justify your answers.

Researcher A identifies factors Researcher B interviews a sample of

that influence/affect discipline, problematic students individually.
develops an inventory and He interacts with them and observes
administers it to a sample of their behaviour. He examines
problematic students. He then counsellorsÊ reports and school
analyses the data and identifies discipline reports. He records all the
significant factors or ranks the information and analyses it for
factors in order of dominance. factors that emerge.

Copyright © Open University Malaysia (OUM)

16 X TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS

ACTIVITY 1.7
1. Write down your definition of research.
2. Suggest how you will go about finding a research problem that
you propose to investigate.
3. Discuss some educational research you have read. (Go to OUMÊs
Digital Library and click on „ProQuest‰, which has a good
collection of journals in education.)
4. List the current thinking on research in your area of interest.

• The word „research‰ has been used in many different ways, sometimes rather
loosely, giving rise to confusion, and at times with the intention to deceive.

• Knowledge can be acquired through six ways - beliefs, intuition, authority,

empiricism, rationalism and science.

• Research is the systematic, controlled, empirical and critical investigation of

natural phenomena guided by theory and hypotheses about the presumed
relations among the phenomena.

• The purpose of the scientific method is to describe, explain, control and

predict phenomena.

• Educational research investigates the behaviour of students, teachers,

administrators, parents and other members of the educational community.

• The seven steps of the educational research process are useful guides for
beginning researchers.

• A good research problem is stated clearly, expresses a relationship between

variables and can be tested empirically.

Copyright © Open University Malaysia (OUM)

 TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS W 17

Educational research Research process

Hypothesis Research question
Research Scientific method

Copyright © Open University Malaysia (OUM)

Topic X Theory and
2 Review of
Literature
LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Define what theory is;
2. Explain the role of theory in educational research;
3. Justify the need for review of literature;
4. Identify the criteria for a good review of literature; and
5. Critique a research article.

X INTRODUCTION
Borg and Borg (1988) mentioned that „without some viable theory to serve as a
guide, many studies address trivial questions or contribute nothing to the slow
accumulation of knowledge needed for the advancement of the science of
education.‰ Do you agree with this statement made by Borg and Borg?

Theory has not been properly understood by some graduate students who are of
the opinion that theory is not relevant to practice. Oftentimes, we hear them
remark that a particular course is „too theoretical‰ or that they prefer courses that
are „practical oriented and not too theoretical‰. Some go to the extent of
denouncing theory as useless! This reflects a lack of understanding of what
theory is, and what practice or practical is. According to Borg and Borg (1988),
theory serves as a guide for research to avoid investigating phenomena that are
irrelevant and do not contribute to our understanding.

Copyright © Open University Malaysia (OUM)

 TOPIC 2 THEORY AND REVIEW OF LITERATURE W 19

2.1 WHAT IS THEORY?

According to Kidder, a theory is a large body of interconnected propositions
about how some portions of the social world operates. It is a statement or a set of
statements that explain and predict phenomena. It is at the same time a statement
of a relationship between two or more events. The more „powerful‰ a theory is,
the more events can be explained by it. Theories consist of generalisations and, in
the physical sciences, some of them are called laws such as the ArchimedesÊ
Principle and BoyleÊs Law.
• According to the Social Learning Theory by Albert Bandura, the observer will
imitate the model's behaviour if the model possesses characteristics such as
talent, intelligence, power, good looks and popularity that the observer finds
attractive or desirable.
• According to the Theory of Meaningful Learning by David Ausubel, learning
takes place when the learner subsumes new information with old information
or oneÊs cognitive structure.
• According to B F SkinnerÊs Theory of Reinforcement, a learner will repeat his
or her performance of a task if he or she is reinforced with a system of
rewards or punishment.

Note that each of these theories explains learning and the variables or factors that
determine learning. Assuming that each of these theories is true, we can predict
that learning will take place when a student connects new information with old
information, is reinforced through a system of rewards, and reproduces a
modelled behaviour if it brings pleasure. However, many areas of education
have virtually no theoretical foundation and have to rely on other behavioural
sciences, such as psychology, sociology, anthropology and social psychology.

Can a theory be „true‰? Not necessarily. The scientific method makes it

impossible to conclude that a theory in the behavioural sciences is definitely true.
It is possible only to disconfirm or confirm a hypothesis or theory. We cannot say
that we have verified a theory because there is always the possibility that some
future research will disconfirm it or that some other theories will account for the
same results. Theories, therefore, are always tentative. They represent the best of
our knowledge for the time being but do not represent some absolute truth.
According to Borg and Borg (1988), they await revision or replacement.

Copyright © Open University Malaysia (OUM)

20 X TOPIC 2 THEORY AND REVIEW OF LITERATURE

2.2 CONFIRMING OR DISCONFIRMING A

THEORY
Let us take an example of a theory and examine how we go about confirming or
disconfirming it as illustrated in Figure 2.1. According to Jean Piaget's theory of
cognitive development, "Children at the pre-operational stage (ages 0 ă 5) are not
able to perform conservation tasks". Conservation is defined as the ability to
recognise that an object remains unchanged when its volume or length has
undergone a transformation with nothing added or taken away. From this broad
theory, a hypothesis is derived, which is a small version of the theory, as shown
in Figure 2.1 below.

Figure 2.1: Confirming or disconfirming a theory

An experiment is set up in which young children are shown two identical

containers (A and B) with the same amount of water. Next, the water from
container B is poured into a flat container C (see Figure 2.2).

Figure 2.2: The experiment

Copyright © Open University Malaysia (OUM)

 TOPIC 2 THEORY AND REVIEW OF LITERATURE W 21

Children are asked whether the amount of water in container A is the same as
that in container C (Is A = C?). Based on these observations, the researcher
concludes whether to confirm or disconfirm the theory. If the theory is
confirmed, it means that children are unable to perform conservation tasks. In
other words, the children would have replied that container C had more water. If
the theory is disconfirmed, then the majority of children would have answered
that the amount of water in container A and C is the same.

SELF-CHECK 2.1
1. What is a theory?
2. Why are there few laws in the field of education?
3. What do you mean by confirming or disconfirming a theory?
4. Identify some theories in your field of interest.
5. What are the implications of PiagetÊs theory of childrenÊs
cognition in the classroom?

Let us examine another well-known theory in psychology which has been used
widely in education. David Clarence McClelland developed a theory of
motivation in 1988, which states that a person is motivated to do something
because of a desire or need for achievement, authority or affiliation, or a
combination of the three characteristics, as provided in Figure 2.3.

Figure 2.3: Motivation characteristics

(a) Achievement Motivation (represented by „n-ach‰) includes people who are

driven by the need or desire to achieve, attain realistic but challenging
goals, and advance in their job.
(b) Authority/Power Motivation (represented by „n-pow‰) includes people
who are driven by the need or desire to be influential, effective and to make
an impact.
Copyright © Open University Malaysia (OUM)
22 X TOPIC 2 THEORY AND REVIEW OF LITERATURE

(c) Affiliation Motivation (represented by „n-affil‰) includes people who are

driven by the need or desire for friendly relationships and are motivated
towards interaction with other people.

ACTIVITY 2.1
1. To what extent does McClellandÊs Motivation Theory describe
your motivation to do something in your daily life?
2. Briefly explain how you would attempt to confirm or disconfirm
McClellandÊs Theory.
[You can find more information about this theory at this site:
http://www.businessballs.com/davidmcclelland.htm
Chapman, A. (1995). David McClellandÊs motivational needs
theory (Electronic version).]
3. Identify some major theories in your area of interest.

As research is focused on testing the behavioural science theory, we gain a better

understanding of the theory, which in turn leads to modification in the theory
and eventually to its acceptance or rejection. It is hoped that this better
understanding of human behaviour will inform practices in terms of what
teachers do in the classroom, how administrators manage and lead their
organisations, and most importantly how students learn and conduct themselves.
Theory can provide a rational basis for explaining or interpreting the results of
research. Studies without a theoretical foundation often produce results that the
researcher will have difficulty explaining. Also, studies based on theory enable
the researcher to make predictions about a wide range of situations. For example,
McClellandÊs theory of motivation can be employed to determine what motivates
teachers, students and school administrators.

2.3 WHAT IS THE REVIEW OF LITERATURE?

According to Borg and Borg (1988) in Educational Research, until you know what
others have done in your area and what has not been done, you cannot
convincingly carry out research that will contribute to furthering knowledge in
your field. Thus, the literature in any field forms the foundation upon which all
future work must be built. If you fail to build this foundation of knowledge
provided by the review of literature, your work is likely to be shallow and naive,
and will often duplicate work that has already been done by someone else.

Copyright © Open University Malaysia (OUM)

 TOPIC 2 THEORY AND REVIEW OF LITERATURE W 23

The review of literature is usually a standard chapter of the research report,

thesis or dissertation. It is an account of the research done in the field of study.
The review forms an important chapter in a thesis or dissertation in which its
purpose is to provide the background to, and justification for, the research
undertaken. It is usual that the review comprises empirical studies done in the
area that is being investigated. It also includes theoretical positions or proposals
related to the study which are not necessarily empirical in nature.

The aim of a literature review is to show what has been done in the field and how
your study relates to earlier research. It also indicates the approaches, the
samples used, the variables examined, the statistical procedure used and, most
important of all, the findings obtained. The review gives an overview of the
findings of various previous studies. The review traces the general patterns of the
findings and the conclusions that can be made based on the findings.

It also provides an insight into how your study is similar to or different from
previous studies. For example, is your study an extension of what others have
done? Are you examining variables that have not been attended to in earlier
studies? Are you attempting to replicate earlier studies in a different cultural
context? Are you applying statistical procedures that have not been attempted by
others?

The review of literature requires you to locate, read and evaluate reports of
research as well as reports of opinions and proposals. The review must be
extensive and thorough because you are aiming to obtain a detailed account of
the topic being studied.

2.4 THE IMPORTANCE OF THE REVIEW OF

LITERATURE
Now, let us see why literature review is important.

(a) The review of literature is an important part of the research process because
it forms the basis of any research and puts your work into perspective. The
literature may inform you about how your research may add to the current
knowledge base. It gives an understanding of previous work (seminal
works in the field), familiarises you with the personalities doing research in
the field and demonstrates that you can access such works.
(b) The review of literature helps the student in delimiting the research
problem by setting parameters. By setting the limits of your study, you
avoid being questioned, "Why didn't you do this or do that?" You can
confidently reply that your study is confined to studying what you had set

Copyright © Open University Malaysia (OUM)

24 X TOPIC 2 THEORY AND REVIEW OF LITERATURE

out to study. Delimiting the research problem can be achieved if you read
extensively and intensively the problem you plan to investigate, and from
the literature specify clearly what it is you want to study.
(c) The literature also provides insights into the approaches and methodologies
adopted by different researchers. A common mistake made by students is
to pay attention to only the findings of studies. Besides findings, students
should also examine the methodologies used to study the phenomenon
they are interested in. There could be unique approaches adopted which
you might want to replicate in the Malaysian context. For example, in most
studies reviewed, the sample tended to be university students. Hence, if
your study is an attempt to use secondary school students as the sample, it
could be a unique contribution to the field because your approach is
different.
(d) Some research studies include a section on „Recommendations‰ for further
research, which indicates whether you are on the right track in studying
something that has not been explored before. These suggestions are
significant because they express the insights of the researcher after having
studied the phenomenon. You can learn from the mistakes made by the
researcher in his/her research.

SELF-CHECK 2.2
1. What is literature review?
2. What is the purpose of the literature review?

2.5 STEPS IN DEVELOPING A LITERATURE

REVIEW
Now we know what is literature review and its importance. Do you know how to
develop a literature review? Let us now discuss the steps involved in developing
a literature review.

Step 1: Select a Research Topic

(a) Choose a topic of current interest. Your goal is to summarise and evaluate
the findings of a line of research. Pick a research topic about which articles
are continuing to be published.
(b) Choose a well-researched area. An area that is well-defined and well-
studied will give you more lines of research to choose from. A line of

Copyright © Open University Malaysia (OUM)

 TOPIC 2 THEORY AND REVIEW OF LITERATURE W 25

research is a series of studies by the same individual. An area of major

research interest will have several lines of research.
(c) Narrow your topic. It is far more satisfying, to both the writer and the
reader, to restrict your topic and cover it in depth. Comprehensiveness and
narrowness of topic go hand in hand.
(d) Write about what interests you. If you are interested in the topic, you are
likely to already know something about it, which will make it easier to
gather information.

Step 2: Collect and Read the Relevant Articles

(a) Do a preliminary search. Visit the library or do an online literature search
before you even decide on a topic.
(b) Search for helpful activities. Some articles will contribute more than others
to your understanding of a topic. Sometimes you can find a pivotal article
that can serve as the foundation for your study (the „References‰ will lead
you to other similar studies).
(c) Find readable articles. Some areas of research will be harder to understand
than others. Scan the research articles in the topic areas you are considering
to decide on the readability of research in those areas.
(d) „Read, Read, Read!‰ That is the bottom line of doing a review. People have
different ways of doing a literature review. A common technique used by
many graduate students is to use a „Note Card‰ as illustrated in Figure 2.4.
It may be a rather old-fashioned technique, but has proven to be most
effective. Many graduate students will testify to this, despite advancements
in computer technology.
(e) Read the easier articles first.
(f) Identify:
(i) The problem statement;
(ii) The research questions or hypotheses;
(iii) The method used;
(iv) The findings; and
(v) How the findings were interpreted.
Jot down the contents of the article using a note card as shown in Figure 2.4.

Copyright © Open University Malaysia (OUM)

26 X TOPIC 2 THEORY AND REVIEW OF LITERATURE

Title: …………………………………………………………………….

Author/s: ………………………………………………………………..

Source: …………………………………………………………………..

Problem Statement: ………………………………………………………

Methodology:…………………………………………………………….

Findings…………………………………………………………………

Conclusion:……………………………………………………………….

Comments:……………………………………………………………….

Figure 2.4: Note card

Step 3: Write the Review

(a) Introduce your research questions (what it is and why it is worth
examining). Begin your review with some themes (or points) that you want
to emphasise. Remember that you are writing for the reader and not for
yourself.
(b) Briefly outline the organisation of the paper. Organisation is of utmost
importance and makes the structure known to your reader. For example,
tell the reader that you will present research supporting one side first, then
the other. Or if you are addressing three methodologies, briefly describe
them and state that you will compare the results from the three methods.
(c) Describe, compare and evaluate studies in terms of the:
(i) Research assumptions;
(ii) Theories;
(iii) Hypotheses stated;
(iv) Research designs used;
(v) Variables selected (independent and dependent); and
(vi) Researcher speculations about future studies.
(d) Discuss implications of studies (e.g. your judgement or what the studies
show, and where to go from here).
Copyright © Open University Malaysia (OUM)
 TOPIC 2 THEORY AND REVIEW OF LITERATURE W 27

(e) Most important of all, avoid plagiarism. Give due recognition to the works
of other people. It does not cost anything to acknowledge sources. In fact, it
shows the breadth and depth of your review, and the thoroughness of your
work.

2.6 COMMON WEAKNESSES

In writing a literature review, at the beginning, researchers make the following
common errors:
(a) The presentation is a mere description of various studies without an effort
to show how the studies are related to the main aim and the research
questions of your study.
(b) The presentation is a mere listing of the studies without an attempt to show
how each study is similar or different. Hence, use connectives such as:
„however‰, „on the other hand‰, „similarly‰ and „but‰.
(c) Poor citations. In education, it is normal practice to adopt the format
proposed in the manual published by the American Psychological
Association (commonly known as the „APA style‰).
(d) The literature has been hurriedly reviewed and relies too heavily on
secondary sources.
(e) At times, there is evidence suggesting that students have not read the
original works but have instead taken someone's work and cited it as
though they had read the primary source.
(f) There is also evidence of „cut and paste‰ which should not be encouraged.
You must read the original works and know in detail every study that you
cite.
(g) Articles or reports that are included have not been critically evaluated.
Hence. You need to critically evaluate the research questions, the
methodology, the statistics used, the conclusions reached and the
recommendations made by the researcher.

SELF-CHECK 2.3
1. „Read, Read, Read!‰. Comment.
2. What are some weaknesses of graduate students when writing a
literature review?

Copyright © Open University Malaysia (OUM)

28 X TOPIC 2 THEORY AND REVIEW OF LITERATURE

2.7 SOURCES
A good literature review requires knowledge of the use of indexes and abstracts
and the ability to conduct exhaustive bibliographic searches. You should be able
to organise the materials meaningfully, describe, critique and relate each source
to the subject of the inquiry, present the organised review logically and, most
importantly, correctly cite all sources mentioned. Generally, there are two main
sources of materials:
(a) Secondary Sources: These include materials written by an author who was
not a direct observer or participant in the events described. If you read a
textbook on educational psychology, it would be a compilation of the views
and empirical works of other authors rearranged into a textbook. The
textbook is a review of research done by others and is as interpreted by the
author. This interpretation by the author of the textbook would be classified
as a secondary source (note that the interpretation may be biased).
Secondary sources are useful because they provide a quick and relatively
easy method for obtaining an overview of current thinking in the field.
(b) Primary Sources: These include materials that are a direct description of
events by researchers who actually conducted the investigation. Most
primary sources are found in research journals. However, there are also
many reports of research conducted by individuals, groups of individuals
and organisations.

How do you search for research articles, research reports, etc? You can start by
referring to preliminary sources or references such as indexes and abstracts,
which are intended to help you identify and locate research articles and other
primary sources of information. The following are well-known indexes and
abstracts in education:

(a) Education Index ă Provides a list of articles published in education journals

and books on education.
(b) Psychological Abstracts ă Contains abstracts of articles
appearing in over 900 journals and other sources in psychology and related
issues.
(c) Educational Resources Information Center (ERIC) ă Transmits
the findings of current educational research to teachers, administrators,
researchers and the public.
(d) Resources in Education (RIE) ă Provides abstracts of papers presented at
education conferences, progress reports and final reports of projects which
may not appear in education journals.

Copyright © Open University Malaysia (OUM)

 TOPIC 2 THEORY AND REVIEW OF LITERATURE W 29

(e) Current Index to Journals in Education (CIE) ă Indexes over 800 education
journals and includes more than 1,000 articles each month.

Specialised Areas
• Child Development Abstracts and Bibliography
• Exceptional Child Education Resources (ECER)
• Education Administration Abstracts
• Physical Education Index

Another way to obtain information about research done is to examine journals,

handbooks and encyclopaedias that locate and review research for particular
topics. The following are the more well-known materials in education:

(a) Review of Educational Research ă A journal published by the American

Educational Research Association (AERA) covering critical issues and
reviews of research literature on important topics and issues.
(b) Review of Research in Education ă Presents critical essays that survey and
synthesise educational research on important problem areas.
(c) Encyclopaedia of Educational Research ă Best single source of information
on educational research with contributions from leading educational
researchers.
(d) Handbook of Research on Teaching ă Contains reviews of various aspects
of research on teaching such as methods and techniques of teaching,
teaching specific school subjects and problems of teaching.

A further source of information are theses and dissertations that have never been
published. The following are important sources which provide abstracts of
masterÊs theses and doctoral dissertations in education:
(a) Dissertation Abstracts International ă A compilation of abstracts of doctoral
dissertations. Under the education section are sub-topics such as adult
education, art education, preschool and teacher training.
(b) MasterÊs Theses in Education ă This is a listing of masterÊs theses in about
40 major educational topics. It includes name of author, title and institution.

Copyright © Open University Malaysia (OUM)

30 X TOPIC 2 THEORY AND REVIEW OF LITERATURE

ACTIVITY 2.2

What do you think are some problems graduate students face when
doing literature review for their theses or research projects?

2.8 EVALUATING SEARCH ARTICLES

As mentioned earlier, in writing your literature review, you must be able to
interpret the works of others. How do you go about evaluating research articles?
The procedure for evaluating research articles is shown in Figure 2.5. You should
keep in mind that in educational research, the findings of previous research tend
to be inconclusive as results are often contradictory. This may leave you at a loss
to decide which, if any, to accept. However, this problem can be resolved
through a critical evaluation of previous works in which the strengths and
weaknesses of each study are carefully weighed as illustrated in Figure 2.5. The
procedure proposed below comprises two parts: the first part relates to
describing the research article and the second part relates to critiquing the article
or doing a critique of the article.

Figure 2.5: The five-step procedure of evaluating an article

Step 1: Read the Abstract

(a) What was the research about? Were the purposes or objectives of the study
specified?
(b) Was the design used described?
(c) What are the reasons or rationale for the research?

Copyright © Open University Malaysia (OUM)

 TOPIC 2 THEORY AND REVIEW OF LITERATURE W 31

Step 2: Read the Introduction

Describe:
(a) Keep in mind that the writer has assumed that the reader is an expert in the
field or at least has some background knowledge of the field.
(b) References made may be brief because it was assumed that the reader
knows the people in the field (e.g. if you are reading about intelligence,
then names like Sternberg, Gardner, Thurstone and Spearman should be
known to you).
(c) The writer has assumed that the reader knows the concepts in the field (e.g.
burnout, metacognition, inductive reasoning, organisational climate).
(d) The rationale given for the study and why the research questions or
hypotheses were put forward.

Critique:
(a) Was the reason for answering the research questions or testing the
hypotheses convincing or was it only attempting to appeal to the readerÊs
emotions and merely seeking endorsement from well-known authorities in
the field?
(b) Did the research questions or hypotheses follow logically from theory?
(c) Was there a tendency to oversimplify the theories or studies reviewed?

Step 3: Read the Methods Section

Describe:
(a) The writer should provide the background of the subjects used, the number
of subjects and the method used to collect data.
(b) The design of the study was described in sufficient detail, and the
justification for the study was given.
(c) The development of the instruments (or the use of someoneÊs instrument)
was described and there was mention of pilot-testing the instruments and
the reliability and validity of the figures given.

Critique:
(a) How clear was the selection of subjects?
(b) Was the instrument or treatment administered adequately explained?
(c) Were issues of validity and reliability discussed?
(d) Was the design of the study appropriate? How was it designed to reduce
different types of biases?
(e) What were the independent and dependent variables?

Copyright © Open University Malaysia (OUM)

32 X TOPIC 2 THEORY AND REVIEW OF LITERATURE

(f) Were the statistical procedures used appropriate?

(g) Were the variables operationally defined?
(h) If the study was an experiment, was the treatment explained in sufficient
detail? Could it have been done in another way?

Step 4: Read the Analysis or Results Section

Describe:
(a) Connection between the results and the research questions or hypotheses.
(b) Report results relating to the research questions or hypotheses (whether
results are statistically significant).
(c) Report other statistically significant results.

Critique:
(a) Were the results clearly reported (e.g. the use of tables and graphs)?
(b) Did the statistics test the predictions made in the Introduction?

Step 5: Read the Discussion section

Describe:
(a) List the main findings.
(b) Relate the findings to what was mentioned in the Introduction.
(c) Speculate about the reasons for the results.

Critique:
(a) Was the author's way the only way to interpret the predicted results?
(b) Can you explain any of the findings the author did not explain or was
unable to explain?
(c) What were the weaknesses or limitations identified by the author or which
you found but were not mentioned?

SELF-CHECK 2.4
1. What major aspects of a study will you examine when
describing a research article?
2. When you critique the Methods section and the Results section,
what are you looking for?

Copyright © Open University Malaysia (OUM)

 TOPIC 2 THEORY AND REVIEW OF LITERATURE W 33

ACTIVITY 2.3

1. Select a research topic in which you are interested and locate a

primary and a secondary source related to it. Explain why each
is a primary or a secondary source.
2. Locate full-text journals in your areas of interest that are
available for free on the Internet. Check to see if they are
referred journals. Share what you have found with your
coursemates.
3. How will you define plagiarism? What constitutes plagiarism?
4. „Who am I to critique the research of experts in the field? I am

• A theory is a statement or a set of statements that explains and predicts

phenomena. It is also a statement of a relationship between two or more
events.

• Theories are tentative; they can be confirmed or disconfirmed.

• Literature review shows what has been done in the field and how the
intended study relates to earlier research.

• Literature review comprises research evidence as well as propositions and

opinions of personalities in the field.

• Literature review delimits a study, relates the methods used by others and
the recommendations of earlier works, and provides the basis for the
intended study.

• There are primary and secondary sources of material in education.

• The best sources of primary material are found in research journals.

• All reviewed materials included should be critically evaluated.

Copyright © Open University Malaysia (OUM)

34 X TOPIC 2 THEORY AND REVIEW OF LITERATURE

Abstracts of research Indexes in education

Confirming a theory Review of literature
Critique of research article Primary sources
Disconfirming a theory Secondary sources
Dissertation and theses abstracts Theory
Evaluating information

Copyright © Open University Malaysia (OUM)

Topic X Constructs,
Variables and
3 Data Types
LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Differentiate between constructs and variables;
2. Identify independent and dependent variables;
3. Explain what is meant by moderator and extraneous variables;
4. Operationally define variables; and
5. Differentiate the different types of data in quantitative research.

X INTRODUCTION
Once you have identified your research topic, reviewed the literature and refined
your research questions, you are ready to plan for the data collection stage of the
research process. At this stage, you need to decide what data to collect and how to
collect it. This involves identifying the constructs and variables to be measured.
Without these measurement targets, you may end up collecting irrelevant data and
wasting your time, money and other resources. It is therefore important to
understand the concepts of constructs and variables in educational research.

3.1 CONSTRUCTS AND VARIABLES

A construct is a theoretical concept that is used to describe a specific observed
behaviour or characteristic of individuals. For instance, "intelligence" is a
construct based on observation of presumably intelligent and less intelligent
behaviours or having a value of more or less. Intelligence is so defined and
specified that it can be observed and measured, such as by administering
intelligence tests or interviewing teachers about their students. Some other
examples of constructs that are commonly found in educational research are

Copyright © Open University Malaysia (OUM)

36 X TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES

academic performance, creativity, self-esteem, motivation, learning readiness,

personality traits, anxiety, distress and leadership styles.

To be noted is that constructs are not measured directly. Rather, they are inferred
from, and measured by, their empirical referents. The word „empirical‰ refers to
observation. Thus, the empirical referents of a construct are those observable
objects, events and behaviours including beliefs and feelings from which the
construct is inferred. For example, the referents of anxiety are behaviours and
events such as crying, certain facial expressions, certain mannerisms and other
non-verbal behaviours. Note that no single empirical referent ă no one event, object
or behaviour ă fully represents a construct. In other words, no one empirical
referent mentioned above by itself fully represents the construct of anxiety. All
these referents need to be taken together as a whole as the indicator of anxiety.
Figure 3.1 illustrates the process of inference involved in defining a construct.

Figure 3.1: The process of inference involved in defining a construct

To measure a construct, it is essential that the measuring instrument used

captures as many empirical referents associated with the construct as possible.
However, this is often not possible; a representative sample of the referents
supported by theory is taken instead. In educational research, the referents of a
construct are very often grouped together and referred to as the dimensions (or
factors) of the construct. For instance, according to the Technology Acceptance
Model, the two dimensions of the „attitudes towards using technology‰ construct
are: the perceived ease of use and the perceived usefulness of technology.

Copyright © Open University Malaysia (OUM)

 TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES W 37

ACTIVITY 3.1
Given below is a set of items that are supposedly reflective of the
empirical referents associated with the construct of reading attitude. The
referents can be divided into three categories, which make up the three
dimensions of the reading attitude construct defined by MathewsonÊs
well-known tripartite model. The three dimensions are:
• A personÊs prevailing feelings about reading (affective dimension);
• A personÊs belief or knowledge about reading (cognitive dimension);
and
• A personÊs action readiness towards reading (behavioural
dimension).

Group the statements into the three dimensions.

Dimension Item No
Affective
Cognitive
Behavioural

Copyright © Open University Malaysia (OUM)

38 X TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES

Reading Attitude Questionnaire

1 Reading is for learning but not for enjoyment. SD D A SA

2 Money spent on books is well spent. SD D A SA

3 There is nothing to be gained from reading books. SD D A SA

4 Books are dull. SD D A SA

5 Reading is a good way to spend spare time. SD D A SA

6 Sharing books with friends is a waste of time. SD D A SA

7 Reading excites me. SD D A SA

8 I feel tired when I am presented with a long text. SD D A SA

9 I make it a point to finish the book I read. SD D A SA

10 Reading is rewarding to me. SD D A SA

11 I feel anxious when I have to read books. SD D A SA

12 I find reading books enjoyable. SD D A SA

13 Leisure reading does not teach anything. SD D A SA

14 I would spend more time for leisure reading during school days. SD D A SA

15 There are many books which I hope to read. SD D A SA

16 I would not read books except for class requirements. SD D A SA

17 Reading is something I can do without. SD D A SA

18 I would set aside a certain amount of school vacation time for SD D A SA

reading.

19 I would buy books as presents for others. SD D A SA

20 Reading is useful. SD D A SA

21 Reading is something I should not miss whenever I am free. SD D A SA

22 I think reading makes a good hobby. SD D A SA

23 I get bored easily when I read. SD D A SA

24 Reading many books widens our outlook. SD D A SA

SD ă Strongly Disagree D ă Disagree A ă Agree SA ă Strongly Agree

Copyright © Open University Malaysia (OUM)

 TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES W 39

When a construct is measured in educational research, it is known as a variable

i.e. something that varies. In other words, constructs are represented by variables
which normally take on different numerical or text values. For example, the
construct "intelligence" is represented by IQ scores, a variable ranging from 50 to
150 in numerical values. Likewise, the construct „language proficiency‰ can be
represented by the variable test scores in a language proficiency test such as
TOEFL. However, it is important to distinguish between variables and constructs
in educational research. A variable is essentially what we can observe or measure
of a characteristic or ability. A construct is the underlying characteristic or ability
itself. In the case of the above examples, each set of test scores is not the ability
but a reflection of intelligence or language proficiency.

Not all variables in educational research are quantitative or take on numerical

values. The variable „gender", for example, consists of only two text values or
labels and they are called dichotomous variables i.e. male (1) and female
(0). Other examples of two-value variables are: graduate ă non-graduate, low
income ă high income, citizen ă non-citizen, treatment ă no treatment as in an
experimental research design. Besides dichotomous variables, some variables are
polytomies i.e. religion ă Islam, Christianity, Buddhism, Hinduism, etc.

3.2 TYPES OF VARIABLES

There are many ways of classifying variables but in educational research, the two
most common methods of classification are as follows:
(a) Independent and dependent variables; and
(b) Continuous and categorical variables.

Let us discuss this further.

(a) Independent and Dependent Variables

An independent variable (IV) is the variable that is presumed to cause a
change in the dependent variable (DV). The independent variable is the
antecedent while the dependent variable is the consequent. See Figure 3.2,
which describes a study to determine which teaching method (independent
variable) is effective in enhancing the academic performance (dependent
variable) of students.
(i) The independent variable (teaching method) can be manipulated.
„Manipulated‰ means the variable can be manoeuvred, and in this
case it is divided into the „discussion method‰ and the „lecture
method‰. Other examples of independent variables are gender (male,
female), race (Malay, Chinese, Indian) and socio-economic status

Copyright © Open University Malaysia (OUM)

40 X TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES

(high, middle, low). Other names for the independent variable are
treatment, factor and predictor variable.
(ii) The dependent variable in this study is academic performance which
cannot be manipulated by the researcher. Academic performance is a
score and other examples of dependent variables are IQ (score from
IQ tests), attitude (score from an attitude scale), and self-esteem (score
from a self-esteem test). Other names for the dependent variable are
outcome variable, results variable and criterion variable.

Put it another way, the dependent variable is the variable predicted to,
whereas the independent variable is predicted from. The dependent
variable is the presumed effect, which varies from changes or variation in
the independent variable.

Figure 3.2: Independent and dependent variables in experimental research

The concepts of independent and dependent variables are only relevant in

research that examines cause-and-effect relationship (e.g. experimental research
that examines the effectiveness of a teaching method on academic performance).

In research that examines correlation between two variables (e.g. the relationship
between studentsÊ English proficiency and their mathematics performance) as
stipulated in Figure 3.3, both variables are dependent variables.

Copyright © Open University Malaysia (OUM)

 TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES W 41

Figure 3.3: Dependent variables in correlation research

ACTIVITY 3.2
Can you identify the independent and dependent variables in the
following research study?

You are interested to find out if students who are exposed to project-
based method (PBM) would perform better in the science subject than
those who are taught using the traditional lecture method (TLM).
Besides, you also want to know if the PBM would change their attitude
towards learning science. Lastly, if the PBM has positive effects, you
want to know if it would benefit:
(a) Male students more or female students more.
(b) High achievers more or low achievers more.
Write your answers in the following table.

Independent Variable Dependent Variable

In the above research study, which intends to examine the effects of

project-based learning (PBL) on studentsÊ performance in science and their
attitudes towards learning science in relation to gender and academic
achievement, you will find that there are two dependent variables i.e.
performance in science and attitude towards learning science. There are,
however, three independent variables i.e. teaching method (PBL and TLM),
gender (male and female) and academic achievement (high and low
achievers). Gender and academic achievement have been included in the
study to discover whether they would modify the cause-effect relationship
between the PBL and performance in science as well as attitudes towards
learning science i.e. if the PBL would have differential effects on both the

Copyright © Open University Malaysia (OUM)

42 X TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES

dependent variables. They are therefore a special type of independent

variables. These two independent variables are also known as moderator
variables.

It is important to take note of another type of independent variable called

extraneous variable. An extraneous variable is an independent variable that
may influence the outcome of a study but has not been taken into
consideration in the research design or has not been controlled.
For example, in the above study on the effects of the PBL on performance in
science and attitudes towards learning science, one independent variable
i.e. language proficiency of the students, has not been taken into
consideration. If the PBL comprises students whose language proficiency is
better than the TLM group, and the findings show that the PBL group
outperforms the TLM group, the better performance could be attributed to
the higher language proficiency of the students. In other words, language
proficiency has confounded the finding of the study. One way to control an
extraneous variable which might influence the results is to make it a
constant (i.e. keeping everyone alike on that variable). In the above
example, the researcher can restrict the study to one proficiency group of
students, such as average proficiency level. In so doing, he can eliminate
language proficiency as an extraneous variable.

(b) Continuous and Categorical Variables

A continuous variable is capable of taking on an ordered set of values
within a certain range. For example, an attitude scale towards smoking may
have values ranging from 5 to 20 which express differing levels of attitude
towards smoking. A categorical variable may be made up of two or more
sub-sets or categories. Each sub-set or category possesses certain
characteristics and individuals are categorised by their possession of those
characteristics that define a sub-set. For example, the variable socio-
economic class (SES) may comprise three values such as high SES, middle
SES and low SES.

In quantitative educational research, continuous variables are sometimes

converted into categorical variables for the purpose of data analysis and
interpretation. Table 3.1 gives an example. The construct of reading ability
as represented by the continuous variable of reading scores (maximum 20
marks) has been converted to the categorical variable: type of readers with
two levels, poor and good. The conversion from the categorical variable to
continuous variable is not possible.

Copyright © Open University Malaysia (OUM)

 TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES  43

Table 3.1: The Construct of Reading Ability

Student Reading Score Type of Reader

(Continuous Variable) (Categorical Variable)

1 5 Poor
2 7 Poor
3 6 Poor
4 8 Poor
5 13 Good
6 16 Good
7 19 Good
8 20 Good

3.3 OPERATIONAL DEFINITION OF VARIABLES

„If you lead a good life, you will not suffer.‰ This is a specific prediction of the
future, but it cannot be scientifically tested. Such a prediction is not scientifically
tested because we cannot define it operationally. How do you define „good life‰
and how do you define „suffer‰? According to Bridgman, „operational‰ means
that the variables used in the study must be defined as they are used in the
context of the study and are publicly observable. This is done to facilitate
measurement and to eliminate confusion. For example, when you state in your
study that you are studying excellent principals, you should be able to explain
what „excellent‰ means. Once the behaviours of an excellent principal have been
identified, the operational definition will be unique to your study as provided in
Figure 3.4.

Figure 3.4: Example of an operational definition of an excellent principal

Copyright © Open University Malaysia (OUM)

44 X TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES

However, it should be borne in mind that in education not all variables are
directly observable. For example, we cannot really observe learning, memory,
reasoning and so forth. Though these cannot be observed, they can be measured.
With enough indirect evidence, researchers can make a convincing case for the
existence of these invisible variables. For example, though we cannot observe
learning directly, we can see its effect on performance i.e. we can operationally
define learning as an increase in performance. Thus, if we see students improve
their performance after carrying out a task, we can conclude that learning has
occurred.

Similarly, we can provide operational definitions for such intangible variables

such as self-esteem, racial stereotype and attitudes.

An operational definition is different from the dictionary type of definition.

The dictionary type of definition only conveys the general meaning of a variable,
but it is not precise enough for research purposes. For example, if you define
„intelligence‰ as the ability to think abstractly or the capacities to acquire
knowledge, just as provided by the dictionary, the term will not be clear to other
researchers who want to replicate the research. However, if you define it
operationally based on scores projected by the Standford-Binet Intelligence Scale,
other researchers will know exactly what you mean by the term. It is important
that variables investigated in research be defined operationally not only to guide
the study, but also for the purpose of replication by other researchers.

3.4 DATA TYPES

Once you have operationalised the variables to be studied, you are ready to
collect the data that you want. In quantitative research, the data collected is in the
form of numbers, not words as in the case of qualitative research. This type of
data is often referred to as quantitative data and there are four types:
(a) Nominal data;
(b) Ordinal data;
(c) Interval data; and
(d) Ratio data.

They are so called in line with the four levels of measurement: nominal, ordinal,
interval and ratio as suggested by Stevens (1951) and cited in Ary, Jacobs and
Razavieh (2002).

Copyright © Open University Malaysia (OUM)

 TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES W 45

3.4.1 Nominal Data

This type of data takes the form of frequency count. For example, you conducted
a Science test based on a topic and collected data on the number of students who
passed or failed the test. Out of the 20 students who took the test, you found that
15 passed but five failed. In this case, the data you have obtained is known as
nominal data. This type of data can also be expressed in percentages as indicated
by Table 3.2.

Table 3.2: Number and Percentages of Students who Passed or Failed their Science Test

PASSED FAILED

Number of Students 15 5
(75 %) (25 %)

In the above example, passed and failed are categorical variables but the
quantitative data is nominal because it is in the form of frequency count.
Sometimes, you may want to collect more information about the students who
passed or failed the Science test. For instance, you wish to find out how many
who passed or failed were male and female students. You have thus included an
additional categorical variable, which is gender, in the data collection process.
The information may be cross-tabulated as shown in Table 3.3 below:

Table 3.3: Number and Percentages of Students who

Passed or Failed their Science Test by Gender

PASSED FAILED
9 1
Male (60%) (20%)
6 4
Female (40%) (80%)

3.4.2 Ordinal Data

This type of data is in the form of rank, either from the lowest to the highest rank,
or vice versa. For example, in the Science test above, instead of tabulating the
performance in terms of the number of passes and failures, you can arrange the
20 studentsÊ marks in order from the highest to the lowest, ranging from rank 1
to rank 20, as shown in Table 3.4.

Copyright © Open University Malaysia (OUM)

46 X TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES

Table 3.4: Rank-Order of StudentsÊ Performance in their Science Test

No. Student Mark Rank (Ordinal data)

1 Henry 56 14
2 Peter 70 8
3 James 83 4
4 Mei Lin 0* 20
5 Muthu 91 2
6 Mary 25* 18
7 Jennifer 77 5
8 David 35* 17
9 Joseph 64 11
10 Tony 55 15
11 Jimmy 61 12
12 Patricia 39* 16
13 Tom 58 13
14 Steven 73 7
15 Christine 23* 19
16 Patrick 88 3
17 Ali 95 1
18 Bakar 67 10
19 Christopher 76 6
20 Karam Singh 69 9
* Failure

In this case, the numbers in the rank are ordinal data. Usually, the highest
score is assigned the smallest number while the lowest score is assigned the
biggest number. It must be emphasised that the numbers 1 to 20 themselves
have no particular meaning other than to establish rank. The intervals
between the numbers have no meaning and the numbers themselves have no
absolute quantities.

Copyright © Open University Malaysia (OUM)

 TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES W 47

3.4.3 Interval Data

This type of data can have a value on a continuous scale but it does not have an
absolute zero. This differs from ordinal data in that the intervals between each
number are the same. An example of interval data is the IQ scores which range
from 50 to 150 as measured by the Stanford-Binet Intelligence Scale. In this case,
there is no such thing as zero intelligence. For this reason, it is inappropriate to
say that a person with an IQ score of 140 is twice as intelligent as a person with
an IQ score of 70. Other examples of interval data are temperatures in degree as
measured by a thermometer, attitude scores as measured by an attitude scale, etc.

ACTIVITY 3.3

Refer to the scale in Activity 3.1 and explain the type of data you can
obtain from it.

3.4.4 Ratio Data

This type of data is measured on a continuous scale just like interval data but, unlike
interval data, it does have an absolute zero, which is meaningful. The difference
between ratio data and interval data is subtle. Again, let us refer to the scores
obtained in the Science test (refer to Table 3.4). The numbers as displayed in the
Marks Column are ratio data because it has a true zero, which indicates a total lack
of knowledge of the Science topic being tested. For instance, Mei LinÊs test mark is 0,
indicating that she has no knowledge about the topic. Moreover, you can also say
that Peter, who obtained 70 marks, had correctly answered twice as many questions
as David, who obtained 35 marks. Other examples of ratio data are age, height and
weight.

Good understanding of the four data types is important because it has far-
reaching implications for quantitative data analysis. The appropriateness of the
statistical techniques to be used in the analysis is partly dependent on the types
of data you have collected in your study. You will learn about this in Topics 8
and 9 of this module and in HMEF5113 Statistics for Educational Research.

Copyright © Open University Malaysia (OUM)

48 X TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES

SELF-CHECK 3.1

1. What is a variable?
2. What is the difference between an independent variable and a
dependent variable? Give specific examples.
3. Why do you need to define variables operationally?
4. Explain the differences and similarities of the four types of data.

ACTIVITY 3.4

Make up operational definitions for the following variables:

• Underachievement
• Parental bonding
• Aspirations (of teenagers)
• Autocratic leader (principal or headmaster)
• Teacher burnout
• Socio-economic status
• Leadership
• Reading ability
• Delinquency
• Interests

• A construct is a theoretical concept that is used to describe a specific observed

behaviour or characteristic of individuals.

• When a construct is measured in educational research, it is known as a variable

i.e. something that varies.

• Variables that comprise only two text values or labels are called
dichotomous variables.

Copyright © Open University Malaysia (OUM)

 TOPIC 3 CONSTRUCTS, VARIABLES AND DATA TYPES W 49

• An independent variable can be manipulated to see its effect on a dependent

variable.

• Moderator variables are variables that modify the cause-effect relationship

between independent and dependent variables.

• An extraneous variable is an independent variable that may influence the

outcome of a study but has not been controlled.

• Operational definition of variables is necessary to allow the measurement and

elimination of confusion.

• Nominal data takes the form of frequency count. This type of data can also be
expressed in percentages.

• Ordinal data is in the form of rank, either from the lowest to the highest rank, or
vice versa.

• Interval data can have a value on a continuous scale but it does not have an
absolute zero.

• Ratio data is measured on a continuous scale just like interval data but unlike
interval data, it has an absolute zero, which is meaningful.

Categorical variable
Constructs Interval data
Continuous variable Moderator variable
Data types Nominal data
Dependent variable Operational definition of variables
Extraneous variable Ordinal data
Independent variable Ratio data

Copyright © Open University Malaysia (OUM)

Topic  Experimental
4 Research
Methodology
LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Define what an experiment is;
2. Explain the components of an experiment in education;
3. Identify the threats to the internal validity of experiments;
4. Explain how to control extraneous variables that affect the internal
validity of experiments;
5. Describe how a random assignment is performed;
6. List other techniques for ensuring the equivalence of groups; and
7. Discuss the role of hypothesis testing in an experiment.

 INTRODUCTION

You may have come across these statements or somewhat similar statements.
Note that each statement is making a claim that their proposed method, product,
technique or procedure is effective in enhancing human performance. Obviously,

Copyright © Open University Malaysia (OUM)

 TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY  51

you would like to know how they went about proving „effectiveness‰. How does
one prove effectiveness? Of all available research methods, the experimental
method is the best. You may have conducted science experiments in the
laboratory or in the field. The experimental method was originally used in the
field of agriculture where experiments were conducted to test the effectiveness of
various kinds of treatments such as fertilisers, water and sunlight on plant
growth. The method is used in medical sciences especially in testing the
effectiveness of various kinds of drugs, procedures and therapies on patients.
The experimental method is widely used in education whereby researchers
observe the occurrence of a phenomenon as a consequence of a particular action
or intervention.

4.1 THE EXPERIMENTAL METHOD

An experiment is a research method used to determine the effectiveness of a
particular action or treatment on a single organism or group of organisms. To
show that a particular treatment has an effect or brings about a particular change,
the researcher has to control all other factors that might influence the occurrence
of the particular change. The experimental method is the best method to show
the effectiveness of a particular treatment (e.g. teaching method and curriculum
innovation). Experiments are ideally suited for the task of causal analysis (claim
to show "cause and effect"). No other methods of scientific inquiry permits the
researcher to say with confidence that "X (praising young learners) causes Y (to
repeat the task) to happen".

Hence, it is important that you use the word "effectiveness" carefully, as it only
applies if you are using the experimental method.

Figure 4.1 shows a simple experiment to determine whether teaching young

learners using analogies (e.g. blood circulation is like a river and its tributaries)
"causes" them to perform better academically in science ("effect"). The experiment
involves administering a treatment (independent variable) such as teaching
science using analogies. A pre-test (dependent variable) is given before the
experiment and the same test or equivalent test is given after the experiment. The
differences between pre-test scores and post-test scores will determine whether
teaching using analogies improves performance in science.

Copyright © Open University Malaysia (OUM)

52  TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY

Figure 4.1: A simple experiment

However, experiments are difficult to conduct. Many experiments in education

are concerned about testing the effectiveness of certain interventions or
educational practices on student learning, attitudes, perceptions and so forth. A
key problem in conducting experiments is establishing suitable control, so that
any change in behaviour can be attributed only to the treatment introduced by
the researcher. Control means ruling out other possible causes for the changes in
the behaviour of subjects as shown in Figure 4.1. There are many extraneous
variables (irrelevant, unrelated or unconnected factors) that need to be controlled
so that they do not contaminate or interfere with the findings of the study. Once
an extraneous variable creeps into an experiment, the researcher can no longer
draw any conclusion regarding the causal relationship that exists between the
independent and the dependent variables.

In education, many experiments are conducted in the classroom (natural setting)

and, so, many factors not related to the treatment may influence performance in
the post-test. With reference to Figure 4.1, some students may have discussed
with their friends at home concerning the science topic, while others may have
viewed a programme about the topic on TV. Hence, improved performance on
the post-test may not be attributed to the treatment but to the influence of other
factors. It is thus necessary to control the influence of these outside factors or
variables in order to attain internal validity.

Some experiments have both an experimental group and a control group. An

experimental group consists of subjects who are exposed to the treatment. For
example, a particular counselling technique is used on a group of juvenile
delinquents. The control group consists of subjects who do not receive the
treatment (i.e. they are not „treated‰ with the counselling technique).
Comparison between the experimental group and the control group determines

Copyright © Open University Malaysia (OUM)

 TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY  53

the effectiveness of the counselling technique. In some experiments, there may be

more than one experimental group; subjects treated with two or three different
methods, techniques or procedures are compared with the control group who
does not receive any of the treatments. You can also compare the effectiveness of
different treatments on the dependent variable.

SELF-CHECK 4.1
1. What is unique about the experimental method compared to other
methods of research?
2. What is „treatment‰?
3. What is the difference between an experimental group and a
control group? Why do you need these two groups?

4.2 EXTRANEOUS VARIABLES TO BE

CONTROLLED TO ENHANCE INTERNAL
VALIDITY OF EXPERIMENTS
In conducting experiments, you should ensure that your design has the highest
internal validity possible. What is internal validity? The internal validity of an
experiment is the extent to which extraneous variables (irrelevant variables) have
been controlled or ruled out by the researcher. Internal validity is an indication
that the results you obtain are caused by the treatment you administered and not
some other variables or factors. For example, in your experiment, you taught
(treated) one group of four-year-olds with the whole-word method of reading
and discovered that their reading ability increased by 50% compared to the
group who were taught (treated) the „phonics method of reading‰. How can you
be sure that the increase in reading scores of the whole-word method group is
due to the method taught and not some other factors or variables? Generally, for
an experiment with high internal validity, the probability that the treatment
causes the change is higher.

Campbell and Stanley state that experiments are internally valid when the
obtained effect can be attributed to the manipulation of the independent variable.
In other words, if the effects (e.g. improved scores in mathematics) obtained in
the experiment are due only to the experimental conditions manipulated by the
researcher and not to any other variables (factors), the experiment has internal
validity. In any experiment, there are always some other factors other than the
independent variable (treatment) that can influence the observed effects
(dependent variable). These variables must be identified and dealt with, or held

Copyright © Open University Malaysia (OUM)

54  TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY

constant. Cook and Campbell list a number of factors that can threaten the
validity of experiments. It is important that you know these threats so that you
can take the necessary steps to control the influence of these outside factors to
enhance internal validity.

4.2.1 Time Interval and Threats to Internal Validity

In conducting an experiment, a pre-test and a post-test are administered to
subjects. The time interval between the pre-test and post-test measurement of the
dependent variable can introduce extraneous factors as shown in Figure 4.2.

Figure 4.2: Time interval between pre-test and post-test and threats to internal validity

(a) The first is History, which includes events that have occurred in the
subjectsÊ environment between the pre-test and the post-test that might
affect the scores. For example, the subjects may have experienced events
during the time lapse that affected their attitude, and this is reflected in the
scores of the dependent measure. Generally, the longer the duration
between the pre-test and the post-test, the greater the possibility of history
threatening internal validity. Short time lapses can generate the history
effect too.

(b) The second is Maturation, whereby subjects may change between the pre-
test and post-test, becoming more mature. The change could be both
biological and psychological such as age, learning, fatigue, boredom and
hunger which are not related to specific external events but reside within
the individual.

(c) The third is Instrumentation, whereby change in instruments used in the

pre-test and the post-test can lead to changes in measurement. For example,
an easier test used in the post-test will result in better performance in the
post-test because of the instrument and not the treatment. Alternatively, in

Copyright © Open University Malaysia (OUM)

 TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY  55

your pre-test, you may use a multiple-choice test to measure the effect of
the treatment, while in the post-test, an essay test is used.

(d) The fourth is Testing, whereby subjects remember the questions in the pre-
test and if the same test is given as a post-test, the chances are they may
score higher in the post-test i.e. they have become "test-wise". The period
between the pre-test and the post-test should not be too short to the extent
that subjects can recall the questions.

4.2.2 Other Threats to Internal Validity

Besides the four factors threatening internal validity that we discussed earlier, there
are three other factors. Let us see those factors now.

(a) Mortality
Which is sometimes referred to as "attrition" when subjects drop out from the
experiment which can affect the experiment. This is especially serious when
subjects of a particular characteristic (e.g. high ability) systematically drop out.
(b) Selection Bias
When the subjects selected for the experimental group and the control group
are not equivalent before the treatment, leading to a misleading conclusion. For
example, if the experimental group comprises 50% high ability subjects while
the control group comprises only 25% high ability subjects, higher
performance on the post-test may not be attributed to the treatment but to
non-equivalent subjects in terms of ability.
(c) Regression to the Mean
When subjects with extreme scores on a test are selected, there is a likelihood
that when they are retested later on a measure that is correlated with the first
test, their scores will move towards the mean. For example, if students who
performed poorly are selected for training, their average post-test scores will
be higher than their pre-test scores because of statistical regression even if no
training was given.

4.2.3 Subject-Experimenter Effects to be Controlled

You should know that in an experiment, the experimenter or researcher interacts
with the subjects. We assume that the subjects taking part in the experiment will
listen to the instructions and perform all tasks according to the way the
researcher planned it. However, you should know that in reality this does not
happen because the subjects are of varied backgrounds and have their own
perceptions and opinions. This may lead to subjects responding to the

Copyright © Open University Malaysia (OUM)

56  TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY

experiment in different ways which may affect the experiment. This is termed as
„subject-experimenter effects‰.

(a) Subject Effect ă The perception of subjects when they enter the experiment
can affect how they respond to the tasks required of them. Their perception
of the purpose of the experiment, the task required and the rumours they
hear about the experiment may cause them to behave differently. For
example, subjects who realise that an experiment is about speed of learning
and intelligence might learn the material presented as rapidly as possible to
appear intelligent. Similarly, if a task suggests something about emotional
stability, the subject may respond in such a way as to appear most
emotionally stable.

If there is an experimental group and a control group, there is tendency for

subjects in the experimental group to succumb to the novelty effect because
the treatment given is different from what they are used to. Subjects tend to
be enthusiastic especially in the beginning, which may wear off as the
treatment continues.

(b) Experimenter Effect ă The experimenter has a motive for conducting the
experiment. He or she is attempting to uncover the laws of human
behaviour through experimentation. Towards this goal, the experimenter
expects subjects to be perfect respondents who will cooperate and follow
instructions carefully. The experimenter may be too keen to obtain findings
that confirm the hypotheses and this desire is communicated unconsciously
to subjects. The subtle cues presented by the experimenter are picked up by
subjects and influence their performance in the direction desired by the
experimenter. Certain attributes of the experimenter have shown to
influence subjects. For example, in some experiments, young children
respond more readily to women experimenters compared to their male
counterparts.

If there is an experimental and a control group, the researcher may pay

special attention to subjects in the experimental group which may influence
their behaviour. This is called the Hawthorne Effect. Likewise, there is also
the John Henry Effect, which refers to a situation in which the control
group performs above its usual average when placed in competition with
an experimental group who is using a new method that threatens to replace
the control method. Teachers in the control group feel threatened by the
new methodology and make a strong effort to prove that their way of
teaching is as good as the new method.

Copyright © Open University Malaysia (OUM)

 TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY  57

SELF-CHECK 4.2
1. What is meant by internal validity and why is it an important
ingredient in experimental research?
2. Identify the major extraneous variables that need to be controlled
in an experiment.
3. How do these extraneous variables affect the internal validity of
experiments?
4. Explain how subjects and the experimenter can bias the results of
an experiment.

4.3 RANDOM ASSIGNMENT TO ENHANCE

INTERNAL VALIDITY
An important issue when conducting experiments is how subjects are assigned to
the groups. This is important because it determines whether your study is a true
experiment or a quasi-experiment. This will be discussed in the next topic.
Random assignment means that each sampling unit (e.g. student, teacher, class)
has an equal chance of being selected in the experiment. In designing an
experiment, you should ensure random assignment as it is the best technique
available in establishing that the two or more groups are equivalent. Equivalent
means that the subjects in the two or more groups have more or less similar
characteristics, such as similar ability levels, similar attitudes, similar number of
males and females, similar experiences and similar socio-economic backgrounds.
If the subjects are not randomly assigned, there is the possibility that you may
have disproportionately high ability subjects in one group. If they score high
after the treatment, it may be attributed to the larger number of high ability
subjects rather than the treatment.

A popular technique used to ensure random assignment is Table of Random

Numbers. Say you have 70 subjects to be assigned to two groups as in Figure 4.3.
Assign number 1 to 70 to the subjects. Then, refer to the Table of Random
Numbers as in Table 4.1 and select a starting point; you may take the third
column which has the numbers 26, 54, 37, 98, 39 and so forth. You will select
subject no. 26 assigned to Group 1 followed by subject no. 54 assigned to Group
2. You will ignore number 98 because it is outside the 70 subjects. You will
continue this procedure until all 70 subjects have been assigned to the two
groups.

Copyright © Open University Malaysia (OUM)

58  TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY

Figure 4.3: Random assignment of subjects to two groups

Table 4.1: Table of Random Numbers

23 34 26 91 73 93 83 59 50 51

76 79 54 45 65 13 11 56 91 27

68 57 37 38 45 45 04 85 66 12

45 25 98 63 52 23 03 36 06 08

89 3 39 34 91 94 12 39 13 31

90 26 83 26 21 34 82 07 34 67

23 61 64 65 37 06 54 26 29 75

87 82 51 02 95 64 62 35 96 49

90 71 25 86 62 39 53 49 48 52

12 38 67 09 67 31 45 40 28 31

SELF-CHECK 4.3
1. Why is it important that subjects are assigned randomly in an
experiment?
2. How do you use the Table of Random Numbers to assign subjects
randomly?

Copyright © Open University Malaysia (OUM)

 TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY  59

4.4 OTHER TECHNIQUES TO ENSURE

GROUPS ARE EQUIVALENT
One of the difficult tasks for a researcher using the experimental method is
getting two or more equivalent groups. Imagine the difficulty of finding two
people who are similar on every characteristic such as IQ, attitude, aptitude and
mathematical ability. As mentioned earlier, random assignment is a powerful
way to ensure that subjects assigned to the various groups will have more or less
similar characteristics. There are other techniques used to increase the probability
that the subjects in two or more groups are equivalent.

4.4.1 Matching
Determine a particular factor such as academic performance which is measurable
and can be categorised as High and Low. From the sample, select two High
Ability subjects and randomly assign them to the control group and the
experimental group. Next, select two Low Ability subjects and assign them
randomly to the control group and the experimental. Continue doing this until
all subjects have been assigned and your two groups are matched in terms of
academic performance.

Another technique of matching is to give the pre-test and, based on the scores
obtained, assign subjects to the control group and the experimental group.
However, you should ensure that the average score or mean score of the pre-test
is the same for the two groups. E.g. two subjects with mean 23; two subjects with
mean 30; two subjects with mean 34 and so forth.

4.4.2 Holding One or More Variables Constant

Another method is to hold a particular variable constant. For example, in an
experiment, you have difficulty ensuring that the control group and the
experimental have an equal number of high socio-economic and low socio-
economic subjects. You could take only low socio-economic subjects and assign
them randomly to the control group and experimental group if you are not
interested in comparing high and low socio-economic subjects (see Figure 4.4).
What you have done is to eliminate the socio-economic factor or variable by
including only low socio-economic subjects i.e. controlling, by holding the
socio-economic factor constant across the two groups you are comparing.

Copyright © Open University Malaysia (OUM)

60  TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY

Figure 4.4: Control by holding a variable constant

4.4.3 Including an Extraneous Variable in the

Research Design
You can control a variable by including it in the design and making it another
independent variable. For example, you design an experiment to test the
effectiveness of getting students to define concepts using their own words on
performance in economics as in Figure 4.5. However, you find it difficult to
control prior knowledge in economics among your subjects. You can include
only those who have low prior knowledge based on a test on economics you
administered or you can categorise prior knowledge as High, Medium and Low
based on test scores and treat prior knowledge levels as an independent variable.
However, you should use this technique only if you are interested in the
influence of prior knowledge on performance. What you have done is to control
the influence of prior knowledge on other independent variables by including it
in the research design.

Figure 4.5: Including extraneous variable in the design

Copyright © Open University Malaysia (OUM)

 TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY  61

4.4.4 Analysis of Covariance (ANCOVA)

It is a statistical method used to ensure that the subjects in the control group and
the experimental group are equivalent on various factors. ANCOVA adjusts the
scores on the dependent measure for the differences found on the pre-test and
statistically equates the subjects in the control and experimental group. For
example, you are conducting a study on the effectiveness of metacognitive
training on the critical thinking skills of Form 4 students. However, you find that
some subjects in your experiment are high achievers while the others are low
achievers which may influence performance on the critical thinking test (i.e. the
dependent variable). To ensure that all subjects in the control group and the
experimental group are equal in academic achievement, the ANCOVA is used to
adjust scores on the critical thinking test for the difference in academic
achievement.

"You should keep in mind that ANCOVA is an imperfect statistical technique for
equating experimental groups prior to the treatment period. Only the variables
that are measured can be used as covariates. The groups may differ on other
variables, but if these variables have not been measured, they cannot be entered
into the ANCOVA."

SELF-CHECK 4.4
1. Besides randomisation, what are the three techniques of
increasing the probability that subjects in two or more groups are
equivalent?
2. Explain the differences between these three techniques.

4.5 HYPOTHESIS TESTING

In Topic 1, it was mentioned that after identifying the research problem, the next
step would be to break down the problem into hypotheses or research questions.
A hypothesis is a conjecture (guess or speculation) about the presumed relations
between variables. Why should you state hypotheses? Why not just go ahead and
answer the question? Hypotheses are essential because they are derived from
knowledge obtained from the review of literature of other experiments, theories
and so forth. Such prior knowledge serves as the basis for the hypothesis. An
experiment can either confirm or disconfirm the hypothesis. There are two types
of hypothesis: directional and null.

Copyright © Open University Malaysia (OUM)

62  TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY

4.5.1 Directional Hypothesis

Directional hypothesis states a relationship between the variables being studied
or difference between treatments that the researcher expects to emerge. For
example, the following are directional hypotheses:
(a) Low ability students who receive training in cognitive strategies will score
higher in geography.
(b) Children taught arithmetic using the inquiry method will make greater
gains in arithmetic than comparable children who were taught using the
didactic method.

4.5.2 Null Hypothesis

Opposite of directional hypothesis is null hypothesis, which states that there is
no evidence that the treatment (e.g. inductive approach) has an effect (e.g.
enhanced creative thinking abilities). In other words, any difference that does
exist between the experimental group and the control group is due to chance.
The null hypothesis states that there is no difference between the two groups,
and in statistical terms it is as follows:

H0: M1 = M2 (Equation A)

H0: M1 – M2 = 0 (Equation B)

H0: M1 – M2 is not equal to 0 (Equation C)

[H0 = notation for the null hypothesis. You can have H1, H2, H3 and so on;
which simply means Hypothesis 1, Hypothesis 2, Hypothesis 3 and so forth]

M1 = Mean for Group 1

M2 = Mean for Group 2

So for Equation A, the mean score for Group 1 is more or less EQUAL to the
mean score of Group 2, and if there is no statistical significance difference, then
the null hypothesis is accepted.

Copyright © Open University Malaysia (OUM)

 TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY  63

Equation B is similar. The mean of Group 1 minus the mean of Group 2 is equal
to 0, indicating no significant differences and so, the null hypothesis is accepted.

If the means of the two groups are different and there is a statistical difference,
then you can reject the null hypothesis (Equation C).

4.6 TEST OF SIGNIFICANCE

In order to enable you to reject the null hypothesis, it is necessary to analyse the
data statistically. Why is this necessary? For example, in your experiment, you
obtained the following:

Mean Standard
Deviation
Experimental Group 30.4 3.7
Control Group 28.3 4.1

To the naive person, he or she might conclude that the experimental group
performed better than the control group because the mean score is higher by 2.1
and so the treatment is effective. This is misleading because it is likely that the
differences in the mean between the experimental group and control group could
have occurred by chance. In order for you to accept or reject the null hypothesis,
it is necessary that you analyse the data statistically because you want to be sure
that the treatment administered produced a real effect. How do you determine
that the difference between the two groups is caused by the treatment and not
some other extraneous variables? You could repeat the experiment and see if you
get the same results which will provide evidence on the reliability of the obtained
findings.

However, this is not an economical approach and for this reason statistical tests
are preferred. The test of significance enables one to determine whether the
amount of difference between the two groups is due to chance or treatment. Does
a large difference between the mean score of the experimental and control group
indicate that the difference is real? Even large differences can occur by chance,
although the probability of this happening would be very low. The most
common practice is to state a significance level that must be reached; which is a
statement of the probability that an observed difference is chance difference. The
most common significance levels are .05 and .01; regardless of whether you are
using the t-test, F-test or chi-square.

Copyright © Open University Malaysia (OUM)

64  TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY

If you decide from the onset of the experiment that the .05 significance level is to
be used, it means that you will accept as a real difference only one that is so large
that it could have occurred by chance only 5 times in 100 (i.e. 95% not due to
chance). If the .01 significance level is selected, then the difference can be
expected to occur only 1 time in 100 by chance (i.e. 99% not due to chance).

SELF-CHECK 4.5

1. Why is the statistical test of significance used to determine

differences between means?
2. Explain the difference between .05 and .01 level of significance.

ACTIVITY 4.1

Age Group Pre-Test Mean Post-Test Mean

------------------------------------------------------------------------------
Males 52.4 57.2
Females 53.1 64.5*
-------------------------------------------------------------------------------
Note: * significant at p < .05
The table above shows the pre-test and post-test means on a critical
thinking skills test. The subjects were taught critical thinking skills
one period (40 minutes) a week for six weeks.
(a) Give a title for the study.
(b) State TWO null hypotheses based on the data above.
(c) State TWO conclusions based on the findings.
(d) What is the independent variable and the dependent variable?
(e) Provide an operational definition for the treatment.

Copyright © Open University Malaysia (OUM)

 TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY  65

ACTIVITY 4.2

1. Explain when you would use an experiment in educational

research.
2. What do you mean by the statement that „Experiments allow the
researcher to make causal statements‰?
3. Why should you be concerned about the internal validity of an
experiment?

 An experiment is a research method used to determine the effectiveness of a

particular action or treatment on a single organism or group of organisms.

 The experimental method is the best method to show the effectiveness of a

particular treatment.

 The internal validity of an experiment is the extent to which extraneous

variables have been controlled or ruled out by the researcher.

 History, maturation, testing, selection and instrumentation threaten the

internal validity of experiments.

 Random assignment means that each sampling unit has an equal chance of
being selected in the experiment.

 Random assignment increases the likelihood that groups are equivalent.

 Other methods of ensuring the equivalence of groups are matching, holding

variable constant, including variable in the design and ANCOVA.

 A hypothesis is a conjecture (guess or speculation) about the presumed

relations between variables.

 The test of significance enables one to determine whether the amount of

difference between two groups is due to chance or treatment.

Copyright © Open University Malaysia (OUM)

66  TOPIC 4 EXPERIMENTAL RESEARCH METHODOLOGY

Equivalent groups Internal validity

 Matching  Selection
 Holding a variable  Maturation
 Including a variable  Testing
 ANCOVA  Instrumentation
Experimenter effect  History
Hypothesis testing Random assignment
 Directional hypothesis  Table of Random Numbers
 Null hypothesis  Test of significance
Subject effects
The experiment
 Pre-test and post-test
 Control and experimental groups

Copyright © Open University Malaysia (OUM)

Topic X Experimental
Research
5 Designs
LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Define what research design is;
2. Distinguish the ways in which good research designs differ from weak
research designs;
3. Explain the differences between a true experimental design and a
quasi-experimental design;
4. Elaborate on the concept of main effects and interaction; and
5. Discuss the ethics of experimental research.

X INTRODUCTION
What is meant by research design? According to Christensen (1988), „research
design refers to the outline, plan or strategy specifying the procedure to be used
in seeking an answer to the research question. It specifies such things as how to
collect and analyse the data‰. The design of an experiment will show how
extraneous variables are controlled or included in the study (refer to the control
techniques discussed in Topic 4). The design will determine the types of analysis
that can be done to answer your research questions and the conclusions that can
be drawn. To what extent your design is good or bad will depend on whether
you are able to get the answers to your research questions. If your design is
faulty, the results of the experiment will also be faulty. How do you go about
getting a good research design that will provide answers to the questions asked?
It is not easy and there is no fixed way of telling others how to do it. The best that
can be done is to examine different research designs and to point out their
strengths and weaknesses, and leave it to you to make the decision.

Copyright © Open University Malaysia (OUM)

68 X TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS

You should have an in-depth understanding of your research problem, such as

the treatment you want to administer, the extraneous variables or factors you
want to control and the strengths and weaknesses of the different alternative
designs. You should be clear about your research questions and what it is you
intend to establish. You should avoid selecting a design and then trying to fit the
research question to the design. It should be the other way round! What is most
important is to see whether the design will enable you to answer the research
question. You should be clear about what factors you wish to control so that you
can arrive at a convincing conclusion. Choose a design that will give you
maximum control over variables or factors that explain the results obtained.

5.1 SYMBOLS USED IN EXPERIMENTAL

RESEARCH DESIGNS
Research design can be thought of as the structure of research i.e. it is the „glue‰
that holds all of the elements in a research project together. In experimental
research, a few selected symbols are used to show the design of a study.

O = Observation or Measurement (e.g. mathematics score, score on an

attitude scale, weight of subjects, etc).

O1, O2, O3 ⁄⁄⁄⁄ On = More than one observation or measurement.

R = Random assignment: Subjects are randomly assigned to various

groups.

X = Treatment which may be a teaching method, counselling technique,

reading strategy, frequency of questioning and so forth.

5.2 WEAK DESIGNS

In the following sections you will learn about three weak designs.

5.2.1 One-Shot Design

Let's say you want to determine whether praising primary school children makes
them do better in arithmetic as in Figure 5.1. You measure arithmetic
achievement with a test. To test this idea, you choose a class of Year 4 pupils,
increase the act of praising the children and you find that their mathematics
performance significantly improves.

Copyright © Open University Malaysia (OUM)

 TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS W 69

Figure 5.1: One-shot design

You conclude that praising children increases their mathematics score. This
design is weak for the following reasons:
(a) Selection Bias: It is possible that the pupils you selected as subjects were
already good in mathematics.
(b) History: The school had organised a motivation course on mathematics for
Year 4 pupils. So, it is possible that it might influence their performance.

5.2.2 One-Group Pre-Test and Post-Test Design

To ensure that there was no pre-existing characteristic among the school
children, a pre-test may be administered as illustrated in Figure 5.2. If the
children performed better in mathematics after praising compared to the pre-test,
then you can attribute it to the practice of praising.

This design is weak for the following reasons:

(a) Maturation: If the time between the pre-test and the post-test is long, it is
possible that the subjects may have matured because of developmental
changes.
(b) Testing: Sometimes the period between the pre-test and the post-test is too
short and there is the possibility that subjects can remember the questions
and answers.

Figure 5.2: One-Group Pre-Test Post-Test design

Copyright © Open University Malaysia (OUM)

70 X TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS

ACTIVITY 5.1

Twenty pupils who were weak in arithmetic were taught arithmetic

using the Zandox method. Three weeks later, when they were tested,
their arithmetic scores improved. Thus, the Zandox method
improves arithmetic performance.
1. Which type of research design is this study based on?
2. What are some problems with this design?

5.2.3 Non-Equivalent Post-Test Only Design

The main weakness of the previous two designs is the lack of a comparison
group and the consequent difficulty of saying conclusively that the treatment
(„praising‰) contributed to higher mathematics score. In the Non-Equivalent
Post-Test Only Design, an attempt is made to include a comparison group (i.e.
control group) that did not receive „praise‰ as in Figure 5.3. The dashed lines
separating the experimental group and the control group indicate that the
children were not randomly assigned to the two groups. Hence, the two groups
are non-equivalent. Matching can be used but there is no assurance that the two
groups can be equated (see Topic 4). The only way one can have assurance that
the two groups are equated is to assign the children randomly.

Figure 5.3: Non-Equivalent Post-Test Only design

This design is weak for the following reason:

Selection Bias: Since there was no random assignment, it cannot be established
that the two groups are equivalent. So, any differences in the post-test may not be
attributable to giving praise but other factors such as ability, IQ and interest.

The three designs described are „weak‰ research designs because they do not
allow for controlling of extraneous factors that might creep into the experiment.
These factors may affect the results of the dependent measure. For example, if the
attitude towards mathematics and outside tuition in mathematics is not

Copyright © Open University Malaysia (OUM)

 TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS W 71

controlled, it may not be possible to conclude that „praise‰ (treatment) affects

mathematics performance (dependent variable). Also, weak research designs do
not attempt to randomly assign subjects to the groups being compared, which
introduces extraneous factors affecting the dependent measure. Random
assignment controls both known and unknown extraneous variables that might
affect the results of the experiment.

SELF-CHECK 5.1
1. Identify the major differences between one-shot design, one-
group pre-test post-test design and non-equivalent post-test
only design.

2. Why are these designs considered weak?

ACTIVITY 5.2

A teacher assigns one class of pupils to be the experimental group and

another class as the control group. Both groups are given a science
post-test. The pupils in the experimental group are taught by their
peers, while pupils in the control group are taught by their teacher.
1. Which research design is the teacher using?
2. How will you challenge the findings of the experiment?

5.3 TRUE EXPERIMENTAL DESIGNS

In this section, we will discuss some „true‰ experimental research designs. What
is a „true‰ experimental design? According to Christensen, „to be a true
experimental design, a research design must enable the researcher to maintain
control over the situation in terms of assignment of subjects to groups, who gets
the treatment condition, and the amount of treatment condition that subjects
receive‰. In this topic, we will discuss two major types of true designs: 1) after-
only design and 2) before-after design as illustrated in Figure 5.4. What is the
difference between the two designs? The after-only design relies only on a post-
test while the before-after design (as the name suggests) relies on both a pre-test
and a post-test.

Copyright © Open University Malaysia (OUM)

72 X TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS

True Experimental Design

Figure 5.4: Types of true experimental designs

5.3.1 After-Only Research Design

The After-Only Research Design gets its name from the fact that the dependent
variable is measured only once after the experimental treatment. In other words,
the post-test is administered once to the experimental group and the control
group as provided in Figure 5.5. It shows an experiment in which the researcher
is attempting to show the effectiveness of the inductive method in improving the
science problem skills of 17-year-old secondary school students. The sample was
drawn from a population and randomly assigned to the experimental and control
group. The experimental group was taught science using the inductive approach
while students in the control group were not taught using the inductive
approach. Instead, students in this group were taught the same science content
using the traditional didactic approach („chalk-and-talk‰ method).

Note: R ă Random Assignment

Figure 5.5: After-only research design

In the above example, the experimental and control groups comprise two
different sets of students. This procedure is called a between-subjects design (also
sometimes known as an independent or unrelated design). One advantage of this
design is that the students are less likely to get bored, tired or frustrated with the
study because each set of students is exposed to only one condition. In a similar
Copyright © Open University Malaysia (OUM)
 TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS W 73

vein, the research is less susceptible to practice and order effects. On the negative
side, you will need more students to participate in your research. There is also a
need to ensure that both groups of students are homogeneous in any
confounding variables that might affect the outcome of the study. This is because
different students bring different characteristics to the experimental setting. Even
though we randomly assign students to experimental and control conditions, we
might by chance allocate students with one characteristic to one condition, and
this might confuse our results.

Another research procedure in the after-only design is within-subjects design

(sometimes known as repeated measures or related design). In this design, the
same students are exposed to two or more different conditions under
comparison. For example, you wish to study the effects of content familiarity on
reading comprehension performance. You can assign the same students to read
two types of passages, one familiar and the other unfamiliar, and then analyse
their comprehension performance. One obvious advantage is that you need
fewer students to participate in your research. Besides, you will have much
greater control over confounding variables between conditions because the same
students are used in both the conditions. By and large, the same individual will
bring the same characteristics to the conditions. However, it is not all rosy in the
within-subjects design. First, since the same students are exposed to different
conditions, they might get bored or tired when receiving the experimental
treatment in the later conditions. Besides, there is an increased likelihood of
practice and order effects. One way to eliminate these effects is to introduce
counterbalancing into your design. In counterbalancing, you get one half of your
students to complete the first condition followed by the second condition. You
then get the other half of your students to do the two conditions in the appositive
order, that is, the second condition followed by the first condition.

5.3.2 Factorial Research Design

Factorial design is an after-design research design that allows the study of two or
more independent variables simultaneously and their interactive effects on the
dependent variable. To explain factorial design, a hypothetical example is shown
in Figure 5.6. The experiment (2 x 2 factorial design) aims to examine the
effectiveness of two teaching methods (Independent Variable A) on performance
in history (Dependent Variable) among a sample of 17-year-old students of
different ability levels (Independent Variable B).

Copyright © Open University Malaysia (OUM)

74 X TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS

(a) Method (Independent Variable A) is made up of two methods:

(i) Deductive teaching method: In this method, students are presented
with a concept followed by the examples.
(ii) Inductive teaching method: In this method, students are presented
with examples and from these examples they derive the concept.
(b) Ability (independent variable B) is divided into two levels based on their
academic performance scores:
(i) High ability; and
(ii) Low ability.

Figure 5.6: A 2 x 2 factorial design

So there are four possible combinations of the two independent variables. Each of
these treatment combinations is referred to as cells (i.e. A1B1, A2B1, A2B1 and
A2B2). Subjects are randomly assigned to these four cells within the design. For
the experiment using this factorial design, you are looking for three different
kinds of effect: the main effect of method, the main effect for ability and the
interaction between method and ability.

In this example, you are able to test three null hypotheses:

1. There is no significant difference between the inductive method and the
deductive method on performance in history (main effect ă Method).
2. There is no significant difference between high and low ability students on
performance in history (main effect ă Ability).
3. There is no significant interaction between method and ability (interaction
effect).
Copyright © Open University Malaysia (OUM)
 TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS W 75

Figure 5.7: Factorial design showing means for ability and method

Results of the hypothetical experiment are shown in Figure 5.7. The main effects
for methods (variable A) showed that there was a significant difference in history
performance between students taught the inductive (Mean = 50.0) and the
deductive approach (M = 40.0). This means that method had an „effect‰ on
history performance. There was also a main effect for ability (variable B) where a
significant difference was observed between high (M = 55.0) and low ability (M =
35.0) students on performance in history. Similarly, it means that ability had an
„effect‰ on history performance.

However, the interaction effect was not statistically significant. What is an

interaction? Let's say a psychologist is asked, „Does listening to a motivation talk
improve academic performance?‰ When the psychologist replies, „Yes, but it
depends on ⁄‰ or „It is more complicated than that‰, he or she is referring to
„interaction‰. An interaction effect tells us about the influence of one
independent variable on another. In the case of our hypothetical example, it is
whether the combination of „method‰ and „ability‰ produced an effect on
performance in history. Though the two variables (method and ability) by
themselves had a significant effect, the combination of method and ability did
not produce an effect on performance in history.

In a 2 X 2 experiment (the hypothetical experiment), you can obtain eight basic

patterns of results (see Figure 5.8).

Copyright © Open University Malaysia (OUM)

76 X TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS

Figure 5.8: Patterns of effects in a 2 X 2 factorial design

(a) Main Effect for Method and Main Effect for Ability and No Interaction (No: 3)
Let us examine what this means with our hypothetical example. The data in
Figure 5.7 indicates that you have main effects for both method and ability.
Look at the first row. You can see that high ability learners treated with the
inductive method (M = 60.0) scored higher than high ability learners treated
with the deductive method (M = 50.0). Looking at the next row, you see
that low ability learners treated with the inductive method (M = 40.0)
scored higher than low ability learners treated with the deductive method
(30.0). You can see this in Figure 5.8.

Figure 5.9: Graph showing no interaction

Copyright © Open University Malaysia (OUM)

 TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS W 77

Looking at the columns tells you about the effect of ability. You see that
high ability learners treated with the inductive method scored 60.0
compared to their low ability counterparts who scored 40.0 (i.e. 20 more) in
the same treatment group. Looking at the second column, you learn that
high ability learners (M = 50.0) treated with the deductive method scored
higher than low ability learners (M = 30.0) treated with the same method.
Thus, it appears that in addition to the method main effect, you have an
ability main effect.

Finally, you also know that there is no interaction because the effect of
method is unaffected by the ability level of students. As Figure 5.7
demonstrates, the effect of method is independent of the ability level and
the effect of ability level is independent of method of instruction. If you
graph the means, your graph should look something like Figure 5.9. The
graph confirms what you saw in Figure 5.7. The high ability line is above
the low ability line. Similarly, ability increases as shown by the fact that
both lines slope upwards as they go from the deductive to the inductive
method. Finally, the graph tells you that there is no interaction between
method and ability on performance in history because the lines are parallel.

(b) No Main Effect for Method and No Main Effect for Ability but an
Interaction (No: 7)
Let us examine what this means with our hypothetical experiment.
According to Figure 5.10, the means obtained on history performance
according to method reveals no significant difference between the inductive
method (M=55.0) and the deductive method (M=55.0). Similarly, for ability,
there was no significant difference between high ability students and low
ability students. However, there was an interaction and the interaction was
significant (see Figure 5.11). In this figure, you noticed that the lines are not
parallel (as in Figure 5.9). Therefore, you have an interaction. What is the
meaning of this interaction since there was no effect for either method or
ability? You would say that method has an effect, but its effect depends on
ability level. Alternatively, you could say that ability has an effect but that
effect depends on the type of method students had been treated with.

Copyright © Open University Malaysia (OUM)

78 X TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS

Figure 5.10: Factorial design showing means for methods and ability

Figure 5.11: Graph showing an interaction

ACTIVITY 5.3

A lecturer doing an experiment finds that students who are given lecture
notes but do not attend the lecture perform better than those who attend
the lecture. Refine the study by using a 2 X 2 factorial design.

Copyright © Open University Malaysia (OUM)

 TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS W 79

5.3.3 Before-After Research Design

Before-After Research Design is perhaps the best example of a true experimental
design that incorporates both an experimental group and a control group to
which the subjects are randomly assigned as shown in Figure 5.12. This research
design is a good experimental design because it does a good job of controlling
extraneous factors such as history, maturation, instrumentation, selection bias
and regression to the mean. How is this done? Any history events (e.g. certain
events subjects may have been exposed to) that may have produced a difference
in the experimental group would also have produced a difference in the control
group. Here, it is assumed that subjects in both groups have experienced the
same set of events.

Figure 5.12: Before-after research design

Thus, we can conclude that a true experimental research has three distinct
characteristics (sometimes referred to as three basic principles of
experimentation).
(a) First, it involves the intervention or treatment of independent variables. The
independent variables are manipulated systematically to examine the
effectiveness of the treatment.
(b) Secondly, there is a control of variables. The purpose of this control is to rule
out extraneous variables that might confound the experiment.
(c) Thirdly, a true experiment requires an appropriate comparison. For instance,
comparison is made between two or more groups that are treated
differently.

SELF-CHECK 5.2
1. What is the main strength of „true‰ experiments?
2. What is the major difference between the two types of true
experiments; after-only research design and before-after research
design?

Copyright © Open University Malaysia (OUM)

80 X TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS

SELF-CHECK 5.3
1. What is the main advantage of using the factorial design?

2. Why is the factorial design considered a true experiment?

3. Identify the differences between main effects and interaction effects.

5.4 QUASI-EXPERIMENTAL DESIGN

So far, we have examined both weak and strong experimental research designs.
However, in educational research, there are times when investigators are faced
with situations in which all the requirements of a true experiment cannot be met.

For example, sometimes it is not possible to assign students to groups which are
a requirement of strong experimental research. Due to logistical reasons, it is
difficult to randomly assign subjects to groups and, so, intact groups such as a
class may have to be used. Is it still possible to do an experiment despite these
limitations? The answer is yes, you can use a quasi-experimental design.

According to Christensen and Johnson, a quasi-experimental design is „an

experimental research design that does not provide for the full control of
potential confounding variables. In most instances, the primary reason that full
control is not achieved is that participants cannot be randomly assigned‰.

5.4.1 Non-Equivalent Control-Group Design

Figure 5.13: Non-equivalent control-group design

Non-equivalent control-group design contains an experimental group and a

control group, but the subjects are not randomly assigned to groups as stipulated
in Figure 5.13. The fact that there is no random assignment means that subjects in
the experimental group and control group may not be equivalent on all variables.
For example, you could have more low ability students in the control group

Copyright © Open University Malaysia (OUM)

 TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS W 81

compared to the experimental group. Hence, it may be difficult to establish

whether the better performance of the experimental group is due to the treatment
or that there are more high ability students in the group.

In non-equivalent control-group design, both groups are given first a pre-test and
then a post-test (after the treatment is given to the experimental group). The pre-
test score and the post-test score are compared to determine whether there are
significant differences.

When you cannot randomly assign subjects, you can be sure that extraneous
variables or factors will creep into the experiment and threaten its internal
validity. Do you leave it alone or do something about it?

Knowing that extraneous factors will creep into a quasi-experiment, a good

researcher will take steps to ensure that the subjects in the experimental group
and control group are as similar as possible, especially on important variables
such as academic ability, attitude, interest and socio-economic status. How do
you go about doing this?

Cook and Campbell proposed the following steps to enhance the internal validity
of non-equivalent control-group design or quasi-experiments in general:
(a) Selection: Ensure that subjects in experimental and control groups are
matched in terms of important variables that may affect the results of the
experiment. For example, match subjects in terms of academic ability, IQ,
attitudes, interests, gender and socio-economic background.
(b) Testing: Ensure that the time period between the pre-test and post-test is
not too short such that subjects are able to remember the questions given to
them earlier.
(c) History: Ensure that events outside the experiment do not affect the
experiment. The problem is most serious when only subjects from one of
the groups are exposed to such events (e.g. motivation talks and private
tuition).
(d) Instrumentation: Ensure that the pre-test and the post-test are similar. If a
different test is used, you should make sure that the two tests are
equivalent in terms of what they measure (i.e. high reliability and validity).

5.4.2 Interrupted Time Series Design

Interrupted time-series design requires the researcher to take a series of
measurements both before and after the treatment. A single group of subjects is
pre-tested a number of times during the baseline phase, exposed to the treatment,

Copyright © Open University Malaysia (OUM)

82 X TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS

and then posted a number of times after the treatment. „Baseline‰ refers to the
testing done before the treatment designed to alter behaviour.

A hypothetical example may illustrate how the interrupted time series design is
used. For example, you want to determine whether positive reinforcement
encourages inattentive low ability learners to be more attentive. You identify a
group of 11-year-olds who are low ability learners and get them to attend an
experimental classroom for at least one period each school day as in Figure 5.14.
In this classroom, subjects were taught reading skills in a positive environment
where they were praised and rewarded for attentive behaviour to the given task
activities. Before the students were sent to the positive treatment classroom, their
behaviour was observed over three sessions in their regular classroom with
regards to their attentiveness. This was to obtain baseline data where their
behaviour was recorded in its freely occurring state. The treatment lasted three
weeks and after the treatment, subjects were observed for their attentiveness and
focused behaviour.

Figure 5.14: Interrupted time-series design

Figure 5.15: Percentage of students observed to be attentive and focused

Copyright © Open University Malaysia (OUM)

 TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS W 83

The result of the hypothetical experiment is shown in Figure 5.15, which

illustrates the percentage of students who were attentive and focused on the
given task. From this graph, you can see that the percentage of students who
were attentive and focused and who were assessed multiple times prior to and
after the implementation of the positive classroom environment, making it an
interrupted time-series design. This assessment reveals that the percentage of
students who were attentive and focused remained rather constant during the
first three baseline class sessions, or the class sessions prior to the
implementation of the positive classroom environment. After the implementation
of the positive classroom environment, the percentage of attentive behaviour
consistently increased over the next three class sessions, suggesting that the
implementation of the positive approach had a beneficial effect on the behaviour
of inattentive students.

SELF-CHECK 5.4
1. What is the meaning of non-equivalent in non-equivalent control
group design?
2. How can you enhance the internal validity of quasi-experimental
research designs?
3. When would you use interrupted time-series design?

5.5 ETHICS IN EXPERIMENTAL RESEARCH

During World War II, Nazi scientists conducted cruel experiments such as
immersing people in ice water to determine how long it would take them to
freeze to death. They also injected prisoners with newly developed drugs to
determine their effectiveness and many died in the process. These experiments
were conducted by individuals living in a demented society and were
universally condemned as unethical and inhumane. Research in education
involves humans as subjects: students, teachers, school administrators, parents
and so on. These individuals have certain rights, such as the right to privacy that
may be violated if you are to attempt to arrive at answers to many significant
questions. Obviously, this becomes a dilemma for the researcher as to whether to
conduct the experiment and violate the rights of subjects, or abandon the study.
You might have heard people say: „I guess we are the guinea pigs in this study!‰
or „We are your white rats!‰

Copyright © Open University Malaysia (OUM)

84 X TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS

Any researcher conducting an experiment must ensure that the dignity and
welfare of the subjects are maintained. The American Psychological Association
published the Ethical Principles in the Conduct of Research with Human
Participants in 1982. The document listed the following principles:
(a) In planning a study, the researcher must take responsibility to ensure that
the study respects human values and protects the rights of human subjects.
(b) The researcher should determine the degree of risk imposed on subjects by
the study (e.g. stress on subjects, subjects required to take drugs).
(c) The principal researcher is responsible for the ethical conduct of the study
and be responsible for assistants or other researchers involved.
(d) The researcher should make it clear to the subjects before they participate in
the study regarding their obligations and responsibilities. The researcher
should inform subjects of all aspects of the research that might influence
their decision to participate.
(e) If the researcher cannot tell everything about the experiment because it is
too technical or it will affect the study, then the researcher must inform
subjects after the experiment.
(f) The researcher should respect the individualÊs freedom to decline to
participate in or withdraw from the experiment at any time.
(g) The researcher should protect subjects from physical and mental
discomfort, and harm and danger that may arise from the experiment. If
there are risks involved, the researcher must inform the subjects of that fact.
(h) Information obtained from the subjects in the experiment is confidential
unless otherwise agreed upon. Data should be reported as group
performance and not individual performance.

SELF-CHECK 5.5

What are some ethical principles proposed by the American

Psychological Association with regard to doing experiments involving
human subjects?

Copyright © Open University Malaysia (OUM)

 TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS W 85

ACTIVITY 5.4

1. Make a case for the superiority of true experimental designs.

2. What are quasi-experimental research designs and how do they
differ from true experiments?
3. Discuss the circumstances in which researchers have to use intact
groups.
4. What can a researcher do to increase the equivalence of subjects in
the control and experimental groups in a quasi-experiment
design?

5. Graph the following data from an experiment on the effect of

lighting and music on anxiety. The scores are means on an anxiety
test.

Music
Classical Rock
Lighting Dim 45 11
Level
Bright 12 44

• A research design is a plan or strategy specifying the procedure in seeking an

answer to the research question.

• „Weak‰ research designs do not allow for controlling of extraneous variables

that might creep into the experiment.

• Examples of weak designs are one shot design, one-group pre-test post-test
design and non-equivalent post-test only design.

• „True‰ experimental designs enable the researcher to maintain control over

the situation in terms of assignment of subjects to groups.

• Examples of true designs are after-only research design, factorial design and
before-after research design.
Copyright © Open University Malaysia (OUM)
86 X TOPIC 5 EXPERIMENTAL RESEARCH DESIGNS

• A quasi-experimental design is a design that does not provide for the full
control of potential confounding variables.

• Examples of quasi-experimental designs are non-equivalent control-group

design and interrupted time-series.

• Researchers conducting experiments involving human subjects should

respect the privacy and confidentiality of subjects.

Experimental design True research designs:

Quasi-experimental design − After-only design
Non-equivalent design − Factorial design
Time series design − Before-after design

Weak research designs:

− One-shot design
− One-group pre-test post-test
− Non-equivalent post-test only

Copyright © Open University Malaysia (OUM)

Topic X Survey Research
6 Methodology

LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Define what a survey is;
2. Compare the different types of survey methods;
3. Explain the process of selecting a sample using different techniques;
4. List seven major steps in conducting survey research;
5. Discuss the advantages and disadvantages of different survey data
collection methods; and
6. Elaborate on the ethics involved in conducting surveys.

X INTRODUCTION
Do you want to know what people are thinking, feeling or doing? If you do, then
a survey would be the best research method. You can survey them orally through
an interview or you can give them a questionnaire. By asking questions, you are
tapping into peopleÊs attitudes, beliefs, perceptions and behaviour. The survey
has proven to be a powerful tool in gathering information about the opinions,
attitudes and intentions of people concerning different social, cultural, economic
and political issues.

It is quite common for newspapers, radio and television to report about the
opinions and perceptions of people obtained through surveys (at times using
survey data to sensationalise issues). Surveys are used by various government
agencies, non-governmental agencies, business and scientific organisations to
probe into peopleÊs reactions to different issues and trends. For example:
• TV stations rely on surveys to profile the people who watch programmes
over their stations.

Copyright © Open University Malaysia (OUM)

88 X TOPIC 6 SURVEY RESEARCH METHODOLOGY

• Automobile companies use surveys to find out about customer satisfaction

about the cars they produce or sell.
• Statistics departments conduct surveys to measure consumer price index
with the purpose of providing advice on government policies.
• Engineering companies conduct surveys to gather information about the
habits of road users.
• Magazine companies conduct surveys to find out about the reading habits
and interests of their subscribers.
• Manufacturing companies conduct surveys to determine consumer
acceptance of their products.

Malaysian Teens Grade Dad a C-

A sample of 3,212 young people from Hong Kong, Indonesia, Philippines,

Singapore, South Korea, Taiwan, Thailand and Malaysia, aged between
14 and 18, took part in the survey. The survey required participants to
grade their parents from A to F based on 35 statements given.

The survey was designed to allow children to reflect on and rank their
parentÊs ability to show respect and love, have integrity and patience and
be open-minded, among a myriad of other skills and characteristics.

Although the Malaysian dad ranks lower than the Malaysian mum, a
general analysis of the survey results reveals fascinating details of
Malaysian fathers as seen through the eyes of young adults. For instance:
• Dad got a B for telling jokes and C for being fashionable.
• Dad does not really allow them much independence (C-) but
respects their privacy (B) and trusts them (B).
• Dad takes the trouble to teach them right from wrong (B+).
• Dad accepts their own taste in music (B).
• Dad talks about sex (D).

Source: Sunday Star, 27 July, 2005, p.22

Copyright © Open University Malaysia (OUM)

 TOPIC 6 SURVEY RESEARCH METHODOLOGY W 89

ACTIVITY 6.1
Read the newspaper report on „Malaysian teens grade dad a C-‰.
(a) What are your views about the findings of the survey?
(b) What other information would you need about the survey to
accept the findings?

6.1 WHAT IS A SURVEY?

What is a survey? According to Kerlinger, survey research involves the study of
large and small populations, and the selecting and studying of samples chosen
from the populations to discover the relative incidence, distribution and
interrelations of sociological and psychological variables. It is a method of
obtaining information about a population from a sample of individuals. Surveys
can provide a quick, inexpensive and accurate means of obtaining information
from a large group of people. If you want to know about the opinions, attitudes
and perceptions of respondents, the survey is an appropriate method of
collecting data. Besides, describing surveys can also be used to explain the
relationship and differences between variables. The term „sample survey‰ is
often used because a sample which is representative of the target population is
used. The survey method is widely used in social sciences, education, business
and medicine. Basically, information is obtained by asking people questions
either orally or by responding to a written paper or computer screen concerning:
(a) What they know: (Who was the first Prime Minister of Malaysia?)
(b) What they believe: (Should students be given freedom to express
themselves?)
(c) What they expect: (Do you expect to be a famous person?)
(d) What they feel: (Do you think your school principal is fair?)
(e) What they have done: (How often do you use the computer in a week?)
(f) What they plan: (Do you intend to continue studying or work?)

Copyright © Open University Malaysia (OUM)

90 X TOPIC 6 SURVEY RESEARCH METHODOLOGY

6.2 TYPES OF SURVEY

Surveys provide an important source of basic scientific knowledge. Economists,
psychologists, health professionals, political scientists, educationists and
sociologists conduct surveys to study such matters as income and expenditure
patterns among households, the roots of ethnic or racial prejudice, the
implications of health problems on people's lives, comparative voting behaviour,
factors influencing academic performance, the effects on family life of women
working outside the home and so on. To serve these different needs, there are
two main types of survey as in Figure 6.1. The types of survey used will depend
on the objectives of the study. If the study aims to get a snapshot of opinions and
practices, then cross-sectional survey would be most appropriate. If the objective
is to compare differences in opinions and practices over time, then longitudinal
survey would be the obvious choice.

Figure 6.1: Types of survey

6.2.1 Cross-Sectional Survey

Just like all surveys, cross-sectional survey collects information from a sample
drawn from a population. It involves collecting data at one point of time.
However, the period for data collection can vary from one week to six months. If
you are using a questionnaire to collect data, you can ask respondents about the
past, present or future. For example, you administered a questionnaire on habits
and attitude towards smoking to 500 students in secondary school aged between
14 and 16 years old on 6 September 2012. The students included males and
females from different socio-economic backgrounds in the state of Selangor. The
data you obtained is a cross-section of the population at one point of time.

Copyright © Open University Malaysia (OUM)

 TOPIC 6 SURVEY RESEARCH METHODOLOGY W 91

6.2.2 Longitudinal Survey

In longitudinal surveys, data are collected at different points in time in order to
study changes. There are two common types of longitudinal surveys:
• Cohort studies: You identify a specific population (e.g. teachers in Perak who
have a masters degree) and list the names of all members of this population.
At each data collection point, you select a sample of respondents from the
population of Perak teachers with a masters degree and administer a
questionnaire (e.g. about their aspirations). At another point, you might select
another sample from the same population of teachers and administer the
same questionnaire. Thus, although the population remains the same,
different individuals are sampled each time. Your aim is to see if there are
changes in perceptions or trends that are present.
• Panel studies: You identify a sample from the beginning and follow the
individuals over a period with the aim of noting changes in specific
respondents and exploring reasons why these individuals have changed. For
example, you want to find out about changes in racial attitudes among a
group of primary school children. You administer an attitude scale to a group
of Year 5 pupils and then administer the same scale when they are in Year 6
and so on. You analyse the data to see if there are changes in racial attitudes
as children grow older. The only problem is the loss of subjects which you
cannot replace.

SELF-CHECK 6.1

1. Define survey in your own words.

2. What is the main difference between cross-sectional surveys and
longitudinal surveys?

6.3 SAMPLING
Unlike a census, where all members of the population are studied, surveys gather
information from only a portion of the population of interest ă the size of the
sample will depend on the purpose of the study. In a good survey, the sample is
not selected haphazardly or only from persons who volunteer to participate. It is
scientifically chosen so that each person in the population will have a measurable
chance of being selected. This procedure is called sampling. This way, the results
can be reliably projected from the sample to the larger population. Two key
words involved in sampling are population and sample. According to Borg and
Borg (1988), the word „population‰ is defined as all people, objects or events
Copyright © Open University Malaysia (OUM)
92 X TOPIC 6 SURVEY RESEARCH METHODOLOGY

found in a particular group the researcher is planning to generalise. For example,

the population of primary school students, graduate teachers, medical doctors in
Malaysia, senior citizens in Selangor and secondary school students in Sarawak.

If the population is small, e.g. school principals in Kuala Lumpur, the researcher
may decide to study all subjects. When the population is large, the researcher is
not able to study all individuals. It would be too expensive and time-consuming
even for large research organisations. Only in a census is the whole population
studied which involves enormous expenditure, time and many research
assistants. Just imagine having to administer a questionnaire to 100,000 sixteen-
year-olds in Malaysian schools!

Figure 6.2: Selection of a sample representative of the population

Copyright © Open University Malaysia (OUM)

 TOPIC 6 SURVEY RESEARCH METHODOLOGY W 93

Hence, the researcher has to select a certain number of subjects or a sample from
the population to study. Regardless of the method used in selecting the sample,
the most important thing is to ensure that the sample is representative of the
population. Figure 6.2 shows a population of 10,000 individuals who possess
certain characteristics in terms of gender, socio-economic status and ethnicity. If
you intend to select 5% of individuals from the population to form a sample, you
should ensure that the sample has similar characteristics to the population. This
is called population validity whereby the researcher is trying to show that the
sample is representative of the population according to the variables specified.
Using a sample is more economical and if the sample is selected appropriately,
the researcher can make conclusions about the population based on the results
from the sample. This is called generalisation.

ACTIVITY 6.2

Refer to Figure 6.2 and explain how the sample selected is

representative of the population.

6.4 SAMPLING TECHNIQUES

As mentioned earlier, surveys rely on samples to make projections about the
population. How does one select a sample? The sampling techniques used will
depend on the objectives of the study and the resources available. Generally,
there are two types of sampling techniques:
(a) Probability sampling: This includes techniques that select samples based on
random selection. Among the techniques are random sampling, systematic
sampling, stratified sampling and cluster sampling.
(b) Non-probability sampling: This includes techniques that are not based on
random selection. Among the common techniques are quota sampling,
purposive sampling and convenience sampling.

6.4.1 Probability Sampling Techniques

(a) Random sampling
One of the techniques to increase the probability that the sample selected
represents the population is random sampling. With this technique, each
member of a population has an equal chance of being selected. For example,
you are interested in surveying the attitudes of graduate teachers towards

Copyright © Open University Malaysia (OUM)

94 X TOPIC 6 SURVEY RESEARCH METHODOLOGY

the teaching profession. The sample you selected will have to represent the
attitudes of the target population as illustrated in Figure 6.3. Due to
financial and time constraints, you are unable to survey the attitudes of all
graduate teachers across the whole of Malaysia (N = 100,000) and so you
decide to confine your study to graduate teachers in Perak (n = 15,000)
which is called the accessible population. From the accessible population, a
sample of 100 teachers is drawn.

Even though the sample is selected from the accessible population, you
may want to know the degree to which the results can be generalised to the
target population which requires two steps. In the first step, you generalise
from the results of the sample to the accessible population (teachers in
Perak). Second, you generalise from the accessible population to the target
population (graduate teachers in Malaysia). The leap from sample to the
accessible population presents no problem if a random sample of the
accessible population is obtained.

However, in order to make the leap from the accessible population to the
target population, you must gather data to determine the degree of
similarity between graduate teachers in Perak and graduate teachers in the
whole of Malaysia. If you can demonstrate, based on a number of variables
(such as gender, age, experience and ethnicity) that the accessible
population is closely comparable to the target population, you have
established population validity i.e. the accessible population is
representative of the target population.

Figure 6.3: Random selection of a sample

How do you select a sample randomly? The usual definition of a random sample
is that it is a procedure in which all individuals in the defined population have an
equal and independent chance of being selected as a member of the sample.
Independent means the selection of an individual is not affected by the selection
of another individual. In other words, each individual, event or object has the
same probability of being selected. For example, the number of graduate teachers

Copyright © Open University Malaysia (OUM)

 TOPIC 6 SURVEY RESEARCH METHODOLOGY W 95

in Perak is 10,000 and you intend to draw a sample of 100 teachers as in Figure
6.3. When you select the first teacher, he has a 1:10,000 chance of being selected.
Once this teacher is selected, however, there are only 9,999 cases remaining so
that each teacher has a 1:9999 chance of being selected as the second case. Thus,
as each case is selected, the probability of being selected next changes slightly
because the population from which you are selecting has become one case
smaller.

The Table of Random Numbers is often used in the selection of a random sample
as in Table 6.1. You need to obtain a list of the 10,000 graduate teachers in Perak
and assign numbers 1 to 10,000. Using the table of random numbers, randomly
select a row or column as the starting point. For example, you select Column 5.
Select all the numbers that follow in that column. So, you will select a teacher
with the assigned number 7,332, followed by a teacher with the assigned number
6,516, and then a teacher assigned with the number 4,553 and so forth. If you
need more numbers, you can proceed to the next column until you have enough
numbers to make up the desired sample (i.e. n = 100).

Table 6.1: Table of Random Numbers

1 2 3 4 5 6 7 8 9 10

1 2345 3445 2678 9156 7332 9332 8345 5950 5023 5189

2 7612 7989 5456 4523 6516 1345 1123 5636 9189 27452

3 6823 5732 3702 3808 4553 4589 0467 8506 6612 12136

4 4598 2564 9860 6360 5245 2347 0391 3623 0620 0850

5 8956 357 3934 3495 9112 9472 1254 3998 1390 3194

6 9059 2691 8395 2634 2189 3465 8223 0745 3487 6709

7 2312 6120 6425 6556 3720 0639 5490 2614 2950 7556

8 8787 8236 5153 0202 9530 6490 6220 3523 9691 4917

9 9063 7190 2590 8696 6267 3923 5360 4937 4854 5223

10 1298 3820 6737 0932 6719 3154 4532 4046 2860 3191

Copyright © Open University Malaysia (OUM)

96 X TOPIC 6 SURVEY RESEARCH METHODOLOGY

(b) Systematic sampling

Similar to random sampling, systematic sampling is used to draw a sample
from a population. It is often used instead of random sampling. It is also
called an Nth name selection technique. You begin by having a list of the
names of members in the population in random order.
(i) You want to select a sample of 100 students from a population of 1,000
students.
(ii) You divide the population by the sample needed (1,000/10) = 10.
(iii) You select at random a number smaller than 10.
(iv) You start with that number (e.g. 7) and select every seventh name
from the list of the population.

After the required sample size has been calculated, every Nth record is
selected from a list of population members. As long as the list does not
contain any hidden order, this sampling method is as good as the random
sampling method. Its only advantage over the random sampling technique
is simplicity. Systematic sampling is frequently used to select a specified
number of records from a computer file.

(c) Stratified sampling

In some surveys, you want to ensure that individuals with certain
characteristics are included in the sample to be studied. For this purpose,
the stratified sampling technique is used. For example, you are interested in
studying inductive reasoning among 12-year-olds according to ability and
gender in the Petaling district. You want to ensure that you have a
proportionate number of high and low academic achievers as well as a
proportionate number of males and females. In order to avoid a sample that
does not include a sufficient number of students of each sex at each ability
level, a stratified sample may be selected. All 12-year-old students in the
district are divided into one of the following four groups: male high
achievers, female high achievers, male low achievers and female low
achievers. Sub-samples are then selected at random from the population to
fill each of the four groups as provided in Table 6.2.

Table 6.2: Sample Stratified according to Gender and Academic Performance

Males Females

High Achievers n=? n=?

Low Achievers n=? n=?

Copyright © Open University Malaysia (OUM)

 TOPIC 6 SURVEY RESEARCH METHODOLOGY W 97

However, the proportion of students randomly selected to fill each of the

groups is the same as the population. For example, if male high achievers
made up 25% of all 12-year-olds in the Petaling district, the proportion of
male high achievers in the sample should also be 25%. If you do not follow
this procedure, results obtained from the sample will produce an inaccurate
view of the population. Stratified samples are most appropriate when you
want to make comparisons between various sub-groups and ensure that the
sample is representative of the population in terms of the critical factors
you want to study.

ACTIVITY 6.3
You have been appointed to lead a research team assigned with the
task of finding the reasons teenagers smoke. The team has decided to
conduct a nationwide survey involving students between 14 and 16
years of age in secondary schools.
(a) Suggest three research questions that will guide data collection.
(b) Explain how you plan to draw the sample of students using
stratified sampling. What sub-groups would you include?
(c) What further information do you need to draw a representative
sample?

(d) Cluster sampling

In the techniques of sampling discussed thus far, the unit of sampling is the
individual student, teacher or principal. In cluster sampling, the unit of
sampling is not the individual but rather a group of individuals. For
example, in a district, there are 25 villages and in each village there are an
estimated 20 preschool children. You need a sample of about 100
preschoolers but this would mean going to many villagers if random
sampling is used. In cluster sampling, you select five villages randomly
from the 25 villages. You study all the preschool children in the five villages
you have selected. The advantage of using cluster sampling is that it saves
time and money, especially if the population is dispersed. The weakness is
that it is less accurate compared to the other techniques of sampling
discussed because the sub-groups may be more heterogeneous than
homogeneous.

Copyright © Open University Malaysia (OUM)

98 X TOPIC 6 SURVEY RESEARCH METHODOLOGY

SELF-CHECK 6.2

1. What is meant by random sampling?

2. What is the difference between stratified sampling and systematic
sampling?
3. How is cluster sampling different from all the other types of
sampling techniques discussed?

ACTIVITY 6.4
N=118
N=134 N=105 N=113
N=154
N=111 N=129 N=98
N=109

The above is a diagram showing the location of nine housing areas

dispersed all over a city with the number of senior citizens indicated in
each area.
(a) Explain how you would select a sample of about 200 senior
citizens you intend to interview regarding how they spend their
time using cluster sampling.
(b) What factors should you consider when using cluster sampling?

6.4.2 Non-Probability Sampling Techniques

In many situations, it is not possible to ensure that the sample will be selected
based on random selection. So the sample has to be chosen by some other ways.
Non-probability sampling is based on the researcherÊs judgment and hence
biasness will enter sample selection and distort the findings of the study.
Nonetheless, non-probability sampling techniques are used because of practical
reasons. For example, non-probability techniques are used to save costs and time,
when its use can satisfactorily meet the objectives of the study and it may be the
only feasible method given the spread and features of the population. Among the
more common non-probability techniques are: quota sampling, purposive
sampling and convenience sampling.

Copyright © Open University Malaysia (OUM)

 TOPIC 6 SURVEY RESEARCH METHODOLOGY W 99

(a) Quota sampling

In situations where you are unable to ensure that subjects are randomly
assigned to the sub-groups, you can use quota sampling. Like stratified
sampling, you first identify the sub-groups and their proportions as they
are represented in the population. You then select subjects based on
convenience and your judgment to fill each sub-group. When using this
method, you must be confident that the chosen sample is truly
representative of the population. Obviously, you should state clearly the
criteria for your selection of the sample, especially when you make
generalisations of the results to the population.

(b) Purposive sampling

The sample is selected on the basis that members conform to certain
criteria. The researcher uses his judgment to select cases to answer certain
research questions. The form of sampling is usually used when the
population is small (such as in case study research) and when the main
purpose is to select cases that are particularly informative. It is very useful
in the early stages of an exploratory study. The main weakness of this
technique is that the sample may have characteristics which differ from
population characteristics.

(c) Convenience sampling

In this sampling technique, researchers have the freedom to choose
whoever they find. You can sample people who are easy to survey. The
sample is chosen rather haphazardly until the required sample size is met.
It is less expensive and easy to conduct, and is considered the most useful
procedure to test ideas in exploratory research. This sampling technique is
considered the least reliable because of its high incidence of biasness.
Newspapers ask people to mail in their responses to a survey question;
radio stations ask people to call in their opinions to a question. As you can
see, the convenience sample can provide you with a lot of data but you do
not know whether the sample represents your population.

SELF-CHECK 6.3
1. What is the major difference between probability and non-
probability sampling techniques?
2. Why are non-probability sampling techniques used despite their
many weaknesses?

Copyright © Open University Malaysia (OUM)

100 X TOPIC 6 SURVEY RESEARCH METHODOLOGY

6.5 SAMPLE SIZE

How large must a sample size be? This is a question that is uppermost in the
minds of many beginning researchers. The general rule is to use the largest
sample possible. The rule is good because we are interested in learning about the
population based on the results of the sample from which it is drawn. The larger
the sample, the more likely the sample mean and the standard deviation will be
representative of the population mean and standard deviation. The sample size
required for a survey partly depends on the statistical quality needed for survey
findings; this, in turn, relates to how the results will be used. Even so, there is no
simple rule for sample size that can be used for all surveys. Much depends on the
professional and financial resources available. Often a moderate sample size that
is sufficient statistically and operationally is preferred. For example, the well-
known Gallup Poll in the United States frequently uses samples of about 1,000
persons to get reasonable information about national attitudes and opinions.

The following are some guidelines you can use in deciding how large your
sample should be:
(a) When the sample selected has to be broken down into smaller groups
involving comparisons of groups, a larger sample is advisable. For
example, you want to compare the self-esteem of males from low socio-
economic backgrounds from single-parent families against males from high
socio-economic backgrounds from single-parent families.
(b) Attrition is expected especially in longitudinal studies. The longer the
duration of a study, the higher the number of subjects who drop out will be.
To reduce attrition, you should keep demands on subjects to the minimum,
fully inform them about the study, obtain a strong commitment from them
before the study and make frequent contact with them to maintain interest
and rapport.
(c) When the population is highly heterogeneous on the variables being
studied, you need to have a larger sample so that the different
characteristics of individuals are satisfactorily represented. If members in
the population are more or less similar, then you may only need a small
sample as most of the characteristics can be captured.

SELF-CHECK 6.4
What are the factors that you will consider when making decisions
about sample size in a survey?

Copyright © Open University Malaysia (OUM)

 TOPIC 6 SURVEY RESEARCH METHODOLOGY W 101

ACTIVITY 6.5

Visit this website, www.surveysystem.com/sscalc.htm, to access a

Sample Size Calculator.
(a) What is meant by confidence level and confidence interval?
(b) Calculate the sample size required from a population of 9,000
students with a confidence level of 95% and confidence interval
(or margin of error) of 5.

6.6 THE PROCESS OF CONDUCTING A SURVEY

Now, let us see the process of conducting a survey.

Step 1: Defining the Objectives of the Study

A successful survey requires careful preparation. The first and often the
most difficult question is „Why am I doing this survey?‰ Many studies
start with a general hope that something interesting will emerge, and
they often end in frustration. Putting together a group of questions you
feel are important to students does not constitute a valid and reliable
survey.

This initial planning requires some idea of the final analysis and it may
be useful at the outset to outline the key tables for the final report
(better still if you are able to create dummy tables), and to consider the
number of cases expected in each major cells or sub-groups. Conduct a
review of literature to define terms and justify theory underlying
research questions.

Step 2: Writing the Items and Construction of the Questionnaire

Survey data is mostly obtained by means of questionnaires. Generally,
questionnaires are an inexpensive way to gather data from a large
number of respondents. They may be either self-administered (that is,
completed by subject), administered at an interview or administered to
a group in face-to-face session. The crux of a survey is the questionnaire
(i.e. what we ask our respondents). A well-designed questionnaire can
provide valuable information about the opinions, beliefs and practices
of groups of individuals. Questionnaire design is a long process that
demands careful attention and should not be taken lightly. In this
section, the general qualities of a good questionnaire are discussed
briefly. Figure 6.4 has laid out seven steps of survey process.

Copyright © Open University Malaysia (OUM)

102 X TOPIC 6 SURVEY RESEARCH METHODOLOGY

Figure 6.4: The seven steps of the survey process

(a) Clarity: Questions must be clear, precise and unambiguous. This

is to eliminate the chance that the question will mean different
things to different people. Avoid the use of colloquial or
ethnically sensitive terms. Technical terms that assume a certain
background should also be avoided.

(b) Leading Question: A leading question forces or implies a certain

type of answer. The researcher feels strongly about an issue and
assumes that everyone will be of the same opinion. e.g. Does
your principal treat men and women teachers differently?

(c) Hypothetical Question: A hypothetical question is one in which

you are asking respondents to indicate what they think they
would do under particular imaginary circumstances. While they
are used in some attitudinal research, they are difficult to
interpret and often give rise to unreliable answers. For example,
if you were buying a house, what features would you most want
it to have?

(d) Question requiring Prior Knowledge: A question that taxes a

respondentÊs memory leads to non-response or inaccurate
replies. For example, what did you do in the last school
holidays?

(e) Order of Questions:

(i) Take into account the sensitivities of the person to whom
they are addressed ă it is better to start with "What is your

Copyright © Open University Malaysia (OUM)

 TOPIC 6 SURVEY RESEARCH METHODOLOGY W 103

date of birth?" rather than jump straight into, "Has there

been a death in the family?"
(ii) Begin with interesting items. If the first items are boring,
there is little chance that the person will complete the
questionnaire.

Step 3: Method of Conducting the Survey

(a) You have to decide the procedures for conducting the survey;
how many people you will survey (the sample size and how
they will be obtained).
(b) Decide how you will survey your subjects (by phone, in class,
mail format and interview).
(c) Decide how you intend to distribute and collect the
questionnaires; make follow-up contacts.
(d) Decide on the level of response that is acceptable (refer to your
research questions and the population the question is being
asked). What percentage of questionnaires returned or answered
will you accept? What will you do if you do not have enough
respondents? Or a certain sector of respondents is under-
represented (e.g. insufficient number of females)?

Step 4: Pilot-Testing of the Questionnaire

(a) Pilot-testing a questionnaire is necessary to avoid problems that
may arise when the questionnaire is administered to the whole
sample. Unfortunately, some people consider pilot-testing
nothing more than a ritual, while others do not do it.
(b) Before pilot-testing, get some of your colleagues to review the
questionnaire, particularly those who are interested or familiar
with what you are doing.
(c) The questionnaire should be pilot-tested with a small group of
individuals who are in many respects similar to the sample in
the actual study. When they have finished, ask for their opinions
and suggestions (e.g. was it too long or which items were
difficult?).
(d) The pilot-test helps you find out if the content and form of the
questions are satisfactory. You can also get information about:
(i) Length of time to complete the questionnaire;
(ii) The appropriate order of the questions;

Copyright © Open University Malaysia (OUM)

104 X TOPIC 6 SURVEY RESEARCH METHODOLOGY

(iii) Sufficient space is provided for responses;

(iv) The instructions or directions are understood;
(v) Extent to which the questionnaire is reliable (if you give the
questionnaire again four weeks later, would you get the same
responses?); and
(vi) Extent to which the questionnaire is valid (does it measure
what it is supposed to measure?).

Step 5: Administering the Questionnaire

(a) You should gain approval from the respondents you intend to
survey. For example, permission should be sought from the
Ministry of Education (Educational, Planning and Research
Division), State Education Departments and school principals.
(b) In a small study, you can administer the questionnaire yourself.
But in a large survey, you will need assistants. Your field
assistants have to be trained, especially if they are required to
observe or conduct interviews. Such training should be
completed before the start of the main study.
(c) Teachers are gatekeepers to students. Gaining the cooperation of
teachers is very important when conducting a survey involving
students. Teachers will be more receptive in conducting the
survey during class time if you explain to them clearly the
objectives and contents of the survey. It is even better if you can
demonstrate that the survey can be incorporated into the
curriculum. Schedule the survey far enough in advance to allow
teachers to make class time available.
(d) Consent may be necessary for students to participate in the
survey. If so, parents may play a key role in studentsÊ
participation. Ensure that parents understand why you are
asking their childrenÊs opinion.
(e) Do not administer the questionnaire on a day when there are
school activities such as open day, celebration of festivals, sports
events, examinations. Students will be distracted with these
events and may not give the survey their full attention.
(f) If you are surveying parents, you can send the questionnaire
home with students. How can you be sure of receiving enough
returned surveys? One suggestion is to reward students for
returning questionnaires e.g. by giving a coupon for a free soft
drink.

Copyright © Open University Malaysia (OUM)

 TOPIC 6 SURVEY RESEARCH METHODOLOGY W 105

(g) Thank the groups that assisted in the survey process. An

appreciation goes a long way toward support and participation.

Step 6: Data Entry and Analysis

(a) Coding and entry of data: Check to ensure the accuracy of data
entry, that all codes are valid (for example, „1‰ is for male, ‰2‰
for female and there should not be a „3‰!) and look for
inconsistencies.
(b) Selection of software package to analyse data: Statistical analysis
should only begin when the data set is as „clean‰ as possible.
(c) If you are unclear of certain statistical procedures, obtain advice
from a person who is well-versed in statistics.

Step 7: Writing the Report

• The final phase of any research process is documentation or
writing the research report. The usual parts of a research report
are:
(i) Introduction (background of the study, rationale for the
study, the aims and research questions, limitations and
significance of the survey);
(ii) Literature review (previous work done in the field,
underlying theory or core of theories);
(iii) Methodology (description of the sample, how the sample
was drawn, description of the questionnaire used,
administration of the questionnaire);
(iv) Results (data presented in the form of tables and graphs;
statistical analysis, description of the findings); and
(v) Conclusion (explanation for the findings, relate to previous
literature and theory, making generalisations and
recommendation for future research).

SELF-CHECK 6.5
1. List some of the qualities of a good questionnaire.
2. Why is pilot-testing necessary?

Copyright © Open University Malaysia (OUM)

106 X TOPIC 6 SURVEY RESEARCH METHODOLOGY

6.7 DATA COLLECTION METHODS USING

SURVEYS
Surveys can be classified by their method of data collection. Generally, there are
two main types of data collection methods: self-administered and investigator
administered as illustrated in Figure 6.5. The most common self-administered
method of data collection is mail survey and more recently, web survey, where
the respondent is expected to respond to the questionnaire without the presence
of the investigator.

Figure 6.5: Data collection methods using a questionnaire

The investigator administered method of data collection requires the presence of

the investigator such as telephone interview, face-to-face interview and group
administered questionnaire. Besides the above, extracting data from samples of
medical records and others are also frequently done. In newer methods of data
collection, information is entered directly into computers using devices attached
to TV sets that automatically record the channels being watched.

(a) Mail Surveys

Surely you would, at one time or another, receive a questionnaire in the
mail (e.g. credit card companies, automobile companies). There are many
advantages to mail surveys. This method of data collection can be relatively
low in cost. You can send the same questionnaire to many people and they
can fill it at their own convenience. Mail surveys can be most effective when
directed at particular groups, such as subscribers to a specialised magazine
or members of a professional association. The disadvantage of mail survey

Copyright © Open University Malaysia (OUM)

 TOPIC 6 SURVEY RESEARCH METHODOLOGY W 107

is low response rate. Also, since the researcher is not present, there is no
way for respondents to seek clarification if questions are unclear.

(b) Web Survey

A more recent method of data collection is using the Web. The
questionnaire is uploaded to a website and respondents are invited to
respond to the questionnaire. While it is less expensive and you can reach
out to a large audience, there are many weaknesses with this method of
data collection. The authenticity of the person responding can be difficult to
prove, response rate may be low and persons responding to the
questionnaire would be confined to those who have Internet access which
may not be representative of the population.

(c) Telephone Interviews

Telephone interviews are an efficient method of collecting some types of
data and are being increasingly used. They lend themselves particularly
well to situations where timeliness is a factor and the length of the survey is
limited. The telephone interview gives respondents the feeling of
anonymity since the interviewer cannot see them.

(d) Face-to-Face Interview

Face-to-face or in-person interviews in a respondent's home or office are
much more expensive than mail or telephone surveys. They may be
necessary, however, especially when complex information is to be collected.

(e) Group-Administered Questionnaire

A sample of respondents are brought together and invited to respond to a
structured sequence of questions. This is a convenient method because you
are able to capture a relatively large sample of respondents in one sitting
(e.g. classroom). Also, the response rate is relatively high. If the respondents
are unclear about the meaning of questions they could ask for clarification.
However, the presence of the researcher may make respondents feel that
their answers are less anonymous and as such they may be less candid.

6.8 ETHICS IN CONDUCTING A SURVEY

What about confidentiality and integrity in surveys? The confidentiality of the
data supplied by respondents is of prime concern to all who conduct surveys. For
example, in many countries the data collected by the census department is
protected by law. There are acts that guarantee the confidentiality of data
collected by the relevant agencies. Several professional organisations dealing
with survey methods have codes of ethics that prescribe rules for keeping survey

Copyright © Open University Malaysia (OUM)

108 X TOPIC 6 SURVEY RESEARCH METHODOLOGY

responses confidential. The recommended policy for organisations or individuals

to safeguard such confidentiality includes:
(a) As far as possible, use only number codes to link the respondent to a
questionnaire and store the name-to-code linkage information separately
from the questionnaires. For example, 001 for respondent Azlina Darus, 002
for respondent Ong Mei Ling and so forth.
(b) The names and addresses of survey respondents should not be made
available to anyone outside those involved in the survey after the responses
have been entered into the computer. (Individuals and organisations have
been known to sell such databases to companies for marketing purposes
without the consent of individuals involved!).
(c) Omitting the names and addresses of survey respondents from computer
files used for analysis.
(d) Presenting statistical tabulation using broad enough categories so that
individual respondents cannot be singled out.

Respondents should be informed about the purpose of the survey and have the
option not to participate or not to divulge information that he or she does not feel
comfortable with. For example, respondents may be reluctant to disclose income.
To overcome this, you may want to use categories (e.g. RM1,500 ă RM2,000 per
month) which may be less intrusive. You should determine in the pilot-test
which items respondents are uncomfortable with, so that you do not have too
many unanswered questions to the point that some research questions cannot be
answered. The questions asked should not in any way attempt to deceive
respondents. The integrity of a survey is enhanced if respondents are clear about
the purpose of the study.

Copyright © Open University Malaysia (OUM)

 TOPIC 6 SURVEY RESEARCH METHODOLOGY W 109

ACTIVITY 6.6
1. You wish to study moral reasoning among 15-year-olds in a large
district. A total of 4,100 15-year-olds are enrolled in 105
classrooms in the district. You plan to obtain a total of 250
students using the cluster sampling technique. Describe the steps
you would take in selecting the sample.
2. Discuss some of the problems with telephone interviews that may
affect the results of a survey.
3. One of the drawbacks of mail survey is the low response rate.
Suggest how you would increase response rates.
4. When would you use a longitudinal survey rather than a cross-
sectional survey?
5. What are some problems with surveys conducted over the
Internet? How can you overcome these problems?

• A survey is a method of obtaining information about a population from a

sample of individuals.

• Surveys are useful for gathering data about what people are thinking, feeling
or doing.

• There are two types of survey: cross-sectional survey (taken at a particular

time) and longitudinal survey (compare changes over time).

• Since surveying an entire population would be costly and time-consuming, a

sample is drawn from the population.

• There are two techniques of sampling: probability and non-probability.

• Probability sampling is based on random selection while non-probability

sampling is not based on random selection.

• Questionnaire is the main tool for data collection in a survey.

• There are two methods of data collection using the questionnaire:

self-administered survey and investigator administered survey.

Copyright © Open University Malaysia (OUM)

110 X TOPIC 6 SURVEY RESEARCH METHODOLOGY

• Self-administered survey comprises mail survey and web survey while

investigator administered survey includes telephone interview, face-to-face
interview and group administered questionnaire.

• The main ethical issues in conducting a survey are confidentiality and

integrity.

Investigator administered survey Sampling techniques

− Telephone interview − Random
− Face-to-face interview − Stratified
− Group administered questionnaire − Systematic
Questionnaire − Cluster
Sample size Self-administered survey
− Mail survey
Survey
− Cross-sectional
− Longitudinal
Survey process

Copyright © Open University Malaysia (OUM)

Topic X Instrumentation
7
LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Differentiate between instrument and instrumentation;
2. Define what a questionnaire is;
3. Explain the construction of a questionnaire; and
4. Design and develop an attitude scale.

X INTRODUCTION
The most important and sometimes difficult aspect of educational research is
determining the instruments to be used in data collection. An example of an
instrument is the Self-Esteem Test shown in Figure 7.1. There are numerous tools
that may be used to obtain information. In this topic, we will discuss in detail the
process of instrumentation and look at some examples of widely used types of
instruments. We will also discuss two very important concepts related to
instrumentation: reliability and validity. Fraenkel and Wallen make a distinction
between instrument and instrumentation. An instrument is a device or procedure
for systematically collecting information while instrumentation includes both the
instrument and „the conditions under which it is used, when it is used, and by
whom it is used‰. Examples of instruments are tests, questionnaire, rating scales,
inventories and checklists.

Copyright © Open University Malaysia (OUM)

112 X TOPIC 7 INSTRUMENTATION

Do you believe in yourself?

Are you confident?
Do you like yourself?

Figure 7.1: Self-Esteem Test

Source: www.cultsock.ndirect.co.uk

This test is designed to test your level of self-esteem. Read each item and state
whether you strongly agree, agree, disagree or strongly disagree with each
statement by referring to yourself.

Strongly Agree Disagree Strongly

Agree Disagree

4 3 2 1
1. I am an attractive person.

2. People will dislike me if I am myself.

3. I am happy with my present height.

4. I like my hair.

5. I have something to say in social situations.

6. I am looked down upon because I am not good

looking.

7. I am a friend to myself and take care of myself.

8. I am popular among my friends.

9. I like the colour of my skin.

10. I think I am useless in the company of others.

Source: Reprinted with permission: © CoPS Project Malaysia (2005). John Arul Phillips.

Copyright © Open University Malaysia (OUM)

 TOPIC 7 INSTRUMENTATION W 113

ACTIVITY 7.1

Examine each of the 10 items of the Self-Esteem Test.

(a) What do you understand by self-esteem?
(b) Identify the two aspects of self-esteem being measured.
(c) Why do you think some items are stated in the positive, while
others are stated in the negative?

7.1 THE QUESTIONNAIRE

In Topic 6, we discussed in detail the survey method. In conducting a survey, the
second step is „writing the items and constructing the questionnaire‰. When you
conduct a survey, a questionnaire is administered in which subjects respond in
writing or orally in the case of an interview. The responses of the subjects to the
questions are summarised, analysed and reported. The aim is to obtain
information about the characteristics of a particular group. The results from the
sample are generalised to a larger group or the population, if the sample
surveyed is randomly selected.

Questionnaires are widely used because they are cost effective and can be
administered to large groups of people. There is greater standardisation in
questionnaires as each respondent receives the same set of questions which
allows interpreting from a large sample. The questionnaire, if properly designed,
can address many issues in a relatively efficient way. Also, there is assurance of
anonymity which increases the likelihood that the responses are genuine and
reflective of the opinions, perceptions and beliefs of respondents. However, the
quality of data obtained from a questionnaire will depend on how well it is
designed. Fowler suggests the following important criteria when designing a
questionnaire:
• Is this a question that can be asked exactly the way it is written?
• Is this a question that will mean the same thing to everyone?
• Is this a question that people can answer?
• Is this a question that people will be willing to answer?
Designing a questionnaire is both an art and science; and takes time and
precision. Before setting out to design a questionnaire from scratch, it is a good
idea to find out whether such questionnaires already exist that could be used or

Copyright © Open University Malaysia (OUM)

114 X TOPIC 7 INSTRUMENTATION

modified to gather the information that you seek. A little bit of research could
save you a lot of work in creating an entirely new questionnaire.

In designing a questionnaire, you have to determine the questions that you want
to ask people; select the question response format for each question, decide on
question sequence and overall layout, pilot-test and implement the instrument.
Deciding what questions to ask will depend on the aims of your survey. For
example, if the aim of your study is to survey the reading habits and interests of
teenagers, one of the questions you will ask is: „What kind of books do you like
to read?‰ You should constantly check to see that there is a close link between the
research aims and the individual questions that you ask. You do not want a
situation where the data collected from your questionnaire does not answer the
research questions.

7.2 QUESTION RESPONSE FORMAT

The questionnaire is designed with the purpose of getting people to respond to a
set of questions. In most questionnaires, you will find two broad types of
questions used: structured (closed) and unstructured (open).

7.2.1 Structured or Closed Questions

Structured questions are questions in which all possible answers (or responses)
are identified and the respondent is asked to choose one or more among the
answers.

Advantages of Structured or Closed Questions

• Structured or closed questions are useful when the answers are limited and
clear-cut. They are most appropriate for asking questions about factual
information, but they are also widely used in obtaining data about opinions
and attitudes.
• Structured questions allow for greater control as they limit the choice of
answers and force the respondent to answer.
• It is easier for subjects to respond to structured questions and this also saves
time.
• Since the questions do not require writing, this may encourage unmotivated
subjects to respond.
• The limited responses make the task of coding data easier.

Copyright © Open University Malaysia (OUM)

 TOPIC 7 INSTRUMENTATION W 115

Disadvantages of Structured or Closed Questions

• The answers are limited to what is stated which may not capture
responses other than those listed.
• Questions that are poorly designed may mislead and frustrate
respondents.

7.2.2 Types of Structured Questions

(a) Dichotomous Questions: These items require two possible responses such
as Agree/Disagree or True/False or Yes/No.

Should the driving age be increased from 17 to 21 years?

Yes
No

(b) Check Only One Answer from a List: The respondent is required to check
or tick only ONE answer.

What is your fatherÊs highest level of education?

Primary school
Secondary school
Form 6/Matriculation
Degree

(c) Checking More than One Answer from a List: The respondent is required to
place a check or tick next to one or more answer.

Which of the following are reasons for school bullying?

Violence on TV
Lack of discipline at home
Insecure with themselves
Peer pressure

Copyright © Open University Malaysia (OUM)

116 X TOPIC 7 INSTRUMENTATION

(d) Ranked Items: Respondents can also be asked their preferences by ranking
the items from 1 to 2 and so forth. We want the respondent to put a 1, 2, 3, 4
or 5 next to the brand, where 1 is the respondentÊs first choice, 2 the second
choice and so forth.
Rank the following brands of cars in terms of prestige:
(1 ă most prestigious, 2 ă second most prestigious and so forth)

Proton _____
Toyota _____
Honda _____
Nissan _____
Ford _____

(e) Rated Responses: The Likert Scale has become a popular tool in getting
people to respond to questions. The Likert scale is a rating scale which
measures the strength of agreement with a statement.

Common types of rating scales:

AGREEMENT
Strongly Disagree Undecided Agree Strongly
Disagree Agree
1 2 3 4 5
FREQUENCY
Very Rarely Rarely Occasionally Frequently Very
Frequent
1 2 3 4 5
IMPORTANCE
Unimportant Of little Moderately Important Very
importance important important
1 2 3 4 5

(f) Filter Questions: This format is used when you want to obtain information
from people who „do‰ or „do not‰ engage in something or believe in
something or have an opinion about something (i.e. you are filtering).

1. Do you smoke cigarettes Yes No

If „YES‰, answer Questions 2-3. If „NO‰, continue to Question 4.

Copyright © Open University Malaysia (OUM)

 TOPIC 7 INSTRUMENTATION W 117

2. How many sticks of cigarettes do you smoke in a day?

< 5 sticks a day 11-15 sticks a day

6-10 sticks a day > 15 sticks a day

3. When did you start smoking?

Primary school Upper secondary school

Lower secondary school After secondary school

7.2.3 Unstructured Questions

In the design of a questionnaire, one issue that will arise is whether you should
use open-ended questions. Some researchers are against open-ended questions
because of the problems associated with using such a format. However, open-
ended questions can be useful. For example, these open-ended questions:
• What do you think are the reasons for school bullying?
• What do you like most about the learning materials?
Such questions would provide many kinds of responses. Some would be long
answers while others would be just a phrase. If you are interested in getting a
variety of reasons and also some in-depth reasons, then the unstructured or
open-ended format would be useful. Unstructured or open questions should be
used when you feel that the particular question cannot be categorised to include
all possible responses. However, if you are concerned about the time-consuming
task of processing many different responses, than closed questions should be
used. But, you should be aware of the disadvantages of such questions.

SELF-CHECK 7.1

1. What are the advantages and disadvantages of using structured or

closed questions in a questionnaire?
2. What is the difference between ranked and rated responses?
3. When would you use a ranked response and a rated response?
Give specific examples.

Copyright © Open University Malaysia (OUM)

118 X TOPIC 7 INSTRUMENTATION

7.3 GUIDELINES ON QUESTIONNAIRE DESIGN

Most problems with a questionnaire can be traced back to the design phase. Your
questionnaire would be better designed if you are clear about the purpose of
your study. You should clarify your goals in relation to how you intend to use
the information. Unfortunately, many researchers neglect this aspect of the
questionnaire design process.

(a) Questions related to objectives:

Only ask questions that directly address the objectives of your study. For
example, if you do not intend to compare differences between ethnic
groups, you do not need to ask about ethnic origin. Avoid the temptation to
ask questions because it would be „interesting to know‰ or „maybe IÊll need
the data later‰ or „what harm is there in collecting more than I need‰.

(b) Length of questionnaire:

As a general rule, long questionnaires get less response than short
questionnaires. You should keep the questionnaire short to sustain
respondentsÊ interest and not exhaust your subjects. However, if the
questions are interesting, respondents might overlook the length of the
questionnaire.

(c) Instructions:
(i) Instructions on how to answer the questions should be clear and
concise. They should be easy to understand and use words that are
not difficult to understand.
(ii) State the purpose of the survey and who is administering or
sponsoring the survey.
(iii) Indicate how confidentiality is protected. This assurance may increase
the likelihood of honest responses.
(iv) Indicate who the respondents can call or write if they have questions
or concerns, and also if they want a copy of the survey results.

(d) Order of questions:

(i) Begin with questions that you think are interesting to your
respondents. For example, questions about their hobbies and interests
would attract the interest of adolescents. Also, the questions should
not be threatening. Questions that are boring and threatening will
cause respondents not to continue.

Copyright © Open University Malaysia (OUM)

 TOPIC 7 INSTRUMENTATION  119

(ii) Group questions that are similar together in terms of format and the
specific covered (e.g. group questions asking respondents about their
background such as occupation, income, age, etc.).
(iii) The transition from one question to another should not be too abrupt.
Avoid questions that jump from one unrelated topic to another.

(e) Scales:
(i) When you want subjects to respond to a scale from lower to higher, it
is usually better to place the lower end of the scale on the left and the
higher end of the scale to the right.
1. Never 2. Seldom 3. Occasionally 4. Always

(ii) Avoid having too many divisions in your scale which strains the
respondent (see example below). Usually a 4-point or 5-point scale
would be adequate.
1. Never 2. Seldom 3. Occasionally 4. Fairly often
5. Often 6. Very Often 7. Always

(f) Wording of questions:

(i) Avoid leading questions such as the following:
 Is it important to treat the disabled fairly?
 Do you agree with most people that the traffic system is getting
worse?
(ii) Watch out for words that are ambiguous or have more than one
meaning.
 Which teacher is the best?
(iii) Avoid jargon or acronyms or terms that a few people may know.
Acronyms should be expanded or stated in full unless the target
audience commonly knows them.
 Do you favour inclusion? (Are you sure most of your subjects
know what „inclusion‰ is?)
 What are your SPM results? (Foreigners may not know what SPM
stands for.)
(iv) Use simple and direct language which is easily understood by the
respondent. Do not use words that will probably not be understood to
avoid misinterpretation. You could bold or underline words that you
want to emphasise to eliminate misunderstanding.

Copyright © Open University Malaysia (OUM)

120 X TOPIC 7 INSTRUMENTATION

(v) Avoid double-barrelled questions such as the following:

• Do you think Malaysians should eat less and do more exercise?
Break up the question into two: one asking about „eating less‰ and the
other asking about „exercise‰.
(vi) Avoid questions involving negatives. Do not confuse respondents
with questions such as the following:
• Are you against the ban on smoking?
• Do you oppose the ban on public speaking?
(g) Layout and design:
(i) Do not clutter the questionnaire with unnecessary headings and
numbers.
(ii) Avoid using a lot of lines, borders and boxes since these can make the
page look too „dense‰.
(iii) Ensure that the questionnaire has a title and a brief introductory
statement on the purpose of the study.
(iv) Use a good legible font (e.g. Times Roman). Make use of italics and
bold types for instructions and headings.

SELF-CHECK 7.2

1. What are some common mistakes in the design of

questionnaires?
2. What is the purpose of pilot-testing a questionnaire?

7.4 PILOT-TESTING THE QUESTIONNAIRE

Test the questionnaire on a small sample of subjects who are closely similar to
your final sample. For example, if you are planning to survey adolescents, make
sure that you pilot-test the questionnaire with a group of adolescents. Encourage
respondents to make comments on each of the questions, on the order and format
of the questions, or on the nature of the questionnaire. You could also have
respondents discuss in small groups to provide insights into the questionnaire.

Note whether any questions are frequently unanswered or are frequently

misunderstood and answered incorrectly. Probably the questions are poorly
worded, located wrongly in the questionnaire, or too difficult or sensitive to

Copyright © Open University Malaysia (OUM)

 TOPIC 7 INSTRUMENTATION W 121

answer. Also, take note of the time taken to complete the questionnaire. The aim
of the pilot-test is to detect any mistakes in your questioning and to correct them
before the main survey. The pilot-testing may also allow you to convert an
unstructured or open-ended question to a structured or closed question by
determining the range of possible answers or responses. Based on the pilot-test,
you will be able to make changes that will help maximise response rate and
minimise confusion among respondents.

ACTIVITY 7.2
Design a 15-item Teacher Workload Questionnaire focusing on the following:
(a) Planning for teaching;
(b) Preparing for assessment and marking papers;
(c) Clerical and record keeping;
(d) Attending meetings; and
(e) Administrative duties.

7.5 DESIGNING AN ATTITUDE TEST

What is an attitude? An attitude is an enduring pattern of belief which may
influence behaviour. So if someone has a positive attitude about his job, you
would expect the person to be committed to the job. The Likert scale was
introduced by Renis Likert in 1932 as the familiar five-point response format. The
scale requests people to indicate how much they agree or disagree or approve or
disapprove. An attitude test or inventory is a cluster of items (or questions) that
measures a unitary dimension or single attitude. The respondent is provided
with a range of possible responses. An appropriate scoring scheme is associated
with each of the five possible responses. For example: strongly agree = 5, agree =
4, undecided = 3, disagree = 2 and strongly disagree = 1. This could of course be
reversed if desired. Sometimes, if the researcher wants to avoid an undecided
category, they may then choose to use an even number of choices, such as 4 or 6.

Now let us see the steps in developing an attitude test.

(a) Define the Construct: Define the attitude (or construct) that you wish to
measure. You should review the literature in the field to see how the
attitude has been defined. Examples of attitudes: attitudes towards racial
integration, attitudes towards women bosses, attitudes towards smoking
and attitudes towards water conservation.

Copyright © Open University Malaysia (OUM)

122 X TOPIC 7 INSTRUMENTATION

(b) Break down the Construct: Usually an attitude (or construct) needs to be
broken down into a number of categories. In other words, you are
providing an operational definition of the attitude. For example, racial
attitudes can be broken down into: marriage, social interaction, cultural
heritage and workplace.

(c) Brainstorm: Discuss and come up with a list of about 8 to 10 statements for
each category of the attitude. Develop an equal number of positive and
negative statements about each category of the attitude object.

(d) Rating the Items: Next is to have a group of judges or panel of experts rate
each statement on a five-point rating scale to ensure content validity (or
face validity). Do the statements cover the breadth of the attitude being
assessed? Measure what a statement is supposed to measure. For example,
is the statement, „I would be or have been in a romantic relationship with a
person of another race‰, describing a racial attitude?

1 = Strongly does not describe the attitude

2 = Somewhat does not describe the attitude
3 = Undecided
4 = Somewhat describes the attitude
5 = Strongly describes the attitude

The above scale is suggested to guide judges or experts in rating the

statements. Note that the judges are not telling you what they believe, but
the extent to which they perceive the statements as describing the attitude
(or construct) of interest.

(e) Scale: Decide on an appropriate scale such as the following which has five
possible responses: 1 = strongly disagree, 2 = somewhat disagree, 3 =
undecided, 4 = somewhat agree, and 5 = strongly agree. If you want to
avoid an „undecided‰ category, you can choose to use an even number of
choices i.e. 4-point Likert scale or a 6-point Likert scale.

(f) Pilot-Testing: Administer the attitude scale to a group of subjects and score
the instrument. Monitor to see if respondents had problems in
understanding the items. The final score for the respondent on the scale is
the sum of their ratings for all of the items (this is sometimes called a
"summated" scale). On some scales, you will have items that are reversed in
meaning from the overall direction of the scale. These are called reversal
items. You will need to reverse the response value for each of these items
before summing up the total. That is, if the respondent gave a 1, you make
it a 5; if they gave a 2, you make it a 4; 3 = 3; 4 = 2; and, 5 = 1.
Copyright © Open University Malaysia (OUM)
 TOPIC 7 INSTRUMENTATION W 123

(g) Reliability: You have to examine the test for reliability which means
whatever the instrument measures, it measures consistently. CronbachÊs
alpha measures how well a set of items measures a single construct (or
attitude). It measures consistency within the instrument where all items are
compared with each other. Alpha coefficient ranges in value from 0 to 1 and
the higher the score, the more reliable is the attitude scale. Nunnaly has
indicated 0.7 to be an acceptable reliability coefficient but lower thresholds
are sometimes used in the literature.

Figure 7.2: Printout of reliability analysis

(h) Reliability Analysis: See Figure 7.2 which is a printout of the reliability
analysis done for a sample of items. The Cronbach alpha for the instrument
is 0.77 (rounded from 0.77102). The second column shows the correlation
between each item with the total or the rest of the items. The third column
shows the correlation coefficient obtained if a particular item is deleted.
Note that if you were to delete Item 4, the Cronbach alpha will increase to
0.82 (rounded from 0.816080).

(i) Final Items: Based on the reliability analysis, the final items to be included
in the instrument can be decided. Generally, the reliability of the scales
tends to increase with the number of items. However, as the number of
items in a scale increases, so the time taken to complete the attitude test will
also increase, and this may demotivate respondents. There is no hard and
fast rule to determine the final number of items in a scale and this will
reflect the nature and complexity of the attitude being assessed. Generally,
fewer than 20 items may reduce reliability acceptably, but more than 30 will
begin to demotivate the respondent. Another point to note is that attitude
scales are summated scales. The scores for individual items are added
together to form the overall score for each respondent. This score is
normally divided by the number of items in the attitude scale to obtain the
Copyright © Open University Malaysia (OUM)
124 X TOPIC 7 INSTRUMENTATION

mean score which is used to indicate the attitude of an individual. As such,

item-by-item analysis of an attitude scale is meaningless.
Example of an Attitude Scale:
Attitude towards Sex Education in Schools
1 = Strong Disagree 2 = Disagree 3 = Agree 4 = Strongly Agree
1 2 3 4
1. Sex education should be the main responsibility of parents
and not teachers.
2. Rather than have students learn about sex from their peers
or on their own, it is better if sex education is taught in
schools.
3. Since some students are likely to experiment with sex, it is a
good idea to have sex education in schools.
4. Allowing sex education to be taught in the schools would
lead to an increase in teenage pregnancy.
5. Sex education should not be taught in schools because it will
lead to students experimenting with sex earlier than they
might otherwise.
6. The reproductive system taught in science and biology is not
sufficient for students to know about sex.

ACTIVITY 7.3
1. Identify positive and negative statements.
2. Complete the attitude scale by adding three positive statements
and three negative statements.
3. Try out the complete attitude scale with some of your colleagues
and friends.

ACTIVITY 7.4

1. Why do you think some people are unable and unwilling to

respond to questions in a questionnaire?
2. What are some common errors you have found in questionnaires?
3. How reliable do you think is the attitude scale in measuring
attitudes?

Copyright © Open University Malaysia (OUM)

 TOPIC 7 INSTRUMENTATION W 125

• Questionnaires are widely used because they are cost effective and can be
administered to large groups of people. If properly designed, questionnaires
can address many issues in a relatively efficient way.

• Questions in a questionnaire can be structured and unstructured.

• Questions in a questionnaire should be clear, not ambiguous and related to

the goals of the study.

• Pilot-testing a questionnaire is essential to detect weaknesses in the items.

• An attitude is an enduring pattern of belief which may influence behaviour.

• Attitude scales may be designed to measure attitudes using the Likert scale.

• A Likert attitude scale is a summated scale. As such, analysis by item is

meaningless.

Attitude scale Structured questions

Chi-square − Rated response
Correlation coefficient − Ranked response
One-way anova − Dichotomous response
Pilot-testing questionnaire Time series design
Questionnaire T-test
Questionnaire response format Unstructured questions
Reliability analysis

Copyright © Open University Malaysia (OUM)

Topic X Quantitative
8 Data Analysis I:
Introduction to
Descriptive
Statistics
LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Use appropriate graphs and charts to describe research data;
2. Use different types of measures of central tendency to describe
research data; and
3. Use different types of measures of dispersion to describe research data.

INTRODUCTION
Descriptive statistics is concerned with summarising data in order to describe the
characteristics of a sample. Examples of descriptive statistics are statistics of
students enrolled in an education programme, studentsÊ score in an achievement
test and survey results on certain behaviour. The commonly used methods in
describing data are presentation using charts and graphs, measure of central
tendency and measure of dispersion. These forms of descriptive analysis are also
known as exploratory data analysis (EDA).

This topic will not delve into descriptive statistics in detail. You will learn more
about it in HMEF5113 Statistics for Educational Research.

Copyright © Open University Malaysia (OUM)

TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE W 127
STATISTICS

8.1 DATA PRESENTATION

Raw data obtained from samples, censuses and experiments is usually in the
form of an unorganised set of numerical values. Until this data is organised in
certain forms, it will not bring much meaning to researchers. It is important to
tabulate the data before analysis. Tabulating the data and presenting it using
certain graphical methods will help the researcher to determine the presence of
outliers in the data set and the nature of distribution (symmetrical or skewed).
This will help the researcher to decide the next course of action, whether to clean
the data set first, transform it into a different form and so on. Among the
commonly used methods of displaying data are: tabulation and charts (bar chart,
pie chart, line graph, stem and leaf plots, histogram, Box and Whiskers plot).
Table 8.1 provides a summary of charts that are commonly used in education
research.

Table 8.1: Commonly Used Charts

Variable Number of
Variables Data presented as Type of Chart
Type
Two Raw Form Scatter Diagram
Raw Form Stem and Leaf
Continuous One Frequency Distribution Histogram
Polygon
Summary Table Bar Chart
One
Pie Chart
Categorical Cross Tabulation
Two Side-by-Side Bar Chart
(Contingency Table)

• Frequency Table, Pie and Bar Charts

Frequency table is a method of organising the raw data in table form while
pie and bar charts are used to display information from the frequency table
in graphical form. This will make interpretation easier.

Example:
The data below shows the number of tuition classes attended by 16
secondary school students in a week:

1, 3, 5, 2, 4, 3, 1, 4, 3, 2, 3, 3, 4, 2, 2, 3

Table 8.1 is the frequency distribution table for the data, while Figure 8.1
displays the pie chart.

Copyright © Open University Malaysia (OUM)

128 X TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE
STATISTICS

Table 8.2: The Number of Tuition Classes Attended in a Week

Number of Tuitions Number of Cases

1 2
2 4
3 6
4 3
5 1
Total 16

Figure 8.1: Pie chart

• Frequency Table using Class Interval

If the data set is large, these data need to be grouped according to
appropriate class intervals before presenting them in the form of tables or
charts.

Example:
The data below describes the average number of hours in a day spent by 35
students doing mathematics revision.

2.5 3.0 4.4 2.9 2.0

3.4 3.1 1.2 2.2 3.0
2.7 2.2 1.8 2.7 3.3
1.9 3.5 3.8 2.7 2.9
3.0 3.4 2.4 2.3 2.4
3.5 3.3 4.6 1.6 3.4
4.7 2.7 3.3 2.2 1.7

Copyright © Open University Malaysia (OUM)

TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE W 129
STATISTICS

To construct a frequency distribution table: first, determine the size of the

class interval. The size can take any value but most importantly, the lowest
and highest points must be included in the range. Usually, the class size
will take values in multiples of 5 (e.g. 5, 10, 15, etc). For the given data, the
lowest value is 1.2 hours while the highest is 4.7 hours. The width of the
class interval can be determined by using the following rule:

Range
Width of interval ≈ .
Number of desired class groupings

Since there are only 35 observations, seven class groupings is sufficient.

4.7 − 1.2
Width of interval ≈ = 0.5. It is appropriate to start with the value
7
1.0 and end with 4.9. Table 8.2 is the frequency distribution table for the
data while Figure 8.2 displays the bar chart.

Table 8.3: Average Hours Spent in a Day Doing Mathematics Revision

Average number Frequency

of hours
1.0 ă 1.4 1
1.5 ă 1.9 4
2.0 ă 2.4 7
2.5 ă 2.9 7
3.0 ă 3.4 10
3.5 ă 3.9 3
4.0 ă 4.4 1
4.5 ă 4.9 2

Figure 8.2: Bar chart

Copyright © Open University Malaysia (OUM)
130 X TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE
STATISTICS

• Side-by-Side Bar Chart

Side-by-side bar chart is used to display bivariate categorical variable,
especially to explore the pattern of relationship between the variables. The
following example illustrates the multiple bar charts for data on job
satisfaction and performance.

Example:
In studying the association between job satisfaction and job performance, a
researcher carried out a retrospective survey based on the annual
performance appraisal of employees of a large organisation. The following
data was obtained as shown in Table 8.3. Figure 8.3 is a side-by-side bar
chart.

Table 8.3: Data on Job Satisfaction and Performance

Job Performance

Low High

Low 104 20
Job Satisfaction

High 40 25

Figure 8.3: Side-by-side chart for job satisfaction and performance

Copyright © Open University Malaysia (OUM)

TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE W 131
STATISTICS

• Stem and Leaf Plots

It is a simple way to organise the data set to understand how the values are
distributed. Stem and leaf plot separates data into leading digits (stems)
and trailing digits (leaves). In stem and leaf, the decimal portion of the
values are rounded and truncated.

Example:
The data below shows the average number of hours spent in a day by a
group of teachers preparing teaching aids.

0.7, 2.7, 0.5, 1.2, 0.5, 2.1, 0.9, 1.4, 1.3, 1.6, 3.3, 1.8, 1.9, 1.9, 2.3.

The following is the stem and leaf plot for this data.

Stem Leaf

0 5 5 7 9

1 2 3 4 6 8 9 9

2 1 3 7

3 3

[Note: In plotting the stem and leaf for this data set, the unit digit becomes
the stem and the following digit becomes the leaf.]

• Histograms
Histograms are used to describe numerical data that have been grouped
into frequency, relative frequency or percentage distributions.

Example:
Suppose the average time spent on preparing teaching aids were collected
from larger samples and tabulated in a frequency distribution table, a
histogram will be an appropriate chart to represent the data. Table 8.4
shows the frequency distribution of the average number of hours spent in a
day on preparing teaching aids by 140 teachers, while Figures 8.4 and 8.5
present the histogram and polygon illustrating the data respectively.

Copyright © Open University Malaysia (OUM)

132 X TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE
STATISTICS

Table 8.4: Average Time Spent on Preparing Teaching Aids

Average Time per No of Respondents

Day

0 ă less than 0.5 9

0.5 ă less than 1.0 10
1.0 ă less than 1.5 16
1.5 ă less than 2.0 23
2.0 ă less than 2.5 37
2.5 ă less than 3.0 16
3.0 ă less than 3.5 11
3.5 ă less than 4.0 11
4.0 ă less than 4.5 7

Figure 8.4: Histogram showing the average time spent on preparing teaching aids

Copyright © Open University Malaysia (OUM)

TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE W 133
STATISTICS

Figure 8.5: Polygon showing the average time spent on preparing teaching aids

• Scatter Diagram
Scatter diagrams are used to describe the linear relationship between two
quantitative variables. The shape of the scatter points on the two-
dimensional graph will give us an idea on how these two variables are
related.

Example:
Suppose that a group of respondents were asked to rate their satisfaction
with the teaching and learning environment at OUM and their motivation
to learn, their scores on these two variables can be plotted as a scatter
diagram to see whether they are linearly related. Figure 8.6 shows the
scatter diagram relating studentsÊ satisfaction with the teaching (T&L)
environment at OUM and their motivation to learn.

Copyright © Open University Malaysia (OUM)

134 X TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE
STATISTICS

Figure 8.6: Scatter diagram

Plotting Graphs with SPSS:

1. Select the Graph menu.
2. Click on the appropriate graph type.
3. Follow the procedure displayed by the dialogue box.
4. Click on Continue and then OK.

ACTIVITY 8.1

A researcher wanted to study the association between the teaching and

learning (T&L) environment at OUM and studentsÊ learning. The T&L
was assessed from two aspects: studentsÊ satisfaction and whether it
motivates them, while their learning was measured by using their
assignment and final examination marks. The following is the data.
Use the SPSS programme to generate appropriate charts for the
variables used in this study.

Copyright © Open University Malaysia (OUM)

TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE W 135
STATISTICS

8.2 MEASURES OF CENTRAL TENDENCY

Measures of central tendency are single value statistics used to describe the
actual observation; in other words, this is a summary of the observed data. For
example, if a mathematics examination was administered to a class of 25
students, we will obtain 25 scores. The average score on the mathematics test is
a single value statistic representing the entire group of 25 students. Among the
commonly used measures of central tendency are mean, median and mode.

Copyright © Open University Malaysia (OUM)

136 X TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE
STATISTICS

The measure we use depends on the type of measurement and its distribution.
Figure 8.7 summarises the commonly used measure of central tendency.

Figure 8.7: Commonly used measures of central tendency

[Note: Since most of the post-graduate research will use primary data and the SPSS will be used for
analysis, this module does not discuss the methods for determining values for the central tendency
measures for grouped data. Students interested in learning this may refer to any standard statistics
books.]

(a) Mean
To calculate the mean, we add up all the observed values and divide them
by the total number.
x1 + x 2 + ... + x n
Mean is defined as, X=
n
Example:
The following is the data on the number of hours in a day spent by 10
students doing mathematics revision.

0.7, 2.7, 0.2, 1.2, 0.5, 2.1, 0.9, 1.4, 1.3, 1.6

The mean time in hours spent doing mathematics revision per day is:

0.7 + 2.7 + 0.2 + 1.2 + 0.5 + 2.1 + 0.9 + 1.4 + 1.3 + 1.6
X=
10
The mean, X = 1.26 hours.
Since mean uses all the information in the data set, it is a more efficient
summary measure. However, the disadvantage of using mean is that it is
affected by extreme values. This is illustrated as follows.

Copyright © Open University Malaysia (OUM)

TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE W 137
STATISTICS

If the value „1.6‰ in the data set is substituted with „10.3‰ hours, the mean
will become:

0.7 + 2.7 + 0.2 + 1.2 + 0.5 + 2.1 + 0.9 + 1.4 + 1.3 + 10.6
X =
10
X = 2.16 hours
(b) Median
Median is also known as a measure of location; that is, it points to the
central location as the summary of the data set. Since median does not use
all the information in the data set, it is less efficient than the mean. One
advantage of using median over mean is that it is not influenced by extreme
values.

Example:
In the example on time spent (in hours) for doing mathematics revision, the
n +1 15 + 1
median is the values at location , which is
2 2
0.7, 0.9, 1.2, 1.3, 1.4, 1.6, 1.8, 1.9, 1.9, 2.1, 2.3

Median Location

The median value is 1.6 hours.

The median will not be affected by the existence of extreme value.

The following illustration explains this.

If the value 2.3 is replaced with 10.3, the median value of the data set will be:

0.7, 0.9, 1.2, 1.3, 1.4, 1.6, 1.8, 1.9, 1.9, 2.1, 10.3

Median Location

The median value for the data set is still 1.6 hours.

Note : In an ordered array, the median is the „middle‰ number.

• If n is odd, the median is the middle number.
• If n is even, the median is the average of the two middle numbers.

Copyright © Open University Malaysia (OUM)

138 X TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE
STATISTICS

If the data set has 12 observations instead of 11:

0.7, 0.9, 1.2, 1.3, 1.4, 1.6, 1.8, 1.9, 1.9, 2.1, 2.3, 2.7 , 3.3

n +1 12 + 1
Then, the median location will become , which is = 6.5:
2 2

0.7, 0.9, 1.2, 1.3, 1.4, 1.6, 1.8, 1.9, 1.9, 2.1, 2.3, 2.7 , 3.3

Median Location

1.6 + 1.8
Thus, the median is , which is 1.7 hours.
2
(c) Mode
Mode of distribution is the value that occurs most frequently. It is not affected
by extreme values and normally used for either nominal or ordinal data. There
could be situation where no mode or several modes are in the data set. Such
distribution of scores would be termed as bimodal, trimodal and so on.

Example:
The following is the data on the number of hours in a day spent by 10
students doing mathematics revision.
0.7, 0.9, 1.2, 1.3, 1.4, 1.6, 1.8, 1.9, 1.9, 2.1, 2.3

Based on this data, the mode is 1.9 hours.

To obtain a frequency table and measure of central tendency using SPSS:
1. Select the Analyse menu.
2. Click Descriptive Statistics and then Frequencies⁄. to open the
Frequencies dialogue box.
3. Select the variable(s) you require (i.e. opinion on sex education) and
click the button to move the variable into the Variables(s): box.
4. Click the Statistics⁄.command push button to open the Frequencies:
Statistics sub-dialogue box.
5. In the Central Tendency box, select the Mean, Median and Mode check
boxes.
6. Click Continue and then OK.

Copyright © Open University Malaysia (OUM)

TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE W 139
STATISTICS

ACTIVITY 8.2
For the data given in Activity 8.1, determine the appropriate Measure
of Central Tendency and interpret them in the context of the study.
Justify your selections.

8.3 MEASURES OF DISPERSION

Mean, mode and median show how an average value could be used to represent
a set of data; in other words, a single value to describe the whole data set.
However, this does not give us any information on how the data is spread
around the average value. Thus, it is not sufficient to compare two different sets
of data using averages alone since the average does not give any indication of the
spread or the variability of the data. Therefore, together with the averages,
dispersion is also reported in describing a set of data. The commonly used
measures of dispersion are: range, interquartile range and standard deviation.
Figure 8.8 summarises the commonly used measure of dispersion.

Figure 8.8: Commonly used measure of dispersion

[Note: Since most of the post-graduate research will use primary data and the SPSS will be
used for analysis, this module does not discuss the methods for determining values for the
measure of dispersion for grouped data. Students interested in learning this may refer to any
standard statistics books.]

(a) Range
It is the difference between the largest and the smallest observations
in the data set.

Copyright © Open University Malaysia (OUM)

140 X TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE
STATISTICS

Range = Largest observed value ă Smallest observed value

Example:
The following is the data on the number of hours in a day spent by 10
students doing mathematics revision.

0.7, 0.9, 1.2, 1.3, 1.4, 1.6, 1.8, 1.9, 1.9, 2.1, 2.3

The range is defined as:

Range = 2.3 ă 0.7 which is 1.6 hours

(b) Standard Deviation

Standard deviation is a summary measure of the differences of each
observation from the mean. It tells us the variation in the observation, the
sample data. A set of data with similar mean values can have different
standard deviations. A closely clustered set of data will have a smaller
standard deviation compared to a dispersed set of data. Standard deviation
is the most widely used measure of dispersion. Figure 8.9 illustrates the
mean scores in a mathematics test of eight students.

Figure 8.9 : Standard deviation and distribution of data

Data set A, B and C have the same mean but different standard
deviation. The standard for data set B is the lowest followed by data
set A and C. The smaller the standard deviation, the higher the
consistency between the data in the data set. This is illustrated clearly
in Figure 8.9, where data in set B are distributed more closely
compared to data in set A and C.

Copyright © Open University Malaysia (OUM)

TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE W 141
STATISTICS

n (X − X )
Standard deviation is defined by the formula: SD = ∑ ni −1 where
i =1
X i is the ith observation, X is the mean value of the data set and n is the
number of observations in the data set. If the data set comprises the entire
population (census data), the population standard deviation is defined as,
σ=

N (X − X )
∑ i .
i =1 N

Sample Computation of Standard Deviation

The following is the number of hours in a day spent by 10 students doing
mathematics revision.

0.6, 2.2, 0.3, 1.3, 0.6, 2.2, 1.1, 1.5, 1.6, 1.9

The following is the computation of standard deviation for the above data.

Number of hours Differences from mean Differences squared

watching TV (X i − X ) (X i − X )2
0.6 -0.73 0.53
2.2 0.87 0.76
0.3 -1.03 1.06
1.3 -0.03 0.00

0.6 -0.73 0.53

2.2 0.87 0.76

1.1 -0.23 0.05
1.5 0.17 0.03
1.6 0.27 0.07
1.9 0.57 0.32

Mean Score
X = 1.33
∑ (X i − X ) 2 = 4.12

Copyright © Open University Malaysia (OUM)

142 X TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE
STATISTICS

n (X − X)
4.12
Standard Deviation = ∑ i
n−1
=
9
= 0.677
i =1
In this example, the mean hour for watching TV is 1.33 and the standard
deviation is:

(c) Coefficient of Variation

Dispersion is a relative measure. Standard deviation alone is not sufficient
to illustrate the dispersion of the data in a data set without reference to the
mean value. For example, a standard deviation of 2 units in a data set with
a mean value of 20 has a different connotation compared to a standard
deviation of 5 in a data set with a mean value of 20. The data set in the later
scenario is less densely clustered compared to the former. Figure 8.10 below
illustrates this scenario.

Figure 8.10: Standard deviation of two units

The coefficient of variation (CV %) is defined as standard deviation divided

by the mean, expressed as a percentage.

⎛ SD ⎞
CV = ⎜ ⎟ *100%
⎝ X ⎠

Copyright © Open University Malaysia (OUM)

TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE W 143
STATISTICS

Coefficient of variation measures the scatter in the data relative to the

mean. The higher value of the coefficient of variation indicates that the data
are scattered further from the mean value. The coefficient of variance is a
measure of consistency, a set of data that are closely clustered will have a
smaller value of coefficient of variation compared to a set of data that are
dispersed. The coefficient of variation is also used as an indicator to
determine the presence of outliers in the data set. The coefficient of
variation is normally used to compare the degree of dispersion between
two samples.

Example:
The table below shows the mean weight and standard deviation of two
groups of children.

Group Mean Weight (kg) Standard Deviation (kg)

1 (n = 30) 50.5 2.0

2 (n = 30) 48.5 2.5

The coefficient of variation for group 1 is:

⎛ 2.00 ⎞
⎜ ⎟ *100% = 3.96%
⎝ 50.5 ⎠

The coefficient of variation for group 2 is:

⎛ 2.5 ⎞
⎜ ⎟ *100% = 5.15%
⎝ 48.5 ⎠

The weights of children in group 2 are more dispersed than those in group 1.

ACTIVITY 8.3
For the data given in Activity 8.1, determine the appropriate Measure of
Dispersion and interpret it in the context of the study. Then, justify your
selection.

Also, compute the coefficient of variation for Assignment and Final

Examination marks for male and female respondents and provide your
comments.

Copyright © Open University Malaysia (OUM)

144 X TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE
STATISTICS

ACTIVITY 8.4
Use the data provided in the appendix, perform the following tasks:
1. Use appropriate graphs to summarise the data;
2. Determine the appropriate measure of central tendency and measure
of dispersion to describe the relevant variables; and
3. Write a simple report describing what the researcher can conclude
about the sample.

• Descriptive statistics is mainly used to describe actual observation (what is

gathered from samples).

• Data gathered from samples are usually presented using some form of
graphics for easy interpretation.

• Some commonly used graphs are: Bar Chart, Pie Chart, Line Graph,
Histogram, Stem and Leaf Diagram, and Scatter Diagram.

• The types of graphs and charts depend on how the variables are measured.

• The two types of numerical summaries are measure of central tendency and
dispersion.

• The commonly used measures of central tendency are mean, median and mode.

• Frequently used measures of dispersion are range, interquartile range,

standard deviation and coefficient of variation.

Copyright © Open University Malaysia (OUM)

TOPIC 8 QUANTITATIVE DATA ANALYSIS I: INTRODUCTION TO DESCRIPTIVE W 145
STATISTICS

Describing data Measures of central tendency

Graphs and charts − Mean
− Bar chart − Median
− Histogram − Mode
− Line graph Measure of dispersion
− Pie chart − Coefficient of variation
− Stem and leaf diagram − Range
− Scatter diagram − Standard deviation

Copyright © Open University Malaysia (OUM)

Topic X Quantitative
9 Data Analysis II:
Introduction to
Inferential
Statistics
LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Determine the confidence interval for population mean and proportion
based on sample statistics;
2. Use confidence interval to make statistical inference;
3. Develop null and alternate hypotheses for addressing research
objectives; and
4. Select the appropriate statistical tests for the testing of hypothesis.

X INTRODUCTION
Inferential statistics are statistical tests used in making inference from sample
observation to the relevant population. Unlike descriptive statistics presented in
the earlier topic, the emphasis of inferential statistics is not the samples but
making generalisation from sample data to the population. Generally, there are
two ways to infer from sample to population, either estimating the population
parameter from sample statistics or testing the hypothesis that what is observed
in the sample is a reflection of the actual phenomena in the population.

Copyright © Open University Malaysia (OUM)

TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL W 147
STATISTICS

9.1 CONFIDENCE INTERVALS

Confidence interval is a statistically defined range that contains the population
parameter. As mentioned earlier, population parameters are generalisations
made from sample observation. It is almost impossible to estimate a population
parameter as a single point estimate due to sampling error. Therefore, population
parameters are estimated to fall within a given range which is statistically
defined. This range is called the Confidence Interval. For example, instead of
giving one point estimate of the population mean, it is more practical to give a
probable range of values where the population mean falls. This range is known
as confidence interval. The lower and upper limits are known as confidence
limits. The probability that a population value lies within this range is known as
confidence level.

The 95% confidence interval for mean is defined as:

σ
μ ± 1.96 ( μ = mean; σ = standard deviation, n = sample size)
n

1.96 when α = 0.05

Copyright © Open University Malaysia (OUM)

148 X TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL
STATISTICS

The 95% confidence interval for proportion is defined as:

p s (1 − p s )
p s ± 1.96 ( p s = proportion of success, n = sample size)
n

1.96 when α = 0.05

95%

p s (1 − p s ) p s (1 − p s )
p s − 1.96 p s + 1.96
n n

Lower Limit Upper Limit

There are two important elements in confidence interval:

(a) The confidence level (how confident we are that the population parameter
will fall within the range); and
(b) Precision (the width of the range).

Confidence interval allows us to use sample data to estimate population

parameters, like the population mean or population proportion. For a normal
distribution, multiples of the standard deviation mark certain limits on the
spread of the observations. For instance, 95% of the observations lie between 1.96
standard deviations above and 1.96 standard deviations below the mean
( ±1.96 SD ). In practice, where multiple sampling is not done, the population
mean ( μ ) is estimated using the sample mean ( x ) .

Example:
A researcher intends to determine the „mathematics problem-solving ability‰ of
Form Five science stream students in his school. He gave a mathematics
problem-solving test to 65 randomly selected Form Five science stream students

Copyright © Open University Malaysia (OUM)

TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL W 149
STATISTICS

and computed the mean score and standard deviation. He then computed the
95% confidence interval for the test scores and reports it as the „mathematics
problem-solving ability‰ of the Form Five science stream students in his school.

The following are his findings:

The mean score is 89 with a standard deviation of 4.6. Since the population
standard deviation is not known, the sample standard deviation is used to
estimate the standard error of mean.

SD
The standard error of mean is , which is 0.57. The following is the
n
computation of the confidence interval.

Lower Limit
SD
x −1.96
n
89 - (1.96 x 0.57) = 87.88

Upper Limit
SD
x +1.96
n
89 + (1.96 x 0.57) = 90.12 ă lower limit

Thus, the 95% confidence interval for the „mathematics problem-solving

ability‰ of Form Five science stream students in the school is [87.88 ă 90.12].

9.2 STATISTICAL INFERENCE WITH

CONFIDENCE INTERVAL
We can interpret the confidence interval in the previous example as follows:
If a Form Five science stream student from the school is randomly selected and
given the test, there is a 95% chance that he will get a score between 87.88 and
90.12. This is what we term as statistical inference.

Example:
Table 9.1 presents the mean „mathematics problem-solving score and standard
deviation‰ of two groups of randomly selected Form Five students from the
science and arts streams in the same school. There, researchers would like to see
if there is a significant difference in the ‰mathematics problem-solving ability‰
between Form Five students from the science and arts streams in his school.

Copyright © Open University Malaysia (OUM)

150 X TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL
STATISTICS

Table 9.1: Mathematics Problem-Solving Test Scores

Stream Number Mean Standard SD

Score Deviation n

Science 65 89 4.6 0.57

Arts 46 77 4.3 0.63

Solution:
The confidence interval for science stream students:
89 + (1.96 x 0.57) = 90.12 ă upper limit
89 - (1.96 x 0.57) = 87.88 ă lower limit
95% Confidence Interval is between 87.88 ă 90.12.
The confidence interval for arts stream students:

77 + (1.96 x 0.63) = 78.24 – upper limit

77 - (1.96 x 0.63) = 75.76. – lower limit

95% confidence interval is between 75.76 and 78.24.

Arts Stream Form Five students Science Stream Form Five students

Interpretation:
Since the confidence intervals for the science and arts stream Form Five students
do not overlap, it can be concluded that there is a significant difference in the
mathematics problem-solving ability between these two groups of students.
From the mean scores, we can further conclude that science stream students have
higher mathematics problem-solving ability than the arts stream students.
Copyright © Open University Malaysia (OUM)
TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL W 151
STATISTICS

ACTIVITY 9.1
A researcher wanted to study the Emotional Intelligence of university
students in the Klang Valley. He defined Emotional Intelligence as a
three-dimensional construct comprising Stress Tolerance, Optimism
and Emotional Self-Awareness. The following table shows the
descriptive statistics of the variables for male and female students.
Compute the confidence interval and interpret your results.

Gender N Mean Std. Dev.

Male 78 4.709 0.7536
Stress Tolerance
Female 96 4.571 0.7041
Male 78 4.821 0.6294
Optimism
Female 96 4.746 0.6191
Male 78 4.708 0.7946
Emotional
self-awareness Female 96 4.617 0.6914

9.3 HYPOTHESIS TESTING

Hypothesis testing allows us to use sample data to test a claim about a
population, such as testing whether a population proportion or population mean
equals to some values or there exists a significant difference between two or more
populations regarding certain phenomena. There are two types of hypotheses:
null and alternative.

(a) Null Hypothesis

A null hypothesis proposes no difference or no relationship between the
variables of interest. It is commonly written as Ho, and is the foundation of
statistical test. In testing a hypothesis, we assume the null to describe the
state of affairs correctly. The main interest of the researcher is to find
enough evidences to reject the null hypothesis ă this is known as statistical
significance. The following are examples of null hypothesis.

Copyright © Open University Malaysia (OUM)

152 X TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL
STATISTICS

One Sample:
The mean mathematics problem-solving score of Form Five science stream
students in the school is 85.

Two Samples:
There is no difference in the mean mathematics problem-solving score
between the science and arts stream Form Five students in the school.

(b) Alternate Hypothesis

An alternate hypothesis (sometimes called research hypothesis) is a
hypothesis that contradicts the null. It is commonly written as Ha. An
alternate hypothesis can indicate the direction of the differences or
relationship, or assume a neutral position. If direction is emphasised
(indicated in the alternate hypothesis), we call it one tailed-test, if not the
test will be a two-tailed test. The following are examples of alternate
hypothesis.

One Sample:
The mean mathematics problem-solving score of Form Five science stream
students in the school is not 85.

Two Samples:
There is a difference in the mean mathematics problem-solving score
between the science and arts stream Form Five students in the school.

General steps in hypothesis testing:

• Write down the null and alternate hypotheses.
• Determine the level of significance ( α level).
• Collect evidence and perform the statistical analysis.
• Based on the available evidence, decide whether to reject the Null
Hypothesis.

Some common types of hypothesis testing are:

• Determining significant difference(s) by comparing sample statistics
with a reference (or benchmark) values.
• Determining significant difference(s) by comparing samples derived
from two or more population of interest.
• Analysing the significance of relationships between variables.

Copyright © Open University Malaysia (OUM)

TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL W 153
STATISTICS

Types of Error
Regardless of how carefully samples are selected for a study, there will always be
some variation in the results when compared with results derived from the entire
population. This variation is due to sampling error. Thus, the inference from
samples to populations are probabilistic; we can be close to 100% certain by
employing strict research design but can never be 100% certain that our inference
is correct. Drawing the wrong conclusion is referred to as error of inference.
There are two types of error: type I and type II errors ă both relate to the rejection
or acceptance of the null hypothesis. Table 9.2 illustrates type I and type II errors
in relation to the decision on the null hypothesis.

Table 9.2: Related Decisions about the Null Hypothesis and Error Types

Null Hypothesis
Decision
True False

Accept Ho OK Type II Error ( β )

Reject Ho Type I Error ( α ) OK

Type I error is committed when the researcher rejects the null when the null is
indeed true; in other words, incorrectly rejecting the null. The probability level
where the null is incorrectly rejected is called the significance level, denoted by
the symbol α a value set a priori (before even conducting the research) by the
researcher.

On the other hand, type II error is committed when the researcher fails to reject
the null when the null is indeed false; in other words, wrongly accepting the null.
The type II error is often denoted as β . In any research, the intention of the
researcher is to correctly reject the null. If the design is carefully selected and the
samples represent the population, the chances of achieving this objective are
high. Thus, the power of the study is defined as 1 - β .

Both the α and β have some implications on the sample size; holding the
sample size constant, α and β are inversely related. This is to say that if the
researcher would like to minimise the probability of committing type I error, the
probability of committing type II will increase.

Copyright © Open University Malaysia (OUM)

154 X TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL
STATISTICS

9.4 STATISTICAL INFERENCE WITH

HYPOTHESIS TESTING
In hypothesis testing, we are trying to test whether the differences or relationship
observed in the sample are true reflections of the population or merely due to
chance (sampling error). The following are the steps involved in hypothesis
testing:
Step 1 : State the Null Hypothesis (it assumes no difference)
Step 2 : State the Alternate Hypothesis (this states that there is a difference)
Step 3: Decide on the significance level and a suitable statistical test
Step 4: Collect data using appropriate samples and determine the test statistics
and p value
Step 5 : Decision making (to reject or not reject the Null Hypothesis)
Step 6: State the conclusion

The following example illustrates the hypothesis testing procedure.

Example:
It is given that the mathematics problem-solving score of 50 randomly selected
Form Five science stream students in the school is 89 with a standard deviation of
10.8. If it is evident from a previous similar study that the mean score for
mathematics problem-solving of Form Five science stream students in the district
is 85, what can be concluded from the results obtained from the sample?

Solution:

Step 1: State the Null Hypothesis:

Ho: The mean mathematics problem-solving score of Form Five
science stream students in the school is 85.

Step 2: State the Alternate Hypothesis:

Ho: The mean mathematics problem-solving score of Form Five
science stream students in the school is greater than 85.

The alternate hypothesis shows that the researcher is being optimistic with his
studentsÊ ability.

Copyright © Open University Malaysia (OUM)

TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL W 155
STATISTICS

Step 3: Decide on the significance level and a suitable statistical test:

The significance level ( α ) is set as 0.05 and the appropriate test is the
one-sample t-test.

The sample size is 50, and is large

enough to use the t-test.

Step 4: Assume that the data was collected appropriately. The sample mean
( X ) and standard deviations (SD) are 89 and 10.8 respectively while
population mean ( μ ) is 85 (obtained from the null hypothesis).

X−μ 89 − 85
The t value is defined as t = = = 2.61
SD / n 10.8 / 50

This is the formula

for the t-test

Step 5 : Decision Making

The t-value is 2.61 and the associated p value is 0.01 (obtained from the
t-table ă see Appendix 1). Since the p-value is less than 0.05, the decision
is to reject the null and conclude the alternative hypothesis.

Step 6: Conclusion
The mean mathematics problem-solving score of Form Five science
stream students in the school is not equal to 85.

Points to Ponder
The hypothesis can be directional (one tailed) or non-directional (two tailed). The
difference between one-tailed and two-tailed tests is how the hypothesis is
formulated. For the one-tailed test, the Alternate Hypothesis will indicate
direction while no direction is indicated for the two-tailed test. The decision on
whether to use a one-tailed or two-tailed test depends on the amount of
information one has based on literature. Hypothesis for the one-tailed test
(directional hypothesis) should be strongly guided by literature. On the
implication on the statistical significance, it is far more difficult to reject the non-
directional hypothesis since the observed probability needs to be compared to

Copyright © Open University Malaysia (OUM)

156 X TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL
STATISTICS

α
the significance level divided by two ( ). If the researcher is unsure of the
2
direction, the non-directional hypothesis is recommended. The following is an
example of how to write the null and alternate hypotheses for one-tailed and
two-tailed tests.

Two-Tailed Test
Null (Ho) : There is no difference in the mean mathematics problem-
solving score between the science and arts stream Form Five
students in the school.

Alternate (Ha) : There is a difference in the mean mathematics problem-solving

score between the science and arts stream Form Five students
in the school.

Rejection region for Two-Tailed Test

One-Tailed Test
(a) Right Tail
Null (Ho): The mean mathematics problem-solving score of science
stream Form Five students in the school is less than or
equal to the scores of Form Five students from the arts
stream.

Alternate (Ha): The mean mathematics problem-solving score of science

stream Form Five students in the school is higher than the
scores of Form Five students from the arts stream.

Rejection region for One-Tailed Test

Copyright © Open University Malaysia (OUM)

TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL W 157
STATISTICS

(b) Left Tail

Null (Ho) : The mean mathematics problem-solving score of science
stream Form Five students in the school is equal to or
greater to the scores of Form Five students from the arts
stream.

Alternate (Ha): The mean mathematics problem-solving score of science

stream Form Five students in the school is lower than the
scores of Form Five students from the arts stream.

Rejection region for One-Tailed Test

ACTIVITY 9.2
Apart from the three variables related to Emotional Intelligence (Stress
Tolerance, Optimism, and Emotional Self-Awareness) the researcher also
collected some demographic information. Among the demographic
variables are gender, ethnicity and types of institution (either
government of private).
Write the appropriate hypotheses that the researcher can test in this study.
You are required to write the hypotheses in null and alternate form.

9.5 PARAMETRIC VERSUS NON-PARAMETRIC

TEST
Descriptive statistics is used to compute summary statistics (e.g. mean, median,
standard deviation) to describe the samples, while statistical test is used for making
inference from sample to the intended population as illustrated in Figure 9.1.

Copyright © Open University Malaysia (OUM)

158 X TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL
STATISTICS

Figure 9.1: Statistical test are used for making inference

There are two categories of statistical test:

(a) Parametric test

A parametric test is a statistical test which requires the distribution of the
population to be specified. Thus, parametric inferential methods assume
that the distribution of the variables being assessed belongs to some form of
known probability distribution (e.g. assumption that the observed data are
sampled from a normally distributed population).

(b) Non-parametric test

In contrast, a non-parametric test (also known as distribution free test) does
not require the distribution to be specified prior to the research, thus this
family of tests does not require the assumption on the distribution. Most
commonly, non-parametric tests rank the outcome variable from low to
high and then analyse the ranks rather than the actual observation.

Choosing the right test will contribute to the validity of the research
findings. Improper use of statistical tests will not only cause the validity of
the test results to be questioned and do little justification to the research but
at times it can be a serious error especially if the results have major
implications (for example, if these are used in policy formulation and so
on).

Parametric tests have greater statistical power compared to their

non-parametric equivalent. However, it is not all the time that parametric
tests are the best tests to use; parametric test should be used if researcher is
sure that the data are sampled from a population that follows a pre-defined
distribution (e.g. a normal distribution).

On the other hand, a non-parametric test should be used if:

(i) The outcome is a rank (e.g. rank order of the test scores);
(ii) No assumption is made on the population distribution; and
(iii) There are outliers.

Copyright © Open University Malaysia (OUM)

TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL W 159
STATISTICS

Sometimes, it is not easy to decide whether a sample comes from a normal

population. The following clues can be used to make decision on normality:
(i) Construct a histogram with normal curve overlapping; it will be fairly
obvious whether the distribution is approximately bell shaped.
(ii) For large data set, use the Kolmogorov-Smirnoff test (sample > 100) or
Shapiro Wilk test (sample < 100) to test whether the distribution of the
data differs significantly from normal. This test can be found in most
statistical software.
(iii) Examine the literature. What matters is the distribution of the overall
population, not the distribution of the sample. In deciding whether a
population is normal, look at all available data, not just data in the
current experiment.
(iv) When in doubt, use a non-parametric test. You may have less
statistical power but at least the results would be valid.

Sample size plays a crucial role in deciding the family of statistical test,
parametric or non-parametric. In large sample, the central limit theorem
ensures that parametric tests work well even if the population is not
normal. Parametric tests are robust to deviations from normal distributions,
when sample size is large. The issue here is how large is large enough. A
rule of thumb suggests that a sample size of about 30 or more for each
category of observation is sufficient for one to use the parametric test.
Non-parametric tests also work well with large samples.
Non-parametric tests are only slightly less powerful than parametric tests
with large samples.

On the other hand, if the sample size is small, we cannot rely on the central
limit theorem. Thus, the p value may be inaccurate if parametric tests were
to be used.

Figures 9.2, 9.3 and 9.4 summarise the common statistical tests used in
education research.

Copyright © Open University Malaysia (OUM)

160 X TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL
STATISTICS

Figure 9.2: Commonly used one-sample test in Education Research

Figure 9.3: Commonly used test in Education Research for comparing two populations

Copyright © Open University Malaysia (OUM)

TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL W 161
STATISTICS

Figure 9.4: Commonly used test in Education Research for comparing

more than two populations

ACTIVITY 9.3
For the hypotheses listed for activity 9.2, determine the appropriate
statistical tests. The sample comprises 78 males and 96 females. In terms
of ethnicity, 75 are Malays, while Chinese and Indians are 53 and 46
respectively. As for the types of institution, 108 are from the government
universities while 66 are from private institutions.

Copyright © Open University Malaysia (OUM)

162 X TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL
STATISTICS

SELF-CHECK 9.1

1. Define confidence interval.

2. List the steps for hypothesis testing.
3. Describe the differences between parametric and non-parametric
statistical tests.
4. Explain the concept of type I and type II error in hypothesis testing.

• Inferential statistics are statistical tests used in making inferences from a

sample to the relevant population.

• There are two ways to infer from sample to population, either by estimating
the population parameter from sample statistics (confidence interval) or
testing the hypothesis.

• Confidence interval allows us to use sample data to estimate population

parameters.

• Hypothesis testing allows us to use sample data to test a claim about a

population.

• The following are the steps involved in hypothesis testing:

Step 1 : State the null hypothesis
Step 2 : State the alternate hypothesis
Step 3: Decide on the significance level and the suitable statistical test
Step 4: Collect data using appropriate samples and determine the test
statistics and p value
Step 5 : Decision Making (to reject or not reject the null hypothesis)
Step 6: Conclusion

Copyright © Open University Malaysia (OUM)

TOPIC 9 QUANTITATIVE DATA ANALYSIS II: INTRODUCTION TO INFERENTIAL W 163
STATISTICS

Alternate hypothesis Null hypothesis

Confidence interval Parametric test
Inferential statistics Type I error
Non-parametric test Type II error

Copyright © Open University Malaysia (OUM)

Topic X Writing the
10 Research Report
LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Describe the elements of a research report; and
2. Write a research report according to the proposed format.

X INTRODUCTION
From Topic 1 to Topic 9, we have discussed the methods and techniques of
conducting research from the quantitative perspective. This topic will focus on
the writing process and will address one important task that most graduate
students will have to do, that is, write the research report. The guidelines
discussed in this topic are more appropriate for quantitative research rather than
qualitative research. Despite this, the format is still relevant to qualitative
research with some adaptations. You may refer to HMEF5103 Qualitative
Research Methodology for specific details.

ACTIVITY 10.1
Take a look at a research report and identify the chapters and major
sections in each chapter.

Read the chapters and the major sections carefully and take note of what
is focused on each chapter and major section.

Copyright © Open University Malaysia (OUM)

 TOPIC 10 WRITING THE RESEARCH REPORT W 165

10.1 WHAT IS A RESEARCH REPORT?

All graduate students pursuing a programme that requires them to take a
masterÊs project will have to write a research report. What is a research report? A
research report is an academic write-up that details the whole research process
undertaken by a graduate student, right from the conceptualisation of a research
problem to the discussion of his/her research findings. Open University
Malaysia recognises a good research report as one which:
(a) Demonstrates the studentÊs understanding of his/her research problem,
strategies to seek solutions to the problem through appropriate use of data
collection methods and analytical techniques, and ability to interpret and
relate the results to the objectives of his/her study.
(b) Attempts to search for explanations to the problems and meanings of
concepts through a comprehensive and critical review of the relevant
literature from scholarly books and journals.
(c) Constitutes a clean piece of work, well-edited and thoroughly checked for
grammatical errors, spelling mistakes, sentence construction, report formats
and others. Proper citations of authors in the texts and proper writing of
references should be observed according to the writing style suggested by
the American Psychological Association (APA).
(d) Demonstrates the studentÊs ability to present ideas, explanations and
results in a smooth and logical flow from the beginning to the end of each
topic. Overall coherence of the report should be maintained through
appropriate explanatory linkages between topics.

The research report as required by Open University Malaysia is a three-part

document: Preliminary Section, the Body and Supporting Materials. Figure 10.1
presents the overall format of the document.

Copyright © Open University Malaysia (OUM)

166 X TOPIC 10 WRITING THE RESEARCH REPORT

Figure 10.1: Format of Final Research Report

Copyright © Open University Malaysia (OUM)

 TOPIC 10 WRITING THE RESEARCH REPORT W 167

10.2 PRELIMINARY SECTION

The preliminary section consists of the title page, the abstract,
acknowledgements, table of contents, list of tables and figures. All pages in the
preliminary section are to be numbered at the centre of the bottom margin using
lower-case Roman numerals (i, ii, iii, iv, etc.). The title page should not be
numbered.

(a) Title Page

Although title pages may differ from one institution to another, they
usually include the:
(i) Name of the topic;
(ii) Name of the author;
(iii) Relationship of the report to a degree requirement;
(iv) Name of the institution where it is to be submitted; and
(v) Date of presentation.

The title should be concise and should indicate clearly the purpose of the
study. Keep in mind its possible usefulness to another researcher who may
search a database where your dissertation may be listed. The title should not
claim more than the study actually delivers. It should not to be stated broadly
and make it difficult for the reader to determine what the study is about. For
example, the title „Mathematics Achievement of Urban Poor Children‰ is too
general and a more precise title would be „Mathematics Achievement of Poor
Children in an Urban School in Sarawak‰. These items should be written in
Times Roman font/16pt. The title should be in capital letters, single-spaced
and centred between the right and left margins of the page. If the title goes
beyond one line, the words in the title should be divided into lines so that each
successive line is shorter than the one above it and is centred below it in an
inverted pyramid style as shown in Figure 10.2.

(b) Abstract
The abstract is a short, one-paragraph summary of the most important
components of the research study. It normally includes a brief statement of
the problem, the objectives of the study, the main research questions, the
sample involved, the research design adopted, the major findings and their
implications. The abstract should not exceed 200 words written in single
spacing. Open University Malaysia requires that you have the translation of
your abstract in Bahasa Malaysia if you have written your report in English.
Likewise, you need to provide the English translation of the abstract if the
Copyright © Open University Malaysia (OUM)
168 X TOPIC 10 WRITING THE RESEARCH REPORT

report is in Bahasa Malaysia. The translated abstract should include the title
as well.

(c) Acknowledgements
An acknowledgement page is included if you have received unusual
assistance in the conduct of the study. The acknowledgements should be
simple and restrained. Do not indulge in flattery and excessive recognition
for routine participation of family members, lecturers, supervisors,
librarians and clerical helpers.

Figure 10.2: Example of a Title Page

(d) Declaration
The declaration is meant for the candidate to declare that the research
report is the result of your own work. You should use the format provided
by Open University Malaysia as provided in Figure 10.3.

Copyright © Open University Malaysia (OUM)

 TOPIC 10 WRITING THE RESEARCH REPORT W 169

DECLARATION

Name:

Matric Number:

I hereby declare that this dissertation is the result of my own work, except for
quotations and summaries which have been duly acknowledged.

Signature: Date:

Figure 10.3: Example of a Declaration

(e) Table of Contents

The table of contents serves an important purpose in providing an outline
of the contents of the research report. Differentiate between headings and
sub-headings using capitalisation and small letters. Page references for each
topic should be indicated. Refer to Figure 10.1.

(f) List of Tables and Figures

If tables and figures are included in the research report, a separate section
should be included to list each table or figure. The full titles of figures and
tables, worded exactly as they appear in the text, should be presented with
corresponding numbers and page locations.

The acknowledgements, table of contents and list of tables and figures

sections should be presented using Times Roman font/12pt.

10.3 THE BODY OF A RESEARCH REPORT

The body section details the major contents of the research you have undertaken.
These contents are presented in five topics. Refer to Figure 10.1. For some reports,
there are six topics with the fifth topic being broken down into two topics, one
focusing on Discussion and the other on Summary and Conclusion.

(a) Chapter 1: Introduction

The introduction should present the research problem or issue that you
intend to study.
(i) The first section in the introduction chapter is the background to the
study. In this section, you should provide a description of the
background to the research problem or issue that you intend to study.

Copyright © Open University Malaysia (OUM)

170 X TOPIC 10 WRITING THE RESEARCH REPORT

This should be as brief as possible. Do not ramble! The background

should be clear and straight to the point. Describe the general field of
research relating to the issue and then narrow down to the specific
area you are concerned with.
(ii) Discuss the issue that you intend to study in terms of problem
statement and show that there is an issue that needs to be addressed
or a „gap‰ in the research that you will fill. When you are able to
identify the issue or gap, then the research question will fall in place
naturally. Think of the introduction as follows:

Imagine a group of academics discussing in general the area of

study you are interested in. You join in the conversation (assuming
they allow you to) and draw their attention to your specific
problem of interest. You tell them that there is something that has
not been resolved or there is a gap or problem. You argue that this
gap or problem has to be addressed and go on to describe it in
detail. Then you tell your listeners the objectives of your study and
how it will attempt to answer the research questions.

In your problem statement, you not only explain what you are
studying i.e. the independent and dependent variables involved but
also provide research and theoretical support for your study.

(iii) In one or two statements, write the objectives of the study. Ask
yourself what you hope to uncover through your research and then
say it in a way that your colleagues will understand. Use simple and
jargon-free language. In this section, you might want to elaborate on
the conceptual framework to further clarify your objectives.
Otherwise, you can have an additional section on Research Design in
Chapter 3 to explain the conceptual framework of your study.

(iv) The objectives are followed by research questions. The research

questions must be in line with the objectives and should indicate the
variables under investigation. They should be clearly and
unambiguously framed as they will influence research methodology
and the type of data analysis to be performed. Include null
hypotheses, if necessary.

Copyright © Open University Malaysia (OUM)

 TOPIC 10 WRITING THE RESEARCH REPORT W 171

(v) Briefly tell the reader the significance of the study (justify doing the
study). You can argue the significance of your study based on the
following criteria:
• The problem or gap demands attention because the findings could
influence practice and policy;
• The methodology you are using is unusual;
• You are studying certain variables which have not been given
attention in previous studies;
• Your study will contribute to the body of knowledge in the field;
• The outcome could be the extension of a theoretical model.

(vi) The limitations of the study refer to elements which are beyond the
control of the researcher. Outline the limitations in terms of the
sample, data collection and/or data analysis so that the reader is
aware of the parameters of your study. Do not be intimidated by the
limitations of the study because it is not possible to expect every
research conducted to be perfect. They merely indicate to the reader
that you are aware of the limitations and that the findings should be
interpreted in light of these limitations.

(vii) Provide operational definitions of key terms, especially the variables

investigated in the study. In this context, you should be aware of the
distinction between a constitutive definition and an operational
definition. A constitutive definition is the dictionary type of definition.
This type of definition helps to convey the general meaning of a variable,
but it is not precise enough for research purposes. You need to define the
variables in your study so that readers know exactly what is meant by
the terms and so that other researchers can replicate the research. This is
only possible if you provide operational definitions of the variables.
There are two types of operational definitions: measured and
experimental. A measured operational definition details the operations
by which researchers measure a variable. For example, intelligence may
be operationally defined as scores on the Stanford-Binet Intelligence
Scale. On the other hand, an experimental operational definition details
the steps a researcher takes to produce certain experimental conditions.
For example, the operational definition of concept mapping strategy in a
reading research study may take the form of a group of students reading
texts with the help of concept maps (experimental condition) and another
group of students reading texts without the help of concept maps
(control condition).

Copyright © Open University Malaysia (OUM)

172 X TOPIC 10 WRITING THE RESEARCH REPORT

SELF-CHECK 10.1

1. What are the major sections in Chapter 1?

2. What is the focus of each major section?

(b) Chapter 2: Review of Literature

Literature review provides details on what other researchers have done in
the area, and what you propose to do. In this section, you need to cover the
following:
(i) The major issues or schools of thought.
(ii) The gaps in the literature (in more detail than that provided in the
introduction).
(iii) Research questions and/or hypotheses (for quantitative research)
which are connected carefully to the literature being reviewed.
(iv) Definition of key terms (this can be done when you introduce each
idea, or in a definition section). You should provide the operational
definitions of the key variables of your study.
(v) Methodological issues arising from the gaps relating to sampling,
instrumentation, data collection procedures and data analysis.
(vi) The theoretical framework, which can form either the first or the final
part of the Literature Review chapter. It describes the theoretical basis
that you are using in conducting your research.

Generally, this chapter provides a background for the development of your

study and brings the reader up to date about research and thinking in the
field. It also gives evidence of your knowledge of the field. You should
avoid an article-by-article presentation but should indicate areas of
agreement or disagreement in findings or gaps in existing knowledge.
The journal Review of Educational Research can be referred to for examples
of good critical reviews of the literature. Also, avoid excessive use of
quotations. Nothing is more tiresome or difficult to follow than a review of
literature that is merely an accumulation of quotations. Furthermore, you
should rely more on primary sources for your review. Although books are a
good source of reference, you should make as much use of the journals
which are well recognised and known in the area of your research. As the
literature review indicates the current state of knowledge in the area, the
references reviewed should also be as recent or up to date as possible.

Copyright © Open University Malaysia (OUM)

 TOPIC 10 WRITING THE RESEARCH REPORT W 173

SELF-CHECK 10.2

Why is the review of literature important in a research?

(c) Chapter 3: Methodology

This chapter describes the methodology used in great detail and with
justifications of its use over other similar methodologies. For example, you
could explain:
(i) Why you are using a certain paradigm or theory.
(ii) Why you are using quantitative research.
(iii) Why you are using a case study of a specific kind.
(iv) Why you are using surveys, correlational studies, experiments, etc.
(v) Why you are researching certain dependent or independent or
moderating variables.
(vi) Why you have chosen a sampling frame and the size of a certain
sample.
(vii) How you are proposing to have access to the data.
(viii) How you are proposing to analyse the data.

The following details the sections of the chapter on methodology.

(i) Sample
It is here that you explain the size of the sample and how you select
them. Indicate the extent to which the sample is representative of the
population. For example, what is the composition of your sample?
Are you going to use random sampling or purposive sampling and
why?

(ii) Instrumentation
You should include a clear description of the data collection
techniques or instruments you will use. For example, if your study is a
survey, you have to explain how you design and develop the
questionnaire or interview checklist. Explain the number and types of
items included in the questionnaire. If you are going to use attitude
scales, achievement tests and other psychological tests, you have to
give evidence regarding the reliability and validity of the instruments.

Copyright © Open University Malaysia (OUM)

174 X TOPIC 10 WRITING THE RESEARCH REPORT

You should also describe the scoring procedures adopted for the
instruments used.

(iii) Data Collection Procedures

You should provide a clear description of the procedures you use in
collecting your data. Among the various issues discussed in this
section include the implementation of pilot studies and the actual
research procedures.

(iv) Framework of Data Analysis

This section elaborates on the techniques used in the data analysis to
obtain the information required to answer the research questions.
Ideally, you should explain how you will analyse the data obtained
under each research question. It would be helpful if you can present a
summary of your analytical framework using the grid shown in
Figure 10.4.

Figure 10.4: Summary of Analytical Framework

SELF-CHECK 10.3

1. What are the major sections in Chapter 3?

2. What is the main focus of each section?

(d) Chapter 4: Data Analysis and Results

In this chapter, you present the analysis of the data obtained from the
study. This is the heart of the research report. For a quantitative study,
tables and figures (e.g. graphs) are commonly used to organise and present
numerical data. Tables and figures are useful in presenting an overall
picture of the data as well as showing trends that have emerged from the
analysis. Describe all findings that are shown in the tables and figures in
detail. You are advised to refer to the Journal of Educational Psychology

Copyright © Open University Malaysia (OUM)

 TOPIC 10 WRITING THE RESEARCH REPORT W 175

and the American Educational Research Journal to see how tables and
graphs are presented and explained.

If you do a qualitative study, there will be less numerical data. Instead your
data would consist of concepts, categories or themes which may be
presented in table form. You would also be presenting data in the form of
anecdotes or excerpts of interviews, observations and documents to
support your arguments. For qualitative studies, Qualitative Report is a
useful journal which presents reports of qualitative studies in education,
nursing and medicine. As the focus here is on quantitative research
methodology, the format of data analysis and data presentation is beyond
the scope of this module.

For clarity, you may want to present your analyses and findings under each
research question. However, demographic data of the sample that are not
findings to your research questions should not be presented in this chapter.
This information should ideally be presented in Chapter 3 under the
heading „Sample‰.

SELF-CHECK 10.4

What is the focus of Chapter 4?

(e) Chapter 5: Discussion and Conclusion

The last chapter comprises two parts. The first part includes a brief
summary of the problem, methodology and results. Focus should be on a
summary of the main findings and it should be as brief as possible. Some
researchers present the main findings in the form of list. The second part is
a discussion of the findings. Here, you identify and interpret the findings.
Give possible reasons why the results occurred. You could also provide
reasons by referring to the findings of previous research. This is where the
studies cited in Chapter 2 are useful. Explain how far your findings are
similar to or different from those of previous research. You should also
relate your findings to the theory/theories upon which your study is based.
Because you are the one who conducted the study, you should have a
deeper understanding of the study compared to most readers. Thus, you
are expected to discuss the findings and to give your own opinion about the
outcomes. Other important sections in this chapter are implications of your
findings and direction for future research.

One of the most common weaknesses found in the writing of graduate

students is that their reports present important and interesting findings but
Copyright © Open University Malaysia (OUM)
176 X TOPIC 10 WRITING THE RESEARCH REPORT

fail to provide a thoughtful interpretation and discussion of the findings in

relation to past research and existing theories. There is also the tendency to
over-generalise on the basis of limited data. Remember, your study is not
attempting to change the whole education system!

You should keep in mind that this chapter is the most used part of the
research report by other readers. Readers who scan research literature to
find significant studies examine this chapter before deciding whether or not
further examination of the report is worthwhile.

SELF-CHECK 10.5
What could be the sections in Chapter 5?

10.4 SUPPORTING MATERIALS

This section of the report comprises the References and Appendices.

(a) List of References

(i) This must be provided in scholarly fashion. It helps to convince your
readers that your proposal is worth pursuing if you can identify
literature in the field and demonstrate that you understand it.
(ii) Use the citation style proposed by the Manual of the American
Psychological Association (APA style for short). You can access the
APA writing style from http://owl.english.purdue.edu/owl/
resource/560/01/
(iii) Make sure that all materials cited in the text are also found in the list
of references, and vice versa. This will reflect your academic honesty
in producing the research report.

(b) Appendices
Include in this section all the relevant support materials especially the
research instruments that are used in the study. This will indicate to
whoever assesses your research report that it is a complete and quality
product.

NOTE:
For details about the scope and depth of the research report, please refer to
Appendix 2. The appendix presents the scope and depth of the research

Copyright © Open University Malaysia (OUM)

 TOPIC 10 WRITING THE RESEARCH REPORT W 177

report for the PhD (Edu) Thesis , the MasterÊs Dissertation (Option 1) and
the MasterÊs Project (Option 2).

ACTIVITY 10.1
Compare what you have read about the research report in this topic with
what you have noted in Activity 10.1.
How are they similar to each other?
How are they different from each other?

Ć A research report is an academic write-up that details the whole research

process undertaken by a graduate student, right from the conceptualisation
of a research problem to the discussion of his/her research findings.
Ć The research report as required by Open University Malaysia is a three-part
document: Preliminary Section, the Body and Supporting Materials.
Ć The preliminary section consists of the title page, the abstract,
acknowledgements, declaration, table of contents, list of tables and figures.
Ć The body section details the major contents of the research and they are
presented in five or six topics.
Ć References and Appendices make up the supporting materials of a research
report.

Conceptual framework Instrumentation

Data collection Methodology
Definitions of terms Problem statement
- Constitute definition Research design
- Operational definition Research questions
Discussion Review of literature
Framework of data analysis Sample
Hypotheses Theoretical framework

Copyright © Open University Malaysia (OUM)

References
Ary, D. Jacobs, L. C., & Razavieh, A. (2002). Introduction to research in education
(6th ed.). USA: Wodsworth/Thomson Learning.

Benz, C., & Newman, I. (1998). Qualitative-quantitative research methodology:

Exploring the interactive continuum. Carbondale: Southern Illinois
University Press.

Black, T. R. (1999). Doing quantitative research in the social sciences: An

integrated approach to research design, measurement and statistics. London:
SAGE.

Borg, W., & Borg, M. (1988). Educational research: An introduction. New York:
Longman.

Christensen, L. (1988). Experimental methodology. Boston: Allyn and Bacon, Inc.

Christensen, B., & Johnson, B. (2000). Educational research: Quantitative and

qualitative approaches. Needham Heights, MA: A Pearson Education
Company.

Firestone, W. A. (1987). Meaning in method: The rhetoric of quantitative and

qualitative research. Educational Researcher, 16, pp. 1-6.

Guba, E., & Lincoln, Y. (1989). Fourth generation evaluation. Newbury Park,
California: Sage Publications.

Helmstadter, G. (1970). Research concepts in human behaviour. New York:

Appleson-Century Crofts.

John Arul Phillips (2007). HMEF 5014 Educational research methodology. Kuala
Lumpur: OUM.

Kerlinger, F. (1973). Foundations of behavioural research. New York: Holt

Rinehart & Winston.

Kuhn, T. (1973). The structure of scientific revolution. Chicago: University of

Chicago Press.

Copyright © Open University Malaysia (OUM)

 REFERENCES W 179

Leedy, P. D. (1993). Practical research: Planning and design. New Jersey:

Prentice-Hall.

Mitchell, M., & Jolley, J. (1988). Research design explained. New York: Holt,
Rinehart and Winston.

Copyright © Open University Malaysia (OUM)

180 X ANSWERS

Copyright © Open University Malaysia (OUM)

APPENDICES

Copyright © Open University Malaysia (OUM)

Copyright © Open University Malaysia (OUM)
 APPENDICES W 183

APPENDIX 1

Copyright © Open University Malaysia (OUM)

 APPENDIX 2

Suggested Scope and Depth of Research

Below are expectations for research for the PhD (Edu) Thesis, the MasterÊs Dissertation (Option 1) and the MasterÊs Project (Option 2)
184 X APPENDICES

Area PhD (Edu) Dissertation M Ed Option 1 Dissertation M Ed Option 2 Master’s Project

(18 credits) (6 credits)

Overview A dissertation is expected to cover a A dissertation is expected to cover a A research report is expected to cover a
particular area of study in depth and to particular area of study in depth and particular area of study and to contribute
contribute to research in the discipline. to contribute to research in the field. to research in the field. A study
Mere description or a study of undertaken for Option 2 may be a small-
perceptions of people in the field is not scale study of a phenomenon or it may
A dissertation should demonstrate encouraged unless it is supported by be a study of perceptions.
evidence of reading seminal work and some qualitative data.
recent contributions to body of research.
The intended study should explore and The emphasis here is more on a
address the existing “gap” in the area of A dissertation should examine student’s understanding of the research
research. contemporary issues in the field by process as well as the awareness of the
presenting evidence of reading seminal limitations of his/her study rather than
The emphasis here is on a student’s
work as well as recent contributions to the in-depth understanding of the
detailed and in-depth understanding of
research in an area. phenomenon investigated or the
the phenomenon investigated.
generalisability of his/her findings.

The emphasis here is on a student’s

understanding of the research process
as well as the awareness of the
limitations of his /her study with some
in-depth understanding of the
phenomenon investigated or the
possible generalisability of his/her

Copyright © Open University Malaysia (OUM)

findings.
 Research Students may use various research Students may use various research Students may use various research
Methodology methods: qualitative, quantitative or methods: qualitative, quantitative or methods: qualitative or quantitative
mixed method mixed method
Students should demonstrate the
Students should demonstrate a high Students should demonstrate a high application of appropriate methodology.
degree of rigour (e.g. justifying use of degree of rigour (e.g. justifying use of
Students may adopt or adapt ready-
appropriate research design) in their appropriate research design) in their
made or established instruments.
work by applying all elements of work by applying all elements of
However, some form of pilot tests to
relevant and appropriate methodology relevant and appropriate methodology
establish their validity and reliability is
Students are required to demonstrate a Students are required to demonstrate expected.
high level of originality their research a high level of originality their research
work. work.

Quantitative If the study employs a quantitative If the study employs a quantitative If the study employs a quantitative
Research approach, it should rest on a sound approach, it should rest on a sound approach, it should be supported by past
theoretical framework and supported by theoretical framework and supported research.
past research. by past research.
For a research report using experiments
For a dissertation using experiments or For a Dissertation using experiments or surveys, the analysis may just focus
surveys, the analysis should include use or surveys, the analysis should include on detailed explanations of descriptive
of inferential statistics, e.g. ANOVA, t- use of inferential statistics, e.g. statistics (means, standard deviation,
tests or chi-square statistics at the very ANOVA, t-tests or chi-square statistics frequency, percentages, etc).
least. Students must explain the at the very least. Students should
However, the use of inferential statistics
appropriateness of the statistical tests explain the appropriateness of the
such as t-test and chi-square analysis is
used and they must only use these tests statistical tests used and they are
highly encouraged. Students should
after providing sufficient evidence for the encouraged to provide evidence for the
show some awareness of their
assumptions underlying each statistical assumptions underlying each statistical
understanding of the statistical
procedure. procedure.
assumptions when inferential statistics is
In situations where these assumptions used.
are not met, appropriate non-parametric
tests should be used and the limitations
of these tests must be discussed.
Apart from reporting the significance of

Copyright © Open University Malaysia (OUM)

the findings, students are also required
to discuss the ‘practical importance’ by
APPENDICES

means of effect size and confidence

intervals whenever appropriate.
W
185
 Qualitative If the study uses the qualitative If the study uses the qualitative Qualitative studies should include “rich
Research approach, the theory building must be approach, the theory building must be thick description” so that understandings
directed by a well developed conceptual directed by a well developed of phenomena in the field are based on
framework. conceptual framework. in-depth analyses of data.

Qualitative studies should include “rich Qualitative studies should include “rich
186 X APPENDICES

thick description” so that understandings thick description” so that

of phenomena in the field are based on understandings of phenomena in the
in-depth analyses of data. field are based on in-depth analyses of
data.
The use of appropriate qualitative data
analysis approaches and techniques The use of appropriate qualitative data
should be clearly explained and justified analysis approaches and techniques
with sound theories and/or past should be clearly explained.
research.

No. of Pages Proposal: 60-75 pages

Proposal: 75 – 150 pages Master’s Project: 80-100 pages
Dissertation: 120-150 pages
Dissertation: 250 - 300 pages (20,000 – 25,000 words)
(30,000 – 40,000 words)
(70,000 – 80,000 words) (Excluding tables, figures, references
(Excluding tables, figures, references
(Excluding tables, figures, references and appendices)
and appendices)
and appendices)

Copyright © Open University Malaysia (OUM)

 MODULE FEEDBACK
MAKLUM BALAS MODUL

If you have any comment or feedback, you are welcome to:

1. E-mail your comment or feedback to modulefeedback@oum.edu.my

OR

2. Fill in the Print Module online evaluation form available on myVLE.

Thank you.

Centre for Instructional Design and Technology

(Pusat Reka Bentuk Pengajaran dan Teknologi)
Tel No.: 03-27732578
Fax No.: 03-26978702

Copyright © Open University Malaysia (OUM)