COURSE MODERATION – M&E

DATE: 15/JAN/2020
AGENDA:

 1. Data Collection Methods.

 2. Data Collection Guidelines.
 3. Data Analysis & Interpretation.
 4. Participants’ Feedback session.

 Stephen Muchami – Course Moderator

INTRODUCTIONS:
SESSION OBJECTIVES

 1. To review the different M&E data collection methods.

 2. To highlight the M&E data collection guidelines for

data quality assurance.

 3. To discuss the approaches of Data Analysis &

Interpretation used in M&E.

 4. To have a participants’ feedback session at the end of

the session.
1. DATA COLLECTION
METHODS:
 INTRODUCTION :

 Monitoring and Evaluation (M&E) process requires selection of the data

collection methods, knowing how to combine them and knowing how to
use them.

 The purpose of data collection, analysis and interpretation is to provide

a factual basis for decision making.

 Data collection, analysis and interpretation is the process of managing

the entire process of data collection, analysis and interpretation and its
use in informing management and planning decisions across stakeholders
and organization levels.

 It entails the following:

 1. Continually assessing whether project/program information needs are

being met.

 2. Managing M&E data collection process and ensuring that data is

captured in time.

 3. Analyzing and using the M&E information in time.

 4. Disseminating the M&E information to ensure its timely used.

QUANTITATIVE AND QUALITATIVE
DATA COLLECTION :

 Data can be classified as quantitative or qualitative.

 Quantitative data are data in numerical form.

 Qualitative data are data in non-numerical form.

 Quantitative data are data that can be precisely measured.

 Examples include data on age, cost, length, height, area,

volume, weight, speed, time and temperature.

 Qualitative data deal with descriptions. They are data that can
be observed, or self-reported, but not necessarily precisely
measured.

 Examples of qualitative data are data on relationships and

behavior.
 QUANTITATIVE AND QUALITATIVE
DATA COLLECTION : EXAMPLE :
Consider the following three descriptions of ‘poverty’. What does
each of them tell you about poverty and how does it do so?
A. Rural poverty in Tanzania has been halved in the period from 1985 to 2001. At present about 38 per
cent of people living in rural areas are classified as poor. This progress is reflected in the United
Nations Development Programme's Human Development Index for Tanzania, which rose from 0.3 in
1991 to 0.4 in 2002.”

B. Photo.

C. “We are cripples; we are afraid of everything; we depend on everyone. No one needs us. We are like
garbage that everyone wants to get rid of.”— a blind woman from the streets of Dareesalaam, Tanzania
QUANTITATIVE AND QUALITATIVE
DATA COLLECTION :
 DATA COLLECTION
METHODS :

 Triangulation – It is collecting the same information using different

methods in order to increase the accuracy of the data.

 Evaluators use method triangulation to strengthen findings.

 The more information gathered using different methods that

supports a finding, the stronger the evidence is.

 There are different triangulation of methods:

 1. Triangulation of sources - Collection of the same information

from a variety of sources in order to increase the accuracy of the
data.

 2. Evaluator triangulation – Collection of the same information

from more than one evaluator in order to increase the accuracy of
an evaluation.
 DATA COLLECTION
METHODS :

 1. Participatory tools

 2. Secondary data

 3. Observation

 4. Surveys and interviews

 5. Focus groups

 6. Diaries or journals and self-reported checklists

 7. Expert judgment

 8. Delphi technique

 9. Measurement tools – Scales, Stop watches, Clinical tools,

Chemical tests tools (water/soil quality), Aptitude tests etc.
 DATA COLLECTION
METHODS :
 1. Participatory tools:  1.3 Transect walks – It is a type of walk an
evaluator might take around a community in
 1.1 Community meetings - These meetings order to obtain a representative observation of
allow members of the community to ask its people, surroundings, and resources.
questions, make comments and discuss issues
of importance to the community.  Transect walks are a kind of spatial data
gathering tool.
 1.2 Mapping - The drawing of or use of existing
maps and is used to generate discussions about  Instead of using aerial images, a walker uses his
local development priorities and the extent to or her observation skills to develop an
which they have been met. understanding of the area.

 Mapping can also capture changes or perceived  A transect walk is planned by drawing a
changes over time (before and after an “transect line” through a map of a community.
intervention).
 The evaluator, accompanied by several
 Types of mapping: resource mapping, historical community members, walks along the area
mapping, social mapping, health mapping, represented by the transect line on the map.
wealth mapping, land use mapping and
demographic mapping.  He or she talks to the community members
while observing conditions, people, problems
and opportunities.
 DATA COLLECTION
METHODS :
 2. Analysis of records and secondary analysis:

 Sometimes data that have already been collected can be used to answer evaluation questions.

 When using data sets gathered by others, it is necessary to understand how the data were
collected, how variables were defined and how the data were coded and cleaned, including
how missing data, nonresponses and low response rates were treated.

 Secondary data sources include not only data sets from prior data collection activities but
also: newspaper articles, journal articles, papers, web pages, blogs, internet discussion
groups etc.

 3. Observation can be a powerful data collection tool. By just using one’s eyes, one can
observe, for example, traffic patterns, land use patterns, the layout of city and rural
environments, the quality of housing, the condition of roads or who comes to a rural health
clinic for medical services. Observation types:

 1. Structured observation – Has a Checklist and Observation Plan.

 2. Unstructured observation - No Checklist and no Observation Plan.

 3. Semi-structured observation - Has a Checklist but no Plan.

 DATA COLLECTION
METHODS :

 4.1 Surveys:

 They are excellent tools for collecting data about people’s

perceptions, opinions, and ideas.

 They are less useful in measuring behavior, because what people

say they do may not reflect what they actually do.

 Surveys can be:

 Structured or semi-structured

 Administered in person or by telephone

 Self-administered by having people respond to a mailed

tool/web form

 Poll a sample of the population or all of the population.

 DATA COLLECTION
METHODS :

 Developing
survey/questionnaire
questions involves:

 1. Choosing the
questions.

 2. Wording the questions

and responses.

 3. Sequencing the
questions.

 4. Choosing a layout for

the survey.

 5. Reviewing, translating,
and pretesting the
questions.
 DATA COLLECTION
METHODS :
 4.2 Interviews:

 An interview is a dialogue between a skilled interviewer and the person

being interviewed.

 The goal is to elicit information that can be used to answer evaluation

questions.

 The quality of the information obtained depends largely on the

interviewer’s skill and personality.

 The key to a good interview is being a good listener and good questioner.

 5. Focus Group:

 It is a qualitative evaluation methodology in which small groups of people

are brought together to discuss specific topics under the guidance of a
moderator.

 A focus group generally includes 6–12 people.

 The composition of the group depends upon its purpose.

 DATA COLLECTION
METHODS :
 6.1 Diaries or Journals:

 A diary (or journal) is a written self-report by a participant, usually

maintained daily.

 Diaries are used to capture detailed information about events in

people’s daily lives.

 They can be used to study such phenomena as: social networks, diet
and nutrition, farm work, study habits, contraception use etc.

 6.2 A self-reported Checklist:

 It is a cross between a questionnaire and a diary.

 Participants are asked to keep track of a specific set of activities or

events, which are listed, so that the respondents can easily check
them off.

 Checklists can be completed on a daily or weekly basis or every time

a particular event or activity occurs.
 DATA COLLECTION
METHODS :
 7. Expert Judgment

 Sometimes it makes sense to engage experts as the source of information or opinion.

 It can be obtained from an individual expert or a panel of experts.

 Government task forces are a form of expert judgment.

 8. Delphi technique

 It enables experts who live in different locations to engage in dialogue and reach
consensus through an iterative process.

 Experts are asked specific questions; their answers are sent to a central source,
summarized, and fed back to the experts.

 The experts then comment on the summary.

 They are free to challenge particular points of view or to add new perspectives by
providing additional information.

 Because no one knows who said what, conflict is avoided.

2. DATA COLLECTION
GUIDELINES:
DATA COLLECTION
GUIDELINES :

 1. Data Capture:  3. Querying

 It is the process of  4. Imputation (process of

converting data from forms replacing missing data with
into a format that can be substituted values)
interpreted and analyzed. It
is also the process of  5. Coding
collecting and entering of
data into a computer, or the  6. Conversion
conversion of data into a
form compatible with  7. Verification
computers.
 8. Validation
 The key steps to data
capture include:

 1. Receipt of forms

 2. Editing
 DATA COLLECTION
GUIDELINES :
 2. Data Preservation:

 Data preservation or digital data preservation refers to the series of

managed activities necessary to ensure continued access to digital
materials for as long as necessary.

 It refers to all of the actions required to maintain access to digital

materials beyond the limits of media failure or technological change.

 A sustainable preservation program should address:

 1. Organizational infrastructure – These includes the policies,

procedures, practices and people being the basic elements of data
preservation requirements.

 2. Technological concerns – This deals with the IT architecture of the

organization i.e. requisite equipment, software, hardware, skills, secure
environment and an updated media monitoring strategy.

 3. Data curation – This refers to the active management of data through

its life cycle of interest and usefulness to a designated community.
 DATA COLLECTION
GUIDELINES :

 3. Data Quality Management/Assurance:

 It is key to ensuring that ongoing generation of useful and

reliable information on the core indicators of the program.

 Most preferably, the data quality assurance should be

undertaken at different levels: field level, then at the regional
level and finally at the national level.

 Other terminologies used include:

 1. Data Quality Control.

 2. Information Quality.

 3. Data Quality Assurance (DQA).

 4. Data Content Quality.

 DATA COLLECTION
GUIDELINES :
 Data Quality Assurance (DQA) - It refers to the reliability and effectiveness of data.

 Maintaining data quality requires going through the data periodically and cleaning it.

 The core principles that must be observed in DQA include: accuracy, consistency,
completeness and timeliness. See example of DQA Protocol below:
3. DATA ANALYSIS &
INTERPRETATION:
 DATA ANALYSIS &
INTERPRETATION :
 M&E data that you have collected will not mean much to you and your
stakeholders in its raw form.

 M&E data only becomes useful when you give it meaning and this is
done through analysis, visualization and interpretation:

 1. Data Analysis – It is the process of bringing order and structure to

the collected data. It turns individual pieces of data into information
you can use. This is accomplished by applying systematic methods to
understanding the data: looking for trends, groupings or other
statistical relationships between different types of data.

 2. Data Visualization – It is the process of putting data into a chart,

graph or other visual format that helps inform analysis. Data
visualization also helps you interpret and communicate your results.

 3. Data Interpretation – It is the process of attaching meaning to data.

Interpretation requires reaching conclusions about generalization,
correlation and causation, and is intended to answer key learning
questions about your project.
 DATA ANALYSIS :
 Data analysis depends on the type of data:

 1. Quantitative Data:

 These are analyzed using quantitative, statistical methods and

computer packages such as Microsoft Excel or SPSS.

 The results of quantitative data analysis are numerical and easily

visualized using a graph, chart or map.

 2. Qualitative Data:

 It is most often done by reading through qualitative data in the form

of data transcripts, such as notes from focus group discussions or
interviews, to identify themes that emerge from the data.

 This process is called content analysis or thematic analysis.

 It can be aided by software, but is most often done using paper, pens
and sticky notes.
DATA VISUALIZATION :
 DATA INTERPRETATION :
 There are several recommended practices for improving your data
interpretation:

 1. Creating visualizations of your results to help people better

understand and interpret your data, making sure your visualizations are
used to give the full picture of the data and are not misleading.

 2. Triangulating your data by presenting the results of both quantitative

and qualitative analysis together so that you can compare the results.

 3. Convening a stakeholder meeting to interpret the data. This meeting

should involve stakeholders with different perspectives on the project.

 4. Planning an adequate amount of time to analyze and interpret data.

The interpretation process takes time.

 5. Making sure roles and responsibilities around interpretation are clear.

Usually, the MEAL team does the initial analysis while project staff
organize and facilitate interpretation events.
PARTICIPANTS ’ FEEDBACK
SESSION
END !