Strategic Review of Child Health

Framework and strategy for integrated monitoring

and evaluation of child health programmes for

BMJ: first published as 10.1136/bmj.k2785 on 30 July 2018. Downloaded from http://www.bmj.com/ on 11 December 2018 by guest. Protected by copyright.
responsive programming, accountability, and impact
Stronger governance and leadership is needed to implement and maintain integrated reporting

M
systems, say Theresa Diaz and colleagues
onitoring and evaluation child mortality and improve child health systems. These non-integrated systems
of health systems, pro- in countries with a mortality rate in the for collection of routine data are in place
grammes, and interven- under 5s of >40/1000. It tackles multiple because each vertical programme, glob-
tions is critical to assess child illnesses through integrated delivery ally and nationally, is funded, governed,
progress, identify problems, of preventive and curative services.5 More and managed separately. In addition, IMCI
and facilitate change to improve service than 100 countries have adopted the IMCI monitoring and evaluation uses data from
delivery and reach the desired outcomes.1-3 strategy for their health programmes and other sources. For example, data on exclu-
Funders increasingly demand monitoring systems. Its three components—improving sive breast feeding, use of bed nets, and
and evaluation so that they can determine health worker skills, strengthening health very often mortality data require house-
whether a programme achieves its intended systems, and improving family and hold population based surveys. Informa-
outcomes. community practices—have been adopted tion on quality of care, experience of care,
As the recognition and importance of to a varying extent.6 However, there have and availability of commodities requires
monitoring and evaluation has grown so been challenges to creating and sustaining health facility surveys. Pulling these mul-
has the push towards integration of health a monitoring and evaluation system to tiple data sources together and conducting
services. For the sustainable development support the programme. integrated analysis is complex and thus
goals adopted by countries at the United Here we describe the challenges created infrequently carried out.
Nations in 2015, the global health policy by the fragmentation of the current
pendulum is swinging towards the need for monitoring and evaluation systems and Lack of resources for integration
“integration” of health service delivery to suggest solutions. We have used data Efforts to support routine monitoring and
achieve universal health coverage (which is mainly from the IMCI implementation evaluation systems have typically focused
one of the sustainable development health survey, 6 interviews with key informants on a single programme, usually owing to
targets).4 However, less attention has been and in-depth country assessments carried availability of resources. For example, in
paid to the consequent need for monitoring out as part of the 2016 strategic review of Malawi 42 types of data tools were in use.
and evaluation systems to be integrated, IMCI.7 Each technical programme also collects
through design and implementation that its own indicators, and data are stored
allows the measurement and analysis of Why monitoring and evaluation are not in disparate systems (H Nsona, personal
multiple outputs and outcomes in a single integrated communication, 2017). In Nigeria, the
report. Between 1997 and 1999 an attempt was well funded polio eradication programme
An integrated strategy for management made to foster integration and harmonise has been given political priority, as have
of childhood illness has been in place for indicators. An interagency working group routine immunisation data. Both of these
20 years. The Integrated Management of on IMCI monitoring and evaluation recom- were allocated a module on the health
Childhood Illness (IMCI) was developed mended a standard set of indicators for use management information systems plat-
by the World Health Organization and of IMCI at first level health facilities and in form (other programmes were not (E Meri-
Unicef in 1995. Its aim was to reduce the community.8 This set of 24 indicators bole, personal communication, 2017)).
included health system indicators as well Multiple single disease reporting systems
as nutrition, prevention (immunisation often result in an individual healthcare
Key messages and bed net use to prevent malaria), and worker reporting multiple programmes
• Integration of health services often caretaker satisfaction. Further attempts to using multiple forms. This can be cum-
neglects monitoring and evaluation bring about integration included a guide bersome, duplicative, time consuming
systems, with negative consequences for for monitoring and evaluation of child and lead to poor quality of data. 10 11 Key
both health workers and service delivery health programmes in 2005. 9 However, informants felt that monitoring and evalu-
• Global partners need to fully support these recommendations failed to deal with ation should avoid an approach in which a
single child health national reporting the reality. Health management informa- global programme dictates what data need
systems tion systems (HMIS) are data collection to be collected. An academic researcher in
systems specifically designed to support a high income country suggested that such
• More investment is needed in staff to
collect, report and analyse data planning, management, and decision mak- an approach was unhelpful and “. . . more
ing. Often immunisation, HIV, malaria, often than not leads to game playing and
• Countries must develop standardised
and nutrition use different information data tampering.” It was suggested that
forms and indicators across health
systems. Similarly, civil registration and the role of WHO should be to encourage
programmes
vital statistics systems and child death country offices to track a short list of key
• New technologies can improve audits, which track births, deaths, and indicators.
collection analysis and use of child potential causes of death, use separate Global and national investment into
health data reporting forms and electronic reporting health monitoring and evaluation systems

the bmj | BMJ 2018;362:k2785 | doi: 10.1136/bmj.k2785 1

Strategic Review of Child Health

has been insufficient, with a notable lack reporting systems, and, most importantly, levels to better use technology and support
of training, supervision, and funding to align with data that are useful and build the ability of staff to collect and
for officers. This absence of investment and needed locally to improve use data.

BMJ: first published as 10.1136/bmj.k2785 on 30 July 2018. Downloaded from http://www.bmj.com/ on 11 December 2018 by guest. Protected by copyright.
can result in a reduced ability to report programmes.14 15 Some progress in alignment of global
findings owing to a lack of supervision initiatives has been made. In 2015, WHO
and insufficient tools to capture data. Weak national leadership and governance collaborated with 19 international and
This was the case in India, where reporting Strong country leadership and clear govern- multilateral partners and countries to
was hampered by poor availability ance are necessary to ensure that standard- develop and agree on a global reference
of forms and registers. Indicators to ised key indicators are available in health list of 100 core health indicators. 15 The
monitor programme performance were management information systems and list reflects indicators of relevance for
not included in the health management collected through surveys when needed. country, regional, and global reporting
information systems, resulting in an Globally, according to the IMCI implemen- across the full spectrum of global health
adverse effect on the flow of information tation survey, monitoring has been rela- priorities, including IMCI. Building on the
at district and state level. tively infrequent and inconsistent.6 Only work of the 100 core health indicators,
The limited capacity to analyse data 33% of countries (30/91) reported having a the Health Data Collaborative was created
from multiple sources and use the findings comprehensive IMCI monitoring and evalu- to provide a platform for countries,
hampers the provision of information ation plan. Although more than two thirds partners, and donors for better alignment
that might improve the efficiency and of countries (70%; 66/94) reported that of funding and technical support for health
quality of health service delivery. All these their health management information sys- monitoring and evaluation systems.16 The
factors can lead to healthcare workers not tems included some monitoring indicators aim is to contribute to reduced reporting
recording data collected and not trusting for IMCI, only about one third of countries requirements and to align investments
the data captured. As one academic (34%; 32/95) in the past 5 years had con- in one common country monitoring and
researcher based in a low income country ducted a health facility survey collecting evaluation system.
suggests, “workers don’t have time to data relevant to IMCI. Responsibility for introducing a national
input the data and nobody trusts the S ever al countries in this review integrated child health monitoring and
data, so nobody uses them and, in turn, highlighted the lack of national leadership evaluation system must reside in national
healthcare workers care even less about and governance. India reported that IMCI institutions with overall stewardship
the data.” indicators are not available in health of monitoring and evaluation. These
management information systems. Kenya institutions are generally ministries of
Global initiatives support parallel monitoring has no legal framework to enforce the use health, which must institute and enforce
and evaluation of standardised data tools or reporting (I integrated data collection using one set
Child health programmes entail not only Maina, personal communication, 2017). of shared indicators and one child health
treatment of major causes of mortality such In Nigeria, separate funding sources, reporting form for health management
as pneumonia, malaria, and diarrhoea but weak leadership and enforcement, and information systems, and supporting
also preventive services such as immuni- lack of ownership have led to vertical integrated analysis.17-19 Box 1 provides some
sations, provision of nutritional services, alignment within child health monitoring examples of improved national governance
and prevention and treatment of HIV and and evaluation programming (E Meribole, and leadership.
tuberculosis (TB). Surveillance, monitor- personal communication, 2017). Similarly, Technology needs to be better used
ing, and evaluation of malaria, HIV, TB, in Malawi each technical programme globally, nationally, and subnationally
nutrition, and immunisation are usu- collects its own indicators (H Nsona, to improve the collection, analysis,
ally done through separate donor funded personal communication, 2017). and use of data. We found innovative
programmes, with global disease specific technological solutions being employed
guidance and reporting forms and single Achieving integration in some countries. These included web
disease monitoring systems. This has fur- Future major goals are to reduce the burden based reporting systems in Bangladesh,
ther increased fragmentation as shown on frontline health workers and improve the Kenya, and Nigeria and real time data
by uncoordinated parallel data collection quality and use of data. This requires global collection in the Democratic Republic
systems, analysis of multiple data sources action to synergise investments and require- of Congo using healthcare workers and
focusing on only one disease, and the lack ments for reporting systems, national action local community volunteers (box 1),
of resources provided to other non-donor to improve leadership and governance by Globally, we suggest that an increased
funded monitoring and evaluation pro- establishing and requiring one reporting use of mobile health technologies,
grammes. system for child health, and action at all adoption of national standards for health
More recently, in alignment with
the sustainable development goals,
a set of indicators and a monitoring
and evaluation framework have been Box 1: Innovative national solutions to improving monitoring and evaluation
developed specifically for maternal,
• Bangladesh: One major innovation used here is eIMCI, a web based management system
newborn, and child health, 12 but large
for the Integrated Management of Childhood Illness (IMCI). Some key informants suggested
gaps in data remain. 13 In part, this may
that IMCI paved the way for improvement of health management information systems.
be because these indicator frameworks
“Web based management information systems were initially started for IMCI and are now
are intended for global reporting. Some
used for everything.”
guidance is available for maternal,
newborn, and child health programmes • Democratic Republic of Congo: The country has a monitoring system, known as “improved
at subnational, facility, and community monitoring for action.” This is used to track progress in effective coverage of maternal,
levels. 9 14 However, this needs to be newborn and child health interventions. Monitoring is carried out by health workers helped
updated to reflect new technologies, to by volunteers from the communities being served. Efforts are now focusing on developing
better integrate facility and community real time data collection and centralised transmission.

2 doi: 10.1136/bmj.k2785 | BMJ 2018;362:k2785 | the bmj

 INPUT / PROCESS OUTPUTS OUTCOMES IMPACTS
IMPACTS
INDICATORS SELECTED TRACERS FOR SERVICE COVERAGE OF INTERVENTIONS LONG TERM OUTCOMES AND EFFECTS *
AVAILABILITY AND READINESS AND RISK FACTORS
Health workforce/infrastructure
• No of IMCI trained health workers per 1000 (INCLUDING COMMODITIES)
population
• No of paediatric hospital beds per 10 000 • Essential newborn care • Postnatal care coverage • Neonatal mortality rate
population Neonatal • Newborn weighed • Early initiation of breastfeeding
• No of health facilities with paediatric essential
medicines and life saving commodities
Health information • Immunisation • Exclusive breastfeeding rate 0-5 • Under 5 mortality rate
Child • Preventive care months of age • Infant mortality rate
• HMIS with IMCI indicators

the bmj | BMJ 2018;362:k2785 | doi: 10.1136/bmj.k2785

• Birth registration coverage • Nutrition screening • Care seeking for pneumonia • Prevalence of stunting under 5
• Death registration coverage • Curative care, including lifesaving • Children with diarrhoea receiving ORS • Prevalence of wasting under 5
• One (1) M&E system for child health commodities • Vitamin A supplementation coverage • Malaria, pneumonia,‡ diarrhoea‡
• Participation rate in organised • Immunisation rate (by vaccine)
Governance incidence rate under 5
• Policies on data (eg, access and use) learning‡
• Use of insecticide treated nets • Childhood disease specific mortality
• National steering committee (for child health rates
• Children under 5 who are
and M&E)
developmentally on track‡
• Partnerships
• Laws on mandatory education, birth • Children aged 6-23 months receive
and death registration a minimum acceptable diet

Administrative sources Quarterly†: Interoperable clinical reporting systems (HMIS, DHIS2, EMR, Lab IS, Medicine IS)
SOURCES: (human resources, REPORTING: Annual: Newborn and child death audits Civil vital registration
infrastructure, financial,
medicines, policy, etc) 2-5 years: Facility assessments Population based surveys

Compile, analyse, report, and disseminate results for review and action:
National : Annual review of data with MOH national steering committee (all indicators and data sources)
REVIEW and ACT
Review data at all levels and various Regional : Semiannual review with regional and district health management teams
time periods to support managerial
decisions and actions
District : Quarterly review with district and health facility management including community participation

* See WHO Global Reference List of 100 Core Health Indicators for additional indicator details. † Reporting from interoperable clinical systems can be more frequent than quarterly.
‡ Not currently in WHO core 100 indicator list but recommended for inclusion.
Fig 1 | Monitoring and evaluation framework for integrated management of newborn and child illness. DHIS2=District Health Information System 2; EMR=electronic medical record; HMIS=health management information
systems; IMCI=integrated management of childhood illness; Lab IS=Laboratory Information System; M&E=monitoring and evaluation; Medicine IS=Medicine Information System; MOH, Ministry of Health; ORS=oral
rehydration solution

3
Strategic Review of Child Health

BMJ: first published as 10.1136/bmj.k2785 on 30 July 2018. Downloaded from http://www.bmj.com/ on 11 December 2018 by guest. Protected by copyright.
Strategic Review of Child Health

management information systems (eg, conclusions with inputs from EM, IM, HN, KMA, BN 8 World Health Organization. IMCI indicators,
and KPO’N. BM created fig 1. All authors contributed monitoring and evaluation. Integrated Management
District Health Information System 2),20-23
intellectual content, edited the manuscript and of Childhood Illness (IMCI). 1999. http://apps.who.
as well as interoperability and integration approved the final version for submission. int/iris/bitstream/10665/65002/12/WHO_CHS_

BMJ: first published as 10.1136/bmj.k2785 on 30 July 2018. Downloaded from http://www.bmj.com/ on 11 December 2018 by guest. Protected by copyright.
of electronic reporting systems must be in CAH_98.1K_eng.pdf
Competing interests: We have read and understood 9 Gage A, Ali D, Suzuki C. A guide for monitoring
place.24 BMJ policy on declaration of interests and have no and evaluating child health programs. 2005.
Finally, some of the approaches used relevant interests to declare. The views expressed in https://www.measureevaluation.org/resources/
for monitoring a specific disease may be this article do not necessarily represent the views, publications/ms-05-15
decisions, or policies of the institutions to which the 10 Phalkey RK, Yamamoto S, Awate P, Marx M. Challenges
adapted for integrated monitoring and authors are affiliated. with the implementation of an Integrated Disease
evaluation. Specifically, methods should Surveillance and Response (IDSR) system: systematic
Provenance and peer review: Commissioned; not
be adopted to improve the ability of externally peer reviewed. review of the lessons learned. Health Policy
health workers to perform monitoring and This article is part of a series based on findings from a
Plan 2015;30:131-43. doi:10.1093/heapol/czt097
11 Lukwago L, Nanyunja M, Ndayimirije N. et al. The
evaluation, such as regional workshops global review of Integrated Management of Childhood
implementation of Integrated Disease Surveillance
supplemented with online courses and Illness (IMCI) funded by the Bill and Melinda Gates
and Response in Uganda: a review of progress
Foundation. Open access fees were funded by the
participatory training.25 26 and challenges between 2001 and 2007. Health
Health Systems Research Unit, South African Medical
We propose a strategy for monitoring and Policy Plan 2013;28:30-40. doi:10.1093/heapol/
Research Council. The BMJ peer reviewed, edited,
czs022
evaluation of child health as shown in fig 1. and made the decision to publish the article with no
12 Unicef. Indicator and monitoring framework for
We emphasise the need for a small set involvement from the Medical Research Council.
the global strategy for women’s, children’s and
of key inputs, outputs, and outcomes Theresa Diaz, coordinator1 adolescent’s health (2016-2030). 2016. https://
Kumanan Rasanathan, chief2 data.unicef.org/resources/indicator-monitoring-
and data sources for collecting common framework-global-strategy-womens-childrens-
measurements and having the systems in Emmanuel Meribole, director3
adolescents-health/
place to collect such data. These include Isabella Maina, head4 13 Every Woman Every Child. Country data, universal
administrative data, interoperable health Humphreys Nsona, programme manager5 accountability. monitoring priorities for the global
strategy for women’s, children’s and adolescent’s
management information systems, facility Kyaw Myint Aung, chief of health6
health (2016-2030). 2016. http://www.who.int/
and population based surveys, and fully Bennett Nemser, senior consultant1 life-course/partners/global-strategy/gs-monitoring-
functional civil registration and vital Kathryn Patricia O’Neill, coordinator7 readiness-report.pdf
14 Maternal and Child Health Integrated Program
statistics supplemented by death audits— 1
Epidemiology and Monitoring and Evaluation Team,
(MCHIP). Indicator guide: monitoring and evaluating
in particular, audits of newborn and child Maternal Newborn and Child Adolescent Health
integrated community case management. 2013.
Department, World Health Organization, Geneva,
death. Using multiple data sources to Switzerland
http://1rqxbs47ujl4rdy6q3nzf554.wpengine.
netdna-cdn.com/wp-content/uploads/2016/07/
analyse and act on findings is a complex 2
Knowledge Management and Implementation iCCM-Indicator-Guide.pdf
process. We suggest increased use of Research Unit, Health Section, Unicef, New York, USA 15 World Health Organization. 2018 Global reference
electronic systems to link data sources 3
Monitoring and Evaluation, Nigeria Ministry of Health, list of 100 core health indicators. 2018. http://www.
and generate automatic reports and Abuja, Nigeria who.int/healthinfo/indicators/2018/en/
4 16 Health Data Collaborative. Better data. Better health.
visualisations (eg, score cards) as well as Health Sector Monitoring and Evaluation Unit-
2015. https://www.healthdatacollaborative.org/
MOH;,Nairobi, Kenya
regular workshops that bring stakeholders 5
what-we-do/
Integrated Management of Childhood Illnesses,
together to review data analysis and plan 17 Atun R, de Jongh T, Secci F, Ohiri K, Adeyi O.
Ministry of Health, Malawi Integration of targeted health interventions into
for action. 6
health systems: a conceptual framework for analysis.
Unicef, Malawi Lilongwe, Malawi
7 Health Policy Plan 2010;25:104-11. doi:10.1093/
Global Platform on Measurement and Accountability,
Conclusion Department of Information, Evidence and Research, heapol/czp055
18 Kasolo F, Yoti Z, Bakyaita N, et al. IDSR as a
Integration of health services is key to Health Systems and Innovation, World Health
platform for implementing IHR in African countries.
universal health coverage and an effec- Organization, Geneva, Switzerland
Biosecur Bioterror 2013;11:163-9. doi:10.1089/
tive health system and makes the most of Correspondence to: T Diaz bsp.2013.0032
tdiaz@who.int 19 Reynolds HWSE, Sutherland EG. A systematic
human and other resources. Experience
1 Frieden TR. Six components necessary for effective approach to the planning, implementation,
with IMCI shows that the integration of public health program implementation. Am J monitoring and evaluation of integrated health
monitoring and evaluation systems is Public Health 2014;104:17-22. doi:10.2105/ services. BMC Health Serv Res 2013;13:168.
often neglected. The global context is once AJPH.2013.301608 doi:10.1186/1472-6963-13-168
2 Frankel N, Gage A. M&E fundamentals: a self-guided 20 Kizito K, Adeline K, Jean Baptiste K, editors, et al.
more favourable for integration, including
mini-course. 2016. Measure evaluation. MS-07-20. TRACnet: a national phone-based and web-based
of monitoring and evaluation, particularly https://www.globalhealthlearning.org/course/m-e- tool for the timely integrated disease surveillance
for child health. Countries now need to fundamentals and response in Rwanda. Online J Public Health
include monitoring and evaluation to pro- 3 Peersman G, Rugg D. Basic terminology and Inform 2013;5:e202. doi:10.5210/ojphi.
frameworks for monitoring and evaluation. v5i1.4598
duce a single health management informa- 2004. http://www.unaids.org/sites/default/files/ 21 Guyon A, Bock A, Buback L, Knittel B. Mobile-based
tion system, with a single set of indicators sub_landing/files/7_1-Basic-Terminology-and- nutrition and child health monitoring to inform
and training and which supports a team Frameworks-MEF.pdf program development: an experience from Liberia.
4 United Nations. The sustainable development Glob Health Sci Pract 2016;4:661-70. doi:10.9745/
of data specialists. This would facilitate GHSP-D-16-00189
goals report. 2016. https://unstats.un.org/sdgs/
the reporting, analysis, and use of data for report/2016/ 22 Negandhi P, Chauhan M, Das AM, Sharma
decision making. Global partners need to 5 Gove SThe WHO Working Group on Guidelines J, Neogi S, Sethy G. Computer tablet-based
for Integrated Management of the Sick Child. health technology for strengthening maternal
support these efforts and fully accept single
Integrated management of childhood illness by and child tracking in Bihar. Indian J Public
national reporting systems. These princi- outpatient health workers: technical basis and Health 2016;60:329-33. doi:10.4103/0019-
ples should be incorporated into the rede- overview. Bull World Health Organ 1997;75 557X.195868
sign of IMCI programmes. (Suppl 1):7-24. 23 Kiberu VM, Matofu JK, Makumbi F, Kyozira C,
6 Dalgish SL. Methods for the strategic review of IMCI Mukooyo E, Wanyenze RK. Strengthening
Contributors and sources: This article is based and iCCM programmes. BMJ 2018;362:k2989. district-based health reporting through the
on findings from a global review of Integrated doi:10.1136/bmj.k2989 district health management information
Management of Childhood Illness (IMCI) commissioned 7 Boschi-Pinto C, Labadie G, Ramachandran DT, software system: the Ugandan experience.
by the WHO and funded by the Bill and Melinda Gates et al, Global implementation of the integrated BMC Med Inform Decis Mak 2014;14:40.
Foundation and three additional case studies. TD management of childhood illness—twenty years doi:10.1186/1472-6947-14-40.
conceived the outline of this paper and drafted the on.BMJ Open 2018:8:e01907. doi:10.1135:bmj- 24 Kariuki JM, Manders EJ, Richards J, et al. Automating
first version, KR revised the paper and drafted the open.2017-01907 indicator data reporting from health facility EMR

4 doi: 10.1136/bmj.k2785 | BMJ 2018;362:k2785 | the bmj

 Strategic Review of Child Health

to a national aggregate data system in Kenya: of malaria control programmes to streamline programmes. Glob J Health Sci 2014;7:192-202.
an interoperability field-test using OpenMRS and M&E systems and enhance information use in doi:10.5539/gjhs.v7n2p192
DHIS2. Online J Public Health Inform 2016;8:e188. malaria endemic countries. Malar J 2016;15:300.
doi:10.5210/ojphi.v8i2.6722 doi:10.1186/s12936-016-1354-y

BMJ: first published as 10.1136/bmj.k2785 on 30 July 2018. Downloaded from http://www.bmj.com/ on 11 December 2018 by guest. Protected by copyright.
25 Garley A, Eckert E, Sie A, et al. Strengthening 26 Bell JS, Marais D. Participatory training in monitoring Cite this as: BMJ 2018;362:k2785
individual capacity in monitoring and evaluation and evaluation for maternal and newborn health http://dx.doi.org/10.1136/bmj.k2785

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits
others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://
creativecommons.org/licenses/by/4.0/.

the bmj | BMJ 2018;362:k2785 | doi: 10.1136/bmj.k2785 5