HANDBOOK ON

GOVERNANCE
STATISTICS
Design and layout by Phoenix Design Aid A/S.
HANDBOOK ON
GOVERNANCE
STATISTICS

Empowered lives.
Resilient nations.
iv |
EXECUTIVE SUMMARY

Governance statistics are fundamental to ensuring that the relationship between the State and
its peoples is inclusive, transparent and accountable. They assist in monitoring the performance of
government and in better understanding the contribution of governance in its various dimensions
to development. Governance statistics contribute to the measurement and realization of human
rights. They can help to identify groups or sub-groups in the population that are most affected
by the dysfunctions of governance systems, with a view to putting in place appropriately targeted
policies, as pledged in the 2030 Agenda “Leaving no one behind”. Governance statistics can also
contribute to preventing and managing conflict and violence, when used as early warning sys-
tems, and they can help foster peace, by periodically informing on State-society relations, which
lie at the centre of sustainable peace.

This Handbook was developed under the auspices of the Praia Group on Governance Statistics,
which was established in 2015 by the UN Statistical Commission and mandated to develop “a
handbook on governance statistics for national statistical offices”. It is a collaborative effort of
national statistical offices, international organizations and civil society organizations. The purpose
of this Handbook is to provide a foundation for the development of international statistical
guidance and standards in all areas of governance statistics. To this end, it conceptualizes and
systematizes the various dimensions of governance statistics, takes stock of and qualifies existing
practices in governance data collection, highlights the most important metrics in each dimension
of governance statistics, and discusses the way forward required to achieve international statistical
standards in each dimension. It is intended as a guide for those wishing to understand, produce
and analyse governance statistics, and is primarily targeted towards national statistical agencies.
Whilst this Handbook aims to contribute to the development of international standards on gov-
ernance statistics, its purpose is not to promulgate such standards. Likewise, the key indicators
recommended in this Handbook do not constitute or imply a reporting obligation for national
statistical systems.

This Handbook is based on a framework for governance statistics that includes eight dimen-
sions of governance:

1) N
 on-discrimination and equality: This dimension focuses on any distinction, exclusion,
restriction or preference or other differential treatment based on grounds such as colour,
sex, language, religion, national or social origin, disability or other status that has the in-
tention or effect of nullifying or impairing human rights and fundamental freedoms.
2) P
 articipation: This dimension focuses on the ways in which individuals take part in the
conduct of political and public affairs, including by registering to vote, voting or standing
as a candidate in elections; being members of legislative, executive and judicial bodies
at all levels of government; accessing positions in the public service; and engaging, indi-
vidually or as members of political parties and other non-governmental organizations, in
political activities.

HANDBOOK ON GOVERNANCE STATISTICS | v

 3) Openness: This dimension focuses on the extent to which public institutions provide ac-
cess to information and are transparent in their decision- and policy-making processes.
More specifically, the dimension covers access to information, open government provi-
sions, freedom of expression and media pluralism.
4) A ccess to and quality of justice: This dimension focuses on the ability of people to
defend and enforce their rights and obtain just resolution of justiciable problems — if
necessary, through impartial formal or informal institutions of justice and with appropriate
legal support.
5) R esponsiveness: This chapter focuses on whether people have a say in what government
does and whether they are satisfied with the government’s performance.
6) A bsence of corruption: This chapter focuses on levels of intolerance to corruption; the
levels and patterns of observable corrupt practices; and the State response to corruption.
7) T rust: This dimension focuses on people’s trust in institutions as well as in other people,
with a primary focus on the former, e.g. the parliament, the national government and the
justice system.
8) S afety and security: This dimension focuses on levels and patterns of crime, perceptions
of safety, measurement of casualties directly provoked by armed operations, and the qual-
ity of law enforcement and criminal justice institutions.

The Handbook has two main parts. Part A discusses cross-cutting issues that are applicable
to all or most dimensions of governance statistics. It provides an overarching framework with
common analytical underpinnings informed by international human rights norms and principles,
which are applicable to all governance dimensions chapters. Part B examines in detail each of the
eight dimensions of governance, including a conceptualization of the dimension, a discussion of
its relevance, a presentation of data and best practices that currently exist, recommended key
indicators (including but not limited to SDG indicators), and a suggested way forward to achieve
international statistical standards in the respective dimension.

The Handbook provides key insights about the current state of play and a call for action on
the way forward in the field of governance statistics, in particular with regard to: methodological
development; data production and use; engaging new actors and exploring new data sources.

In terms of methodological development across the eight dimensions, the Handbook shows
that only a limited number of statistical standards and international technical guidelines/recom-
mendations and tools exist at this point in time – notable examples being a number of manuals
and an international classification in the area of crime and victimization statistics, a manual on
corruption surveys, a guide for legal needs surveys, a guideline on measuring trust, a set of tech-
nical documents for harmonizing governance statistics in Africa, guidance material on a human
rights based approach to data, and the various metadata documents that exist for SDG indicators
falling into the remit of this Handbook. In consequence, statistical methodologies can be con-
sidered to be more advanced for the measurement of bribery, crime prevalence/victimization,
access to criminal justice, system responsiveness (i.e. political efficacy) and trust in institutions. In
contrast, substantial further methodological work and/or harmonization is required in the areas
of measuring discrimination experiences, participation in political and public affairs, openness,
access to civil justice, satisfaction with services, “grand” corruption, nepotism, and other forms of
corruption. Further methodological work in the above-mentioned thematic areas is encouraged
both in the realm of surveys and administrative data systems. Various chapters emphasize the
potential benefits of consolidating governance-themed survey modules in an integrated way, and

vi |
the limited usability of the many composite indices that exist in the field of governance for gener-
ating actionable insights.

When it comes to data production and use, the Handbook showcases that governance statis-
tics are already a matter of reality in a variety of countries and – to a varying extent – for all eight
dimensions. The current state of play closely mirrors that of methodological development dis-
cussed above. In areas where more advanced guidance exists, there tends to be more widespread
data collection and use, and in more comparable/harmonized ways. As a general observation,
governance statistics – perhaps more often than other, more established fields of statistics – are
not yet routinely or comprehensively produced at the national level and, all too often, are left to
international and/or non-public institutions. For some dimensions of governance, considerable
regional variations in data production and use can be observed. The need for better coverage of
hard-to-reach population groups in all relevant data collection exercises has been identified as a
common challenge across countries and regions. The need for extensive capacity development in
the area of governance statistics is a key prerequisite to facilitate more widespread data collection
and use.

In terms of engaging new actors and exploring new data sources, several chapters empha-
size the benefits of strengthening the collaboration between National Statistical Offices and other
entities, both inside and outside government. National Statistical Offices have a central role to
play as a coordinating and harmonizing centrepoint of the national statistical system. This includes
the better integration of often-underutilized administrative data systems across government in-
stitutions and the exploration of unofficial data sources to complement more traditional ones.
Relevant actors listed across the various chapters include electoral management bodies, public
service commissions, ministries of public administration, parliamentary and ministerial secretariats,
judicial service commissions, ministries of justice, health and education institutions, national hu-
man rights institutions, equality bodies, civil society organisations, academia and representatives
of groups at risk of discrimination. Finally, the majority of chapters identify potential benefits from
further exploring and integrating insights from “big data” sources and experimental data.

HANDBOOK ON GOVERNANCE STATISTICS | vii

 ACKNOWLEDGEMENTS

This Handbook was developed under the leadership and co-ordination of the Praia City Group
on Governance Statistics, whose main objective is to promote and enable the production of gov-
ernance statistics based on sound and internationally agreed methodologies as part of the official
statistical system. It is one of the nine (at time of writing) city groups created by the UN Statistical
Commission as platforms of experts to address select issues in statistical methods. The chairman
of the Praia City Group is Osvaldo Borges, president of the National Statistical Institute of Cabo
Verde, who heads the City Group’s secretariat, comprising Cabo Verde INECV staff including
Celso Soares, Carlos Mendes, Elina dos Santos, João Mendes and Malene Almeida, who led
in the coordination of the Handbook, and a Steering Committee comprising AfDB, IRD, INEGI,
OHCHR, PARIS21, PRIO and UNDP.

The development of this Handbook would not have been possible without generous contribu-
tions from a wide range of organizations and stakeholders.

The development of the substantive chapters of the Handbook was led by:

•  ross-cutting principles and considerations: Adrián Franco Barrios (INEGI), Héctor Duarte
C
Ortiz (INEGI), Oscar Jaimes Bello (INEGI).
• Human rights – a cross-cutting normative framework for governance statistics: Grace
Sanico Steffan (OHCHR), Marc Cebreros (OHCHR), Nicolas Fasel (OHCHR).
• Non-discrimination and equality: Grace Sanico Steffan (OHCHR), Marc Cebreros
(OHCHR), Nicolas Fasel (OHCHR). The development of the two chapters led by OHCHR
was financially supported by the European Union.1
• Participation: Alexandra Wilde (UNDP), Marie Laberge (UNDP), Ionica Berevoescu (UN
Women), Julie Ballington (UN Women).
• Openness: El Iza Mohamedou (PARIS21), François Fonteneau (PARIS21), Frédéric Saint
Martin (PARIS21), Paz Patiño (PARIS21).
• Access to and quality of justice: Tatyana Taplova (OECD), Jennifer Tsai (OSJI), Peter
Chapman (OSJI), Zaza Namoradze (OSJI), Pascoe Pleasence (University College London),
Enrico Bisogno (UNODC), Michael Jandl (UNODC), Salomé Flores (UNODC), Sarika
Dewan (UNODC).
• Responsiveness: Santiago González (OECD), Zsuzsanna Lonti (OECD).
• Absence of corruption: François Roubaud (IRD), Mireille Razafindrakoto (IRD), Enrico
Bisogno (UNODC), Michael Jandl (UNODC), Salomé Flores (UNODC), Sarika Dewan
(UNODC).
• Trust: Lara Fleischer (OECD), Marco Mira d’Ercole (OECD).
• Safety and security: Adrián Franco Barrios (INEGI), Oscar Jaimes Bello (INEGI), Håvard
Mokleiv Nygård (PRIO), Håvard Strand (University of Oslo), Grace Sanico Steffan
(OHCHR), Enrico Bisogno (UNODC), Giulia Mugellini (UNODC), Michael Jandl (UNODC),
Salomé Flores (UNODC).

1 The contents of these chapters can in no way be taken to reflect the views of the European Union.

viii |
 The development of this Handbook also benefited greatly from an explorative analysis carried
out by UNECA and AfDB into the complex area of government effectiveness, which has provided
important insights reflected throughout this Handbook.

The Praia Group expresses its deep appreciation to the editor of the Handbook, Felix
Schmieding (independent consultant).

The Handbook is the result of extensive consultations with national and international statistical
and governance experts. In particular, the Handbook benefited greatly from the many comments
and suggestions made by national statistical offices and other experts, including the following:
ARMSTAT Armenia; Cameroon Policy Analysis and Research Center; Central Agency for Public
Mobilization and Statistics (CAPMAS) Egypt; Caribbean Community (CARICOM); Central Bureau
of Statistics of the Government of Sudan; Central Bureau of Statistics of the State of Israel; Central
Statistical Bureau of The State of Kuwait; Central Statistical Organization of Iraq; Central Statistical
Organization of The Government of the Republic of the Union of Myanmar; DANE Colombia;
Danish Institute for Human Rights; DFID UK; EUROSTAT; European Central Bank; Fundamental
Rights Agency of the European Union; General Authority for Statistics, Kingdom of Saudi
Arabia; General Statistics Office Of Vietnam; Hague Institute for Innovation of Law; IBGE Brazil;
Instituto Nacional de Estadística y Censos, Ecuador; INE Angola; INE Bolivia; INE Cabo Verde;
INE España; INEGI México; INEI Peru; INS Cote d’Ivoire; INSEE France; INSTAT Madagascar;
IRD France; ISTEEBU; KNBS Kenya; Hungarian Central Statistical Office; the Law and Justice
Foundation of NSW Australia; Legal Aid South Africa; Lithuania Statistics; MoSPI India; Ministerio
de Desarrollo Social del Uruguay; Ministerio del Interior del Gobierno de España; Ministerio del
Interior del Uruguay; Mo Ibrahim Foundation; National Bureau of Statistics of China; National
Statistical Committee of the Republic of Belarus; National Statistical Office of Malawi; Prof.
Rebecca Sandefur; OHCHR; OECD; OSJI; PARIS21; Palestinian Central Bureau of Statistics, State
of Palestine; PSA Qatar; Saferworld; SERNAC Chile; Statistical Office of the Republic of Serbia;
Statistical Office of the Republic of Slovenia; Statistics Austria; Statistics Botswana; Statistics
Canada; Statistics Lithuania; Statistics Korea; Statistics Mauritius; Statistics Norway; Statistics
South Africa; Statistics Suriname; Statistics Sweden; Statistik Austria; Swiss Federal Statistical
Office; Transparency International; TURKSTAT; UK Office of National Statistics; UNDESA; UNDP;
UNECA; UNECLAC; UNECE; UNESCAP; UNODC; UN Women; World Bank; World Justice Project;
the World Values Survey Association; and the Zimbabwe National Statistics Agency.

Funding for the development of the Handbook was provided by the Norwegian Ministry of
Foreign Affairs and the United Kingdom’s Department for International Development (DFID). The
Praia Group also wishes to acknowledge the support from the UNDP Oslo Governance Centre,
including the advisory role of Alexandra Wilde during the process.

Finally, thanks go to Bruce Hamm for copyediting this publication.

HANDBOOK ON GOVERNANCE STATISTICS | ix

 TABLE OF CONTENTS
HANDBOOK ON
GOVERNANCE STATISTICS

Executive summary v
Acknowledgements viii
Abbreviations/acronyms xviii

Introduction 1
Background 1
About the Praia Group and this Handbook 2
Who is this Handbook for? 3
Conceptualizing governance 3
Structure of the Handbook 5
Alignment of the Handbook with the 2030 Agenda 5

 Part A: Cross-cutting issues

I. Cross-cutting principles and considerations 9
I.1 Basic principles for producing, analysing and disseminating governance statistics 10
I.1.1 User focus 10
I.1.2 Methodological consistency and comparability 10
I.1.3 Importance of data production at country level and through national statistical systems 10
I.1.4 Importance of data disaggregation 11
I.1.5 Objective and subjective data go hand-in-hand in governance statistics 11
I.2 Managing the cost implications of producing governance statistics 11
I.3 Differentiating between structural, process and outcome indicators 12
I.4 A framework for discussing data quality in governance statistics 13
I.5 Data sources to measure governance 14

II. Human rights: A cross-cutting normative framework for governance statistics 21

II.1 Why do human rights matter for the development of governance statistics? 22
II.2 What are human rights? 22
II.3 Statistics and the international human rights normative framework 24
II.4 Human rights-based approach to data 26

x |
 Part B: Eight dimensions of governance
III. Non-discrimination and equality 31
III.1 Conceptualizing this dimension 32
III.1.1 What is discrimination? 32
III.1.2 What are the prohibited grounds of discrimination? 33
III.2 Why is this dimension important? 34
III.3 Data and best practices currently available 35
III.3.1 Understanding discrimination requires a comprehensive view of both
direct and indirect discrimination 35
III.3.2 Measuring direct discrimination 38
III.3.3 Measuring indirect discrimination 44
III.3.4 Quality of available statistics 46
III.4 Recommended key indicators 48
III.5 Way forward 49

IV. Participation in political and public affairs 53
IV.1 Conceptualizing this dimension 54
IV.1.1 Normative framework and definition of “participation in political and public affairs” 54
IV.1.2 Sub-dimensions of participation in political and public affairs 55
IV.2 Why is this dimension important? 59
IV.3 Data and best practices currently available 60
IV.3.1 Administrative data 61
IV.3.2 Surveys 66
IV.3.3 Expert assessments 70
IV.4 Recommended key indicators 73
IV.5 Way forward 77

V. Openness 81
V.1 Conceptualizing this dimension 82
V.2 Why is this dimension important? 85
V.3 Data and best practices currently available 86
V.3.1 Overview of the most established openness metrics 86
V.3.2 Deconstructing composite indices into stand-alone indicators 86
V.3.3 Alternatives to openness data sourced only from expert assessments 92
V.4 Recommended key indicators 95
V.5 Way forward 96

HANDBOOK ON GOVERNANCE STATISTICS | xi


VI. Access to and quality of justice 99
VI.1 Conceptualizing this dimension 100
VI.1.1 Normative framework and definition of concepts 100
VI.1.2 A possible taxonomy 104
VI.2 Why is this dimension important? 104
VI.3 Data and best practices currently available 107
VI.3.1 Administrative data 107
VI.3.2 Population survey data 110
VI.3.3 Good practices and tools 115
VI.3.4 User surveys 116
VI.4 Recommended key indicators 118
VI.5 Way forward 121

VII. Responsiveness 125
VII.1 Conceptualizing this dimension 126
VII.2 Why is this dimension important? 128
VII.3 Data and best practices currently available 128
VII.3.1 System responsiveness 128
VII.3.2 Satisfaction with services 133
VII.3.3 How can survey design mitigate methodological issues in
system responsiveness and satisfaction with services questions? 137
VII.4 Recommended key indicators 143
VII.5 Way forward 143

VIII. Absence of corruption 147
VIII.1 Conceptualizing this dimension 148
VIII.1.1 Normative framework 148
VIII.1.2 The different types of corruption 148
VIII.1.3 What sub-dimensions can be measured? 148
VIII.2 Why is this dimension important? 151
VIII.3 Data and best practices currently available 152
VIII.3.1 Administrative data 152
VIII.3.2 Sample surveys among the population and businesses 155
VIII.3.3 Sample surveys among civil servants 163
VIII.3.4 Composite indices 165
VIII.3.5 Other data sources 166
VIII.4 Recommended key indicators 167
VIII.5 Way forward 168

xii |

IX. Trust 171
IX.1 Conceptualizing this dimension 172
IX.1.1 What is trust? 172
IX.1.2 Can trust be measured? 173
IX.1.3 Is trust policy-amendable? 174
IX.2 Why is this dimension important? 174
IX.3 Data and best practices currently available 176
IX.3.1 Data sources 176
IX.3.2 Best practice for specifying survey questions on trust 178
IX.3.3 The state of trust 180
IX.3.4 How many institutions should be considered when asking trust questions? 182
IX.3.5 What do we know about the quality of trust statistics? 183
IX.3.6 How can survey design mitigate methodological issues in trust questions? 184
IX.4 Recommended key indicators 186
IX.5 Way forward 187

X. Safety and security 189
X.1 Conceptualizing this dimension 190
X.2 Why is this dimension important? 192
X.3 Data and best practices currently available 192
X.3.1 Administrative data on crime reported to or detected by law enforcement and
criminal justice authorities 194
X.3.2 Administrative data on violence produced by public health institutions 199
X.3.3 Victimization surveys 200
X.3.4 Administrative data on documented deaths and injuries related to armed conflict 203
X.3.5 Estimates of conflict fatalities 206
X.3.6 Administrative data on the performance of law enforcement and criminal justice
institutions 209
X.3.7 Surveys on law enforcement and criminal justice institutions
(population surveys and surveys of employees of these institutions) 211
X.4 Recommended key indicators 215
X.5 Way forward 218

List of references 220

Annexes 242

HANDBOOK ON GOVERNANCE STATISTICS | xiii

 LIST OF BOXES, FIGURES
AND TABLES

Box I.1 Structural, process and outcome indicators in the SDG indicators framework 13
Box I.2 Quality framework for OECD statistics 14
Box I.3 Public transportation in Mexico City: A successful case for the complementarity
of sources 16
Box II.1  Obligations to respect, protect and fulfil Human Rights 24
Box II.2  Human Rights and the data revolution 26
Box II.3  Examples of initiatives to operationalize a HRBAD at country level 28
Box III.1  The Universal Declaration of Human Rights, 1948 (excerpts) 34
Box III.2  Identifying non-discrimination indicators in Bolivia 35
Box III.3  Measuring discrimination through household surveys in Mexico 39
Box III.4 Measuring discrimination experiences in the European Union 40
Box III.5 Measuring discrimination through different household surveys in Canada 40
Box III.6 Recommended survey module for SDG indicator 10.3.1 and 16.b.1 on discrimination 42
Box III.7  The impact of data on violence against women in the European Union 43
Box III.8  Identifying and sharing good practices in collecting and using equality data 43
Box III.9 Identifying and sharing good practices in collecting and using equality data 48
Box IV.1 On the importance of disaggregating data on participation in political and public
affairs 57
Box IV.2 Statistics Norway’s experience in producing voter turnout statistics disaggregated
by immigrant background 62
Box IV.3 Institutionalizing the collection and dissemination of disaggregated data by EMBs 63
Box IV.4 Measuring representation in the public service - SDG 16.7.1.b. 65
Box IV.5 Selected questions on political participation and its enabling environment in
the African Union SHaSA module on Governance, Peace and Security 67
Box IV.6 Methodological issues to keep in mind when measuring sub-dimensions of
participation in political and public affairs through a population survey 69
Box IV.7 Public Service Employee Survey of the Canadian Public Service 70
Box IV.8 Using expert assessments to report on SDG 5.1.1
(“Whether or not legal frameworks are in place to promote, enforce and
monitor equality and non‐discrimination on the basis of sex”) 71
Box IV.9 The role of NHRIs in monitoring elections in Zimbabwe 72

xiv |
Box IV.10 Regional initiatives in compiling and disseminating data on women’s representation in
positions of power and decision-making: the case of Latin American and the Caribbean 77
Box VI.1 Mapping legal needs to better plan legal assistance services 106
Box VI.2 Access to Justice Centres in Argentina 109
Box VI.3 South Africa’s Governance, Peace, Security and Justice Survey 111
Box VI.4 A Stand-alone Access to Justice Survey in South Korea 112
Box VI.5 Index of Effective Access to Justice in Colombia 120
Box VII.1 Perceptions of system responsiveness in the USA 129
Box VII.2 The NOSAY question in the PIAAC and ESS surveys 130
Box VII.3 Questions on system responsiveness recommended as indicators of SDG indicator 16.7.2 132
Box VII.4 Satisfaction with education and health services 134
Box VII.5 Questions on satisfaction with health care for the measurement of SDG indicator 16.6.2 136
Box VII.6 Questions on satisfaction with education for the measurement of SDG indicator 16.6.2 138
Box VIII.1 Administrative Data: Best Practices/Useful Tools 155
Box VIII.2 Benefits and drawbacks of corruption surveys 156
Box VIII.3 Pros and cons of dedicated surveys and integrated modules 159
Box VIII.4 Surveys: Best Practices/Useful Tools 160
Box VIII.5 Sample surveys among civil servants: Best Practices/Useful Tools 164
Box VIII.6 The “mirror survey”: A comparison between people’s experiences and expert opinions 166
Box IX.1 Trustlab: Measuring trust and social norms through experimental techniques 177
Box IX.2 Survey questions recommended by the OECD as a “core module” on trust 179
Box X.1 Examples of rights relevant to safety and security 190
Box X.2 Intentional Homicide Statistics 196
Box X.3 International Classification of Crime for Statistical Purposes (ICCS): Improving the
quality and comparability of data on crime and justice at national and global level 197
Box X.4 Factors influencing the willingness to report crime to the police 198
Box X.5 Harmonization of homicide data: The case of Chile 200
Box X.6 Standardizing Victimization Surveys: The Latin America and Caribbean Crime
Victimization Survey Initiative (LACSI) 201
Box X.7 Experience and perception of crime, safety and security: Results from the GPS-SHaSA
modules in Africa 202
Box X.8 SDG 16.1.2 on the number of conflict-related deaths per 100,000 population,
by sex, age and cause 205
Box X.9 Law Enforcement Core Statistics (LECS) – United States 211
Box X.10 National Survey of Standards and Police Professional Training (ENECAP) 213

HANDBOOK ON GOVERNANCE STATISTICS | xv

 Figure I.1 Structural, process and outcome indicators 12
Figure II.1 Main principles of the Human Rights-Based Approach to Data 26
Figure III.1 Perception of discrimination in 9 African countries 43
Figure III.2 Primary completion rate in the Philippines, gender and regional disparities,
2006 & 2015 45
Figure III.3 Functional illiteracy in South Africa between 2002 and 2018 46
Figure V.1 Sub-dimensions and components of openness 84
Figure VII.1 Percentage of the voting age population who feel they have a say in what
the government does, 1952-2016 129
Figure VII.2 Percentage of the working-age population who feel they have a say in what the
government does, around 2012 130
Figure VII.3 Percentage of the population over age 15 who feel that the political system allows
them to have a say in what the government does 130
Figure VII.4 Percentage of the population satisfied with health and education in OECD members,
2017 134
Figure VII.5 Percentage of the population who are satisfied with the education and health
they have access to, 2016 134
Figure VIII.1 Patterns of bribery in Nigeria 160
Figure VIII.2 Corruption incidence and poverty in Peru, 2003-2018 161
Figure VIII.3 Perception and experience of corruption in seven African countries 162
Figure VIII.4 Perception and experience of corruption, Mexico 162
Figure VIII.5 Prevalence of corruption (victims per 1000 police officers), by beneficiary type
and by jurisdiction 164
Figure VIII.6 Civil servants’ exposure to corrupt practices, by frequency of interaction 164
Figure IX.1 Policy determinants of trust in government in selected OECD countries 175
Figure IX.2 Trust in people you meet for the first time, by country 181
Figure IX.3 Trust in national government between 2006 and 2017 in OECD countries 181
Figure IX.4 Trust in institutions by socio-economic status 182
Figure X.1 Crime victimization and perception of safety in selected countries of Africa
(2013-2016) 202
Figure X.2 Comparison of homicide statistics and armed conflict death rates
(per 100 000 inhabitants), 2015 206

xvi |
Table 0 SDG 16 indicators discussed in this Handbook 7
Table I.1 Sources for Governance Statistics (strengths, limitations, indicators and examples) 17
Table II.1 Main principles of the Human Rights-Based Approach to Data 27
Table III.1 List of illustrative indicators on non-discrimination and equality
(Universal Declaration of Human Rights, arts. 1, 2 and 7) 36
Table III.2 Average, deprivation and inequality perspectives 44
Table III.3 Recommended key indicators 49
Table IV.1 Internationally agreed normative frameworks on participation of specific groups
in political and public affairs 55
Table IV.2 Recommended key indicators 74
Table V.1 Available indices and comparative assessment frameworks on openness 87
Table V.2 Openness indicators identified from composite indices and comparative frameworks 88
Table V.3 Recommended key indicators 95
Table VI.1 Sub-dimensions of access to justice 105
Table VI.2 Recommended key indicators 119
Table VII.1 Summary evaluation of the accuracy of survey measures of system responsiveness 132
Table VII.2 Summary evaluation of the accuracy of survey measures of satisfaction with services 135
Table VII.3 Recommended key indicators 143
Table VIII.1 Examples of sample surveys on the experience and perception of corruption 157
Table VIII.2 Examples of international composite indices on corruption 165
Table VIII.3 Recommended key indicators 167
Table IX.1 Overview of surveys in the OECD Trust Database 177
Table IX.2 A summary evaluation of the accuracy of survey measures of trust 183
Table IX.3 Recommended key indicators 186
Table X.1 SDG indicators derived from Victimization Surveys 201
Table X.2 Recommended key indicators 216

HANDBOOK ON GOVERNANCE STATISTICS | xvii

 ABBREVIATIONS / ACRONYMS

AD Administrative data
ADR Alternative dispute resolution
AFDB African Development Bank
ATI Access to information
AU African Union
CAPMAS Central Agency for Public Mobilization and Statistics, Egypt
CARICOM Caribbean Community
CEDAW Convention on the Elimination of all Forms of Discrimination Against Women
CEPEJ Council of Europe European Commission for the Efficiency of Justice
CSO Civil society organization
DANE Colombia National Administrative Department of Statistics
DFID United Kingdom Department for International Development
DIAL Développement, Institutions et Mondialisation (France)
EA Expert assessment
EBS Establishment and business survey
EMB Electoral management body
EUROSTAT European Union statistical office
FRA Fundamental Rights Agency of the European Union
HRBAD Human rights-based approach to data
HRMIS Human Resource Management Information System
HS Household surveys
IAEG Inter-Agency and Expert Group on SDGs
IBGE Brazil National Statistics Office
ICESCR International Covenant on Economic, Social and Cultural Rights
ICCPR International Covenant on Civil and Political Rights
ICCS International Classification of Crime for Statistical Purposes
ILO International Labour Organization
INECV National Institute of Statistics, Cabo Verde
INE Instituto Nacional de Estatística/ Instituto Nacional de Estadística
INEGI Mexico National Statistics Office
INEI Peru National Statistics Office
INSEE National Institute of Statistics and Economic Studies, France
INSTAT Institut National de la Statistique, Madagascar
IRD The French National Research Institute for Sustainable Development
xviii |
ISTEEBU Institute of Statistics and Economic Studies of Burundi
KDI Korea Development Institute
LAC Latin America and Caribbean region
MDG Millennium Development Goal
MENA Middle East and North Africa
MICS Multiple indicator cluster survey
MoSPI Ministry of Statistics and Programme Implementation, Government of India
NHRI National human rights institution
NGO Non-governmental organization
NSO National statistics office
NSS National statistical system
OECD Organisation for Economic Co-operation and Development
OHCHR Office of the United Nations High Commissioner for Human Rights
OSCE Organisation for Security and Co-operation in Europe
OSJI Open Society Justice Initiative
PARIS21 The Partnership in Statistics for Development in the 21st Century
PCBS Palestinian Central Bureau of Statistics
PRIO Peace Research Institute of Oslo
PSA Qatar Planning and Statistics Authority
SDG Sustainable Development Goal
SERNAC Servicio Nacional del Consumidor, Chile
SHaSA-GPS Strategy for Harmonization of Statistics in Africa on Governance, Peace and Security
TSM Temporary special measure
TURKSTAT Turkey national statistics office
UDHR Universal Declaration of Human Rights
UNCAC United Nations Convention against Corruption
UNDESA United Nations Department of Economic and Social Affairs
UNDP United Nations Development Programme
UNECA United Nations Economic Commission for Africa
UNECLAC United Nations Economic Commission for Latin America and the Caribbean
UNESC United Nations Economic and Social Council
UNESCAP United Nations Economic and Social Commission for Asia and the Pacific
UNICEF United Nations Children's Fund
UNODC United Nations Office on Drugs and Crime
UNSC United Nations Statistical Commission
UNTS United Nations Treaty Series
WJP World Justice Project
WVS World Values Survey
HANDBOOK ON GOVERNANCE STATISTICS | xix
xx |
INTRODUCTION

Background

1. Governance statistics are fundamental to ensuring that the relationship between the State and
its citizenry is transparent and accountable. They assist in monitoring the performance of government
and in better understanding the contribution of governance in its various dimensions to develop-
ment. Governance statistics contribute to the measurement and realization of human rights. They can
help to identify groups or sub-groups in the population that are most affected by the dysfunctions of
governance systems, with a view to putting in place appropriately targeted policies, as pledged in the
2030 Agenda “Leaving no one behind”. Governance statistics can also contribute to preventing and
managing conflict, when used as early warning systems, and can help foster peace, by periodically
informing on State-society relations, which lie at the centre of sustainable peace.

2. The last three decades have seen many organizations and scholars working on the develop-
ment of measures of governance. The first efforts in integrating new modules in traditional soci-
oeconomic surveys appeared in the mid-1990s as add-on modules to traditional socioeconomic
surveys in developing countries, Madagascar being the first country to carry out such project.
Many countries followed this approach, like Benin, Vietnam and Peru, among others. In parallel,
Transparency International placed corruption measurements on the agenda by publishing the
first Corruption Perceptions Index in 1995. Subsequently, international organizations such as the
Kellogg Institute, the World Justice Project and the Bertelsmann Foundation summarized many
governance-related concepts into composite indices, of which the most emblematic are the World
Bank’s World Governance Indicators. Although governance featured prominently in the values and
principles of the United Nations Millennium Declaration,1 it was not included in the monitoring
framework for the Millennium Development Goals, owing to a lack of consensus at the time on how
and by whom it was to be defined and measured. In 2012, international guidance for human rights
measurement was published by the UN Human Rights Office (UN High Commissioner for Human
Rights, 2012). The next milestone of governance statistics was the approval of the 2030 Agenda for
Sustainable Development in 2015 in which the international community, through the leadership of
the United Nations, recognized the role of governance, peace and security as crucial prerequisites
for achieving sustainable development. It is worth noting that developing countries have led much
of the progress in governance measurement over the last 25 years (Razafrindakoto and Roubaud,
2015), and more countries are joining these efforts, thereby furthering a better understanding of the
current state of governance at a global level.

1 Governance measurement before 2000 is well covered in the earlier part of Wilde, “The democratization of governance assessments”, in Hydén and
Samuel, eds. (2011).

HANDBOOK ON GOVERNANCE STATISTICS | 1

 About the Praia Group and this Handbook

3. Following the conference of the International Association for Official Statistics held in 2000, 2
the Friends of the Chair of the United Nations Statistical Commission (UNSC) stated in its report
that the development of statistical indicators on governance would not be easy and would take
time. It recommended that the Commission establish a mechanism to develop statistical indicators
of human rights and good governance (UN Social and Economic Council, 2001). To this end, the
National Institute of Statistics of Cabo Verde (INECV) — having successfully piloted the “Strategy
for Harmonization of Statistics in Africa on Governance, Peace and Security” (SHaSA-GPS) initiative
in 20133 — proposed the creation of a new city group, the “Praia Group on Governance, Peace and
Security Statistics” (the Praia Group) at the forty-fifth session of the UNSC in 2014. 4 Subsequently,
the vast majority of actors consulted expressed the view that peace and security are in fact constitu-
tive dimensions of governance and should be investigated under the broad conceptual framework
of governance. Therefore, the proposed group was renamed the “Praia Group on Governance
Statistics”. In March 2015, during the forty-sixth session of the UNSC, the Praia Group was estab-
lished to “contribute to establishing international standards and methods for the compilation of
statistics on the major dimensions of governance”. To this end, the Group was charged to develop
“a handbook on governance statistics for national statistical offices, which will cover the conceptu-
alization, measurement methodology and dissemination of governance statistics”.5

4. Whilst the Praia Group aims to contribute to the development of international standards on
governance statistics, the purpose of the Handbook itself is not to promulgate such standards. The
development of such standards is a more substantial endeavour that requires extensive pilot-testing
and in-depth examination of the validity and reliability of proposed standardized methodologies
across time and space, and as such will unfold over a longer time frame. Likewise, the key indica-
tors recommended in this Handbook do not constitute or imply a reporting obligation for national
statistical systems. However, this Handbook is considered an important stepping stone in eventu-
ally achieving international statistical standards in all areas of governance statistics. To this end, it
conceptualizes and systematizes the various dimensions of governance statistics, takes stock of and
qualifies existing practices in governance data collection, highlights the most important metrics in
each dimension of governance statistics, and discusses the way forward required to achieve interna-
tional statistical standards in each dimension.

5. Given that governance statistics are a “work in progress”, this Handbook is considered a “liv-
ing document” that will be updated in the future to reflect the latest developments in the eventful
field of governance statistics. Each of the substantive chapters of this Handbook outlines a way
forward in terms of furthering the methodological development in the respective area with a view to
eventually achieving internationally-agreed statistical standards. The Praia Group will continuously
monitor all major accomplishments made on this way forward, and produce revised editions of this
Handbook when this is justified.

2 http://www.portal-stat.admin.ch/iaos2000.
3 The GPS-SHaSA initiative develops, tests and institutionalizes measurement instruments and is designed for use by the continent’s National Statistics
Offices (NSOs). The initiative’s experience contributes to the Praia Group’s work on international standards for governance statistics worldwide. See the
working paper analysing the eight dimensions of governance covered in this Handbook using GPS-SHaSA household surveys in nine sub-Saharan African
countries. Razafindrakoto M., Roubaud F., Assany Y. A. (2019): Populating the Handbook on Governance Statistics with Empirical Evidence: Illustrations
from the GPS-SHaSA survey modules in Africa, IRD DIAL Working paper No. 2019-15.
4 Specifically, ongoing work on crime statistics led by the National Institute of Statistics and Geography of Mexico (INEGI) and UNODC include the road
map to improve the quality and availability of crime statistics at national and international levels (see UN, E/CN.3/2013/11); the International Classification
of Crimes for Statistical Purposes (see UN, E/CN.3/2015/7); and the Manual on Victimization Surveys (Geneva, UNODC and Economic Commission for
Europe, 2010), available from https://www.unodc.org/unodc/en/data-and-analysis/Manual-on-victim- surveys.html).
5 Cabo Verde Report on governance, peace and security statistics to the UN Statistical Commission 46th Session – Annex: “Proposed Terms of Reference
for the Praia group on governance statistics”.

2 |
Who is this Handbook for?

6. Demands for governance statistics are increasing. Governments, municipalities, academia,

civil society, private sector entities, scientific communities, media organizations, youth groups and
regional human rights and international organizations are currently the main users of governance
statistics. This Handbook is intended as a tool for those wishing to understand, produce and ana-
lyse governance statistics, and is primarily targeted towards national statistical agencies. National
statistical offices offer important comparative advantages for the production of official statistics on
governance. They have the official mandate to coordinate national statistical production, they hold
ample methodological expertise to do so, and they are best placed to ensure the sustainability of
national data collection systems. Furthermore, since governance is a public good, there are impor-
tant benefits to be gained if public bodies produce statistics on these issues.

Conceptualizing governance

7. While the concept of “good governance” has been around in both political and academic dis-
course for a long time, it does not have a widely agreed single definition. This is not surprising: it is
a multifaceted concept; its various terrains – market, state and society – are emphasized by different
actors with particular purposes; and the operationalization of the concept has altered significantly
over time, with processes like globalization, devolution and outsourcing, and the digital age.6

8. Beginning in the broadest terms, “governance” is generally understood to concern the various
institutions, mechanisms and established practices through which a country exercises governmental
authority, discharges its responsibilities and manages its public resources. While there is a consensus
on this basic concept, different authors and institutions tend to have specific focuses within this uni-
verse — indeed, the same institution may define “governance” differently over time and for different
purposes.7

9. Examples of definitions of governance suggested by international organizations include:

• The United Nations Development Programme (UNDP) in 1995 defined governance as the
exercise of political, economic and administrative authority at all levels in the management
of a country’s affairs. It comprises the complex mechanisms, processes, relationships and
institutions through which citizens and groups articulate their interests, exercise their legal
rights and obligations and mediate their differences. Subsequent UNDP documents have
expanded and elaborated the concept and, since 2002, UNDP has been using the term
“democratic governance” to refer to a system of governance that allows people’s rights
and freedoms to be respected and for them to have a say in the decisions that affect their
lives.8
• The Office of the United Nations High Commissioner for Human Rights (OHCHR) provided
a working definition of good governance: “the process whereby public institutions conduct
public affairs, manage public resources and guarantee the realization of human rights in

6 Useful overviews of the various definitions of governance, including those discussed in academia, are provided in UNDESA (2007) as well as in INEGI
(2017).
7 The United Nations Economic and Social Council conducted an extensive review of the literature defining governance in 2006 (UNESC, 2006). For further
examples of approaches to defining governance in different organizations see Center for Global Development (2013), International Fund for Agricultural
Development (IFAD, 2016), United Nations Economic and Social Commission for Asia and the Pacific (2009), United States Institute of Peace (USIP, 2010),
and World Bank (2011).
8 UNDP Human Development Report, 2002, “Deepening democracy in a fragmented world”. See page p. 51 for a full explanation of the term “democratic
governance”.

HANDBOOK ON GOVERNANCE STATISTICS | 3

 a manner essentially free of abuse and corruption, and with due regard for the rule of law
(OHCHR, 2000).”
• OECD brings in a political aspect with the following definition: “the use of political authority
and the exercise of control in a society in relation to the management of its resources for
social and economic development (OECD, 2015a).”

10. In the face of multiple definitions, a 2016 in-depth review of governance statistics undertaken
by the OECD, INEGI and TURKSTAT (the CES Report, UNECE, 2016) — and which played a par-
ticularly central role in informing and inspiring the Praia Group’s early work — concluded that “the
only option for operationalizing complex concepts such as governance is to ‘deconstruct’ them”
(González, Fleischer and Mira d’Ercole, 2017). This Handbook does the same. In a pragmatic effort
to simplify the conceptual framework, the primary conceptualization of governance is limited to
eight dimensions.9 These eight dimensions have been reformulated to be consonant with current
usage in many of the available governance definitions, and to meet the needs of users in policy,
popular and academic areas.10 However, the eight dimensions proposed here are by no means con-
sidered a comprehensive or closed set, and do not have the ambition to cover all that is relevant to
the concept of governance. Additional chapters may be needed in future editions of this Handbook
with the advance of statistical research and practice.

11. The eight dimensions of governance used by this Handbook are:

1) N on-discrimination and equality: This dimension focuses on any distinction, exclusion,

restriction or preference or other differential treatment that is based on grounds such as
colour, sex, language, religion, national or social origin, disability or other status, with the
intention or effect of nullifying or impairing human rights and fundamental freedoms.
2) Participation: This dimension focuses on the ways in which citizens take part in the con-
duct of political and public affairs, including by registering to vote, voting and standing as
a candidate in elections; being members of legislative, executive and judicial bodies at all
levels of government; accessing positions in the public service; and engaging, individually
or as members of political parties and other non-governmental organizations, in political
activities.
3) Openness: This dimension focuses on the extent to which public institutions provide access
to information and are transparent in their decision- and policy-making processes. More spe-
cifically, the dimension covers access to information, open government provisions, freedom
of expression and media pluralism.
4) Access to and quality of justice: This dimension focuses on the ability of people to defend
and enforce their rights and obtain just resolution of justiciable problems — if necessary,
through impartial formal or informal institutions of justice and with appropriate legal support.

9 Also drawing on the CES report, the conceptualization presented here uses as its starting point two decisions taken by the CES report to restrict the
scope of its review: first, it is agreed to focus on “public institutions serving the common good of a community of people”, which implies excluding cor-
porations and other private institutions; second, it is further specified that the focus is on public institutions operating at the national level (i.e. excluding
international or supranational ones).
10 The conceptualization presented here also draws critically on the conceptualization endorsed by the UN Statistical Commission, and as outlined in the
foundational report of the Praia Group (https://undocs.org/E/CN.3/2015/17). The report underlines the “important conceptual consensus reached in the
course of the [international] deliberations” on the proposed City Group held in 2014 prior to its creation, on two aspects. First, the inception report of the
Praia Group notes “the vast majority of actors consulted were of the view that peace and security are in fact constitutive dimensions of governance and
should be investigated under the broad conceptual framework of governance. It is in that context that the present proposal refers to establishing a Praia
group on governance statistics.” Second, the same report acknowledges the important ongoing work in the area of crime statistics led by the National
Institute of Statistics and Geography of Mexico (INEGI) and UNODC, and stresses the importance for the Praia Group to “avoid duplication of efforts” in
this area.

4 |
 5) R esponsiveness: This chapter focuses on whether people have a say in what government
does and whether they are satisfied with the government’s performance.
6) Absence of corruption: This chapter focuses on the levels of intolerance to corruption; the
levels and patterns of observable corrupt practices; and the State response to corruption.
7) Trust: This dimension focuses on people’s trust in institutions as well as in other people, with
a primary focus on the former, e.g. the parliament, the national government and the justice
system.
8) Safety and security: This dimension focuses on levels and patterns of crime, perceptions of
safety, measurement of casualties directly provoked by armed operations, and the quality of
law enforcement and criminal justice institutions.

Structure of the Handbook

12. Following this introduction, the Handbook has two main parts. Part A discusses cross-cutting
issues that are applicable to all or most dimensions of governance statistics, including the funda-
mental cross-cutting normative framework for governance statistics, overarching frameworks that
allow all subsequent chapters to be based in common analytical underpinnings and provided by
the international human rights norms and principles. Part B examines in detail each of the eight
dimensions of governance listed above, along the following structure:

• Concepts and definitions

• Why is this dimension important?
• Data sources and good practices
• Recommended key indicators
• Way forward

Alignment of the Handbook with the 2030 Agenda

13. The inclusion of governance in the indicator framework of the 2030 Agenda constituted a
major milestone in the evolution of governance statistics. It reinforced worldwide attention on this
area of statistics, and will undoubtedly be a major driver of demand for such data over the coming
years. Acknowledging the centrality of monitoring Sustainable Development Goal (SDG) 16 in users’
interest in this Handbook over the coming decade, SDG 16 indicators play an important role in the
discussions found across the book’s dimensions.

14. However, governance statistics did not begin with the SDGs and will not end with their con-
summation in 2030. As outlined above, the primary objective of this Handbook is to be a stepping
stone in the development of international statistical standards on governance statistics, whose con-
ceptual scope and lifetime will exceed the 2030 Agenda. Moreover, it is widely understood that the
SDG indicators provide only a basic set of indicators for monitoring in the various interconnected
fields of sustainable development, but cannot comprehensively measure all issues of concern to the
international community. Hence, this Handbook goes well beyond the global SDG 16 indicators.
Considering that the SDG indicator framework explicitly encourages the complementing of the

HANDBOOK ON GOVERNANCE STATISTICS | 5

 global indicator set through additional indicators that are highly relevant at the national level, this
Handbook may thereby also play a role in aiding National Statistical Systems to identify and select
such “localized SDG indicators” to monitor progress on SDG 16.

15. Four11 of the global indicators are not covered by this Handbook, due to the fact that they
did not fit naturally under any of the eight dimensions along which the Praia Group decided to
conceptualize it; future editions may include them as part of an expanded conceptual scope.

16. The mapping below shows where SDG 16 indicators are discussed in this Handbook.

11 The four indicators not covered are: 16.4.1 Total value of inward and outward illicit financial flows (in current United States dollars); 16.6.1 Primary
government expenditures as a proportion of original approved budget, by sector (or by budget codes or similar); 16.8.1 Proportion of members and voting
rights of developing countries in international organizations and 16.9.1 Proportion of children under 5 years of age whose births have been registered with
a civil authority, by age.

6 |
TABLE 0 SDG 16 indicators discussed in this Handbook

SDG indicator Primary discussion Further mentions

16.1.1 Number of victims of intentional homicide per 100,000 population, by sex and age Safety and security

16.1.2 Conflict-related deaths per 100,000 population, by sex, age and cause Safety and security

16.1.3 Proportion of population subjected to (a) physical violence, (b) psychological violence and Safety and security
(c) sexual violence in the previous 12 months

16.1.4 Proportion of population that feel safe walking alone around the area they live Safety and security

16.2.1 Proportion of children aged 1–17 years who experienced any physical Safety and security
punishment and/or psychological aggression by caregivers in the past month

16.2.2 Number of victims of human trafficking per 100,000 population, by sex, Safety and security
age and form of exploitation

16.2.3 Proportion of young women and men aged 18–29 years who experienced sexual violence Safety and security
by age 18

16.3.1 Proportion of victims of violence in the previous 12 months who reported their victimization Access to and
to competent authorities or other officially recognized conflict resolution mechanisms quality of justice

16.3.2 Unsentenced detainees as a proportion of overall prison population Access to and

quality of justice

16.4.1 Total value of inward and outward illicit financial flows (in current United States dollars) Not applicable

16.4.2 Proportion of seized, found or surrendered arms whose illicit origin or context has been Safety and security
traced or established by a competent authority in line with international instruments

16.5.1 Proportion of persons who had at least one contact with a public official and who paid a bribe Absence of corruption
to a public official, or were asked for a bribe by those public officials, during the previous 12 months

16.5.2 Proportion of businesses that had at least one contact with a public official Absence of corruption
and that paid a bribe to a public official, or were asked for a bribe by those public officials,
during the previous 12 months

16.6.1 Primary government expenditures as a proportion of original approved budget, by sector Not applicable
(or by budget codes or similar)

16.6.2 Proportion of population satisfied with their last experience of public services System responsiveness and
satisfaction with services

16.7.1 Proportions of positions in national and local institutions, including (a) the legislatures; Participation in political Safety and security (16.7.1 c)
(b) the public service; and (c) the judiciary, compared to national distributions, by sex, age, and public affairs
persons with disabilities and population groups

16.7.2 Proportion of population who believe decision-making is inclusive and responsive, System responsiveness and Participation in political
by sex, age, disability and population group satisfaction with services and public affairs

16.8.1 Proportion of members and voting rights of developing countries in international organizations Not applicable

16.9.1 Proportion of children under 5 years of age whose births have been registered with Not applicable
a civil authority, by age

16.10.1 Number of verified cases of killing, kidnapping, enforced disappearance, arbitrary detention Openness
and torture of journalists, associated media personnel, trade unionists and human rights advocates
in the previous 12 months

16.10.2 Number of countries that adopt and implement constitutional, statutory and/or Openness
policy guarantees for public access to information

16.a.1 Existence of independent national human rights institutions in compliance with the Paris Principles Safety and security [tbc] Human rights: A cross-cutting normative
framework for governance statistics/
Non-discrimination and equality /
Participation in political and public
affairs / Access to and quality of justice

16.b.1 Proportion of population reporting having personally felt discriminated against or harassed Non-discrimination
in the previous 12 months on the basis of a ground of discrimination prohibited under and equality
international human rights law

HANDBOOK ON GOVERNANCE STATISTICS | 7

 PART A
CROSS-CUTTING ISSUES

CHAPTER I
Cross-cutting
principles and
considerations
 17. As discussed in the introduction, this Handbook conceptualizes governance statistics along eight
different dimensions. While subsequent chapters of this book will discuss each of those specific dimen-
sions in detail, this chapter introduces a set of considerations, principles and frameworks that are appli-
cable across dimensions and that provide a common underpinning for subsequent elaborations of this
book: first, basic principles for producing, analysing and disseminating governance statistics; second,
implications regarding the cost of producing governance statistics; third, an approach for distinguishing
between different types of statistical indicators; fourth, a framework for assessing data quality; and fifth,
a brief overview of the different types of data sources that are most relevant for measuring governance.

I.1 Basic principles for producing, analysing and disseminating governance statistics
I.1.1 User focus
18. Data needs to be generated first and foremost with users in mind. As with any other field of statistics,
it is essential to remember that the production of governance statistics is not an end in itself, rather, it must
serve those who put the data to use for better policy and better outcomes. Accordingly, data producers
should invest in identifying and engaging those in a position to use data to drive action (IAEG-SDGs, 2014).

I.1.2 Methodological consistency and comparability

19. Data must be produced with congruency. If different institutions apply different counting rules
and different methods, then data is not an equivalent and direct comparison is not possible. Moreover,
contextual factors such as culturally different notions of concepts like satisfaction or trust, or the particu-
larities of a country’s political system, pose challenges for the international comparability of governance
statistics. The good news is that there is a priori no reason why governance statistics cannot be generat-
ed and adhere to the same standards as metrics in other disciplines such as economics or social or envi-
ronmental statistics (González, Fleischer and Mira d'Ercole, 2017; OECD, 2017a), although this requires
significant efforts in terms of developing and consistently applying statistical standards and building
statistical capacity around them. Although there is not necessarily a criterion regarding methodological
consistency in the examples given in this Handbook (for example, with respect to scales), further work
should consider it a prime issue for the advancement of governance statistics.

I.1.3 Importance of data production at country level and through national statistical systems
20. Governance statistics are at current not routinely or comprehensively produced at the national
level. A study on governance statistics in the OECD-UNECE area found that, of the 29 countries consid-
ered, only about half collected statistics in the various dimensions of governance. Recent reports on data
availability for SDG 16 indicators also revealed that many indicators are not regularly produced at country
level (e.g. IAEG, 2019). All too often, the production of governance statistics is left to international and/
or non-public institutions (UNECE, 2016). This contrasts sharply with the unique advantages that national
statistical offices and systems offer when it comes to the production of statistics on governance. As pointed
out in the introduction of this Handbook, they hold ample methodological expertise and are well placed
to ensure the sustainability of national data collection efforts. Furthermore, since governance is a public
good, there are important benefits to be gained if public bodies produce statistics on these issues.

21. A commonly stated concern is that sensitive data on governance collected by government
institutions are potentially biased due to self-censorship by respondents. However, recent evidence
suggests that such an effect is not observed, and thereby confirms the legitimacy of government-led
data collection on governance (Calvo and others, 2019).1

1 The Calvo paper studies the potential bias in responses to what are seen as sensitive questions, here governance issues, in surveys conducted by public
organizations. It compares Afrobarometer (AB) survey data, collected in eight African countries by self-professed independent institutions, with other survey
data collected by National Statistics Offices (NSOs). The paper first compares responses from AB survey respondents based on who they believe the survey
sponsor to be. No systematic response bias is found between respondents who believe the government to be behind the AB survey and those who consider

10 |
I.1.4 Importance of data disaggregation
22. In our world, there are rising inequalities within and among countries, many of them associated with
the outcomes of governance, which translate to disparities of opportunity, wealth and power (A/Res/70/1
by the United Nations General Assembly). Hence, the measurement of governance dimensions requires
representative and meaningful disaggregation to understand differences within and between population
groups. Moreover, disaggregated data allow us to better understand the differences in everyday life
and in the interactions with government of vulnerable or marginalized groups (IAEG-SDGs, 2017).
In accordance with the principle of “Leaving no one behind” of the 2030 Agenda, it is important to
pay special attention to those who are furthest behind and who too often are invisible to official data
(IAEG-SDGs, 2014).2 In order to fulfill this commitment, however, NSOs must make significant efforts to
overcome the challenges of collecting disaggregated data, such as those relevant to methodological
issues, prioritization and the protection of personal data and the right to privacy.

I.1.5 Objective and subjective data go hand-in-hand in governance statistics

23. In governance statistics, frequent use is and should be made of both objective and subjective data.
On the one hand, objective data refer to the existence of tangible concepts such as the number of workers,
a budget, items found in a public office, or the number of processes carried out by an institution in
a given year. Objective data also comprise the experiences of respondents based on real events.
On the other hand, subjective data concern the respondent’s personal perspective and appreciation
regarding an issue, such as people’s feelings of trust or fear. While objective measures look to approx-
imate reality, subjective data reflect the assessment society makes of institutions, structures or certain
behaviours, as well as whether governance “works” for the people.

I.2 Managing the cost implications of producing governance statistics

24. Generating high-quality governance statistics can come with considerable cost implications. These
costs may be particularly high in countries that do not have a tradition of measuring governance indicators
(González and others, 2017). Moreover, other types of efforts (besides pecuniary ones) must be con-
sidered. Inter-institutional coordination, for example, is essential to strengthen statistical systems and
could be a prime challenge for many nations. However, there are ways to manage the cost implications
of collecting governance statistics without compromising data quality. On the one hand, countries can
focus attention on the improvement and expansion of existing administrative data systems, which can
be an important source of governance statistics. On the other hand, where measuring governance
phenomena requires the use of surveys, such operations can be made less burdensome through inte-
grating governance modules into existing surveys. Various countries in Africa and Latin America have
followed this approach, e.g. when introducing the SHaSA-GPS module (African Union-UNDP, 2017).
Italy and France also integrated questions on corruption experiences into existing victimization sur-
veys. Hence, surveys related to governance need not be stand-alone surveys. The modular approach
represents a lower burden not only for NSOs, but also for respondents, by limiting the questions
to be answered. This approach also provides the possibility to link data on governance with other
information included in the main survey (UNODC-UNDP, 2018), sometimes even increasing levels of
disaggregation for sociodemographic variables (Razafrindakoto and Roubaud, 2015). Despite these
advantages, covering a variety of topics within the same survey can also have negative effects, e.g.
on the attitude and attention of respondents, pressure to reduce the number of questions to explore
each dimension of interest, or reduced flexibility in terms of sample design. Data producers need to
weigh these and other aspects when deciding for or against an integrated survey approach (UNODC-
UNDP, 2018).

it to be conducted by an independent institution. The absence of a systematic self-censorship or attenuation bias is further evidenced by means of an
experimental design, whereby responses from surveys conducted by NSOs are compared with AB surveys sponsored by reportedly independent bodies.
2 As mentioned by the Inter-Agency and Expert Group on SDG indicators, this is something especially important in developing countries (2014).

HANDBOOK ON GOVERNANCE STATISTICS | 11

 I.3 Differentiating between structural, process and outcome indicators

25. When discussing and developing statistical indicators, it can be useful to systematically consider
the indicators’ nature by broad categories or types, in order to ensure a balanced and comprehensive
approach. Measurement can be enhanced when different angles of a single governance dimension are
explored and linkages between them are examined.

26. Building on methodological work in the area of human rights, the various chapters of this
Handbook categorize indicators as structural, process or outcome indicators, as outlined in Figure I.1.3

27. An example of a structural indicator is the existence of independent national human rights
institutions in compliance with the Paris Principles4 (SDG indicator 16.a.1). An example of a process
indicator is the proportion of children under 5 years of age whose births have been registered with
a civil authority (SDG indicator 16.9.1). An example of an outcome indicator is the percentage of the
population satisfied with their last experience with public services (SDG indicator 16.6.2).

28. The categorization of indicators suggested here does not imply that any measurement or data
collection initiative in the field of governance statistics should necessarily produce indicators within
each of these categories. In practice, a comprehensive compilation of structural, process and outcome
indicators may not always be feasible, since the relevant metrics in one dimension may fall predomi-
nantly under one or the other category. However, data producers and users can use this categorization
as a tool to further inform their choices of potentially relevant indicators.

FIGURE I.1 Structural, process and outcome indicators

Measure Structural inindicators

commitments
and adoption
of legal, Measure Process indicators
institutional implementation
and policy of policies and
frameworks programmes Measure the Outcome indicators
to realize the results of policies
commitments and programmes
on individuals/
groups/
population

Source: UNRISD 2016 Flagship Report.

3 This conceptual framework and terminology were endorsed by the international human rights treaty bodies, following two years of research and vali-
dation work involving human rights experts and statisticians from national and international organizations, including civil society organizations (UN High
Commissioner for Human Rights, 2008).
4 “Principles relating to the status of national institutions” (The Paris Principles). Adopted by General Assembly resolution 48/134 of 20 December 1993
(A/RES/48/134).

12 |
BOX I.1 Structural, process and outcome indicators in the SDG indicators framework

OHCHR analysed about two thirds of the indicators in the SDG framework (which were considered
relevant to the measurement of human rights). Out of these, about 50% of indicators can be
categorized as outcome indicators, 40% as process indicators and 10% as structural indicators.

Structural Process Outcome

10% 40% 50%

Source: Office of the United Nations High Commissioner for Human Rights

I.4 A framework for discussing data quality in governance statistics

29. As stated in the introduction, one of the key objectives of this Handbook is to take stock of
existing data and best practices in governance data collection. While subsequent chapters do make
occasional reference to data quality, it is beyond the scope of this Handbook to provide a comprehen-
sive data quality assessment of such existing data. Nevertheless, where a discussion of quality does
occur, the terminology applied will follow loosely the OECD quality framework for statistics (OECD,
2012). While various other internationally agreed data quality frameworks exist and may have provided
an equally suitable basis for the subsequent elaborations5, this framework was chosen because it is
explicitly applicable to both official and non-official data (as mentioned above, the latter remain highly
prevalent in the field of governance statistics). Box I.2 briefly outlines the terminology and meaning of
the different dimensions of the framework.

5 Plenty of relevant work in this matter is available online, for example: the UN National Quality Assurance Framework Manual, Eurostat’s Quality of Life
Framework, and the European Statistics Code of Practice.

HANDBOOK ON GOVERNANCE STATISTICS | 13

 BOX I.2 Quality framework for OECD statistics 6

1. Relevance: a qualitative assessment of the value contributed by these data.

2. A ccuracy: the degree to which the data correctly estimate or describe the quantities or characteristics they
are designed to measure, and the closeness between the values provided and the (unknown) true values.
a. R eliability: the degree to which the measure produces consistent results over time and across
different measurement vehicles;
b. V alidity: the degree to which a measurement reflects the underlying concept looking to
be estimated. This may include face-validity (a qualitative assessment based on intuitive
plausibility), construct validity (assessment of the degree to which a measure behaves in a
way that is consistent with expectations), and convergent validity (a measure bears statistical
resemblance with other measures of the same construct).
3. C redibility: the confidence that users place in the data, based simply on their image of the data producer.
4. T imeliness: the length of time between data availability and the event or phenomenon they describe.
5. A ccessibility: how readily the data can be located and accessed by users.
6. I nterpretability: the ease with which the user can understand and properly use and analyze the data
7. Coherence: the degree to which data products are logically connected and mutually consistent.
This includes coherence within a dataset, across datasets, over time and across countries.

Source: Adapted from OECD, Quality framework and guidelines for OECD statistical activities, Statistics Directorate STD/QFS (2011) 1.

I.5 Data sources to measure governance

30. The recent growth in the demand for governance statistics has led to an increased focus on
quantitative country-level data related to these topics. Consequently, NSOs and other governmen-
tal agencies have resorted mainly to five “conventional” statistical sources for the measurement of
governance. The following briefly outlines – at a general level – what the subsequent chapters are
referring to when discussing each of these sources.

•  ousehold surveys: These focus on a representative sample of the general population, es-
H
pecially convenient for gathering data on what people feel/perceive or experience about a
specific phenomenon. Especially for household surveys (although it might be relevant for
other sources), there are various survey methods available (face to face, online or telephone
surveys), each one having advantages and disadvantages that should be considered in or-
der to meet the survey’s goals and quality standards. Respondent burden and accessibility
standards for specific populations (e.g. for the population with a disability) should be contem-
plated in the questionnaire design as well as in the survey method selection. With a sufficient
sample size, survey results may be representative of the population of a country, of a specific
area (such as a province or a small locality) or a specific group (e.g. urban/rural populations).
However, such disaggregation for governance statistics are not commonly found in national
statistical systems and will require special attention from the statistical community. Household
surveys are particularly useful to give voice to marginalized populations in contexts in which
the mechanisms to respond to citizens’ demands are not yet consolidated (AU and UNDP,
2017).
• Business surveys: These are like household surveys in the sense that they collect statistically
representative data for a population directly from the informants, although their sampling
population are companies.7 However, they do not necessarily raise the same considerations

6 The quality framework presented in the 2011 publication does not further differentiate the accuracy component into reliability and validity. However, in
recent years this has become an established practice, see e.g. González, Fleischer and Mira d’Ercole (2017) or OECD (2013).
7 See, for instance, World Bank Group, Enterprise Surveys, What Businesses Experience, available at: https://www.enterprisesurveys.org/data

14 |
 or issues (methodological or regarding disaggregation, for example) and must be under-
stood as independent sources.
• Other surveys: Although most commonly surveys will collect information from households
and businesses, different units of analysis could be surveyed as well, for example, govern-
ment units, non-governmental organizations, prison populations, or police officers, among
others. Methodological considerations must be assessed for each unit of analysis.
• Censuses: A census produces statistics by collecting data from each unit within the scope of
the study (INEGI, 2012). These units of analysis are varied and may include, for example, in-
dividuals, businesses or government units. Data from censuses can be highly disaggregated.
• Administrative records: Unlike censuses and surveys that are specifically designed to meet
statistical needs, administrative records are part of the procedures that government agencies
routinely follow as part of their operations (UNECE, 2016). Administrative records are particu-
larly relevant in assessing the coverage and quality of services provided by the government.
• Expert assessments: These typically refer to professionals such as academics, lawyers or civil
servants who answer a questionnaire in the area of interest (UNECE, 2016). The resort to
experts is thought to be advantageous because they can assess complex topics related to
governance through an informed judgment (UNECE, 2016). Expert assessments are frequent-
ly combined to create composite governance indexes that summarize multi-concept and
complex phenomena into rankings and reference data. There are, however, reasonable con-
cerns on how representative a sample of experts is of the universe of people with knowledge
in the matter of interest (González, Fleischer and Mira d’Ercole, 2017) as well as the degree
of validity and reliability of data obtained through expert assessments (Kaufmann and Kray,
2007).8

31. In addition to the conventional sources described above that are regularly employed by statis-
ticians to measure governance, technological advancements have given rise to new data sources that
are of high volume, velocity and variety. This information is commonly generated by a considerable
number of data producers of a great heterogeneity (MacFeely, 2019). These sources are often referred
to simply as “big data”, but in fact include a variety of different sources and techniques, such as satel-
lite and aerial images, crowdsourcing and web scraping (see the UN Big Data Global Working Group
inventory for a rich set of examples). The common and differentiating feature of these sources is their
high reliance on technologies not frequently employed in the statistical field a few years ago. Overall,
sources based on new technologies bring opportunities for tackling challenges associated with more
conventional sources of governance statistics. For instance, since the underlying data is often auto-
matically collected, the focus shifts from spending resources on raw data generation to managing and
curating the data (IAEG, 2014). Not surprisingly, national and international statistical organizations are
dedicating more resources to the development of these projects. However, data sources based on
new technologies have shortcomings of their own, often associated with their accessibility and the
representativeness of the population that they cover. Thus, they are not yet at a stage where they can
replace conventional statistical sources.

32. Finally, as has been noted above, data from non-official sources (which may come in any of the
above data types) remains highly prevalent in the field of governance statistics. While this Handbook is
clear in that it encourages more government-led production of governance statistics, it also acknowl-
edges the value and usefulness of many of the non-official data sources. Needless to say, all sources
of governance data, whoever produces them, should be assessed for their quality based on common
criteria. In the case of non-official statistics, statistical offices should try to review their methodological

8 For instance, a comparison of household surveys in eight African countries on experiences on corruption with a mirror survey conducted among experts
shows a systematic overestimation of corruption by the experts compared to the everyday experiences of the population as well as a lack of a significant
correlation in the aggregate country results (Razafrindrakoto and Roubaud, 2010) .

HANDBOOK ON GOVERNANCE STATISTICS | 15

 standards and explore the possibility of working jointly with NGOs in the collection of data. In that
sense, non-official governance statistics have more of a complementary rather than a substitutive role.
The case of the Netherlands is illustrative of good practice in the adoption of this data following these
guidelines. Starting with a coverage of 30% of the SDG indicators in 2016, the Dutch National Statistics
Office (CBS) embarked on an extensive process of consultation with 30 different data-producing or-
ganizations, many coming from civil society with a record of independence and being responsible
with data protection. This consultation led to a significant number of supplementary data that met a
set of criteria and guaranteed compliance with standards of data produced by CBS. The result of this
process was a rise to 51% in coverage of the SDG indicators (Statistics Netherlands, 2018).

33. Needless to add, the above data sources do not necessarily present an either-or decision. Good
statistical practice suggests the use of multiple sources where possible, as a way of data triangulation
and enhancing measurement. For instance, a comparison of statistics from victimization surveys to
administrative data on crime may not only lead to a better understanding and validation of one’s data,
but also contribute to greater conceptual harmonization between both sources. An illustrative case of
multi-source data use is provided in Box I.3.

BOX I.3 Public transportation in Mexico City:

A successful case for the complementarity of sources

An example of how a multitude of sources coming from various data producers works to guarantee robust
governance statistics and innovative public policy is public transportation in Mexico City. Mexico City is one of
the largest cities in the world (almost 21 million population according to the Mexican NSO – INEGI - estimation
of 2017). An estimated 34 million trips are done every day in Mexico’s capital by people over age six. Obtaining
a complete and interdisciplinary picture of public problems related to urban mobility motivated the local
government to open a public “hackathon”, in which administrative records of trips (both historical and in real-
time) were made available for various forms of public transportation. In addition, a private insurance company
published in open-data format its records on traffic incidents through the hackathon platform. In addition to
this information, there is an existing origin-destination survey of households as well as a survey that estimates
levels of satisfaction with public transportation (both carried out by INEGI).

The initiative motivated interesting bottom-up public policy recommendations by participants, which were
designed to improve the mobile application of the police, to provide incentives for the use of green-transport
alternatives and to enhance the timely communication of delays in public transportation.

The case of Mexico City is illustrative of similar efforts undertaken in countries such as Colombia, which has
organized hackathons in several cities for a few years. There is also a strong potential for other countries to
employ this multi-method approach, considering both the availability of origin-destination surveys and that
transport and mobility were until 2018 the third most common topic for big-data projects registered by national
and international organizations in the Big Data Project Inventory of the UN Working Group.

Source: MacFeely, 2019 "The (Big) Data Bang: Opportunities and challenges for compiling SDG indicators". In Global Policy, Vol. 10, Supp.1, January.

34. Table I.1 summarizes the main practical and methodological strengths and limitations of the
different data source types. The fourth column also shows which specific sources are best suited to
produce indicators according to the categorization of the structures, processes and outcomes.

16 |
TABLE I.1 Sources for Governance Statistics (strengths, limitations, indicators and examples)

Source Strengths Limitations Best fit for measuring Example

Surveys 1. Often the most efficient way to obtain 1. Social desirability bias: people often Structures Corruption surveys as
(household a representative sample of a population answer what is expected (i.e., corruption May estimate endorsement a rigorous source to
and business of interest (people, businesses or others). involvement, electoral participation). by society to principles such estimate certain forms
and others) as democracy/Rule of Law of corruption, such as
2. Surveys can register events not 2. Memory-decay reporting
and rejection of practices bribes (chapter VIII).
present in government records bias (based on the experiences
such as corruption.
(e.g., unreported crime). remembered by the respondent, not
Trust in institutions
the actual number of events).
3. Ideal for perception-measurement Outcomes and in other people
(reduced bias in comparison with other 3. Not everyone interprets questions the Only citizens can judge (chapter IX).
sources), but also fit for obtaining same way, social constructs and their the actual coverage of
objective data (i.e., the prevalence of perception and interpretation can vary government programmes
crime or corruption experiences). across a wide variety of people, especially and how these goods and
for those phenomena where judgments may services affect their lives
4. The sample of a survey can be and shape their decisions.
be most culture-bound such as satisfaction,
designed to provide multiple levels of
trust and government effectiveness.
disaggregation and still be representative. Perceptions
4. There are worldwide and differentiated Surveys provide an
5. Household surveys provide a voice
response rates for surveys. Causing a fall unfiltered channel to
for the weak, especially in developing
in accuracy and possible bias in results. attain opinions towards
countries where mechanisms for
representation are not yet consolidated. 5. Costs: surveys are expensive compared a specific topic.
to administrative records, 9 expert
6. Flexibility: surveys can change
assessments or new technology-based
according to different policy interests
methods and need specialized human
(unlike, say, administrative data).
capital to assure their quality.

6. Several levels of data

disaggregation can cause a loss of
statistical representativeness when
samples are of moderate size.

7. Possible bias if the sampling frame

excludes unusual residence groups (e.g.,
student living in university halls, migrants
living in temporary accommodation,
military personnel living in barracks).

Administrative 1. Reduce the burden on respondents. 1. May reflect underreporting to Processes Measures on open
records governments (e.g., high levels Objective data on data, public requests
2. Low-cost. Once in place, the
of non-reported crime). government operations. for information
generation of statistics depends
received and
only on the regular operation of 2. Recorded data may change because
processed are
government and not ad-hoc programs. of legal or administrative changes (e.g.
based on reviews
definition of certain types of crime).
3. They are in a privileged of administrative
position to control and identify 3. Demand high levels of statistical data (chapter V).
government procedures. capacity from institutions across the
State (not just NSOs) or a specialized Caseload- Issues,
4. They facilitate obtaining disaggregated events and outcomes
agency in charge of quality control.
statistics (thanks to larger “samples”). in the justice system
4. Having harmonized definitions across (chapter VI).
countries might be problematic. Building
classifications for international purposes
tends to be lengthy and costly.

9 Assuming statistical systems for the operation of government services that comply with minimum quality requirements are already in place.

HANDBOOK ON GOVERNANCE STATISTICS | 17

TABLE I.1 Sources for Governance Statistics (strengths, limitations, indicators and examples), CONT.

Source Strengths Limitations Best fit for measuring Example

Censuses 1. Universal coverage of unit of analysis. 1. Due to its universality, costs are high Structures 5. Socio-economic
compared to surveys and administrative Can measure the official and demographic
2. Data disaggregation can be
records. For this reason, the collection of adoption of norms data of the national
done extensively without worries
information is usually more time-sparse. and frameworks. population collected
of losing representativeness.
from a census can
2. Commonly lengthy and
3. Some census specifically in areas Processes then be used as a
burdensome for the informant.
of government are less costly than Estimate regular benchmark when
administrative data when statistical 3. They tend to be less frequent and operation of governments assessing the
systems are non-existent or not reliable. more lengthy operations reducing the in a precise way. representation in
availability of data and its timeliness. public service and
Outcomes administration of
Measure the delivery of traditionally excluded
government programmes groups (chapter IV).
according to official records.

Expert 1. For certain aspects of governance, 1. More often than household surveys, they Structures Compliance of national
Assessments experts are best placed to can reflect ideological biases, the agenda Experts tend to have legal frameworks
provide an informed opinion. of a sponsoring organization or irrelevant expertise evaluating with internationally
considerations for governance (i.e., elements that form agreed standards
2. Experts assessment can provide
economic performance of a country may complex concepts. on civil and political
qualitative information of interest
have pervasive effects in other dimensions). rights (chapter III).
(e.g. all possible reasons for an event,
Processes
or knowledgeable explanations). 2. There is no comprehensive sampling
To measure processes often Corruption country
frame of all potential experts in
3. Indicators based on expert assessments requires expertise that only rankings such as the
a subject, therefore cannot be
from a standard template facilitate experts or civil servants of Corruption Perceptions
considered as statistical indicators.
better cross-country comparisons. a certain rank may have. Index (chapter VIII).
3. Their results are based on the strong
4. Less costly to collect, especially when
assumption that there is no correlation
compared to data collected through
between the errors (in simple terms,
household/business surveys or census.
that there is no “herd behavior”).
5. They synthesize complex concepts into
4. Issues with reliability and validity
simple figures. Thus, they excel to raise
when compared with other sources.
awareness or provide benchmarks.
5. Lack of transparency in the
criteria to choose experts.

Sources based on 1. Enable lower-cost monitoring of 1. It is not always feasible to assess Processes & Outcomes (Web-based)
new technologies public services demands by citizens the relationships between the covered These sources may systematic gathering
Crowdsourcing/ and delivers more systematized population and the target population on estimate the operation of of media and
Webscraping/ information in terms of data. the internet. Thus, it is challenging to government and the delivery other reports on
Big Data/ estimate bias for this kind of sources. of goods and services conflict-related
2. Facilitates data reports in
Satellite and in automatized ways. fatalities (chapter X).
real-time. Thus, data timeliness 2. Risks regarding privacy and security.
aerial sources/
represents an advantage.
Text mining 3. Data quality and representativeness
3. Fewer resources are needed to are highly dependent on the
produce data in favor of more for specific application domain.
managing and curating statistics.
4. Much of it is proprietary and
not accessible by the public.

5. There might be a lack of skill on

how to benefit from big data or other
sources related to new technologies.

Source: Own elaboration from various sources.

18 |
HANDBOOK ON GOVERNANCE STATISTICS | 19
 PART A
CROSS-CUTTING ISSUES

CHAPTER II
Human rights:
A cross-cutting
normative
framework for
governance
statistics
 II.1 Why do human rights matter for the development of governance statistics?

35. The international concept of governance has evolved over time from a narrow focus on economic
growth and efficiency towards policies and institutions that can best promote fundamental freedoms,
genuine peoples’ participation and the realization of civil, economic, political and social rights, without
discrimination. A number of declarations and outcome documents of global conferences adopted by
Member States confirm this evolution and recognize the nexus and mutually reinforcing relationship be-
tween good governance and human rights.1 The connection between human rights and governance also
lies at the heart of the Praia City Group that developed this Handbook. The recommendations formulated
under the auspices of the United Nations Statistical Commission (2002) that led to the creation of this City
Group called for developing statistical indicators on both human rights and good governance.2

36. International recognition of the overlapping nature of the two concepts is the first reason for
placing human rights at the centre of the development of governance statistics. The second rea-
son is that human rights, as defined and elaborated in international law, provide an adequate
basis for further development of definitional and methodological frameworks of governance statistics.
Moreover, bridging human rights helps to expand the range of data producers and users, and to cre-
ate new partnerships, such as between National Statistical Offices (NSOs) and National Human Rights
Institutions (NHRI). Finally, given the risks associated with data (e.g. violation of the right to privacy),
human rights provide legal safeguards in the compilation and dissemination of statistical information.

II.2 What are human rights?

37. Human rights are legal entitlements considered inherent in all human beings, whatever their
sex, colour, national or ethnic origin, religion, language, place of residence or any other status. Their
aim is to protect the dignity of the person. They stem from human values that are common across
societies and must be respected regardless of economic, political or cultural systems. They have been
enshrined in the Universal Declaration of Human Rights (UDHR) adopted by the United Nations in
1948.3 They cover civil, cultural, economic, political and social rights, such as: the right to life, the right
to food, the right to liberty and security, the right to the enjoyment of the highest attainable standards
of physical and mental health, the right to participate in public affairs, the right to education, the
right not to be subjected to torture or to cruel, inhuman or degrading treatment or punishment, the
right to adequate housing, the right to freedom of opinion and expression, the right to work and fair
conditions of work, the right to a fair trial, the rights to peaceful assembly and association, the rights
to water and sanitation, or the right to non-discrimination and equality.

38. Human rights have also been codified in international human rights treaties, adopted by the
General Assembly of the United Nations and ratified by Member States across all regions:

• International Convention on the Elimination of All Forms of Racial Discrimination, 1966

• International Covenant on Civil and Political Rights, 1966
• International Covenant on Economic, Social and Cultural Rights, 1966
• Convention on the Elimination of All Forms of Discrimination against Women, 1979

1 For instance, the UN Human Rights Council’s resolutions 7/11 and 25/8 both recognize the importance of a conducive environment, at both the national
and the international levels, for the full enjoyment of human rights and fundamental freedoms and of the mutually reinforcing relationship between good
governance and human rights, and also that transparent, responsible, accountable and participatory government, responsive to the needs and aspirations
of the people, including women and members of vulnerable and marginalized groups, is the foundation on which good governance rests and that such a
foundation is an indispensable condition for the full realization of human rights, including the right to development (2008).
2 https://unstats.un.org/unsd/statcom/doc02/2002-26e.pdf
3 The UDHR is one of the most translated documents in the world. In April 2019, there were 520 different translations officially available on the website of
the Office of the United Nations High Commissioner for Human Rights (see https://www.ohchr.org/EN/UDHR/Pages/Introduction.aspx).

22 |
 •  onvention against Torture and Other Cruel, Inhuman or Degrading Treatment or Punishment,
C
1984
• Convention on the Rights of the Child, 1989
• International Convention on the Protection of the Rights of All Migrant Workers and Members
of Their Families, 1990
• Convention on the Rights of Persons with Disabilities, 2006
• International Convention for the Protection of All Persons from Enforced Disappearance, 2006

39. Each of these treaties has a Committee of independent and regionally representative experts
(the so-called Treaty Bodies) to monitor States parties’ implementation of the treaty.4 These inter-
national human rights treaties, together with interpretative practice by the Treaty Bodies, form the
backbone of international human rights law. General comments and other recommendations adopted
by these bodies provide both authoritative and practical guidance for the definition and operational-
ization of the rights and obligations enunciated in international human rights instruments. There are
other international instruments (e.g. declarations, guidelines, principles), such as the Declaration on
the Right to Development,5 that also contribute to the understanding, implementation and develop-
ment of international human rights law (UN General Assembly, 1986). There are also regional human
rights instruments, and most States have adopted constitutions or other laws that formally protect
human rights and freedoms.

40. The primary characteristics of human rights are their universality, indivisibility, interrelatedness,
interdependence and inalienability. In other words, all human rights are to be realized for all people
and at all times. Improvement in the enjoyment of a right should not be at the expense of the en-
joyment of other rights.6 For instance, the right to vote might be of little or no immediate interest to
someone who has nothing to eat. Similarly, the enjoyment of the right to education will also be of less
value for someone who will be discriminated against because of her sex, colour or disability on the
labour market.

41. Each right corresponds to one or several State obligations. For instance, enjoyment of the
right to free and compulsory primary education (UDHR, art. 26) requires States to adopt a plan of
action and non-discriminatory measures, allocate appropriate resources and monitor the quality of
education. Freedom from torture (UDHR, art. 5) requires among other things the formal prohibition by
the authorities of any act of torture, the establishment of accountability mechanisms and appropriate
conditions of detention. Given the importance of these obligations for human rights and their nexus
with governance or the conduct of public affairs, it is worth elaborating on their content.

42. A distinction is made between obligations that are immediate and obligations that may be
discharged progressively if resources are lacking. The obligation not to discriminate between different
population groups is considered an immediate obligation, requiring no resources. Realizing the rights
to water and sanitation to all will, however, usually require the provision of adequate resources to en-
sure accessibility, affordability, safety, acceptability, sufficiency and continuity.7 Whereas the full realiza-
tion of economic and social rights is likely to be achieved only progressively, States have an immediate
obligation to satisfy a “minimum essential level” of these rights and to take tangible steps towards
their full realization. States have a legal duty to demonstrate that all available resources, including, if
need be, resources obtained through international cooperation, are being used to realize these rights.

4 A type of treaty called an optional protocol supplements some of these treaties. Further information on the list of international human rights treaties and
optional protocols is accessible at https://www.ohchr.org/en/professionalinterest/pages/coreinstruments.aspx
5 https://www.ohchr.org/en/professionalinterest/pages/righttodevelopment.aspx
6 For a more detailed description of human rights characteristics and obligations, see OHCHR (2012), Human Rights Indicators: A Guide to Measurement
(pp. 10-11). https://www.ohchr.org/Documents/Publications/Human_rights_indicators_en.pdf
7 See, for instance, General Comment 15 on the right to water adopted by the UN Committee for Economic, Social and Cultural Rights (E/C.12/2002/11).

HANDBOOK ON GOVERNANCE STATISTICS | 23

 This fundamental principle is referred to as the “use of maximum available resources”.8 Similarly, any
retrogressive measures (e.g. halting adjustment for cost-of-living increases for pensions of the elderly)
would have to be duly justified and assessed from a perspective of maximum available resources.

43. Three main obligations are distinguished in international human rights law. These are the
obligations to respect, protect and fulfil. The obligations to respect and protect are seen essen-
tially as immediate obligations. A violation of the obligation to respect would be, for instance, the
use of unnecessary or disproportionate force by the police against demonstrators. An example
of a breach of the obligation to protect would be a State failing to ensure that private employers
comply with the right to just and favourable conditions of work. The obligation to fulfil relates
to obligations to promote (e.g. adoption of legal, institutional and policy instruments, aware-
ness-raising campaigns) and to provide (e.g. allocation of appropriate resources and provision of
public or private services). Table II.1 summarizes the three main human rights obligations.

BOX II.1 Obligations to respect, protect and fulfil Human Rights

RESPECT PROTECT FULFIL

Duty-bearers must Duty-bearers must prevent Duty-bearers must take

refrain from interfering private actors or third positive measures,
with the enjoyment parties from interfering including adopting and
of human rights with the enjoyment implementing appropriate
of human rights legislations, policies and
programmes, to ensure the
realization of human rights

II.3 Statistics and the international human rights normative framework

44. The link between human rights and statistics is multifaceted indeed. Sound statistics are essen-
tial to the measurement of human rights. For a long time, statisticians, sometimes without knowing
it, have helped measure the realization of economic and social rights (e.g. the right to education, the
right to health, the right to work). Conversely, their role in the measurement of rights more traditionally
referred to as civil and political rights (e.g. the right to participate in public affairs, the right to personal
security and the right to a fair trial) has been more recent.

45. In addition to statisticians measuring the realization of rights “incidentally” through their work,
statistics are also produced and reported on specifically with the aim of assessing the implementation
of human rights standards. The role and relevance of statistical information in the measurement and
realization of human rights has long been recognized in international human rights law and in the work
of international human rights mechanisms, such as the Treaty Bodies, the Universal Periodic Review
(a peer review of the human rights performance of all UN Member States under the auspices of the
Human Rights Council) and Special Procedures (independent human rights experts with a mandate
to report and advise the Human Rights Council on human rights from a thematic or country-specific

8 See article 2 of the International Covenant on Economic, Social and Cultural Rights: “Each State Party to the present Covenant undertakes to take steps,
individually and through international assistance and co-operation, especially economic and technical, to the maximum of its available resources, with a
view to achieving progressively the full realization of the rights recognized in the present Covenant by all appropriate means, including particularly the
adoption of legislative measures” (UN General Assembly, 1966).

24 |
perspective).9 States provide statistical information to these mechanisms and other interlocutors, like
civil society organizations, which is then used by all stakeholders for assessing progress of human
rights compliance over time and across population groups. Statistical information is explicitly referred
to in international human rights treaties. For instance, article 31 of the Convention on the Rights of
Persons with Disabilities (A/RES/61/106), entitled “Statistics and data collection”, clearly spells out the
role and manner of producing and disseminating statistical information for the realization of human
rights:

1) S tates Parties undertake to collect appropriate information, including statistical and

research data, to enable them to formulate and implement policies to give effect to the
present Convention. The process of collecting and maintaining this information shall:
(a) Comply with legally established safeguards, including legislation on data protec-
tion, to ensure confidentiality and respect for the privacy of persons with disabilities;
(b) Comply with internationally accepted norms to protect human rights and fundamental
freedoms and ethical principles in the collection and use of statistics.
2) The information collected in accordance with this article shall be disaggregated, as appro-
priate, and used to help assess the implementation of States Parties' obligations under the
present Convention and to identify and address the barriers faced by persons with disabilities
in exercising their rights.
3) States Parties shall assume responsibility for the dissemination of these statistics and ensure
their accessibility to persons with disabilities and others.

46. Enshrined in international human rights law, the right to be registered immediately after birth10
is a major contribution to the development of population registers and vital statistics. Respecting this
right is often a condition for the enjoyment of other rights (e.g. education, social security, to vote,
etc.). With regard to the coverage and visibility of groups in disaggregated statistics, the registration
of all children serves as an acknowledgment of their importance as individuals and of their recognition
in legal and policy frameworks.

47. International human rights mechanisms have issued recommendations to States about the need
for using statistical indicators, including data that are disaggregated according to grounds of discrim-
ination prohibited by human rights law, in monitoring and reporting human rights compliance. It is
easy to find these recommendations online in the Universal Human Rights Index.11

48. People’s entitlement to statistical information, treasured in the first principle of the Fundamental
Principles of Official Statistics,12 is also an attribute of a broader right to information and freedom of
expression enshrined in article 19 of the Universal Declaration of Human Rights. In short, statistics
matter for human rights, and human rights matter for statistics.

49. According to its Charter, human rights are one of the pillars of the United Nations, along with
development, peace and security.13 The 2030 Agenda for Sustainable Development,14 adopted by
all heads of States and Governments in 2015, explicitly seeks to realize the human rights of all, and

9 To see all human rights bodies and their functions, see https://www.ohchr.org/EN/HRBodies/Pages/HumanRightsBodies.aspx
10 Articles 24 (2) of the International Covenant on Civil and Political Rights and 7 (1) of the Convention on the Rights of the Child.
11 https://uhri.ohchr.org/en. Using this tool, a “basic search”, applying the “Statistics and Indicators” theme, revealed not less than 2218 recommen-
dations, namely an average of more than 10 by country (as of 13 June 2019). These recommendations are also relevant to the implementation and
measurement of the 2030 Agenda for Sustainable Development.
12 Principle 1 of the Fundamental Principles of Official Statistics (A/RES/68/261), accessible at: https://unstats.un.org/unsd/dnss/gp/fundprinciples.aspx
13 The late Kofi Annan, former Secretary-General of the United Nations (1997-2006), said that his experience taught him that “there can be no long-term
development without security and no long-term security without development. Nor will any society remain prosperous for long without the rule of law and
respect for human rights" (“The next steps for Africa to meet its potential”, The Washington Post, 13 August 2014. Retrieved 2 April 2019).
14 A/RES/70/1 accessible at: https://sustainabledevelopment.un.org/post2015/transformingourworld

HANDBOOK ON GOVERNANCE STATISTICS | 25

 BOX II.2 H
 uman Rights and the data revolution

“Any legal or regulatory mechanisms, or networks or partnerships, set up to mobilize the data revolution for
sustainable development should have the protection of human rights as a core part of their activities, specify who is
responsible for upholding those rights, and should support the protection, respect and fulfilment of human rights”

- UN Report on the Data Revolution for Sustainable Development (IEAG, 2017)

Member States have committed to implement the Agenda in a manner consistent with the rights and
obligations of States under international law.15 The overarching calls in the Agenda to leave no one
behind, eliminate discrimination and reduce inequality, including gender inequality, and the related
commitment to improve the disaggregation of indicators – by income, gender, age, race, ethnicity,
migratory status, disability, geographic location and other characteristics relevant in national contexts
– open new opportunities as well as new challenges for upholding human rights. It requires therefore
novel approaches and partnerships in statistics that can be brought together under the framework of
a Human rights-based approach to data (HRBAD).

II.4 Human rights-based approach to data

50. International human rights norms and principles provide legal safeguards and guidance for the
development of statistical information and national and international statistical systems more gen-
erally. The UN guidance on HRBAD outlines the main human rights norms and principles relevant to
the development and use of indicators, in particular for data collection and disaggregation. HRBAD
provides timely guidance for supporting efforts in the 2030 Agenda for Sustainable Development to
leave no one behind and ensure the protection of human rights in doing so.

FIGURE II.1 Main principles of the Human Rights-Based Approach to Data

ion
ti c i p at
Pa r Dis
ag
gre
g
at i
on
ilit y
Ac c o u n t a b

Self-identi
ficat
on i
y
ac

v
Pr i
Tran
sp ar e
nc y

15 Ibid., paragraph 18.

26 |
TABLE II.1 Main principles of the Human Rights-Based Approach to Data16

PARTICIPATION • Participation of relevant population groups in data collection exercises, including the planning, collection, dissemination and analysis
of data, is central to HRBAD.
• 
Participation is instrumental to the realization of the other components of the approach and in increasing trust and use of relevant
statistics.
• 
“Nothing about us without us” is a well-known motto in human rights approaches and is enshrined in international human rights
treaties (e.g. article 31 of the Convention on the Rights of Persons with Disabilities).
• 
Groups left behind or at risk of discrimination, civil society organizations, National Human Rights Institutions (see SDG indicator 16.a.1)
and other relevant organisations, provided they are competent to represent the group’s interest, should be able to participate in data
collection efforts.
• 
Participation, in particular for the most vulnerable, marginalized or discriminated groups, should be free, active and meaningful.17

DISAGGREGATION • 
Inclusive data collection and disaggregation that allow for comparison of population groups are central to HRBAD and form part of
human rights obligations to monitor the extent of possible inequality and discrimination, including multiple and intersecting forms.
• 
Disaggregation by grounds of discrimination prohibited by international human rights law such as: sex, age, colour, ethnicity, migration
or displacement status, disability, religion, civil status, socio-economic status, sexual orientation, gender identity, place of residence
and other status.
• 
Going beyond traditional data collection to cover the homeless population (including street children), travellers or nomadic populations,
undocumented migrants and persons in institutions (e.g. orphans, elderly, prisoners).
• 
Measuring national averages and revealing the most deprived population groups and quantifying the extent of inequality.

SELF-IDENTIFICATION • 
O verriding “do no harm” principle to be always respected.
• 
Respect and protection of personal identity is central to human dignity.
• 
Q uestions about personal identity characteristics (e.g. ethnicity, religion, sexual orientation and gender identity) should be voluntary.
“Other” or “non-response” options should also be provided.
• 
A pplying gender and culturally sensitive data collection approaches.

TRANSPARENCY • 
O fficial statistics are part of the public’s right to information, including of relevant statistical information (see article 19 on freedom of
expression in the International Covenant on Civil and Political Rights and principle 1 of the Fundamental Principle of Official Statistics).
• 
Information about how data is collected should be publicly available (e.g. metadata).
• 
Data should be disseminated as quickly as possible after collection.
• 
Data should be accessible, considering disabilities, language, literacy/numeracy levels and diverse cultural backgrounds.

PRIVACY • 
Individual data collected to produce statistical information must be strictly confidential and used exclusively for statistical purposes
and regulated by law (see article 17 of the International Covenant on Civil and Political Rights and principle 6 of the
Fundamental Principles of Official Statistics).
• 
Data should not be published or publicly accessible in a manner that permits identification of individual data subjects,
either directly or indirectly.
• 
Data should be secured against both natural and human dangers and disposed of appropriately when no longer required.
• 
Data protection should also be ensured in the context of big data and other data collection by non-State actors.

ACCOUNTABILITY • 
A ccountability in HRBAD refers to data collection for accountability as well as accountability in data collection.
• 
National Statistical Offices are human rights duty-bearers and are accountable for respecting, protecting and fulfilling human rights.
• 
Data can, and should, be used to hold human rights actors to account.
• 
P utting collected data back in the hands of disadvantaged population groups and strengthening their capacity to use
them is essential for accountability.

16 Further information on HRBAD is accessible at www.ohchr.org/HRBAD

17 In a manner consistent with the Declaration on the Right to Development adopted by UN General Assembly resolution 41/128.
See: https://www.ohchr.org/en/professionalinterest/pages/righttodevelopment.aspx

HANDBOOK ON GOVERNANCE STATISTICS | 27

 51. Statisticians working at national and international levels have pointed out the consistency of
HRBAD with the fundamental principles of official statistics. They have also underlined the value of
the international human rights normative framework to strengthen the implementation of the UN
Fundamental Principles of Official Statistics (A/RES/68/261). The link between human rights and sta-
tistics has been explicitly recognized in both international human rights treaties (as described above)
and these Principles. Concerning the latter, they state that “the essential trust of the public in the
integrity of official statistical systems and confidence in statistics depend to a large extent on respect
for the fundamental values and principles that are the basis of any society seeking to understand itself
and respect the rights of its members”. The references to people’s entitlement to public information
(principle 1) and to data confidentiality (principle 6) have direct anchorages in international human
rights law.

52. The implementation of the national population census by the Kenya National Bureau of Statistics
(KNBS) in 2019 illustrates concrete steps taken by national stakeholders to operationalize HRBAD at
country level. KNBS collaborated with the Kenya National Commission on Human Rights (KNCHR) on
the design and implementation of the census. The census has facilitated the counting of groups that
had never been counted before (e.g. persons with albinism, indigenous peoples, intersex people, the
stateless population); these are groups particularly at risk of being left behind, in the words of the 2030
Agenda for Sustainable Development (Bearak and Ombuor, 2019; OHCHR, 2019).

BOX II.3 Examples of initiatives to operationalize a HRBAD at country level

Cooperation agreements were signed between the statistics office and the national human rights institution
of Kenya and the State of Palestine, respectively, to support the operationalization of the HRBAD. The primary
objective of this collaboration is to systematize identification and data collection on groups potentially left
behind.

“Our signing of a Memorandum of Understanding (MoU) with the Kenya National Commission on Human Rights
(NCHR) is a great milestone in the development of official statistics. We look forward to ensuring that the
official statistics we generate allow for effective identification, planning and evidence-based policymaking for
all groups,”
Zachary Mwangi, Director General of the Kenya National Bureau of Statistics.

“Human rights advocacy is the primary mandate of national human rights institutions and entails persuading
governments to act in compliance with their human rights obligations. This work is advanced when advocacy is
backed up with credible data. A partnership guarantees that the NCHR builds its capacity in terms of collecting
credible data and that the statistics body expands the lens that it uses to collect data to incorporate traditionally
marginalized groups.”
Patricia Nyaundi, the former Secretary of Kenya National Commission on Human Rights.

“Signing the MoU with the Independent Commission for Human Rights (ICHR) is part of our strategy to
strengthen the national statistical system and provide the data needed to realize the commitment to leaving
no one behind in the implementation of the 2030 Agenda. It is absolutely crucial for us to join forces to address
the data challenges of revealing the situation of the most vulnerable and marginalized groups in Palestine.”

Dr. Ola Awad, President of the Palestinian Central Bureau of Statistics.

“Since the signing of the MoU, ICHR has played an important role in providing guidance on the development
of the planned domestic violence survey and the identification and development of human rights indicators
and related SDG indicators for measuring Palestine's implementation of the Convention for the Elimination of
Discrimination Against Women.”
Dr. Ammar Dweik, General Director of the Independent Commission for Human Rights

28 |
HANDBOOK ON GOVERNANCE STATISTICS | 29
 PART B
EIGHT DIMENSIONS OF
GOVERNANCE STATISTICS

CHAPTER III
Non-
discrimination
and equality
 III.1 Conceptualizing this dimension
III.1.1 What is discrimination?
53. Non-discrimination and equality are fundamental standards of international human rights law
and a core dimension of good governance. Negative forms of discrimination undermine the reali-
zation of civil, cultural, economic, political and social rights for many population groups. Economic
growth, by itself, does not ensure sustainable development, and individuals and groups continue to
face various inequalities, often resulting from or contributing to the entrenchment of certain forms of
discrimination.

54. Under international human rights law, discrimination is any distinction, exclusion, restriction or
preference or other differential treatment that is directly or indirectly based on the prohibited grounds
of discrimination (e.g. colour, sex, language, religion, national or social origin, disability), and which
has the intention or effect of nullifying or impairing the recognition, enjoyment or exercise, on an
equal footing, of human rights and fundamental freedoms in the political, economic, social or cultural
fields or any other field of public life.1

55. Elimination of discrimination is enshrined in the Universal Declaration of Human Rights and in all
international human rights treaties. Under the 2030 Agenda for Sustainable Development, Heads of
States and Governments have made resounding commitments to leave no one behind and eliminate
discriminatory laws, policies and practices and all forms of discrimination against women and girls.
The Agenda also emphasizes the responsibilities of all States, in conformity with the Charter of the
United Nations, to respect, protect and promote human rights and fundamental freedoms for all,
without distinction of any kind as to race, colour, sex, language, religion, political or other opinion,
national or social origin, property, birth, disability or other status.2

56. Discrimination can occur both formally and substantively. Formal discrimination or de jure dis-
crimination may be found in laws and policy documents (e.g. a law denying equal social security bene-
fits to women on the basis of their marital status). Substantive discrimination refers to discrimination in
practice or de facto. This refers to behaviours, attitudes and conditions against individuals or groups,
preventing them from having equal access to adequate housing, health care, judicial mechanisms,
education, employment and other services or opportunities.3

57. Discrimination also encompasses direct and indirect discrimination. Direct discrimination occurs
when an individual is treated less favourably than another person in a similar situation for a reason
related to a prohibited ground (e.g. job applicants with a certain skin colour or ethnic origin are sys-
temically excluded). Indirect discrimination refers to laws, policies or practices that appear neutral at
face value, yet are discriminatory for population groups with certain characteristics (e.g. requirement
of a birth registration certificate for school enrolment may discriminate against ethnic minorities or
non-nationals who do not possess, or have been denied, such certificates).4

1 UNESC (2009), General Comment 20, Non-discrimination in economic, social and cultural rights (E/C.12/GC/20).
2 A/RES/70/1, paragraph 19. The combined reference to the principles of non-discrimination and equality in this chapter is consistent with the internatio-
nal human rights normative framework and draws upon existing measurement practices. Non-discrimination and equality are fundamental components
of international human rights law and are essential to the exercise and enjoyment of human rights by everyone (see, for instance, General Comment No
20 of the Committee of Economic, Social and Cultural Rights, E/C.12/GC/20). Paragraph 19 of the 2030 Agenda replicates article 2, paragraph 2 of the
International Covenant on Economic, Social and Cultural Rights (ICESCR) and article 2, paragraph 1 of the International Covenant on Civil and Political
Rights (ICCPR). In terms of measurement practices, inequality indicators, using for instance data disaggregated by sex, income, disability and other
relevant characteristics, are essential for assessing disparities resulting (even if only partly) from historical or contemporaneous, direct or indirect, forms of
discrimination.
3 See General Comment 20 (ibid.) and General recommendation No. 28 of the Committee on the Elimination of Discrimination against Women
(CEDAW/C/GC/28) on the core obligations of States parties under article 2 of the Convention on the Elimination of All Forms of Discrimination against
Women.
4 Ibid.

32 |
 58. Harassment because of some real or perceived characteristics (e.g. sex, gender identity, social
origins) of the victim is also considered a form of discrimination by international human rights mech-
anisms. Harassment can be defined as any improper and unwelcome conduct that might reasonably
be expected or be perceived to cause offence or humiliation to another person. Harassment may
take the form of words, gestures or actions that tend to annoy, alarm, abuse, demean, intimidate, be-
little, humiliate or embarrass another or that create an intimidating, hostile or offensive environment.
Harassment can be of a sexual nature. While typically involving a pattern of behaviour, harassment can
take the form of a single incident5.

59. There are important ramifications flowing from the concept and measurement of discrimina-
tion and other thematic domains that are also the subject of statistical investigations, such as hate
crime, hate speech, gender-based violence, stigma, stereotypes or bias. Hate crimes refer to crimes
(e.g. homicide, assault, theft) in which victims are specifically targeted because of their character-
istics, ascribed attributes, beliefs of values (e.g. race, religion, ethnic origins, sexual orientation or
disabilities). Hate speech can be broadly defined as derogatory, attacking, prejudiced or otherwise
negative comments about individuals because of some traits (e.g. religious or other beliefs or absence
of beliefs, national origin or disability).6 Violence against women or gender-based violence is also a
form of discrimination. The Committee on the Elimination of Discrimination against Women defines
gender-based violence as “violence which is directed against a woman because she is a woman or
that affects women disproportionately”. 7 Stigma, bias and negative stereotypes relate to prejudicial
views, attitudes, beliefs or assumptions held by an individual toward a particular population group
and which may consciously or unconsciously influence the individual’s behaviour towards members of
that group. The existence of stigma, stereotypes and bias is generally a factor in discriminatory acts.

60. It is important to keep in mind that discrimination may be permissible under certain circum-
stances (e.g. affirmative action or positive measures to remedy the historical, systemic or embed-
ded marginalization of discrete minorities). However, justification for differential treatment based on
prohibited grounds has to be reasonable, objective and conform with human rights standards. For
instance, international human rights mechanisms recommend the adoption of temporary special
measures (e.g. quotas) to eliminate discrimination and accelerate equality. 8

III.1.2 What are the prohibited grounds of discrimination?

61. Lists of prohibited grounds of discrimination can be found in the international human rights
normative framework, starting with the Universal Declaration of Human Rights. The Declaration lists
race, colour, sex, language, religion, political or other opinion, national or social origin, property, birth
or other status, as prohibited grounds of discrimination. The list is not complete and ends with a
mention of “other status”. It recognized and anticipated that other grounds of discrimination can
prevail and should also be prohibited (e.g. persons with albinism, HIV/AIDS, etc.). Based on the review
of international human rights law, a more complete, albeit non-exhaustive, list of prohibited grounds
of discrimination includes: race,9 colour, sex, language, religion, political or other opinion, national
origin, social origin, property, birth status, disability, age, nationality, marital and family status, sexual

5 Websites of national human rights institutions or other equality bodies often provide examples of incidents of discrimination and harassment typically
occurring in their countries (see, for instance, the examples provided by the Equality and Human Rights Commission of Great Britain: https://www.
equalityhumanrights.com/en).
6 Definition provided in the International Classification of Crime for Statistical Purposes (ICCS), version 1.0.
7 General recommendation No. 35 on gender-based violence against women, updating general recommendation No. 19, Committee on the Elimination
of Discrimination against Women, CEDAW/C/GC/35.
8 See temporary special measures in, for example, article 4 of the Convention on the Elimination of Discrimination against Women, and General recom-
mendation No. 25 of the Committee on the Elimination of Discrimination against Women on article 4, paragraph 1, of the Convention.
9 Importantly, the use of the term “race” in the international human rights normative framework does not imply the acceptance of theories that attempt
to determine the existence of separate human races. In this regard, the 2009 outcome document of the Durban Review Conference (para. 6) did reaffirm
“that all peoples and individuals constitute one human family, rich in diversity, and that all human beings are born free and equal in dignity and rights; and
strongly rejects any doctrine of racial superiority along with theories which attempt to determine the existence of so-called distinct human races.”

HANDBOOK ON GOVERNANCE STATISTICS | 33

 orientation, gender identity, health status, place of residence, economic and social situation, pregnan-
cy, indigenous status and other status.10

III.2 Why is this dimension important?

62. Consistent with an international concept of governance that evolved from a narrow economic
focus to broader human rights-related issues, non-discrimination and equality is a fundamental dimen-
sion of (good) governance. The commitments to “leave no one behind”, “eliminate discrimination”
and “achieve gender equality” in the 2030 Agenda for Sustainable Development call for the develop-
ment and use of indicators helping to measure direct and indirect discrimination. Also, several targets
of the Agenda are about non-discrimination and equality, including:

•  arget 4.5 “Eliminate gender disparities in education and ensure equal access to all levels
T
of education and vocational training for the vulnerable, including persons with disabilities,
indigenous peoples, and children in vulnerable situations”;
• Target 5.1 “End all forms of discrimination against all women and girls everywhere”;
• Target 10.3 “Ensure equal opportunity and reduce inequalities of outcome, including by
eliminating discriminatory laws, policies and practices and promoting appropriate legislation,
policies and action in this regard”.
• Target 16.b “Promote and enforce non-discriminatory laws and policies for sustainable
development”.

63. Non-discrimination and equality

BOX III.1 The Universal Declaration of
are really at the heart of international
Human Rights, 1948 (excerpts)
human rights law, starting with the
Universal Declaration of Human Rights
stating that everyone is entitled to all Article 1. All human beings are born free and equal
in dignity and rights. They are endowed with reason
the rights and freedoms set forth in and conscience and should act towards one another in a
the Declaration, without distinction of spirit of brotherhood.
any kind. Discrimination prevents vic- Article 2. Everyone is entitled to all the rights and
tims from fully enjoying their human freedoms set forth in this Declaration, without distinction
rights and fundamental freedoms. of any kind, such as race, colour, sex, language, religion,
Against this background, there is a le- political or other opinion, national or social origin,
property, birth or other status.
gal requirement to monitor and report
on discrimination and equality, paying
attention to the populations most at
risk. One good example is the Convention on the Rights of Persons with Disabilities, ratified or signed
by 189 States,11 that stipulates that appropriate information, including statistical and research data, is
to be collected to formulate and implement policies to give effect to the treaty.12

10 Further to the UDHR, international human rights treaties adopted by the United Nations General Assembly, include treaties on specific population
groups: the International Convention on the Elimination of All Forms of Racial Discrimination, Convention on the Elimination of All Forms of Discrimination
against Women, Convention on the Rights of the Child, International Convention on the Protection of the Rights of All Migrant Workers and Members
of Their Families, and the Convention on the Rights of Persons with Disabilities https://www.un.org/disabilities/documents/convention/convoptprot-e.
pdf. For more information on prohibited grounds of discrimination: http://www.ohchr.org/Documents/Issues/HRIndicators/HumanRightsStandards.pdf
11 Information on 19 June 2019, accessible at: http://indicators.ohchr.org/
12 Convention on the Rights of Persons with Disabilities, Article 31, available at: https://www.un.org/development/desa/disabilities/

34 |
III.3 Data and best practices currently available

64. Measuring discrimination calls for the use of multiple indicators, data sources and methods. An
indicator of the number of convictions for discrimination in courts, using administrative records, will
be useful, yet insufficient for assessing the prevalence of discrimination in a country, as some victims
may not report discrimination. Household surveys reaching out directly to the population to assess
the extent to which they may have been personally a victim of discrimination are therefore necessary.
These survey-based indicators will however not provide a legal determination of any reported cases
of discrimination. Indicators, based on expert assessments, on the existence of discriminatory laws
and programmes will help complete assessments of the prevalence of discrimination. In line with the
recommended framework for governance and human rights statistics, it is therefore necessary to draw
on configurations of structural, process and outcome indicators, and to use multiple data sources.

III.3.1 Understanding discrimination requires a comprehensive

view of both direct and indirect discrimination BOX III.2 I dentifying non-discrimination
indicators in Bolivia
65. As detailed above, non-discrimination and equality are
complex issues that go well beyond the existence of legislation
or the absence of harassment. Hence, taking a comprehensive In Bolivia, the Ministry of Justice and the national
statistical office (INE) led a process of identification of
approach to measuring and understanding non-discrimination indicators relevant to human rights, including indicators
and equality is an important best practice. Comprehensively relevant to non-discrimination. Drawing on the list of
identifying all indicators relevant to non-discrimination at the illustrative tables of human rights indicators developed
at international level, nationally relevant and available
national level may often involve a consultation and contextualiza- indicators on several rights were identified through
tion processes, as illustrated in Box III.1 on Bolivia. participatory processes and populated based on
available information.1 National statistical offices of other
66. Table III.1 provides an illustrative list of statistical indicators, countries have also engaged in similar national processes
of contextualizing the tables of illustrative human rights
including but not limited to SDG indicators, that can be relevant indicators developed internationally. These processes
when measuring non-discrimination and equality (see also the are often led by national human rights institutions or
subsequent discussion in this chapter of the prioritization of mechanisms.
non-discrimination indicators). The table thereby showcases the
importance of considering both direct and indirect discrimina-
tion, as elaborated further in subsequent sections.13

13 Further examples of indicators relevant to the measurement of discrimination are provided in other tables of OHCHR human rights indicators:
https://www.ohchr.org/Documents/Issues/HRIndicators/SDG_Indicators_Tables.pdf

HANDBOOK ON GOVERNANCE STATISTICS | 35

TABLE III.1 List of illustrative indicators on non-discrimination and equality
(Universal Declaration of Human Rights, arts. 1, 2 and 7)

Direct or indirect discrimination by public and

private actors nullifying or impairing
Special measures,
Equality before the law and Access to an adequate standard Equality of livelihood including for participation
protection of the person of living, health and education opportunities in decision-making

Structural • International human rights treaties relevant to the right to non-discrimination and equality (right to non-discrimination) ratified by the State
• 
Date of entry into force and coverage of the right to non-discrimination, including the list of prohibited grounds of discrimination,
in the Constitution or other forms of superior law
• 
Date of entry into force and coverage of domestic laws for implementing the right to non-discrimination, including on the prohibition
of advocacy constituting incitement to discrimination and hatred
• Date of entry into force and coverage of legal act constituting a body responsible for promoting and protecting the right to non-discrimination
• Periodicity and coverage of the collection and dissemination of data relevant to assessing the implementation of the right to non-discrimination
• 
Number of registered or active NGOs and full-time equivalent employment (per 100,000 persons) involved in the promotion and
protection of the right to non-discrimination

• Time frame and coverage • 

T ime frame and coverage of • 
T ime frame and coverage • 
T ime frame and coverage of
of policy and programmes policy or programme for equal of policies for equal policy to implement special
to ensure equal protection, access to education at all levels access to decent work and temporary measures to
security and handling of ensure or accelerate equality
• 
T ime frame and coverage • 
T ime frame and coverage
crimes (including hate in the enjoyment of
of policy and programmes of policy for the elimination
crimes and abuse by law human rights
to provide protection from of forced labour and other
enforcement officials)
discriminatory practices abuse at work, including • 
Date of entry into force and
• 
Date of entry into force and interfering with access domestic work coverage of quotas or other
coverage of domestic laws to food, health, social special measures for
• 
P roportion of countries where
ensuring equal access to security and housing targeted populations in
the legal framework (including
justice and treatment legislative, executive,
customary law) guarantees
including for married, unmarried judicial and other
women’s equal rights to land
couples, single parents and appointed bodies
ownership and/or control
other target
[5.a.2] • 
Date of entry into force
groups
and coverage of legal
frameworks to promote,
enforce and monitor equality
and non‑discrimination on
the basis of sex [5.1.1]

Process • 
P roportion of received complaints on cases of direct and indirect discrimination investigated and adjudicated by the national human rights
institution, human rights ombudsperson or other mechanisms (e.g. equal opportunity commission) and the proportion
responded to effectively by the Government
• 
P roportion of targeted population (e.g. law enforcement officials) trained on implementing a code of conduct for the elimination
of discriminatory practices
• Proportion of countries with systems to track and make public allocations for gender equality and women’s empowerment [5.c.1]

36 |
TABLE III.1 List of illustrative indicators on non-discrimination and equality
(Universal Declaration of Human Rights, arts. 1, 2 and 7), CONT.

Direct or indirect discrimination by public and

private actors nullifying or impairing
Special measures,
Equality before the law and access to an adequate standard equality of livelihood including for participation
protection of the person of living, health and education opportunities in decision-making

Process • 
P roportion of victims of • 
Parity indices (female/male, • 
P roportion of enterprises • 
P roportion of targeted
discrimination and bias-driven rural/urban, bottom/top wealth (e.g., government contractors) population groups accessing
violence provided with legal aid quintile and others such as that conform with certified positive action or preferential
disability status, indigenous discrimination-free business treatment measures aiming to
• 
Number of persons (including
peoples and conflict-affected, and workplace practices (e.g. promote de facto equality (e.g.
law enforcement officials)
other target group) in primary no HIV test requirements) financial assistance, training)
arrested, adjudicated,
and higher education and by
convicted or serving a • 
P roportion of job vacancy • 
P roportion of education
kind of school (e.g., public,
sentence for discrimination announcements stipulating institutions at all levels
private, special school) [4.5.1]
and bias-driven violence that among equally qualified teaching human rights and
per 100,000 population • 
P roportion of health-care (or comparable) candidates promoting understanding
professionals [landlords] a person from a targeted among population groups
• 
P roportion of women reporting
handling requests from population group will be (e.g. ethnic groups)
forms of violence against
potential patients [tenants] selected (e.g. women, minority)
themselves or their children • 
P roportion of members of
in a non- discriminatory
initiating legal action or • 
P roportion of employers trade unions and political
manner (source: discrimination
seeking help from police handling applications of parties who are women
testing survey)
or counselling centres candidates in a non- or from other targeted
• 
P roportion of public buildings discriminatory manner (e.g. ILO population groups and the
• 
P roportion of requests for
with facilities for persons discrimination testing survey) proportion thereof presented
legal assistance and free
with physical disabilities as candidates for election
interpreters being met (criminal • 
P roportion of employees (e.g.
and civil proceedings) • 
P roportion of population using migrant workers) reporting
safely managed drinking water discrimination or abuse at
• 
P roportion of lawsuits
[6.1.1], sanitation services work who initiated legal
related to property where
[6.2.1], electricity [7.1.1] and or administrative action
women appear in person
waste disposal [11.6.1]
or through counsel as • 
P roportion of time spent on
plaintiff or respondent • 
Number of countries that have unpaid domestic and care work,
implemented well-managed by sex, age and location [5.4.1]
migration policies [10.7.2]

Outcome • 
P revalence/incidence of • 
P roportion of population in a • 
Unemployment rate, by • 
P roportion of relevant
crimes, including hate crime given age group achieving at sex, age and persons positions (e.g. managerial)
and domestic violence, by least a fixed level of proficiency with disabilities [8.5.2] in the public and private
target population group in functional (a) literacy and (b) sectors held by targeted
• 
Average hourly earnings of
numeracy skills, by sex [4.6.1] population groups [5.5.2]
• 
Reported cases of female and male employees,
arbitrary killing, detention, • 
Birth, mortality and life by occupation, age, persons • 
P roportion of seats in
disappearance or torture expectancy rates disaggregated with disabilities, and elected and appointed
from population groups by targeted population group other target groups bodies at subnational and
ordinarily subject to risk of local level held by targeted
discriminatory treatment population groups [5.5.1]
• 
C onviction rates for indigent
defendants provided with • 
P roportion of targeted populations below national poverty line
legal representation as a [1.2.1] (and Gini indices) before and after social transfers
proportion of conviction • 
(a) Proportion of total agricultural population with ownership or secure
rates for defendants with rights over agricultural land, by sex; and (b) share of women among
lawyer of their own choice owners or rights-bearers of agricultural land, by type of tenure [5.a.1]

• 
Reported number of victims of direct and indirect discrimination and hate crimes and proportion of victims (or relatives)
who received compensation and rehabilitation in the reporting period
• 
P roportion of population reporting having personally felt discriminated against or harassed in the previous 12 months
on the basis of a ground of discrimination prohibited under international human rights law [10.3.1/16.b.1]

HANDBOOK ON GOVERNANCE STATISTICS | 37

 III.3.2 Measuring direct discrimination

III.3.2.1 Administrative data and victimization surveys to measure hate crimes

67. Hate crime statistics are compiled through administrative records and victimization surveys. The
availability of robust crimes statistics from administrative sources means that law enforcement officials (e.g.
police officers) are systematically recording crime motives. They record cases where victims were specifi-
cally targeted because of their characteristics, perceived attributes or beliefs. The quality of hate statistics
rests on the quality of recording by law enforcement officials. To this end, mechanisms for recording hate
crime should follow standard operating procedures that provide law enforcement officials with the tools to
flag and record possible bias motivations, and require that they are used. Practices vary significantly across
countries, and evidence has shown that hate crimes are constantly underreported (see also the access
to and quality of justice chapter of this Handbook). Victims encounter difficulties in reporting, and law
enforcement officials may be reluctant to record and acknowledge hate crime. An international initiative
(European Commission, 2017) seeking to improve the recording of hate crime by law enforcement author-
ities identified key guiding organizational, structural and operational principles, including:

•  ultivating a human rights culture within law enforcement agencies;

C
• Developing or adapting recording mechanisms corresponding to national needs and
capacities;
• Cooperating actively with civil society organizations;
• Using indicators to identify bias motivation when recording hate crime; and
• Flagging potential hate crimes when they are reported.

68. Underreporting of administrative data is one of the reasons for using additional data sources, es-
pecially in contexts where the main law enforcement officials (e.g. the police) may not be immune from
committing hate crime themselves or from being complicit in such crimes. Alternative sources include
data collected by national human rights institutions and civil society organizations as well as crime
victimization surveys conducted by NSOs or other relevant entities. Surveys can include hate crime-spe-
cific questions enhancing the measurement of the prevalence of hate crimes. Survey respondents are
asked if they think that the incident(s) they were victim of happened partly or completely because of
their characteristics or perceived attributes.14 These and other related household surveys allow policy
makers and civil society to have a more representative picture of the discrimination problem.

III.3.2.2 Household surveys to measure experiences, awareness and attitudes about discrimina-
tion and equality

69. Surveys of individuals or households are an essential tool to assess directly experiences, aware-
ness and attitudes regarding discrimination. A range of questions have been included in victimization
surveys, health surveys, labour force surveys, multiple indicator cluster surveys (MICS) and increasingly
discrimination-dedicated surveys implemented by NSOs, human rights institutions (see SDG indicator
16.a.1) and civil society organizations. Such surveys may target or oversample population groups more
at risk of discrimination, which are often not sufficiently covered by traditional surveys. For instance,
this concern guided the design of a targeted discrimination survey in Mexico, undertaken by the
National Institute of Statistics and Geography (INEGI) in collaboration with the national human rights
institution and other relevant stakeholders (see Box III.3).

14 See, for instance, questionnaires designed by the European Union Agency for Fundamental Rights (FRA) used in surveys on immigrants and ethnic
minorities (2016) (https://fra.europa.eu/sites/default/files/fra_uploads/fra-2017-eu-midis-ii-questionnaire_en.pdf) and on LGBT people (2012) (https://fra.
europa.eu/sites/default/files/eu-lgbt-survey-technical-report_en.pdf)

38 |
BOX III.3 Measuring discrimination through household surveys in Mexico

In 2017, INEGI-Mexico, in collaboration with the National Human Rights Commission and the National Council
for Discrimination Prevention, conducted the National Survey on Discrimination, ENADIS. Overall, its purpose
was to recognize the extent of discrimination and its various manifestations in daily life, deepening knowledge
about who discriminates, in which areas of life this problem occurs more frequently, and the socio-cultural
factors that are related to it. In order to precisely approach the phenomenon, the ENADIS studies the following
populations:

Results at Population aged 18 and over

national and Adolescents and young people (aged 12-29)
state level Women aged 18 years and over

Indigenous people aged 12 and over

Results at People who have a disability aged 12 and over
national level People of a religion other than Catholic aged 12 and over
People aged 60 and over
Girls and boys (aged 9-11)

The ENADIS focuses on the following key goals:

• Assessing discrimination prevalence and its main expressions and characteristics

• Collecting information about attitudes, prejudices, social stigmas and stereotypes faced by the different
groups suffering discrimination regarding ethnicity, age and sexual orientation, among others
• Gathering data concerning discrimination experiences within different aspects of social life
• Strengthening awareness among different populations regarding the respect of human rights

Source: INEGI. ENADIS 2017

70. The Fundamental Rights Agency (FRA) of the European Union follows a similar approach in con-
ducting surveys on minorities, such as immigrant or ethnic minorities, Roma, lesbian, gay, bisexual, trans
and intersex people (see Box III.4).15 Disaggregation is highly desirable for surveys seeking to measure
discrimination.

71. In support of disaggregation of survey-based SDG 16 indicators, a joint project by the OHCHR,
UNODC and UNDP brought together a list of survey questions. Building on existing standard questions
(e.g. Washington Group Set of Questions on Disabilities16) and emerging practices by national statistical
offices, questions that relate to sex characteristics, gender identity, education levels, economic and social
situation, age, citizenship, ethnic background, disabilities, religion, marital and family status, sexual orienta-
tion and place of residence have been included and are now being piloted.17 In the past 20 years, Statistics
Canada has also extended the disaggregation of social indicators in connection with the measurement of
discrimination (see Box III.5).

72. SDG indicator 10.3.1 and 16.b.1 (proportion of the population reporting having personally felt
discriminated against or harassed in the previous 12 months on the basis of a ground of discrimination

15 Each release of survey results by FRA is accompanied with a detailed technical documentation to allow a transparent assessment of the sampling and
data collection methods. The technical documentation of the surveys can also assist countries in and outside the EU to replicate FRA’s surveys or further
develop their national data collection tools to improve data on minorities. Several survey microdata sets of FRA’s surveys are also available for researchers
to carry out further analysis to allow for the collected data to reach maximum impact. For more information: http://fra.europa.eu
16 They are accessible from: http://www.washingtongroup-disability.com/washington-group-question-sets/short-set-of-disability-questions/
17 See Guidance Note for Implementation of Survey Module on SDG Indicator 16.b.1 & 10.3.1, Annex on Examples of survey questions to facilitate
indicator disaggregation by relevant characteristics available at: https://www.ohchr.org/EN/Issues/Indicators/Pages/SDGindicators.aspx

HANDBOOK ON GOVERNANCE STATISTICS | 39

 prohibited under international human rights law) is an important example of
BOX III.4 Measuring discrimination
experiences in the European Union
an indicator seeking to assess discrimination prevalence by asking individu-
als directly via surveys about their personal experience. The internationally
agreed module for the compilation of this SDG indicator recommends that
The sample of the second European Union
Minorities and Discrimination Survey (EU- data collectors engage in participatory processes to identify and formulate
MIDIS II) covered persons belonging to ethnic contextually relevant grounds. A Human Rights-Based Approach to Data
or national minorities, Roma and Russians, (HRBAD) is necessary for this process (see the chapter in this Handbook
as well as persons born outside the EU (first- “Human rights: A cross-cutting normative framework for governance statis-
generation respondents) and individuals
with at least one parent born outside the EU tics”). National Institutions with mandates related to human rights or non-dis-
(second-generation respondents). Similar crimination and equality are ideal partners for these activities. Data collectors
to SDG indicator 10.3.1/16.b.1, the survey may also be able to identify civil society organizations or other government
included a question on experience of felt
discrimination. The question was posed agencies with relevant expertise.
for different domains and activities, such
as in (access to) employment, education, 73. Critically, participatory approaches to the selection of grounds of dis-
housing, health, and when using public or
private services; on experiences of police crimination should consider: a comprehensive review of groups at risk of being
stops, criminal victimization (including hate discriminated; any sensitivities related to engaging with and collecting data on
crime); on awareness of rights and redress groups at risk of being discriminated; and the need for effective community
mechanisms; and on societal participation engagement and relationship-building to facilitate participation in data collec-
and integration, including trust in public
institutions. For instance, the question on tion activities and coverage of the groups concerned. The survey module rec-
discrimination when looking for work was: ommends the use of a show or prompt card with a definition of discrimination/
When looking for work in the past 5 years harassment and listing of prohibited grounds to facilitate the respondent’s
in [COUNTRY] (or since you have been in
[COUNTRY]), have you ever felt discriminated comprehension and recall. Box III.6 showcases the related survey module.
against for any of the following reasons?
Based on a showcard, the respondent could 74. This indicator provides a total population prevalence measure of
then select one or more grounds (skin colour,
ethnic origin or immigrant background,
experience of perceived discrimination/harassment. The indicator will not
religion or religious beliefs, sex, age, necessarily inform on the prevalence of discrimination within specific pop-
disability, sexual orientation) as well as ulation groups. This will depend on the covered populations. For example,
“other” grounds. if disability is included within the selected grounds, the resulting data for
The survey revealed that descendants of discrimination on the ground of disability will represent only the proportion
immigrants and minority ethnic groups
faced extensive discrimination in all areas
of life. Four out of 10 respondents (38%) BOX III.5 Measuring discrimination through
felt discriminated against in the five years different household surveys in Canada
before the survey because of their ethnic
or immigrant background in one or more
areas of daily life, and one in four (24%) Statistics Canada has been collecting information through its General Social
experienced this in the 12 months preceding Survey (GSS) on Canadians’ Safety since 1988 to better understand how Canadians
the survey. The highest rate of discrimination perceive crime and the justice system and to capture information on their
based on ethnic or immigrant background experiences of victimization. Information on the prevalence of discrimination and
concerned access to work. Almost a third the types of situation experienced has been collected through this module every
(29%) of respondents who looked for work 5 years since 2004. Statistics Canada has also collected data on discrimination
in the five years preceding the survey felt through its Ethnic Diversity Survey, starting in 2013. Different types of questions
discriminated against, and one in 10 (12%) and modules concerning discrimination and harassment have been applied as
experienced this in the year before the survey. part of the General Social Survey (Social Identity in 2013; Canadians at Work and
The results also showed that only 12% of Home in 2017) and the Public Service Employee Survey since 1999. Depending
respondents who felt discriminated against on the sample size of each survey, the data collected are disaggregated by many
reported the most recent incident. One in sociodemographic characteristics, including age, sex/gender, marital status,
four respondents (24%) experienced hate- religion, visible minority status, disability status, immigrant status, income,
motivated harassment, and 3% experienced urban/rural and education.
a hate-motivated physical attack in the 12 Source: https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getInstanceList&Id=1235019;
months before the survey. https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=4508;
https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=5024;
https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=5221;
https://www23.statcan.gc.ca/imdb/p3Instr.pl?Function=assembleInstr&lang=en&Item_Id=387773

40 |
of the total population who feel that they had personally experienced discrimination on the ground
of disability. Unless the sample provides adequate coverage of people with disability to allow disag-
gregation on this characteristic, the data cannot be understood as an indication of the prevalence of
discrimination (on the ground of disability) within the population of people with a disability.18 Against
this background, the module recommends oversampling population groups at risk of discrimination.19

75. To ensure optimal data quality when applying these questions, it is important to consider the
appropriateness of the profiles and training of interviewers in light of the priority grounds of discrim-
ination identified. For instance, if discrimination on a certain ground is expected to be a prevalent
issue, it may be appropriate to include among the interviewers or interview teams members of the
concerned groups (e.g. to provide female interviewers for female respondents). Interviewers should
be trained before collecting data of this nature. They should have at least a basic understanding of
the concepts involved and the potential sensitivity of the questions. Training may include observation,
role play and practical examples, in addition to training on the survey instrument and data collection
methodology. Another measure that needs to be taken is ensuring that respondents are alone when
they answer these questions and assuring respondents of the confidentiality of their responses. For
some surveys, it is also crucial to assess the possible impact of relying on proxy respondents versus
self-reporting respondents when analysing the results. This is because proxy responses could lead to
under-reporting discrimination/harassment when compared to direct responses.

76. Surveys on violence against women, beyond the measurement, can be useful instruments for
strengthening awareness-raising and support to victims (see Box III.7).20 Measuring stigma, stereo-
types, bias, values or awareness, whether at the level of possible perpetrator or victim of discrim-
inatory acts, complements the measurement of direct experience of discrimination. For instance,
Multiple Indicator Cluster Surveys (MICS) implemented by national statistical offices, with the support
of UNICEF, include a victimization module asking women and men if they consider that a husband is
justified in hitting or beating his wife in situations that make him annoyed or angered by things his
wife does (i.e. if she goes out without telling him; if she neglects the children; if she argues with him; if
she refuses to have sex with him; if she burns the food).21

77. In 2017, the OECD released Guidelines on Measuring the Quality of the Working Environment,
focusing on non-pecuniary aspects of work. The guidelines considered intimidation and discrimina-
tion at the workplace as one the fundamental issues affecting the physical and social environment of
work. Based on survey questions used in OECD countries, they recommended a question to assess
discrimination at the workplace (“I feel unfairly treated through discrimination at work”). Comparing
experience versus perception of discrimination can provide interesting information on the correlation
between different measurements concerning the prevalence of discrimination (i.e. an actual versus per-
ceived or assumed prevalence of discrimination by comparing survey responses to questions like “have
you been personally a victim of discrimination?” and “how often do you think there is discrimination
in your country?”). An example of a comparative analysis for countries in Africa is provided in Box III.8.

18 A similar listing of grounds is used as part of the Strategy for Harmonization of Statistics in Africa (SHaSA) and its harmonized module on Governance,
Peace and Security (GPS). Its question on discrimination experience relates to ethnicity, regional origin, religion, economic situation (poverty), gender,
disability and sexual orientation (https://au.int/pt/node/32847). A question on discrimination perception for the same grounds is also included in the
module.
19 For more information about SDG indicator 10.3.1 and 16.b.1, see https://www.ohchr.org/EN/Issues/Indicators/Pages/SDGindicators.aspx
20 See also United Nations Guidelines for Producing Statistics on Violence against Women — Statistical Surveys, ST/ESA/STAT/SER.F/110, https://unstats.
un.org/unsd/gender/docs/Guidelines_Statistics_VAW.pdf
21 See MICS6 tools, including survey questionnaires, available at: http://mics.unicef.org/tools.

HANDBOOK ON GOVERNANCE STATISTICS | 41

BOX III.6 Recommended survey module for SDG indicator 10.3.1 and 16.b.1 on discrimination

Interviewer: ensure privacy, check for the presence of others. Read the following introductory text and then ask Question 1.

Let me assure you again that your answers are completely confidential.

I will now ask you about discrimination. Discrimination happens when you are treated less favourably compared to others or harassed because
of the way you look, where you come from, what you believe or for other reasons. You may be refused equal access to work, housing, healthcare,
education, marriage or family life, to the police or justice system, shops, restaurants, or any other services or opportunities. You may also
encounter comments, gestures or other behaviours that make you feel offended, threatened or insulted, or have to stay away from
places or activities to avoid such behaviours.

Question 1: In [COUNTRY], do you feel that you personally experienced any form of discrimination or harassment during the last 5 years,
namely since [YEAR OF INTERVIEW MINUS 5] (or since you have been in the country), on the following grounds?

Interviewer: Show the attached card (Showcard) to the respondent or read it if the respondent has difficulty to read. Go through the list of grounds one
by one. If necessary, help the respondent to establish the recall period and make sure that you allow adequate time for the recall. You may reassure:
“It can be difficult to remember this sort of incident, so please take your time while you think about your answers.” Code “YES”, “NO” or “NR”
(No Response, in case the respondent prefers not to say or does not know). If the respondent answered at least with one “YES” to any of the grounds,
including “other grounds” under Question 1 (Q1.1 to Q1.12), then ask Question 2.

[Multiple responses]

1. SEX: such as being a woman or a man

2. AGE: such as being perceived to be too young or too old
3. D ISABILITY OR HEALTH STATUS: such as having difficulty in seeing, hearing, walking or moving, concentrating or communicating, having a
disease or other health conditions and no reasonable accommodation provided for it
4. ETHNICITY,
 COLOUR OR LANGUAGE: such as skin colour or physical appearance, ethnic origin or way of dressing, culture, traditions, native
language, indigenous status, or being of African descent
5. M
 IGRATION STATUS: such as nationality or national origin, country of birth, refugees, asylum seekers, migrant status, undocumented migrants
or stateless persons
6. S OCIOECONOMIC STATUS: such as wealth or education level, being perceived to be from a lower or different social or economic group or class,
land or home ownership or not
7. G EOGRAPHIC LOCATION OR PLACE OF RESIDENCE: such as living in urban or rural areas, formal or informal settlements
8. R ELIGION: such as having or not a religion or religious beliefs
9. MARITAL
 AND FAMILY STATUS: such as being single, married, divorced, widowed, pregnant, with or without children, orphan or born from
unmarried parents
10. SEXUAL ORIENTATION OR GENDER IDENTITY: such as being attracted to persons of the same sex, self-identifying differently from sex assigned
at birth or as being either sexually, bodily and/or gender diverse
11. POLITICAL OPINION: such as expressing political views, defending the rights of others, being a member or not of a political party or trade union
12. OTHER GROUNDS, SPECIFY: _________________

Question 2: In [COUNTRY], do you feel that you personally experienced any form of discrimination or harassment during the past 12 months,
namely since [MONTH OF INTERVIEW] [YEAR OF INTERVIEW MINUS 1], on any of the grounds you identified under question 1?

Interviewer: only for the ground(s) to which the respondent said (YES) under Question 1, select either “YES”, “NO” or “NR” based on respondent’s
answer to Question 2. Repeat the ground(s) mentioned by the respondent under the preceding question to facilitate recall and answer.

42 |
BOX III.7 The impact of data on violence against women in the European Union

Results of the survey on violence against women – carried out by the European Union Agency for Fundamental
Rights (FRA) – have been used extensively by policy makers at the EU level, as well as in the 28 EU Member
States for which the survey provided for the first time comparable data concerning the extent, nature and
consequences of violence against women. Recognising the gravity of the situation in the EU Member States,
the European Commission used the survey results to justify the use of EU funds to support awareness-raising
campaigns as well as the work of civil society organizations that support victims of violence. The survey results
have been frequently quoted in the work of GREVIO, an expert body monitoring the implementation of the
Council of Europe Convention on Preventing and Combating violence against women and domestic violence,
the first legally binding instrument of its kind in Europe. The survey also had an impact outside the EU, leading
the Organization for Security and Co-operation in Europe (OSCE) to apply the methodology developed by FRA to
collect comparable data in countries beyond the EU.

Grevio: www.coe.int/en/web/istanbul-convention/grevio
FRA: https://fra.europa.eu/en/publication/2014/violence-against-women-eu-wide-survey-main-results-report
OSCE: https://www.osce.org/projects/survey-on-the-well-being-and-safety-of-women

BOX III.8 Identifying and sharing good practices in collecting and using equality data

Looking at the results of the first round of the GPS-SHaSa surveys conducted in nine African countries between
2013 and 2016, it appeared that the prevalence of experienced discrimination on ten grounds (e.g. ethnic,
sex/gender, religion, disability) ranged from 10% to 61%. Compared to the perceived or assumed prevalence
of discrimination (i.e. percentage of the population considering that there is “always” or “often” discrimination),
the results showed an overall positive correlation with the prevalence of experienced discrimination.
Not surprisingly, the perceived or assumed prevalence is also always lower for self-declared non-victims
than it is for self-declared victims.

FIGURE III.1 Perception of discrimination in 9 African countries

100
90
80
70
60
50
40
30
20
10
0
Burundi Mali Madagascar Ivory Benin Cap Uganda Malawi Cameroon
Coast Vert
 Perception of discriminations  Perception of discriminations (for the discriminated)
 Incidence of discriminations (experience)

Sources: GPS-SHaSA modules, 2013-2016, NSOs; see also Razafindrakoto and Roubaud (2018), “Responding to the SDG 16 measurement challenge: the
governance, peace and security survey modules in Africa”, Global Policy, 20, pp. 146-182.

HANDBOOK ON GOVERNANCE STATISTICS | 43

 III.3.2.3 Discrimination testing surveys

78. Inspired by incidents of discrimination reported by victims and related testing of potential discrim-
inatory behaviours by civil society organizations (e.g. testing if entrance to a restaurant or bar is systemati-
cally refused to individuals of a certain skin colour), discrimination testing methods have been developed
and applied by institutional actors. For instance, discrimination testing surveys on access to employment
were developed by ILO and implemented in several countries (e.g. Belgium, Germany, France, Italy, the
Netherlands, Spain, Sweden, and the United States). The surveys measure the treatment by employers of
two applications submitted for job vacancies in different economic sectors and for different levels of skills.
The profiles of the two applicants are rigorously equivalent (i.e. same educational background and working
experience, both born in the country and nationals, etc.), except for one criterion, such as “origins or
migrant background”, revealed by names. The survey tests then statistically the number of times a person
with rigorously equivalent profiles needs to apply before being invited for an interview. The employers
tested are not aware of the experiment, but their anonymity is strictly protected. ILO testing showed dis-
crimination in access to employment to be a major issue. Candidates of immigrant origin generally needed
to make three to four times as many applications as candidates of “national extraction” in order to get to
the next step of the hiring process (see, for instance, Attström, 2007, and Cediey and Foroni, 2008).

III.3.3 Measuring indirect discrimination

79. As discussed above, discrimination extends well beyond direct discrimination. Any socioeco-
nomic and other relevant statistics disaggregated by grounds of discrimination prohibited by human
rights law are also essential to the measurement of discrimination, in particular indirect discrimination.
Collecting and disseminating disaggregated data is not a norm- or value-neutral exercise and pro-
vides an opportunity to make groups visible and enhance the realization of their human rights. Given
the associated risks and possible misuse of data, these efforts should be guided by the principles of
the Human Rights-Based Approach to Data (see the chapter in this Handbook on human rights).

80. Listing all potentially relevant socioeconomic indicators to the measurement of non-discrimina-
tion and equality would go much beyond the scope of this Handbook. In complement to the table of
illustrative indicators provided above, this chapter limits itself to providing some pointers regarding
how best to derive meaningful insights.

81. A useful approach beyond simple disaggregation of descriptive statistics to revealing possible
indirect discrimination is the average, deprivation and inequality approach:22

TABLE III.2 Average, deprivation and inequality perspectives

Time horizon Average Deprivation Inequality

One period What is the national average? Who are the most deprived? (e.g. by What is the disparity between the groups? (e.g.
sex, income quintile, ethnic groups) women/men, bottom and top income quintiles)

Two or more How has the national average changed? How has the situation evolved How have the disparities evolved
periods for the most deprived? (widened or narrowed)?

Source: Adapted from OHCHR, Human Rights Indicators: A Guide to Measurement and Implementation, and UNDP, Human Development Report 2000.

22 There are efforts emerging to develop new statistical practices to enhance data disaggregation, including on characteristics that may still be considered
sensitive in country contexts. For instance, the OECD released studies on data collection practices regarding disaggregation by race and ethnicity, sexual
orientation and gender identity. See OECD (2018) and OECD (2017).

44 |
 82. The following examples from the Philippines, South Africa and India illustrate applications of the
average, deprivation and inequality approach. In the Philippines, the percentage of learners complet-
ing the last year of primary school (Figure III.1) has on average risen 12.3 percentage points between
2005 and 2015. Disaggregated data, however, reveals the gender and regional dimensions of disparity.
In the same period, the gap between the National Capital Region (NCR) and in the Autonomous
Region of Muslim Mindanao (ARMM) – the country’s richest and poorest regions in terms of family
income – has narrowed by 11.9 percentage points. That more boys and girls are completing primary
grade in ARMM, whereas the primary completion rates for both gender categories in NCR has fallen,
explains both the increase in the national average and the narrowing gap. Among the groups, the
boys in ARMM remain the most deprived while girls in NCR are the most advantaged.

FIGURE III.2 Primary completion rate in the Philippines, gender and regional disparities, 2006 & 2015

Primary completion rate in the Philippines, gender and regional disparities, 2006 & 2015

100 Girls, National Capital

Region (NCR) Both genders in NCR declining
compared to increasing National
90 national average average, 2015

Boys, NCR
80 National
average, 2006
70 Narrowing gap between
boys and girls in the
richest (NCR) and poorest
60
%

(ARMM) regions

50
Girls, Autonomous
Region of Muslim
40 Mindanao (ARMM)

Boys in ARMM at
30
greatest risk of
being left behind
20 Boys, ARMM

2006 2015
Sources: Philippine Statistics Authority, National Child Poverty Indicators

83. As shown in Figure III.3, inequality in respect of South Africans over 20 years old considered
functionally illiterate (defined as having no formal education or education below Grade 7) has de-
clined on average for all non-white populations. Between 2002 and 2018, functional illiteracy has
dramatically fallen among Black Africans, Coloureds (persons of mixed racial descent) and people of
Asian or Indian descent. The incidence of functional illiteracy has declined the highest amongst Black
Africans (by 19.3 percentage points, from 34.3 per cent in 2002 to 15.0 per cent in 2018), followed by
Coloured (15.6 percentage points), whilst the gap between Black Africans and whites decreased from
33.8 percentage points in 2002 to 14 percentage points in 2018. From a deprivation perspective, Black
Africans were still the most behind in 2018.

HANDBOOK ON GOVERNANCE STATISTICS | 45

FIGURE III.3 Functional illiteracy in South Africa between 2002 and 2018

Percentage (%) of persons aged 20 years and above in South Africa with no formal education or
highest level of education less than grade 7, by population group, 2002 & 2018

40

Black African
35
National
30 average, 2002
Coloured Black Africans
remain most
25 deprived

Inequality declining
20
%

across all groups

Black African National
15 Indian/Asian average, 2018
Coloured
10
Indian/Asian
5
White White
0

2002 2018
Sources: Statistics South Africa, General Household Survey

84. In India, a young woman aged 20–24 from a poor, rural household was 5.1 times as likely as one
from a rich urban household to marry before the age of 18, 21.8 times as likely to have never attended
school, 5.8 times as likely to become an adolescent mother, 1.3 times as likely to have no access to
money for her own use, and 2.3 times as likely to report she has no say in how money is spent (UN
Women, 2018). These examples of inequalities, prevailing in many other contexts, will often reflect
entrenched direct or indirect discriminatory practices.

85. Another advanced approach to uncovering differences in outcomes is statistical modelling, e.g.
of the wage differential between men and women, potentially attributable to discrimination. Using
the Blinder-Oaxaca decomposition method, multiple regression analysis can be performed to decom-
pose the pay gap into elements that are more easily observed (Blinder, 1973; Oaxaca, 1973).

III.3.4 Quality of available statistics

86. The accuracy of the available statistics outlined above depends on their respective data sources,
the methodology applied for data collection and the nature of the information capture (e.g. verifiable/
observable incidents or opinion).

87. Structural indicators, like the proportion of international treaties relevant to non-discrimination
and equality ratified by States, have high accuracy. 23 They are based on comprehensive adminis-
trative records, the binary nature of information captured (i.e. ratified or not ratified) ensure their

23 See http://indicators.ohchr.org/

46 |
reliability. These structural indicators demonstrate validity as long as they are used to measure de jure
acceptance of standards or commitments, and not the actual implementation of non-discrimination
standards in practice.

88. Hate crime statistics based on administrative records usually underestimate the prevalence of
hate crimes. Victims do not systematically report incidents to law enforcement officials. Moreover,
the latter do not always rigorously assess and record possible motives behind crimes. The use of
additional data sources, such as data collected by national human rights institutions and relevant civil
society organizations, can help improve the accuracy of such incidents-based data. The credibility of
data producers affects trust in hate crime statistics. It will be particularly problematic when the data
producer, for instance, police officers, are themselves known or perceived as perpetrators of hate
crimes.

89. The accuracy of discrimination statistics obtained through household surveys can be measured
or assessed in terms of the error associated with sampling, non-response, questionnaire quality, inter-
viewer skills and other data collection and dissemination issues. The possibility to cover population
groups particularly at risk of discrimination will affect the accuracy and interpretation of statistics
about discrimination prevalence, attitudes or opinions. Research has shown that lack of awareness
or understanding of “discrimination” among the population affects the validity of the measurement.
This calls therefore for spelling out the concept, by providing respondents with the same definition
before asking about their experience, or by enumerating specific discriminatory acts and asking them
if one of these happened to them. There are pros and cons in these approaches. While helping to
identify the prevalence of specific discriminatory acts, the second approach will hardly be able to list
all possible acts of direct as well as indirect discrimination. In addition, this approach, requiring more
extensive survey modules, can discourage potential data collectors with limited capacity. These were
among the main considerations behind the development of the relatively short survey module for
SDG indicator 10.3.1/16.b.1. The combination of discrimination and harassment in the same survey
module reflects the overlapping character of the two concepts, both at the level of respondents’
own understanding and, as mentioned earlier, in the international normative framework. It was also
guided by the need to mitigate a telescoping risk, requiring the inclusion of an additional question
in the module.24 Cognitive testing on the module has revealed that providing respondents with the
same definition of discrimination/harassment, highlighting possible areas, places or incidents, even in
a succinct manner, helps to recall relevant incidents. A similar observation was made on the need for
providing contextually relevant and accessible formulations of grounds of discrimination.25

90. Accessibility to statistics and related data on discrimination should be, if anything, non-discrim-
inatory as well. People with disabilities, minorities speaking other languages and groups with limited
resources, all at risk of discrimination, should have access to the statistics, the related data and the
analysis. Proper interpretability of the statistics in this field calls for further collaboration between NSOs
with national human rights institutions, non-discrimination and equality bodies and other relevant
stakeholders.

91. For SDG indicator 10.3.1/16.b.1, achieving coherence across datasets, countries and over
time depends on the extent to which methodologies applied at country level are consistent with the

24 “Forward telescoping” describes a pattern of reporting events as having occurred more recently than they actually did. This phenomenon is commonly
observed in the context of crime victimization surveys.
25 For instance, rather than referring to ethnicity, more “plain language” may be helpful to respondents (e.g. the colour of your skin, your physical
appearance or way of dressing, culture, traditions or language).

HANDBOOK ON GOVERNANCE STATISTICS | 47

 internationally agreed methodology, including consistent use of general definitions of discrimination/
harassment and the listing of prohibited grounds of discrimination (see Box III.926). Differences should
be appropriately documented to support interpretation and comparability assessment.

BOX III.9 Identifying and sharing good practices in collecting and using equality data

In February 2018, the European Union High Level Group on Non-discrimination, Equality and Diversity set up
the Subgroup on Equality Data which is facilitated by the EU Fundamental Rights Agency (FRA). The subgroup
provides a forum for relevant national authorities and EU institutions to discuss challenges and opportunities
in improving the collection and use of equality data. Member States’ representatives are experts from national
statistical institutes, governmental departments dealing with non-discrimination policies, and equality bodies.

The subgroup combines thematic discussions on issues related to data collection on different grounds of
discrimination with the work on specific outputs that can support EU Member States in their efforts to improve
the collection and use of equality data: Guidelines on improving the collection and use of equality data
(European Commission, 2018); an on-line compendium of practices related to equality data collection and use;
and a diagnostic mapping tool. Furthermore, the subgroup supports the Member States by organizing study
visits to countries with advanced structures and instruments in place to allow other Member States to get a
deeper understanding of how these structures and instruments function and cooperate with each other.

III.4 Recommended key indicators

92. The chapter has shown that there are many statistical indicators relevant to the measurement
of non-discrimination and equality. Given the importance of analysing common socioeconomic in-
dicators by groups or grounds of discrimination prohibited by international human rights standards,
it is not possible to provide an exhaustive list of all potentially relevant indicators. Therefore, the
indicators listed in this section were selected primarily for their contribution to the measurement of
direct discrimination, the existence of available methods and data collection practices (particularly in
relation to the SDGs), and the role and contribution of NSOs in their compilation (Table III.3).

93. To also support the measurement of indirect discrimination, a more systematic dissemination
and analysis of disaggregated socioeconomic and other relevant statistics drawing on the average,
deprivation and inequality approach outlined in this chapter is also recommended.

26 For more on the EU High Level Group, see: https://ec.europa.eu/info/policies/justice-and-fundamental-rights/combatting-discrimination/tackling-dis-

crimination/network-experts-field-anti-discrimination_en; on the Sub-group, see: https://fra.europa.eu/en/project/2019/subgroup-equality-data; for the
Compendium of Practices, see: https://fra.europa.eu/en/theme/racism-related-intolerances/racism-compendium-practices.

48 |
TABLE III.3 Recommended key indicators

Sub-dimension Indicator Data source Type

Direct Proportion of the population reporting having personally felt discriminated against Household surveys Outcome
discrimination or harassed in the previous 12 months on the basis of a ground of discrimination
prohibited under international human rights law (SDG indicator 10.3.1/16.b.1)

Whether or not legal frameworks are in place to promote, enforce and monitor Expert assessment Structural
equality and non-discrimination on the basis of sex (SDG indicator 5.1.1)

Prevalence rates, disaggregated by the different domains where discrimination typically Household surveys, Outcome
occurs, especially when people are seeking to access different opportunities and discrimination
services – e.g. when looking for work, at work, in education or when in contact with their testing surveys
children’s school personnel, in access to healthcare, in connection with housing, when
using public or private services (such as public transport, administrative offices, when
entering a night club, restaurant or a hotel, and when being in or entering a shop)

Prevalence of hate crimes, namely crimes (e.g. homicide, assaults, theft) where victims Administrative data, Outcome
are specifically targeted because of their characteristics, ascribed attributes, beliefs household surveys
or values (e.g. race, religion, ethnic origins, sexual orientation or disabilities)

Proportion of victims of discrimination/harassment/hate crime who reported the Household surveys Process/ outcome
crime to relevant entities

Prevalence rates and proportions, disaggregated by relevant characteristics of Administrative data, Structural/ process/
groups at risk, such as sex characteristics, gender identity, education levels, household surveys, outcome
economic and social situation, age, citizenship, ethnic background, disabilities, discrimination testing
religion, marital and family status, sexual orientation and place of residence surveys, census

III.5 Way forward

94. The chapter outlined several concepts and definitions of non-discrimination and equality, based
on international human rights standards and existing data collection practices. Referring to the 2030
Agenda for Sustainable Development and core international human rights treaties, the chapter high-
lighted the importance of measuring this dimension in support of States’ commitments and efforts to
eliminate discrimination and reduce inequality at national level. Internationally agreed SDG indicators
(SDG indicators 5.1.1, 10.3.1/16.b.1) and additional indicators contributing to the measurement of this
dimension are recommended. Examples of related methodology and existing data collection prac-
tices from NSOs and other organizations (e.g. NHRIs and CSOs) from different regions have provided
a solid basis for the further development of methodological tools and data collection using surveys
and administrative records.

HANDBOOK ON GOVERNANCE STATISTICS | 49

 95. Further developments and considerations in the following areas would contribute to advancing the
measurement of discrimination and inequalities:

• Discrimination surveys or discrimination survey modules need to be developed and imple-

mented at country level as a matter of priority to measure specific SDG targets. Discrimination
survey modules need to be integrated in relevant surveys, such as victimization surveys, la-
bour force surveys, multiple indicator cluster surveys (MICS) and other relevant social surveys.
• The process of the contextualization and implementation of discrimination surveys or mod-
ules should be consistent with international statistical and human rights guidance, including
the principles of the Human Rights-Based Approach to Data (see Part A, Chapter II on human
rights) to uphold the rights of groups potentially left behind or at risk of discrimination.
• NSOs need to strengthen their collaboration with National Human Rights Institutions (SDG
indicator 16.a.1), equality bodies and similar organizations, including with civil society organ-
izations, to enhance the measurement of discrimination and equality and the compilation of
corresponding structural, process and outcome indicators at national level.
• In support of the efforts to leave no one behind in the 2030 Agenda for Sustainable
Development, new questions for disaggregation (e.g. on gender identity) need to be con-
sidered for inclusion in censuses and relevant surveys and the average, deprivation and in­
equality perspectives (Table III.2) need to be systematically applied to the dissemination and
analysis of relevant SDG and other social indicators.
• Further methodological guidance needs to be developed at international level on:
- how to collect and disseminate data on hard-to-reach and left-behind groups;
- more comprehensive statistical survey modules to measure experience of discrimi-
nation, including multiple and intersectional forms of discrimination; and
- the compilation of hate crime statistics based on administrative sources.
• To meet these objectives, there is a need for further capacity-building activities and sharing of
good practices, both at national and international levels.

50 |
HANDBOOK ON GOVERNANCE STATISTICS | 51
 PART B
EIGHT DIMENSIONS OF
GOVERNANCE STATISTICS

CHAPTER IV
Participation
in political and
public affairs
 IV.1 Conceptualizing this dimension
IV.1.1 Normative framework and definition of “participation in political and public affairs”
96. Participation in political and public affairs is a universal human right and a key component of
good governance. As a concept, it is grounded in various internationally agreed normative frame-
works, particularly the 1966 International Covenant on Civil and Political Rights (ICCPR). Article
25 of the ICCPR provides for the right of every citizen – “without distinction of any kind, such as
race, colour, sex, language, religion, political or other opinion, national or social origin, proper-
ty, birth or other status” – to participate in public affairs, including the following three elements:

a) the right to take part in the conduct of public affairs, directly or through freely chosen
representatives;
b) the right to vote and to be elected; and
c) the right to have access on general terms of equality to public service positions.1

97. General Comment no. 25 on article 25 of the ICCPR, adopted by the UN Human Rights
Committee, explains that the conduct of public affairs referred to in paragraph (a) is a “broad concept
which relates to the exercise of political power, in particular the exercise of legislative, executive and
administrative powers. It covers all aspects of public administration, and the formulation and imple-
mentation of policy at international, national, regional and local levels” (UN Human Rights Committee,
1966, para. 5).2 The General Comment further specifies that the right to take part in the conduct of
public affairs can also be realized “through public debate, dialogue with elected representatives, and
people’s capacity to organize themselves” (GC, para. 8). In this regard, the Committee specifically
refers to the “freedom to engage in political activity individually or through political parties and other
organizations, freedom to debate public affairs, to hold peaceful demonstrations and meetings, to
criticize and oppose, to publish political material, to campaign for election and to advertise political
ideas (GC, para. 25)”.

98. Other human rights norms and standards refer to equal rights and opportunities to participate
in political and public affairs for specific population groups. Table IV.1 briefly illustrates internationally
agreed normative frameworks emphasizing equal rights for women, ethnic, religious and linguistic
minorities, indigenous people, youth and persons with disabilities, while Table ANX.IV.1 in Annex IV
lists relevant regional human rights treaties.

99. Based on the above framework, this chapter proposes the following working definition of “par-
ticipation in political and public affairs”:

Taking part in the conduct of public affairs, including by registering to vote, voting and stand-
ing as a candidate in elections; being members of legislative, executive and judicial bodies at
all levels of government; accessing positions in the public service; and engaging, individually
or as members of political parties and other non-governmental organizations, in political activ-
ities such as publicly expressing political opinions, campaigning, holding peaceful demonstra-
tions or taking part in other forms of collective mobilization.

1 ICCPR (1966), https://www.ohchr.org/en/professionalinterest/pages/ccpr.aspx

2 General Comment No. 25: The right to participate in public affairs, voting rights and the right of equal access to public service. Para. 5. https://tbinternet.
ohchr.org/_layouts/15/treatybodyexternal/Download.aspx?symbolno=CCPR%2fC%2f21%2fRev

54 |
TABLE IV.1 Internationally agreed normative frameworks on participation of specific groups in political and public affairs

Participation Convention on the Political Rights of Women (1953) Articles I-III state that women shall be entitled to vote in all elections, shall be
of women eligible for election to all publicly elected bodies, and entitled to hold public office and exercise public functions on equal terms with men,
without any discrimination.
Convention on the Elimination of All Forms of Discrimination against Women (1979) Article 7 protects women’s right, on equal terms
with men, to vote in all elections and public referenda and to be eligible for election to all publicly elected bodies; to participate in the formulation
of government policy and the implementation thereof; and to hold public office.
UN General Assembly Resolution on Women and Political Participation (2011) Article 2 calls on all States to eliminate laws,
regulations and practices that, in a discriminatory manner, prevent or restrict women’s participation in the political process.

Participation UN Convention on the Elimination of All Forms of Racial Discrimination (ICERD) (1965) Article 5 requires States Parties to…
of minorities guarantee the right of everyone, without distinction as to race, colour, or national or ethnic origin, to equality before the law, notably in
the enjoyment of … (c) political rights, in particular the right to participate in elections – to vote and to stand for election – on the basis of
universal and equal suffrage, to take part in the Government as well as in the conduct of public affairs at any level and to have equal access
to public service.
Declaration on the Rights of Persons belonging to National or Ethnic, Religious & Linguistic Minorities (1992) Article 2 states
that persons belonging to minorities have the right to participate effectively in public life, and in decisions on the national and, where
appropriate, regional level.
UN Declaration on the Rights of Indigenous Peoples (2007) Article 5 states that indigenous peoples have the right to maintain and
strengthen their distinct political... institutions, while retaining their right to participate fully, if they so choose, in the political... life of the
State.

Participation Security Council Resolution 2250 (2015) Article 1 urges Member States to consider ways to increase inclusive representation of youth in
of youth decision-making at all levels in local, national... institutions and in mechanisms for the prevention and resolution of conflict.

Participation of UN Convention on the Rights of Persons with Disabilities (2006) Article 29 states that States Parties shall ensure that persons with
persons with disabilities can effectively and fully participate in political and public life on an equal basis with others, directly or through freely chosen
disabilities representatives, including the right and opportunity for persons with disabilities to vote and be elected.

Participation of International Convention on the Protection of the Rights of All Migrant Workers and Members of Their Families
migrants (MWC) (1990) Article 41 states that Migrant workers and members of their families shall have the right to participate in public
affairs of their State of origin and to vote and to be elected at elections of that State, in accordance with its legislation.

IV.1.2 Sub-dimensions of participation in political and public affairs

100. Participation in political and public affairs has several sub-dimensions, as derived from the
normative framework and illustrated by Figure IV.1. Each sub-dimension entails one or more specific
areas for statistical measurement. For instance, measuring participation in electoral processes requires
statistical information on the voting-age population, registered voters, voter turnout and candidates
standing in elections. As further shown in Figure IV.1 and explained in Box IV.1, capturing inclusive
participation and realization of the right to participate for all individuals requires that statistics are
disaggregated by sex, age, disability status, income/wealth, sub-national administrative level and
other relevant characteristics and that this is looked at in relation to the environment of participation,
including whether other rights are fully respected and enjoyed by all individuals.

HANDBOOK ON GOVERNANCE STATISTICS | 55

FIGURE IV.1 Conceptual and measurement framework for participation in political and public affairs

Sub-dimensions of participation in political and public affairs

A. Participation in electoral D. Representation in judicial bodies G. Enabling environment of

processes and referendums, as participation, including whether
primary means of exercising the right to AREAS FOR STATISTICAL MEASUREMENT: other rights are fully respected
participate in political and public affairs - C ourt staff dealing with criminal, civil, and enjoyed by all individuals
and administrative matters, by level
AREAS FOR STATISTICAL MEASUREMENT: of court and category of occupation. AREAS FOR QUALITATIVE ASSESSMENT:
- Voting-age population; - E xtent to which national legal
- Registered voters; frameworks comply with international
- Voter turnout in presidential, standards on civil and political rights;
legislative and local government - E xistence of Temporary Special
elections, and in referendums; Measures (TSMs) to facilitate the
- C andidates standing in elections. access of disadvantaged groups
to political and public office;
- Instances of fraud and other
B. Participation through E. Representation in bodies of irregularities in conduct of elections.
association in political parties public service/ administration
and other organizations
AREAS FOR STATISTICAL MEASUREMENT: AREAS FOR STATISTICAL MEASUREMENT:
AREAS FOR STATISTICAL MEASUREMENT: - Employment in public service by category - C ampaign finances;
- Members of political parties, of occupation. - A dult population who fears becoming
including in leadership positions; a victim of political intimidation or
- Members of non-governmental violence during electoral campaigns;
organizations whose aim is to influence - C andidates standing for elections
the conduct of public affairs. who experienced violence;
- Members of political and public
office who experienced violence and
C. Representation and participation F. Representation in informal discrimination while in the office;
in political office, including as governance bodies with power - A dult population with interest,
members of national and sub-national or influence over local matters information and knowledge on
legislative and executive bodies political and public affairs;
AREAS FOR STATISTICAL MEASUREMENT:
AREAS FOR STATISTICAL MEASUREMENT: - A dult population with stereotyping
- M
 embers of informal popular assemblies attitudes and values;
- Members of parliaments, including in with decision-making power over
leadership positions; and by manner of local issues and communities. - Self-reported levels of political efficacy
selection; among adult population and youth;
- Ministers, including by type of - Perceived levels of freedom to express
portfolios held; any political opinion, to join any
political organization, and to criticize
- Members of deliberative and executive government actions or performance/to
bodies of local government. participate in protests/demonstrations’.

Potential data disaggregation: sex, age, disability status, income/wealth, sub-national administrative level and other relevant characteristics.

101. The following sub-dimensions of participation in political and public affairs are noted:

102. Participation in electoral processes and referendums (sub-dimension A) is a crucial sub-di-

mension of participation considering that elections are “the primary means through which individuals
exercise their right to participate in public affairs” (Guidelines, para. 25). Inclusive electoral processes
enable the direct participation of eligible persons as voters or as candidates in elections and the
exercise of governmental power by freely chosen and accountable representatives. Referendums also
enable direct electoral participation, by giving people a say in “choos[ing] or chang[ing] their consti-
tution or decid[ing] public issues” (GC, para. 6).

56 |
 103. Participation through association in political parties and other organizations (sub-dimension
B) allows citizens to “exert influence through public debate… or through their capacity to organize
themselves” (GC, para. 8). The UN Human Rights Committee specifically refers to the “freedom to
engage in political activity individually or through political parties and other organizations, freedom
to debate public affairs, to hold peaceful demonstrations and meetings, to criticize and oppose, to
publish political material, to campaign for election and to advertise political ideas” (GC, para. 25).

104. Representation and participation in political office (sub-dimension C) refers to representation and
participation in political office including as members of legislative and executive bodies of government.
The Office of the UN High Commissioner for Human Rights underlines that “adequate representation
of the diversity within societies should be reflected, as appropriate, in State institutions and governmen-
tal bodies” (Guidelines, para. 20(d)).3 Consistent with the 2008 SNA (System of National Accounts), this
Handbook distinguishes between three spheres of government: central government, state (intermediate)
government, and local government (2008 SNA, paras 4.129-4.130). Central government is the government
sphere with legislative, judicial and executive authority over the entire territory of the country (paras 4.134
and 4.117). Its political responsibilities include national defence, the maintenance of law and order and
relations with foreign governments, ensuring the efficient working of the social and economic system by
means of appropriate legislation and regulation, and providing collective services for the benefit of the
community as a whole (para 4.134). The state (intermediate) government sphere consists of multiple state
government institutional units whose fiscal, legislative and executive authority extends only over the states
(or equivalent largest geographical areas) into which the country as a whole may be divided for political or
administrative purposes (paras 4.140 – 4.143). Considerable powers and responsibilities may be assigned
to state (intermediate) governments, especially in countries with federal constitutions (para 4.141). In some
countries, especially small countries, individual states and state governments may not exist (para 4.141).
The local government sphere consists of multiple local government units whose fiscal, legislative and exec-
utive authority extends over the smallest geographical areas distinguished for administrative and political
purposes (para 4.145). The scope of their authority is generally much less than that of central government
and state governments, they are often heavily dependent on higher levels of government, and they may
act as agents of central or state governments to some extent (paras 4.145-4.146).

BOX IV.1 On the importance of disaggregating data on participation in political and public affairs

Systemic inequalities impact specific population groups differently and can limit their participation in political and public affairs. As such, collecting
and disseminating disaggregated data on participation is fundamental to the development of adequate measures to strengthen the participation
of individuals and groups that are marginalized or discriminated against. The Guidelines for States on the effective implementation of the right
to participate in public affairs (A/HRC/39/28) specify that the adverse impact of discrimination (including multiple and intersecting forms of
discrimination) on the right to participate in public affairs should be recognized ‘in particular for women and girls, young people, persons with
disabilities, indigenous peoples, older persons, persons belonging to minority groups, persons with albinism, lesbian, gay, bisexual, transgender and
intersex persons and other groups that are discriminated against’. Furthermore, disaggregation by wealth/income and occupational status is key in
showing potential economic imbalances in the make-up of the public office and the political participation of the underprivileged. Disaggregation
by migratory status may also be important, considering the growth of migration streams across the globe which has made the civic and political
integration of migrants an increasingly important policy imperative. Finally, in most countries, participation in political and public affairs is not
evenly distributed across administrative levels. As such, data on participation should also be disaggregated at relevant sub-national levels to enable
policymakers and human rights bodies to more carefully target programmes aimed at expanding the effective exercise of the right to participation.
In many countries, meaningful analysis may require disaggregation down to the level of individual municipalities, counties or wards.

3 Guidelines for States on the effective implementation of the right to participate in public affairs, based on report Office of the UN High Commissioner
for Human Rights A/HRC/39/28 and presented by the Human Rights Council to member states and other relevant stakeholders through Resolution 39/11.
Available at: https://www.ohchr.org/Documents/Issues/PublicAffairs/GuidelinesRightParticipatePublicAffairs_web.pdf

HANDBOOK ON GOVERNANCE STATISTICS | 57

 105. Representation in judicial bodies (sub-dimension D) refers to the extent to which various pop-
ulation groups have equal access to judicial positions and are equally represented in the judiciary,
especially among judicial officials holding decision-making power. The judiciary is the system of courts
that constitutes the branch of central authority in a country concerned with the administration of jus-
tice. In the judiciary, decision-making power and leadership roles are essentially held by individuals in
two types of positions, namely judges and registrars. Judges play important roles in decision-making
by carrying out their core functions of interpreting laws and adjudicating controversies over the appli-
cation of laws to particular circumstances. Registrars assist judges in performing their functions and
play an important role in case management, including by scheduling hearing dates, registering court
documents, receiving fees emanating from court matters, preparing case files, drafting decisions and
executing court decisions. Additionally, in certain circumstances, they can perform judicial or quasi-ju-
dicial functions themselves, including making decisions on interlocutory applications, assessment of
damages, and applications for the entry of default judgments.

106. Representation in bodies of public service/administration (sub-dimension E) refers to the ex-

tent to which various population groups have equal access to public service positions and are equally
represented in the public service. Article 25 of the ICCPR underlines people’s right to access public
service positions based on “equal opportunity and general principles of merit” (para. 23). This implies
that “the criteria and processes for appointment, promotion, suspension and dismissal must be ob-
jective and reasonable”, in order to minimize political interference. The UN Human Rights Committee
further adds that “affirmative measures may be taken in appropriate cases to ensure that there is
equal access to public service for all citizens (para. 23).”

107. Representation in informal governance bodies (sub-dimension F) refers to popular assemblies

with power “to make decisions about local issues or about the affairs of a particular community (para.
6)” such as indigenous or custom-based local governance bodies.

108. An enabling environment for participation in political and public affairs (sub-dimension G) is
grounded in the consideration that “the right to participate in public affairs cannot be considered in a
vacuum (Guidelines, para. 14)”: it requires an enabling environment where other rights are fully respect-
ed and enjoyed by all individuals. Particularly relevant are the freedom of expression (GC 25, para. 8),4
“the right to form and join organizations and associations concerned with political and public affairs
(Ibid.)”,5 “a free press and other media able to comment on public issues without censorship or restraint
and to inform public opinion (Ibid., paras. 12, 25, 26)”, the right of access to information (Guidelines,
para. 15) and the right to equality and non-discrimination (GC 25, paras. 15, 16, 21, 23, 24).

109. Furthermore, the right to participate in political and public affairs requires States to adopt leg-
islative and other measures to ensure that individuals have an effective opportunity to enjoy the right
(GC 25, para. 1), including, among others, temporary special measures to increase representation in
political and public office among groups that are marginalized or discriminated against6 and meas-
ures enabling electoral processes that are not distorted by disproportionate distribution of campaign
finances and are free of violence or the threat of violence (GC 25, para 19). A prevalent form of vio-
lence in political context is violence against women in political life, defined as any act of, or threat of,

4 Note: Freedom of expression is guaranteed under article 19 of ICCPR.

5 Note: Freedom of assembly is guaranteed under article 21 of ICCPR and freedom of association is guaranteed under article 22 of ICCPR.
6 Guidelines, para 19 (e). Temporary special measures (TSMs) are specific targeted measures taken to accelerate the equal participation of women or other
underrepresented population groups in the political, economic, social, cultural, civil or any other field. Examples of legislated TSMs in electoral processes
include reserved seats and election candidate quotas (UN Women and UNDP, 2015).

58 |
gender-based violence, resulting in physical, sexual, psychological harm or suffering to women, that
prevents them from exercising and realizing their political rights, whether in public or private spaces,
including the right to vote and hold office, to vote in secret and to freely campaign, to associate and
assemble, and to enjoy freedom of opinion and expression. Such violence can be perpetrated by a
family member, community member and or by the State. Violence against women in elections is a
form of violence against women intended to impact the realization of women’s political rights in an
electoral context (UN Women and UNDP, 2017).

110. Furthermore, States are required to “ensure that [all members of society] are empowered and
equipped with the knowledge and capacity necessary to claim and exercise their rights” (Guidelines,
para. 17). People must also perceive that they have abilities to understand politics and to participate
in political processes (internal political efficacy) and feel that their views can impact on public deci-
sion-making for them to find it “worthwhile” to perform their civic duties (external political efficacy).
When internal and external political efficacy are stronger, levels of political participation, including
voting in elections, are also higher (see Condon and Holleque, 2013; Finkel, 1985; and Vecchione and
Caprara, 2009).

IV.2 Why is this dimension important?

111. As mentioned above, participation in political and public affairs is a human right and no-
body should be denied opportunities to realize this right. Furthermore, and as recognized by the UN
Human Rights Council, equal and effective participation in political and public affairs is of “critical
importance… for democracy, the rule of law, social inclusion, economic development and advancing
gender equality and empowerment of women and girls, as well as for the realization of all human
rights and fundamental freedoms”.7 Empirical evidence shows that inclusive participation significantly
benefits sustainable development, as summarized below:

• Inclusive participation in political and public affairs leads to more effective governance and
more sustainable development outcomes. Research in OECD countries, for instance, shows
that when citizens are able to participate in selecting their government and have freedom
of expression, freedom of association and a free media, governments are more effective.8
Allowing public participation in decision-making deepens the understanding by authorities
of people’s demands and preferences, helps them identify significant deprivations, and
encourages them to better assess the impact of various policy and legislative options on
population groups (Guidelines), with considerable payoffs for economic efficiency and social
equity (OECD, 2017b).
• When political structures are more representative of society, the level of public trust in public
institutions is higher. People feel closer to elected representatives and public servants who
resemble them, and they perceive more representative political bodies to generate better
quality and fairer policy decisions, and to be less prone to the influence of vested interests
over decision-making. On the other hand, it has been shown that people or communities

7 Human Rights Council, Thirty-ninth session, 10–28 September 2018, Agenda item 3, Promotion and protection of all human rights, civil, political,
economic, social and cultural rights, including the right to development, A/HRC/39/L.14 https://undocs.org/A/HRC/39/L.14/Rev.1
8 As measured by the World Bank’s Worldwide Governance Indicators. OECD (2017); OECD Public Governance Reviews, available at: https://doi.
org/10.1787/9789264268920-en

HANDBOOK ON GOVERNANCE STATISTICS | 59

 with low levels of trust in governing institutions are less likely to engage with them, including
through voting in elections (OECD, 2017a).
• Inclusive decision-making is fundamental to sustaining peace. When women take leadership
roles and participate meaningfully in peace negotiations, the resulting agreements tend to
last longer and there is greater satisfaction with the outcomes. Similarly, societies that offer
more opportunities for youth participation in the political realms tend to experience less
violence (United Nations/World Bank, 2018; O’Reilley, O’Suilleabhain and Paffenholz, 2015).
• At a more holistic level, inclusive participation in political and public affairs increases people’s
satisfaction with their life. It has been shown that people’s feeling of “having a say in what
the government does” – people’s sense that their views can impact the political processes
and therefore, that they can influence their living conditions – affects individual levels of life
satisfaction (Flavin and Keane, 2011).

112. Accordingly, the 2030 Agenda for Sustainable Development places particular emphasis on the
promotion and protection of the participation of all members of society in political and public affairs.
This is well reflected in Goal 16 on Peaceful, Just and Inclusive Societies, which includes a target aimed
at ensuring responsive, inclusive, participatory and representative decision-making at all levels (target
16.7), another aimed at promoting and enforcing non-discriminatory laws and policies (target 16.b),
and another aimed at ensuring public access to information and protecting fundamental freedoms, in
accordance with national legislation and international agreements (target 16.10). Ensuring equal op-
portunities to participate in political and public affairs is also at the heart of Goal 5 on Gender Equality
(with target 5.1 aimed at ending all forms of discrimination against all women and girls everywhere,
and target 5.5 aimed at ensuring women’s full and effective participation and equal opportunities
for leadership at all levels of decision-making in political, economic and public life) and Goal 10 on
Reducing Inequalities (with target 10.2 aimed at empowering and promoting the social, economic and
political inclusion of all, irrespective of age, sex, disability, race, ethnicity, origin, religion or economic
or other status). Beyond Goals 16, 5 and 10, the implementation and monitoring of the 2030 Agenda
as a whole rest on the meaningful participation of all actors in society, in particular those most at risk
of discrimination and being left behind.

IV.3 Data and best practices currently available

113. Participation in political and public affairs is a multidimensional concept, and its measurement
involves multiple sources of data – including administrative sources, censuses, 9 surveys and expert
assessments – some of which can be used to measure the same sub-dimensions. For example, aspects
of participation in electoral processes may be measured through administrative data produced by
Electoral Management Bodies (EMBs)10, but also through survey data. Similarly, representation in the
public service may be measured through administrative data, but also through surveys of government
employees and population censuses.

9 Censuses, because of their unparalleled coverage, remain the best source of baseline data on the demographic and socioeconomic characteristics of
the population in general. This data is needed to measure the extent to which representation in political office, in the public service and in the judiciary
resembles the make-up of the national population.
10 An EMB is the body (or bodies) responsible for planning and administering the conduct of elections, and on occasion, referendums. Tasks of an EMB
may include: voter registration, receiving and validating the nominations of electoral contestants; conducting polling; counting the votes; and tabulating
the results.

60 |
 114. Developing a coherent and cost-effective statistical system on participation in political and
public affairs requires careful consideration of the strengths and pitfalls of each data source. This
section provides basic information to guide users in selecting appropriate sources to measure the
various sub-dimensions of participation covered in this chapter.

IV.3.1 Administrative data

115. Administrative sources are cost-effective for statistics on (a) participation in electoral processes
and referendums; (b) representation in legislative and executive bodies of government at national
and sub-national levels; (c) representation in the public service; and (d) representation in the judiciary.
This type of data may be produced by various governmental entities, including EMBs or equivalent,
secretariats of national parliament and government, ministries of local government, Public Service
Commissions/Ministries of Public Administration, and Judicial Services Commissions/Ministries of
Justice, as described below. It is recommended that these entities be integrated into the National
Statistical System (NSS) and that their mandate in producing official statistics on participation in polit-
ical and public affairs be clearly defined.

IV.3.1.1 Administrative data produced by Electoral Management Bodies (EMBs)

116. EMBs are entities engaged in organizing formal elections and referendums through a series
of core activities, including voter registration, registration of candidates, polling place management,
counting, tabulation and dissemination of election results, and post-election assessments. Additional
tasks – such as voter outreach and information, electoral boundary delimitations, campaign finances
disclosure and adjudication of electoral disputes – may also be undertaken by some EMBs. The over-
all process involves managing a vast amount of information that is essential for election administration
and accountability and can provide the basis for statistics on electoral and political participation.

117. Voter registration is a preparatory activity that involves recording eligible voters on a voters list.
It may be generated through updating existing voter lists; extracting new lists from civil or population
registries, national ID card databases or residency lists; or the targeted enumeration/registration of
eligible persons for the purpose of elections. The outcome is a voter list or registry comprising the
names and selected sociodemographic characteristics of the persons who are eligible to vote and the
polling places where they are expected to cast their vote.

118. The criteria for voter eligibility are indicated in the legal framework of a country, and typically
refer to age, citizenship and residence. In addition to these characteristics, it is recommended that the
registration process integrates, at a minimum, information on the sex and age of registered voters.
The information obtained can be used to generate statistics on voter registration, overall and disag-
gregated, at a minimum, by sex, age and place of residence. In countries where registration records
are in electronic form, the disaggregation is straightforward and may inform, including in real time,
targeted interventions and overall strategies for more inclusive registration.

119. Voter turnout refers to the number of registered voters casting a vote (whether valid or not)
on election day(s). A key advantage of the EMBs as the source of data (by comparison with surveys
as source of data) is their ability to provide data on the number of voters disaggregated at different
geographical levels. Other possible disaggregations of EMB data on voter turnout include individual
voter characteristics, such as sex and age. This information is directly accessible from the scanning of
voter ID cards or from an electronic register. For instance, in South Africa, the Independent Electoral
Commission uses the South African Identity document (ID) number as the basis for voter registra-
tion, verification of the voter’s identity at the polling stations, and the record of voter turnout. The

HANDBOOK ON GOVERNANCE STATISTICS | 61

 ID number is issued for all citizens and individuals qualifying for South African residence status and
incorporates numeric information on date of birth, sex (female or male), and citizen or permanent
resident status. This enables obtaining data on registered voters and voter turnout disaggregated
by sex, age and geographical area. ID numbers can further be used to link electoral data with data
from other administrative sources. This has been done by Statistics Norway, which has developed a
system to monitor the electoral participation of migrants in local and parliamentary elections, by using
national ID numbers to link microdata from the electoral registers with microdata from population
and other registers supplying information on the country of birth, reason for migration and length of
residence (see Box IV.2).

BOX IV.2 Statistics Norway’s experience in producing voter turnout statistics disaggregated by immigrant background

All foreign nationals with three years of legal residence in Norway were given the right to vote in local elections in 1983 and in parliamentary
elections in 1997. Eligible voters in local elections can also stand as candidates for municipal and county councils. In order to facilitate the integration
of immigrants into the country’s democratic political processes and institutions, Statistics Norway has established a system to produce data
disaggregated by immigrant status.11 The system builds on a provision of the Statistics Act of Norway, which states that the country’s national
statistical office shall have access to all official registers. It is based on the unique identifier (national ID number) provided to each person living in
Norway, which is also used to identify eligible voters in the electoral register. With this ID number, Statistics Norway can merge microdata from
the municipal electoral registers with other population registers that provide data on the country of birth, the country of birth of the person’s
father and mother, and reasons for migration. Information about refugees’ background is also available for linkage. This has enabled, since 1983,
the production of official voter turnout statistics by reason for immigration (i.e. refugee background, labour immigration, education, other family
immigration) and by length of residence in the country (i.e. 0-9 years, 10-19 years, 20-29 years, more than 30 years) (Statistics Norway, 2015, 2017).

These disaggregated statistics have revealed that voter turnout among immigrants is significantly lower than among the general population (by
about 25 percentage points), and that it varies considerably according to what country an immigrant is from. They have also shown that longer
periods of residence and older age groups are correlated with a higher turnout. Detailed statistics are also available to examine who among the
immigrant population stood as a list candidate and who was elected as a representative (by country of origin, by reason for immigration, by length
of residence, etc.). Based on these statistics, introduction programmes that teach immigrants about Norwegian society and the political system can
now be designed with specific target groups in mind, thus making them more effective and helping to enhance levels of immigrant participation in
the country’s political life.

120. When voter turnout is manually recorded at polling stations, it is recommended that the tally
or results form at polling places records the overall number of eligible voters on the list who voted.
Polling staff should record the sex of the voters as they turn out during the polling period. Regardless
of manual or electronic methods, voter turnout results (by sex and other characteristics) need to be
aggregated at district or regional level, and then aggregated for the whole country.11

121. Candidate registration refers to receiving and validating nominations for candidates to stand in
elections, including candidate lists from political parties, coalitions of parties or individuals, and inde-
pendent/non-affiliated individual candidacies. The submission of nominations for candidacies typically
includes standardized forms covering information related to party affiliation, a ranking of candidates
on a candidate list, and information on individual characteristics of the candidates as required by the
electoral laws and regulations. These individual characteristics often refer to sex and age, and, less
frequently, education and occupation. Information on the ranking of women and men on candidate
lists is particularly relevant in contexts where existing legislated candidate quotas for women require

11 This is in line with a provision in the Election Act of Norway which states that the electoral register should be used for election research and governmen-
tal planning (without jeopardizing the anonymity of the persons listed).

62 |
not only that a certain proportion of the total number of candidates are women, but also that women
and men be distributed in a balanced manner across the list.

122. The systematic electronic registration of information on sex, age and other sociodemographic
characteristics of candidates provides the basic information needed to generate statistics to measure
whether different sub-groups of the population have equal chances to stand for elections. These
statistics are key in informing political decisions on special measures to increase the participation
of underrepresented groups and can only be generated in a cost-effective manner by EMBs or the
bodies charged with registering parties and candidates.

123. The counting of votes and tabulation of results are sensitive and often complex processes
during which the winners and losers of an electoral contest are determined according to the electoral
law. Election results are recorded in official results forms and reported by individual polling stations
to higher levels of the electoral administration. While public dissemination of election results often
takes the form of lists of party names and/or candidates who won the elections and the number of
votes obtained, it is recommended that the dissemination of information on election results goes a
step further and provides short summary statistics about the composition of elected representatives
in terms of sex, age, geographical areas and other characteristics relevant in the national context (see
also Box IV.3).

BOX IV.3 Institutionalizing the collection and dissemination of disaggregated data by EMBs

Administrative data generated by EMBs can be one of the most cost-effective and timely data sources for monitoring progress towards inclusive
political participation by people as voters, candidates and elected officials. This requires that a minimum of individual and geographical characteristics
be systematically recorded during the electoral process and used as a basis for data processing/analysis and dissemination.

The following series of steps are recommended to be undertaken to institutionalize the collection and dissemination of disaggregated data by EMBs.
Some of these steps are best taken in partnership with NSOs, given their role as coordinators of statistical activities within NSSs and in providing
quality assurance for official statistics production.

1. The statistical mandate of the EMB is made clear in the electoral laws, rules or regulations guiding the work of EMBs. If changes need to be
made to the legal framework, they should be incorporated and communicated before the new election period starts.
2. A n assessment of existing potential disaggregations in electoral registration forms and electronic information systems is conducted to examine
whether the following procedures are in place:
• A geographic information system (GIS) is integrated in the information system of EMBs. This enables the recording of information
on the geographical location of eligible voters, voters and polling stations in line with the geospatial standards used by NSSs, thus
ensuring the coherence and integration of EMBs’ information systems within the system of official statistics.
•  Paper and electronic forms for voter and candidate registration include, at a minimum, information on sex and age. For candidates,
information on additional characteristics may be required by law, including education or occupation. However, information on
sensitive aspects that may breach the privacy of persons in the public setting of elections – such as disability status, ethnicity or
race, religion, income/wealth status – should be carefully considered before initiating data collection.
•  The information system used by EMBs to collect and store data electronically enables linkages between various types of data,
particularly the information on registered voters with voter turnout, and information on candidates with elected candidates.
3. Post-election dissemination of information on electoral participation and election results is done in a statistical format that is easily accessible
to ensure that data generated by EMBs – a valuable public good – is effectively used by all interested stakeholders. It is recommended that a
brief statistical summary is prepared and includes data on registered voters, voter turnout, election candidates, and the winners of elections
disaggregated by sex, age categories, other relevant individual characteristics and relevant territorial-administrative units.
4. Data-sharing agreements are developed to ensure that EMB data can easily be accessed by NSOs and other entities of the NSS. Such data may
be tabulated / aggregated at the level of sub-national administrative units used for electoral management. Some data – such as data on
candidates and elected candidates - may also be shared at individual level.

HANDBOOK ON GOVERNANCE STATISTICS | 63

 124. In addition to these core tasks, EMBs may also gather information on potential obstacles to
inclusive electoral and political participation, including, for example, on campaign finances (including
direct and indirect, as well as monetary and in-kind expenditures and contributions), and accessibility
to polling stations for persons with physical disabilities.12

IV.3.1.2 Parliamentary and ministerial secretariats

125. Parliamentary and ministerial secretariats can provide data on representation and participation in
national political office, including data on the sex, age, nationally relevant population groups, education,
occupational categories and wealth of individuals holding positions in national legislative and executive
bodies. While the format and scope of the information collected vary, nearly all parliamentary secretar-
iats record the sex and date of birth (which enables disaggregation by age) of appointed and elected
members of parliament (MPs). Information about the wealth and occupational background of MPs is less
commonly documented but important for showing the imbalances in the make-up of MPs, typically to
the advantage of white-collar professionals and above-average income groups.13

126. Information is also commonly recorded on the sex and age of parliamentarians in leadership
positions (speakers and deputy speakers) and those who are members of committees and parliamen-
tary groups. In the context of global reporting on SDG 16.7.1(a), which measures how representative
of the general population are the individuals occupying key decision-making positions in national
legislatures, information on the sex and age of MPs in three types of positions is to be collected,
namely, (1) Members, (2) Speakers and (3) Chairs of permanent committees in charge of the following
portfolios: Foreign Affairs, Defence, Finance, Human Rights and Gender Equality.

127. Similarly, ministerial secretariats (or, potentially the President or Prime minister’s office) can
provide data on senior members of the executive body, including ministerial positions and the head
of government, by sex and age. Ministerial positions refer to the highest decision-making positions in
the executive branch of government, immediately following the Head of Government, and generally
leading key policy areas of governmental action. Ministerial positions typically include all members of
the Council/Cabinet of Ministries (or equivalent) and the Deputy Prime Minister (in countries where
applicable). Additional information on types of ministerial portfolios held by women and men minis-
ters should also be provided.

128. Although sociodemographic information on MPs and senior government officials are usually
publicly available and simple to process, parliamentary and ministerial secretariats need to be tasked
with summarizing this information in a statistical format and reporting it regularly. It is recommended
that NSOs provide them with guidance to this end and request the regular dissemination of informa-
tion on the composition of these high-level decision-making bodies.

IV.3.1.3 Public Service Commission/Ministry of Public Administration

129. The most common and most comprehensive method for collecting public servant data (to
measure representation in the public service) is a Human Resource Management Information System
(HRMIS), which is typically maintained by a Public Service Commission (or related institution such as

12 Further readings on EMB organization and their data collection functions: International Institute for Democracy and Electoral Assistance (IDEA) (2014);
IDEA (2017); UN Women and UNDP (2015).
13 Research has shown that imbalances in the occupational and income group make-up of MPs are common across countries with different political
systems, and that these imbalances impact on the decisions taken by parliaments. See, for example, OECD, 2017, p.173.

64 |
a Ministry of Public Administration or a Ministry of Finance). In a global study conducted by UNDP’s
Gender Equality in Public Administration (GEPA) Initiative on the availability and quality of data on
women in public service in over 130 countries, such systems were found to produce more robust data,
in developing and developed countries alike (i.e. no sampling error compared to public service em-
ployment statistics derived from surveys), more up-to-date data (HRMIS are updated on a continuous
basis, compared to surveys conducted once a year or once every two years) and to have the greatest
potential for expansion on various disaggregations.

130. To measure the extent to which the composition of the public service is representative of
the national population at various levels of decision-making (as requested by SDG global indicator
16.7.1.b.), public servant data needs to be disaggregated by occupation14 (“job”). For purposes of
international comparison, countries are encouraged to align national classifications of public service
jobs with the International Standard Classification of Occupations (ISCO-08). Table ANX.IV.2 in Annex
IV outlines the main categories of occupations of relevance to the public service, as defined by ISCO-
08, namely among managers, professionals, technicians and associate professionals, and clerical
support workers. Among front-line service workers, a particular focus is placed on police, education,
health and front-desk administrative personnel. Such information can then be easily disaggregated by
multiple individual characteristics – such as sex, age, nationally relevant population groups or disabil-
ity status15 – thus enabling an analysis of the representation of various population groups at various
levels of decision-making across the public service.

131. National institution maintaining an HRMIS should collaborate with the NSO to ensure quality
standards are followed in the production of data.

BOX IV.4 Measuring representation in the public service - SDG 16.7.1.b.

SDG 16.7.1.b. aims to measure how representative of the general population are the individuals occupying
decision-making positions in the public service. It has been found that when public servants resemble the people
they provide services to, in respect to sex, age, nationally relevant population groups (e.g. ethnic, linguistic,
indigenous or other groups) and disability status, levels of popular trust in public institutions are higher (see
OECD, 2017a). Countries are encouraged to report data that is available, understanding that public servant data
disaggregated for disability status and nationally-relevant population groups may not be currently available
in many jurisdictions. Countries are encouraged to build additional capacities to disaggregate data by these
demographic groups.

IV.3.1.4 Judicial Service Commission/Ministry of Justice

132. The administrative data needed to measure representation in the judiciary is commonly col-
lected by a country’s Judicial Services Commission, Ministry of Justice or a similar body managing
human resources for the judiciary, handling appointments for judicial positions, or having an oversight
role over the judiciary. Data collection should refer to court staff dealing with criminal, civil and admin-
istrative cases.

14 “Occupation” refers to the kind of work performed in a job. More specifically, the concept of occupation is defined in ISCO-08 as a “set of jobs whose
main tasks and duties are characterized by a high degree of similarity”.
15 Countries are encouraged to produce self-reported data on disability, using the Short Set of Questions on Disability elaborated by the Washington
Group (http://www.washingtongroup-disability.com/washington-group-question-sets/short-set-of-disability-questions/). This short questionnaire is best
administered directly to public servants by a neutral sponsor, such as an NSO, which can provide adequate reassurances to respondents on the anonymity
and confidentiality of their responses.

HANDBOOK ON GOVERNANCE STATISTICS | 65

 133. To measure representation in the judiciary, data on court staff need to be disaggregated by
functionality, with a specific focus on staff holding decision-making power, such as judges and regis-
trars – two positions prioritized for global reporting under SDG 16.7.1c. Additionally, it is critical to
disaggregate court staff data by the level of court, including lower-level courts (i.e. first-instance or
frontline courts of local jurisdiction), higher-level courts (e.g. courts of appeal) and the highest courts
(including supreme courts and constitutional courts), and by the geographic location of courts. Court
staff data should also be disaggregated by sex, age, disability status (OECD, 2017cx) and national-
ly-relevant population groups.

IV.3.2 Surveys
134. Sub-dimensions of participation that require respondent self-reporting are best measured
through surveys. These include the following: experiences of participation in electoral processes and
referendums, and through political parties or other organizations; a wide range of sub-dimensions
related to the enabling environment of participation, such as perceived levels of freedom of associ-
ation and freedom of expression, social norms and stereotypes relative to participation, experiences
of discrimination and violence by voters, candidates and individuals holding political or public office;
and finally, public access to information and public knowledge on political and public affairs. Such
surveys may be representative of a country’s population (household and population-based surveys),
or may target specific governmental bodies (e.g. parliamentary surveys). In comparison with censuses
and administrative sources, surveys are flexible instruments that can collect in-depth information on
the topics covered, including on sensitive topics, while providing a range of demographic and socio-
economic individual characteristics that can be used for data disaggregation.

IV.3.2.1 Household and population surveys

135. Broadly speaking, household and population surveys can be used to measure:

•  elf-reported experiences of participation, including experiences of registering to vote and

S
voting in elections or in referendums, and being a member of a political party or of other or-
ganizations whose aim is to influence the conduct of public affairs. Other survey questions can
ask whether people have attended meetings and/or activities organized by political parties
or other civic organizations during an electoral campaign, taken part in peaceful demonstra-
tions, signed a petition, posted or shared anything about public affairs online and/or on social
media, or contacted a public official or a political actor (e.g. a local member of parliament) to
voice an opinion about a matter of concern or to request assistance to solve a problem.
• Public perceptions related to the quality of the enabling environment of participation, includ-
ing, for instance, regarding public perceptions about freedom of expression and freedom
of association, the extent to which any candidates or parties are prevented from running in
elections, whether the media are biased in reporting during elections, or whether voters are
bribed or vote-buying occurs during elections. In non-electoral contexts, surveys can be used
to measure the extent to which people feel there is media freedom in their country, or the
extent to which they feel they have access to important information on political and public
affairs.
• Self-reported experiences of, or fear of becoming a victim of, harassment and/or violence
in elections, as part of voter registration processes, during the electoral campaign or at the
polls. This may include intimidation or harassment of voters and family voting practices.
• The prevalence of social norms and stereotypes enabling or discouraging the exercise of
participation in political and public affairs for certain groups, including harmful gender

66 |
 stereotypes, and negative assumptions around the capacity of women, young people, minor-
ities, migrants and persons with disabilities to contribute to public affairs.
• Internal and external “political efficacy”: Surveys can be used to measure “individual beliefs
in the responsiveness of the political system, that is the extent to which people think that
politicians and/or political institutions will listen to and/or act on the opinions of ordinary
citizens”16 (typically called “external efficacy”) and “the confidence or belief that an individual
has in his or her own abilities to understand politics and to participate in the political pro-
cess”17 (typically called “internal efficacy”).

136. The measurement of participation in political and public affairs in official surveys conducted by NSOs
is a fairly recent practice in many countries, driven by specific interests in individual countries or regional initi-
atives. For instance, the General Social Survey (Social Identity) conducted by Statistics Canada since 2003 in-
cludes, among others, a set of questions on voting and engagement in politics.18 The respondents are asked
about their participation in the last elections at federal, provincial and local levels, intention to participate
in the next round of elections, interest in politics, political information-seeking, expressing political views,
volunteering for a political party, signing petitions and attending public meetings. As a further example, the
community of African statisticians, as part of the Strategy for the Harmonization of Statistics in Africa (SHaSA)
adopted in 2012, has developed and successfully pilot-tested a survey module on governance, peace and
security in 16 African countries, which includes several questions on participation in political and public
affairs (Box IV.5). This experience19 has demonstrated that official survey-based statistics on participation that
are comparable across countries are feasible and methodologically robust in varied political contexts. Efforts
are currently underway to institutionalize the production of harmonized official survey statistics on participa-
tion in political and public affairs across the African continent, under the auspices of the African Union.

BOX IV.5 S
 elected questions on political participation and its enabling environment
in the African Union SHaSA module on Governance, Peace and Security

• H ow often do you think the following listen and act on issues that people like you raise?
a) Members of Parliament/Senate; b) Local elected officials; c) Traditional leaders (4-point answer scale)
• H ow much information is provided by the national government to citizens on government decisions?
(4-point answer scale)
• H ow well do you think your local authority is handling the following: A. Reporting back to the people; B.
Consulting people in decision-making; C. Consulting traditional/community leaders (4-point answer scale)
• W hich of these characteristics of democracy do you consider essential? Are they respected in this country?
A) Freedom of expression; B) Freedom of the press/media; C) Equality before the law; D) Political freedom;
E) Free and fair elections; F) Freedom of Association; G) Absence of discrimination (among others)
(4-point answer scale)
• O verall, how satisfied are you with the way democracy works in this country? (4-point answer scale)
• Do you think that women should have the same chances as men of being elected to political office?
(4-point answer scale)
• D o you think that women actually have the same chances as men of
being elected to political office? (4-point answer scale)

16 Definition of “external political efficacy” used by the European Social Survey (2016), https://www.europeansocialsurvey.org/
17 Definition of “internal political efficacy”, ibid.
18 More information on the survey’s methodology is available at https://www150.statcan.gc.ca/n1/pub/89f0115x/89f0115x2019001-eng.htm
19 See “GPS-SHaSA Fact Sheets” here: http://www.austat.org/fact-sheet/. Also see Razafindrakoto, M., F. Roubaud and Y.A. Assany (2019). More informa-
tion on the GPS-SHaSA experience is available at UNDP (2017a).

HANDBOOK ON GOVERNANCE STATISTICS | 67

 137. Integrating a battery of questions on participation in multi-topic household or population
surveys conducted by NSOs, as was done by African NSOs using the SHaSA module on Governance,
Peace and Security, is a cost-effective solution by comparison to conducting a stand-alone survey on
the topic. Household surveys on living conditions, such as the Living Standard Measurement Surveys
(LSMS), labour force surveys, or the EU Statistics on Income and Living Conditions survey, may be
particularly well-suited “survey vehicles” for incorporating questions on political participation.

138. Some EMBs and NSOs have also embarked on conducting pre- and post-election surveys.20
Typically, pre-election surveys capture political attitudes; access to electoral information and knowl-
edge; and information related to voter registration.21 For example, the Baseline Study on Gender
Equity in the Electoral Process in Zimbabwe22 undertaken by the Zimbabwe Electoral Commission
with the support of the Zimbabwe National Statistics Agency collected data on media use and prefer-
ences, knowledge on electoral processes, and a series of characteristics related to voter registration
(whether the respondents had the necessary documentation, reasons for not being registered, and
voter roll-checking). The data were used to develop a targeted information and education campaign
for women voters and strengthen the capacity of election officers to administer gender-sensitive
elections.

139. Post-election surveys23 can capture the profiles of voters and absentees and the impact of
strategies for inclusive suffrage, perceptions of electoral processes, and obstacles faced when voting.
For example, the US Census Bureau uses its Current Population Survey (a monthly labour force survey
in which interviews are conducted in approximately 54,000 households across the country) to integrate
a set of questions on reported voting and registration, preference for methods of voting, and reasons
for not voting and not registering, immediately after national elections.24 The data collected are disag-
gregated by multiple characteristics, including sex, age, race and educational attainment. While the
voting estimates obtained tend to be higher than those based on administrative records, the survey is
nevertheless considered “the most comprehensive data source available for examining the social and
demographic composition of American voters in federal elections, particularly when examining broad
historical results” (File, 2018).

140. Several non-NSS entities are also using surveys to measure people’s experiences, expectations
and opinions on various sub-dimensions of participation. Such surveys have been implemented since
the early 1980s by academic networks (e.g. International Social Survey Programme, World Values
Survey, European Social Survey, Comparative Study of Electoral Systems; with regional barometers –
Afrobarometer, Arab barometer, Latinobarometro – following suit in the late 1990s). Others have been
initiated by commercial providers (e.g. Gallup World Poll), and others still are conducted by public
institutions (e.g. Eurobarometer) or in response to a specific mandate from a public agency (e.g. the
European Quality of Life Survey and the European Quality of Governance Survey).

20 Some of the post-election surveys involve re-interviewing respondents from the pre-election surveys.
21 Political preferences in voting may also be covered by surveys conducted by entities outside the National Statistical Systems, including private com-
panies, academic groups and non-governmental organizations. However, the engagement of NSOs in measuring voting preferences could potentially
undermine their status of impartiality and impact the trust in official statistics.
22 Zimbabwe Election Commission, 2018. Voice, Choice and Access to Information: Baseline Study on Gender Equity in the Electoral Process in Zimbabwe
23 Post-electoral surveys should be distinguished from exit polls, which cover samples of voters only. Exit polls are brief surveys of voters as they leave the
polling station, no later than election day. They may be used to measure voter turnout, particularly whether the suffrage was inclusive, by collecting a range
of sociodemographic data about voters. They may also be used to collect information on voting irregularities, including whether voters experienced or
witnessed vote-buying, harassment, intimidation or other forms of violence during election day.
24 https://www.census.gov/topics/public-sector/voting.html

68 |
BOX IV.6 M
 ethodological issues to keep in mind when measuring sub-dimensions of
participation in political and public affairs through a population survey

“Social desirability bias” – when survey respondents do not reveal their true beliefs or behaviors but rather
provide a response that they believe to be more socially acceptable – can affect responses on political
participation, potentially to a larger extent than responses to typical demographic or economic questions. To
increase the accuracy of data, statisticians may want to apply the list experiment method (see, for instance, Blair
and Imai, 2012), enabling the calculation of reliable aggregate estimates of potentially sensitive survey items. By
comparing the way people respond to a direct question on a potentially sensitive topic with the way they respond
to the same question asked indirectly (i.e. in a way that does not require them to openly reveal their true opinion
on the sensitive item), statisticians can infer the extent of the social desirability bias affecting the survey item
of interest.25 Other activities typically employed to ensure quality of data in a survey – including testing of the
questionnaires and training of the enumerators – should also be systematically implemented.

In surveys conducted through face-to-face interviews or over the phone, preventing non-response through
training the interviewers is crucial. A number of aspects should be particularly emphasized when training the
interviewers. First, explicit guarantees of confidentiality should be provided to respondents when introducing
the survey, as well as a brief context on why their honest responses are important to improve opportunities
for participation in the country. Also, enumerators should refrain from reading out loud to respondents the
“don’t know” or “refuse to answer” options, as they provide an easy way for respondents to avoid engaging
with the subject of the question.26 Finally, if respondents do not understand certain terms in a survey question,
interviewers should refer to the alternative wording specifically provided to them for this purpose. Trying to
explain the meaning of certain words in their own terms would jeopardize the consistency of data produced over
time and across countries.

Enabling self-reporting through adequate respondent selection protocol is key to data quality. It is important to
keep in mind that the respondent selection protocols of most household surveys differ from the protocol required
for a governance survey. While for many household surveys the head of household is often selected to act as
an informant about the status and experiences of the entire household, in a governance survey all household
members should be (or have an equal chance to be) interviewed, as a male household head has a different
experience of governance than his wife, son or daughter. As such, if the survey vehicle used to administer the
subset of questions on political participation targets household heads only, a different respondent selection
protocol should be applied for that particular subset of questions.

Sample size and structure should respond to data disaggregation needs for capturing inclusive participation. One
advantage of surveys is collecting information that can be used for data disaggregation, including by location
(administrative level), age, sex, migration status, disability, education level and income level. Yet disaggregation
places specific demands on sample design. For instance, collecting data that is representative of relevant sub-
populations, for example older persons or migrants, may require increasing sample sizes or over-sampling in
the particular sub-population of relevance. For some categories of population – such as the population living
in institutions – a different sampling framework or a dedicated survey instrument administered specifically to
these sub-populations may be considered.

IV.3.2.2 Surveys targeted to specific governmental bodies and public offices

141. Surveys targeted to specific governmental bodies and public offices can be used to collect infor-
mation on members’ experiences of discrimination and violence. For example, public service employee
surveys can be used to measure public servants’ own experiences and perceptions about the inclusive-
ness of the public service, particularly workplace diversity issues and experiences of discrimination (see

25 In a list experiment, when respondents are asked about a sensitive item indirectly, they are presented with a list of statements and asked to indicate
the number of items they agree (or disagree) with, without specifying their opinion on each one of them. The list experiment method is based on the
assumption that a reliable survey item should not show a statistically significant difference between the direct and indirect estimation techniques.
26 In electronic or mail surveys, “don’t know” or “refuse to answer” may be excluded as categories of answers.

HANDBOOK ON GOVERNANCE STATISTICS | 69

 Box IV.7 for examples of questions used in such surveys). While some of these surveys are conducted
by NSOs, other countries choose to subcontract the exercise to a private survey firm, as this can make
public servants more comfortable about providing honest responses to more sensitive questions.

BOX IV.7 Public Service Employee Survey of the Canadian Public Service

In its annual Public Service Employee Survey,27 Canada’s Office of the Chief Human Resources Officer asks public
servants’ opinion on the following statements on workplace diversity:
• My department or agency implements activities and practices that support a diverse workplace.
• I think that my department or agency respects individual differences (e.g., culture, work styles, ideas).
• Overall, my department or agency treats me with respect.

With respect to discrimination, the survey asks questions about the type of discrimination experienced
by public servants (on the basis of sex, age, race, national or ethnic origin, disability, family status), the
source of discrimination (co-workers, individuals with authority over them, etc.), the action taken (or not
taken) by public servants to address discrimination (e.g. filed a grievance or formal complaint; discussed the
matter with their supervisor or a senior manager; contacted their union representative; contacted a human
resources advisor; resolved the matter informally on their own; etc.), and the respondent’s satisfaction with
the organization’s response to discrimination and efforts to prevent it. For instance, public servants are
asked to rate statements such as the following:
• I am satisfied with how matters related to discrimination are resolved in my department or agency.
• My department or agency works hard to create a workplace that prevents discrimination.

142. Similarly, parliamentary surveys can be used to ask MPs about their own experiences and percep-
tions of the inclusiveness of legislative bodies.27 For instance, an annual survey in Canada collects data
on obstacles faced by MPs.28 One survey question asks MPs, “What are the biggest obstacles you face
doing the work of a Parliamentarian?”, and one answer choice reads as follows: “Discriminatory attitudes
toward me (e.g. racist, sexist, ageist attitudes)”. The data are used to advocate for parliamentary reforms
to “strengthen Canada’s democracy, making it more accessible, responsive, and inclusive”.

143. Surveys targeted to legislative bodies at national and local levels are also the recommended
quantitative method to investigate participation in decision-making while in the office and the key
challenges involved. In addition to the aforementioned issues of discrimination and inclusion, these
surveys can collect data on violence against women in politics and access to leadership positions.
These types of surveys have been pioneered by international entities working on democracy, electoral
systems and parliaments, and are yet to be mainstreamed in the work of NSSs. For instance, the 2018
IPU and Council of Europe’s survey on violence against women MPs and parliamentary staff in 45
European countries inquired about perceptions and experiences of psychological, sexual, physical
and economic abuse during the parliamentary term or in the course of their work in parliament (IPU,
Parliamentary Assembly and Council of Europe, 2018).

IV.3.3 Expert assessments

144. Expert assessments may be used to investigate selected sub-dimensions of an enabling environ-
ment for participation, including the extent to which national legal frameworks comply with internationally

27 https://www.canada.ca/en/treasury-board-secretariat/services/innovation/public-service-employee-survey/2018/2018-public-service-employee-sur-
vey-summary-report-results.html
28 The survey is conducted by a non-governmental organization, the Samara Centre for Democracy, https://www.samaracanada.com/research/
political-leadership/the-2018-member-of-parliament-survey

70 |
agreed standards on civil and political rights, the existence of temporary special measures (TSMs) to
facilitate the access of disadvantaged groups to political and public office, and observed irregularities
in the conduct of elections. While measures based on expert assessment have not been typically under-
taken by NSSs, they are becoming more relevant for national stakeholders, as such measures are being
used by some global SDG indicators. One such example is SDG indicator 5.1.1 on discriminatory legal
frameworks, which draws on an assessment of national legal frameworks (Box IV.8).

BOX IV.8 U
 sing expert assessments to report on SDG 5.1.1 (“Whether or not legal frameworks
are in place to promote, enforce and monitor equality and non-discrimination
on the basis of sex”)

Global reporting on SDG indicator 5.1.1 draws on an assessment of national legal frameworks carried out by
national counterparts, including NSOs and/or National Women’s Machineries, and legal practitioners/researchers
on gender equality, using a questionnaire comprising 45 yes/no questions under four areas of law: (i) overarching
legal frameworks and public life; (ii) violence against women; (iii) employment and economic benefits; and (iv)
marriage and family.

The first section of the questionnaire, which examines “overarching legal frameworks and public life”, includes
three specific questions on key elements promoting women’s representation in political and public institutions,
including through TSMs aimed at enhancing women’s participation in public and political office, and provisions
for the enforcement and monitoring of such TSMs:
•  Do women and men enjoy equal rights and access to hold public and political office (legislature, executive,
judiciary)?
•  Are there quotas for women (reserved seats) in, or quotas for women in candidate lists for, national parliament?
•  Are there sanctions for non-compliance with mandated candidate list quotas, or incentives for political parties to
field women candidates in national parliamentary elections?

145. For certain sub-dimensions of participation in political and public affairs, experts may be best
placed to provide the information required and to make informed judgements. For instance, in electoral
contexts, experts can assess whether the electoral laws have been fully implemented and make availa-
ble the information assessed through election observation reports.29 In particular, international human
rights monitoring mechanisms play a key role in ensuring that States adhere to agreed principles and
standards on political rights. Experts can assess the compliance of national electoral laws and regulatory
frameworks with international human rights obligations, media freedom30 (including the extent to which
contesting candidates and political parties have equitable access to the media – especially publicly
funded media), the right of access to information (including public access to voter registration statistics,
candidate nomination information, polling station locations, and political party finances and campaign
expenditures, among others), and the right to equality and non-discrimination when people exercise
their right to vote and to be elected, or when they try to access public service positions. This includes
assessments of the quality and implementation of TSMs taken by States “to strengthen the representa-
tion and equal participation of women and groups that are discriminated against in electoral processes”
(Guidelines, para. 31), such as legislated candidate quotas and reserved seats in elected bodies.

29 See, for example, international election observation reports disseminated by: European External Action Service (EEAS) of EU at https://eeas.europa.eu/
topics/election-observation-missions-eueoms_en?page=1; OSCE office for Democratic Institutions and Human Rights at https://www.osce.org/elections;
and the Carter Center at https://www.cartercenter.org/peace/democracy/.
30 This also includes looking at biased or unfair media reporting, especially by public media against opposition parties; the ability of the media to work
freely and to operate without censorship (including self-censorship) or interference during electoral periods; incidents of violence, threats, detention of,
criminal prosecution or sanctions against journalists and representatives of media organizations owing to their reporting on elections; and the existence
of a pluralistic media environment that provides access to a broad range of political opinions.

HANDBOOK ON GOVERNANCE STATISTICS | 71

 146. At national level, a key role may be played by National Human Rights Institutions (NHRIs),
which can provide expert assessments on several sub-dimensions of participation. Through their peri-
odic monitoring activities and reports on the status of national implementation of relevant internation-
al human rights obligations, NHRIs can provide evidence-based analysis on the specific groups most
at risk of not enjoying their right to participate in public affairs (OHCHR, 2018).31 They can also play
an active role in monitoring human rights violations during elections (as illustrated in Box IV.9 for the
case of Zimbabwe). Finally, as promoted by the Merida Declaration on “The Role of National Human
Rights Institutions in implementing the 2030 Agenda for Sustainable Development”,32 some NHRIs
are actively collaborating with NSOs to contribute data to the monitoring of the Agenda.33 They do
so by “disclos[ing] inequality and discrimination in [the implementation of the Agenda], including
through innovative approaches to data-collection and partnerships with rights-holders, vulnerable
and marginalized groups”.34 35

BOX IV.9 The role of NHRIs in monitoring elections in Zimbabwe

During the 2018 general elections in Zimbabwe, the Zimbabwe Human Rights Commission and the Zimbabwe
Gender Commission – two independent commissions mandated by the country’s Constitution to protect and
promote human rights – played an important observation and monitoring role.

The Zimbabwe Human Rights Commission (ZHRC) carried out a series of monitoring activities during the pre-
election period, on the election day and in the post-election period. For instance, during the pre-election period,
the ZHRC monitored the media coverage and whether the voter registration procedures and regulations were
followed, while also receiving and investigating complaints on electoral processes. For the 2018 general elections,
77 pre-election complaints were received, investigated and referred to the police as needed, including 21 on
instances of electoral interference, 39 on voter intimidation, 13 on vote-buying/partisan food distribution and 4
on violence. On election day, the ZHRC noted difficulties for disabled persons and older persons to access polling
stations and some instances of forced assisted voting. The two categories of citizens who faced the greatest
challenges in exercising their right to vote were prisoners and citizens living in the diaspora, as no logistical and
administrative mechanisms were put in place to support the vote of these two groups. Finally, in the post-election
period, the ZHRC documented violence related to the delay in announcing the winning presidential candidate,
which resulted in the loss of lives, injuries and destruction of property (ZHRC, 2018).

Meanwhile, the Zimbabwe Gender Commission (ZGC) established a Gender Observatory, which brought together
independent constitutional commissions and other relevant stakeholders from government, academia and civil
society to document women’s experiences throughout the electoral cycle and coordinate responses to violence
against women during elections. Some of the key components of this work involved field interviews with
potential voters and candidates during the pre-election period, observing elections at selected polling stations,
and monitoring media content in selected print publications, radio and TV stations and social media (Facebook
and Twitter). The information gathered suggested that violence against women in elections is most prevalent
and visible in the media, including a widespread negative portrayal of women in politics through social media,
hate speech against women candidates and sexual harassment of women in leadership positions.35

31 At global level, another valuable source of “expert assessments” available to all countries to help identify particular groups most at risk of being left
behind in relation to their participation in public affairs and/or related rights is the SDG – Human Rights Data Explorer (https://sdgdata.humanrights.dk/
en/explorer). This database was developed by the Danish Institute for Human Rights and categorizes all recommendations made by the UN Human Rights
Council’s Universal Periodic Review (UPR) to individual countries, treaty bodies and special procedures by SDG Goal and target, and right-holder groups.
This makes it possible to filter searches related to participation-related SDG targets, such as 16.7, 16.10, 5.5 and 10.2, and retrieve information for specific
groups, such as migrants, persons with disabilities or members of minorities.
32 The Merida Declaration was adopted at the twelfth International Conference of the International Coordinating Committee of National Institutions for
the Promotion and Protection of Human Rights (ICC) that took place in Mérida, Yucatàn, Mexico from 8 to 10 October 2015. See https://nhri.ohchr.org/EN/
ICC/InternationalConference/12IC/Background%20Information/Merida%20Declaration%20FINAL.pdf
33 See paragraph 17(5), Merida Declaration.
34 See paragraph 17(6), Merida Declaration.
35 Zimbabwe Gender Commission, 2018. Gender Observatory Election Report.

72 |
 147. Expert assessments on participation in political and public affairs are often used in global
composite indicators, on their own, or in combination with data from administrative sources and/or
population surveys (see Table ANX.IV.3 in Annex IV for a selected list of composite indicators based
on expert assessments and details on their measurement). Such global composite indicators can be
used to obtain an overall picture of various sub-dimensions of participation across countries and over
time, to call attention to challenging contexts, and to prompt additional national data collection ef-
forts and further research around specific issues unveiled by such assessments.

IV.4 Recommended key indicators

148. This section presents a minimum set of indicators for monitoring participation in political and
public affairs and their recommended data sources. The indicators were selected based on the fol-
lowing criteria:

•  overing all sub-dimensions identified in the conceptual framework outlined in Figure IV.1,
C
thus providing a comprehensive statistical picture of participation in political and public
affairs;
• Being relevant for policymaking / strong links with specific policies and strategies;
• Being simple, clear and easy to understand by policymakers and other stakeholders;
• Providing a direct and unambiguous measure of progress;
• Being universal, relevant across all or most countries.

149. Relevant indicators already established in global monitoring frameworks, such as the SDG
monitoring framework and the UN Minimum Set of Gender Indicators, are included and flagged
accordingly.

150. Countries are advised to consider this minimum set of indicators for inclusion in national sta-
tistical plans and development plans for the purpose of monitoring trends in participation in political
and public affairs. Additional statistics and indicators will nonetheless be needed for electoral plan-
ning and for managing the implementation of specific measures to strengthen the representation and
participation of disadvantaged groups in public institutions.

HANDBOOK ON GOVERNANCE STATISTICS | 73

TABLE IV.2 Recommended key indicators

Sub-dimensions
and indicator Recommended Structure/ process/
areas Indicators data sources outcome

A. Participation in electoral processes and referendums

Registered voters Proportion of registered voters in voting-age population by Administrative data Process/ outcome
administrative divisions of the country, sex and age produced by EMBs or
equivalent bodies and
population censuses
or registers.

Voter turnout Proportion of registered voters who voted in previous elections by Administrative data Process / outcome
type of elections (presidential, legislative, local) and disaggregated by produced by EMBs or
administrative divisions of the country, voters’ sex and age 36 equivalent bodies

Inclusive Proportion of candidates in elections for (a) national legislature and (b) local Administrative data Process / outcome
representation legislatures who are women, youth, in various educational/ occupational/ produced by EMBs
among electoral income categories or other specific relevant population group
candidates

B. Participation through association in political parties and other organizations

Inclusive Proportion of political party members, by sex, age, geographic Surveys targeted Process/ outcome
representation in area, nationally relevant population groups to political party
political parties secretariats / Registrar
Proportion of leadership positions of political parties (party leaders/
of political parties
vice-leaders), by sex, age, nationally relevant population groups

Membership in Proportion of the population taking part in the activities or meetings Household/ Process/ outcome
non-governmental of a trade union, political party or political action group, by sex, age, population surveys
organizations disability status and nationally relevant population groups.
whose aim is
to influence the
conduct of political
and public affairs

C. Representation and participation in political office

Inclusive Proportion of positions (by age, sex, persons with disabilities and nationally Administrative data from Outcome
representation relevant population groups) in public institutions, including the legislatures, parliamentary secretariat
in parliament compared to national distributions (SDG indicator 16.7.1a)
Proportion of MPs in high-income categories compared to national distributions

Inclusive Share of women among ministers (UN Minimum set of gender indicators) Administrative data Outcome
representation
among ministers

Inclusive Proportion of elected seats held by women in deliberative Administrative data Outcome
representation in bodies of local government (SDG indicator 5.5.1b) / local government
local government censuses/surveys

36 For comparisons across countries, it is important to distinguish those with compulsory voting, where the voter turnout is expected to be higher.

74 |
TABLE IV.2 Recommended key indicators, CONT.

Sub-dimensions
and indicator Recommended Structure/ process/
areas Indicators data sources outcome

Inclusive Attrition rate (proportion of MPs who resigned, retired or died before the end of Administrative data from Outcome
participation in their mandate) in national legislatures, by sex and other relevant characteristics parliamentary secretariat
national legislatures

D. Representation in judicial bodies

Inclusive Proportion of positions (by age group, sex, persons with disabilities Administrative data Outcome
representation and population groups) in public institutions, including the judiciary,
in the judiciary compared to national distributions (SDG indicator 16.7.1c)

E. Representation in bodies of public service / administration

Inclusive Proportion of positions (by age, sex, persons with disabilities and Administrative data / Outcome
representation in population groups) in public institutions, including the public service, population censuses
public service compared to national distributions (SDG indicator 16.7.1b)

F. Representation in informal governance bodies

Inclusive Share of women and youth among members of informal popular assemblies Targeted surveys in Outcome
representation in with decision-making power over local issues and communities relevant contexts
informal bodies of
local governance

G. Enabling environment of participation

Compliance of Whether or not legal frameworks are in place to promote, enforce and monitor Expert assessments Structure
national legal equality and non-discrimination on the basis of sex (SDG indicator 5.1.1)
frameworks with
Ratification status of five core international conventions related
internationally
to participation in political and public affairs 37
agreed standards
on civil and Compliance of national legal frameworks on civil and political
political rights rights with the five aforementioned core international conventions
related to participation in political and public affairs

Barriers to voter Proportion of the population with access to documents required for voting (IDs, etc.) Household/ Process
registration population surveys

Perception of Perceived levels of freedom to express any political opinion, to join any Household/ Process/ outcome
freedom of political organization, and to criticize government actions or performance/ population surveys
expression and to participate in protests/demonstrations, disaggregated by sex, age,
freedom of disability status, other nationally relevant population groups
association

Stereotyping Proportion of people holding stereotyping attitudes and values discouraging Household/ Structure
social norms the exercise of the right of participation, disaggregated by sex, age, nationally population surveys
relevant population groups and geographic area – e.g. proportion of people who
perceive women to be equally legitimate and effective political leaders as men

37 Namely: 1) International Covenant on Civil and Political Rights (United Nations, Treaty Series, vol. 999, p. 171 and vol. 1057, p. 407); 2) Convention on the Political Rights of Women (https://treaties.
un.org/doc/Tr...707%2000-40%20AM/Ch_XVI_1p.pdf); 3) Convention on the Elimination of All Forms of Discrimination against Women (http://www.un.org/womenwatch/daw/cedaw/cedaw.htm); 4)
International Convention on the Elimination of All Forms of Racial Discrimination (A_RES_2106(XX).pdf); and the 5) Convention on the Rights of Persons with Disabilities (A/RES/61/106).

HANDBOOK ON GOVERNANCE STATISTICS | 75

TABLE IV.2 Recommended key indicators, CONT.

Sub-dimensions
and indicator Recommended Structure/ process/
areas Indicators data sources outcome

External political Proportion of people who feel they have a say in what the government does / who feel Household/ Outcome
efficacy 38 the political system allows them to have an influence on politics (SDG indicator 16.7.2) population surveys

Equitable access to Proportion of parties with access to public funding for political campaigns Register of political Process
public funding for parties or equivalent
Percentage distribution of campaign finances by age and sex of candidates
political campaigns administrative data

Access to Proportion of people who say there is “some”/ “a lot” of information Household/ Process
information and on the forthcoming (local/national) election, disaggregated by sex, population surveys
knowledge on age, nationally relevant population groups, geographic area39
political and
public affairs

Voter intimidation Proportion of voters who say they fear becoming a victim of political Household/ Process
/ violence during intimidation or violence during election campaigns population surveys
electoral processes

Experience of (a) Proportion of candidates who experienced physical, sexual or psychological Surveys targeted to Process
violence and (b) violence and/or discrimination as candidate in elections, disaggregated by election candidates in
discrimination by sex, age, disability status and nationally relevant population groups elections for national
candidates standing legislatures and
for elections local government

Experience of Proportion of women members of national and local legislatures who Surveys targeted to Process
gender-based (a) experienced physical, sexual or psychological violence and/or discrimination, members of national
violence and (b) by age, disability status and nationally relevant population groups and local legislatures
discrimination while
in political office

151. Some of these indicators, particularly the SDG indicators, have established global method-
ologies, and countries have started to produce and report the relevant data. Other indicators may
currently be defined and agreed upon only at regional level, in the context of the work led by UN
Regional Commissions and other regional statistical bodies. One such example is the data compila-
tion on women’s decision-making power undertaken by the Gender Equality Observatory of ECLAC
(see Box IV.10). Finally, for a set of indicators, methodological work will need to be undertaken from
now on.

38 The chapter in this Handbook on “Responsiveness” provides the primary discussion on external efficacy, including on SDG indicator 16.7.2.
39 While this indicator has been used for 20 years by the Afrobarometer across the African continent, further research is required to identify and test-pilot other similar indicators in other regions of
the world.

76 |
BOX IV.10 R
 egional initiatives in compiling and disseminating data on women’s
representation in positions of power and decision-making: the case of
Latin American and the Caribbean

In Latin America and the Caribbean, gender parity in positions of power and decision-making is recognized as
one of the driving forces of democracy and for eradicating structural discrimination and exclusion of women. In
2007, member States of the Economic Commission for Latin America and the Caribbean (ECLAC) requested the
creation of a Gender Equality Observatory that would compile and disseminate data on gender inequality and
make available analytical tools for policymaking (ECLAC, 2007). The Observatory was established in 2009 and
has since focused on three areas of women’s autonomy: physical, economic and decision-making (https://oig.
cepal.org/en). One of the focus areas – women’s decision-making autonomy – refers to women's engagement in
decision-making at various levels of government and is monitored through a set of seven indicators:

• National executive power: percentage of women in ministerial cabinet positions

• National legislative power: percentage of women among parliament members
• National judicial power: percentage of women judges in the highest court or Supreme Court
• Municipal executive power: percentage of elected mayors who are women
• Municipal deliberative power: percentage of elected city council members who are women
• Countries that have signed and ratified the Optional Protocol to the Convention on the Elimination of All
Forms of Discrimination against Women
• Level of National Machineries for the Advancement of Women within the governmental hierarchy

Data for most of these indicators are based on national administrative sources and are compiled from countries
annually, with the support of the National Machineries for the Advancement of Women. The result is a data
series on women's representation in decision-making bodies dating back more than two decades. These data
are complemented by qualitative contextual information and analytical tools for policymaking. A repository
of national laws and public policies on quotas and gender parity has been developed and is available online.
Analytical reports and studies are periodically published with the aim of analysing women´s progress and
setbacks and providing the region’s countries with relevant information for developing public policies.

IV.5 Way forward

152. This final section outlines a longer-term agenda on measuring and monitoring participation in
political and public affairs for the international statistical community and national statistical systems.
Currently, data on participation in political and public affairs are collected by NSSs in many countries
as well as by independent national, regional and global research organizations and other entities.
However, in the absence of international standards on data collection methods and indicators, efforts
to produce comparable official statistics in this thematic area have been fragmented. This chapter pro-
vides a conceptual and measurement framework for producing statistics on participation in political
and public affairs, linking definitions to key dimensions of measurement, to recommended indicators
and to potential data sources. Nevertheless, some elements of this framework will need to be further
developed through international statistical guidelines before they can be applied consistently in the
production of official statistics.

153. Further development in the following four key areas would contribute significantly to advanc-
ing the statistical measurement of participation in political and public affairs:

1) Developing guidelines on producing statistics on participation in political and public affairs

based on administrative sources. Existing guidelines on using administrative data sources for
official statistics do not provide in-depth guidance on how to use these sources specifically

HANDBOOK ON GOVERNANCE STATISTICS | 77

 for participation statistics (see, for example, United Nations, 2011). Such existing guidelines
nonetheless do provide a good basis for developing focused guidance materials to this end.
2) Developing a survey module on participation in political and public affairs for integration into
national multi-topic surveys. The design of such a module, with accompanying guidelines for
NSOs on how to administer it, should be informed by the key sub-dimensions of participation
reviewed in this chapter, including experiences of participation in electoral processes and ref-
erendums, and through political parties and other organizations, public perceptions around
the enabling environment for participation, access to information and public knowledge on
political and public affairs, stereotyping social norms, and experiences of discrimination and
violence during electoral processes. The design of such a module should also build on the
lessons learned from the SHaSA surveys on Governance, Peace and Security and from inde-
pendent survey initiatives led by international and regional research institutions or consortia.
3) Measuring violence in electoral processes and violence against women in politics through
targeted surveys. The design of such surveys or survey modules with accompanying guide-
lines would build on existing guidelines on crime and victimizations surveys and violence
against women surveys and as well as on experimental approaches in measuring violence in
political settings.
4) Measuring representation and participation in informal local governance bodies, including
indigenous and custom-based decision-making bodies. Despite the high visibility of issues
faced by indigenous populations, this area of measurement has long been neglected in offi-
cial statistics.

154. More research is also needed on using new sources of data to measure participation in political
and public affairs, including big data and social media data. In this regard, one particularly under-
measured area relates to violence against political actors that takes place in traditional and social
media, including through hate speech and intimidation through the web.

155. Furthermore, methodological work is needed to define headline indicators for all sub-dimen-
sions of participation in political and public affairs. The development of the SDG indicator framework
has provided a great opportunity to advance the methodologies of several key indicators of partici-
pation in political and public affairs – for instance, SDG indicators 5.5.1b, 16.7.1a, 16.7.1b, 16.7.1c and
16.7.2. However, many other indicators recommended in this chapter are at the stage of a blueprint of
what needs to be measured and would need more clarification in terms of their exact scope, defini-
tions and methods of calculation.

156. At national level, the measurement and monitoring of participation in political and public
affairs is yet to be integrated by NSSs into regular programmes of official statistics. The following three
steps are key in achieving this goal:

1) Governmental institutions producing administrative data on participation should be fully

integrated into the NSS and their mandate in producing official statistics in this thematic
area should be clearly defined, recognized, and put into practice. These institutions include
EMBs, Parliamentary and Ministerial secretariats, Ministries of local government, Public
Service Commissions/Ministries of public administration, and Judicial Services Commissions/
Ministries of justice.
2) An assessment of existing statistics and available sources of data on participation in polit-
ical and public affairs should be undertaken by NSOs, with the objective of identifying (a)
data gaps; (b) “new” or underutilised sources of data that could be used in a cost-effective

78 |
 manner, inclusive of existing multi-topic surveys that could integrate a module on political
participation; and (c) needs for capacity-building assistance.
3) Measuring inclusive participation in line with the Agenda 2030 principle of “leaving no one
behind” requires regular reporting of data disaggregated by various sociodemographic
and economic characteristics of individuals. Information on some characteristics is currently
collected but may not be disseminated on a regular basis (for instance, sex and age), while
other information may not be currently collected (for instance, wealth/income group and
disability status). Some sources are better suited than others for certain types of disaggrega-
tion, and it is important that NSOs keep these aspects in mind when developing statistical
programmes to measure participation.

HANDBOOK ON GOVERNANCE STATISTICS | 79

 PART B
EIGHT DIMENSIONS OF
GOVERNANCE STATISTICS

CHAPTER V
Openness
 V.1 Conceptualizing this dimension

157. There are many definitions of government openness1 that significantly vary in scope. Some
definitions focus exclusively on the ability of citizens to access specific government data (Lathrop and
Ruma, 2010), while other definitions are much broader and use government openness almost as a
synonym of good governance (Wei, 1999). It is paramount to choose a widely endorsed definition of
openness that is broad enough to include all relevant aspects, while being precise enough to indicate
the key components of government openness.

158. The definition of openness used in this chapter has been drawn from commonalities among
definitions from different international organizations and experts. Williamsson and Eisen (2016), from
the Centre for Effective Public Management at Brookings, define openness by focusing on three gov-
ernance processes, i.e. initiatives to increase transparency; interventions intended to expand public
engagement and participation; and efforts to improve responsiveness and accountability. The World
Bank also suggests that government openness implies “increased transparency, citizen participation
and collaboration between government and citizens” (for more see World Bank, 2015). Similarly, the
Organisation for Economic Co-operation and Development (OECD) defines “an open government”
as “a culture of governance that promotes the principles of (1) transparency, (2) integrity, (3) accounta-
bility and (4) stakeholder participation in support of democracy and inclusive growth”2 (OECD, 2016).

159. This chapter will focus on two components of openness as defined by the OECD: transparency
(i.e. the disclosure and subsequent accessibility of relevant government data and information) and
accountability (a government’s responsibility and duty to inform its citizens about the decisions it
takes as well as to provide an account of the activities and performance of the entire government and
its public officials) to avoid overlap with other chapters. This is because the concepts of integrity and
stakeholder participation are already covered by chapter VIII (absence of corruption) and chapter IV
(participation in political and public affairs).

160. Therefore, the working definition of openness adopted for this chapter is: the extent to which
public institutions are transparent about their processes in decision-making and policy-making, and
accountable for making sufficient information available to the media, members of the public and
businesses. More specifically, the chapter will look into four sub-dimensions of government openness
as defined below.

•  ccess to information, which refers to the right of citizens to request access to official re-
A
cords held by the government on a topic defined by the information user, subject to narrowly
defined exceptions.
• Open data, which refers to the proactive disclosure of large datasets of information by the
government that have been defined by law or policy.
• Freedom of expression for the media, which refers to the ability of media outlets to access
and disseminate factual and objective information to the public, express an opinion on public
policy- and decision-making, advocate for the public interest, as well as scrutinize and hold
accountable public and private bodies, including their leaders and officials. The scope of this
sub-dimension is limited to media outlets that are licensed to operate and hence subject to
methodological and quality standards, as opposed to informal opinion makers, influencers

1 For the purpose of this chapter, both the terms “government openness” and “openness” are used interchangeably and refer to the openness of public
institutions.
2 See also the recommendation at: https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0438

82 |
 and advocacy groups, which are covered by the chapter of this Handbook on participation in
political and public affairs.
• Media plurality, which refers to the media ecosystem more broadly, including diversity of
media ownership, financing models, relationship between governments and the media, and
the media outlets’ relationship with civil society.

161. For the purpose of this chapter, the definition of a “public institution” is aligned with the
OECD’s definition of “government sector”. This definition includes all units of central, state or local
government; all social security funds at each level of government; and all non-market, non-profit in-
stitutions that are controlled and financed by government units. However, public institution does not
include a public corporation, even when all the equity of the corporation is owned by government
units. It also does not include quasi-corporations that are owned and controlled by government units.3

162. Government openness is an essential building block for the implementation of several human
rights, including the right to participation in public affairs, freedom of expression and the right to
non-discrimination and equality. As such, this chapter’s definition of openness is aligned with article
19 of the International Covenant on Civil and Political Rights (ICCPR) on freedom of expression. In this
article, freedom of expression is framed not only as the right to hold opinions without interference,
but also as the right to seek, receive and impart information and ideas of all kinds, regardless of fron-
tiers, orally, in writing or in print, in the form of art, or through any other media of a person’s choice.
According to the ICCPR, freedom of expression may be restricted only under certain circumstances,
as necessary, to respect the rights and reputations of others, and to protect national security, public
order, public health or morals.

163. It is also worth noting that to allow for a more tailored approach to measuring openness, the
concept of “accountability” in this chapter refers to processes that make governments accountable for
sharing sufficient information with the public on their management of public affairs, as opposed to the
broader processes for sanctioning elected and public officials for the exercise of their decision-making
powers. Other issues concerning government and public service accountability are addressed in other
chapters of this Handbook under different angles, including the chapters on participation, responsive-
ness and absence of corruption.

164. Each of the four sub-dimensions identified above is linked to different aspects of the OECD
definition of openness this chapter focuses on, i.e. transparency and accountability:

•  ccess to information promotes transparency by allowing citizens and businesses to request

A
access to specific information held by government, and accountability by providing the ra-
tionale for not sharing such information, the means to challenge government decisions and
the appropriate oversight mechanisms.
• Open data promotes transparency by allowing citizens and businesses to access the vast
quantity of data collected as part of government activities, and accountability by allowing for
consultations and feedback on the relevance and usability of the information provided, as
well as for the monitoring, assessment and oversight of relevant open data processes.
• Freedom of expression of the media promotes transparency by exposing poor decision-mak-
ing or misconduct in government, and accountability by allowing for effective mechanisms to
avoid interference with the work of media outlets.

3 See the OECD’s Glossary of Statistical Terms, available at: https://stats.oecd.org/glossary/detail.asp?ID=1139.

HANDBOOK ON GOVERNANCE STATISTICS | 83

 •  edia plurality promotes transparency by reflecting a diversified and representative range of
M
opinions and political views, as well as accountability by allowing for fair procedures for both
granting licences to media outlets and journalists and fairly assessing journalistic quality.

165. To sum up, this chapter’s conceptual framework consists of two components of openness
(transparency and accountability) and four sub-dimensions (access to information; open data; freedom
of expression for the media; and media plurality) to give a comprehensive picture of the different
mechanisms allowing for open government operations and decision-making (Figure V.1).

FIGURE V.1 Sub-dimensions and components of openness

Openness

Components Accountability Transparancy

Freedom of
Sub-dimensions Access to Open Media
expression by
information Data Plurality
the media

166. In the last decade, more than 100 countries worldwide have passed access to information or
freedom of information laws, including 65% of countries in the Latin America and Caribbean region
(LAC) and 97% of OECD countries (OECD, 2016). While this undoubtedly constitutes important pro-
gress towards government accountability and transparency, there is a lack of systematic statistical data
collection and analysis on what governments are doing to foster openness, and a significant dearth of
hard evidence on the impact of practical measures for openness taken by governments.

167. The guidance developed in this Handbook is primarily developed for National Statistics Offices
(NSOs) to address these data gaps. However, other monitoring and evaluation bodies (e.g. supreme
audit institutions, internal control bodies, anti-corruption agencies and parliamentary oversight bod-
ies) may also benefit from guidance on the measurement of openness.

84 |
V.2 Why is this dimension important?

168. Populations, civil society organizations (CSOs), international organizations, policy makers and
experts at large have advocated for greater government openness as a necessary condition for better
government quality for a number of years (see UNODC, 2004; Stiglitz, 2002; Islam, 2006; Kosack and
Fung, 2014; Bauhr and Grimes, 2014; Bauhr and Nasiritousi, 2012). A considerable and diverse com-
munity promotes the principles of government openness, including anticorruption activists, grass-
roots community-driven development associations, advocates of state modernization and public
sector innovation, as well as advocates of liberalization.

169. The tangible benefits of government or institutional openness are by now well established,
and there is substantial evidence that openness increases government performance (Alt, Lassen and
Skilling, 2002; Besley and Burgess, 2002; Brunetti and Weder, 2003; Reinikka and Svensson, 2005;
Winters and Weitz-Shapiro, 2013; Mitchell, 1998). Openness has been described as an important con-
dition for good governance, economic growth and effective environmental policies (IMF, 2010; Islam
2006; Kaufmann, Mehrez and Gurgur, 2002; Kurtzman, Yago and Phumiwasana, 2004; UNODC, 2004;
and Wapner, 1996; Bauhr and Nasiritousi, 2012). Openness can also improve public service delivery
(Bauhr and Carlitz, 2019), increase competition and reduce corruption risks in public procurement
(Ochrana and Pavel, 2013). Incidentally, openness has been deemed effective in light of research both
on negotiation and institutional design (Stasavage, 2004) and on ensuring that officials safeguard the
public interest rather than their own private interests (Florini, 2002).

170. Beyond economic and administrative benefits, institutional openness is of intrinsic value for the
well-being of citizens and how they relate to the country they live in. Freedom of expression and me-
dia plurality are fundamental for the formation of public opinion and for debate, and help democratic
systems to flourish and people to declare their voice (Beetham, 2004; Rodriguez and Zechtmeister,
2018). Open government practices, i.e. access to information, open data and citizen engagement,
are linked to other important themes in this Handbook, as they can increase public satisfaction with
government services, strengthen accountability, and enhance popular understanding of government
processes and results (OECD, 2017). They hence play an important role in shaping people’s trust both
in government and in other members of society, a relationship that has been empirically documented
(see Murtin and others, 2018, and chapter 10 in this Handbook).

171. The beneficial effect of openness shown in academic research is often derived from the logic
of the principal-agent theory (Besley, 2006). In this context, information asymmetries are seen as an
important obstacle for citizens (the principal) in holding the government or public officials (the agent)
accountable. As a result, increasing transparency by removing information asymmetries is deemed to
allow for greater opportunities for detecting and sanctioning abuses of public power and for enhanc-
ing the use of public resources for the greater good.

172. Finally, the extent of government openness will define how journalists and ultimately citi-
zens and businesses will be able to influence their government’s efforts to achieve the Sustainable
Development Goals (SDGs). In particular, Goal 16 "Peace, Justice and Strong Institutions” reflects
this consideration by promoting inclusive societies for sustainable development and seeking to build
effective, accountable and inclusive institutions at all levels. Many of these objectives overlap with
those included in the openness agenda (OECD, 2016). The substance of openness principles is par-
ticularly relevant to some of the substantive targets under Goal 16, including: the development of ef-
fective, accountable and transparent institutions (target 16.6); the promotion of responsive, inclusive,

HANDBOOK ON GOVERNANCE STATISTICS | 85

 participatory decision-making (target 16.7); and the expansion of access to information (target 16.10).
Government openness is also key to monitoring and evaluating progress towards the SDGs. Openness
strategies and practices can inform both the pre-defined objective for achieving all of the SDGs, as
well as the process to successfully achieve this objective. In other words, openness allows citizens and
journalists to be aware of what their government intends to do to achieve the SDGs, as well as how
it intends to achieve these objectives, including the evidence and rationale underlying each option.
Openness also allows citizens to keep apprised of their government’s progress towards the achieve-
ment of pre-defined objectives for the achievement of the SDGs.

V.3 Data and best practices currently available

173. The development of comprehensive metrics on openness is a relatively recent phenomenon,

and it is taking place mostly outside of national statistical systems (NSS). Currently, the debate is dom-
inated by composite indices produced by international organizations, CSOs, think tanks and private
firms. Moreover, the vast majority of indicators found within these composite indices are sourced from
expert assessments. While composite indices and expert assessments have unquestionable merit in
the policy debate, a detailed technical discussion and suggestions for further statistical development
– in particular in the field of official statistics – require the discussion to focus on stand-alone indicators
derived from quanitative data sources such as surveys or administrative data. Hence, this section ex-
amines first the existing indices, then disentangles them into their respective building blocks, and last-
ly discusses opportunities for measuring increasingly more of the key stand-alone indicators through
surveys and administrative data rather than expert assessments alone.

V.3.1 Overview of the most established openness metrics

174. Comprehensive efforts to measure government transparency and openness within and across
countries have started to emerge only in recent years (Bauhr and Grimes, 2017). Such efforts include
composite indexes building on a broad spectrum of indicators sourced mostly from expert assess-
ments of legal frameworks and their effective implementation or from more qualitative data in govern-
ment publications of norms, administrative processes and structures.

175. Table V.1 lists the most established indices and comparative assessment frameworks currently
available on openness. These frameworks have been chosen as being most closely aligned with the
sub-dimensions of this chapter’s definition of government openness, i.e. access to information, open
data, freedom of expression for the media, and media plurality.

176. It should be noted that beyond the list provided here, several additional frameworks or com-
posite indices relating to openness exist in specific policy areas, such as open budgets, transparency
in public procurement, and immigration / emigration flows, or even official statistics (Open Data
Inventory - ODIN), just to name a few.

V.3.2 Deconstructing composite indices into stand-alone indicators

177. In order to allow for a deeper analysis of openness indicators and their potential measurement
through NSOs, the composite indices identified in Table V.1 need to be deconstructed to identify their
precise building blocks, i.e. stand-alone indicators.

86 |
TABLE V.1 Available indices and comparative assessment frameworks on openness

Index / framework name Organization Sub-dimension measured Link

Open Government OECD Access to information https://www.oecd.org/publications/open-

Framework and open data government-9789264268104-en.htm

Open Government Guide Open Government Partnership Access to information https://www.opengovpartnership.org/sites/default/

and open data files/open-gov-guide_summary_all-topics.pdf

The Right to Information Centre for Law and Access to information https://www.rti-rating.org/
Rating Index Democracy and Access
Info Europe

The Open Data Open Data Watch Open data https://opendatawatch.com/

Inventory (ODIN)

Global Open Data Index Open Knowledge Foundation Open data https://index.okfn.org

World Press Freedom Index Reporters Without Borders Freedom of expression https://rsf.org/en/detailed-methodology
and media plurality

Freedom of the Press Index Freedom House Freedom of expression https://freedomhouse.org/report/freedom-press-2017-methodology

and media plurality

Open Government Index World Justice Project Access to information, open https://worldjusticeproject.org/sites/default/
data, freedom of expression files/documents/variables%20used%20to%20
construct%20open%20govt%20data.pdf

178. Table V.2 lists these stand-alone indicators and categorizes them under both the four sub-di-
mensions of openness and the structures/process/outcome categorization used throughout this
Handbook. Most of these indicators are derived from expert assessments (EA), but a few are from
household surveys (HS), establishment and business surveys (EBS) and administrative data (AD). Each
of the indicators identified in Table V.2 are linked to a subscript (e.g. EA, HS, EBS or AD) that identifies
their main data source. This information is largely indicative.

179. Some of the outcome indicators listed in Table V.2 were added from other chapters in this
Handbook, from the indicators under targets for SDG Goal 16, or they have been identified through
academic research (Williamsson and Eisen, 2016).

180. Needless to say, not all the indicators identified in Table V.2 should form the basis of recom-
mended key indicators to be considered by NSOs. Nevertheless, such a comprehensive list may help
NSOs understand the current state of openness measurement and what their strategic role could be
for collecting useful statistics on openness. To facilitate navigating through Table V.2, process indica-
tors have been categorized according to whether they measure the immediate objective of each of
the sub-dimensions (i.e. the clarity and effectiveness of access to information and open data process-
es, the absence of reprisals against journalists / media outlets, and the diversity of media), or whether
they contribute to measure components of openness that are common to each of the sub-dimensions
(i.e. transparency and accountability).

181. For a close look at how each composite index referred in Table V.1 links to Table V.2, Table
ANX.V.3 links survey questions or components of assessment frameworks with recommended key
indicators discussed in section V.4 of this chapter.

HANDBOOK ON GOVERNANCE STATISTICS | 87

88 |
TABLE V.2 Openness indicators identified from composite indices and comparative frameworks

Type of
indicator Sub-dimension of openness

Access to information Open data Freedom of expression of the media Media plurality

Structural • The State has developed an overarching openness strategy with a clear definition of openness, which promotes access to information requests, open data, freedom of expression and plurality of the media (EA).
• The openness strategy applies to regional, local and municipal governments, as well as to the legislative branch and the judiciary (EA).
• There is a government body that is responsible for coordinating the implementation of the openness strategy for different levels of government and different branches of government (EA).
• There is an awareness-raising, education and outreach campaign to promote the national openness strategy (EA).
• The government is showing support for the implementation of the national openness strategy, including by (EA):
o Statements by Cabinet Members and senior public officials;
o Openness initiatives in the government’s programme or platform;
o Assignment of overall coordination mandate to a specific institution;
o Discussions of openness issues in parliament;
o Targeted communications to specific groups.

• The law mandates responses by the government • 

T he law mandates the release of specific datasets • 
T he Constitution or law contains provisions to protect • 
T here are privately owned print press, television
to access to information (ATI) requests and and classes of information, along with a policy that freedom of the press and of expression (EA). networks and radio stations in the country (EA).
warrants a broad interpretation of ATI law (EA). defines the minimal requirements and procedures • 
T he penal code or other laws do not prohibit • 
T he proportion of the media that is owned
that must be met by public bodies (EA). journalists to report about specific issues (e.g. / controlled by government (EA).
• T he law allows everyone to make ATI requests,
including non-citizens and legal entities (EA). • 
T here is a national coordinating body for national security) or authorize State censorship (EA). • 
T he concentration of media ownership (EA).
the implementation of open data initiatives • 
W hether national laws include opinion crimes such • 
T he proportion of media owners that hold
• T he law requires the appointment of information
across the public sector (EA). as blasphemy or disrespect for authority (EA). economic interests going beyond the
officials in each public body to ensure they
comply with their obligations (EA). • 
A n advisory body is established with members • 
W hether laws require a right of response information / communications industry (EA).
from within and outside the government to to an individual or organization who is • 
T here is an independent regulatory body with
• T he law does not require motives
guide the elaboration and implementation the subject of a media article (EA). transparent procedures that is responsible for
for ATI requests (EA).
of the open data policy (EA). • 
E xistence of adequate defense against libel granting licenses / frequencies to media outlets (EA).
• T he law protects the anonymity
• 
T here is an independent body responsible for charges (e.g. telling the truth) (EA).
of the ATI process (EA).
exercising oversight, assessing performance, • 
T here are explicit legal guarantees protecting
• T he law requires that ATI requests and investigating complaints and resolving conflicts
reuse of information be free of charge, the independence of media regulatory bodies,
in relation to open data processes (EA). as well as of public and private media (EA).
except for minimal costs for reproducing
• 
T here is an open data officer with the relevant • 
T he media has established a self-
and sending the information (EA).
skill set who is responsible for defining and regulatory body (EA).
• A central body coordinates a process to gather making available open data for each public
information to produce an annual report on the • 
Journalists are free to form associations and
body at all levels of government (EA).
implementation of the ATI process (EA). unions to represent their interests (EA).
• 
T he law requires the coordinated and consistent
• T here is an independent oversight public body • 
T here are no legal restrictions on the
disclosure of the following categories of
responsible for supporting and monitoring freedom to become a journalist (EA).
information for each public institution (EA):
the implementation of ATI processes for each • 
A ccess to foreign news is not censored
o Institutional: Core legal documents,
public body across the public sector, for all or otherwise restricted (EA).
roles and functions;
government levels and branches (EA). • 
T here are restrictions on the ability of journalists
o Organisational: Information on
• T he oversight body is independent and has the to accept gifts, travel or other benefits that
personnel, names and contact;
necessary mandate to perform its powers and could impact on their objectivity (EA).
functions. It reports annually to parliament, which o Operational: Strategy and plans, policies,
• 
T here are protections for journalists against
also determines the budget, the appointment activities, procedures, reports and evaluations;
interference from media owners (EA).
/ dismissal of senior officials, and conflict-of- o Decisions and formal acts: Decisions and
• 
T here are legal protections for the
interest due diligence of senior officials (EA). acts along with background documents;
confidentiality of journalists’ sources (EA).
• T here are appropriate protections from • 
L aws on cybercrime, bullying and
liability for those who decide to release harassment are balanced with consideration
information according to ATI law (EA). for freedom of expression (EA).
 Structural o P ublic services: Descriptions of services, • 
T he availability of journalism training at
guidance, forms, fees and deadlines. a professional level (e.g. developing the
o Budget: Projected budget, actual income critical judgment capacity) (EA).
and expenditure, audit reports;
o Open meetings: Meetings that are open
or not, and how to attend them.
o Decision-making and participation:
Decision-making procedures and
public consultation mechanisms;
o Subsidies: Beneficiaries of subsidies,
objectives, amounts and implementation.
o P ublic procurement: Relevant details on
processes, criteria, outcomes of tenders,
contracts and contract implementation
o L ists, registers, databases: Lists, registers and
databases and how they are accessible online.
o P ublications: Publications, free of
charge or with purchase price.
o Dispute resolution: Mechanisms available for
raising concerns or complaints and appealing
decisions or actions by institutions.
o Information about ATI requests: How to
make requests, complaints and appeals,
contact of information officer.

Process • Whether citizens believe they have sufficient access to public data and in-depth information to monitor management of the public service (HS).
• Whether citizens believe they have sufficient means to hold the government and public services accountable for the information they share (HS).
• Whether it is easy for businesses to obtain information about policy or regulatory changes that affect their activities (Global Competitiveness Index) (EBS).
• A single institution monitors the implementation and impact of the openness strategy according to specific requirements across all branches and levels of government (EA).
• The national openness strategy is communicated to promote its implementation across the public sector (EA).
• Appropriate consultations with civil society, journalists, academia, sub-national governments and the private sector have been organized to develop the national strategy (EA, HS).
• The impact of openness initiatives is measured based on the following (EA):
o Ad hoc mechanism focusing on impact;
o Usual evaluation activities conducted by each public institution;
o Surveys among citizens;
o Surveys among stakeholders;
o Surveys among government employees;
o Government-conducted studies on the impact of openness initiatives in specific areas (e.g. quality of services);
o Independent assessments conducted by academic or research institutions;
o Independent assessments conducted by NGOs;
o Independent assessments conducted by private companies.
• Measuring the impact of the openness strategy is achieved through indicators linked to pre-determined objectives (EA).
• The extent to which the results of the monitoring activities are made publicly available (EA, HS).
• The data derived from monitoring and evaluation are used to strengthen the openness strategy (EA).

HANDBOOK ON GOVERNANCE STATISTICS | 89

90 |
Type of
indicator Sub-dimension of openness

Access to information Open data Freedom of expression of the media Media plurality

Process Clarity and effectiveness of ATI processes Clarity and effectiveness of open data process Absence of reprisals against Diversified media
• Degree of satisfation with processes reporters / media outlets • W hether the State may be able to distort the media
• Members of the public are aware of their rights
and of the processes to make ATI requests (HS). for sharing open data (HS, EA). • T he media are generally free to publish revelations market by allocating subsidies or advertising (EA, AD).
• Open data is made available through digital, about (1) political power; (2) major economic • W hether the overall economic situation of traditional
• C entral guidance assists public bodies
user-friendly, machine-readable formats in interests; (3) religious or spiritual authorities; (4) media negatively affects their sustainability (EA).
to meet their obligations under the ATI
relevant languages without any restrictions on the military; (5) police and law enforcement; and • W hether government advertising is distributed
Law, including in relation to (EA):
reuse, and it is regularly updated (EA, HS, AD). (6) organized crime (EA, other targeted surveys). equally among different media (EA, AD).
o E nsuring requests are received and processed in
• T here is a central access point for government • P olitical or religious leaders do not contribute to • P erceived administrative or financial constraints
accordance with standards, including timelines;
information that enables users to search for create a hostile environment for the press (EA). to establish independent media (EA).
o E nsuring that internal appeals are
information across all public bodies (EA). • Journalists and media owners are not • Media ownership is transparent to allow
processed according to standards;
• T here are unique identifiers for public organizations prosecuted for what they write or broadcast consumers to judge their impartiality (EA).
o C oordinating with all institutions that may (EA, other targeted surveys).
that allow for cross-referencing across data sets (EA). • T he costs associated with the establishment
have relevant data, as well as relevant
• T he data is available in non-proprietary format (EA). • T he number of civil or criminal prosecution cases and operation of media outlets (EA).
third parties affected by the request.
• W hether users are able to select the specific against journalists and whether they are subject
• ATI procedures are simple, can be supported to fair procedures (EA, other targeted surveys).
by technical assistance (e.g. to precise data they are interested in (EA). Transparency of information
• W hether users are able to establish an API connection • E xcessive monetary fines, criminal
requests) and can be made by any means of • T he media reflect the range of opinions and
to the data and link it to other applications (EA). prosecutions and violence are used against
communication (e.g. mail, internet, phone) (EA). political views of the general public (EA).
journalists / bloggers for libeling officials or
• T he information requested is made • W hether metadata is available (EA). • T he media reflect the population’s
the State (EA, other targeted surveys).
available in the preferred format for users, • E valuation of value-for-money of proactive language diversity (EA).
• L ocal and foreign journalists are able
including those with disabilities (EA). disclosure of government data (EA). • T he extent to which media ownership is perceived
to cover the news freely and safely in
• T he average delays to respond to ATI person (EA, other targeted surveys). as affecting news coverage / content (EA).
requests for each public body (HS, AD). Transparency of relevant information • T he presence of a sufficient community
• Reported cases of media subject to
• Degree of satisfaction with the process • E ase of obtaining information on changes to policies, intimidation or violence for their reporting of investigative journalists (EA).
of ATI requests (HS, EA). laws and regulations (Executive survey, EA?); (EA, other targeted surveys). • T he extent to which news and information
• ATI requests and reuse of information are free except • T he disclosure of data for the categories of • T he penalties that have been imposed on information content is determined by vested interests (EA).
for minimal costs for reproducing and information identified in “structures” indicators is providers in the last 12 months (EA, AD). • A ccess to public officials granted equitably
sending the information (EA). coordinated and consistent across public sector regardless of editorial lines (EA).
• Number of verified cases of killing, kidnapping,
• E valuation of value-for-money of access institutions, and disaggregations are available (EA). enforced disappearance, arbitrary detention or
to government initiatives (EA). • P ublic institutions help identify relevant torture of journalists, associated media personnel, Accountability for providing information
data to be disclosed proactively (EA). trade unionists and human rights advocates in the • 
T he extent to which journalistic quality is
Transparency of relevant information • T he information deemed most useful or frequently previous 12 months (EA, AD, other targeted surveys). assessed openly and transparently (EA).
• E ase of obtaining information on changes to policies, subject to ATI requests is made publicly • T he confidentiality of journalists’ sources is not • 
T he extent to which political or religious factors
laws and regulations (Executive survey, EA); available by default in priority (EA, AD). undermined (EA, other targeted surveys). prevent the establishment of independent media (EA).
• W hether the information provided by ATI • T he information released pursuant to ATI
• 
T he transparency of the process for granting
requests is pertinent and complete (HS). requests is made available proactively Transparency of relevant information television and radio licenses (EA).
• Implementation of exceptions and other legal in a searchable database (EA, AD).
• T he extent to which the media provide • 
T he transparency of the process for appointing
restrictions to ATI law are consistent with • T he open data is available for the substantive in-depth information on directors of TV and radio regulatory agencies (EA).
good international practices, 4 including a harm last 5 or 10 years (EA, AD). decision-making by the government (EA). • 
T he transparency of accreditation
test and a public interest override (EA). • A ll public data disclosed are updated • T he government holds regular press conferences procedures of foreign journalists (EA).
• W hen part of the information may not be released regularly / in real time (EA). that all media are able to attend (EA). • 
Reported cases of journalists being bribed
based on exceptions to ATI law, it is redacted to • T here are public listings of public data available (EA). • W hether courts excessively impose gag to cover or not cover certain topics (AD).
allow the rest of the document to be released (EA). orders or bans on the media (EA, AD).

4 Standard exceptions include : national security; international relations; public health and safety; the prevention, investigation and prosecution of legal wrongs; privacy; legitimate commercial and other economic interests; management of the economy; fair
administration of justice and legal advice privilege; conservation of the environment; and legitimate policy-making and other operations of public authorities.
 Process • 
T he right of access applies to all 3 • 
A ppropriate training is provided to public • 
W hether defamation laws pose a
branches of government with no classes officials responsible with sharing open data. threat to public debate (EA).
of information excluded5 (EA). • 
Civic education training is conducted to • 
W hether public and private media are perceived to
• 
T here is a specific body responsible for raise awareness about enhancing public conduct self-censorship on sensitive topics (EA).
promoting the right to ATI (EA). sector accountability through open data. • 
W hether public institutions exercise
• 
A ppropriate training is provided for all censorship on sensitive topics (EA).
relevant functions or public institutions Accountability for providing information
(e.g. courts, information officers) (EA). • 
Regular consultations within the public service Accountability for providing information
and with the public are conducted to define the • 
W hether broadcasting authorities and
Accountability for providing information information deemed most useful to be released, other relevant media authorities are able
• 
Effectiveness of complaint mechanisms (EA, HS). including for vulnerable communities (EA). to operate independently (EA).
• 
Reasons given for not providing the • 
F eedback is requested to measure the relevance • 
Crimes against journalists are prosecuted (EA, AD).
information are satisfactory (HS). and usability of the information provided (EA). • 
W hether the judiciary and high-level public officials
• 
F eedback is requested from the public • 
Open government mechanisms are subject to uphold legal protections for media freedoms (EA).
about access to information processes to internal audit or audits from supreme audit • 
T here are professional groups that support
strengthen government openness by both the institutions to assess performance (EA). journalists’ rights and interests (EA).
government and oversight bodies (EA). • 
E ach public institution periodically assesses • 
Journalists are not subject to electronic or physical
• 
C ompliance with ATI law is part of the public how citizens use its information and how surveillance with the object of interfering in
service performance management systems it may be more accessible (EA). their work or ascertaining their sources (EA).
at individual and institutional levels (EA). • 
Independent oversight authority has sufficient • 
T here is a transparent process to assess the
• 
ATI law courses are integrated in school / powers to address resistance and non- quality of media outlets and of journalists (EA).
university programmes to raise awareness compliance with recommendations (EA).
• 
W hether the courts are perceived as
and build capacity and accountability (EA). • 
Effectiveness of complaint mechanisms (EA). independent in their decisions about the
• 
F eedback is requested to measure the relevance media (EA, other targeted surveys).
and usability of the information provided (EA). • 
W hether the practice of journalism is restricted
• 
T here is a simple, free internal appeal completed based on nationality, ethnic origin, social class,
within clear timelines (20 days or less) (EA). religion, gender (EA, other targeted surveys).
• 
T here is a right to external appeal to an
independent administrative or judicial body
that is free, does not require legal assistance
and is governed by clear timelines (EA).
• 
Grounds for not providing the information
are clearly stated and the burden of proof
is on public bodies to justify their decisions
not to release the information (EA).
• 
Grounds for external remedies are broad (e.g.
refusal to disclose data in requested format, lack
of responsiveness, breach of timelines) (EA).
• 
T he independent oversight body monitors
the implementation of its recommendations
by the relevant public bodies (EA).
• 
T he decisions of the oversight body are binding,
including remedies (e.g. release of information),
structural measures on public authorities (e.g.
training), or sanctions on those undermining ATI (EA).

Outcome • Whether citizens and businesses believe government officials show favouritism to well-connected firms and individuals when deciding on policies and contracts (EA, HS).
• Whether public officials and citizens believe the government is responsive to concerns expressed by citizens and businesses (HS).
• Whether citizens believe they have a role to play in exercising oversight over their government(s) and public services (HS).
• Whether public officials believe the government has achieved a culture of transparency in the public sector (HS).
• Proportion of population who believe decision-making is inclusive and responsive, by sex, age, disability and population group (HS).

HANDBOOK ON GOVERNANCE STATISTICS | 91

5 These include : Cabinet, ministries, SOEs, sub-national government, public schools, health care bodies, police, army, intelligence services, independent oversight bodies, and private bodies performing public functions or receiving substantial public funding.
 V.3.3 Alternatives to openness data sourced only from expert assessments
182. As reflected in Table V.2, the vast majority of openness indicators found across the established
composite indices on openness are based mostly on expert assessments. Moreover, all the predomi-
nant indices listed in Table V.1 and most indicators listed in Table V.2 are currently produced by inter-
national organizations, CSOs, think tanks and private firms. It is relatively infrequent that government
agencies are themselves involved in the collection, production and dissemination of this data through
NSS-led data sources such as surveys or administrative data.

183. In aiming to make openness more easily measurable also by NSOs, this section shines a
light on the (few) cases where openness statistics are produced through sources other than expert
assessments.

V.3.3.1 Administrative data

184. Few indicators identified in Table V.2 are drawn from administrative data. Indeed, this data source
is largely untapped in relation to collecting data on government openness. Two factors largely explain
this situation. First, the legal basis to access these administrative data sources is missing in many coun-
tries, making their use impossible. Second, when these sources are accessible and understandable,
they are sometimes lacking some key quality and credibility criteria – making their analytical value
dubious.

185. Nevertheless, governments and public agencies at large collect a wide range of administrative
data that, if shared with relevant bodies, could be controlled for quality and used to measure different
aspects of government openness. This kind of data collection may be mandated by law or done on an
ad hoc basis for the purpose of good governance, transparency and efficiency.

186. An example of administrative data put to use for measuring openness:

•  he Treasury Board of Canada is mandated by the Access to Information Act to collect sta-
T
tistics on an annual basis for the purpose of assessing the compliance of government insti-
tutions with the provisions of the Act and the regulations relating to access. The minister
acting as the President of the Treasury Board of Canada also defines the information that
must be included in the annual report that each head of a government institution must submit
to Canada’s Parliament on the administration of the Act. The information collected by the
Treasury Board of Canada includes the following:
- Number of requests made and completed each year;
- Institution where the request was made;
- Origin of the requests (e.g. private sector, citizens, media, academia);
- Timeliness of responses and motives for extension;
- Exceptions used not to disclose; and
- Fees and costs associated with access to information regimes.

187. Some of the indicators currently proxied through expert assessments could be collected more
systematically via administrative data sources if partnerships with relevant institutions are explored.
These include: the number of appeals of decisions not to share government information following
ATI requests; the penalties imposed on information providers in the last 12 months; the number of
verified cases of killing, kidnapping, enforced disapearance, arbitrary detention or torture of reporters,
associated media personnel, trade unionists and human rights advocates in the previous 12 months in

92 |
connection with their profession; the number of gag orders or bans on the media; the prosecution of
crimes against journalists; the reported cases of journalists being bribed for covering or not covering
certain topics; the number of media licences issued per capita; and the proportion of media licences
that have been denied and the motives.

V.3.3.2 Household surveys

188. Household surveys and other surveys are also rarely used to assess government openness.
Increasing the use of large-scale surveys for measuring openness would undoubtedly contribute to
strengthening data collection and analysis, and ultimately, to use in building blocks for accountability
systems in the area of public governance. Using NSO surveys to gather data on openness would
improve statistical capacity and data availability in the area of openness and beyond, including by
allowing multivariate analysis with other topics captured through the questionnaires.

189. Even though the concept of openness remains relatively rare in survey questionnaires, a
small but growing number of surveys carried out by both international organizations and NSOs are
introducing questions on openness, particularly at the outcome level. Below are a few examples of
international surveys that include questions on openness:

•  he World Justice Project Open Government Index. This index is derived from two data
T
sources collected by the World Justice Project (WJP) in each country for the WJP Rule of Law
Index: (1) a general population poll (GPP), conducted by leading local polling companies,
using a representative sample of 1,000 respondents in the three largest cities per country;
and (2) a qualified respondents’ questionnaire (QRQ) consisting of close-ended questions
completed by in-country practitioners and academic experts. The Open Government Index
includes questions that contribute to measuring some of the key indicators for the purpose
of this report, including:
- The process for access to information requests is clear, simple, and user-friendly,
and it is supported by appropriate technical assistance;
- Up-to-date open data is available in user-friendly, machine-readable format on stra-
tegic issues for all public institutions;
- There are no excessive civil / criminal sanctions, violence or other reprisals against
reporters and media outlets in reaction to media reports;
- Members of the public have substantive and in-depth information on government
decision-making;
- The relevant institutions are accountable for the amount of information shared with
the public.
• 2012 OECD Survey of Adult Skills (PIAAC). This survey was fielded in large and represent-
ative samples of the adult population in 32 countries and subnational entities. For more
details about the survey’s methodology, please refer to the chapter of this Handbook on
Responsiveness. The Responsiveness chapter also provides references to national surveys
that seek to measure public perceptions of government responsiveness. The OECD PIAAC
Survey measures one of the key indicators used for the purpose of this report:
- Members of the public believe the government is responsive to their views and
concerns.

HANDBOOK ON GOVERNANCE STATISTICS | 93

 190. At the national level, there is limited but growing NSO experience worldwide to engage in
measuring openness via surveys, either as stand-alone dedicated surveys or as thematic modules
embedded in a larger, multi-topic survey.

• In Mexico, the Instituto Nacional de Estadistica y Geografia’s (INEGI) conducts the National
survey on access to public information and personal data protection (INEGI, 2016). The survey
collects data on citizens’ knowledge about their right to access government and its underly-
ing mechanisms. The openness questions concern:
- Trust in government information in different sectors such as security, budgeting, etc.;
- Whether access to government information is user-friendly (e.g. government
websites);
- Whether citizens’ information requests to a governmental institution were answered
and how satisfactory the process was;
- The regularity and clarity of bills for public services (e.g. water, electricity).
• The Governance, Peace and Security Surveys (GPS), under the Strategy for Harmonization
of Statistics for Africa (SHaSA), are an initiative to add thematic sections to socioeconomic
surveys at the national level (see Annex V, Tables ANX.V.1 and ANX.V.2). Under this initiative,
since 1995 a few NSOs in West Africa have conducted GPS surveys to collect data on issues
related to openness, including:
- Freedom of expression;
- Government transparency;
- Contact with government administration;
- Views on transparent elections.
• Although it was not led by an NSO, in 2016 the OECD and the Korean Development Institute
undertook a survey about trust in public institutions in Korea. Some of the indicators on open-
ness were:
- Expected satisfactory answer to citizens’ complaints;
- Expected availability of relevant government information;
- Expected incorporation of an opinion following a consultation process (OECD/KDI,
2018).
• Similarly, in Costa Rica, the General Comptroller Office runs three national surveys on trans-
parency to collect information on access to information, information for accountability, and
citizens’ participation. The surveys include the following indicators:
- The efficiency of access to government information processes;
- Whether enough information is published by the government to hold its agencies
accountable;
- The existence of spaces for public participation (República, 2019).

V.3.3.3 New data sources: social media

191. A relatively untapped data source is social media big data (tweets, Facebook posts, etc.),
which could potentially generate insights into the public’s perception of their government’s openness,
and its outcomes. The debate here is not over how government can use social media to improve
openness, but how social media data can be used to measure government openness. While this type
of text-mining analysis of big data is gaining momentum to inform specific research questions in other
domains, including health, economics, migration, etc., the literature is scarce on how social media
data analysis can be used to report on government openness. Social media could be a useful data
source for measuring government accountability and transparency because of the global trends of

94 |
increased connectivity and digitalization, which lead many to join this online conversation. Big data
techniques are also evolving to generate smart evidence from unstructured big data.

V.4 Recommended key indicators

192. This section identifies indicators that may be prioritized by NSOs to fill some of the most
urgent data gaps on openness, based on the indicators listed in Table V.2, i.e. the indicators already
in use to measure the implementation of SDGs and academic work. The overall aim in recommending
these indicators is to enhance systematic and coherent data collection through NSOs, based on the
conceptual framework presented earlier in this chapter, in order to fully depict and understand the
state of openness at national and global levels.

193. Many of the indicators recommended here are currently collected only through expert as-
sessments and as part of composite indices. In line with the discussion throughout this chapter, the
recommendation of this Handbook is to further develop these indicators so that they become more
easily measurable through surveys or administrative data, and thereby by NSOs. Given the fact that
many of these recommended indicators are not yet developed in this way, their specification may
be less concrete – in statistical terms – than the key indicators recommended in other Handbook
chapters. A more precise specification will evolve as they are developed methodologically.

TABLE V.3 Recommended key indicators

Sub-dimension Indicator Data source Type

All four sub-dimensions Number of countries that adopt and implement constitutional, statutory and/ Expert assessment Structural
or policy guarantees for public access to information (SDG indicator 16.10.2)

All four sub-dimensions Members of the public perceive public institutions are Surveys Outcome
responsive to their concerns in a transparent manner

All four sub-dimensions Members of the public perceive public institutions make objective, Surveys Outcome
evidence-based decision-making without favouritism

All four sub-dimensions Members of the public have substantive and in-depth Surveys Outcome
information on decision-making by public institutions

All four sub-dimensions Relevant institutions (including the media) are accountable for the Surveys, administrative Outcome
nature and amount of information that is shared with the public data, expert assessments

Access to information The process for access to information requests is clear, simple, and Surveys, administrative Process
user-friendly, and is supported by appropriate technical assistance data, expert assessments

Open data The availability of up-to-date open data in user-friendly, machine- Surveys, administrative Process
readable format on strategic issues for all public institutions data, expert assessments

Freedom of expression There are no excessive civil / criminal sanctions, violence or other reprisals Surveys, administrative Process
against reporters and media outlets in reaction to media reports data, expert assessments

Freedom of expression Number of verified cases of killing, kidnapping, enforced disappearance, arbitrary Administrative data, Outcome
detention or torture of journalists, associated media personnel, trade unionists expert assessment
and human rights advocates in the previous 12 months (SDG indicator 16.10.1)

Media plurality News coverage is diversified and media ownership does not interfere with Surveys, expert assessment Process
a comprehensive representation of public opinions and news coverage

HANDBOOK ON GOVERNANCE STATISTICS | 95

 194. For each of the indicators identified in Table V.3, examples of associated questions from exist-
ing data sources and instruments (surveys, expert assessment questionnaires or administrative forms)
are provided in Annex V, Table ANX.V.3.

V.5 Way forward

195. As discussed throughout this chapter, the development of metrics, tools and guidance that
enable NSOs to measure government openness is still in its infancy. Many international organizations
rely primarily on expert assessments to measure immediate outcomes of openness, and a great deal
of methodological work will be needed to identify the key indicators among them and make them
amenable to more quantitative measurement by NSOs.

196. The following actions are recommended with a view to achieving robust, country-led and con-
sistent measurement of openness:

•  esigning and piloting a survey module on openness for integration into national, multi-topic
D
household surveys. A survey module developed in collaboration with international organi-
zations, subject-matter experts and NSOs would allow for the development of harmonized
indicators of openness and consistent assessment methodologies across jurisdictions. Such a
survey module should be built on anticipated outcomes of openness, as well as on process in-
dicators such as the clarity and effectiveness of processes for sharing government information
/ data as well as on perceived intimidation and threats of media outlets and the perceived
diversity of media outlets and news coverage in a given jurisdiction.
• Developing guidelines to support an increased use of administrative data for measuring
openness. Administrative data is severly underused today to assess openness, in large part
because this data is not accessible and/or readily available for use by NSOs and other State
or non-State actors. Additional guidance on relevant administrative data sources as well as
proposed frameworks (including quality frameworks) to make such data available to NSOs
and other relevant organizations should be developed to ensure a greater and more consist-
ent use of administrative data to assess openness.
• Improving how the concept of “accountability” is reflected in the assessment framework
to measure openness. There is value in better reflecting the concept of “accountability”
in the context of measuring government openness. This could be done by measuring (1)
the accountability of the executive branch to an independent oversight institution; and (2)
whether there is an open environment to raise potential issues or concerns in the public
sector. Assessing the ongoing independent oversight by an independent institution over
government policy-making and operations and the transparent management of breaches of
established standards by public sector officials would provide significant insight into the state
of government openness. Methodologies underlying the collection of this data already exist,
as such data is collected through the Transparency International Corruption Perception Index
and the Global Integrity Public Integrity Index.

197. These proposed next steps have the potential to generate greater investment in the statis-
tical capacity to generate new types of data on openness. Indeed, the appropriate monitoring and

96 |
evaluation by a variety of actors, including NSOs, will be front and centre in defining the effectiveness
and impact of openness policies, as well as in assessing progress towards the achievement of all the
SDGs.

HANDBOOK ON GOVERNANCE STATISTICS | 97

 PART B
EIGHT DIMENSIONS OF
GOVERNANCE STATISTICS

CHAPTER VI
Access to
and quality
of justice
 VI.1 Conceptualizing this dimension
VI.1.1 Normative framework and definition of concepts
198. Several international and regional human rights norms concern access to and the quality of
justice. For example, many articles of the Universal Declaration of Human Rights (UDHR) and the
International Covenant on Civil and Political Rights (ICCPR) — two cornerstones of international hu-
man rights norms — directly address access to justice, by articulating procedural due process safe-
guards necessary for people involved with the legal system. For example, Article 14 states clearly that
everyone should be entitled, among other rights, “to be tried in his presence, and to defend himself
in person or through legal assistance of his own choosing; to be informed, if he does not have legal
assistance, of this right; and to have legal assistance assigned to him in any case where the interests
of justice so require, and without payment by him in any such case if he does not have sufficient
means to pay for it”.1 Other articles that are more substantive in nature, such as ones that guarantee
the right to life, liberty, family, education and other basic needs, indirectly but firmly support a right
to legal assistance (including legal representation). Since its establishment, the United Nations (UN)
Human Rights Committee has led the way among UN treaty bodies in interpreting concepts relating
to access to justice. Access to justice is also safeguarded in UN conventions, such as the 1998 Aarhus
Convention on Access to Information, Public Participation in Decision-Making and Access to Justice in
Environmental Matters and the 2006 Convention on the Rights of Persons with Disabilities. In addition,
the 1995 Beijing Declaration and Platform for Action provides guidelines for guaranteeing equal rights
for women and girls, and confirms a determination to “ensure the full enjoyment by women and the
girl child of all human rights and fundamental freedoms and take effective action against violations of
these rights and freedoms”. The “effective action” referenced in this Declaration includes access to
justice as a mechanism for women and girls to realize their rights.

199. While there is no single definition of access to justice,2 it is broadly concerned with the ability
of people to defend and enforce their rights and obtain just resolution of legal3 problems in compli-
ance with human rights standards,4 if necessary, through impartial formal or informal institutions of
justice and with appropriate legal support.5 While legal institutions and services are often a primary
focus of access to justice measurement and policy, they are frequently peripheral to ensuring access
to justice. Access to justice is concerned ultimately with lived experience. Approaches for people-cen-
tered justice measurement are expanding, and this will help deliver on the promise at the heart of the
of the 2030 Agenda to “leave no one behind”.

1 International Covenant on Political and Civil Rights, entered into force Mar. 23, 1976, 999 U.N.T.S. 171, S. Exec. Doc. No. E, 95-2, Articles 2, 10 and 14
available at http://www.ohchr.org/en/professionalinterest/pages/ccpr.aspx.
2 Garth and Cappelletti (1978, p182) observed the concept is “not easily defined”, but focused on “the system by which people may vindicate their rights
and/or resolve their disputes under the general auspices of the state”. However, beyond this focus there are different theoretical, ideological, cultural and
religious conceptualizations of access to justice. These can place different emphasis on “access” and “justice”, generally extend beyond formal process
to informal dispute resolution and, sometimes, to social justice and the distribution of welfare, resources and opportunity (e.g. Maranlou, 2015; Cornford,
2016; OECD and Open Society Justice Initiative, 2019).
3 Legal, or “justiciable”, problems are those that raise legal issues, whether or not these are recognized as such by the individuals facing them, and
whether or not action taken to deal with them involves lawyers or legal process (Genn, 1999).
4 United Nations Development Programme (2005). A reference to conformity with human rights standards is necessary to both extend the concept beyond
local legal frameworks (which may conflict with accepted international norms) and indicate standards for independent adjudication.
5 The debate around access to justice “has many strands” (Paterson 2012) (including citizens’ behaviour in resolving legal problems, the availability and
accessibility of legal services and state-sanctioned dispute resolution processes, the sustainable provision of legal aid, and the role of non-legal services
in delivering justice outcomes) and ideological dimensions. While the concept of access to justice is generally discussed in relation to access to legal
services and processes, debates frequently also extend to other human services that have utility in addressing problems existing within such frameworks.

100 |
 200. Although the sub-dimensions of access to justice are best conceived as being universal across
justice systems, for practical and institutional reasons a distinction is generally drawn between criminal
and civil justice (UN Principles and Guidelines).6

201. A fair, humane and efficient criminal justice system based on the rule of law must respect and
protect the rights of all those affected by crime and/or involved in criminal justice processes.

202. Evidently, effective access to criminal justice involves protecting and enabling proper defence
of the rights of those suspected, accused or convicted of crime. This process starts with appropriate
law enforcement prosecutorial behavior (e.g. UN Women, 2016; Green, 2016) and extends, vitally, to
the provision of legal assistance (including representation) for those subjected to criminal processes
and to proper judicial oversight of such processes. Countless persons are arrested and tens of millions
prosecuted each year (Harrendorf and others, 2010). Globally, nearly one-third of the world’s total
prisoners are held without trial (UN Statistics Division, 2017), which amounts to more than three million
people being held in pretrial detention.7 All are at risk of having their rights ignored or violated. Many
are detained for excessive periods, and an unknown number suffer ill-treatment, coerced confessions
and wrongful convictions. The social, financial and health consequences of pretrial detention can be
severe and disproportionately impact those already disadvantaged (e.g. those who can least afford
bail), reinforcing disadvantage and compounding the underlying causes of crime within and across
generations (Travis and others, 2014). Consequences extend beyond detainees, to “their families,
communities, and even State” (Open Society Justice Initiative and UNDP, 2011).

203. The UN Principles and Guidelines on Access to Legal Aid in Criminal Justice Systems8 calls for
States to “ensure that anyone who is detained, arrested, suspected of, or charged with a criminal of-
fence punishable by a term of imprisonment or the death penalty is entitled to legal aid at all stages of
the criminal justice process.” As the Principles and Guidelines declare, a functioning legal aid system,
including both primary and secondary legal aid (UNDP and UNODC, 2016), can reduce the length
of time suspects are held in detention, congestion in the courts, wrongful convictions, the prison
population, prison overcrowding, reoffending and revictimization.

204. Effective access to criminal justice involves victims of crime being able to identify and report
wrongdoing and see just outcomes obtained through processes that are fair and inclusive of all con-
cerned.9 Significant barriers to justice persist for victims of crime, particularly in the case of the poor
and vulnerable. For example, more than one-third of women worldwide have experienced intimate
partner violence or sexual violence by a non-partner (WHO Department of Reproductive Health and
Research and others, 2013), but only a tiny fraction report such violence to authorities (UN Statistics

6 Criminal justice generally involves State control and sanction, while civil justice concerns relationships and disputes between individuals, communities,
organizations and government.
7 Persons held in pretrial detention are “persons who, in connection with an alleged offence or offences, are deprived of liberty following a judicial or legal
process but have not been definitively sentenced by a court for the offence(s)” (Walmsley, 2016). They can fall into one of four categories, although not all
legal systems and not all cases will involve each category: (1) detainees who have been formally charged and are awaiting the commencement of their trial;
(2) detainees whose trial has begun but has yet to conclude with a finding of guilt or innocence; (3) detainees who have been convicted but not sentenced;
and (4) detainees who have been sentenced by a court of first instance but who have appealed against their sentence or are within the statutory time limit
for doing so (Open Society Justice Initiative, 2014).
8 United Nations Principles and Guidelines on Access to Legal Aid in Criminal Justice Systems, A/RES/67/187, annex (henceforth “UN Principles and
Guidelines”).
9 Within a retributive criminal justice system, inclusivity can refer to, for example, victims being informed of progress or voice being given to victims
through impact statements. In a restorative criminal justice system, it may refer to, for example, the involvement of victims and communities in processes
aimed at restoring relationships broken by crime.

HANDBOOK ON GOVERNANCE STATISTICS | 101

 Division, 2015). The Principles and Guidelines also highlight the importance of legal aid support for
victims and witnesses in the criminal justice process.10

205. Effective access to criminal justice also involves protecting and enabling proper defence of
the rights of those suspected, accused or convicted of crime (e.g. UNODC, 2013). This starts with
appropriate law enforcement and prosecutorial behavior (e.g. UN Women, 2016; Green, 2016) and
extends, vitally, to provision of legal assistance for those subjected to criminal processes and to
proper judicial oversight of such processes (UNODC, 2013; EU Directive 2016/1919). Countless peo-
ple are arrested and tens of millions prosecuted each year (Harrendorf and others, 2010). Globally,
nearly one third of the world’s total prisoners are held without trial (UN Statistical Division, 2017),
which corresponds to more than three million people being held in pretrial detention.11 All are at risk
of having their rights ignored or violated. Many are detained for excessive periods, and an unknown
number suffer ill-treatment, coerced confessions and wrongful convictions. The social, financial and
health consequences of pretrial detention can be severe and disproportionately impact those already
disadvantaged (e.g. those who can least afford bail), reinforcing disadvantage and compounding the
underlying causes of crime within and across generations (Travis and others, 2014). Consequences
extend beyond detainees, to “their families, communities, and even State” (Open Society Justice
Initiative and UNDP, 2011).

206. Effective access to civil justice concerns the people and communities that the civil justice
system is intended to benefit. It includes administrative justice, which is concerned with the laws
surrounding decision-making and dispute resolution of government agencies. It involves enabling
people, communities and organizations to resolve fairly and peacefully the vast number of civil and
administrative legal problems faced in everyday life (OECD and Open Society Justice Initiative,
2019).12 It also involves enabling people and communities to understand, use and shape the law. This
is known as “legal empowerment” (e.g. Maru, 2019; Golub and McQuay, 2001; UN Commission on
Legal Empowerment of the Poor, 2008). As well as giving meaning to the possession of rights (Garth
and Cappelletti, 1978), civil justice promotes general well-being, government accountability and in-
clusive and sustainable development. Although not always needed, legal assistance and legal aid can
be essential, particularly for vulnerable groups.

207. Civil law touches on nearly all aspects of people’s lives, with common legal problems concern-
ing, for example, children, families, consumerism, education, employment, access to public services,
housing, land, natural resources and welfare. These problems are ubiquitous and can disproportion-
ally affect disadvantaged groups (OECD and Open Society Justice Initiative, 2019), which have fewer
resources to avoid, mitigate and resolve problems. They also create and exacerbate disadvantage.
Problems can affect whole communities, such as where a community’s land is taken without any
compensation or where a family is denied identity documents because of their ethnicity, making it
impossible to apply for a job (Maru, 2019). They can substantially impact the lives of those facing

10 Further information on how to ensure quality of legal aid and also on how to evaluate such quality at national level can be found in the UNODC
Handbook on Ensuring Quality of Legal Aid Services in Criminal Justice Processes, 2019, available at: https://www.unodc.org/documents/justice-and-pri-
son-reform/HB_Ensuring_Quality_Legal_Aid_Services.pdf
11 Persons held in pretrial detention are “persons who, in connection with an alleged offence or offences, are deprived of liberty following a judicial or
legal process but have not been definitively sentenced by a court for the offence(s)” (Walmsley, 2016). They can fall into one of four categories, although
not all legal systems and not all cases will involve each category: (1) detainees who have been formally charged and are awaiting the commencement of
their trial; (2) detainees whose trial has begun but has yet to conclude with a finding of guilt or innocence; (3) detainees who have been convicted but
not sentenced; and (4) detainees who have been sentenced by a court of first instance but who have appealed against their sentence or are within the
statutory time limit for doing so (Open Society Justice Initiative, 2014).
12 Civil law here refers to all law that can be applied to or by individuals (or, in a business context, businesses) other than criminal law.

102 |
them,13 affecting health, relationships, income, employment, housing and confidence. This increases
vulnerability to further problems, contributes to the phenomenon of problem clustering and can initi-
ate vicious social cycles. Unaddressed, problems “can cause an economic or social shock that pushes
vulnerable persons into poverty” (OECD and Open Society Justice Initiative, 2019). While difficult to
estimate, the knock-on cost to public services is substantial.14

208. Each year, many millions of people, more often disadvantaged, take no action to resolve even
serious civil legal problems. Many do not recognize their problems as having a legal dimension.15
People are often ignorant of their rights, sources of assistance and options,16 and/or believe action
would make no difference, despite generally “making this judgment without the benefit of any advice”
(Genn, 1999). They also worry about consequences on relationships and personal safety and, in some
places, fairness of processes and corruption.17 When people do act, many do so without assistance.
This can be unproblematic, but many lack the legal capability18 to resolve problems effectively and
perceive the processes of resolution to be unfair and complex.

209. The concept of access to justice is closely linked to the constituent concept of legal need.
Legal need arises when a deficit of legal capability necessitates legal assistance to enable a legal issue
to be dealt with appropriately (OECD and Open Society Justice Initiative, 2019). Legal assistance is
not always necessary to ensure access to justice but must be accessible for those who need it. Access
to justice “is about just resolution, not legal services” (Sandefur, 2019). Appropriate assistance may
be secured from beyond recognized legal professions, from government, independent civil society
organizations and community-based paralegals and may constitute only information.

210. Relatively few civil legal problems are resolved through formal legal processes. In some
countries, traditional dispute resolution processes are more common than court processes.19 Many
problems are addressed through administrative processes, governance mechanisms and complemen-
tary justice mechanisms, such as Ombudsman programmes, national human rights institutions and
national mechanisms for gender equality. Overall, most problems are addressed through informal
methods, directly between parties, unilaterally or not at all. Thus, assessing the achievement of access
to civil justice requires data that reach beyond formal institutions.

13 For example, in Macedonia, 32% of survey respondents described non-trivial problems as “destroying my life” (Srbijanko and others 2013, p. 82).
14 In Canada, the impact of legal problems has been estimated to cost public services “approximately $800 million (and perhaps significantly more)” an-
nually (Farrow and others, 2016). By comparison, the study noted that individuals were spending approximately $6,100 on legal problems, which is “almost
as much as Canadian households spent on average in 2012 on food ($7,739)” and “almost half as much as Canadians spent on average in 2012 on shelter
($15,811).” In the United Kingdom, a similar estimate of the cost to individuals and public services was put at US$ 5 billion per year (Pleasence, 2006, p. i).
15 Adams and others (2017) found that only ~1 in 5 people in their 2018 survey identified their problem as being legal as opposed to bad luck, part of life,
an economic problem, a family problem, etc.
16 Recent findings of legal needs surveys from England and Wales indicate that levels of understanding of legal rights and responsibilities are low
(Pleasence and Balmer, 2012; Pleasence and others, 2015, 2017).
17 For example, in the case of Indonesia, Gramatikov and others (2014) reported that “a deeper look at three of the most frequent and serious categories
of problems — land disputes, crimes and money-related disputes — reveal that people are concerned about the time it takes, the stress and negative
emotions as well as the fairness of the process”. And in Ukraine, Kobzin and others (2011) have categorized the broad barriers to access to justice as
including emotional, informational, physical, financial, effectiveness, bureaucratic, corruption and secondary victimization barriers.
18 Legal capability refers to the capabilities necessary for a person — or, at a higher level, a household or community — to make and carry through
informed decisions so as “to resolve legal problems effectively” (Coumarelos and others, 2012). Some elements include legal confidence, the ability to
recognize legal issues and awareness of law, services and processes. The concept of legal capability links to Sen’s (1980, 1999) capability approach to
disadvantage. For a discussion of legal needs within the context of the concept of legal capability, see Pleasence and Balmer (2019).
19 The World Justice Project’s 2016 General Population Poll found this to be the case in Bangladesh, Thailand and Uganda (Adams and others, 2017). In
many developing countries “legal pluralism is common” (Barendrecht, 2011).

HANDBOOK ON GOVERNANCE STATISTICS | 103

 VI.1.2 A possible taxonomy
211. Access to justice is a complex concept with multiple sub-dimensions. Various taxonomies of
its sub-dimensions have been proposed,20 and several recent efforts have focused on putting people at
the centre of this framework (Task Force on Justice, 2019). Access to justice is usually framed differently in
criminal and civil spheres. However, there is much commonality between their sub-dimensions. Both
extend to, for example, the substance of law, legal capability and empowerment, the accessibility and
quality of legal assistance, the accessibility and quality of processes, and the effectiveness and quality
of decisions and outcomes. It is therefore better to think of criminal and civil justice as requiring lower,
rather than higher, level distinctions within a taxonomy such as that presented in Table VI.1. 21

212. This taxonomy incorporates sub-dimensions from various frameworks for measuring access
to justice (see, for example, Teehankee, 2003; Open Society Justice Initiative, 2015; UN Women,
2016; Dandurand and Jahn, 2017; Rangel, 2018; OECD and Open Society Justice Initiative, 2019) and
accommodates different ideological and cultural conceptualizations of access to justice (e.g. retribu-
tive and restorative approaches to criminal justice), and both individual and systemic approaches to
tackling rights violations (see, for example, Begiraj, Garahan and Shuttleworth, 2018). The domains of
structure, process and outcome refer, respectively, to the resources that are within the justice system,
the use of those resources and the outcomes that their use delivers.

213. The taxonomy makes evident the broad range of sub-dimensions important within different
parts of justice systems and to different stakeholders. The taxonomy exposes the core concerns with,
on the one hand, the needs and capabilities of those who face legal problems and, on the other
hand, the capacity and performance of justice institutions (broadly defined) and wider communities
to deliver just and effective outcomes to such problems. This makes clear the importance of both
demand and supply in the context of access to justice.

214. Any meaningful overall measurement of access to justice must capture the multiple sub-dimen-
sions of access to justice and start from the perspective of people and communities. All of the key
sub-dimensions of access to justice must be represented. To this end, the functioning of justice insti-
tutions and services (central to the process domain in Table VI.1) must be set within the context of the
broad experience of those who need recourse to them (central to the structure domain in Table VI.1) and
of the quality of the outcomes they deliver (central to the outcome domain in Table VI.1). A meaningful
picture should expose the needs of individuals and communities, and how and whether these are met.

VI.2 Why is this dimension important?

215. Access to justice is a fundamental component of the rule of law and central to SDG target
16.3 (“promote the rule of law at the national and international levels, and ensure equal access to
justice for all”). It is foundational for all governance dimensions addressed in this Handbook and is
vital for securing an inclusive and free society. Legal empowerment is crucial. It enables people and
communities to protect and make real their broad array of rights and entitlements — established by
appropriate laws in compliance with human rights standards — and address grievances in a fair and
non-violent manner. Effective access to justice provides checks to State power and promotes govern-
ment accountability. It impacts the functioning and outcomes of public institutions and contributes
20 Often in the context of in national, regional and global measurement frameworks.
21 Particularly as the distinction between them will often not feature in customary and religious justice systems (Barendrecht, 2011).

104 |
TABLE VI.1 Sub-dimensions of access to justice
Domain Primary Sub- Secondary Sub-dimension Examples and areas of interest
dimension
Structure Environment Favourability (i.e. factors outside system IT and transport infrastructure, security,
impacting on experience/outcomes) structural inequalities, etc.
Legal frameworks Conformity (e.g. with international human rights
standards, free of discrimination, etc.)
Population Legal issues Incidence, nature, seriousness, individual/collective, etc.
Capability Legal capability and Awareness of rights/rights violations/services, confidence, etc.
empowerment
Public legal education resources Government/civil society/community, funding, staffing, etc.
Legal State legal aid schemes Type (lawyer/paralegal, government/independent), level/
assistance Independent legal assistance form of funding (incl. pro bono), level of staffing, level
(including of experience/expertise, coverage, eligibility criteria/
representation) Community level legal advice, assistance and empowerment level, integration in other services, geographical
Justice Police/prosecutorial authorities accessibility, security (of staff, buildings, etc.), etc.
institutions Formal courts, quasi-judicial bodies, etc.
Complementary bodies (Ombudsman schemes, human rights
commissions, community-based monitoring systems, etc.)
(see, for example, Begiraj, Garahan and Shuttleworth, 2018)
Traditional / religious / non-formal dispute resolution mechanisms
Other community bodies
Process Capability Empowerment Quality, independence, accessibility, efficiency, etc.
Public legal education practice
Legal Legal aid functioning Caseload, quality (process, etc.), independence,
assistance Other service functioning timeliness, accessibility (legal issue, cost, language,
(including etc.), unmet need, perception (trust, etc.), efficiency,
representation) protection of staff, monitoring, etc.
Justice Police/prosecutorial authorities Caseload, quality (procedural, interpersonal, informational -
institutions Formal courts, etc. see, for example, Klaming and Giesen, 2008), overall fairness,
independence, duration, accessibility (cost, language,
Community institutions, traditional bodies, etc. etc.), perception (trust, etc.), efficiency, protection of
Complementary bodies (Ombudsman schemes, human rights staff, monitoring, etc.
commissions, etc.)
Other paths Individual Volume, quality, duration, accessibility (cost, language, etc.),
to justice Community perception (trust, etc.), efficiency, alignment
with other mechanisms, proximity, etc.
Detention Frequency, legitimacy, etc.
(pretrial, etc.)
Outcome Form Resolved, ongoing, etc.
Quality Case Form Retributive, restorative, distributive, etc.
Transparency Reasoning, public record, etc.
Functionality Attrition, compliance, enforcement, delay, etc.
System Effectiveness, etc.
Accessibility/ reach of legal assistance and
dispute resolution mechanisms, etc.
Perception Trust, fairness, confidence, satisfaction, etc.
Impact Individual outcomes Conclusion, empowerment, social, economic, health, well-being, etc.
Systemic outcomes Change in law, process, policy, etc.
Broader outcomes (community, etc.) Empowerment, economic, social, health, well-being, etc.

HANDBOOK ON GOVERNANCE STATISTICS | 105

 BOX VI.1 Mapping legal needs to better plan legal assistance services

In Australia, evidence from legal needs surveys and other legal needs research has been used to guide the
development of new indicators of the need for legal assistance services. Insights concerning the vulnerability of
different groups and predictive factors for experiencing legal problems have been used to develop Census-based
measures of the geographic distribution of potential demand for legal assistance services. Separate indicators
serve as proxy measures of the minimum number of people likely to need the support of legal assistance services
from different groups, including people with low legal capability, indigenous people and people from culturally
and linguistically diverse backgrounds. Significantly, these indicators of legal need can be mapped down to
various geographic levels — something that legal needs survey data and administrative data do not generally
allow.

Having mapped the legal need indicators, policy makers can then assess whether services are available and
delivered to meet needs. The Law and Justice Foundation’s data digests allow the data from a number of legal
assistance services to be combined and mapped to the same geographies as the legal needs indicator, thus
allowing a comparison of services delivered to the need for those services. While not being a precise measure of
which services were delivered to which person in need, it nevertheless provides a sound approximation for policy
makers and service providers to identify the gaps in service delivery to meet legal need.

Source: Cain, M., 2007, Data digest reports, Law and Justice Foundation of NSW, Sydney.

to the building of trust in such institutions. Failure to ensure effective, timely and equal access to
justice results in substantial individual and societal costs (spanning economic, social, environmental,
and health and well-being costs), which disproportionately impact on the poor, women, children and
other vulnerable groups (OECD and Open Society Justice Initiative, 2019). This means that access to
justice and its key sub-dimensions are relevant not just in the context of SDG target 16.3 but also in
the context of the other SDGs, for example, those concerning poverty (SDG 1), health and well-being
(SDG 3), gender equality (SDG 5) and inequality (SDG 10). Such an intersectional approach is core to
the SDGs.

216. Gaining a multi-dimensional perspective on access to justice as discussed in the preceding

section can be transformative. Expanding focus to people, agency and outcomes, rather than just
the functioning of institutions and services, allows reimagining of justice systems with the population,
users and practical outcomes at their heart. The impact on policy and practice in countries that have
expanded their focus in this way has been profound (Pleasence and others, 2014). Exposing unmet
needs and systemic barriers at a level of disaggregation enables the targeting of resources and the
development of strategies to ensure no group is left behind. Exposing obstacles that prevent people
from acting or accessing assistance or processes promotes innovation to build on trusted intermedi-
aries, remove obstacles, and empower people and communities to overcome them. Exposing inad-
equacies in outcomes enables policy makers to orient reform efforts towards what Amartya Sen calls
“actual realizations and accomplishments, rather than only the establishment of what are identified
as the right institutions and rules” (Sen, 2009). Moreover, a focus on people and outcomes enables
integration of justice policies with wider social policies, highlighting connections between sectors and
allowing upstream and downstream costs and benefits to be mapped.

106 |
VI.3 Data and best practices currently available

217. Insight into different sub-dimensions of access to justice requires the use of multiple data
sources. Administrative data are essential for producing measures and indicators relating to institutions,
formal processes, legal services and citizens’ interactions with these. However, while administrative
data provide important information about those who come into contact with institutions and services,
they provide no information about those who do not. Thus, their reach is limited. In order to establish
broad patterns of experience of, and responses to, criminal and civil legal problems, general popu-
lation surveys are essential. However, the ability of population surveys to capture details of relatively
rare processes is inherently limited and, in any event, respondents generally have only a superficial
understanding of technical processes. Beyond administrative and general population survey data, user
surveys and qualitative reviews, such as expert assessment, can provide information to assess the quality
of processes and services and their outcomes. Expert review can also provide data concerning the legal
and institutional framework within which justice is delivered, which constitutes essential context for any
full picture of access to justice. Evidently, measuring access to justice requires triangulating data sources.

218. Because many actors are involved in enabling access to justice, data production is spread
across a broad institutional landscape. Public institutions, civil society, the private sector and research-
ers all play important roles in the production, analysis and use of justice data. Sharing and linking
such data can be important to addressing the complexity of access to justice. To build capacity and
meet statistical needs, the 2017 Cape Town Global Action Plan for Sustainable Development Data
recommends that efforts should be made to develop quality frameworks and strengthen partnerships
that enable statistics from non-official sources to be used by national statistical systems, provided they
are in full adherence with UN Fundamental Principles of Official Statistics.

VI.3.1 Administrative data

219. Law enforcement and justice sector institutions routinely collect administrative data in the course
of their everyday activities. Official data are usually collected by police and prosecutorial authorities,
courts, prison administrations, Ombudsman schemes and regulators; other valuable data are pro-
duced by civil society organizations such as legal aid schemes, alternative dispute resolution (ADR)
mechanisms, legal empowerment groups, legal services and lawyer associations. They are collected
for management purposes and to fulfill accountability, regulatory and reporting requirements. They
can relate to many sub-dimensions of access to justice but are particularly suited to providing three
types of justice measures: operations of the justice systems, persons in contact with them, and the
availability and distribution of resources.

220. Data on systems operations refer to events, activities, actions and outcomes of justice system
procedures, both criminal and civil. Operations measures can include, for example: the number of
arrests executed by law enforcement agencies, the volume of spending, active and concluded cas-
es, and the length of proceedings. These data can reveal performance in processing cases through
the system as well as systemic barriers and, when demographic data are available, challenges facing
particular sections of the population. Data available across organizations can be used to create, for
example, flow statistics (for example, the movement of people through different parts and stages of
justice processes) and linkages across justice sector components (including between criminal and
civil justice). Depending on the degree of standardization adopted by the various entities (including
definitions and counting rules), these data can be used to analyse the flow of activities across the
justice system, for example in relation to selected types of offences (for example, on analysing arrests,
prosecutions and convictions on violence against women, corruption offences or environmental crime)

HANDBOOK ON GOVERNANCE STATISTICS | 107

 or for selected sectors (for example, the number of legal issues taken to a legal aid scheme compared
to the number of cases filed in court).

221. Data on people having contact with different parts of the justice system, especially when dis-
aggregated by sociodemographic and other characteristics, can be very useful to reveal whether par-
ticular groups are over- or underrepresented within parts of the justice system, present different needs
or receive different treatment. For example, data on persons arrested, convicted and then serving
sentences or admitted to alternatives to incarceration can illustrate how different persons and different
sectors of society are treated by the criminal justice system. In the same vein, other data can provide
valuable information concerning persons obtaining independent legal assistance, using forms of ADR,
pursuing a particular administrative action or appearing in court with or without legal representation.

222. Various statistics on the resources allocated to the justice sector, along with their distribution,
can provide information on how accessible and prepared the justice system is. Resource measures in-
clude, for example: the budgets of the justice system, such as courts and legal aid schemes. They also
include the number, staffing, capacity, location and types of courts, tribunals, civil society providers,
ADR services, legal aid and other legal services, prison staff and detention centres.

223. In addition to administrative data produced by the justice system, data from a range of other
sectors — health, land and property, labour, etc. — can also potentially provide insight into legal
needs, the accessibility of the justice system and the effectiveness of the justice system as regards
the broader social outcomes it realizes, as well as identifying key areas for reform. Such data could
concern, for example: reports of abuse within health services, exclusions from schools, accidents and
failed safety inspections in workplaces, etc. These data can help identify the universe of issues that
can potentially involve the use of legal services and informal and formal justice processes. Thus, when
combined with data from other sources, they flag areas of potential unavailability or inaccessibility of
justice solutions. They can also, for example, reveal social patterns of experiencing legal problems
and the impact of justice sector activity/interventions on social behaviour and organizational practice
outside of the justice sector. However, the full potential of such data is far from realized.

VI.3.1.1 Good practices and tools

224. Administrative data systems generate large quantities of records, but it is only when raw informa-
tion is purposefully, accurately and congruently collated into statistical form that these records become
valuable for policy makers, analysts, the media and advocates to understand meaningful progress.

225. An effective system of justice-related administrative data must be centred on clear responsibili-
ties for data definition, collection and management. Such a system requires consistent data collection,
to allow linking and comparison. It requires clear data management protocols and strictly adhering
to relevant guidelines and regulations, and must be directed towards regular and accurate reporting
of the working of the justice system. It must allow disaggregation by, for example, institution, issue
and process. Data concerning people interacting with the justice system should allow disaggregation
by variables such as age, sex, citizenship and ethnic affiliation. Such disaggregation can highlight
disproportionate experiences and treatment faced by distinct population groups.

226. Operations of the justice system entail a multitude of registrations that — when compiled and
properly assembled — is able to produce statistics that can be used to assess the effectiveness and
accessibility of the justice system.

108 |
 227. Guidance for collection of data on criminal justice systems (on system operations, persons
passing through the system, and resources) is provided by the United Nations Survey on Crime Trends
and Operations of Criminal Justice Systems (UN-CTS), which every year collects selected statistics on
crime and criminal justice from national authorities through a network of national Focal Points and
publishes derived indicators. In Europe, a dedicated data collection on the formal justice system is
managed by the Council of Europe European Commission for the Efficiency of Justice (CEPEJ), which
also produces guidelines around “judicial maps” (2008).

228. To ensure data of high quality, the various components of the justice system should produce
data in a coherent manner, using standardized procedures and concepts, and sharing data in a trans-
parent manner.

229. To improve data consistency, the United Nations Manual for the Development of a System of
Criminal Justice Statistics suggests that common concepts and classifications (e.g. offence classifica-
tions) should be developed for use across the different components of the criminal justice system,
along with unique identifiers to enable persons and cases to be tracked through the system. Similar
suggestions have also been made in relation to the civil justice system.22 Standards such as those set
out in the International Classification of Crime for Statistical Purposes (ICCS) provide frameworks and
guidance for the collection of accurate, consistent and comparable data. The ICCS is a classification
of criminal offences based on internationally agreed concepts, definitions and principles. In addition,
CEPEJ has produced guidelines to promote quality, transparency, accountability and accessibility of
justice statistics (2008) and the Australian National Legal Assistance Data Standards Manual offers
best practice guidance to facilitate the collection of compatible civil and criminal data across dispa-
rate legal assistance services, ranging from public education to legal representation.

230. Key tools:

• OECD: Equal Access to Justice for Inclusive Growth (2019)

• CEPEJ: Guidelines, Strasbourg: Council of Europe (2008)
• UN: Manual for the Development of a System of Criminal Justice Statistics (2003)
• UNODC: International Classification of Crime for Statistical Purposes (2015)

22 See, for example, the Australian Government Productivity Commission’s (2014) recommendation that “Governments should work together, and with the
legal services sector — including courts, tribunals, ombudsmen, private law firms and legal assistance providers — to develop and implement reforms to
collect and report data that have common definitions, measures and collection protocols.”

BOX VI.2 Access to Justice Centres in Argentina

In 2016, the Government of Argentina began providing people-focused primary legal aid services through
a network of 90 Access to Justice Centres around the country. One of the foundations of this strategy is that
efforts are based on systemic evidence that can be used to inform other branches and levels of government.
A case management system within the centres produces significant sociodemographic information about the
populations served at the primary justice centres, the history of their legal needs (cases, evolution, etc.) and
the services provided by the centres, most of which are addressed without engagement of the legal profession.
Alongside Argentina’s legal needs surveys, these administrative data are proving vital information for the centres
to document the specific access to justice gaps affecting particular populations.

HANDBOOK ON GOVERNANCE STATISTICS | 109

 VI.3.1.2 Data quality

231. Administrative data are collected routinely as part of an organization’s normal operations; they
are generally up-to-date, and they are relatively inexpensive as they rely on administrative information
systems that are already in place. Furthermore, they tend to be comprehensive in their coverage and
provide opportunities to report rare phenomena at a small area level (see, for example, Smith and
others, 2004).

232. Data quality challenges often arise from the fact that administrative data are a by-product of
administrative activities and follow operational or regulatory criteria rather than statistical principles,
all of which change over time and differ between organizations and jurisdictions. Furthermore, the jus-
tice sector, in particular the civil justice sector, is composed of a plurality of entities often characterized
by institutional independence and a diversity of organizational functions and responsibilities.

233. A specific issue may arise in relation to units of measurement, which can vary considerably
across justice systems, reflecting different perspectives and operational concerns. Legal issues, pro-
cesses and assistance can be counted in different ways. For example, a single victim might have
experienced multiple offences, and the counts of crime victims and criminal offences may diverge
significantly. Or the police may arrest and charge two suspects in relation to the same offence, or
one suspect in relation to multiple offences. Similarly, a single civil dispute may involve one or many
discrete legal problems. A person may seek help from a single adviser concerning multiple civil legal
problems, or multiple advisers concerning a single problem, and a dispute may involve a single or
multiple distinct dispute resolution processes (e.g. negotiation, mediation, litigation). If different or-
ganizations and jurisdictions apply different counting rules in relation to persons, offences, cases and
the like, then data are not equivalent and direct comparison is not possible.

234. Increased inter-institutional coordination and the use of common statistical standards are ways
to improve the quality and availability of administrative data from the justice sector. International sta-
tistical standards such as the ICCS and the metadata section of the UN-CTS provide guidance on how
to produce standardized statistics on criminal offences across the various stages of the criminal justice
system, while the UN-CTS suggests standards counting units and counting rules when producing data
at the different stages of the criminal justice process.

235. Administrative data in the justice sector are usually subject to management oversight, and it
is important that the statistical activities strictly follow the Fundamental Principles of Official Statistics.
This is particularly important when the data produced by an institution are used — directly or indirect-
ly — to assess its performance. In such cases, measures to promote data transparency and integrity
are important to avoid any possible conflict of interest and to maintain trust in the data by external
users and the public.

VI.3.2 Population survey data

236. Population survey data are collected through surveys of the general population, whether of
individuals, households or specific sub-populations, such as indigenous peoples or persons with dis-
abilities, or through user surveys.

237. Population surveys related to access to justice are conducted for broad sector monitoring and
strategic purposes. They are a very important data collection tool for capturing the lived experiences
of particular populations. They are thus important for supporting people-centred access to justice

110 |
policies. They may be implemented by NSOs, government departments, research organizations, civil
society organizations or other stakeholders in the justice system.

238. Reflecting the common policy split between criminal and civil justice, the two main kinds of
access to justice population survey are the “crime victimization survey” and the “legal needs survey”.
In looking beyond specific institutions and services,, these surveys provide a unique overview of the
experience of, in the first case, crime and, in the second case, civil legal problems faced by individuals,
groups and communities.

239. Crime victimization surveys are a standard source of crime statistics. As only a small propor-
tion of criminal incidents are reported to the police, administrative data on its own cannot provide a
comprehensive picture of crime (UNODC-UNECE, 2010). Crime victimization surveys capture criminal
incidents, both reported and not reported to the police, by randomly selecting a sample of the pop-
ulation and asking them directly about their experiences of victimization. They allow estimation of
the amount of unreported or undiscovered crime (see also the “Safety and Security” chapter of this
Handbook for further information on crime victimization surveys).

240. As well as establishing the level and characteristics of a broad range of criminal offences,
victimization surveys can also investigate, for example: perceptions of safety; characteristics of vic-
tims; the impact of crime on victims; patterns of crime reporting and reasons for non-reporting to
competent authorities; attitudes towards the police; awareness/use of victim support services; and
experiences of criminal justice processes. Thus, victimization surveys can identify at-risk populations,
reveal barriers to crime reporting and the use of victim services, expose shortcomings in criminal
justice processes from the victim’s perspective and, in general, assist with the prioritizing and targeting
of resources.

BOX VI.3 South Africa’s Governance, Peace, Security and Justice Survey

In 2018, Statistics South Africa adapted their victims of crime survey into a large-scale Governance, Peace, Security
and Justice Survey (GPSJS) that aims to improve understanding of the scope of legal needs, sources of information
and advice, and strategies that people rely on to resolve them. It is exploring ways to gather victimization data
and legal needs data in rotating years to provide a more comprehensive picture of access to justice.

The 2018/19 GPSJS measured trends in various types of crime over the past five years. Respondents were asked
about possible measures they took to protect themselves against crime.

About 31% of adults aged 16 and older took various physical protection measures such as installing burglar doors,
21% resolved to walk only during safe hours, 18% decided to be more alert about their surroundings and to use
safer routes, 13% resolved not to walk alone and 7% procured the services of private security. However, 62%
of adults aged 16 and older did not take any action to protect themselves from crime. About 27% of the total
did not take any measures against crime because they felt that there was nothing they could do, 22% thought
crime-fighting was the work of the police, 19% did not know what to do, 17% took no action because of a lack
of money, 10% felt that their action would not make any difference, and 4% were still thinking about the action
they would take.

Source: Governance, Public Safety and Justice Survey: 2018/2019, Statistics South Africa.

241. Similarly, legal needs surveys are increasingly a standard source of civil justice statistics, with
more than 60 national surveys conducted since the mid-1990s, in over 30 jurisdictions, along with many

HANDBOOK ON GOVERNANCE STATISTICS | 111

 sub-national surveys (see, for example, OECD and Open Society Justice Initiative, 2019). Legal needs
surveys investigate the experience of legal problems from the perspective of those who face them
(“bottom-up”), rather than from that of justice professionals and institutions (“top-down”). Thus, they
are able to identify and explore the full range of responses to problems and, within these, of sources
of help and institutions utilized in problem resolution. They are also able to establish the manner of
resolution of legal problems, whether or not they involve justice institutions or services. They are vital
to measuring and monitoring the implementation of the SDG commitment (target 16.3) of leaving no
one behind in accessing justice.

242. Legal needs surveys are distinct from victimization surveys, as their focus is on civil legal is-
sues (family, commercial, administrative, etc.), although they sometimes inquire into the experience
of crime. Beyond the nature of problems (including the parties involved, seriousness and impact),
sources of help, and the processes employed to resolve problems (including reasons for use/non-use,
modes of communication and quality), legal needs surveys routinely investigate problem impact, the
costs and duration of problem resolution, “legal capability”,23 and (although it is not their prime focus)
perceptions and attitudes. Analysis of individual cases may point to structural problems. Legal needs
surveys have had a profound impact on the direction of justice policy and forms of service delivery
in countries in which they have become established, shifting the focus towards targeting, outreach,
joined-up services, timeliness of service delivery and legal capability/empowerment (see, for example,
Pleasence and others, 2014).

BOX VI.4 A Stand-alone Access to Justice Survey in South Korea

In 2018, a Nationwide Legal Needs and Access to Justice Survey was undertaken to support policy in South Korea.
The survey piloted the illustrative longer form questionnaire contained in the OECD-OSJI Legal Needs Surveys and
Access to Justice. Diverse methods were used, such as face-to-face and online. Among the general public, 15.2%
(324 of 2,317 respondents) had experienced a legal problem/dispute in the last four years. Some key findings
included: the most serious problems were related to housing, employment/labour and family; the highest
source of information, advice or representation were family, friends or acquaintances, with 32%, followed by
a lawyer, professional adviser, advice service or advice helpline (24.9%). A high percentage (61.2%) of people
tried to resolve the problem by communicating with the other party; and the percentage of people resolving the
problem on their own was high among well-educated and above middle-income people, with about 72.6% of
respondents expressing their trust in the courts.

Source: Kim, S. and Choi, S. (2018) Reflections and Lessons from the 2018 Nationwide Legal Needs and Access to Justice Survey in South Korea, unpublished
report prepared for the Judiciary of the Republic of Korea.

VI.3.2.1 Good practices and tools

243. Surveys on access to justice should, first and foremost, follow best practice in the survey re­
search field.24 Well-designed and implemented surveys can yield invaluable insight into the nature,
extent and impact of people’s experience of legal problems.

23 Relatively little scrutiny has been given to the quality of legal capability data, although the psychometric properties of some scales have been tested.
See, for example, Pleasence and Balmer (2019).
24 General guidance is readily available on survey research, including more particular guidance on its conduct in developing and transitional countries and
in the justice sector. For general guidance see, for example, United Nations Department of Economic and Social Affairs (2005a), Groves and others (2009)
and Wolf and others (2016). For guidance in the context of developing and transition countries see, for example, UN Department of Economic and Social
Affairs (2005b). For guidance in the context of justice see, for example, Himelein and others (2010).

112 |
 244. Beyond this, general guidance on victimization surveys is provided by the UNODC-UNECE
Manual on Victimization Surveys, while the International Classification of Crime for Statistical Purposes
(ICCS) provides global standards for classifying criminal offences and the OECD-OSJI Legal Needs
Surveys and Access to Justice provides comprehensive guidance for the conduct of legal needs surveys.

245. Much literature also surrounds the many national crime victimization and legal needs sur-
veys that have been undertaken across the world in recent years,25 along with cross-national surveys
such as the International Crime Victim Survey (ICVS)26 and World Justice Project’s General Population
Poll. Further resources are also disseminated by, for example, the Center of Excellence in Statistical
Information on Government, Crime, Victimization and Justice.27

246. Surveys related to access to justice need not be conducted annually. Periods between surveys
need only reflect periods over which significant change is likely. Moreover, surveys need not be stand-
alone. A number of countries have included victimization modules in broader surveys of social issues.
Legal needs modules have also been adopted into seven broad national government surveys, as well
as the World Justice Project’s General Population Poll, routinely implemented in over 100 countries
(Adams and others, 2017, 2018). See Annex VI.

2474. Key tools:

• OECD-OSJI Legal Needs Surveys and Access to Justice (2019)

• UNODC International Classification of Crime for Statistical Purposes (2015)
• UNODC-UNECE Manual on Victimization Surveys (2010)
• OECD Access to Justice for Inclusive Growth: Putting People at the Centre (2019)

VI.3.2.2 Data quality and methodological considerations

248. Surveys on access to justice give rise to a variety of unique data quality and methodological
considerations that are situated within a unique conceptual framework and concerned with a unique
and broad range of issues.

249. Access to justice surveys often have as their subject matter issues that are sensitive, can cre-
ate embarrassment, are potentially traumatic, and could place respondents in jeopardy if they are
divulged. Thus, a key consideration in relation to survey mode and conduct must be to provide a safe
situation for respondents to divulge accurate information.

250. Physical privacy is clearly an important consideration in the conduct of interviews about sen-
sitive issues, and this is particularly important in relation to, for example, questioning about sexual
assault, intimate partner violence, child protection and the like. Without adequate protocols and
assurances of confidentiality, data quality may be poor. Furthermore, questions and interview settings
should reduce the possible impact of social desirability bias on respondents’ willingness to answer
sensitive questions accurately.

25 Further insight into the design and use of legal needs surveys can also be found in the many publications coupled to the major repeated surveys in the
field, including the English and Welsh Civil and Social Justice Survey and HiiL’s Justice Needs and Satisfaction Survey.
26 Details of the ICVS can be found at http://wp.unil.ch/icvs/
27 http://www.cdeunodc.inegi.org.mx/index.php/en-2/

HANDBOOK ON GOVERNANCE STATISTICS | 113

 251. Survey questions on access to justice often concern complex and/or technical concepts, such
as the nature of criminal offences, civil legal problems and aspects of formal dispute resolution pro-
cesses. It can be difficult to formulate questions that clearly articulate the nature and/or scope of the
information sought. If clear articulation is not achieved, error is inevitable. Small changes in the form
of questions can affect the way in which they are understood and significantly alter findings (see, for
example, Pleasence and others, 2016).

252. For example, people’s understanding of criminal law precludes asking respondents whether
they have been “victims of crime”. Good practice is to ask simply whether respondents have expe-
rienced defined sets of circumstances that, for analytical purposes, can be taken to be equivalent to
criminal victimization.

253. Similarly, people’s poor understanding of law and legal concepts more generally precludes
legal needs surveys from asking directly about legal problems. People’s generally narrow preconcep-
tions of law are such that it has been found that simply referencing law in framing surveys significantly
reduces problem reporting (Pleasence and others, 2016). Consequently, good practice is to avoid
reference to all legal and technical terminology, both in framing surveys and within questions. As with
victimization surveys, the identification of legal problems ideally involves presenting respondents with
sets of circumstances that equate to problems.

254. The technical nature of legal processes and service delivery also means that people often may
not recognize or understand complex and/or technical aspects of dispute resolution processes. Thus,
data concerning such aspects of dispute resolution tend to be poor.

255. Another issue affecting surveys on access to justice is that legal issues extend to almost all
aspects of life. Not only do types of criminal offences vary hugely, but the diversity of civil legal prob-
lems is almost as broad as the diversity of people’s behaviours. Population surveys are generally highly
inefficient tools for collecting data concerning rarer legal issues. The prevalence, value, impact and
cost of problems are common criteria for prioritizing issues for inclusion in surveys.28

256. A specific issue affecting the comparability of survey results is that justice institutions and
procedures, as well as possible sources of assistance and dispute resolution processes, vary between
countries and over time. If comparison is important, data must be collected in a conceptually and
methodologically consistent manner. As with other surveys, small differences can significantly impact
on findings. Terminology is important, but may sometimes be difficult to translate between countries,
cultures and languages.

257. In the case of crime, officially recognized authorities may include police, prosecutors, other
criminal investigators and, in some jurisdictions, bodies involved in various informal justice or dispute
resolution processes (e.g. tribal or religious leaders, village elders).

258. As with SDG indicator 16.3.1, victimization survey data offer the potential for comparison
across countries and time, even if patterns of reporting to authorities vary. But comparisons between
different countries “should always be approached with caution” (UNODC-UNECE, 2010). Beyond
methodological differences, which can be controlled to a reasonable extent, there are also differences
between populations as regards perceptions of crime or the cultural acceptances of criminal practices.

28 See, for example, Barendrecht and others (2008). Surveys tend to focus on problems concerning family, employment, housing/land, neighbours, money,
government services (including welfare), consumer issues and injuries (due to negligence)

114 |
It is difficult to control for these. Nevertheless, while flawless comparison may be impossible, “data
from victim surveys are likely to be more easily internationally comparable than [administrative data,
for example] police records” (UNODC-UNECE, 2010).

259. In the case of civil justice, more than 40 different terms have been used to describe lawyers
in past surveys, and the term “mediation” is applied to various forms of intermediation, conciliation,
arbitration and adjudication (OECD and Open Society Justice Initiative, 2019). To assist survey design-
ers, taxonomies of sources of help and of dispute resolution processes, along with model questions,
are included in the 2019 OECD-OSJI legal needs survey guidance, Legal Needs Surveys and Access
to Justice.

260. Importantly, in the case of legal needs surveys, in order to generate a comprehensive picture of
people’s problem-resolving behaviour, three separate areas of activity must be addressed: help-seek-
ing, the use of dispute resolution processes, and other activities that support problem resolution. It is
important not to conflate help and process. The former is specific to the survey respondent, whereas
the latter can occur with or without the respondent’s initiation or engagement.29

261. Also, as with administrative data, for both victimization and legal needs surveys consideration
must be given to units of measurement. As was detailed above, counting crimes and civil legal prob-
lems is far from straightforward. As well as potentially involving many different combinations of par-
ties, crimes and civil legal problems may or may not be discrete events.30 For example, harassment at
work may transpire over a period of time and potentially be counted as a series of individual episodes
or as one ongoing problem. Moreover, as surveys can be of households or individuals, prevalence and
incidence rates can relate to either. The UNODC-UNECE Manual on Victimisation Surveys examines
in depth how crimes and victims might be counted by victimization surveys.

262. Units of measurement must also be considered when comparing survey and administrative
data to ensure that like is compared with like (at least as far as this is possible).31

263. Finally, legal needs surveys owe their name to the concept of legal needs, yet few have sought
to operationalize the concept for the purposes of measurement. But unmet legal need is often a
key policy concern. Looking at the definition of legal need in the first section in this chapter, proxy
measures require data on the quality of process and/or outcome, legal capability, legal assistance and
the appropriateness of assistance. Examples of questions for such measures and further discussion are
set out in the OECD-OSJI Legal Needs Surveys and Access to Justice.

VI.3.3 User surveys

264. User survey data can provide unique insight into the experience of and challenges faced by
users of justice institutions and services. This includes surveys of those detained by the police, being
tried for criminal offences (see, for example, Kemp and Balmer, 2008; Open Society Justice Initiative

29 To maintain conceptual clarity, questions about help and process should clearly distinguish them and allow a distinction to be made between them.
30 Counting requires rules addressing such complexities. In broad terms, rules can be prevalence-based (e.g. the percentage of a population victimized
during a given period) or incidence-based (e.g. the number of crimes occurring during a given period). The number of crimes can see multiple victim
offences counted as single crimes or as a number of episodes of individual victimization. High-frequency repeat victimization, common in intimate partner
violence, school violence and workplace violence, can be counted as multiple crimes or single crimes or, as is now the case for the US National Crime
Victimization Survey, multiple crimes subject to a cap — in this case, ten (Lauritsen and others, 2012).
31 For example, adjustments should be made to ensure that geographical coverage, offences included and populations are the same. Plus, as emphasi-
zed by the most recent report of the South African Victims of Crime Survey, because victimization surveys use lay language to describe the circumstances
of crime, “victim surveys deal with incidents that do not necessarily match the legal definition of crime” (Statistics South Africa, 2018).

HANDBOOK ON GOVERNANCE STATISTICS | 115

 and UNDP, 2011) or, as in the case of the Mexican National Survey of Population Deprived of Liberty,
of inmates, as well as users of civil courts, alternative dispute resolution processes, legal information
and assistance services, etc. Such surveys can be undertaken by institutions and services themselves
but also frequently by external researchers.

265. User surveys generally benefit from complete sample frames, although access to users may
need to be negotiated. Access to users can be problematic while processes are ongoing or during
service use, as users are likely to be preoccupied with the justice issues at hand or otherwise be in an
unfavourable state to provide information for other purposes.

266. As in the case of population surveys, data quality may be poor in relation to technical aspects
of processes and service use. However, users can provide unique insight into, for example, the nav-
igability and clarity of processes or perceptions of process and service quality. In the case of pretrial
detainees, their perspectives help fill a gap in what is known about the socioeconomic consequences
of pretrial detention. Survey framing and questions should avoid the use of technical language when-
ever possible.

267. CEPEJ have produced a handbook for conducting court user satisfaction surveys (2016).
Guidance in relation to quality was discussed above in the context of population surveys. OECD has
helped develop user satisfaction surveys for ADR in some contexts. There are also many examples of
user surveys available for reference.32

VI.3.4 Expert assessments

268. Together, administrative and population survey data can provide extensive insight into justice
system resources, the operation of processes, and the experience of crime and civil legal problems.
However, administrative and population survey data are less well suited to assess the quality of legal
and political frameworks within which resource allocation, processes and experience unfold.

269. Expert assessment involves the collection of data from “experts” in a field (researchers, practi-
tioners, responsible government officials, etc.) using, primarily, questionnaires structured to reflect the
conceptual framework for assessment.

270. In the field of access to justice, expert assessment can provide otherwise valuable information
concerning the quality of legal frameworks and policies supporting access to justice as well as about
their implementation. This is a highly complex task, and experts can complement quantitative infor-
mation with their informed and comprehensive evaluations.

271. Expert assessments are relatively low cost; they involve subjective appraisal and are generally
non-statistical in nature. Nevertheless, they can provide critical information on the quality of legal/
policy frameworks and on their implementation, and they are common in the context of governance
(González, Fleischer and Mira d’Ercole, 2017).

272. Expert assessments can be implemented and overseen by international organizations, think-
tanks and research networks — for example, the CEPEJ evaluation scheme for judicial systems
(CEPEJ, 2018a) and the EU Justice Scoreboard (CEPEJ, 2018b). In some cases, expert assessments

32 See, for example, LaGratta and Jensen (2015).

116 |
are consolidated in composite indicators, such as the World Justice Project’s Rule of Law Index, which
references access to justice.

273. The type of expert best suited to assessment varies with the subject matter. In general, re-
searchers and practitioners tend to be well placed to assess the quality of laws and manner of im-
plementation, while responsible officials tend to be better placed to assess technical details about
process and service functioning.

274. Data provided by individual experts are susceptible to bias, reflecting their political perspec-
tives, and in some cases they are exposed to conflicts of interest in delivering assessments. Inevitably,
different experts will sometimes have different opinions on the same matter. More generally, the
validity and reliability of data obtained through expert assessment is unclear. Non-trivial differenc-
es between similarly constructed measures derived from expert assessments have been reported
(Kaufmann and Kraay, 2007). Also, the opinions of experts may differ from those of the public. While
this does not speak to their validity or reliability, it is important to recognize that such differences may
exist.

275. No comprehensive guidance has yet been produced for the conduct of expert assessments.
Nevertheless, methodological accounts of expert assessment are available in relation to various in-
dicators on access to justice, such as the Rule of Law Index and Justice Index, that draw on expert
assessment. Although not specifically directed towards expert review, the UNODC Criminal Justice
Assessment Toolkit also provides frameworks and guidance for assessing criminal justice systems and
the implementation of criminal justice reform. The toolkit includes four sections on access to justice
(the courts; the independence, impartiality and integrity of the judiciary; the prosecution service; and
legal defence and legal aid). Each section sets out basic issues, a detailed conceptual framework and
list of key documentary data sources. The Global Study on Legal Aid of the UNODC/UNDP also in-
cludes a framework and indication of data sources. (See also the chapter in this Handbook on “Cross-
cutting considerations” for general guidance on expert assessments.)

276. As understanding the reliability and validity of expert assessment data on access to justice is as
yet relatively limited, users of such data “should be cautious about relying overly on one set of expert
assessments” (Kaufmann and Kraay, 2007).

HANDBOOK ON GOVERNANCE STATISTICS | 117

 VI.4 Recommended key indicators

277. Illustrating the breadth of potential access to justice indicators, the British Columbian Access
to Justice Measurement Framework incorporates 100 measures (Dandurand and Jahn, 2017), the UN
Women Framework for Measuring Access to Justice Including Specific Challenges Facing Women
incorporates almost 70 indicators (UN Women, 2016), and the Open Society Justice Initiative’s nar-
rowly focused guide to the effective use of indicators in relation to pretrial justice incorporates nine
indicators (Open Society Justice Initiative, 2015).

278. These examples illustrate that there is no optimal number of indicators to measure the wide
range of potential access to justice priorities. Different countries will have different capacities to em-
ploy indicators and priorities to address. However, if an indication of the effectiveness of access to
justice policy is required, then indicators should extend across the spectrum of sub-dimensions set out
in Table VI.1 and address the key domains. Ideally, they should refer to: a population’s level of legal
capability and empowerment; the level of need of those who experience civil legal problems; the level
of need of those who experience crime or are suspected/prosecuted by criminal justice authorities;
the accessibility of criminal justice institutions for those in need; the accessibility of independent legal
advice through lawyers and non-lawyers; the quality and efficiency of resolution processes (including
the safety/well-being of those involved in criminal processes); and the quality and effectiveness of
outcomes. All indicators should also be disaggregated by issue, process type and participants’ demo-
graphic traits. Such indicators should demonstrate the extent to which the justice system delivers its
objectives and meets the needs of those who rely on it.

279. Table VI.2 sets out a number of recommended indicators. Together, they provide the type of
broad overview necessary to understand and implement people-centered access to justice policies.
They also illustrate the diversity of forms and data sources of the many potential indicators that could
provide insight across the wide spectra of access to justice sub-dimensions. Other relevant indicators,
such as those found in the chapters on non-discrimination, absence of corruption and trust, can be
taken into account, as can initiatives such as UN-CTS.

280. The first structural indicator is focused on the de jure legal frameworks that govern justice
systems, along with the de facto reality of practice in them. The second and third structural indicators
are survey-based indicators designed to capture aspects of legal capability and legal empowerment,
a foundation for people-focused access to justice policy. The fourth structural indicator is focused on
access to justice resources — specifically, the resources allocated to criminal and civil legal assistance
and legal aid. This is routinely reported by CEPEJ. By using a denominator of overall justice system re-
sources, the indicator points to the balance of resources within justice systems (European Commission
for the Efficiency of Justice, 2018). Changes over time would suggest changes in the relative power of
different players or institutions within systems. The final structural indicator focuses on access to justice
personnel — specifically, lawyers and paralegals. A denominator of people in poverty, as employed in
the National Center for Access to Justice US Justice Index, highlights cost barriers for securing legal
assistance, particularly in civil cases (2016).

118 |
TABLE VI.2 Recommended key indicators

Domain Sub-dimension Indicator Data source(s)

Structure Environment The extent to which national law and practice concerning access to Expert assessment
justice are in accordance with international human rights norms

Capability Proportion of population aware of specific laws/ rights relevant in the national context Population surveys

Proportion of population confident in their ability to access

legal information and assistance if needed

Annual expenditure for criminal and civil legal assistance and legal aid Administrative data
as a proportion of total justice budget, by expenditure type

Number of legal aid providers [lawyers and community-based paralegals] per

100,000 general population / population in poverty, by type and practice area

Process Legal assistance Proportion of those legal problems that were experienced in the last two years and Legal needs surveys
(including for which people needed assistance and for which people were able to obtain it
representation)
Proportion of persons in police custody, in pretrial detention, and appearing before Administrative data
courts / tribunals who have legal representation / assistance, by type of proceeding / user surveys

Quality of services received (e.g. legal aid service or other service or process) Expert assessment, third-
party (e.g. peer, supervisor)
assessment), user surveys

Justice institutions Proportion of victims of violence in the previous 12 months who Crime victimization surveys
reported their victimization to competent authorities or other officially
recognized conflict resolution mechanisms (SDG indicator 16.3.1)

Proportion of the population who have experienced a dispute in the past Legal needs surveys
two years and who accessed a formal or informal dispute resolution
mechanism, by type of mechanism (proposed SDG indicator 16.3.3)

Existence of a system for monitoring and evaluating the justice system Expert assessment

Average length of process in days, by type of process Administrative data

Detention Number of unsentenced detainees as a proportion of the

overall prison population (SDG indicator 16.3.2)

Unsentenced detainees by length of detention

Number / proportion of pretrial detainees who are

acquitted or receive a non-custodial sentence

Outcome Form Proportion of those who experienced a legal problem in the last two years, could Legal needs surveys
access appropriate information or expert help, and were able to resolve the problem

Form Number / proportion of pretrial detainees who are Administrative data

acquitted or receive a non-custodial sentence

Quality / Perception Proportion of legal problems experienced in the last two years that were resolved Legal needs surveys
within a reasonable period of time to the satisfaction of those facing them

Quality / Impact Number of suicides in prison, per 100 of untried / unsentenced Administrative data
prison population, disaggregated by prisoner status

Number of intentional homicides in prison, per 100 of prison population

Quality / System Ratio between number of persons convicted of intentional Administrative data
effectiveness homicide and number of intentional homicides

Ratio between persons convicted and persons arrested/suspected (per 100

persons arrested/suspected), disaggregated by type of criminal offence

HANDBOOK ON GOVERNANCE STATISTICS | 119

 BOX VI.5 Index of Effective Access to Justice in Colombia

The 2014-2018 Colombian National Development Plan included the measurement of people’s legal needs as an
objective in the Handbook chapter on strengthening access to justice. In 2017, the National Planning Department,
as the technical state agency in charge of budget allocation, led the design of a composite indicator to measure
the state of access to justice in the country. The survey measures progress towards access to justice associated
with the implementation of the Ten Year Justice Plan 2017–2027. The initial results highlighted the importance
of preventing the denial of access to healthcare services, which was the most declared legal need by Colombians,
and generated new policy responses.

281. Central to thinking about people-centered justice, the first process indicator listed in Table VI.2
focuses on the experience and needs of those who face legal problems. It is of particular importance
in the context of legal capability, legal aid and legal empowerment. It is similar in form to a govern-
ment indicator used in England and Wales in the early 2000s, which drew on legal needs surveys (Her
Majesty’s Treasury, 2000). The second process indicator also concerns people’s needs but focuses on
experiences within institutions and draws on administrative data (or user surveys in the absence of
such data). The third process indicator is focused on the quality of services and processes. In relation
to service quality, documentation on quality assurance indicators for legal aid can provide a useful
resource (see, for example, Sherr, Moorhead and Paterson, 1994, and Sherr and Paterson, 2008). In
relation to process quality, the documentation for HiiL’s Measuring Access to Justice in a Globalising
World project and Justice Needs and Satisfaction Surveys provide valuable theoretical and practical
guidance (see, for example, Barendrecht and others, 2010, and Klaming and Giesen, 2008). The fourth
process indicator is the SDG 16.3.1 indicator of access to criminal justice by victims of crime, which
looks beyond justice services and processes. The fifth process indicator is the similarly conceived pro-
posed SDG 16.3.3 indicator of access to civil justice dispute resolution mechanisms. The sixth process
indicator concerns the infrastructure for monitoring access to justice, rather than access to justice
itself. Such an indicator could identify specific elements required of a monitoring and evaluation sys-
tem. The final four process indicators represent relatively simple and inexpensive indicators that could
draw on administrative data. The seventh process indicator concerns process duration, an important
dimension of access to justice, as recognized in the adage “justice delayed is justice denied”. The
eighth process indicator is the SDG 16.3.2 indicator and is included in UN-CTS. It focuses on pretrial
detention, about which comprehensive administrative data should generally be available. The ninth
indicator is focused on the length of duration of pretrial detention. The tenth process indicator is
focused on the extent to which pretrial detainees are convicted and receive custodial sentences. It is
a reliable indicator of the excessive and arbitrary use of pretrial detention and can lead to inquiries to
identify weaknesses in the criminal justice system (Open Society Justice Initiative, 2015).

282. The first outcome indicator listed in Table VI.2 focuses on whether people facing civil legal
problems obtain legal information or assistance and are ultimately able to resolve their problems.
The indicator is people-centred and so extends to justice issues occurring both inside and outside of
formal institutions. It is feasible to implement, as it has only a small number of survey questions that
have been tested using a comparable methodology in over 100 countries. Further guidance on the
use of such survey-based indicators can be found in the OECD-OSJI’s Legal Needs Survey and Access
to Justice. The third outcome indicator is a simple measure of whether people obtain outcomes to
legal problems that they regard as satisfactory. This represents the simplest form of people-centred

120 |
outcome indicator. The fourth outcome indicator focuses on suicides in prisons, which can be dis-
aggregated by prisoner status. The fifth outcome indicator provides valuable information on prison
conditions, for example, the rehabilitation of offenders and human rights safeguards, which are at the
core of access to justice. The final two outcome indicators are focused on conviction rates and are
important for assessing the effectiveness of the justice system and the functioning of its components.
They represent simple indicators that should be possible to populate using administrative data and
are included within UN-CTS. The first of those – the homicides indicator – is reliable, as “in many
countries, most intentional homicides are brought to the attention of the police” (UN Commission on
Crime Prevention and Criminal Justice, 2017).

283. Finally, it is worth noting that baskets of indicators (composite indices), which bring togeth-
er single indicators to provide a combination of supplementary and complementary indicators,
can offer a more holistic and balanced assessment of progress.33 An example is the Colombian
Index of Effective Access to Justice, which was conducted under agreement between the National
Administrative Department of Statistics and the National Planning Department, and incorporates 24
indicators drawn from both administrative and survey data. However, given the nature and primary
objectives of this Handbook, it generally refrains from recommending composite metrics.

VI.5 Way forward

284. Recent years have seen significant steps towards the measurement of a more holistic perspec-
tive of justice, with people and communities being at the core of the system. A key principle going
forward will be to build on these achievements and to further strengthen approaches focused on
users/people, including a consistent focus on outcomes — on whether legal needs are met and the
process is seen as fair and efficient. Some key steps in achieving further progress will be:

•  romote implementation of the ICCS: The adoption of this standard across all criminal justice
P
institutions will dramatically improve the availability, quality and comparability of data stem-
ming from the criminal justice system.
• Development of new statistical standards: There is a growing demand for standardized defi-
nitions, classifications and measurement tools — building on, for example, the ICCS and
OECD-OSJI legal needs survey guidance — for use in the field of access to justice more
broadly, and particularly within civil justice.
• Expand person-focused data: Promote the implementation of surveys, including on victimiza-
tion, access to justice and legal needs so as to increase the availability and quality of data on
persons directly or indirectly affected by justice problems.
• Better disaggregation of data: Better understand population and user-specific needs, by
demographic pattern and type of need. This can then be compared to the differentiation of
service providers by the range and extent of services. Strategies on how to close the gaps
need to be informed by data linking users to the web of service providers in the public,
community, civil society and private sectors to better appreciate unmet legal needs.
• Increased collaboration in data production: NSOs should seek ways to complement official
statistics with data generated by civil society, academia, the private sector and global survey

33 The Virtual Network for the Development of Indicators on Peaceful, Just and Inclusive Societies for Goal 16 defined complementary indicators as
“those that are necessary to complete measurement of a complex concept”, and supplementary indicators as those that “round out” measurement, to
allow countries to “adapt the universal goals to their own contexts and identify other dimensions of the target that are important to them” (2015).

HANDBOOK ON GOVERNANCE STATISTICS | 121

 programmes. Many countries do not have the resources to measure some of the SDG 16
indicators related to access to justice.
• Raising awareness and strengthening demand for data: Investment by governments in access
to justice measurement will occur only if there is robust demand and if data are used in the
design of policies, strategies and interventions around this topic. Raising awareness can lead
to sustainability and a lasting commitment from stakeholders.
• Further debate around data openness: There is a need to continue to balance data own-
ership rights, the right to confidentiality as guaranteed in many Statistics Acts, and issues
around open access to data.
• Capacity building: Training materials and targeted interventions can help countries to im-
plement international best practices and guidelines on access to justice measurement
frameworks. Targeted regional workshops also contribute to sharing experiences and demon-
strating the value of standardizing definitions, standards and measurement tools.
• Promote digitization: Increased digitization across administrative data sets increases the
possibilities to link various data sets and the ability to mine aspects pertinent to improving
services related to access to justice.

122 |
HANDBOOK ON GOVERNANCE STATISTICS | 123
124 |
 PART B
EIGHT DIMENSIONS OF
GOVERNANCE STATISTICS

CHAPTER VII
Responsiveness
 VII.1 Conceptualizing this dimension
285. Responsiveness, in general, has been defined as “how quickly and well a person or organiza-
tion reacts to something”1. One insight from this general definition is that responsiveness implies a
challenge or stimuli that triggers some type of reaction. The term has been used in several contexts
as, for example, mothers’ response to the frequency and duration of an infant crying (Bell and Salter
Ainsworth, 1972), the reaction of the human immune system to a specific substance (Piani and others,
2000) or the motivations that will make companies adopt an environmentally friendly approach (Bansal
and Roth, 2017).

286. Political science and public management have analysed the responsiveness of public institu-
tions. Yet these institutions could be different in nature; some authors focus on the political sphere
and emphasize the role of political competition and to what extent a high degree of contestation
influences policy responsiveness2 (Hobolt and Klemmensen, 2008); others have emphasized the insti-
tutional aspects of responsiveness as the pursuit of collective interests while respecting formal (rule of
law) and informal norms, by providing services that are aligned and could evolve according to citizens
preferences (Ivanyna and Shah, 2018). Common to most uses in fields related to public governance is
that institutional responsiveness is driven by people’s preferences and that public institutions react to
those preferences. In turn, when referring to responsiveness, the OECD/Korea Development Institute
(2018)3 report emphasizes the role of expectations, but also signal their evolving nature and the role
of institutions in coping with them. For the purpose of this Handbook, responsiveness is defined as
the degree to which public institutions listen to what people want and act on that, i.e. whether public
policies and institutions respond to the needs of citizens and uphold their rights. For the purpose
of this chapter public institutions are understood as administrative agencies of the central or local
government that develop and execute public policy involving education and public health, among
others.4

287. The stimuli to which public institutions are subject are of a diverse nature and hard to encom-
pass in a detailed taxonomy. This chapter will look at the general concept of system responsiveness
and its measurement as well as one specific aspect or application, in the field of satisfaction with se-
lected services (i.e. health and education). However, it recognizes that public institutions are expected
to respond when other circumstances arise, for example the occurrence of a natural disaster. While
still belonging to the overall responsiveness dimension, these stimuli will require different responses
and may call for different measurement approaches, leaving the field open to the further develop-
ment of this dimension.

288. System responsiveness or external political efficacy5 refers to people’s feeling of having a
say in what the government does. This is core to the legitimacy of public institutions that should
be acting on behalf of and for people. As such, people expect that their views and needs will affect
1 Cambridge Dictionary.
2 The authors analyse two aspects of responsiveness: rhetorical responsiveness or the extent to which a government’s selective policy emphases in
speeches reflect public issue preferences, and effective responsiveness, that is, the correspondence between public issues preferences and budgetary
priorities.
3 The OECD has indentified responsiveness as one of the drivers of trust in government institutions and has defined it as the provision of accessible,
efficient and citizen-oriented public services that effectively address the needs and expectations of people, and evolve over time along with these needs.
4 The term government in a broader sense includes all the public entities, such as the central government, local governments, public enterprises, and
other public institutions.
5 Research on political efficacy often distinguishes between internal efficacy (i.e. having personal competence to participate in politics) and external
efficacy or system responsiveness (i.e. a belief in the responsiveness of public institutions and government officials to citizens’ demands). Only the second
concept will be treated in the chapter as it addresses perceptions of government responsiveness.

126 |
the decisions taken by public institutions. In turn, system responsiveness can be built and destroyed
by people’s experiences when interacting with public institutions and also by institutions that are
not perceived as being responsive to people’s needs (e.g. policy-making processes and government
decisions that do not respond to public preferences). Participation and system responsiveness are
intertwined, as both informal and formal channels of participation are crucial in shaping government
responses at different levels (i.e. political, institutional); however, this chapter will focus exclusively on
people’s perception of those responses rather than on mechanisms of participation, which are treated
in greater detail in the chapter on “Openness” in this Handbook.

289. The second aspect of people’s expectations regarding public institutions analysed in this
chapter refers to the performance of these institutions, or more precisely a narrow category of this:
satisfaction with key services, reflecting among others the ability of institutions to respond to peo-
ple’s expectations. In many instances the performance of public institutions has been associated with
the quality of public services, traditionally measured through user satisfaction (Van Ryzin, 2004; Roch
and Poister, 2006). Public institutions are expected to perform according to citizens “expectations”,
with citizens expecting high returns for their tax payments, mainly in the form of service provision
(Van de Walle, 2018). Finally, access to services on the basis of equality, as the basic precondition to
assess performance, is grounded in international legal instruments: Article 25 (c) of the International
Covenant on Civil and Political Rights6 recognizes people’s right “to have access, on general terms of
equality, to public service in his country”.

290. The scope of government action in different areas of service provision varies widely, and the
extent to which government should be the main provider of services is a source of debate (Feldstein,
1996). Public services range from obtaining an identity record, a driver’s licence or a marriage certif-
icate to kids attending school or patients going to hospital. This chapter focus specifically on health
and education services, as these represent a substantial share of government activity and spending
(OECD, forthcoming) and have the longest tradition of measurement. However, and although at a
lower level of maturity, it also acknowledges the relevance of general services and existing measure-
ment efforts in the field. Overall, satisfaction with government services is an outcome of government
activity that captures elements that are essential to people’s lives (OECD, 2017a; OECD, 2017b). This
satisfaction is shaped by people’s expectations as well as by experience with these services and in-
formation about them from other sources (e.g. media, internet, acquaintances, etc.) (Jacobsen and
others, 2015; James 2009).7

291. To sum up, this chapter covers two main aspects: the general concept of system responsive-
ness and one specific aspect of responsiveness as captured by satisfaction with services, particularly
health and education.

292. It should be stressed that all indicators discussed in this chapter are considered as outcomes
under this Handbook’s cross-cutting framework. While reaching those outcomes is often depend-
ent on the existence of structures (e.g. legal instruments guaranteeing participation opportunities)

6 UN Treaty Series, vol. 999, p. 171 and vol. 1057, p. 407.

7 An alternative approach consists of analysing the operational qualities – the actual performance – of public services including data from administrative
records, and, if possible, explore their direct link to satisfaction. For example, the OECD’s Serving Citizens framework evaluates three key public services
– health, education and justice – along three fundamental dimensions – access, responsiveness, quality –, and nine sub dimensions. This approach
recognizes that satisfaction with services is influenced by a variety of factors considering as unit of observation individuals with direct exposure to the
services (e.g., patients, students,). However, sub dimensions for different services cannot be measured by the same metrics (e.g., timeliness cannot be
measured by the same way in health or education, but can be to health and justice); comparing performance across services on the same dimensions is
very challenging, if not impossible.

HANDBOOK ON GOVERNANCE STATISTICS | 127

 or processes (e.g. availability of information or the existence of feedback mechanisms), establishing
these links is beyond the remit of this chapter.

VII.2 Why is this dimension important?

293. System responsiveness, or the belief that people can influence what governments do, matters,
as it captures the perception of people’s capacity to influence policy formulation and policy making,
often through participation and engagement opportunities in traditional or non-traditional channels
(e.g. social media, online platforms). This is of paramount importance to democratic systems, as it
relates to the belief that political and social change are possible and that people can play a part
in bringing about this change. This belief is also associated with the idea that it is worthwhile to
perform civic duties (OECD, 2017c). In the academic literature, levels of system responsiveness are
related to citizen engagement, satisfaction with democracy and trust in public institutions (González,
2020). Persistently low or diminishing levels of system responsiveness will raise doubts in people’s
minds about whether governments are working for the interests of the majority and not just a few. In
turn, resulting disenchantment could erode the foundations of democratic systems and nurture the
emergence of populist responses.

294. Benefitting from high-quality public services, such as education and health, is essential for
people to develop capabilities, obtain skills and make economic and social progress. All public institu-
tions owe their legitimacy to the presumption that, notwithstanding the existence of different delivery
channels (e.g. direct provision or regulation of private providers), they act to secure the constitutional
entitlements of those being governed by them (Nussbaum, 2011). While research has shown that
reported satisfaction is a combination of experience and expectations (Oliver, 2010), comparative and
consistent evidence is still missing on how these expectations and experiences are shaped (i.e. their
constitutive elements). Developing that evidence alongside satisfaction metrics will allow reflecting on
those links, which in turn are key for formulating and implementing policies that could improve these
services and through this channel people’s well-being.

295. Satisfaction with services also matters, as it shapes political attitudes and behaviours towards
policies (Young Mok and others, 2017). People who are dissatisfied with public services are likely to
demonstrate their dissatisfaction by raising their voice, filing a complaint, engaging in protest or opt-
ing out if private providers are available (Van de Walle, 2018). Improving the quality of public services
can lead to more satisfied users, which, in turn, can increase trust in government institutions (see the
chapter on “Trust” in this Handbook), a mechanism referred in the literature as the “micro-perfor-
mance hypothesis” (Van de Walle and Bouckaert, 2003; Yang and Holzer, 2006).

VII.3 Data and best practices currently available

VII.3.1 System responsiveness
296. Statistics in the field of system responsiveness are increasingly being collected, both by nation-
al statistics institutes and as part of non-official household surveys. In particular, the so-called NOSAY
and INFLUENCE IN POLITICS questions stand out as those capturing the concept well. Furthermore,
they have been incorporated as part of the indicators suggested for measuring SDG 16. Indicator
16.7.2 refers to the proportion of the population who believe that decision-making is inclusive and
responsive.

128 |
BOX VII.1 Perceptions of system responsiveness in the USA

The figure below shows that the percentage of the American population who feel they have a say in what
the government does has declined by 25 percentage points from 68.5% in 1952 to 43% in 2016. Originally
the question was asked using a dichotomous scale (“agree” and “disagree”) but since 1988 has included a
third choice, “neither agree nor disagree”. The high levels of volatility of this measure suggest that this metric
could be influenced by a range of factors, such as the political cycle or the type of policies implemented by
governments, thus calling for cautious interpretation of the data. However, a long-term diminishing trend is
consistent with the erosion of institutional trust and social capital documented for the US over similar long-
term periods (OECD, 2017a; Putnam, 2005).

FIGURE VII.1 P
 ercentage of the voting age population who feel they
have a say in what the government does, 1952-2016

85

75

65

55

45

35

25
1950 1952 1954 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016 2018

Note: The figure depicts the percentage of the population who disagree with the following statement: people like me don't have any say about what
the government does. Before 1988, the question had only three answer choices (agree, disagree and don't know). Since then the answer choice “neither
agree nor disagree” was added.
Source: OECD calculations based on the American National Electoral Study combined dataset.

297. There is a long tradition of collecting political efficacy measures through household surveys,
which started in 1952 with the inclusion of a battery of questions8 in the American National Election
Studies9 (see Box VII.1). Originally, political efficacy was considered a unidimensional construct; how-
ever, researchers realized early on the existence of two distinct dimensions: internal efficacy (beliefs
about one’s own competence to understand and participate effectively in politics) and external efficacy

8 Before 1982, the Center For Political Studies (CPS) through its American National Election Studies (ANES) measured political efficacy, using an agreed/
disagreed scale, by asking survey respondents if they “agreed” or “disagreed” with several statements: 1) People like me don’t have a say about what
the government does, 2) Voting is the only way that people like me can have any say about how the government runs things, 3) Sometimes politics and
government seem so complicated that a person like me can’t really understand what’s going on, 4) I don’t think public officials care much about what
people like me think, 5) Generally speaking, those we elect to Congress in Washington lose touch with the people pretty quickly, and 6) Parties are only
interested in people’s votes but not in their opinions.
9 The American National Election Studies (ANES) are academically-run national surveys of voters in the United States, conducted before and after every
presidential election. The ANES was formally established by a National Science Foundation grant in 1977; however, the data are a continuation of studies
going back to 1948. The study has been based at the University of Michigan since its origin and, since 2005, has been run in partnership with Stanford
University.

HANDBOOK ON GOVERNANCE STATISTICS | 129

 BOX VII.2 The NOSAY question in the PIAAC and ESS surveys

According to the latest available PIAAC data from 2012, one third of people in OECD countries believe having
some influence on what government does, with the share ranging between 20% or less in Italy, Slovenia and
France to 60% or more in Chile, Greece and Lithuania.

FIGURE VII.2 P
 ercentage of the voting age population who feel they
have a say in what the government does, 1952-2016

80
70
60
50
40
30
20
10
0
FRA SVN ITA CZE TUR SVK ESP DEU JPN EST IRL ISR AUT GBR OECD AUS BEL CAN POL KOR NLD NZL USA SWE FIN NOR DNK CHL GRC RUS LTU
28
Source: OECD calculations based on data from the OECD Survey of Adult Skills (PIAAC database), http://www.oecd.org/site/piaac/

The ESS results show that about 37% of the population in European countries consider that the political system
allows people to have a say in what the government does.

FIGURE VII.3 P
 ercentage of the population over age 15 who feel that the political
system allows them to have a say in what the government does

80
 2016
70  2014

60
50
40
30
20
10
0
ITA SVN EST RUS LTU ESP POL ISR FRA IRL HUN PRT AUT FIN EUR BEL CZE GBR SWE ISL DEU NLD NOR CHE DNK

Source: OECD calculations based on data from the OECD Survey of Adult Skills (PIAAC database), http://www.oecd.org/site/piaac/

130 |
(generally referred to as system responsiveness in this chapter) and tried to fit existing questions to
those dimensions. Yet evidence on the reliability and validity of measures under this model was unsat-
isfactory (Morrell, 2003).

298. From the onset of efforts to measure political efficacy, the so-called NOSAY question (common-
ly formulated as “To what extent do you agree or disagree with the following statements? People like
me don’t have any say in what the government does”) has been included in surveys. An early wave of
research associated the NOSAY question to internal political efficacy (Miller and others, 1980; Craig and
Maggiotto, 1982; Acock and Clarke, 1990). However, additional research showed that the NOSAY ques-
tion is actually a measure of external political efficacy, i.e. system responsiveness (Niemi and Mattei, 1991),
a result confirmed empirically by subsequent waves of the survey and field experiments (Morrell 2003) as
well as more recent research (Borgonovi and Pokropek, 2017; OECD, 2017a). Table ANX.VII.1 in Annex VII
shows the questions on system responsiveness included in the different surveys analysed in this chapter.

299. The NOSAY question has also been included in the 2012 OECD Survey of Adult Skills (PIAAC).
Since 2014, the European Social Survey (ESS) also collects responses on system responsiveness
through the following two questions: 1) how much would you say the political system in [country]
allows people like you to have a say in what the government does? and 2) how much would you say
that the political system in [country] allows people like you to have an influence in politics. While the
wording of the PIAAC questionnaire is identical to the ANES, PIAAC uses a 5-point Likert-type scale
ranging from 1 (“strongly agree”) to 5 (“strongly disagree”). Differently from ANES and PIAAC, ESS
formulates the NOSAY and INFLUENCE IN POLITICS questions in a positive way (have a say instead of
don’t have a say) and asks about the political system instead of government, finally the items included
are formulated as questions rather than agreement or disagreement with statements. According to
researchers, the use of questions instead of statements leads to higher quality10 of the data (Saris and
Torcal, 2009).

300. Some countries have also included questions on system responsiveness in their official sur-
veys. For example, the Mexican Ministry of Interior has conducted five rounds of a national survey
on Political Culture and Citizenship (ENCUP), three of them in partnership with the Mexican National
Statistical Office (INEGI). While the questions have varied, the topic of system responsiveness has
been included in all waves of the survey. The most recent survey, fielded in 2012, included the follow-
ing question: “Do you agree or disagree with this statement? People like me have influence on what
the government does”,11 with a three-point scale as response choices (see Table ANX.VII.1 in Annex
VII). Previous versions of the survey in 2003, 2005 and 2008 included the NOSAY question12.

10 According to these researchers the use of questions as opposed to statements improves the reliability of the tested model and provides a better
representation of the system responsiveness concept.
11 The original question in Spanish is formulated in the following way: Para cada una de estas frases, por favor dígame, ¿está usted de acuerdo o en
desacuerdo? La gente como yo tiene influencia sobre lo que hace el gobierno. a) muy de acuerdo b) algo de acuerdo c) ni de acuerdo ni desacuerdo d)
algo en desacuerdo e) muy en desacuerdo.
12 The original question in Spanish is formulated in the following way: Dígame si usted está de acuerdo o no con cada una de las siguientes frases: Las
personas como usted no tienen nada que opinar sobre lo que hace el gobierno. Sí esta de acuerdo, ni de acuerdo ni en desacuerdo, esta en desacuerdo.

HANDBOOK ON GOVERNANCE STATISTICS | 131

TABLE VII.1 Summary evaluation of the accuracy of survey measures of system responsiveness

Face validity Construct validity Convergent validity Reliability

NOSAY question √√ √√ √√ √
Note: √√ mean strong validity or reliability while √ means weak validity or reliability
Source: González (2020).

VII.3.1.1 Accuracy of system responsiveness measures

301. The accuracy of a metric is the degree to which it captures the concept that it is intended to
measure. Typically accuracy is thought of as having two dimensions: reliability and validity. The relia-
bility of a metric is the degree to which repeated measurement of the same thing produces consistent
results. Validity, on the other hand, is concerned with whether the measure in question is biased.
Table VII.1 above gives a snapshot of the different criteria for statistical quality considered in this
chapter, indicating that the accuracy (i.e. reliability and validity) of measures of system responsiveness
is generally robust, and a strong case exists to promote the collection of these statistics. For a detailed
description of the procedure followed to reach this conclusion see González (2020) and UNDP (2019a).

BOX VII.3 Q
 uestions on system responsiveness recommended
as indicators of SDG indicator 16.7.2

SDG indicator 16.7.2 “Proportion of population who believe decision-making is inclusive and responsive” aims
to measure both the inclusiveness and the responsiveness of decision-making. As such, the methodology
consists of two separate survey questions addressing these two distinct dimensions, namely:

To measure the inclusiveness of decision-making:

How much would you say the political system in [country X] allows people like you to have a say in what the
government does?

• Not at all (1)

• Very little (2)
• Some (3)
• A lot (4)
• A great deal (5)
• Refusal
• Don’t know
• No answer

To measure the responsiveness of decision-making:

And how much would you say that the political system in [country] allows people like you to have an influence
on politics?

• Not at all (1)

• Very little (2)
• Some (3)
• A lot (4)
• A great deal (5)
• Refusal
• Don’t know
• No answer

To derive a single combined estimate for the SDG indicator, data producers can simply calculate the cumulative
rating on each question (on a scale of 1 to 5) and then calculate the average rating for the two questions.

132 |
VII.3.1.2 Best practice for specifying survey questions on system responsiveness

302. Building on the long tradition of including system responsiveness metrics in political partici-
pation surveys outlined above, and the ample research that has gone into validating their statistical
quality (González, 2020; UNDP, 2019a; ESS, 2016; Saris and Revillas, 2012; Saris and Torcal, 2009), two
survey questions on system responsiveness have been adopted as part of the SDG indicator frame-
work (SDG indicator 16.7.2, “Proportion of the population who believe decision-making is inclusive
and responsive”). Data producers wishing to gather data on system responsiveness are therefore
recommended to use the questions detailed above13 (see Box VII.3).

VII.3.2 Satisfaction with services

303. Satisfaction with services is recognized as a key measure of government performance and
is widely measured by many NSOs and in non-official household surveys. While general satisfaction
questions could provide meaningful information, attributes-specific questions are more informative
and meaningful for policy makers. The specificity of the information generated by such questions, as
well as the focus on citizen experiences rather than simply perceptions, have greater policy use than
stand-alone perception data on overall satisfaction, which may not reveal “what needs to be fixed”.

304. Public management researchers have long experience with specifically focused surveys to
measure satisfaction with services conducted mainly at the local level and with varying degrees of
representativeness (Bouckaert and Van de Walle, 2003).

305. Questions on satisfaction with services are included in various national data collection efforts
(e.g. Mexico, Peru, Brazil, Germany, Norway and South Africa). A non-comprehensive review found
that 11 NSOs14 include questions about satisfaction with services, albeit with different scopes and fre-
quencies (UNDP, 2017). Satisfaction with services is also captured in several cross-country comparative
surveys (Gallup World Poll, European Quality of Life Surveys, European Social Survey, European Quality
of Government Index, Latinobarometer — see Table ANX.VII.2 in Annex VII for more). However, both
at the national and the cross-country level, no consensus exists on which services to cover.15 Health
and education services, those most commonly covered, are the focus of this chapter. Another chapter
of this Handbook focuses specifically on access to justice.

306. In terms of how the issue is being measured, some surveys ask specifically about satisfaction
in the respondent’s city or area (e.g. Gallup World Poll), while others ask about the state of the health
and education systems in the respondent’s country (e.g. ESS). Some surveys (e.g. EQLS and QoG) ask
generally to rate the quality of health and education, while Latinbarometer restricts the questions on

13 The proposed set of questions were piloted by seven NSOs across various regions and development contexts (Cabo Verde, Ghana, Kenya, Republic of
Korea, Mexico, Palestine and Uganda. The World Values Survey Association (WVSA) also pilot-tested one of the two questions (i.e. How much would you
say the political system in [country] allows people like you to have a say in what the government does?) as part of its 7th survey wave (2018-19), with results
published for 15 countries (Andorra, Argentina, Australia, Bangladesh, Brazil, Egypt, Indonesia, Iraq, Jordan, Kazakhstan, Lebanon, Malaysia, Nigeria,
Pakistan, Peru). Positive feedback was received from the WVSA as to the relevance and viability of the recommended questions, which have now been
integrated into the core WVS questionnaire for future survey rounds. Piloting by CIVICUS at community level in 22 countries (in Africa: Ethiopia, Liberia,
Togo, Uganda, Gambia, Cameroon, Nigeria, South Sudan, Zambia; in Europe: Albania, Spain; in Latin Amerca: Argentina, Bolivia, Colombia, Mexico, Peru,
Uruguay; in the MENA region: Syria, Iraq, Morocco, Turkey, Lebanon) also confirmed the high interest of local communities for the proposed questions and
their willingness to answer them. Statistical analysis on the accuracy of these questions was also conducted by UNDP and piloting entities (UNDP, 2019a).
The questions agreed for re-classification are presented below.
14 The countries are Cameroon, Germany, Kenya, Mexico, Viet Nam, the Philippines, South Africa, Latvia, Norway, Tunisia and New Zealand.
15 Different surveys include different services. For example, in addition to health and education the Gallup World Poll also asks about satisfaction with
public transportation, air quality and water quality. In addition to health and education the European Quality of Life Survey also collects data on long-
term care (e.g. nursing, etc). In several waves the Latinobarometer has included questions about satisfaction with administrative services (e.g. obtaining
passports, licences, permits, etc).

HANDBOOK ON GOVERNANCE STATISTICS | 133

 BOX VII.4 Satisfaction with education and health services

Based on Gallup World Poll data, the figure shows the percentage of the population in OECD member and
accession countries who are satisfied with the availability of health care and the education system or schools
at the local level using a binary choice (i.e. satisfied/dissatisfied). According to the latest available data, 68% of
the population in OECD countries are satisfied with the education system, while almost 70% are satisfied with
the availability of health care.

FIGURE VII.4 P
 ercentage of the population who are satisfied with health
and education in OECD member countries, 2017

100
 Education
 Health
80

60

40

20

0
GRC CHL TUR COL LTU LVA KOR ESP ITA HUN MEX SVK EST ISR USA AUS DEU SWE OECD AUT FRA PRT GBR CZE NZL LUX CAN POL DNK BEL ISL GRI NLD SVN CHE IRL FIN NOR

Source: OECD calculations based on the Gallup World Poll.

Since 1995, Latinbarometer has collected annual data for eighteen LAC countries on citizen satisfaction with several
services and since 2016 it has linked the question to having access to these services. In addition, the questionnaire
also includes questions about specific experience with public hospitals and schools. According to the latest available
data, 56% of the population in LAC countries are satisfied with the health system they have access to and slightly less
than half with the education system.

FIGURE VII.5 P
 ercentage of the population who are satisfied with the
education and health they have access to, 2016
90
 Education
80  Health
70
60
50
40
30
20
10
0
BRA VEN CHL PRY PER MEX GTM BOL HND COL LAC PAN SLV ARG URY ECU NIC DOM
(18)

Source: OECD calculations based on Latinobarometer.

134 |
health and education to the services respondents have access to (see Annex VII, Table ANX.VII.2 for
the exact questions included in these cross-country comparative surveys).

307. Within the SDG framework, indicator 16.6.2 refers to the proportion of the population satisfied
with their last experience of public services, specifically a) health-care services, b) education services
and c) government services (i.e. services to obtain government-issued identification documents and
services for the civil registration of life events such as births, marriages and deaths). A survey module
assessing education and health care (the two policy fields analysed in this chapter) as well as “gov-
ernment services” based on four general criteria (accessibility, affordability, quality of facilities, equal
treatment for everyone; and a specific criterion for each service (courtesy and treatment for health
care and effective delivery for education) has been put forward as a tool for measuring satisfaction
with these services.

VII.3.2.1 Accuracy of measures of satisfaction with services

308. The analysis of the statistical accuracy of existing satisfaction with services questions leads
to mixed results (González, 2020). Questions about satisfaction with health care have higher statis-
tical quality than questions on satisfaction with education. While the former perform well in terms
of reliability, construct and face validity, the evidence for testing convergent validity (e.g. correlation
with proxy measures of the same construct) is not available. The lower performance of questions on
satisfaction with the education sector could be explained by respondents without direct exposure or
reference answering these questions. Table VII.2 shows the summary of statistical quality criteria for
questions on satisfaction with health and education services.

TABLE VII.2 Summary evaluation of the accuracy of survey measures of satisfaction with services

Face validity Construct validity Convergent validity Reliability

Satisfaction with services-health √√ √√ √ √√

Satisfaction with services-education √ √ √ √

Note: √√ mean strong validity or reliability while √ means weak validity or reliability
Source: González (2020).

VII.3.2.2 Best practice for specifying survey questions on satisfaction with services

309. Building on the existing experiences of capturing satisfaction with public services in surveys,
and the recent body of research that allows for some preliminary conclusions about their quality
(González, 2020; UNDP, 2019a), two sets of survey questions on satisfaction with services have been
put forward for measuring SDG indicator 16.6.2 “Proportion of the population satisfied with their
last experience of public services, specifically a) health-care services, b) education services and c)
government services (i.e. services to obtain government-issued identification documents and services
for the civil registration of life events such as births, marriages and deaths)”. Data producers wishing
to gather data on satisfaction with services are therefore recommended to use the questions detailed
below (see Boxes VII.5 and VII.6).

310. As previously discussed, properly capturing satisfaction with health and education requires
taking into account a series of considerations on direct exposure to those services as well as the

HANDBOOK ON GOVERNANCE STATISTICS | 135

BOX VII.5 Questions on satisfaction with health care for the measurement of SDG indicator 16.6.2

1. Was there any time during the past 12 months when you (or a child in your household) really needed a medical examination or treatment?
A. Y es (There was at least one occasion in the past 12 months when I [or a child in my household] really needed medical examination or treatment)
[go to 2]
B. No (There was no occasion in the past 12 months when I [or a child in my household] really needed medical examination or treatment)
[End here. Go to next service area]
99. R efuse to answer

2. Did you [or a child in your household] have a medical examination or treatment each time you [or a child in your household] really needed it?
A. Y es (I [or a child in my household] had a medical examination or treatment each time I [or a child in my household] needed it) [go to 4]
B. No (there was at least one occasion when I [or a child in my household] did not have a medical examination or treatment when I [or a child in
my household] needed it) [go to 3]
99. R efuse to answer

3. What was the main reason for not having the medical examination or treatment?
A. C ould not afford to (too expensive)
B. Long waiting list (to get an appointment, or when turning up to a health facility without an appointment)
C. Too far to travel or no means of transportation to get there
D. D idn’t know any good medical doctor or health professional
E. Could not take time because of work, care for children or for other reasons
F. Wanted to wait and see if problem got better on its own
G. Fear of medical doctors, hospitals, examination or treatment
H. H ealth-care facilities are not clean
I. Health-care facilities are not adequately equipped or lack medicine
J. O ther reasons: _______________

4. I now want to ask you some questions about the last time you [or a child in your household] had a medical examination or treatment, in the past
12 months. Thinking about this last experience, would you say that:

Strongly Agree Disagree Strongly NA DK RA

agree disagree

4.1 It was easy to get to the place where I received medical treatment. 3 2 1 0 97 98 99

4.2 Expenses for health-care services were affordable to you/your household. 3 2 1 0 97 98 99

4.3 The health-care facilities were clean and in good condition. 3 2 1 0 97 98 99

4.4 All people are treated equally in receiving 3 2 1 0 97 98 99

health-care services in your area.

4.5 The doctor or other health-care staff you saw spent enough time 3 2 1 0 97 98 99
with you [or a child in your household] during the consultation.

5. Overall, how satisfied or dissatisfied were you with the quality of primary health-care services you [or a child in your household] received on that
last consultation? (i.e. the last time you [or a child in your household] had a medical examination or treatment in the past 12 months)

Very satisfied Satisfied Dissatisfied Very dissatisfied NA DK RA

3 2 1 0 97 98 99

136 |
specific attributes pertaining to each of them. In the case of health services, the module is struc-
tured to start with a question on direct exposure to health examination or treatment over the past 12
months. Throughout the questionnaire, respondents are asked to answer for themselves as well as for
children in their care who may have needed a medical examination or treatment. It continues with two
questions on the existence and nature of access barriers. Question 4, composed of five sub-questions,
asks respondents to rate the specific attributes of services to which they had access in the past 12
months, including geographic and financial access, state of the facilities, courtesy of treatment and
time devoted to the patient. The questionnaire ends with a general satisfaction question, as previously
explained. It is worth mentioning that the proposed questions do not enquire about specific health
conditions or treatments and remain rather general in the approach to health goods and services.16

311. In the case of satisfaction with education services, where the statistical quality of existing indi-
cators is considered generally lower (González, 2020), the module put forward should be considered
a place-holder until further evidence is generated. The set of questions follows a similar structure as
the one for health care. In order to reflect on respondents having a recent experience, the question-
naire is administered to households where there are children between age 4 and 16, a range that can
be adapted to specific national contexts. The module further emphasizes that it enquires exclusively
about public schools. The second question asks whether children attend education, and in the case
of a negative answer the next question asks why this is the case. For subsequent questions, further
clarification is required, as respondents should clarify if they are responding for primary or secondary
school. If in the household there are children attending both levels they will be asked independently
for both. Question 9 enquires about the specific attributes of the service (e.g. geographic and financial
access, state of the facilities, fairness of treatment and perceived quality of treatment). The module
ends with the question on the overall quality of the education system.

VII.3.3 How can survey design mitigate methodological issues in system responsiveness and
satisfaction with services questions?
312. System responsiveness and satisfaction with services measures are inherently subjective, i.e.
only people themselves can report on them. Subjective measures are more sensitive to response bi-
ases and measurement error than more objective measures drawn from administrative sources (such
as educational attainment or life expectancy) or from surveys (based on self-reports of, say, health or
labour market status). However, these biases are not unique to system responsiveness and affect many
of the other self-reported measures that are regularly collected by NSOs. While it is important to be
aware of these biases, the existence of measurement error per se is not an argument against gather-
ing data on system responsiveness; what is important is to understand what are the most appropriate
strategies for question and survey design to mitigate these errors. No matter which approach to ques-
tion design is adopted by data collectors, standardization is critical to ensure meaningful comparison
over time, between population groups or countries. In order to expand the collection of these metrics
to other contexts it is recommended to observe the following methodological considerations.

16 In its Health at a Glance report the OECD regularly reports comparable measures of patient-related experiences (PREMs) such as the doctor spending
enough time with patients in consultation. Patient-related outcome measures (PROMs) are also being increasingly used by countries (OECD, 2015).

HANDBOOK ON GOVERNANCE STATISTICS | 137

BOX VII.6 Questions on satisfaction with education for the measurement of SDG indicator 16.6.2

The next few questions focus on your experience with the primary and secondary public school system. By this, we mean public schools that are
funded by the State.
6. Are there children in your household whose age falls between 4 and 16 years old?
A. Y es (There are children in my household whose age falls between 4 and 16 years old)
B. No (There are NO children in my household whose age falls between 4 and 16 years old) [End here. Go to next service area]
99. R efuse to answer
7. Does this child (do all of these children) attend a public school regularly?
A. Y es [go to 9] (All children in in my household whose age falls between 4 and 16 years old attend a public school regularly)
B. No [go to 8] (There is at least one child in my household whose age falls between 4 and 16 years old who does NOT attend a public school regularly)
98. Don’t know
99. R efuse to answer
8. What is the main reason for this child/some children in your household not to attend a public school regularly?
A. C hild/children in my household attend a private school [End here. Go to next service area]
B. Child/children in my household are home-schooled [End here. Go to next service area]
C. Cannot afford to (school-related expenses, including administrative fees, books, uniforms and transportation, are too expensive)
D. T he nearest school is too far away and/or transportation is not available
E. School facilities are in poor conditions
F. The school and its compound are not safe
G. Teachers and other school staff do not treat children with respect
H. T eachers are ineffective/not adequately trained
I. Teachers are often absent
J. C hild/children need to stay home to help with housework/farm work
K. No culturally or religiously appropriate educational programs available
L. S chool not equipped for children with special learning needs
M. Other reasons: _______________
9. Please tell me more about the primary and/or secondary public schools attended by this child/children in your household:
• If necessary, replace “primary” and “secondary” schools with terms more commonly used in the national context: In some contexts,
primary school may be referred to as “elementary school” and secondary school may be referred to as “high school”, “middle school”,
“junior high” and/or “senior high”.
• Ask respondents to respond separately for primary and secondary schools if children in their household attend school at different levels,
i.e. if some respondents have two or more children in their household attending different school levels, ask the below set of questions
twice: first in relation to primary schools, and second in relation to secondary schools.
Are you reporting on:
A. P rimary school in your area ___
B. Secondary school in your area ___

Strongly Agree Disagree Strongly NA DK RA

agree disagree

9.1 The school can be reached by public or private transportation, 3 2 1 0 97 98 99

or by walk, in less than 30 minutes and without difficulties.

9.2 School-related expenses (including administrative fees, books, 3 2 1 0 97 98 99

uniforms and transportation) are affordable to you/your household.

9.3 School facilities are in good condition. 3 2 1 0 97 98 99

9.4 All children are treated equally in the school attended 3 2 1 0 97 98 99
by the child/children in your household.

9.5 The quality of teaching is good. 3 2 1 0 97 98 99

10. Overall, how satisfied or dissatisfied are you with the quality of education services provided by the primary and/or secondary public schools
attended by this child/children in your household?
Are you reporting on:
A. P rimary school in your area ___
B. Secondary school in your area ___

Very satisfied Satisfied Dissatisfied Very dissatisfied NA DK RA

3 2 1 0 97 98 99

138 |
VII.3.3.1 Question wording

313. Evidence on question wording shows that good question wording improves the quality of results.

314. System responsiveness

•  sing a split sample experiment in the Political Support in Canada (PSC) study, researchers have
U
analysed the effects of having negatively worded questions (i.e. NO SAY) versus positive fram-
ing (SAY). Overall, results for both formulations are similar, and no evidence was found that neg-
atively framing the statements led to acquiescence bias, though some mild evidence indicates
that negative formulations were more stable over time in a highly contested election (Clark and
others, 2010). In turn, the ESS has decided to include a positive formulation of the question and
found evidence of high reliability and validity (Saris and Revilla, 2012; Saris and Torcal, 2009).
As no conclusive evidence is found that either formulation induces meaningful changes, it is
suggested that a positive formulation be used to adhere to generally good practice in survey
design of avoiding any confusion generated by double negatives (Sheatsley, 1983).
• The NOSAY/SAY question has been formulated using two alternative framings: some surveys
ask about influence in what government does, while others formulate the question in terms
of influence in the political system. No direct testing has been conducted on the potential
implications of the different wordings. Evidence on the reliability of system responsiveness
questions using both formulations interchangeably shows no substantial difference between
surveys (González, 2020). But in order to enhance the applicability of questions to different
contexts, the use of references to the political system is recommended (see definition below).
• As the concepts encompassed by system responsiveness questions are complex, survey re-
spondents may refer to the interviewer for additional guidance on the terminology. In turn,
interviewers should refer to the specific wording provided below if respondents do not under-
stand certain terms (UNDP, 2019a). In particular:
- The political system: A particular form of government. For example, democracy is a
political system in which citizens govern themselves through free and fair elections.
Other political systems include republics, monarchies, communist systems and dic-
tatorships or a combination of these (e.g. democratic/constitutional monarchy).
- Having a say in what government does: This means having a channel to express
one’s demands, opinions or preferences about what the government does, and
feeling listened to.
- Have an influence on politics: This means feeling that decision makers listen to and
act on one’s demands, opinions or preferences.

315. Satisfaction with services

• Different question formulations are used to ask about an overall assessment of key services.
While some surveys refer to satisfaction (e.g. Gallup), others enquire about the overall state of
health or education in a given country (e.g. ESS), while others ask about the quality of services
(e.g. EQLS, EQOG). Analysis conducted across surveys shows that these formulations are
closely related (see González, 2020). Recently, a formulation based on the quality of public
services, similar to that used by the EQLS, has been tested by several NSOs. This confirmed
the technical feasibility and pertinence of these questions, which respondents found relevant
and easy to understand (UNDP, 2019b). These questions are being proposed for measuring
target 16.7.2 of SDG 16.

HANDBOOK ON GOVERNANCE STATISTICS | 139

 •  eople interact with service providers at several moments and for several purposes, yet
P
service quality can evolve over time. In order to avoid telescoping effects (e.g. perception
of distant events as being more recent, or conversely, recent events as being more distant
than they are) and to minimize memory bias, satisfaction questions should refer to the last
experience of public services and include a reference time frame; “the past 12 months” is
commonly used and suggested. In turn, to improve the accuracy of the answers, it is advised
that only respondents who used services should report on perceived quality or satisfaction
(González, 2020; UNDP, 2019b).
• Question wording should be precise enough to be understood by respondents, without get-
ting into subtle nuances that might also pose problems for translatability across countries.
If the concepts that questions try to capture are closely related, respondents might have
difficulty differentiating between them. Therefore it is recommended to keep consistency in
terminology by, for example, avoiding references to the health services or the health system
in the same questionnaire.

VII.3.3.2 Response formats

316. Answers to survey questions are also affected by the response options available. Best practice
in this field includes the following:

317. System responsiveness

•  ystem responsiveness questions have been asked using different response scales. While
S
ANES uses a three-item scale (Agree/Disagree/Neither agree nor disagree), PIAAC opted
for a five-point Likert-type ranging from 1 (strongly disagree) to 5 (strongly agree). In turn, the
European Social Survey used a scale of 0 (not at all) to 10 (completely) in wave seven, fielded
in 2014, and shifted to a five-point Likert-type scale ranging from 1 (not at all) to 5 ( a great
deal) in wave eight (2016). The results across both waves using the different response scales
are consistent (González, 2019). Piloting by seven NSOs and an additional 18 jurisdictions by
the World Values Survey has confirmed the plausibility of this scale for the implementation
of political efficacy questions (UNDP, 2019a). Whatever format is used, consistency across
countries is essential to guarantee the comparability of political efficacy measures.
• The response order used for different questions should be presented consistently (i.e. 1-5
instead of 5-1) in order to minimize mental switching by respondents between positive and
negative normative outcomes.
• Verbal descriptions of the scale anchors should represent absolute responses either in the
extremes 0 (not at all) to 10 (completely) or throughout the scale if a five-point scale is chosen
(see Table ANX.VII.1, Annex VII) in order to minimize acquiescence bias and socially desirable
responding, and to allow for the full spectrum of possible responses.
• The response options “don’t know”, “refuse to answer” or “not applicable” are possible;
however, they should not be read out loud to respondents: Providing a “don’t know” or “re-
fuse to answer” option provides an easy way for respondents to avoid engaging with the
subject of the question. Even when respondents say they “don’t know”, enumerators should
repeat the question and simply ask respondents to provide their best guess before selecting
this option.

140 |
 318. Satisfaction with services

319. Several response scales have been used for questions on satisfaction with services, for exam-
ple, the Gallup World Poll uses a binary YES/NO scale. At the other end, the European Social Survey
uses a scale from 0 (very poor quality/extremely bad) to 10 (very high quality/extremely good) with
anchors at the extremes. Other surveys such as Latinobarometer use a four-point verbal response
scale. Using split samples, the overall satisfaction/dissatisfaction with services was tested by the Cape
Verde National Statistical Office. Half of the sample was given a 4-level scale (“very dissatisfied”, “dis-
satisfied”, “satisfied”, or “very satisfied”) and the other half a 10-level scale (from 1 to 10, where 1
is “very dissatisfied” and 10 is “very satisfied”). The results showed no evidence to suggest that the
4-point scale provides insufficient alternatives that would prompt respondents to give a non-response.
Furthermore, the non-descriptive thresholds on the 10-point scale (which has only upper and lower an-
chors) was deemed problematic where the interpretation of such thresholds differ among respondents
and sub-groups (despite having the same actual level of satisfaction). In terms of interpretability and
potential cross-country comparability, the 4-level scale was considered preferable by this experiment.
Whatever format is used, consistency across countries is recommended to guarantee the compa-
rability of measures (see the sub-section above on system responsiveness for recommendation on
consistency of the response order, verbal descriptions, and don’t know and alternative choices).

VII.3.3.3 Survey context

320. System responsiveness and satisfaction with services measures should be considered within
the broader context of the survey in which they are placed. To minimize the impact of holidays, sea-
sons and elections, data collection should be spread throughout the year or at least over multiple
weeks. Best practice in this field includes the following:

321. System responsiveness

•  enerally, system responsiveness questions should not be asked immediately after items that
G
are likely to elicit strong emotional responses or that refer to experiences with institutions.
• Questionnaire designers should reflect on the potential effect that system responsiveness
questions could have on subsequent items, in particular those dealing with similar issues.
• To the extent possible, system responsiveness measurement should be detached from the
elections cycle, particularly as new technologies are allowing mechanisms of communication
and engagement between people and public institutions to become permanent.

322. Satisfaction with services

•  esearch has documented the existence of the so-called “halo” effect, the influence that
R
general attitudes towards the public sector could have on people when assessing public ser-
vices in broad terms (Van de Walle, 2017). This type of bias calls for two types of solutions. On
the one hand, not placing questions on satisfaction with services after general assessments
of government or satisfaction with the system (e.g. trust, system responsiveness) or, if that is
not possible, otherwise using text as a buffer. Second, the inclusion of questions on service
attributes that are as detailed as possible could help overcome this bias (González, 2020).

HANDBOOK ON GOVERNANCE STATISTICS | 141

 • If attribute questions are asked, another placement issue relates to the positioning of the
overall satisfaction question. If this question is included before the specific attributes, there
is a risk that it will be interpreted differently by respondents. For example, evidence from the
Mexican National Survey of Quality and Governmental Impact (ENCIG) found that more ed-
ucated respondents who interact more frequently with public institutions have higher expec-
tations in terms of what constitutes a good quality service than do the rest of the population
(ENCIG, 2017). In order to generate a common understanding of the aspects considered
as elements of good quality, it is suggested to “prime” respondents with a set of attributes
of good quality of service provision prior to asking about their overall satisfaction (UNDP,
2019b). It is therefore recommended to include the overall satisfaction question after the set
of attributes.
• Since order effects (e.g. the different order in which questions and answer choices are pre-
sented influences respondents’ answers in a more or less systematic way) are more common
when two or more questions deal with the same or closely related issues, it is recommended
to randomize the order of services when enquiring about satisfaction with services.
• Whenever lists of institutions are used, it is recommended that the order of these institutions
be randomized across respondents to minimize order effects.

VII.3.3.4 Survey mode

323. System responsiveness and satisfaction with services can be highly sensitive topics, triggering
respondents to answer in a socially desirable way or making them unwilling to answer at all. This might
especially be the case in contexts where freedoms are restricted, and by extension there might be a
low level of trust in the official data collector. Self-administered surveys, compared to interviewer-led
ones, perform better in terms of minimizing social desirability biases, so if possible they should be
preferred. Whatever the survey mode, sensitivity-related response biases can be reduced by lowering
the respondent’s concerns about data protection (e.g. via confidentiality assurances provided at the
beginning of the survey) or by controlling the survey situation (e.g. avoiding that enumerators provide
information about the person’s social identity). Moreover, to avoid other potential causes for bias, for
instance answering filter questions to abstain from answering follow-up questions due to response
fatigue, the general principle of having parsimonious surveys should be observed. The following
considerations can reduce these risks:

VII.3.3.5 Response styles and cultural context

324. Cross-cultural response styles are very difficult to verify externally against a common standard
or actual behaviour. Having different response styles does not imply that data are of lower quality;
however, it is possible to reduce their effects by adhering to the considerations below.

• In many national contexts, minority groups have special rights or are entitled to special
provision of services (e.g. in their own language or free of charge). In these contexts the
questionnaire should be “indigenized” to fit the national context. For further general recom-
mendations see the similar section on system responsiveness in this chapter.
• The overall survey design (including its length and how it is introduced) needs to pay particu-
lar attention to respondent burden, motivation and fatigue in order to maximize data quality.

142 |
VII.4 Recommended key indicators
325. As described in detail above, for both areas covered by this chapter — namely system re-
sponsiveness and satisfaction with services — the basic concepts are well captured in the two SDG
indicators that directly relate to them. This chapter therefore limits itself to recommending only these
two SDG indicators (measured as specified above) as a primary focus for NSOs seeking to measure
system responsiveness and satisfaction with services.

TABLE VII.3 Recommended key indicators

Sub-dimension Indicator Data source Type

System responsiveness Proportion of population who believe decision-making is Survey Outcome

inclusive and responsive (SDG indicator 16.7.2)

Satisfaction with services Proportion of the population satisfied with their last experience of public services, Survey Outcome
specifically a) health-care services, b) education services and c) government services (i.e.
services to obtain government-issued identification documents and services for the civil
registration of life events such as births, marriages and deaths) (SDG indicator 16.6.2)

VII.5 Way forward

326. This chapter analyses a general aspect (e.g. system responsiveness) and a detailed one (e.g.
satisfaction with services) of the overall public governance dimension of responsiveness. It argues that
it is possible to close some of the gaps in the field of system responsiveness (i.e. external political
efficacy) and satisfaction with services, while also presenting some areas where further exploration will
be required. Most importantly in terms of recent methodological achievements, the questionnaires
developed for measuring the corresponding SDG 16 targets provide a good basis to standardize and
expand the collection of statistics in these areas, and recent research on the statistical quality of these
measures further strengthens the case for capturing them globally.

327. System responsiveness has a long, but somehow inconsistent measurement tradition.
Measurement efforts have often been driven by researchers and relied on non-official household
surveys. Nevertheless, important progress has been achieved for understanding and measuring this
concept. By taking stock of available evidence for examining criteria for statistical quality, this chapter
has found that the so-called NOSAY and INFLUENCE IN POLITICS questions perform well in terms
of reliability and validity and are suggested to NSOs interested in collecting these indicators. Further
exploring potential differences in the wording between “the political system” and “the government”
and elucidating equivalences for different types of scales could further contribute to strengthen the
proposed set of questions.

328. Satisfaction with health and education services are commonly generated as official statistics
by some NSOs and also measured in non-official comparative surveys. While most surveys treat these
equivalently, recent evidence referred to in this chapter shows that they are at different levels of ma-
turity, and evidence on the accuracy of satisfaction with health metrics is stronger than for education
services. In the case of health measures, there is good evidence of reliability and face validity, while

HANDBOOK ON GOVERNANCE STATISTICS | 143

 evidence on construct validity is not conclusive, and convergent validity cannot be tested due to the
absence of proxy metrics. In the case of education, the evidence on reliability and validity is weaker.
The poorer performance of questions on satisfaction with the education sector could be explained
by respondents without direct exposure or reference answering these questions. It is therefore rec-
ommended that these questions be restricted to relevant segments of the population. While these
findings have the potential to be extended to other services (e.g. general services), these have not
been systematically studied in this chapter.

329. Additionally, no international agreement exists in terms of the question formulations and re-
sponse scales to be used. However, existing evidence sheds light on the concepts to be measured (i.e.
attributes and general satisfaction) as well as the biases to be avoided. General biases towards gov-
ernment or the public sector that could negatively influence responses on satisfaction with services
can be substantially mitigated by questions about direct experiences with the use of services and by
observing good practices in survey design (e.g. ordering of items, etc.). In terms of the response scale,
and while no conclusive evidence exists, recent testing suggests that a 4-point bipolar Likert scale with
scale labels could be used without losing precision.

330. While general satisfaction questions could provide relevant information, attribute-specific
questions are more informative and meaningful for policy makers. The specificity of the information
generated by such questions, directly linked to satisfaction, as well as the focus on citizen experienc-
es rather than simply on perceptions, have greater policy use than stand-alone perception data on
overall satisfaction, and this will help NSOs build a stronger case for collecting these questions. While
there has been progress in testing the statistical quality of general satisfaction questions, rigorous sta-
tistical testing of questions on specific attributes questions is still lacking. The module put forward for
measuring target 16.6.2 could be subject to further refinement as additional evidence is generated.

331. All in all, substantial progress has been achieved for the measurement of system responsive-
ness and satisfaction with services, on the basis of which NSOs can produce and refine these statistics.
Finally, there is an agenda ahead to explore further elements of the responsiveness dimension and the
response by public institutions to different type of stimuli.

144 |
HANDBOOK ON GOVERNANCE STATISTICS | 145
 PART B
EIGHT DIMENSIONS OF
GOVERNANCE STATISTICS

CHAPTER VIII
Absence of
corruption
 VIII.1 Conceptualizing this dimension
332. Corruption is a complex and evolving phenomenon; it takes on many forms and is perpetrated
by various actors. One of the challenges to measuring corruption is how to define this phenomenon
in order to develop adequate indicators. The concept is broad, and there is no consensus on an
exhaustive definition. To overcome this difficulty, the United Nations Convention against Corruption
(UNCAC) identified a list of specific behaviours to be criminalized in national legislations, instead of
proposing an overarching definition. In this chapter a broad framework is presented and three main
dimensions are identified with the objective of making progress towards the elimination of corruption.

VIII.1.1 Normative framework

333. Corruption is directly addressed in internationally agreed normative frameworks. The United
Nations Convention against Corruption (UNCAC) was adopted in 2003 and entered into force in 2005.
With 186 States parties, UNCAC is approaching universal adherence, making it one of the most ratified
United Nations conventions. The adoption and widespread ratification of UNCAC is a demonstration
of the commitment of UN Member States to tackle corruption.

334. Furthermore, the universal human rights convention provides an overarching normative frame-
work. Indeed, different reports of the Human Rights Council underline the “negative impact” of cor-
ruption on the enjoyment of human rights.1 By way of illustration, corruption in the health or education
sector affects the right to health or to education (Articles 12 and 13 of the International Covenant on
Economic, Social and Cultural Rights2). Similarly, corruption in the judicial sector endangers the rights
that guarantee every person access to an independent and impartial process and the opportunity
to receive a fair and just trial (Article 14, International Covenant on Civil and Political Rights3). More
generally, corruption leads to the violation of the prohibitions against discrimination found in the
universal human rights convention.

335. Article 61 of UNCAC, on the collection, exchange and analysis of information on corruption,
underscores the need to analyse trends in corruption and the circumstances in which corruption of-
fences are committed. Importance is given to developing and sharing statistics, analytical expertise
and information on corruption, with a view to producing common definitions, standards and method-
ologies, as well as information on best practices to prevent and combat corruption. In addition, Article
61 emphasizes the monitoring of policies and actual measures to combat corruption and to assess
their effectiveness and efficiency.

VIII.1.2 The different types of corruption

336. UNCAC does not contain one single definition of corruption. Instead, it recognizes that cor-
ruption comprises various types of offences and requires States to criminalize the following conducts:

• Bribery (active and passive) of national public officials (article 15)

• Bribery of foreign public officials and officials of public international organizations (article 16)
• Embezzlement, misappropriation or other diversion of property by a public official (article 17)
• Trading in influence (article 18)

1 See for example the foreword to UNCAC (2003) by UN Secretary-General Kofi Annan: “Corruption is an insidious plague that has a wide range of
corrosive effects on societies. It undermines democracy and the rule of law, leads to violations of human rights, distorts markets, erodes the quality of life
and allows organized crime, terrorism and other threats to human security to flourish.”
2 https://www.ohchr.org/en/professionalinterest/pages/cescr.aspx
3 https://www.ohchr.org/EN/ProfessionalInterest/Pages/CCPR.aspx

148 |
 • Abuse of functions (article 19)
• Illicit enrichment (article 20)
• Bribery in the private sector (article 21)
• Embezzlement of property in the private sector (article 22)
• Laundering of proceeds of crime (article 23)
• Concealment (article 24)
• Obstruction of justice (article 25)

337. Several approaches have been proposed in the literature to define and/or classify the various
types of corruption that exist. For example, a distinction can be made between two types of corrup-
tion, depending on the actors and the objective of the corruptive act:

• Bureaucratic Corruption. Corrupt acts of civil servants/bureaucrats in their dealings with

either their superiors (the political elite) or with the public. In its most common form, usually
known as petty corruption, the public may be required to bribe public servants either to re-
ceive a service to which they are entitled or to speed up a bureaucratic procedure. Corruption
in the judiciary, where bribes can lower either the costs or the chances of legal penalties, is
another form of this type of corruption (Rose-Ackerman, 1998).
• Legislative/political Corruption. This refers to the manner and the extent to which the vot-
ing behaviour of legislators can be influenced. Legislators can be bribed by interest groups,
for example to enact legislation that can change the economic rents associated with assets.
This type of corruption includes also "vote-buying" (Rose-Ackerman, 1999).

338. Another distinction is frequently made between grand corruption and petty corruption. Grand
corruption refers to the acts of high-level officials (political or administrative elites) by which they
exploit their power to make policies (Krueger, 1993a, 1993b). But the differentiation between grand
and petty corruption is also related to the amount of the transaction and/or the size of its impact. A
corrupt elite can change either national policies or their implementation to serve its own interests; for
example, public spending can be diverted to sectors where gains from corruption are possible to the
detriment of the needs of the collectivity (Porta and Vannucci, 1997).

339. Furthermore, corruption can also take place in the private sector (private-to-private corrup-
tion), and it can also be classified as grand or petty corruption depending on the level of power of the
person involved and the amount of the transaction.

VIII.1.3 What sub-dimensions can be measured?

340. Broadly speaking, the absence of corruption can be assessed by examining three
sub-dimensions:

1) the level of intolerance to corruption (i.e. ethical values, principles and norms that strengthen
resistance to corruption practices);
2) the levels and patterns of corrupt practices; and
3) the State response to corruption.

341. As such, the absence of corruption is the result of a combination of these three components
and can be measured by “triangulating” data and information on them. Each one of these three
components can be viewed as one part of an entire raft of processes. The level of intolerance to
corruption is related to the upstream phase where corruption can be prevented. State responses to

HANDBOOK ON GOVERNANCE STATISTICS | 149

 corruption intervene further downstream in the process (even if actions to tackle corruption include
also the promotion of integrity and transparency). Therefore, instead of focusing on just one stage
of the process, the objective is to broaden the focus as a way to understand and identify precisely
at which level and at which stage the sources of potential problems and discontent are to be found.

VIII.1.3.1 Intolerance to corruption

342. The underlying principle of this sub-dimension is to foster integrity within public sector institu-
tions as a strategy to prevent corruption. As integrity means using power for officially authorized and
publicly justified purposes, it is the opposite of corruption. The 2017 OECD Recommendation of the
Council on Public Integrity provides a global framework for implementing policies to improve several
dimensions of public sector integrity, a concept whose scope goes well beyond reducing corruption.
Various definitions of integrity exist, and they broadly refer to “behaviours and attitudes following eth-
ical standards that can counter corruption4.” Therefore, relevant methodologies should be developed
to capture the way public officers, or ordinary citizens, behave when confronted with certain situations
and understand the concept of corruption, as well as their awareness and their perception of the
phenomenon, their values and beliefs.5

VIII.1.3.2 Corrupt practices

343. The measurement of corruption is challenging. Corruption is often carried out in a clandestine
manner, not least because it is legally a crime and public exposure could lead to legal sanctions.
Collecting accurate data on corruption is therefore at least as challenging as gathering evidence on
any other type of crime. In the case of corruption, the collection of statistical evidence is further com-
plicated by three main factors (UNODC, 2010a and 2010b):

• When national legislation is not fully consistent with the UN Convention against Corruption,
borders between licit and illicit, or appropriate and inappropriate behaviours, are often
blurred;
• As UNCAC calls for the criminalization of a catalogue of corruption offences, an accurate
assessment would require collection of data on each of those offenses, a daunting task;
• In comparison to other offences, persons involved in corruption cases are less prone to report
to competent authorities for reasons such as fear of retaliation or reluctance to fight an estab-
lished practice, or because they are to some extent co-responsible for the crime.

344. However, adequate approaches can be put in place to overcome these difficulties, and it must
be stressed that people involved in corruption cases are willing to report if a dedicated body and
appropriate systems and procedures are in place.

345. Assessing the extent and modalities of corruption affecting the public sector usually focuses
on forms of administrative bribery taking place during contacts between civil servants and users of
public services. Information on personal experiences of bribery is collected from the general public in
order to understand the extent and modalities of bribery, to identify sectors, processes and functions
more vulnerable to the risk of bribery, and to monitor trends over time.

4 See for example the definition provided by Transparency International, which is, “Behaviours and actions, consistent with a set of moral and ethical
principles and standards, embraced by individuals as well as institutions that create a barrier to corruption” (Transparency International, 2009). See also the
definition of integrity in UNDP (2015), pp. 29-30.
5 Methodologies for assessing integrity within private sector institutions are not part of this chapter.

150 |
 346. The usual focus on the public sector and on experiences of bureaucratic corruption should
not detract from trying to capture other types of corruption. First, corruption in the private sector
(private-to-private corruption) is understudied and difficult to assess. Even if actions to fight this type
of corruption fall mainly under the responsibility of the private companies concerned, its effects can
be detrimental to society in general (cases of corruption in private schools or hospitals are an exam-
ple). Second, grand corruption is a phenomenon that is difficult to measure, but media reports and
public discourses about high-profile corruption cases often strongly influence individual perceptions
of corruption per se.

347. Clearly, perception indicators are by nature subjective and may not reflect actual levels of
grand corruption. At the same time, subjective opinions and private judgements on the scope and
extent of corruption (including grand corruption) matter for countries’ socioeconomic and political
stability. Although difficult to measure, prevailing attitudes and perceptions may also contribute to a
climate where corruption seems to be generally accepted, thereby reducing moral barriers to engage
in similar corrupt acts. Thus, indicators on both perceptions and experiences, two elements not nec-
essarily correlated, are important for policy-making and/or policy evaluation and useful to measure
(Razafindrakoto and Roubaud, 2018). In both cases, though, it is vital that statistical information is
produced on the basis of transparent and solid methodologies.6

VIII.1.3.3 State responses to corruption7

348. One of the main ways to fight corruption is to ensure that channels for reporting to authorities
are functioning correctly and that appropriate subsequent action is taken by the criminal justice sys-
tem. Information on reporting patterns (e.g. “whistle-blowers”) and on the response of the criminal
justice system to detected cases of corruption is fundamental for assessing whether the State response
to corruption is adequate. In this context, data on investigations and legal proceedings relating to
corruption illustrate the challenges and difficulties of investigating and substantiating corruption cases
reported to official authorities.

349. But actions can also be taken further upstream to prevent corruption, for instance through an
information campaign or a sensitization program addressed either to specific categories of the popu-
lation (youth, civil servants, etc.) or to the general public. Therefore, data that provide information on
whether citizens are aware of the existence of any official anti-corruption mechanism or programme,
and on their views on its effectiveness, are useful. It is one way to assess government effectiveness in
controlling corruption.

VIII.2 Why is this dimension important?

350. Corruption has a detrimental impact on political, social, cultural, institutional and organization-
al structures, on economic and structural policies, and on human rights, and it can affect numerous
aspects of everyday life. Corruption affects the patterns of resource allocation as well as the distribu-
tion of income within a society. It raises the cost of transactions: it has a negative effect on investment
levels and patterns, and more globally on the effectiveness of economic policy. While some authors

6 See the box on the surveys implemented by INEGI-Mexico in Box VIII.4.

7 Responses to corruption can also involve civil society. Actions can be undertaken by NGOs to contribute to fighting corruption (information, sensitization,
etc.). But the idea is to put the focus on the State or para-State institutions in charge of implementing government anti-corruption policy.

HANDBOOK ON GOVERNANCE STATISTICS | 151

 have suggested that corruption can help to overcome bureaucratic rigidities (Leff, 1964; Lui, 1985;
Dreher and Gassebner, 2013; Vial and Hanoteau, 2010), globally research fails to support this “grease-
the-wheels” argument. Most analyses conclude that corruption acts mainly as “sand-in-the-machine”
(Jain, 2001). Beyond the economic sphere, corruption also has an impact on the socio-political envi-
ronment. It negatively effects trust in institutions, satisfaction with democracy, and government effec-
tiveness (Anderson and Tverdova, 2003; Tavits, 2008).

351. Because of corruption’s negative impact, it is important to foster a corruption-free environment

and, for that purpose, to monitor national and international efforts that can result in a situation where
corruption is absent. The importance of reducing corruption is also recognized explicitly in the 2030
Agenda for Sustainable Development (SDG target 16.5).The three identified dimensions can be mon-
itored through a number of different metrics that can be generated in different manners.

VIII.3 Data and best practices currently available

VIII.3.1 Administrative data

VIII.3.1.1 Description

352. Official administrative data on reported cases of corruption provides an experience-based

source of statistical information on corruption. Such data can come from a variety of sources (police,
prosecutors, courts, anti-corruption agencies). Administrative data may also refer to data on processes
related to key State functions (public procurement, recruitment of civil servants, granting of permis-
sions/licenses, etc.) in relation to duration, accessibility, access and transparency.

353. Administrative statistics on corruption are mostly limited to counts of criminal justice or ad-
ministrative responses, in terms of crimes reported to the police or anti-corruption bodies, persons
arrested, persons charged and persons convicted.8 Of course, these data, although crucial for bench-
marking and monitoring the implementation of anti-corruption measures, cannot be used to measure
the extent of the phenomenon, as they reflect only cases that are detected and recorded by official
authorities.

354. The following types of administrative data relevant to the measurement of corruption exist:

355. Reported cases of corruption: Official data on reported cases of corruption can represent
an initial step towards the assessment of corruption, its extent and societies’ vulnerability to it. But
caution needs to be applied in the interpretation of trends in reported corruption offences. Changes
in recorded offences might have several explanations (changes in recording criteria, impact of sensi-
tization campaign, changes in anti-corruption efforts, etc.), which are not necessarily linked to actual
changes in the level of corruption. Even though the number of reported cases is unreliable as a cor-
ruption measure at large, the availability of detailed data on offences committed and officials involved
can provide insights into specific areas of vulnerability to corruption and into the effectiveness of the
State’s anti-corruption policy.

8 We will see below the key role NSOs can play to produce other type of official statistics through surveys.

152 |
 356. Criminal justice response to corruption: Given the notoriously low reporting rates for cor-
ruption, criminal justice data on corruption mainly reflect the response of law enforcement and
criminal justice systems rather than providing information about the true extent of corruption itself.
Administrative data on corruption may refer to the number of bribery offences as well as the number
of people arrested, prosecuted and held in prison for bribery offences. In addition, other types of data
are often collected, such as reports on corruption and investigations; cases and persons referred to
court; and persons convicted of corruption offences. When data are available on each phase of the
criminal justice process, the overall efficiency of the criminal justice system related to corruption could
be evaluated using tools such as attrition rates.9

357. Asset declaration: Administrative data on the assets held and reported by public officials have
been collected by a great number of countries around the world in order to prevent and combat
corruption (OECD, 2011). The main aims of asset declarations include the following:

a) To increase transparency and the trust of citizens in public administration, by disclosing infor-
mation about the assets of politicians and civil servants;
b) To help heads of public institutions prevent conflicts of interest among their employees and to
resolve such situations when they arise;
c) To monitor variations in the wealth of individual politicians and civil servants in order to dis-
suade them from misconduct and protect them from false accusations and to help clarify the
full scope of illicit enrichment or other illegal activity by providing additional evidence.

358. Public procurement: Implementing an effective public procurement system based on trans-
parency, competition and integrity is not simple. A procurement system that lacks transparency and
competition is the ideal breeding ground for corrupt behaviour, and thus the most important interna-
tional codes on anti-corruption and public procurement rest heavily upon these fundamental princi-
ples. UNCAC calls for the establishment of appropriate systems of public procurement based on the
fundamental principles of transparency, competition and objective criteria in decision-making. Thus,
collecting administrative data on public procurement processes is fundamental to the measurement
of absence of corruption.

359. Recruitment of civil servants: One crucial area for ensuring the integrity of the civil service, as
well as its capacity to deliver results, is the selection and recruitment of new staff. Full transparency
and fairness in the recruitment process is not only necessary for implementing integrity standards
throughout the selection process, but also for ensuring the best match between applicants’ experi-
ence and skills and the position’s objective requirements.

VIII.3.1.2 Methodological and quality issues

360. The lack of comparability of data across administrative levels can be a problem. For instance,
institutions operating at the local level can record data differently than at the central level. This can
also be the case between institutions operating in a given sector (the police, the courts, communes,
etc.). Coordination between the services in charge of data repositories in different institutions is a
key task. There is a need to harmonize definitions, formats and schedules between institutions and
along the “data production chain” in a given sector so as to obtain compatible and consistent data
over time. The piloting experience of the GPS-ShaSA initiative (AUC-IRD/DIAL, 2018) in Kenya, Côte

9 The attrition rate is defined as the ratio of persons referred to court and convicted out of all those investigated in corruption cases.

HANDBOOK ON GOVERNANCE STATISTICS | 153

 d’Ivoire and Cape Verde shows the need to establish “collaboration protocols” between the NSO
and data-producing institutions (e.g. courts, parliament, national anti-corruption commission, police
stations, national human rights commission, etc.) to address issues concerning the non-comparability
of data collected by various institutions.

361. Some of the major methodological issues that require careful consideration during the de-
velopment of administrative data on criminal justice (see also the more detailed discussion in the
Handbook chapters on the access to and quality of justice and on safety and security), including data
on the State response to corruption, include the following:

1) Counting units and rules: typically, each criminal justice institution uses counting units based
on its own operational requirements. Police may use charges, suspects, victims and incidents,
while courts may use cases, convictions and sentences. Linking data across institutions re-
quires the use of the same counting units (such as persons suspected, and persons convicted);
2) The principal offence rule: This means that when more than one offence is committed si-
multaneously, only the most serious offence is recorded. Countries that apply the principal
offence rule record only the most serious offence if simultaneous offences are committed;
others record each offence separately, which results in a higher count of recorded crimes;
3) The moment of inclusion of incidents into statistics: Criminal justice institutions can collect
data from different stages of their respective criminal justice process. For example, ‘‘input
statistics’’ refer to data being collected at the time the offence is first reported to authorities;
on the other hand, “output statistics’’ refer to data being collected after the offence has been
investigated (UN Department of Economic and Social Affairs, 2003).

362. One of the most important considerations when comparing data on corruption offences (or
any other criminal offence) between criminal justice institutions is the use of consistent definitions
and classification categories. Both at the national and international levels, the implementation of
the International Classification of Crime for Statistical Purposes (ICCS) (UNODC, 2015) provides a
framework for the systematic production and comparison of statistical data across different criminal
justice institutions and jurisdictions, in particular for State responses and trends over time for the
same categories of crime. In this context, Section 07 of the ICCS (Acts involving Fraud, Deception or
Corruption) provides a comprehensive classification of corruption offences that is consistent with the
United Nations Convention against Corruption, and it should be applied at all levels of the criminal
justice system.

363. Administrative data on reported cases of corruption suffer from a high “dark figure”, i.e.
the share of corruption offences that is not reported to, or detected by, criminal justice institutions.
Contrary to other crimes, there is no clear victim (such as in a case of robbery or car theft) who has an
interest in reporting their experience to authorities. This is also due to the limited and varying capacity
of criminal justice institutions to detect corruption-related crimes, as well as the usually low interest of
persons directly or indirectly involved in corruption cases to report such cases.

364. Findings from recent surveys conducted at national level indicate that the level of reporting of
bribery cases to relevant authorities is, on average, only 12 per cent across European Union Member
States (European Commission, 2014) and well below 10 per cent in other countries in Europe, Asia
and Africa (UNODC, 2010c, 2011a, 2013a, 2017). Moreover, reported cases may not be considered as
representative of the whole of the corruption affecting a given country, as some corruption offences

154 |
or types are more likely to be reported or detected than others.10 Administrative data on corrup-
tion should therefore be interpreted with caution, as they may provide more information about the
activity and response of criminal justice systems than about the actual extent of the phenomenon.
Furthermore, there is generally limited availability and comparability of administrative data on corrup-
tion at national, regional and global levels.

365. In contrast, administrative data on integrity and transparency may be more accurate (at least for
the limited scope of their measurement), but they are not yet widely available, and standard method-
ologies and concepts are needed (see the Handbook chapter on “Openness” for further discussion).

BOX VIII.1 Administrative Data: Best Practices/Useful Tools

Example from Mexico: Integrating administrative data on corruption

Since 2011, Mexico’s NSO, INEGI, has conducted so-called “Government National Censuses” (not to be
confused with a population census). The primary goal of this undertaking is to gather and systematize all the
administrative records and information available inside every government institution at federal, state and
municipal level, and concerning the executive, legislative and judicial branches, as well as the autonomous
organisms. These Government Censuses centre on State capacity and organization (human, material and
financial resources) and the institutions’ core functions. Regarding corruption, Government National Censuses
measure, predominantly, internal mechanisms to prevent or sanction acts of corruption. However, they also
integrate and harmonize the records of the criminal justice system relevant to corruption acts (e.g. people
incarcerated for corruption-related crimes).

Structural Process

• Federal Government • Anti-corruption programmes

• Government, Public Safety and State • Internal control offices
Prison Systems • Auditing
• Munipal Governments • Assets declaration registries
• Federal Justice • Sanctions and sanctioned public servants
• State Justice • Personnel in charge of transparency
• Federal Prosecution • Crime committed by public servants
• Federal Prison System

Useful tools:

UNODC International Classification of Crime for Statistical Purposes (ICCS) (2015), https://www.unodc.org/
unodc/en/data-and-analysis/statistics/iccs.html
OECD Asset Declarations for Public Officials: A Tool to Prevent Corruption (2011), https://www.oecd.org/daf/
anti-bribery/assetdeclarationsforpublicofficialsatooltopreventcorruption.htm

VIII.3.2 Sample surveys among the population and businesses

VIII.3.2.1 Description

366. Sample surveys on corruption were first implemented in the 1990s, and thanks to a large body
of experience, relevant methodologies have gradually improved. This experience has recently been

10 The issue of selective reporting is also known as selection bias. Selection bias could also arise due to the characteristics of people that report the
corruption case.

HANDBOOK ON GOVERNANCE STATISTICS | 155

BOX VIII.2 Benefits and drawbacks of corruption surveys

Benefits
Probably the most apparent advantage of measuring corruption through sample surveys is that this is a widely-used methodology which, when
implemented correctly, can rely on a solid theoretical background and long-standing experience at academic and institutional level. The possibility
of computing estimates of the indicators of interest through transparent methods, along with measures of their accuracy, is a fundamental
advantage of this approach.
Corruption surveys enable direct access to those who have experienced corruption. For that reason, information can be collected on the variable
of interest (such as the experience of bribery) as well as on several characteristics, behaviours and phenomena associated with it. For example,
detailed information on the individuals involved (bribe payers or receivers) and their demographic, social and economic background can help
identify whether features specific to households, individuals, businesses or countries raise or lower the risk of corruption.11
Moreover, by requesting information about the type of public official or private entity involved (e.g. customs officers, police officers, tax/revenue
officials, court officials, etc.), about the situation or administrative/business procedure during which the bribe was requested/offered (e.g. public
procurement, customs clearance, issuance of building permits) or about the reason for its request (e.g. speeding up the procedure, obtaining an
advantage over other participants in a bid), it is possible to acquire a comprehensive understanding of the mechanism of bribery. All these details are
fundamental to understanding the drivers of and vulnerabilities to corruption and, thus, to providing actionable and policy-relevant information.
Another important benefit of corruption surveys is their capacity to overcome the undercounting problem affecting administrative statistics on
crime. Sample surveys guarantee the anonymity of responses and provide a context that makes respondents more amenable to disclosing their
experiences. When correctly implemented, corruption surveys can even produce estimates of the “dark figure” of bribery.
In addition to overcoming underreporting problems, sample surveys enable comparability of data, which is key to monitoring trends and assessing
anti-corruption measures. They also enable the coverage of different target populations (e.g. individuals, businesses, civil servants), which is of
utmost importance in understanding how types of corruption and risks vary among different actors. While the focus of this Manual is both on
the general population and business, other target populations of surveys can be considered, such as public officials, users of public services, or
businesses participating in public sector bids. Within target populations, data collected through sample surveys enable the description of the
phenomenon of interest, its dynamics and mechanisms in relation to various subpopulation groups of particular interest, for example, groups that
are particularly vulnerable to, or at risk of, corruption.
Finally, sample surveys enable the gathering of micro-level data, providing analysis at the highest level of disaggregation: the crime incident and
its victim. Collecting data at the individual level helps to overcome the “ecological fallacy”, which occurs when individual behaviours are explained
solely through data collected at an aggregated level. Furthermore, the dissemination of micro-data, in accordance with legislation to protect
privacy, can stimulate further research and analysis of corruption patterns and trends.
Drawbacks
The most common critique of surveys on corruption is linked to social desirability bias, or unwillingness to admit socially undesirable behaviour.
As corruption is recognized as a socially undesirable issue, respondents’ fear or shame of admitting their experience may lead them to underreport
bribery. This bias varies among populations (e.g. businesses may be more sensitive to reputational damage than individual citizens), and it is also
influenced by “whether the respondents benefited or not from corruption and how detrimental or justified the respondent views his or her actions
to be”. The impact of non-disclosure can be a particular problem when a bribe is very large; for this reason, surveys are not considered to provide
accurate results for cases of corruption involving very large sums or assets. However, experiences from household and business surveys indicate
that well-designed surveys can also collect information on bribery cases entailing substantial amounts. This issue can be partially addressed by
selecting interviewing techniques that can maximize the confidentiality of responses. Nevertheless, research has shown that social desirability
bias is an issue that is difficult to control completely.
11
Other possible limitations of corruption surveys relate to reporting bias. As in all surveys in which information is directly elicited from those in
possession of it, two main issues can impact the accuracy of the replies provided by respondents: 1) they misunderstand the question; and 2)
they fail to remember the correct answer. The first issue is strongly related to the fact that crime and corruption are social constructs and the
perception and interpretation of them can vary across citizenry, particularly in the case of types of crimes for which perceptions may be most
culture-bound. These concerns can, however, be taken under control through sound questionnaire design, including ad-hoc sections for “tackling
cultural bias” and proper question wording. The second issue is related to non-recall and mis-recall. Non-recall usually depends on the memory
decay of respondents, while mis-recall is based mainly on the “telescoping effect”.
Given the cost of conducting sample surveys, the sustainability of this approach is also a critical element. As with any other sample survey, the cost
is dependent on several factors (sample size, scope of survey, type of data collection method, etc.), and there is usually a direct trade-off between
survey cost and overall quality. Using short modules on corruption in already existing sample surveys at national level, instead of developing
ad-hoc surveys, can be a valid strategy to address this problem.

Source: Manual on Corruption Surveys (2018), UNODC, UNODC-INEGI CoE, and UNDP

11 In sample surveys accurate information on perceptions. attitudes and opinions about corruption can be collected. When properly gathered, information on corruption perception provides
important information on acceptability or impact of anti-corruption policies, as well as on the context where such policies are implemented.

156 |
consolidated in the Manual on Corruption Surveys: Methodological guidelines on the measurement
of bribery and other forms of corruption through sample surveys, jointly produced by UNODC, the
UNODC-INEGI Centre of Excellence and UNDP (2018). The Manual provides comprehensive guid-
ance on measuring corruption through population- and businesses-based sample surveys and on the
production of the two SDG indicators on bribery prevalence (SDG indicators 16.5.1 and 16.5.2).

367. Historically, two approaches have been adopted for measuring corruption through sample
surveys. On the one hand, victimization surveys often include a section to measure the experience of
bribery among the population. These surveys were launched in different countries by or in keeping to
the tradition of the International Crime Victims Survey (ICVS), an international programme for measur-
ing the direct experience of crime initiated in 1987 by a group of European criminologists.12 The first
ICVS survey that included a corruption module was implemented in 1996.

368. At international level, such efforts have been promoted by UNODC, which supported popu-
lation-based surveys on corruption in Afghanistan (2009 and 2012), Iraq (2011), the western Balkans
(2010) and Nigeria (2016 and 2019), and business surveys on corruption were conducted in the western
Balkans (2013). At regional level, the GPS-SHaSA programme13 aims at producing harmonized house-
hold statistics on governance, peace and security at continental level in Africa. Questions on bribery
have been included since the first GPS-SHaSA pilot surveys.

369. On the other hand, a group of surveys aimed at corruption measurement were designed using
a socioeconomic approach. On the business side, the World Bank Enterprise Surveys and Business
Enterprise Economic Surveys have the largest firm-level survey data on the experience of bribery. On
the household side, Madagascar’s NSOs conducted a survey with a module on the perception and
experience of corruption in 1995 (Razafindrakoto and Roubaud, 2003). Since then, the experience in
developing this kind of survey has been expanded to other continental African countries, some Latin
American countries (Andean countries, with Peru’s NSO taking the lead with quarterly national time
series since 2002) and Asia (Viet Nam).

TABLE VIII.1 Examples of sample surveys on the experience and perception of corruption

Surveys Institutions Sources

International Crime Victims Survey (ICVS) Household

National experience-based corruption surveys UNODC Household

Global Corruption Barometer (GCB) Transparency International Household

Afrobarometer surveys AB network Household

GPS – SHaSA Surveys (+ other national surveys) NSOs - AU Household

Business Enterprise Economic Surveys World Bank Firm

Enterprise Surveys World Bank Firm

Sources: Transparency International (2016), How-to guide for corruption assessment tools (2nd ed.), https://www.transparency.org/files/content/corruptionqas/
How_to_guide_corruption_assessment_tools_2016.pdf; UNODC-INEGI Center of Excellence, Repository of Corruption Measurement Tools.

12 The first surveys were implemented by the ministries of justice of 14 developed countries in 1989. See Van Dijk, Mayhew and Killias (1990).
13 The GPS-SHaSA programme (Governance, Peace and Security statistics under the Strategy for the Harmonisation of Statistics in Africa) was launched in
2012 at the initiative of the African Union Commission’s Economic Affairs Department in partnership with UNDP and DIAL/IRD.

HANDBOOK ON GOVERNANCE STATISTICS | 157

 VIII.3.2.2 Methodological and quality issues

370. When using sample surveys, it is important to be aware of the benefits and drawbacks of
measuring corruption through this approach. Many of the challenges (e.g. the impact of non-disclo-
sure, high economic costs, sustainability) can be overcome through sound methodological planning,
in particular, in the sample design, the questionnaire design and ordering, and the choice of survey
mode. The UNODC-UNDP Manual on Corruption Surveys (2018) provides comprehensive guidance
on the methodology for conducting corruption surveys from start to finish.

371. The importance of a lightweight, flexible, modular mechanism should be stressed. Due con-
sideration of data production conditions and constraints is key to ensuring the mechanism’s feasibility,
relevance, reliability and sustainability, especially in developing countries.

372. Asking about corruption in interviews can be a sensitive issue. In order to get reliable data,
the interviewer’s attitude is crucial. The interviewer must be completely neutral so that the respondent
feels totally at ease. Special attention must be paid to the wording of the questions on corruption. In
accordance with consistent practice, the inclusion of introductory sentences (which can also include
examples) is advisable. To avoid social desirability biases, neutral introductory, illustrative sentences
improve the contextualization of sensitive questions.

373. If interviewers are well trained, and if the surveys rely on questions already piloted and tested,
experience has shown that respondents are ready to answer them, minimizing non-response rates; in these
conditions, it is possible to produce reliable indicators for monitoring the extent and patterns of corruption.

374. Sampling frames and corresponding target populations of surveys among the business sec-
tor tend to exclude the informal sector. This also applies to corruption surveys among businesses,
which means that the bribery experience of informal economic entities is not reflected in such surveys
(Lavallée and Roubaud, 2019; Lavallée and Roubaud, 2014). In countries where the informal sector
is very significant, a recommended approach is to include a dedicated module to identify informal
sector entities in a household survey: for each individual belonging to the active working population
and who states that he or she is the owner or a self-employed worker of a unit satisfying the conditions
for membership of the informal sector (criterion relating to size or non-registration), a second enter-
prise questionnaire is collected (International Labour Organization, 2013), which includes business
corruption questions.

158 |
BOX VIII.3 Pros and cons of dedicated surveys and integrated modules

An important choice at the planning stage is whether to conduct a dedicated survey on corruption or to
develop a module (a set of questions on corruption) to be integrated into a broader household or business
survey. Victimization surveys, business environment and performance surveys, and surveys on the quality and
integrity of the public administration are examples of surveys that can include a corruption module. The two
approaches have advantages and disadvantages, and in general a trade-off exists between the accuracy and
comprehensiveness of data, on the one hand, and sustainability and costs, on the other.

a) Dedicated surveys
Dedicated surveys on corruption thoroughly address corruption and related issues (perception and acceptability
of corrupt behaviour, and awareness and effectiveness of anti-corruption agencies).

Pros:
• C omprehensiveness due to the possibility to include detailed questions both on the experience and
perception of corruption and on the characteristics of the most recent/serious incidents
• Accuracy of results due to the ability to devote more resources to screening for contact with public
officials and corruption incidents, unlike surveys that do not specifically focus on this issue
• Possibility for respondents to concentrate on one main topic and gradually arrive at core questions on
corruption experience; this approach may reduce memory decay
• Possibility to address other related topics, such as the characteristics of services/procedures, the
acceptability of corrupt behaviour, etc.
• Opportunity to develop an ad-hoc methodological design for the survey

Cons:
• Conducting a high-quality sample survey is a costly exercise
• Survey sustainability may be affected by the high cost of a dedicated survey
• Burden on survey respondents

b) Integrated modules
An integrated module consists of a core set of questions on the experience and perceptions of bribery and
the type of public official involved. Integrated modules on corruption are usually included in surveys that are
focused on other related issues, such as crime in general, governance, aspects of everyday life, the integrity of
public institutions, and business environments, among others.

Pros:
• S avings in cost and time of survey design and data collection, since a large part is absorbed by the main
survey cost (e.g. sampling design, fieldwork cost)
• Lower burden on respondents by limiting the number of questions to be answered
• Possibility to link to data from the same population on interlinked topics, which are collected by the
main survey (e.g. level of income, occupation, quality of life, quality of public services, etc.)
• When the main survey is financially well established, the sustainability of the collection of corruption
data will be periodically guaranteed. This would ensure the possibility of monitoring progress towards
reducing corruption and bribery (SDG indicator 16.5.1)

Cons:
• L imited set of questions on corruption and related follow-up on the most recent/serious incidents, in
the interest of covering other topics
• Possible impact on accuracy, as switching topics within the same survey can have an impact on the
attitude and attention of respondents. In addition, the introduction of “context effects”, i.e., one topic
in a multi-topic survey, may influence answers to other topics
• Need to adapt to the methodological design of the main survey. This might hinder, for example, the use
of a specific survey mode or sample design that is better suited to the objectives of a corruption survey
• Risk of insufficient and dedicated training for interviewers on how to address sensitive issues, such as
corruption

Source: Based largely on UNODC-UNDP (2018), Manual on corruption surveys: methodological guidelines on the measurement of bribery and other forms of
corruption through sample surveys, Vienna.

HANDBOOK ON GOVERNANCE STATISTICS | 159

BOX VIII.4 Surveys: Best Practices/Useful Tools

Corruption measurement in household surveys: The pioneering case of Madagascar

A first attempt to measure corruption within an official household survey was made in 1995 in Madagascar. The publishing of the results in the
local media had a strong impact on civil society and the national authorities, leading the Ministry of Justice to draft a law to fight corruption
(which was unfortunately rejected in 1999 by the government). The approach to measure corruption was renewed in 1998, and since 2000 a
“corruption module” has been systematically included in every labour force survey. Among the rich analytical findings provided by the Malagasy
survey data, two particular insights can be stressed: first, a strong negative correlation between the level of corruption and state employee wages.
This suggests that public administration performance depends largely on the salaries of its employees. Second, from 2002 to 2004 the incidence
of corruption halved. The burden of corruption on household budgets also fell. In 2004, the total amount spent on corruption stood at just 1.2 per
cent of annual household income (as opposed to 3.3 per cent in 2002). This implies that the government’s anti-corruption commitment, with the
establishment of an Anti-Corruption High Council (CSLCC) in 2003 and an Independent Anti-Corruption Bureau (BIANCO), played a significant role
in improving the situation. The Madagascar experience illustrates the usefulness of statistical household surveys as an effective tool to monitor
corruption and to inform policy-making. Given the success of Madagascar’s experience, the same methodology, with the same corruption module,
was applied by NSOs in seven West African capitals at the end of 2001, and in five Andean Pact countries in 2002-2003. The corruption questions
in the GPS-SHaSA module are mainly based on these experiences.

Corruption in Nigeria: Measuring the experience of bribery and the public response to it
A 2016 survey on corruption in Nigeria was carried out by the National Bureau of Statistics of Nigeria in partnership with UNODC. This experience-
based corruption survey provides a solid evidence base on the nature and extent of administrative bribery, perhaps the most familiar and
widespread form of corruption among the general population in Nigeria today. Among other topics it covers: the reach of bribery; how bribery
works; who takes bribes; who pays bribes; and how citizens respond to bribery. The report thus provides concrete guidance for the design of
anti-corruption strategies and policies. It generates actionable data on patterns and modalities of bribe-paying. It highlights particular areas of
vulnerability to bribery in the public administration and examines the State response to corruption.

The report finds that almost a third of Nigerian adults (32.3 per cent) who had contact with a public official between June 2015 and May 2016 had
to pay, or were requested to pay, a bribe to that public official. The magnitude of public sector bribery in Nigeria becomes even more palpable
when factoring in the frequency of those payments, as the majority of those who paid a bribe to a public official did so more than once over the
course of the year. According to the survey, bribe-payers in Nigeria pay an average of some six bribes per year, or roughly one bribe every two
months. The vast majority of bribery episodes in Nigeria are initiated either directly or indirectly by public officials (85.3 per cent), and almost 70
per cent of bribes are paid before a service is rendered.

FIGURE VIII.1 Patterns of bribery in Nigeria

Type of bribe request Timing of payment

Nobody asked for it,

I did it to facilitate/
Do not know After the Do not know
accelerate the procedure
/ NA service was / NA
8.2%
3.7% delivered 5.6%
8.2%

Direct Before the

request from service was
the official delivered
65.4% Partly 68.9%
A third before/
person partly after
requested the the service
extra payment The official indirectly was delivered At the same time that the
2.8% requested a payment 19.9% 2.0% service was delivered 10.0%

160 |
BOX VIII.4 Best practices/Useful tools, CONT.

Poverty and corruption in Peru

A module on corruption is included in the periodic household survey (ENAHO) implemented by Peru’s NSO. The survey is carried continuously
on a sample of around 40,000 households, including a rotating panel. According to this survey, the incidence of corruption shows a downward
trend after a peak attained in 2008 during the mandate of President Garcia (he committed suicide in 2018 when about to be arrested on charges
of corruption). The percentage of victims is always lower among poor people than non-poor people (respectively 3 per cent and 6.7 per cent in
2008; 1.3 per cent and 2.9 per cent in 2017). However, even if the poor seem to be less affected, the amounts paid by households to corrupt civil
servants is non-negligible, as it represents globally around 1 per cent of their food expenses and approximately a third of government transfers to
households through anti-poverty social programmes. Considering the 2018 survey, when the question on the incidence of corruption is posed in
a disaggregated way, institution by institution, corruption incidence appears to be higher with respect to an aggregated question (4.8 per cent vs
2.9 per cent in 2018). The survey module includes a question on why corruption is not reported to the authorities. The reasons are, by importance,
fear of retaliation (25.8 per cent), because authorities don’t care (27.2 per cent) or because of lack of time (23 per cent). Interestingly enough,
around 15 per cent do not report because they obtained benefits by paying bribes.

FIGURE VIII.2 Corruption incidence and poverty in Peru, 2003-2018

8.0
 Total
7.0  Poor
 Non poor
6.0
5.0
4.0
3.0
2.0
1.0
0
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018

Source: Estimates based on Peru’s National Household Survey (ENAHO), INEI 2003-2018.
* Since 2012 the question has been answered by the household head, so interpreting changes between 2011 and 2012 should be cautious.

HANDBOOK ON GOVERNANCE STATISTICS | 161

BOX VIII.4 Best practices/Useful tools, CONT.

Comparing perceptions of corruption and the actual resulting victimization

As discussed in this chapter, indicators on both perceptions and experiences — two elements not necessarily correlated — are important for fully
understanding corruption. Using GPS-SHaSA data from seven countries, the figure compares the perceptions of male and female respondents on
corruption (bar charts) with their actual experiences of corruption in the past 12 months (dots/squares).

FIGURE VIII.3 Perception and experience of corruption in seven African countries

To what 100 100 In the past 12

extent do you months, have
think that 90 90 you had to
corruption 80 80 give money
is a problem or to offer a
in this 70 70 gift to a civil
country? 60 60 servant?

% who 50 50 % users
consider 40 40 victim of
corruption is corruption
often/always 30 30
a problem
20 20
10 10
0 0
Cape Burundi Mali Madagascar Uganda Cameroon Côte
Verde d'Ivoire

 Perception corruption – Male  Perception corruption – Female

 Victim of corruption – Male (right scale)  Victim of corruption – Female (right scale)

Similarly, data from Mexico shows the divergence between what people or businesses hear or suspect, what they know from a third party, and
what they have experienced themselves.

FIGURE VIII.4 Perception and experience of corruption, Mexico

 Perception. Do you believe or have

62% you heard that… public servants or
government employees sometimes
receive money, gifts or favours from
citizens/enterprises to expedite,
47%
approve or avoid procedures?
46%  Acknowledgement from a third party.
Do you remember someone you know
(family, friends or colleagues at work or
school/another enterprise) who has lived
an experience in which a public servant or
government employee asked for money,
24% gifts or favours to make or to expedite,
approve or avoid any of the procedures,
15% services or payments listed above?
 E xperience. Did a public servant, a
6% government employee or someone on their
behalf hint or attempt to appropriate or
directly solicit any benefit (money, gifts or
favours) that you could grant him or her?

Population aged 18+ Business

Sources: National Survey on Regulatory Quality and Government Impact on Enterprises (ENCRIGE, 2016). Available at; https://www.inegi.org.mx/programas/encrige/2016/ And: National Survey of
Quality and Government Impact (ENCIG, 2017), INEGI. Available at: https://www.inegi.org.mx/programas/encig/2017/.

162 |
BOX VIII.4 Best practices/Useful tools, CONT.

Corruption measurement in high-income countries: The case of Italy

In developed countries, NSOs in the past were more reluctant to design surveys or survey modules on corruption. One interesting example is
the case of Italy. For the first time, Istat introduced a series of questions in its 2015-2016 survey on the safety of citizens in order to examine the
phenomenon of corruption. 43,000 individuals between the age of 18 and 80 were interviewed and asked if it had ever been suggested, to them
directly or to a co-habiting member of their household, to give a gift or a favour in exchange for facilitating access to a service or receiving a
concession. It is estimated that 7.9 per cent of households have been directly involved in corruption events during the course of their lives, and 1.2
per cent during the year preceding the survey. In 2017, the French NSO Insee (in collaboration with the statistical department of the Ministry of the
Interior) also embarked in an equivalent but less ambitious project by including a few add-on questions on corruption grafted onto the periodic
victimization survey Cadre de Vie et Sécurité (ECVS). The results were not publicly available at the time of writing of this Handbook.

VIII.3.3 Sample surveys among civil servants

VIII.3.3.1 Description

375. Information on the factors and circumstances facilitating or preventing corruption can also be
collected through sample surveys among civil servants, e.g. on procedures and practices related to
the functioning of public administration (internal functioning, procedures towards the public, provision
of services, subcontracting).

376. Sample surveys among civil servants directly target public officials and frequently focus on
specific groups such as the police, the judiciary or employees within public administration offices.
These surveys aim to collect information on the working conditions of civil servants with the view
to identifying weak practices and vulnerabilities to corrupt behaviour. Information on issues such as
recruitment and promotion practices, job mobility, the frequency of training, work incentives, and
salary and career satisfaction is crucial to elaborate policies and measures to enhance integrity in the
civil service, especially when coupled with information on corruption experiences.

377. Sample surveys among public servants (also called integrity surveys) cannot be used to meas-
ure the prevalence of bribery or other forms of corruption in the public sector. However, such surveys
can produce a range of useful and actionable indicators that allow an evidence-based diagnosis of
integrity challenges in this sector. Key data in this field include indicators on recruitment to the public
service; mobility and promotion within the civil service; transparency measures and complaint mecha-
nisms in place; integrity awareness and training; reporting channels and whistleblowing mechanism in
place; working conditions and job satisfaction; and internal inspection and sanction regimes.

VIII.3.3.2 Methodological and quality issues

378. As with the other sample surveys, relevant methodology is necessary to ensure the reliability
of the data collected (representativeness, adequate concepts and questions, precision as regards the
reference period, etc.). For obvious reasons, civil servants may be more inclined to conceal corruption
than ordinary citizens, thus information on factors affecting the integrity of civil servants might be
more difficult to obtain.

HANDBOOK ON GOVERNANCE STATISTICS | 163

BOX VIII.5 Sample surveys among civil servants: Best Practices/Useful Tools

Corruption among police officers in Mexico

The National Survey of Police Standards and Professional Training, ENECAP, is an official national survey conducted among police officers in Mexico,
specifically within national and local police forces (federal and local agencies with preventive and/or investigative functions). Although ENECAP
is not exclusively about corruption and has no particular module on corruption, it contains relevant indicators on the experience and perception
of corruption throughout its principal sections. Concretely, it examines issues related to corruption such as the prevalence of corruption (victims
per 1,000 police officers), total and per jurisdiction and the prevalence of corruption in the recruitment and promotion process. Regarding internal
corruption, the survey allows a disaggregation per beneficiary (co-worker, superior, other) and asks for the approximate economic cost of the
corruption act (including money, goods, gifts or favours); knowledge and perception of the effectiveness and transparency of internal complaint
mechanisms; and the prevalence of bribery initiated by a citizen.

FIGURE VIII.5 Prevalence of corruption (victims per 1000 police officers), by beneficiary type and by jurisdiction

90 96
71
24 40
5
Total Bribery from Extortion from Bribery from Federal State and
citizens co-workers other inner municipal
or superiors personnel level

Beneficiary Jurisdiction
Source: INEGI. ENECAP, 2017

Corruption and integrity challenges in the public sector of Iraq

A survey conducted in 2011 by the National Statistical Office of Iraq with support from UNODC provides an example of indicators that can be
produced by integrity surveys of civil servants. This survey found that around 4 per cent of the civil servants interviewed stated that they were
offered a bribe at least once in the previous 12 months. The figure below in this Box shows that the frequency of interactions with external actors
can have a significant impact on civil servants’ exposure to corrupt practices. Officials with daily interactions with external entities, especially with
citizens as customers and with private companies as contractors, are more frequently offered bribes. The risk of being offered a bribe decreases for
staff who have weekly or monthly contacts with outside counterparts, or no contacts at all. The integrity survey conducted in Iraq also illustrates
the difficulty of eliciting information on actual engagement in corrupt practices from civil servants. In the survey, only 7 per cent of those stating
that they had been offered a bribe admitted that they had accepted the offer (at least once), which means that only 0.3 per cent of all civil servants
admitted having accepted a bribe in the previous 12 months. At the same time, the survey also illustrates the value of integrity surveys for
assessing areas of vulnerability and priority for integrity policies that are not directly linked to acts of bribery. For example, over 20 per cent of civil
servants in Iraq admitted that they had received some help from family members, friends or their own political party in their own recruitment.

FIGURE VIII.6 Civil servants’ exposure to corrupt practices, by frequency of interaction

5%
 Citizens as customers
 Private enterprises as contractors
4%  Units of the same institution
 Private enterprises as customers
 Units of other institutions
3%

2%

1%

0%
Daily contacts Weekly contacts Monthly contacts (Almost) no contacts

Source: UNODC, Corruption and Integrity challenges in the public sector of Iraq. An evidence-based study, 2012, https://www.unodc.org/documents/publications/2013_Report_
on_Corruption_and_Integrity_Iraq.pdf

164 |
 379. As with other forms of sample surveys, adherence to strict quality standards in the conduct of
the survey is necessary to guarantee the quality of the data produced. In the case of integrity surveys,
special consideration should be given to protecting the anonymity of the respondents’ answers. The
confidentiality of the responses can be enhanced — and thus the willingness of civil servants to disclose
sensitive information increased — through the careful choice of the interview mode. For example,
self-administered questionnaires (that can be filled in and submitted anonymously by the respondents)
can safeguard anonymity and should work well as most respondents are likely to be literate.

VIII.3.4 Composite indices

VIII.3.4.1 Description

380. The rise in prominence of governance issues has been accompanied by an increase in the
number of indicators aiming to provide quantitative measures of various complex concepts of govern-
ance, and in particular global composite indicators on corruption.14 Regardless of the methods used,
these indicators are characterized by being mainly (but not exclusively) based on experts’ perceptions.

TABLE VIII.2 Examples of international composite indices on corruption

Indicators/Databases Institutions Sources Information

Control of Corruption, World Governance Indicators World Bank Institute Expert/HH/Firm Perception

Corruption Perceptions Index Transparency International Expert/HH/Firm Perception

International Country Risk Guide Political Risk Services Group Expert Perception

Index of Economic Freedom Heritage Foundation Expert Perception

Corruption Index Economic Intelligence Unit Expert Perception

Country Policy and Institutional Assessment World Bank Expert Perception

Bribe Payers Index Transparency International Firm Perception

Executive Opinion Survey World Economic Forum Firm Perception

World Business Environment Survey World Bank Firm Perception

Sources: Razafindrakoto and Roubaud (2010); Sudders and Nahem (2004).

381. The two main producers of benchmark corruption indicators are currently the NGO
Transparency International (Corruption Perception Index, CPI) and the World Bank (Control of
Corruption Index, CCI). In both cases, metadata or “indicators of indicators” are used to compile
and aggregate the different primary sources into composite indicators. The CPI was developed by
Transparency International in 1995 and is probably the oldest and most well-known corruption index.
As its name suggests, the CPI is based on perceptions of corruption by experts (businesspeople and
country risk analysts), both residents and non-residents, nationals and expatriates. The CCI is one of
the six indicators of the Worldwide Governance Indicators (WGI). This database draws on 33 individual
sources from 30 different institutions. The CCI is also a composite index of perceptions of corruption,
combining different sources and dominated by the opinions of experts and businessmen. Other types
of perceptions, including by households, are also taken into consideration.

14 A UNDP report talks about the “mushrooming industry of indicators” (2010).

HANDBOOK ON GOVERNANCE STATISTICS | 165

 BOX VIII.6 T
 he “mirror survey”: A comparison between people’s
experiences and expert opinions

The corruption questions attached to regular household surveys between 2001 and 2003 in eight West African
capitals were compared with the responses of some 250 specialists to the same questions as those contained
in the household surveys. A comparison between the data obtained from the two sources showed that experts
consistently overestimated the level of corruption experienced by citizens and had a much more negative view
of the actual situation than the general population. Such a significant overestimation would constitute a minor
problem if it were uniform; however, strong discrepancies in the relative classification of countries shows that
this is far from the case. Therefore, it can be concluded that experts do not have a good appreciation of the actual
level of corruption. Having said that, these results do not necessarily negate the relevance of these indicators,
but it is essential to combine them with a new generation of indicators based on objective measurements in
order to assess occurrences of corruption in all their complexity.

Source: Razafindrakoto & Roubaud (2010): Are International Databases on Corruption reliable? A comparison of Expert Opinion Surveys and Household
Surveys in Sub-Saharan Africa’, World Development, 38 (8).

VIII.3.4.2 Methodological and data issues

382. A number of studies have examined the pertinence and validity of composite indicators on
corruption in international databases. Different shortcomings have been singled out by a certain num-
ber of critiques (Arndt and Oman, 2006; Arndt, 2009; Razafindrakoto and Roubaud, 2010). In particular,
as recalled in the 2018 Manual on Corruption Surveys, “assessments based on indirect methods have
major weaknesses in relation to their validity and relevance. The construction of expert assessments
and composite indicator metrics is based on a number of subjective assumptions, such as the selec-
tion of variables or sources and the determination of the algorithm used to combine heterogeneous
data. Furthermore, indirect methods cannot be used to produce the disaggregated data or detailed
information on corruption needed for policy-making purposes” (UNODC, UNODC-INEGI Centre of
Excellence, UNDP).

383. Most of the time, indicators in international databases do not really permit comparison over
time. The indicators provide no way of assessing governance trends at worldwide level, and only
allow for relative positions. They provide no indication of the policies that might reduce corruption.
Moreover, the aggregation procedure and the choice of the different primary sources used to put
together the indicators are often questionable. Last but not least, it is difficult for any expert to have
an objective appreciation of the actual level of corruption.

384. That said, an obvious advantage of these global indices is that they facilitate inter-country
and large-scale international comparisons, thus providing international users (e.g. investors, donors
and other bodies) with indicators on governance that they believe fulfil their needs. Those users do
not usually require indicators that reflect context-specific features. Another important concern with
this approach is that these “top-down” tools are relatively unsuited for effectively monitoring and
evaluating national and local policies or strategies aimed at fighting corruption.

VIII.3.5 Other data sources

385. In addition to the administrative data sources, surveys and composite indices discussed in
detail above, other “indirect” measurement tools may have some secondary relevance in better un-
derstanding corruption. However, these tools can capture only presumptions of corruption and not
corruption itself.

166 |
 386. Public Expenditure Tracking Surveys (PETS), part of a vast programme launched by the World
Bank in the late 1990s (Reinika and Svenson, 2006), are designed to track and quantify the flow of re-
sources through the various layers of government bureaucracy, down to the service facilities, in order to
determine how much of the originally allocated resources reach each level (and in particular the public
services at the end of the line that are supposed to receive the funds). The initial application of this meth-
od in Uganda showed that the State schools received less than 20 per cent of the allocation (excluding
wages) that should have come to them, since the money was diverted en route for “pork barrel” projects.

387. Quantitative Service Delivery Surveys (QSDS) form a second approach. They collect detailed
information on budgets, prices, inputs, products, service quality and operating modes to assess the
economic efficiency of the institutions. For example, they can be used to quantify and qualify the
phenomenon of absenteeism or “ghost” jobs in the services studied.

VIII.4 Recommended key indicators

388. Based on the insights from the previous section, a list of recommended key indicators that all
countries should aim to develop in the short or medium term is provided below.

TABLE VIII.3 Recommended key indicators

Sub-dimension Indicator Data source Type

Corrupt practices Proportion of persons who had at least one contact with a public official Surveys of households Outcome
and who paid a bribe to a public official, or were asked for a bribe by those
public officials, during the previous 12 months (SDG indicator 16.5.1)

Corrupt practices Proportion of businesses that had at least one contact with a public official Surveys of businesses Outcome
and that paid a bribe to a public official, or were asked for a bribe by those
public officials, during the previous 12 months (SDG indicator 16.5.2)

Corrupt practices Sectors (or type of transactions) in which bribes were paid Surveys of households Outcome
and businesses

Corrupt practices Perceptions on the level of corruption in different sectors Surveys of households Outcome
(police, justice, health, education, etc.) and businesses

Corrupt practices Percentage of citizens being exposed to vote-buying in recent national elections Surveys of households Outcome

State response to Reporting rate of bribery experiences Surveys of households Outcome/Process

corruption/ Integrity and businesses

State response to Ratification status by the country of the United Expert assessment Structural
corruption/ Intolerance Nations Convention against Corruption
to corruption

State response to corruption Existence of an effective anti-corruption body Surveys of households Structural
and businesses,
expert assessment

Intolerance to corruption Percentage of civil servants being recruited or promoted Surveys of households/ Structural/Process
through public, transparent and merit-based procedures Admin data

Corrupt practices/ State Reported cases of corruption; cases and persons referred Administrative data Process
response to corruption to court; persons convicted of corruption offences

HANDBOOK ON GOVERNANCE STATISTICS | 167

 VIII.5 Way forward
389. As shown in this chapter, a certain number of countries have already begun to collect data
that can be useful to monitor corruption. Reliable and relevant best-practice cases exist in the area of
household surveys. However, despite this progress much still remains to be done. There is sometimes
reluctance and scepticism regarding the legitimacy and the capacity of public institutions (in particular,
NSOs) to collect data on corruption, while further standardization and improvement of methodologies
can be promoted on the basis of their valuable experiences. The following key areas of development
are therefore identified:

1) Institutionalization of the monitoring of corruption at the national level. This implies that in-
stitutional arrangements need to be defined: For example, a dedicated unit or team with
expertise in corruption measurement, located at (or closely linked to) the NSO, should be put
in place, and the different services or institutions that can collect or provide data on the phe-
nomenon need coordination. This includes putting in place harmonized definitions, formats
and schedules between institutions and along the “data production chain” in a given sector,
in order to obtain compatible and consistent data over time.
2) Standardized household or enterprise surveys or survey modules (core set of standardized
questions) on the experience of corruption need to be integrated into the statistical pro-
gramme of NSOs to allow the regular monitoring of corruption trends and the production of
data for SDG indicators 16.5.1 and 16.5.2. The UNOCD-UNDP Manual on Corruption Surveys
provides adequate guidance to conduct such surveys.
3) Further research on methodological issues, and development of new tools:
a) New survey tools to cover additional aspects of corruption experience beyond bribery, for
example, nepotism or electoral corruption need to be identified and developed.
b) Methodologies for mapping the risks of exposure to corruption in the public administration
and the private sectors need to be identified.
c) The link between specific values and norms (sub-dimensions of integrity) and vulnerability
to corruption is still an avenue of research, in particular to identify indicators which allow to
monitor the “intolerance to corruption” sub-dimension.
d) Reliable and relevant indicators to monitor grand corruption need to be identified. Despite
its critical importance, methodology to measure this form of corruption is crucially lacking.
e) Exploration is needed of “new” sources of data, including big data, social platforms and
social media data.
4) Systematic assessments of the validity, reliability, relevance, and consistency over time and
across countries of corruption indicators should be planned.

168 |
HANDBOOK ON GOVERNANCE STATISTICS | 169
 PART B
EIGHT DIMENSIONS OF
GOVERNANCE STATISTICS

CHAPTER IX
Trust
 IX.1 Conceptualizing this dimension
IX.1.1 What is trust?
390. Trust can be conceptualized as “a person’s belief that another person or institution will act consist-
ently with her/his expectations of positive behaviour” (OECD, 2017a). This definition builds on approaches
from different disciplines (political science, sociology, economics, psychology), capturing both behavioural
and attitudinal aspects. It reflects the view that, in any type of interaction, a trusting person consciously
places resources at the disposal of another party without the means to guarantee that these resources
will be well used or returned (Fehr, 2009). Beyond an individual’s observable behaviour, this definition also
recognizes cognitive and normative aspects of trust: Trust is influenced by a person’s expectations about
other people’s trustworthiness and about whether others (including strangers) and institutions share and
embody the same fundamental values (Hardin, 2004; Uslaner, 2002).

391. Trust is one of the key aspects of the wider notion of “social capital”, a concept broadly under-
stood as encompassing the set of shared norms and values that facilitate co-operation and collective
actions within and between groups. While this concept is used in research with different connotations,
the term “capital” conveys the idea that these co-operative relations are crucial for improving various
aspects of people’s life (from health to skills, political voice and more), that it represents a stock affected
by current decisions and behaviours, and that this stock should be preserved and enhanced to ensure
the sustainability of societal well-being over time. Other aspects of social capital beyond trust include
personal relations and social networks, the types of support that these networks provide, and civic
engagement activities through which people contribute to community life (Scrivens and Smith, 2013).

392. Because of its nature as a belief, trust belongs to the domain of “outcomes” (rather than “pro-
cesses” or “structures”) of this Handbook’s cross-cutting indicator categorization. Unlike in most other
chapters in this Handbook, where dimensions are defined starting from individual sub-components,
trust is inherently a single concept, although it can be measured at the level of different institutions or
groups of people. Trust (and other “outcome” concepts described in this Handbook, such as “political
efficacy”) is unique, as it reflects countries’ performances in other government dimensions (e.g. human
rights, the rule of law, integrity). It thus lends itself for use as a “thermometer” or summary indicator of
broader governance performance.

393. The OECD Guidelines on Measuring Trust distinguish between people’s trust in institutions
(i.e. institutional trust) and in other people (i.e. interpersonal trust). These notions can be further bro-
ken down into specific types of institutions or groups of people:

• Trust in institutions refers to people’s trust in different types of public and private institutions.
The focus of this chapter is on trust in the public institutions of each country (rather than in
private or international organizations). These public institutions encompass specific bodies
such as the parliament, the national government, the civil service or the justice system.
• Trust in other people refers both to people who are not known to the respondent (gener-
alized trust) and to persons known to the respondent, such as family members, friends and
neighbours (limited trust).

394. The theoretical literature generally distinguishes between two main aspects of institutional
trust: “trust in competence”, i.e. in the skills and knowledge of administrative staff; and “trust in inten-
tions”, i.e. in the honesty and integrity of public officials or in the legitimacy of those taking decisions
(Nooteboom, 2007; OECD, 2017e). Recent experimental research has distinguished these two aspects

172 |
empirically. Trust in institutions strongly correlates with perceptions of institutional performance, espe-
cially with people’s views of government integrity, openness and participation, effectiveness, and with
their satisfaction with public services (Murtin and others, 2018), which link to themes covered in other
chapters of this Handbook.

IX.1.2 Can trust be measured?

395. While measuring an inherently intangible concept such as trust is challenging, there is now
sufficient evidence to conclude that questions on trust produce valid results and to make the case that
they should be measured in the official household surveys undertaken by national statistical offices
(OECD, 2017a). The evidence base on the validity of survey questions on trust in institutions is some-
what less developed than for measures of interpersonal trust, hence extending data availability in this
space is especially important.

396. This chapter builds on key insights of the 2017 OECD Guidelines on Measuring Trust, which
was prepared — under the guidance of an advisory group of experts from statistical offices, research-
ers and policy makers1 — in response to a demand from OECD Ministers to improve the quality of
existing trust metrics. These Guidelines reflect comments provided by delegates from both the OECD
Committee on Statistics and Statistical Policy (CSSP) and the OECD Public Governance Committee.
As such, these Guidelines represent the natural reference point for the international statistical com-
munity in this field. While this process has been focused mainly on OECD countries, CSSP gathers
the Chief Statisticians from both OECD and a number of “partner countries”. Also, the data used to
assess statistical quality of trust measures covers a significant number of non-OECD countries.2

397. Recent national and international initiatives, in addition to the efforts of the UN Praia City
Group, have underscored the importance of measuring trust. They are collected by several OECD
statistical offices in their general social surveys, and are already regularly published by the OECD in
its publications How’s Life? (OECD, 2017e) and Governance at a Glance (OECD, 2017c). They are also
collected by several African countries in the context of the Strategy for the Harmonization of Statistics
in Africa (SHaSA), a regional initiative – supported by UNDP, EU, DIAL/IRD and the African Union – to
inform the African Union's Charter on Democracy, Elections and Governance (Razafindrakoto and
Roubaud, 2015; African Union, 2019). The Inter-American Development Bank (IADB) recently decided
to prioritize research on trust and social capital among citizens and with governments in the Latin
American and Caribbean region (IDB, 2019). Moreover, trust is also the focus of a 2018 Eurofund
report on societal change in Europe (Boda and others, 2018), and it was one of the topics singled out
in an in-depth review of governance statistics in the UNECE region (UNECE, 2016). Finally, improving
the measurement of trust through a combination of surveys and experimental tools is one of the 12
recommendations of the High-Level Expert Group on the Measurement of Economic Performance
and Social Progress3 (Stiglitz and others, 2018).

1 Members of the OECD advisory group included Adrian Franco, INEGI, Mexico; Sophie Pontieux, INSEE, France; Dawn Snape and Veronique Siegler,
ONS, UK; Fiona Carnes and Joanne Baker, ABS, Australia; Prof. Soonhee Kim and Prof. Dong-Young Kim, Korea Development Institute; Prof. Jacob S.
Hacker, Yale University; Prof. Yann Algan, Paris Institute of Political Studies; Dr. Monica Ferrin, University of Turin; Jacob Saeger, UK Cabinet Strategy Office;
Prof. Marc Hetherington, Vanderbilt University; and Prof. Eugene Kandel, Hebrew University of Jerusalem and former Chairman of the Israeli National
Economic Council.
2 Data on trust were sourced from the Gallup Word Poll, the World Values Survey and Latinobarometer, all of which extend beyond OECD members. See
section IX.3.2 on data and the best practices currently available for further detail.
3 The High-Level Expert Group on the Measurement of Economic Performance and Social Progress, attached to the OECD, was established to follow up
on the recommendations of the Commission on the Measurement of Economic Performance and Social Progress (also known as the Stiglitz-Sen-Fitoussi
Commission) and to provide impetus and guidance to the various initiatives currently ongoing on measuring people’s well-being and societies’ progress.

HANDBOOK ON GOVERNANCE STATISTICS | 173

 IX.1.3 Is trust policy-amendable?
398. A critical question for the application of trust measures in practical policy-making is whether
people’s level of trust is fixed or changes over time and, by extension, whether it is influenced by
specific government policies and by how institutions function.

399. Does a person’s environment influence her trust over the life course? Or is trust an engrained
cultural value of a country or region that is inherited through socialization from one generation to the
next, i.e. a stable psychological trait? This latter notion of “moral trust” (as opposed to the former “ra-
tional trust”) suggests that trust is a belief about how a person should behave towards others rather
than a belief about how others are likely to behave towards you. The distinction between “rational”
and “moral” trust is important, as the room for policy intervention is smaller and more long-term in the
first case, and larger and more immediate in the second (Algan, 2018).

400. Both views on how trust is formed have elements of truth, as illustrated by Putnam’s seminal
work on the evolution of social capital. Making Democracy Work (1993) contrasted the high interper-
sonal trust prevailing in the regions of northern Italy (whose cities experienced democratic self-rule
in the medieval period) with those in southern Italy (ruled by a succession of foreign powers), arguing
that current trust levels in Italy’s regions are largely determined by this historical legacy. Conversely,
Bowling Alone (2000) documented the sharp decline of various measures of social capital (such as
survey measures of interpersonal trust and membership in voluntary associations) in the United States
over the past 50 years. Putnam attributed this fall in trust to the demise of the more public-minded
generation that experienced the Great Depression and the Second World War, as well as to individual-
ization of leisure activities and watching television more. In this second narrative, trust is more variable
and influenced by changes in one’s environment.

401. For institutional trust, concrete aspects of institutional performance seem to influence how
much people trust their governments. For example, people’s views of government integrity, openness
and participation, effectiveness, and satisfaction with public services all strongly correlate with trust in
institutions (Figure IX.1), although broader societal aspects (such as neighbourhood connectedness or
people’s financial security) are also important.

IX.2 Why is this dimension important?

402. Both trust in institutions and trust in other people are essential for good governance, econom-
ic growth and societal cohesion (see Algan and Cahuc, 2014, and Algan, 2018, for a detailed review).

403. Early perspectives on economic development stress the role of technological progress and the
accumulation of physical and human capital. Since these fail to explain a large share of the cross-coun-
try differences in GDP per capita, development narratives have progressively shifted to the role of for-
mal and informal institutions. The laws, rights and regulations enforced by official authorities, as well
as the (unwritten) social norms and traditions that shape how people think and behave, can support
or weaken market incentives to accumulate wealth and to innovate (North, 1990; Stiglitz and Arnott,
1991; Acemoglu, Robinson and Johnson, 2001; World Development Report, 2002).

404. In this perspective, institutional trust is both a prerequisite for and a result of institutional
quality and political participation. Institutional trust correlates with objective measures of bureaucratic

174 |
FIGURE IX.1 Policy determinants of trust in government in selected OECD countries

0.45
0.40
0.35
0.30
Per cent

0.25
0.20
0.15
0.10
0.05
0.00
y

ss

it y

es

ion

it y

lot
es

es

es
fit
rit

te
ne

s te

v ic
bil

ur
en

rn

dn

at

ra
ne
ys
eg

en

ec
er
sy

lia

gr
f ai
es

si v

ee
be

te
int

op

ys

ls
mi
ion

re

ec

nt
ar

on

nt
re

c ia
nt

rit
nt

im
nn
hc

nt

lu
me
lf a

at
sp
me

an
cu
me

Vo
me

co
uc
alt

to
re

we

rn

se

Fin
rn

ed
rn

ld
rn
he

ss
nt

ve
ve

ith

ith

oo
ve

ne
ve
me

ith

Go
ith
Go

nw
Go

nw

rh
Go

en
nw
rn
t. w

bo

Op
ve

tio

tio
tio

igh
Sa

Go

f ac

f ac
f ac

Ne
tis

tis
tis
Sa

Sa
Sa

Note: The figure shows the most significant determinants of self-reported trust in government in an ordinary least squares estimation that controls
for individual characteristics. All variables depicted are significant at the p < 0.01 level. Countries covered are France, Italy, Germany Slovenia, South
Korea, and the United States.
Source: Murtin, F. and others (2018), "Trust and its determinants: Evidence from the Trustlab experiment", OECD Statistics Working Papers, No.
2018/02, OECD Publishing, Paris, https://doi.org/10.1787/869ef2ec-en.

efficiency, foreign direct investment and perceptions of government corruption4 (Knack and Keefer,
1997; Razafindrakoto and Roubaud, 2007; Zhao and Kim, 2011; OECD, 2013). It also shapes people’s
willingness to cooperate with others, to comply voluntarily with regulations (such as taxes or environ-
mental codes) and to support reforms that might pay off only in the long term (Murphy and others,
2009; Daude and others, 2012; OECD, 2016; Boda and others, 2018). Trust in institutions hence sets
the policy options that governments can use to attain their goals and deliver public goods.

405. Trust is also of intrinsic value for people’s quality of life. Without trust in institutions such as
the parliament, the civil service, the justice system and the police, it is impossible for a community
to perform effectively or for people within it to live the sort of lives that they wish to pursue. Trust
in institutions is directly related to how satisfied citizens are with their lives: for example, it has been
shown that “procedural utility” (i.e. the process through which people are involved in making im-
portant collective decisions) matters for people’s subjective well-being, independently of the actual
outcome of the decision (Frey, Benz, and Stutzer, 2004; Frey and Stutzer, 2005). Trust in institutions
requires institutions that are perceived as competent and effective in achieving their goals and that
operate consistently with a set of values that reflect people’s expectations of how institutions should
function and treat citizens.

4 Most empirical studies in this field are based on correlations, whose interpretation is complicated by the possibility of reverse causality, i.e. people are
less likely to trust inept or corrupt institutions. Further empirical research is needed to establish a causal link between trust in institutions and economic
progress.

HANDBOOK ON GOVERNANCE STATISTICS | 175

 406. Interpersonal trust, one of the best available proxies of social capital,5 also matters for de-
velopment outcomes. Arrow (1972) linked trust in strangers to economic growth based on the argu-
ment that “virtually every commercial transaction has within itself an element of trust, certainly any
transaction conducted over a period of time. It can be plausibly argued that much of the economic
backwardness in the world can be explained by the lack of mutual confidence” (see also Putnam,
1993; Temple, 2000; Bouckaert, 2012). Trust in others is also correlated with measures of a population’s
health status or health-related behaviours (Lochner and others, 2003; Lindström, 2005; Brown and
others, 2006), crime rates (Buonanno and others, 2009) and subjective well-being (Helliwell and Wang,
2010, Boarini and others, 2012).

407. Interpersonal trust is conceptually distinct from institutional trust, but is nonetheless an impor-
tant indicator of the quality of governance. Trust in others and trust in institutions are linked through
complex “feedback loops”: Only when public institutions are performing fairly and competently will
people extend trust to strangers without putting themselves at risk (Sønderskov and Dinesen, 2016). A
causal pathway also runs in the opposite direction: Public institutions are likely to be fairer and more
effective in societies where civil servants believe that other officials and citizens are trustworthy and
where the risk of being caught in such behaviour is higher (Farrell and Knight, 2003).

IX.3 Data and best practices currently available

IX.3.1 Data sources
408. Statistics on trust are available from both official and non-official sources, with most of these
measures elicited through household surveys. More recently, quasi-experimental measures of trust
have started to be produced as well (e.g. the OECD Trustlab platform described in Box IX.1). Overall,
regular, timely and consistent measurement at the international level needs to be improved. In the
European context, for example, comparable official statistics on trust were collected only through ad
hoc modules to one of the main community surveys (the 2013 and 2018 well-being module of the EU
Statistics on Income and Living Conditions, EU-SILC6).

409. At the national level, collection of trust data has been somewhat more frequent. Since 2010,
some European countries have implemented national well-being versions of EU-SILC (e.g. France),
and many recurrent general social surveys include trust questions (such as Poland’s Social Cohesion
Survey, the Australian, Canadian and New Zealand General Social Survey, or the Social Cohesion and
Well-being Survey of the Netherlands7). The national statistical offices of Ecuador, Chile, Colombia,
Mexico and Peru collect official data on institutional trust, and some African and Andean NSOs al-
ready introduced such questions in the early 2000s (Herrera and others, 2007). Since 2012, several
African countries have included institutional trust questions in their harmonized surveys modules on
“Governance, Peace and Security” conducted in the context of the Strategy for the Harmonization of
Statistics in Africa (SHaSA).

5 Social capital broadly refers to the networks, social norms, trust and values that foster cooperation within or among different groups in society (Scrivens
and Smith, 2013).
6 EU-SILC is input harmonized across countries and includes recommendations on how to address trust survey questions. However, national implementa-
tion might vary to a slight degree (e.g. translations are not regulated and question order may vary).
7 Statistics Canada first introduced trust questions in its 2002 Ethnic Diversity Survey.

176 |
TABLE IX.1 Overview of surveys in the OECD Trust Database

Survey Inception Frequency Number of countries in the database Time-period covered

Gallup World Poll (GWP) 2006 Yearly 115 2006-18

World Values Survey (WVS) 1981 Every 5 years Wave 4 (6) Wave 4 (1999-2004)
Wave 5 (46) Wave 5 (2005-09)
Wave 6 (45) Wave 6 (2010-14)

European Social Survey (ESS) 2002 Every 2 years Round 1 (22) Round 1 (2002)
Round 2 (25) Round 2 (2004)
Round 3 (23) Round 3 (2006)
Round 4 (28) Round 4 (2008)
Round 5 (27) Round 5 (2010)
Round 6 (29) Round 6 (2012)

European Quality of Life Survey (EQLS) 2003 Every 3 years Round 2 (31) Round 2 (2007/08)
Round 3 (34) Round 3 (2011/12)

Eurobarometer 1973 Yearly 34 2003-15

Latinobarometer 1995 Yearly 19 2002-16

EU-SILC 2003 2013 ad-hoc module 33 2013

Source: OECD Guidelines on Measuring Trust, OECD Publishing, Paris (2017), https://doi.org/10.1787/9789264278219-en.

BOX IX.1 Trustlab: Measuring trust and social norms through experimental techniques

Trustlab is an innovative OECD initiative to improve existing measures of trust and to understand its drivers and how policymakers might go
about restoring it. Trustlab combines cutting-edge techniques drawn from behavioural science and experimental economics with an extensive
questionnaire on the policy and contextual determinants of trust in others and trust in institutions within an integrated online platform.

Following a pilot phase in 2016, Trustlab has now been implemented in France, Germany, Italy, Korea, Slovenia, the United Kingdom and the
United States, with academic and governmental partners joining the effort. In each country, a representative sample of 1,000 people participated.
The combination of survey and experimental data allows assessing the convergent validity of self-reported trust. For trust in institutions, this is
done by comparing individuals’ responses to trust questions to their scores from an Implicit Association Test (IAT). IAT is a technique widely used
in experimental psychology that relies on the time taken by participants to relate a concept (e.g. “government”) with an attitude or evaluation
(e.g. “competent”), which allows determining whether automatic positive associations between the two are made more easily than negative
associations (Greenwald, McGhee and Schwartz, 1998). For generalized trust, Trustlab relies on measures of how participants behave in a range of
games drawn from experimental psychology, comparing these measures to those based on people’s self-reports.

Modules featuring in Trustlab

Module Focus Technique

1. Behavioural games (trust game, Generalized trust Experimental

public goods game, dictator game)

2. Implicit Association Tests Trust in institutions Quasi-experimental

Generalized trust

3. Survey and demographic module Trust in institutions Traditional self-reported survey questions

Drivers of trust

Empirical analysis of Trustlab data (see Figure IX.1) shows that people’s trust in institutions is driven mainly by institutional aspects specific to each
country (such as people’s perceptions of government integrity, responsiveness and reliability, and their satisfaction for a range of services provided
by public institutions) as well as by societal factors (such as people’s social ties in the neighbourhood, economic conditions and volunteering)
(Murtin and others, 2018).

For more information, visit: http://www.oecd.org/sdd/Trustlab.htm

HANDBOOK ON GOVERNANCE STATISTICS | 177

 410. To start working with some data, internationally comparable data on trust can be drawn from
unofficial surveys. The most important of these are the Gallup World Poll (for trust in institutions only),
the World Values Survey, the Barometers carried out in many regions around the world, as well as the
European Social Survey and the European Quality of Life Survey. Based on these surveys, the OECD
has compiled data on people’s trust in the OECD Trust Database (Table IX.1).8 This database goes
beyond OECD members, covering 124 countries worldwide, and provides a suitable base to analyse
the statistical quality of trust measures for this Handbook.

411. Finally, experimental data on trust is available from the OECD Trustlab initiative (see Box IX.1).

IX.3.2 Best practice for specifying survey questions on trust

412. Generally, survey questions simply ask respondents to evaluate their level of trust in a person
or institution. For example, EU-SILC asks, “How much do you trust the political system/ the legal
system/ the police in [country]? Please answer on a scale of 0 to 10, where 0 means do not trust at all
and 10 means completely trust”; the New Zealand General Social Survey asks “a general question
about trust (0 = you do not trust at all, 10 = you have complete trust). In general, how much do you
trust most people in New Zealand?”

413. While there are commonalities across surveys, questions differ in their wording (e.g. some ask
about “confidence” in institutions rather than “trust”), response scale (e.g. some use 0-10 numeri-
cal scales, as in the examples above, others use 5-point scales, and still others use verbal response
categories such as “a great deal of confidence, some confidence, not very much confidence, no
confidence at all”), and the specific types of institutions and people that trust is placed in. Annex IX,
ANX.IX.1 and ANX.IX.2 provide an overview of selected trust questions included in international and
national surveys, showing that there is ample scope for international harmonization.

414. In addition to evaluative questions, trust questions sometimes also refer to expectations as to
what would or will happen in a given situation, or to respondents’ past experiences and behaviours.
Probably the most famous example of the former is the “wallet question” used by the Gallup World
Poll to elicit the expected trustworthiness of a third party (“In the city or area where you live, imagine
that you lost your wallet… or something holding your identification or address… and it was found by
someone else. Do you think your wallet (or your valuables) would be returned to you if it were found
by a stranger/ a neighbour/ the police?”). Examples of questions on people’s past trusting behaviour
are lending personal possessions, lending money, leaving the door unlocked, or voicing opinions to
public officials.

415. The OECD Guidelines recommend that, to allow cross-country comparisons, statistical of-
fices should include a core set of trust questions in their surveys for which evidence on validity and
relevance is the strongest and which take about 90 seconds to complete (Box IX.2). These questions
directly translate into indicators on trust (such as the mean or median level of trust in a community, or
the share of respondents reporting trust above or below a given threshold).

416. The first two questions in Box IX.2 are about trust in others, distinguishing between “people”
in general (A1) and “people you know personally” (A2). The second set of three questions pertain to
trust in public institutions, distinguishing between the national parliament of each country, as the key
political institution (A3), the police (A4) and the civil service (A5).

8 This database was extended and updated in early 2019 to include the newest waves of Latinobarometer and of the Gallup World Poll.

178 |
 417. The single question on generalized trust (A1), focusing on the aspect of trust with the widest
general use and strongest validity, captures the most important information on generalized trust with
the minimal burden on respondents. The OECD envisaged that question A1 will serve as the primary
measure of trust when limited resources allow for only a single question in a household survey.9 The
three questions on institutional trust (A3 to A5) capture two of the main components of public institu-
tions, namely the political system and the justice system, as identified through the empirical analysis
summarized by Table IX.2. Question A5 aims to establish whether respondents trust the (non-political)
civil service differently than political institutions. The order of the three institutions should be rand-
omized across the survey to minimize order effects.

BOX IX.2 Survey questions recommended by the OECD as a “core module” on trust

A1. And now a general question about trust. On a scale from zero to ten, where zero is not at all and ten is completely,
in general how much do you trust most people?

Not at all Completely DK

00 01 02 03 04 05 06 07 08 09 10 88

A2. On a scale from zero to ten, where zero is not at all and ten is completely, in general how much do you trust most
people you know personally?

Not at all Completely DK

00 01 02 03 04 05 06 07 08 09 10 88

The next questions are about whether you have trust in various institutions in [COUNTRY]. Even if you have had very
little or no contact with these institutions, please base your answer on your general impression of these institutions.
Using this card, please tell me on a score of 0 10 how much you personally trust each of the institutions I read out. 0
means you do not trust an institution at all, and 10 means you have complete trust. Firstly…READ OUT

A3. [COUNTRY’S] Parliament?

Not at all Completely DK

00 01 02 03 04 05 06 07 08 09 10 88

A4. The police?

Not at all Completely DK

00 01 02 03 04 05 06 07 08 09 10 88

A5. The civil service? 1

Not at all Completely DK
00 01 02 03 04 05 06 07 08 09 10 88

1. The term “civil service” is used in the OECD Guidelines to refer to government employees outside the political sphere.
Source: OECD Guidelines on Measuring Trust, OECD Publishing, Paris. https://doi.org/10.1787/9789264278219-en.

9 The primary question recommended by the OECD is based on the generalized trust question from the New Zealand General Social Survey in 2014.
This, in turn, is derived from the widely used Rosenberg question on generalized trust (Rosenberg, 1957) but focuses on trust rather than on caution when
dealing with people, and is amended to use a 0 10 scale with end labels and with neutral question wording at both ends of the scale.

HANDBOOK ON GOVERNANCE STATISTICS | 179

 418. This core set of questions should be implemented as is and in its entirety, but could be com-
plemented with supplementary questions when policy demand is strong. Possible extensions, for
which additional survey modules are provided in the OECD Guidelines, can be grouped under the
three main categories mentioned earlier:

• Evaluative questions can be extended to a broader range of public institutions — such as the
courts, political parties, politicians, the police and armed forces — but also to private ones
such as the media, banks or major companies.
• Respondents can be asked about their expectations as to what could happen in a given sit-
uation, focusing directly on the conduct expected of a third party such as neighbours, police
officers or strangers, hence providing a measure of the trustworthiness of a given institution
or group of people.10
• Finally, questions could be asked to collect information based on the respondent’s experienc-
es and behaviours. This implies confronting respondents with questions related to situations
that they may have experienced in the past and that are typically associated with a trusting
behaviour. In the case of institutional trust, these questions relate to behaviours that reflect
confidence in that specific institution.11

IX.3.3 The state of trust

419. There are large differences in levels of interpersonal trust across countries. In Norway, the
country with the highest level of trust, more than 68 per cent of the population report trusting others
(Figure IX.2; OECD, 2018). At the opposite end of the ranking lie Trinidad and Tobago, where only
4 per cent of the population report high levels of interpersonal trust. In general, northern European
countries lead the ranking, while people in African and South American countries report much lower
levels of trust in others. The extent to which people trust others varies not only across nations, but also
across regions in the same country, as shown by Algan and Cahuc (2014) for regions across European
countries, the United States and several other countries.

420. Similarly, trust in national government (in both levels and change between 2006 and 2017)
varies markedly across OECD countries and beyond (Figure IX.3), ranging from more than 80 per
cent in Switzerland to below 20 per cent in Greece. Up until 2017, trust in government has declined
in a number of OECD countries, in particular among those most affected by the 2009 financial crisis,
though most recent data show that confidence in institutions has now rebounded to pre-crisis levels
(OECD, forthcoming-b).

421. There is also (more limited) evidence of differences in trust in institutions within countries,
across people with different characteristics. For example, the GPS-SHaSA modules conducted in nine
African countries between 2013 and 2016 (Benin, Cameroon, Cape Verde, Cote d’Ivoire, Madagascar,
Malawi, Mali and Uganda) show that, while there are almost no differences in institutional trust be-
tween different age groups or between men and women, people living in urban areas are systemati-
cally more distrustful of their administration than are their rural counterparts (Figure IX.4). However, the
gap varies substantially across countries (from 17 and 12 percentage points in Mali and Cote d’Ivoire
to insignificant differences in Burundi and Malawi). Education, one aspect of people’s socio-economic

10 For example: “If you were to complain about bad quality of a public service, how likely is that the problem would be easily resolved?”; “If a natural
disaster occurs, do you think that the provision by government of adequate food, shelter and clothing will be timely and efficient?”; “If a decision affecting
your community were to be taken by the local or regional government, how likely is it that you and others in the community would have an opportunity
to voice your concerns?”, etc.
11 For example, “Have you done any of the following in the past month? How about: Voiced your opinion to a public official? or Signed a petition?”

180 |
 FIGURES

Figure 1: World distribution of Trust

FIGURE IX.2 Trust in people you meet for the first time, by country

Source: Algan, Y. and P. Cahuc (2014), “Trust, growth and well-being: New evidence and policy implications”, in Aghion, P. and S. Durlauf (eds.), Handbook
Sources: Trust is computed
of Economic as theNorth
Growth, Vol. 2, Elsevier, country
Holland,average from
Amsterdam, responses to the trust question in the five waves of the World
pp. 49-120.
Values Survey (1981-2008), the four waves of the European Values Survey (1981-2008) and the third wave of the
Afrobarometer (2005). The trust question asks "Generally speaking, would you say that most people can be trusted or that you
need to be very careful in dealing with people?" Trust is equal to 1 if the respondent answers ''Most people can be trusted'' and 0
otherwise.
FIGURE IX.3 Trust in national government between 2006 and 2017 in OECD countries

1.00
 2017
 Percentage points change since 2006
0.80

0.60

0.40

0.20

0.00

-0.20

-0.40

-0.60
POL DEU NLD CAN CHE SVK ISL KOR SWE CZE JPN FRA ISR PRT NOR HUN ITA TUR NZL LVA EST IRL GBR AUT LUX AUS BEL DNK FIN MEX USA SVN ESP CHL GRC

Source: Gallup World Poll, https://www.gallup.com/services/170945/worldpoll.aspx.

59
HANDBOOK ON GOVERNANCE STATISTICS | 181
 status, shows the steepest gradient: Trust in the public administration is lowest for those who attended
university and highest for those who never went to school, with the gap reaching 20 percentage points
in Madagascar and Cote d’Ivoire. One explanation is that people with higher social status (urban, edu-
cated, wealthy) are more prone to express critical views, possibly due to skills developed at school and
to better access to information. In contrast, in OECD countries, trust in institutions is higher among
people with higher education (Murtin and others, 2018). Consequently, how educational skills translate
into viewing government is highly dependent on countries’ institutional context.

FIGURE IX.4 Trust in institutions by socio-economic status

100% 100%
 Male  [18-24] years  [45-59] years
80%  Female 80%  [25-44] years  60 and + years

60% 60%

40% 40%

20% 20%

0% 0%
Cameroon Madagascar Benin Uganda Mali Cape Côte Malawi Burundi Cameroon Madagascar Benin Uganda Mali Cape Côte Malawi Burundi
Verde d'Ivoire Verde d'Ivoire

100% 100%
 Urban
 No school  Secondary
 Rural
80% 80%  Primary  Superior

60% 60%

40% 40%

20% 20%

0% 0%
Cameroon Madagascar Benin Uganda Mali Cape Côte Malawi Burundi Cameroon Madagascar Benin Uganda Mali Cape Côte Malawi Burundi
Verde d'Ivoire Verde d'Ivoire

Source: GPS-SHaSA modules, 2013-2016, NSOs; authors calculations.

IX.3.4 How many institutions should be considered when asking trust questions?
422. Several of the questions on trust recommended above or included in Annex IX ask about trust
in a range of institutions (e.g. the government, the parliament, the civil service, the police) and in dif-
ferent groups of people (e.g. people you know personally, people living in the same neighbourhood,
strangers). An important issue when determining how to collect trust data is whether these questions
provide fundamentally different information. In other terms, can respondents differentiate in their
answers between them?

423. This empirical question can be answered through statistical techniques when surveys probe
respondents about their trust in different institutions or groups of people. One suitable approach is
based on factor analysis, a method that describes measured differences across respondents in terms
of their trust in different institutions or population groups in terms of a lower number of “unobserved

182 |
factors”: when measured variables are strongly correlated to the same “factors”, this implies that
respondents do not, empirically, make strong distinctions among different institutions or groups of
people. Analyses based on World Values Survey data included in the OECD Trust Database, and
reported in OECD (2017), highlight a number of patterns. First, they confirm that survey respondents
make a conceptual distinction between interpersonal trust and institutional trust. Second, in the case
of institutional trust, it suggests that respondents provide different assessments when asked about
their trust in some types of institutions but not others. In particular, individual respondents seem to
distinguish among three types of public institutions, i.e. the political system (which includes the gov-
ernment, political parties and parliament), the judicial system (which includes the police, military and
courts) and non-political institutions (NGOs, banks, universities). Third, in the case of interpersonal
trust, this analysis shows that respondents distinguish in their answers between trust in people they
know (i.e. family members, people from the same neighbourhood, people they know personally) and
trust in strangers (i.e. those met for the first time, people from a different country or religion).

424. Conversely, many of the finer distinctions made between different institutions are not very
informative empirically. This does not necessarily imply that there is no value in asking more specific
questions about trust: different trust questions may closely co-vary for reasons other than that they are
measuring the same thing (e.g. they may have very similar drivers). If users have a sufficiently strong
need, even relatively minor differences between closely related concepts may be important. However,
it suggests that, when deciding which trust measures are worth collecting, a relatively narrow range of
questions will cover the most important aspects.

IX.3.5 What do we know about the quality of trust statistics?

425. Statistical quality can be assessed in terms of relevance and accuracy. Relevance concerns
the degree to which measures of trust are sufficiently useful to justify collecting them in the context
of the official statistical system. Accuracy refers to the degree to which measures of trust capture
the intended concept, and it is typically assessed by looking at a measure’s reliability and validity.
Reliability captures the degree to which a measure produces consistent information over time and
across different measurement vehicles, whereas validity asks whether a measure actually reflects the
underlying concept being measured.

426. The policy relevance of trust measures is strong, especially for institutional trust (see section
IX.2, “Why is trust important?”). Detailed information on the accuracy of trust measures is provided
in Annex IX, Box ANX.IX.1 (see also OECD, 2017a; González and Smith, 2018). Overall, the OECD
Guidelines conclude that the validity and reliability of measures of interpersonal trust is robust.
Evidence on the accuracy of measures of institutional trust is more mixed, albeit positive (Table IX.2).
As a result, measures of interpersonal trust can be considered as strongly “fit for purpose”, while
those for institutional trust should be viewed as “promising experimental measures”. In both cases,
the OECD Guidelines and this Handbook recommend their inclusion in official surveys.

TABLE IX.2 A summary evaluation of the accuracy of survey measures of trust

Face validity Construct validity Convergent validity Reliability

Institutional trust √ √√ √ √√

Interpersonal trust √√ √√ √√ √√
Source: OECD Guidelines on Measuring Trust, OECD Publishing, Paris, https://doi.org/10.1787/9789264278219-en.

HANDBOOK ON GOVERNANCE STATISTICS | 183

 IX.3.6 How can survey design mitigate methodological issues in trust questions?
427. Trust measures are inherently subjective, i.e. only people themselves can report on them.
Subjective measures are more sensitive to response biases and measurement error than are more
objective measures drawn from administrative sources (such as educational attainment or life expec-
tancy) or from surveys (based on self-reports of, say, labour market status). However, these biases
are not unique to trust and affect many of the other self-reported measures (e.g. on availability to
work, as collected through labour force surveys to measure unemployment) that are already regularly
collected by NSOs. While it is important to be aware of these biases, the existence of measurement
error per se is not an argument against gathering data on trust; what is important is to understand the
most appropriate questions and survey design strategies to mitigate these errors. No matter which
approach to question design is adopted by data collectors, standardization is critical to ensure mean-
ingful comparison over time and between population groups and countries. This section summarizes
best practice to mitigate risks of biases and measurement errors when collecting trust data (for further
details on best practice on sampling and fielding trust data, see OECD, 2017a).

428. Question wording. The evidence on question wording (especially that drawn from split sample
experiments – see Annex IX, Box ANX.IX.2) shows that good question wording improves the quality of
results. Best practice in this field includes the following:

• Question wording should avoid referring to concepts other than trust and be tailored to the
situation of interest.
• For institutional trust, specifying what institutions are expected to do can affect responses.
For example, as compared to a neutral wording, people in some countries may state that
they trust public institutions more if the question specifies that these institutions are to be
assessed for their capacity to pursue the “national interest” (see Annex IX, Box ANX.IX.2).
• For interpersonal trust, a neutral question wording (“how much do you trust most people”)
is recommended: in particular, data collectors should refrain from referring to “caution in
dealing with other people” (the wording used, for example, in the World Values Survey), as
this phrase can prime more vulnerable groups (e.g. women, or elderly people) to report lower
trust (see Annex IX, Box ANX.IX.2).
• Question wording should be precise enough to be understood by respondents, without get-
ting into subtle nuances that might also pose problems for translatability across countries. If
the concepts (or institutions) that questions try to capture are closely related, respondents
might have difficulty differentiating between them (as in the case of surveys that differentiate
between the notions of “trust” and “confidence” in English-speaking countries).

429. Response formats. Answers to survey questions are also affected by the response options
available. Best practice in this field includes the following:

• For trust questions, the OECD Guidelines recommend using a numerical 0-10 scale with verbal
scale anchors, based on empirical evidence that this allows for greater variance in responses,
increases overall data quality and facilitates translatability across languages. Current practices
by data producers on response formats vary, with some NSOs using verbal response scales
and others using fewer numerical categories (e.g. the GPS-SHaSA questionnaire uses a 1-4
scale; DANE Colombia’s Political Culture Survey uses a 1-5 scale). Whatever the format used,
consistency across countries will be essential to guarantee comparability of trust measures.

184 |
 • The response order used to different questions should be presented consistently (i.e. not al-
ternating 0-10 and 10-0) in order to minimize mental switching between positive and negative
normative outcomes.
• Verbal descriptions of the scale anchors in the 0-10 scale should represent absolute responses
(e.g. completely/not at all) to minimize acquiescence bias and socially desirable responding
and to allow for the full spectrum of possible responses.

430. Survey context. Trust measures should be considered within the broader context of the survey
in which they are placed. Best practice in this field includes the following:

• Since order effects are more common when two or more questions deal with the same or
closely related issues, trust items that deal with different types of trust (e.g. interpersonal vs
institutional) should be buffered by intervening text.
• Whenever a list of institutions is used, it is recommended that the order of these institutions
be randomized across respondents to minimize order effects.
• Generally, trust questions should not be asked immediately after items that are likely to elicit
strong emotional responses or that refer to experiences with other people or institutions.
• Questionnaire designers should reflect on the potential effect that trust questions could have
on subsequent items, in particular those dealing with similar issues.
• To minimize the impact of holidays, seasons and elections, data collection should be spread
throughout the year or at least over multiple weeks.

431. Survey mode. Evidence suggests that trust questions can be sensitive, triggering respondents
to answer in socially desirable ways or making them unwilling to answer at all. This might especially be
the case in contexts where trust in government, and by extension in the official data collector itself, is
low. The following considerations can reduce these risks:

• Self-administered surveys, compared to interviewer-led ones, perform better in terms of min-

imizing social desirability biases.
• Whatever the survey mode, sensitivity-related response biases can be reduced by lowering
the respondent’s concerns about data protection (e.g. via confidentiality assurances provided
at the beginning of the survey) or by controlling the survey situation (e.g. avoiding that enu-
merators provide information about their own social identity).
• If face-to-face interviews are the only option, the use of innovative interviewing methods, such
as the sealed envelope or unmatched count technique, should be considered. The former in-
volves handing a separate self-administered questionnaire to the respondent for the sensitive
questions, with respondents asked to complete the questionnaire, place it in an envelope,
seal it and return it to the interviewer. The latter involves randomly splitting the sample into
two. One group of respondents is asked to answer a short list of questions that includes only
the non-sensitive items, while the other group has to respond to a longer list of questions,
consisting of the same non-sensitive items plus sensitive ones. Respondents in both groups
count the number of positive answers and report solely the sum of these items. An unbiased
estimate of the population’s proportion not trusting specific groups or institutions is obtained
by calculating the difference between the two subsample means.12

12 While the unmatched count technique has been successfully applied across a range of stigmatized behaviours, it can deal only with “yes/no” binary
response formats and does not allow for individual-level analysis.

HANDBOOK ON GOVERNANCE STATISTICS | 185

 432. Response styles and cultural context. Cross-cultural response styles are very difficult to verify
externally against a common standard or actual behaviour. While response styles do not necessarily
lower overall data quality, their effects can be reduced by the practices detailed below:

• As with all survey questions, data producers should design the questionnaire so that items are
as simple, easy to interpret and minimally burdensome as possible, rather than relying on ex
post statistical adjustment techniques.
• The overall design of the survey (including its length and how it is introduced) needs to re-
duce respondent’s burden and fatigue and increase their motivation to participate.
• Question formats that are more prone to response biases should be avoided: for example,
“agree/disagree” and “yes/no” response formats are more likely to prompt acquiescence.

IX.4 Recommended key indicators

433. As was noted above, trust is inherently a single concept, although one that can be measured
at the level of different institutions or groups of people. As a result, the list of recommended key indi-
cators for this chapter (Table IX.3) is straightforward and is based directly on the five survey questions
recommended as a “core module on trust” (presented in section IX.3.2, Box IX.2):

TABLE IX.3 Recommended key indicators

Sub-dimension Indicator Data source Type

n/a Mean value or proportion of the population above/below a specified threshold of trust in others (“most people”) Surveys Outcome

Mean value or proportion of the population above/below a specified threshold Surveys Outcome
of trust in others (“most people known personally”)

Mean value or proportion of the population above/below a specified threshold of trust in parliament Surveys Outcome

Mean value or proportion of the population above/below a specified threshold of trust in the police Surveys Outcome

Mean value or proportion of the population above/below a specified threshold of trust in the civil service Surveys Outcome

186 |
IX.5 Way forward
434. The available methodology for basic measures of trust is well developed, and evidence on
their accuracy is positive. Room for improvement remains in standardizing core survey modules across
countries, developing a more broad-based evidence base (in particular on institutional trust) and fur-
ther investigating some of the more complex methodological aspects and challenges in measuring
trust.

435. By encouraging NSOs to collect trust data more systematically, i.e. by including a core set of
internationally comparable questions in their surveys (or by aligning existing questions with a view to
international comparability), this chapter supports the development of the evidence base that might
underpin future decisions about an international statistical standard for measuring trust.

436. Going forward, several methodological questions remain unanswered, and NSOs and other
data producers can play a key role in conducting empirical research in this area. Priority research issues
for survey questions of trust include the following:

• Regarding question wording for institutional trust, experimental testing should be used to
establish which specifications (e.g. “to act in the national interest”, or “to improve the life of
someone like me”, or “to do what is right”) matter the most for each institution. Ideally, these
experiments should be carried out across more than one country.
• It is not yet clear in which cases the order of trust questions affects answers. While there is
some evidence that transitional text between questions can mitigate order effects, various
text versions should be tested for their impact on answers.
• More research is needed to assess how cultural differences and response styles might impact
on results. This can include comparing survey data with external “objective” references, such
as actual trusting behaviour in real life or experimental games. New experimental tools that
rely on insights from psychology and behavioural economics administered to representative
samples of respondents, as in the OECD Trustlab, can be used (Stiglitz and others, 2018;
Box IX.1).

HANDBOOK ON GOVERNANCE STATISTICS | 187

 PART B
EIGHT DIMENSIONS OF
GOVERNANCE STATISTICS

CHAPTER X
Safety and
security
 X.1 Conceptualizing this dimension
437. Safety and security are key components of good governance and are grounded in various
internationally agreed normative frameworks. The right to life, liberty and security of person and
freedom from torture or other cruel, inhuman or degrading treatment or punishment are examples
of rights relevant to conceptualizing safety and security. These rights are enshrined in the Universal
Declaration of Human Rights (UDHR) and other human rights treaties.1

BOX X.1 Examples of rights relevant to safety and security

Right to life

Definition: The right to life is a fundamental right, the effective protection of which is the prerequisite for protecting the safety and security of
rights holders. It is the supreme right from which no derogation is permitted, even in situations of armed conflict and other public emergencies
that threaten the life of the nation. The right to life is a right which should not be interpreted narrowly. It concerns the entitlement of individuals
to be free from acts and omissions that are intended or may be expected to cause their unnatural or premature death, as well as to enjoy a life
with dignity.

State parties have the obligation to: 1) adopt any appropriate laws or other measures in order to protect life from all reasonably foreseeable
threats, including from threats emanating from private persons and entities; 2) organize all State organs and governance structures through
which public authority is exercised in a manner consistent with the need to respect and ensure the right to life, including establishing by law
adequate institutions and procedures for preventing deprivation of life, investigating and prosecuting potential cases of unlawful deprivation of
life, meting out punishment and providing full reparation; 3) enact a protective legal framework that includes effective criminal prohibitions on
all manifestations of violence or incitement to violence that are likely to result in deprivation of life, such as intentional and negligent homicide,
unnecessary or disproportionate use of firearms, infanticide, “honour” killings, lynching, violent hate crimes, blood feuds, ritual killings, death
threats and terrorist attacks, etc.

Source: For more information, see General Comment 36, Human Rights Committee, CCPR/C/GC/36

Liberty of person

Definition: concerns freedom from confinement of the body, not a general freedom of action. Deprivation of liberty involves more severe
restriction of motion within a narrower space than mere interference with liberty of movement. Examples include police custody, arraigo, remand
detention, imprisonment after conviction, house arrest, administrative detention, involuntary hospitalization, institutional custody of children
and confinement to a restricted area of an airport, as well as being involuntarily transported. Examples also include certain further restrictions on
a person who is already detained, for example, solitary confinement or the use of physical restraining devices. During a period of military service,
restrictions that would amount to deprivation of liberty for a civilian may not amount to deprivation of liberty if they do not exceed the exigencies
of normal military service or deviate from the normal conditions of life within the armed forces of the State party concerned.

State parties have the obligation to: 1) take appropriate measures to protect the right to liberty of person against deprivation by third parties;
2) protect individuals against abduction or detention by individual criminals or irregular groups, including armed or terrorist groups, operating
within their territory; 3) protect individuals against wrongful deprivation of liberty by lawful organizations, such as employers, schools and
hospitals; 4) do their utmost to take appropriate measures to protect individuals against deprivation of liberty by the action of other States within
their territory; 5) when private individuals or entities are empowered or authorized by a State party to exercise powers of arrest or detention, the
State party must limit those powers and must provide strict and effective control to ensure that those powers are not misused, and do not lead to
arbitrary or unlawful arrest or detention; 6) provide effective remedies for victims of arbitrary or unlawful arrest or detention, etc.

1 Other human rights treaties and standards also contain provisions relevant to the right to life, liberty and security of person, e.g. International Convention
on Civil and Political Rights (ICCPR); Convention against Torture and Other Cruel, Inhuman or Degrading Treatment or Punishment (CAT); Convention on
the Elimination of All Forms of Discrimination against Women (CEDAW) and UN Resolution 1325 on Women and Peace and Security; Convention on the
Rights of the Child (CRC) and its optional protocols; International Convention for the Protection of All Persons from Enforced Disappearance (ICPED); and
the Convention on the Rights of Persons with Disabilities (CRPD).

190 |
BOX X.1 Examples of rights relevant to safety and security, CONT.

Security of person

Definition: concerns freedom from injury to the body and the mind, or bodily and mental integrity. The right to security of person protects
individuals against intentional infliction of bodily or mental injury, regardless of whether the victim is detained or non-detained. For example,
officials of States parties violate the right to personal security when they unjustifiably inflict bodily injury. The right to security of person does not
address all risks to physical or mental health and is not implicated in the indirect health impact of being the target of civil or criminal proceedings.

State parties have the obligation to: 1) take appropriate measures in response to death threats against persons in the public sphere, and more
generally to protect individuals from foreseeable threats to life or bodily integrity proceeding from any governmental or private actors; 2) take
both measures to prevent future injury and retrospective measures, such as enforcement of criminal laws, in response to past injury; 3) respond
appropriately to patterns of violence against categories of victims such as intimidation of human rights defenders and journalists, retaliation
against witnesses, violence against women, including domestic violence, the hazing of conscripts in the armed forces, violence against children,
violence against persons on the basis of their sexual orientation or gender identity, and violence against persons with disabilities; 4) prevent and
redress unjustifiable use of force in law enforcement, and protect their populations against abuses by private security forces, and against the risks
posed by excessive availability of firearms; etc.

The right to liberty and security of person are substantive rights and their deprivation have historically been principal means for impairing the
enjoyment of other rights or for impairing the protection of the safety and security of rights holders These rights are guaranteed to everyone.
“Everyone” includes, among others, girls and boys, soldiers, persons with disabilities, lesbian, gay, bisexual and transgender persons, aliens,
refugees and asylum seekers, stateless persons, migrant workers, persons convicted of crime, and persons who have engaged in terrorist activity.

Source: For more information, see General Comment 35, Human Rights Committee, CCPR/C/GC/35

Freedom from torture or other cruel, inhuman or degrading treatment or punishment

Definition: means any act by which severe pain or suffering, whether physical or mental, is intentionally inflicted on a person for such purposes
as obtaining from him or a third person information or a confession, punishing him for an act he or a third person has committed or is suspected
of having committed, or intimidating or coercing him or a third person, or for any reason based on discrimination of any kind, when such pain or
suffering is inflicted by or at the instigation of or with the consent or acquiescence of a public official or other person acting in an official capacity.
It does not include pain or suffering arising only from, inherent in or incidental to lawful sanctions.

State parties have the obligation to: 1) take effective legislative, administrative, judicial or other measures to prevent acts of torture in any
territory under its jurisdiction. No exceptional circumstances whatsoever, whether a state of war or a threat of war, internal political instability
or any other public emergency, may be invoked as a justification of torture; 2) not return (refouler) or extradite a person to another State where
there are substantial grounds for believing that he would be in danger of being subjected to torture; 3) ensure that all acts of torture are offences
under its criminal law; etc.

Source: For more information, see Convention against Torture and Other Cruel, Inhuman or Degrading Treatment
or Punishment (https://www.ohchr.org/Documents/ProfessionalInterest/cat.pdf)

438. The two terms safety and security are often used interchangeably. In this chapter, the term
safety refers mainly to the condition of absence of threats and risks, including to the right to life, liber-
ty and security of person and freedom from torture or other cruel, inhuman or degrading treatment or
punishment, to individuals, households and communities. Security refers here mostly to the activities,
policies and institutions developed and implemented by the State to protect citizens’ safety.

439. While citizens’ safety can be endangered by a large range of possible threats, including for
example diseases affecting human health, natural disasters, traffic and labour accidents, in the context
of this Handbook, reference is made primarily to specific threats related to intentional human activities
that can cause harm to persons or communities either in the form of criminal offences and human

HANDBOOK ON GOVERNANCE STATISTICS | 191

 rights violations/abuses perpetrated by individuals or groups, or in relation to conflicts between States
or between States and other actors.

440. Accordingly, security refers to the activities of official law enforcement and criminal justice
agencies directly responsible for protecting citizens from criminal activities, human rights violations/
abuses and groups.2

441. Gaining a comprehensive understanding of these issues requires analysing them from different
perspectives. For the purpose of this Handbook, the following sub-dimensions of safety and security
are therefore considered:

• Actual levels and patterns of specific types of crime, human rights violations/abuses and per-
ceptions of safety;
• Casualties directly linked to conflicts;
• Quality of law enforcement and criminal justice institutions.

X.2 Why is this dimension important?

442. Citizens´ safety is a prerequisite for protecting fundamental human rights, ensuring minimal
conditions of personal and socioeconomic well-being, and promoting effective institutions. Preserving
citizens´ safety is one of the essential functions of the State through its security sector institutions and
legislation. Establishing and maintaining a State security apparatus that operates according to the rule
of law is fundamental to ensure citizens´ safety, as well as to allow citizens to thrive and to participate
meaningfully in public life and promote inclusive governance.

443. Reliable evidence about the state of safety and security affecting men, women, boys and
girls is essential for the development of sound crime prevention policies and programmes aimed
at preventing or reducing crime and conflict, improving public safety and reducing fear of crime in
communities. Assessing the capacity, efficiency and effectiveness of the security sector is key to devel-
oping effective, inclusive and accountable security institutions, taking into account issues of gender
and age, and contributing to sustainable development. Tracking security-relevant indicators aims at
measuring change in security sector governance.

X.3 Data and best practices currently available

444. In alignment with the sub-dimensions of safety and security outlined above, various measure-
ment approaches exist to disentangle these issues. First, the level of safety of a certain community
or population in a country can be assessed by tracking and analysing actual levels and patterns of
specific types of crime that can harm the physical, sexual or psychological integrity of people and
communities, including the safety of those defending fundamental freedoms, such as human rights
defenders and journalists. Moreover, given the importance of perceptions in shaping the safety of
community members, it is also of key importance to assess and track the feeling of safety as perceived

2 State security institutions responsible for protecting from external threats – i.e. the defence system – will not be covered in this chapter.

192 |
by men and women. Second, in countries and areas affected by armed conflicts, the measurement of
conflict-related deaths is a first instrument to assess the impact of violence, including violence against
women and girls, generated in such circumstances. Third, the quality of security institutions (such as
police, courts, prisons and relevant human rights institutions) can be measured by structural, process
and outcome indicators concerning legal, institutional and policy frameworks on security, including
in the handling of criminality and prevention of abuse by law enforcement officials as well as their
resources, activities and procedures and on the experience of law enforcement officers; on human
rights violations committed by law enforcement officials on duty; and on the general public’s attitudes
towards and perceptions of these institutions.

445. Hence, this section presents available data sources along the following structure:

A. Data on actual levels and patterns of specific types of crime, human rights violations/abuses
and perceptions of safety.
1) Administrative data on crime reported to or detected by the criminal justice system.
2) Administrative data on reported or detected violence produced by public health institu-
tions and other authorities or institutions (such as national human rights institutions, inter-
national human rights mechanisms, violence observatories or victim organizations).
3) Victimization surveys and other relevant surveys such as specialized surveys on violence
against women.

B. Data on conflict-related deaths.

4) Administrative data on casualties related to armed conflict (e.g. data collected by nation-
al human rights institutions, United Nations field presences operating in armed conflict
situations).
5) Estimates of conflict fatalities combining multiple data sources/records and statistical
methodology (e.g. multiple systems estimation).

C. Data on the performance and quality of law enforcement and criminal justice institutions.
6) Administrative data on law enforcement and criminal justice institutions to capture their
performance (e.g. SDG indicator 16.a.1 and related administrative records of international
peer review mechanisms for compliance of national human rights institutions with interna-
tional standards).
7) Surveys on law enforcement and criminal justice institutions (i.e. general population surveys
and surveys of employees of these institutions).

HANDBOOK ON GOVERNANCE STATISTICS | 193

 X.3.1 Administrative data on crime reported to or detected by law enforcement and criminal
justice authorities

X.3.1.1 Description

446. This section addresses administrative data on crime, both its victims and perpetrators, pro-
duced by law enforcement and criminal justice authorities during the process of recording, inves-
tigating and managing judicial proceedings related to a crime event (UNODC, 2011b; UN DESA,
2003). Each of the four main institutions of the criminal justice system – police, prosecution, courts and
prison administration – usually produce statistics on recorded criminal offences, victims, perpetrators
and several other relevant characteristics – such as place and time of occurrence, modus operandi,
relationship between victims and perpetrators, etc. – that can provide valuable information for under-
standing and preventing crime.

447. Police statistics refer to the number and characteristics of criminal offences reported to or
detected by the police; the number of victims recorded; and the number of suspected perpetrators
brought into initial formal contact with the police and/or the criminal justice system. Such statistics on
crime, victims and perpetrators are routinely collected by law enforcement agencies (e.g. national,
state, regional and/or local police). Police statistics represent the first stage and input data in a system
of crime and criminal justice statistics.

448. Prison statistics can provide information on persons detained, by legal status (awaiting trial,
sentenced, on parole, etc.) and disaggregated by several other characteristics such as the demo-
graphics of the persons in prison, type and length of sentence and the criminal offence committed.

449. Data on alternative sanctions (or non-custodial measures) should also be produced at country
level, and they include information on non-custodial sanctions imposed by courts or on persons under
non-custodial supervision by prisons. The institutions collecting these data vary greatly from coun-
try to country and may include monetary fine programmes, communities responsible for alternative
measures, and programmes alternative to incarceration, etc. (UN DESA, 2003).

450. For crime and criminal justice data, the methods for collection, aggregation and flow (from
local to central level), validation and dissemination vary from country to country on the basis of the
legal framework and organizational structure of the criminal justice system. International standards
(UN DESA, 2003; UNODC, 2010a) suggest that statistics on crime should ideally be collected by:

• Developing a unit-based system containing details of each individual incident and person
accused (or brought into initial formal contact with the police).
• Collecting and storing individual records in a safe and computerized environment, such as
automated processes; this can better support regular data aggregation and reporting from
the local to the central level (UN, 2010).
• Using standardized forms for data recording. A standard “incident information form”, for
example, may be used by the police to record information about the crime incident, the
victim and any suspected/identified offender.
• Categorizing records according to a standard offence classification system that is consistent
with the International Classification of Crime for Statistical Purposes (ICCS).3

3 UNODC, International Classification of Crime for Statistical Purposes, Version 1.0, 2015.

194 |
 • Disaggregating victim, alleged offender and criminal offence by characteristics and variable
indicated in the ICCS4 (such as age and sex of victim and perpetrator, offender-victim relation-
ship, place and time of the criminal event, etc.).
• Enabling the transfer of information from the local, operational level to a central data col-
lection point according to standard statistical processes within the relevant criminal justice
agencies (e.g. Ministry of the Interior, Ministry of Justice, etc.). These data can be validated
by the statistical units and disseminated by the criminal justice agencies themselves, or they
can be transferred to the NSOs for a further validation process and dissemination through the
main national official statistical channels.
• Involving NSOs in developing concepts, categories and classifications for collecting and pro-
ducing crime statistics, as well as recording and coding rules (UN DESA, 2003).

451. International initiatives to compile crime and criminal justice statistics try to harmonize the na-
tional data by requiring countries to adapt their national-level statistics to fit the standard categories
of crime and to indicate potential deviations from their standard definitions and the counting rules
used to collect the crime incidents and perpetrators (Mugellini, 2011). At global level, the main source
of data on crime and criminal justice is the United Nations Survey of Crime Trends and Operations
of Criminal Justice Systems,5 which is conducted annually by the United Nations Office on Drugs
and Crime (UNODC). At the regional level, Eurostat and the Organisation of American States (OAS)
operate their annual data collections jointly with UNODC respectively for the European countries6 and
for the Americas.7

452. Other initiatives include the European Sourcebook of Crime and Criminal Justice Statistics,
an initiative of academics and criminal justice experts, which has collected and disseminated data on
crime and criminal justice for European countries.8 The Small Arms Survey9 has published estimates on
violent deaths since 2004 on the basis of ad hoc research and using various administrative data sources
(e.g. NSOs, national police, UNODC, Eurostat, WHO). Among violent deaths, data are published on
intentional homicide, conflict deaths, violent deaths by firearm and female victims of lethal violence.

4 See ICCS (UNODC, 2015), pages 98-104.

5 The United Nations Survey of Crime Trends and Operations of Criminal Justice Systems (UN-CTS) started in the 1970s. Its aim is to collect data on the
incidence of reported crime and the operations of criminal justice systems with a view to improving the analysis and dissemination of that information
globally (UNODC, 2007). The UN-CTS is conducted annually and is managed by UNODC in collaboration with regional organisations (Eurostat for EU
Member States and the Organisation of American States – OAS – in the Americas).
6 The Eurostat data collection on crime started in 2004 to produce crime and criminal justice statistics of the member States, complemented with data
from the European Union candidate countries and the countries in the European Free Trade Association (EFTA)/ European Economic Area (EEA). Since
2014, Eurostat has run its annual data collection jointly with UNODC.
7 http://www.oas.org/IOS/indicators.aspx?lang=en
8 The first European Sourcebook project started in 1996, when the Council of Europe established a committee to prepare a compendium of crime and
criminal justice data for its member states. The European Sourcebook (ESB) covers all criminal justice sectors: police, prosecution, courts and corrections.
The fifth and most recent edition considers crime data from 2007 to 2011 and was published in 2014.
9 http://www.smallarmssurvey.org/tools/interactive-map-charts-on-armed-violence.html

HANDBOOK ON GOVERNANCE STATISTICS | 195

BOX X.2 Intentional Homicide Statistics

Data on intentional homicide can provide valuable and comprehensive information to monitor and analyse violence around the world, both
across time and across countries. Due to the seriousness of the crime and its lethal outcome, homicide events are carefully reported by law
enforcement authorities, making for more accurate data, especially when compared to other types of crime. In addition, when homicide statistics
are disaggregated into different typologies, it is possible to achieve a better understanding of what causes violence and – accordingly – to design
targeted and effective homicide prevention policies.

When measuring the scale of homicide, it is important to use a clear definition that provides guidance on which specific acts of killing are to be
considered intentional homicide. For example, certain definitional challenges may arise when intentional killings have to be distinguished from
other killings during situations of collective violence, such as armed conflict or civil unrest. The International Classification of Crime for Statistical
Purposes (ICCS)10 provides a framework for the definition and classification of unlawful killings, both in conflict and non-conflict situations.
According to the ICCS, intentional homicide is defined as the “unlawful death inflicted upon a person with the intent to cause death or serious
injury”.

At international level, an important source of information on homicide is provided by the UNODC Homicide Statistics dataset (2019)11, which
presents available data for more than 200 countries and territories. Data included in the dataset are sourced from either criminal justice or public
health systems. In the former, data are generated by law enforcement or criminal justice authorities in the process of recording and investigating a
crime event, whereas in the latter, data are produced by health authorities certifying an individual’s cause of death. Although discrepancies exist,
especially when coverage and quality of administrative records are poor, the two sources often provide very similar results.

The UNODC Global Study on Homicide (2019) draws on the 2019 Homicide Statistics dataset to give a comprehensive overview of intentional
homicide across the world. It provides statistical evidence to help improve the understanding of trends and patterns in lethal violence, and it aids
governments in their effort to address root causes and effectively reduce violence. Among several findings, the report highlighted that criminal
activity is responsible for many more deaths worldwide than armed conflict and terrorism combined. Shooting remains the most common
instrument of lethal violence, with slightly more than half of all homicides carried out with firearms in 2017. 81 per cent of homicide victims
recorded were men and boys, and the global homicide rate for the male population is roughly four times the rate for women and girls. However,
women were the victims in 82 per cent of homicides carried out by an intimate partner, which shows that they bear the greatest burden of
victimization in the context of intimate partner violence.

Firearms were involved in more than half

of all homicides worldwide in 2017

97,183 victims 104,341 victims

22% 24%
81% 82%

64%

36%

19% 18%

238,804 victims Total homicide Intimate partner/ Intimate partner/

54% family-related homicide homicide
Source: UNODC, Global Study on Homicide, 2019

10 UNODC, International Classification of Crime for Statistical Purposes (ICCS): Version 1.0 (Vienna, 2015).
11 The UNODC Homicide Statistics (2019) dataset is available at https://dataunodc.un.org/GSH_app

196 |
X.3.1.2 Methodological and quality issues

453. Administrative data on crime have a number of strengths, such as the fact that they derive directly
from administrative activities and therefore tend to be accurate and comprehensive; furthermore, such
data can be very detailed in terms of a number of characteristics, such as geographic location of crime
events or characteristics of victims and/or perpetrators. Furthermore, they can often be available in a
timely manner. When complying with the ICCS, administrative data on crime can provide accurate and
granular information on crime, which can be comparable across and within countries.

BOX X.3 I nternational Classification of Crime for Statistical Purposes (ICCS): Improving the quality
and comparability of data on crime and justice at national and global level

National statistics on crime typically produce data on criminal offences as defined by each country’s criminal law system. Without legal
harmonization, differences in the definition of offences are inevitable, and international comparison must always be placed in the context of
these differences. The comparison of crime statistics across time, between countries or with other available statistics is particularly difficult due to
the lack of standardized concepts and the absence of an internationally agreed statistical framework for making such comparisons.

In order to overcome such challenges, the International Classification of Crime for Statistical Purposes (ICCS) was developed. Endorsed in 2015 by
the United Nations Statistical Commission (UNSC) in its 46th session and by the Commission on Crime Prevention and Criminal Justice (CCPCJ) in
its 24th session, the ICCS is a classification of criminal offences that is based on internationally agreed concepts, definitions and principles in order
both to enhance the consistency and international comparability of crime statistics and to improve analytical capabilities at both the international
and national levels.

At the national level, the ICCS can be used as a model to provide structure and organize statistical data that are often produced according to
legal rather than analytical categories. Moreover, the ICCS can harmonize data across domestic criminal justice institutions (police, prosecutions,
courts, prisons) and across different data sources (administrative records and statistical surveys). Likewise, the ICCS can be used as a tool to
standardize data from sub-national entities that may have different statistical systems or legal frameworks. At the international level, the ICCS
improves the comparability of crime data between countries. Standardized concepts and definitions allow for the systematic collection, analysis
and dissemination of data, and also respond to the demand for in-depth research and analysis of transnational crime.

The ICCS is the bedrock for producing accurate, comprehensive, disaggregated and internationally comparable statistics for several SDG indicators
in the area of crime, violence, illicit trafficking, access to justice and corruption.

Understanding crime
The adoption and mainstream utilization of the ICCS ensures a deep understanding of the characteristics of crime by disaggregating data by:

Victims Perpetrator's Context Motives

Counting all crimes

• The ICCS allows to discover criminal conducts that had not previously been identified or registered.
• It enhances spatial and temporal analysis through geo-referencing.
• And helps to observe the full scope of all crimes, according to:

1 Targets of crime 2 Seriousness of the act 3 Modus operandi 4 Perpetrators state of mind 5 Policy relevance

Source: UNODC, International Classification of Crimes for Statistical Purposes (ICCS, 2015).

HANDBOOK ON GOVERNANCE STATISTICS | 197

 454. On the other hand, the main limitation of criminal justice statistics is that they reflect only
those incidents reported to or otherwise detected by law enforcement agencies and/or brought into
formal contact with the criminal justice system. Therefore, they suffer from undercounting due to the
“dark figure”.12 Not all victims report a crime to the police for several reasons (e.g. level of trust in
the police, perception that the police do not take incidents seriously, an insufficient awareness of the
crime suffered, level of own involvement in the crime, fear of reprisal, etc.) (van Dijk and others, 2007).
For example, according to the results of the European Crime and Safety Survey 2004/2005 (EUICS), on
average, roughly 50 per cent of five crimes (burglary, thefts from cars, robbery, sexual incidents, and
assaults and threats) were reported to the police (Van Dijk and others, 2005).

BOX X.4 Factors influencing the willingness to report crime to the police

Skogan (1984) identifies two major determinants of reporting crimes to the police: the type of crime, and its
seriousness.

Some types of crime are typically reported more often than others (e.g. property crimes vs violent crimes)
(Tolsma and others, 2012; van Dijk and others 2007) or are more visible and therefore more easily discovered
than others (e.g. homicides and car theft vs bribery). In developed countries, car theft and burglary are usually
reported to the police, including because this is required for reimbursement from insurance (Mugellini 2010).
Results from the last five waves of the International Crime Victim Survey (ICVS) show that, on average, 89
per cent of car thefts and 77 per cent of burglary were reported to the police, while only 17 per cent of sexual
incidents were reported to the police (Van Dijk and others, 2007). The average reporting rate for car theft and
sexual offences is confirmed also across African countries such as Egypt (71.4 per cent; n.a.); Ghana (83.3 per
cent; 13.5 per cent); Kenya (94 per cent; 19.7 per cent); Tanzania (71.4 per cent; 33.5 per cent) and Uganda (71.4
per cent; 13.4 per cent). However, the reporting rate for burglary with entry is much lower (Kenya 38.2; Ghana
13.8 per cent; Egypt 32.1 per cent; Tanzania 44.5 per cent; Uganda 30.2 per cent) (see UNODC Data for Africa,
Collection and analysis of data and trends on drugs, crime and victimization in Africa 2009-201013).

The severity of the crime measured by the amount of violence or the material loss is an important offence
characteristic that influences the decision to report (Isenring and others, 2016). Research reveals that violent
crimes are more likely to be reported if the victim was injured (Harlow, 1985). In contrast, incidents of sexual
violence such as unwanted touching or other non-consensual sexual contact are more likely not to be reported
– and also to be considered unfounded by the police – than are more violent sexual assaults. This affects not
only the count of sexual violence but also access to justice for women.

One of the most common reasons for non-reporting is related to a low level of trust in the work of the police or
to a belief that reporting is pointless and a waste of time that leads to no concrete results (Isenring and others,
2016; Goudrian, 2006).

Also, the stigma and/or reputational damage associated with the crime can affect the willingness to report
(e.g. sexual violence experienced by an individual, but also fraud or unfair competition experienced by a private
business). The results of the Swiss Business Crime survey revealed that when dealing with fraud, financial
companies prefer not to report or ask for any police intervention and prefer instead to deal with the incident
discreetly through internal processes in order to avoid reputational damage and the loss of clients’ trust
(Isenring and others, 2016).

The “dark figure” also varies according to the level of the police’s professionalism, efficiency, recording
procedures and ability to collect data (Howard and others, 2000; Van Dijk 2008). Well-developed systems for
gathering and processing statistics might have a strong influence on the number of recorded crimes (Von Hofer,
2000).

12 The difference between the number of crimes actually committed and the number of officially recorded crimes.
13 https://www.unodc.org/unodc/en/data-and-analysis/Data-for-Africa-publications.html

198 |
 455. Another limitation of administrative data on crime is that they are collected for operational and
management purposes and therefore often lack the degree of standardization needed to produce sta-
tistics and may not be complete in relation to information that is not fully relevant from an operational
perspective (i.e. some characteristics of victims or perpetrators).

456. While taking these issues into account, statistics produced by the police can be very valuable
to measure crime, for example in relation to time trends or geographical patterns. Moreover, they are
usually more relevant than those produced by prosecution or courts, as they are closer to the criminal
event in terms of time lag, and they also include crime incidents for which no alleged offender has
been identified (Mosher, Miethe and Phillips, 2002).

457. Comparability issues related to police statistics may also result from differences in the methods
used to elaborate statistics. In this regard, the following elements may affect the comparability of
crime statistics: 1) the moment of inclusion of incidents into statistics (i.e. at the moment the crime is
first reported/detected or after a first investigation is conducted); 2) counting units and rules (e.g. of-
fence vs case vs decision); 3) the application of a principal offence rule; 4) the way in which an offence
committed by more than one person is recorded; 5) the way in which multiple offences are counted,
for example a rape that has occurred in a case of femicide, or a homicide occurring during a robbery
(Aebi, 2008; Mugellini, 2010).

X.3.2 Administrative data on violence produced by public health institutions

X.3.2.1 Description

458. National health authorities produce data on the causes of death and injuries recorded by pub-
lic health system and civil registration. Data on violence may also be collected through domestic vio-
lence shelters and services, anti-violence centres, crime and violence observatories, and safe houses.

459. Data on the number and type of injuries recorded by health authorities are sometimes used to
estimate trends in violent crimes (e.g. Injury Surveillance Workgroup, 2003).

460. The World Health Organization14 collects and compiles comprehensive data on causes of
deaths from national sources.

461. Of particular value is the fact that data on homicide can be produced at country level from two
independent processes. Criminal justice data on homicide are typically recorded by police or public
prosecutors, based on information collected when they receive details of a crime. Homicide data are
also produced by public health or civil registration systems on the basis of information on the causes
of death. Where both criminal justice and public health sources exist, the two sources often provide
similar results, although discrepancies often exist where coverage and quality of administrative re-
cords are poor (UNODC, 2013, p. 100).

14 WHO Mortality Database, https://www.who.int/healthinfo/mortality_data/en/

HANDBOOK ON GOVERNANCE STATISTICS | 199

 BOX X.5 Harmonization of homicide data: The case of Chile

In many countries, there are multiple criminal justice and public health institutions recording data on homicide,
using a variety of different indicators and definitions specific to their individual purposes. This is the case in Chile,
where several sources of data on homicide are available in independent data collection systems. The diversity
and variability of these sources is but one national-level example of the challenges extant in determining the
overall “national” count of intentional homicides. For the purposes of international comparison, based on
standardized definitions, Chile was able to effectively coordinate between its various agencies through the
exchange of information and discussion to produce internationally comparable data, based on timeliness and
coverage of the data. As a result, data from the Subsecretarías de Prevención del Delito was selected as the best
source of data matching the standardized definition of intentional homicide (UNODC, 2014).

X.3.2.2 Methodological and quality issues

462. National and international data sources on causes of death usually publish data on medically
certified deaths. In most cases, the cause of death is recorded according to the WHO International
Classification of Diseases (ICD).15

463. Public health data on death and violence are produced to identify the factors that caused
an individual’s injury or death. Therefore, they provide information on injuries of victims but cannot
provide information on perpetrators or the relationship between victim and perpetrator. They also
do not provide information on intentionality, underlying motives or the circumstances that led to the
violent death.

X.3.3 Victimization surveys

X.3.3.1 Description

464. Sample surveys are used to measure the extent and patterns of victimization from crime, the
perception of safety of citizens, as well as the trust in and performance of law enforcement and crim-
inal justice agencies. They provide an important complement to administrative data, as many crimes
and violent events go unrecorded. Victimization surveys produce crime measures independently of
police activities and free from legal, political or administrative influences, by interviewing a repre-
sentative sample of individuals about their experiences and perception of crime. Surveys have been
particularly useful in settings with little or no data availability and/or have been used to complement
administrative statistics (as for example the National Crime Victimization Survey in the US or the Crime
Survey for England and Wales). Similarly, the GPS-SHaSA initiative has developed survey modules on
“Governance, Peace and Security” across African countries. A comprehensive review of victimization
surveys implemented at country level is included in the Atlas of Victimization Surveys developed by
the Centre of Excellence UNODC-INEGI16, which shows that more than 140 surveys have been carried
out in over 70 different countries by a wide variety of actors in recent decades.

465. Besides overcoming underreporting and undercounting problems, the use of victimization sur-
veys, which follow standardized definitions, allow collecting comparable information across different
countries Moreover, they provide detailed information on victims’ demographic characteristics and
15 https://www.who.int/classifications/icd/en/
16 http://www.cdeunodc.inegi.org.mx/index.php/mapa/

200 |
their social, economic and political environment. This can help in identifying whether specific charac-
teristics or behaviours are associated with an increased risk of crime. This information is fundamental
for designing and implementing crime prevention measures and policies and for effecting institutional
reforms and monitoring their effectiveness.

466. Victimization surveys are a fundamental tool also for monitoring progress towards a number of
SDG targets (Table X.1) as they are the source to produce at least four SDG indicators.

TABLE X.1 SDG indicators derived from Victimization Surveys

16.1 Significantly reduce all forms of violence 16.1.3(a) Proportion of population subjected to (a)
and related death rates everywhere physical violence in the previous 12 months

16.1.3(b) Proportion of population subjected to (b)

psychological violence in the previous 12 months

16.1.3(c) Proportion of population subjected to (c)

sexual violence in the previous 12 months

16.1.4 Proportion of population that feel safe

walking alone around the area they live

11.7 By 2030, provide universal access to safe, 11.7.2: Proportion of persons victim of physical or
inclusive and accessible green and public sexual harassment in the previous 12 months
spaces, in particular for women and children,
older persons and persons with disabilities

16.3 Promote the rule of law at the 16.3.1 Proportion of victims of violence in the previous 12 months
national and international levels and who reported their victimization to competent authorities or
ensure equal access to justice for all other officially recognized conflict resolution mechanisms

BOX X.6 S
 tandardizing Victimization Surveys: The Latin America and
Caribbean Crime Victimization Survey Initiative (LACSI)

In 2013, the UNODC-INEGI Centre of Excellence in Statistical Information on Government, Crime, Victimization
and Justice analysed victimization surveys in 17 countries and 2 cities in the Latin American and Caribbean
region and found that there was a need to harmonize methodologies in order to produce comparable data.

The efforts of the initiative, involving 12 countries from Latin America and several regional and international
organizations, have led to the development of a standardized methodology for measuring crime victimization
by considering a number of core and non-core crimes, the dark figure, the reasons for not reporting victimization
experiences, perceptions about safety and attitudes towards the criminal justice system. Currently, it is
focusing its efforts on developing additional modules on cybercrime and harassment. To date, nine countries
have adopted the LACSI methodology.

Source: UNODC-INEGI Centre of Excellence in Statistical Information on Government, Crime, Victimization and Justice, 2013-2019.

HANDBOOK ON GOVERNANCE STATISTICS | 201

 467. Besides general victimization surveys, specialized sample surveys on violence against women
proved to be very effective in estimating the extent and patterns of violence against this vulnerable
population. Even if the majority of victimization surveys include questions on sexual violence targeted
at both women and men, dedicated modules are able to capture more details on the perception
and experience of both physical and psychological violence against women. Given the sensitivity of
the topic, dedicated surveys can be structured, and interviewers trained, in a way to gain the trust
of respondents and allow them to be ready to disclose events that have a strong emotional impact.
Specific guidelines for Producing Statistics on Violence against Women through surveys have been
developed by the Department of Economic and Social Affairs of the United Nations (UN, 2014). As
mentioned above, much of the violence experienced by women goes unreported to the police and
public institutions (i.e. marital rape), and some forms of violence experienced by women may not be
criminalized in some countries (i.e. sexual harassment). This makes surveys on violence against women
a key instrument for data collection to obtain estimates on this topic.

BOX X.7 E
 xperience and perception of crime, safety and security:
Results from the GPS-SHaSA modules in Africa

As part of the Strategy for the Harmonization of Statistics in Africa (SHaSA), the Governance, Peace and Security
(GPS)-initiative developed a harmonized methodology to collect survey data on crime, safety and security. The
GPS-SHaSA questionnaire addressed various dimensions of criminal (except homicide) violence and political
violence, and it collected information on both the perception and experience of safety and security. The figure
below provides some illustrative results for nine countries that conducted the first round of the GPS-SHaSA
survey between 2013 and 2016: Benin, Cameroon, Cape Verde, Cote d’Ivoire, Madagascar, Malawi, Mali and
Uganda.

Some interesting findings can be highlighted: in the nine countries, the rate of crime victimization varies
hugely; it ranges from 6% to 9% in Mali and Madagascar respectively, to 25% in Burundi, 27% in Malawi and
29% in Cameroon. Second, the share of the population who do not feel safe (“always” or “often”) when walking
in their neighborhood is much higher than the share of the population who were victims of crime. Furthermore,
the correlation between crime and the feeling of insecurity is far from perfect, demonstrating that these are
two distinct phenomena. Finally, those who personally experienced a crime also tend to feel more insecure.
Benin is emblematic in this respect: while 52% of the population declare they feel unsafe when walking alone
around the area they live, this share rises to 77% among those who have actually been a victim of crime.

FIGURE X.1 C
 rime victimization and perception of safety
in selected countries of Africa (2013-2016)

100%
80%
60%
40%
20%
0%
Mali Madagascar Cape Verde Benin Ivory Coast Uganda Burundi Malawi Cameroon

 Fear of walking alone  Fear if victim  Victim of crime

Sources: GPS-SHaSA modules, 2013-2016, implemented by National Statistical Offices.

202 |
 468. In addition to victimization surveys, self-report delinquency studies and retrospective mortality
surveys are additional ways to collect data on crime and violence. Self-report studies of offending
and victimization aim to measure the prevalence and incidence of offending; to test theories about
the correlates of offending; and to describe the dimensions and trajectories of delinquent careers, by
asking individuals if they have engaged in delinquent or criminal behaviour (Junger-Tas and Marshall,
2012; Thornberry and Krohn, 2000). This type of survey has been developed mainly in the US, Canada,
New Zealand, Australia, the UK and other European countries, but also in Japan, India, China and
South Africa (Junger-Tas and Marshall, 2012). The first International Self-Report Study (ISRD) was
launched in 1992 by the Dutch Research and Documentation Centre (WODC) and covered juvenile
delinquency across several European countries (https://web.northeastern.edu/isrd/summary/).

X.3.3.2 Methodological and quality issues

469. It is important that those planning to undertake a survey understand not only the strengths of
the survey method (as outlined above) but also the limitations. A comprehensive discussion of best
practices and tools for designing and conducting victimization surveys – as well as on methodological
and other challenges – is given in the UNODC-UNECE Manual on Victimization Surveys (2010).

470. Although data from crime victimization surveys are likely to elicit a fuller disclosure of criminal
incidents than data from police records, they can also be subject to undercounting, as some victims
may be reluctant to disclose information to the interviewer, particularly for incidents of a sensitive
nature, such as sexual assault (social desirability bias). Some criminal incidents are more difficult to
measure, for example when surveys rely on the respondents realizing that they have been a victim
of crime but this has not been apparent to them before. The accuracy of sample survey data is also
influenced by respondents’ ability to recall past victimizations. However, there are methodological
tools that have proven successful for reducing such risks of misunderstanding or recall issues.

471. Criminal offences that are not as prevalent in the community will require larger samples to
be representative of the population and data for very low prevalence offences (such as intentional
homicide or kidnapping).17 A large sample size will have a positive impact on the precision of survey
results, while the downside is that big sample sizes are expensive, and quality control of a large survey
operation may be more challenging.

X.3.4 Administrative data on documented deaths and injuries related to armed conflict

X.3.4.1 Description

472. Armed conflict and other acts of mass violence continue to be a scourge of humanity, resulting
in the loss of many thousands of lives every year. Efforts to avert the risks of armed conflict and to
strengthen international peace and security are among the most important safeguards of the right to
life and the safety of rights holders.

473. This section addresses administrative data on conflict-related deaths and injuries. The mech-
anisms, bodies and institutions that have the mandate, capacity and independence to document and
investigate incidents related to conflict (e.g. UN peacekeeping operations, commissions of inquiry,
humanitarian operations and human rights offices and National Human Rights Institutions, NHRIs) all
produce casualty records.

17 For an example of measuring homicide through large-scale sample surveys in Nigeria, see UNODC (2019).

HANDBOOK ON GOVERNANCE STATISTICS | 203

 474. UN human rights entities operating in situations of armed conflict, given their mandate,
proximity to hostilities, extensive local networks and cooperation with various organizations, are well
placed to provide accurate data on documented conflict-related deaths, focusing on civilians. In most
instances, the focus is on direct deaths. The methodology used ensures that the data generated is
verified, comprehensive and disaggregated according to multiple factors.18 This methodology has
been endorsed as part of the methodology for the SDG indicator on conflict-related deaths.

475. NHRIs play an important role in addressing issues related to conflict and its consequences.
Although faced with difficulties and challenges during conflict, NHRIs monitor the human rights sit-
uation in accordance with international and regional human rights standards. They document and
respond to international humanitarian emergencies and violations of human rights law and report to
international, regional and national human rights mechanisms.19

476. NSOs, with their work in supporting government entities in strengthening their administrative
records, including those working on vital, crime, health, justice and security statistics, are also impor-
tant data providers.

477. Examples of data sources in this field, i.e. the source used to record and verify information
on the persons killed and related incidents, include: eyewitnesses or those directly affected; hospital
records; community elders, religious and civil leaders; security forces and conflict parties; local author-
ities; prosecution offices, police and other law enforcement agencies, and health authorities; govern-
ment departments and officials; UN and other international organizations; media reports; members
of the international community; and civil society organizations. In addition, relevant information may
be obtained from digital sources (e.g. videos, photos), documentary sources (e.g. medical, police and
judicial records) and open sources (e.g. online information).

X.3.4.2 Methodological and quality issues

478. In the absence of a single set of agreed definitions and categories for the information required
for the SDG 16.1.2 indicator, the OHCHR, as the custodian agency for SDG 16.1.2, has held a series of
consultations and coordination events between institutions working on conflict-related issues to har-
monize and build upon existing standards and methodologies, and to integrate available data into a
single collection that serves the purposes of this indicator. This has involved consultations with a range
of stakeholders, including national statistical offices, as part of the work of the Praia City Group on
Governance Statistics and the Global Alliance of National Human Rights Institutions (GANHRI). This
methodology will be further developed in collaboration with NSOs, NHRIs, UN entities and CSOs.

479. For this methodology, definitions and categories for disaggregation were developed based
on international legal standards, existing statistical classifications and UN methodological standards
on casualty recording. “Conflict” has been defined according to International Humanitarian Law (IHL),
and “conflict-related deaths” include direct and indirect deaths. Direct deaths are deaths where there
are reasonable grounds to believe that they resulted directly from war operations and that the acts,
decisions and/or purposes that caused these deaths were in furtherance of or under the guise of
armed conflict. Indirect deaths are deaths resulting mainly from a loss of access to essential goods

18 See the OHCHR Guidance on Casualty Recording, OHCHR, 2019, available at: https://www.ohchr.org/Documents/Publications/Guidance_on_Casualty_
Recording.pdf.
19 See the Kyiv Declaration on the Role of National Human Rights Institutions in Conflict and Post-Conflict Situations, available at http://ennhri.org/IMG/
pdf/the_kyiv_declaration.pdf

204 |
BOX X.8 S
 DG 16.1.2 on the number of conflict-related deaths per
100,000 population, by sex, age and cause

In March 2019, the Inter-Agency and Expert Group on SDG indicators (IAEG-SDGs) approved the methodology
to measure the number of conflict-related deaths, establishing new statistical standards and data collection
efforts at international level. SDG indicator 16.1.2 monitors conflict-related deaths and will help assess policies
on the protection of civilians and other victims, respect for humanitarian and human rights standards, and the
prevention of armed conflicts. The approved methodology:

• was developed based on already agreed legal, statistical and casualty recording standards and practices at
international level;
• provides harmonized definitions of “armed conflict”, internationally and non-internationally, and of
“conflict-related deaths”, including direct deaths and indirect deaths;
• provides an approach to identifying situations of armed conflict based on the assessments, as contained
in publicly available reports and other relevant documents, issued by the UN and other internationally
mandated entities;
• takes a concentric approach in the methodology development, data collection and compilation of the
indicator, starting with counting documented direct deaths, then estimating undocumented direct deaths
and last, estimating indirect deaths;

Documented deaths caused by use of

weapons or other means and methods
1. Civilians*  2. Other persons killed

(“direct”)
[casualty recording]

Estimation of undocumented deaths caused by use

of weapons or other means and methods
(“direct”)
[statistical estimates/surveys]

Estimation of all deaths caused by use of weapons or

other means, methods or factors (excess mortality)
(“direct/indirect”)
[statistical estimates/surveys]

• provides a basic list of verifiable data that altogether define the case following the analytical syntax of
“Who did what to whom, when, where and why?”
• provides guidance on how to verify each case and on the data collection and compilation process.

Source: OHCHR, Metadata and Technical Guidance on SDG 16.1.2 on conflict-related deaths, available at https://www.ohchr.org/EN/Issues/Indicators/
Pages/SDGindicators.aspx

and services (e.g. economic slowdown, shortages of medicines or reduced farming capacity resulting
in lack of access to adequate food, water, sanitation, health care and safe conditions of work) that is
caused or aggravated by the situation of armed conflict.

HANDBOOK ON GOVERNANCE STATISTICS | 205

 casualties. In most instances, the number of cases reported will depend on access to conflict
zones, access
480. Into information,
situations and
of armed the motivation
conflict, a large shareand perseverance
of deaths of may
and injuries possible
remaindata providers.
unreported.
Often, the presence of conflict heavily affects normal registration systems. Additionally, actors on both
Depending
sides ofona conflict
the availability and quality
may have incentives of administrative
for misreporting, deflatingdata over the
or inflating course Inofmost
casualties. the in-
armed
conflict, statistical
stances, the numbersurveys and
of cases techniques
reported mayonbe
will depend used
access to estimate
to conflict undocumented
zones, access to information, direct
conflict-related deaths. National administrative datasets with sufficiently well documented
and the motivation and perseverance of possible data providers.
direct deaths constitute an essential source for further statistical analysis and estimations of
481. Depending on the availability and quality of administrative data over the course of the armed
undocumented direct deaths. As indirect deaths would typically fall outside the scope of
conflict, statistical surveys and techniques may be used to estimate undocumented direct conflict-re-
commonlatedcasualty recording
deaths. National practices datasets
administrative (that rather focus on well
with sufficiently direct deaths), they
documented direct may
deathsbeconsti-
captured
using tute
additional
an essential source for further statistical analysis and estimations of undocumented direct deaths. of
administrative records and/or statistical surveys allowing the measurement
excessAsmortality, namely
indirect deaths wouldall the deaths
typically (direct
fall outside and indirect)
the scope of commonthat would
casualty not have
recording occurred
practices (that in
time ofrather
peace, ason
focus defined and measured
direct deaths), they mayby beepidemiologists.
captured using additional administrative records and/
or statistical surveys allowing the measurement of excess mortality, namely all the deaths (direct and
The methodology for these
indirect) that would estimations
not have will ofbepeace,
occurred in time further developed
as defined in collaboration
and measured with NSOs,
by epidemiologists.
NHRIs, UN entities and CSOs.
482. The methodology for these estimations will be further developed in collaboration with NSOs,
NHRIs, UN entities and CSOs.
X.3.5 Estimates of conflict fatalities
X.3.5 Estimates of conflict fatalities
X.3.5.1 Description
X.3.5.1 Description
Victims of armed conflicts are very rare compared to homicides, and very few conflicts result
in death483.
ratesBroadly
comparable with
speaking, the of
victims toparmed
countries in the
conflicts homicide
are very statistics.
rare compared Figure X.1
to homicides, andillustrates
very
the difference
few conflictsbetween homicide
result in direct statistics
death rates and statistics
comparable with those on conflict-related
recorded deaths.
in countries with The red
high levels
pointsofare global homicide
intentional statistics,
homicide. Figure ranked from
X.2 illustrates highest between
the difference to lowest. The black
homicide points are armed
and conflict-related
conflict death
death ratesrates,
for thealso
yearranked
2015. Thefrom highest
red points to lowest. In
are country-level 2015, rates,
homicide which wasfrom
ranked among
highestthetomost
lethal years in recent history, the most lethal conflict (Syria) had a higher rate of persons to
lowest. The black points are armed conflict death rates at national level, also ranked from highest killed
than Ellowest.
Salvador,
In 2015,which wasamong
which was on top
the of thelethal
most homicide rate statistic
years in recent formost
history, the thatlethal
year. However,
conflict (Syria) the
black points quickly drop towards zero.31 The gap between the red and the black points show
that globally homicide is by far the larger challenge for individual lives and safety.
FIGURE X.2 Comparison of homicide statistics and armed conflict
Figure X.1. Comparison of homicide statistics and armed conflict death rates (per 100,000
death rates (per 100 000 inhabitants), 201520
inhabitants)

However, armed conflict often coincides with a limited governmental capacity, which means
that the governmental
20 Data institutions
for Figure come from UCDP responsible
for battle deaths for the
and UNODC for homicide collection
statistics, of itofficial
sources cited above; includes moststatistics
countries in themay
world. very
well be unable to do so. In such situations, estimates based on official and unofficial death
206 |
31
If we had high-quality subnational data on armed conflict deaths, we would probably see an even starker contrast. As
had a higher rate of persons killed than any other country, including countries with high homicide
rates. However, the black points quickly drop towards zero. The gap between the red and the black
points show that globally homicide is by far the largest challenge for individual lives and safety.

484. However, armed conflict often coincides with a limited governmental capacity, which means
that governmental institutions responsible for the collection of official statistics may very well be una-
ble to do so. In such situations, estimates based on unofficial death counts produced by experts may
offer a valid approach to produce statistical figures consistent over time and across countries.

485. Expert estimates of conflict fatalities are based mainly on news reports and other eyewitness
accounts. Such reports are not equally available across the world, and they are vulnerable to manipu-
lation by the parties to the conflict and others.21

486. However, by using transparent and objective validation rules, international and third-party
experts can make valid use of available information on conflict fatalities. Independent expert groups
exist – with a proven track record in this field – and while differing in their coverage and definitions,
they offer interesting results for outside validation. Among these groups are the Uppsala Conflict Data
Programme22 (UCDP) and the Armed Conflict Location Event Data Project23 (ACLED).

487. These two projects differ in several ways. UCDP covers the entire world, whereas ACLED fo-
cuses on the world’s most conflict-prone regions (but still covers most conflicts). UCDP has a strict
definition of conflict that is applied consistently to all conflicts, and reports on a given year in the
following May. ACLED has a more pragmatic approach and works closely with local partners to gain as
much information as possible. As a consequence of this different approach, ACLED typically reports
higher casualty figures than UCDP.24

X.3.5.2 Methodological and quality considerations

488. A key issue when measuring conflict-related deaths is to be able to apply the same definitions
and methods over time and for different countries. First, a clear definition for differentiating between
armed conflict and other forms of violence is required. Second, a clear definition of which deaths
should be counted as resulting from an armed conflict is required. Third, it is important to make use of
available sources of information in a transparent manner.

489. The approaches developed by UCDP and ACLED meet the three criteria outlined above in
different ways and, accordingly, provide different statistical results. On the other end, thanks to the
consistent use of the same concepts and methods, these initiatives provide data that can be useful for
the long-term monitoring of conflict violence.

21 Weidman (2015) compared UCDP’s coding of conflict-related deaths in Afghanistan with leaked US reports known as SIGACTS, and found that larger
events were more likely to be included in the former, and that while the location of an event could be quite off, the number of casualties was generally
unbiased.
22 https://ucdp.uu.se
23 https://acleddata.com
24 The merits and reliability of ACLED and UCDP GED are subject to technical discussion. See http://www.acleddata.com/wp-content/uploads/2015/04/
ACLED_Reporting-Sources-Working-Paper-No.-5_2015.pdf and https://journals.sagepub.com/doi/abs/10.1177/0010836711434463 for supportive argu-
ments from the two projects. Global assessments of how well the two datasets correlate can be misleading, as there can be large local differences with
significant consequences for the performance of individual countries.

HANDBOOK ON GOVERNANCE STATISTICS | 207

 490. The UCDP bases its calculations on the following definition of armed conflict: “An armed
conflict is a contested incompatibility that concerns government and / or territory where the use of
armed force between two parties, of which at least one is the government of a State, results in at least
25 battle-related deaths in one calendar year” (Gleditsch and others, 2002). Battle-related deaths in-
cluded in UCDP’s calculations include those that result from: “the use of armed force between warring
parties in a conflict dyad, be it State-based or non-State, resulting in deaths”; one-sided violence,
defined as the “use of armed force by the government of a State or by a formally organized group
against civilians which results in at least 25 deaths in a year”; and non-State violence, defined as the
“use of armed force between two organized armed groups, neither of which is the government of a
State, which results in at least 25 battle-related deaths in a year”.25

491. The ACLED definition starts out with a typology of different events, which are then described
in detail. Among the dimensions coded is the name and type of actor, from which the type of conflict
can be derived. An event that includes a military actor and a rebel actor will be classified as an armed
conflict, but there is no minimum threshold that this dyad must pass in order to qualify. ACLED also
included non-violent events such as protests.26

492. A somewhat more encompassing attempt at measuring and recording deaths due to armed
conflict is a definition used by Every Casualty: “recording of deaths from armed conflict only, though
the term casualty can also include people who are injured”. This approach focuses on documenting
either “the deaths of individual people from conflict violence (e.g. listing individual victims and the
circumstances of their deaths)” or “separate events or incidents in which deaths from conflict violence
occurred (e.g. listing dates and places of separate incidents of violence and the numbers killed in
each)” (Minor, 2012).

493. In counting direct conflict deaths, the Geneva Declaration Secretariat approach in its GBAV
database is to record victims of lethal violence in different settings affected by collective or organized
forms of violence or armed conflict. Various incident-based reporting sources are integrated in this
process; the applied methodology is to choose the best available estimate for each country identified
as suffering from armed conflict. For more information, see the online methodological annex of the
2011 edition of the GBAV (Geneva Declaration Secretariat, 2012).

494. Victims of terrorism are not necessarily accounted for in the data on conflict-related deaths, al-
though most are generally recorded in the databases that cover conflict countries. Defining terrorism
is a difficult matter, and the circumstances of terrorist killings in situations of armed conflict may mean
such deaths are not recorded in conventional statistics on homicide; in theory, “deaths as a result of
terrorist activities”27 should be recorded as intentional homicides, but this does not solve the question
of which deaths should be counted as victims of terrorist violence in a concrete conflict situation. One
point to note is that victims of terrorism are often included in conflict deaths data (Geneva Declaration
Secretariat, 2011). The main existing international data sources on victims of terrorism are the Global
Terrorism Database, the International Institute for Strategic Studies and the National Counterterrorism
Center (US) (Geneva Declaration Secretariat, 2015, p. 35).

25 https://www.pcr.uu.se/research/ucdp/definitions/
26 https://www.acleddata.com/resources/general-guides/
27 According to the ICCS, “Death as a result of terrorist activities” refers to killing due to an act intended to cause death or serious bodily injury by a
person who is not a combatant (i.e. party to a conflict), when the purpose of such an act, by its nature or context, is to intimidate a population, or to compel
a government or international organization to do or abstain from doing any act. (United Nations General Assembly. International Convention for the
Suppression of the Financing of Terrorism. 1999. E/RES/54/109, Article 2(1b).

208 |
X.3.5.3 Measuring conflict-related deaths beyond expert assessments

495. Sample surveys, applied after a conflict subsides, have also been used to measure conflict-re-
lated deaths. Such surveys can be used to measure all three parts of the conflict death definition.
Specifically, household surveys known as retrospective mortality surveys (RMSs) can be used to ask
respondents about information on past violent and non-violent deaths. The accuracy and broader util-
ity of such surveys, however, remains unclear. Perhaps the most famous example is the second Lancet
study in Iraq, which made a claim of a much higher casualty estimate than any other source (Burnham
and others, 2006). The Lancet study exemplifies three main challenges that survey-based estimations
have yet to overcome: methodology, ethics and costs. The main methodological issues are the sam-
pling procedures and the estimates for pre-war mortality. The ethical issues became apparent as an
independent investigation concluded that the Lancet study was not executed according to the ethical
standards of the industry, in addition to “some evidence suggesting” that the survey data was in part
fabricated (Spagat, 2010). Finally, there is a financial issue. The costs of performing a single survey are
very high compared to alternative methodologies. Beyond the issues of the Lancet survey, epidemio-
logical surveys, yielding more reliable results, have also been used to measure indirect deaths in, for
instance, the on-going war in Syria (Guha-Sapir, 2015).

496. Multiple systems analysis (MSE) has also been used to estimate violent deaths, most famously
by the Human Rights Data Analysis Group (HRDAG). HRDAG used MSE to, for instance, estimate the
number of deaths in the civil war in Guatemala. MSE can be used when you have access to several
incomplete lists of, for instance, war deaths that are partially overlapping. The MSE methodology has
been extensively tested and showed to produce reliable estimates, and results from MSE analyses
of deaths have been used in court proceedings and by truth and reconciliation commissions (see
Chapman and Ball, 2001).

X.3.6 Administrative data on the performance of law enforcement and criminal justice institutions

X.3.6.1 Description

497. Administrative crime and criminal justice data can be used to provide information on overall
State response to crime and to offer insights into the performance of law enforcement and criminal
justice institutions, in particular in relation to dedicated resources and their activities.

Resource data
498. Resource data quantify the costs of administering the justice system. They include items such
as the number of persons employed, the functions of persons employed, expenditures on wages and
salaries, operating costs and revenues (UN DESA, 2003).

499. The key types of resource indicators by criminal justice agency include:

• Police personnel or law enforcement personnel: “Personnel in public agencies whose princi-
pal functions are the prevention, detection and investigation of crime and the apprehension
of alleged offenders”.
• Prosecution personnel: “Officials whose duty is to initiate and maintain criminal proceedings
on behalf of the State in relation to a criminal offence”.
• Court personnel: “Officials authorized to hear specifically criminal cases, including in appeal
courts, and to make dispositions in a court of law”.

HANDBOOK ON GOVERNANCE STATISTICS | 209

 • Prison personnel: “All individuals employed in penal or correctional institutions, including
management, treatment, custodial and other (maintenance, food service, etc.) personnel”.

Performance indicators
500. The United Nations Manual for Criminal Justice Statistics (2003) and the UNODC Criminal
Justice Assessment Toolkit (2006) highlight the importance of producing “response/ attrition”28 rates
in order to develop performance indicators for the criminal justice system.

501. To calculate actual attrition rates, it would be necessary to follow the path of individual cases
throughout the criminal justice system (Heiskanen and others, 2014). Recording systems employing
Integrated File Numbers (IFN) allow to directly follow a “cohort” of individuals through the system,
from arrest to prosecution to acquittal/conviction. In this case, the actual attrition rate would measure
the percentage out of the same cohort of persons arrested who were prosecuted and the percentage
who were convicted (UNODC, 2010a; UN DESA, 2003). However, few countries have the possibility of
exactly combining person-related data from different sources in the field of crime and criminal justice
statistics, and even when this is possible, the linkage does not necessarily cover all parts of the criminal
justice process (Heiskanen and others, 2014).

502. An approximation to actual attrition rates may be calculated using ratios between the numbers
of cases or persons at the various stages of the criminal justice process, as recorded at a specific point
in time. Approximate attrition rates do not follow the same individuals or cases through the system
but refer to individuals or cases managed during the same reference period, for example a calendar
year. These ratios can normally be computed at country level and produce valuable information to
assess the performance of the criminal justice system. Several types of approximated attrition rates
can be computed, as for example:

1) Police: clearance ratio (cases for which an offender can be identified per all cases known to the
police).
2) Prosecution: prosecution ratio (suspects or cases passed on to the prosecution service per sus-
pects or cases known to the police); indictment ratios: persons/cases brought before a court
by the prosecution service per persons/cases prosecuted or persons/cases brought before a
court by the prosecution service per suspects/cases known to the police.
3) Conviction: conviction ratios: persons convicted per persons brought before a court or per-
sons convicted per suspects known to the police.
4) Prison: punitivity ratio, for example as all persons sent to prison in a given year, per all convict-
ed persons.29

503. Other combinations of the above, such as the ratio between persons sentenced and suspects
known to the police, are theoretically possible. However, some methodological caveats should be
considered.30

28 Attrition can be defined as the “loss” of cases or, more technically, the filtering out of cases during the criminal justice process (Jehle, 2012; Heiskanen
and others, eds., 2014).
29 “Developing a Methodology to Collect Data on Community Sanctions and Measures and Attrition Rates in Europe” (DECODEUR): project funded by
an EU action grant and carried out by the experts group for the European Sourcebook of Crime and Criminal Justice Statistics (ESB) together with experts
from CEP, the Confederation of European Probation.
30 For a detailed assessment of the indicators see Heiskanen and others (2014).

210 |
Workload indicators
504. Resource data – when combined with data on activities by the various institutions – can pro-
vide workload indicators for the various law enforcement and criminal justice agencies (UN DESA,
2003; Smit and Harrendorf, 2010), as for example:

• Average number of criminal offences and persons suspected per police officer (Smit and
Harrendorf, 2010).
• Average number of persons prosecuted per prosecutor (ibid.).
• Average number of persons brought before a court per prosecutor (ibid.).
• Average number of persons convicted per prosecutor (ibid.).

505. Other indicators related to the timeliness and length of the criminal proceedings (UNODC,
2010a; Smit and Harrendorf, 2010) can be calculated as follows:

• Timeliness of prosecution decisions and actions (prosecution).

• Average length of trials (court).
• Average time spent in pretrial detention (court).

506. At international level, the main initiatives collecting administrative data on law enforcement
and criminal justice institutions, which also allow to compute many of the above-mentioned indicators,
are: the United Nations Survey of Crime Trends and Operations of Criminal Justice Systems (UN-CTS),
Eurostat, and the Council of Europe Annual Penal Statistics (SPACE).

BOX X.9 Law Enforcement Core Statistics (LECS) – United States

The Law Enforcement Core Statistics Program (LECS) is coordinated by the Bureau of Justice Statistics (BJS)
of the United States. For over 25 years, this Bureau has been the authoritative source for national statistics
regarding the personnel, operations, policies, and procedures of law enforcement agencies (LEAs). The BJS
develops two main data collections: 1) the Law Enforcement Management and Administrative Statistics
(LEMAS), which is the BJS’ core law enforcement data collection programme, and 2) the Census of State and
Local Law Enforcement Agencies (CSLLEA). The LECS combines these two collections into a cohesive law
enforcement collection programme and reshapes their content to focus on agency performance as well as
contemporary topics related to agency staffing, regulations and policies. In particular, the LEMAS collects data
“on agency responsibilities, operating expenditures, job functions of sworn and civilian employees, officer
salaries and special pay, demographic characteristics of officers, weapons and armour policies, education and
training requirements, computers and information systems, vehicles, special units, and community policing
activities” (https://bjslecs.org/).

X.3.7 Surveys on law enforcement and criminal justice institutions (population surveys and
surveys of employees of these institutions)

X.3.7.1 Description

507. Collecting information on the activities of law enforcement and criminal justice institutions
through surveys is important for strategic planning and for assessing their performance. Similarly, as-
sessing the level of public trust in justice institutions is crucial to evaluate the rule of law and in order
to better target criminal policies and support the legitimacy of justice institutions. Legitimacy can

HANDBOOK ON GOVERNANCE STATISTICS | 211

 affect law-related behaviour (Jackson and others, 2012) and thus influence the overall level of safety
and security.

508. Surveys on law enforcement and criminal justice institutions can be conducted among:

• The general population or business representatives, to collect their experiences and opinions
on law enforcement and justice institutions.
• Employees of law enforcement and criminal justice institutions, to collect information on their
working environment and activities.

General population and business surveys on law enforcement and justice institutions
509. Surveys collecting information on law enforcement and justice institutions from the general
population can be of three main types:

1) Larger multi-purpose surveys may include a module on trust in public institutions. See the
chapter in this Handbook on “Trust” for further detail.
2) Victimization surveys and surveys on access to justice cover concrete experiences of victims of
crime when turning to official authorities to address injustices and receive assistance. Besides
general information on victims and their experiences with crime, these surveys often also cover
the reporting of crime (whether and to whom the crime was reported), the response of the
police or other authorities, and satisfaction with this response. See the Handbook chapter on
access to and quality of justice for further detail.
3) Community surveys are usually developed by law enforcement agencies to collect informa-
tion from local residents on crime, fear of crime and satisfaction with law enforcement. This
information can be of great value both in strategic planning, but also in assessing the past
performance of law enforcement agencies (see Henning and others, 2017).

Surveys among officers of law enforcement and justice institutions

510. These surveys can be conducted potentially among all actors of the criminal justice systems
(police officers, prosecutors, judges and correctional officers) and aim to collect information on the
attitudes, behaviours and activities of the persons working within law enforcement and justice institu-
tions (Nix and others, 2019):

• Surveys of police officers are the most frequently developed. Nix and others (ibid.) identified
497 police surveys reported in journal articles from 2008 to 2017. A recent and frequently
quoted example is Skogan’s survey of police officers in Chicago developed in 2015.
• Surveys of prosecutors aim to provide data on prosecutorial activities as well as on a variety
of administrative and legal issues facing prosecutors. The United States National Survey of
Prosecutors is a consolidated example of this type of survey.31
• Correctional officer surveys include questions on job satisfaction, work stress, personal safety
and security, attitudes towards inmates and professional orientation.32 Recent examples of
this type of survey are the California Correctional Officer Survey developed in 2006 and the
European Public Service Union survey of the impact of the economic crisis on prisons, devel-
oped in 2015 across 12 European countries.33

31 https://www.bjs.gov/index.cfm?ty=dcdetail&iid=265
32 https://amylerman.weebly.com/uploads/2/3/4/0/23402306/attitudes_towards_rehabilitation.pdf
33 https://www.epsu.org/sites/default/files/article/files/Prison%20staff%20perspectives%20EPSU%20survey%20June%202016%20EN.pdf

212 |
BOX X.10 National Survey of Standards and Police Professional Training (ENECAP)

As a federal country, public security in Mexico is provided by municipal, state and federal polices with varying
degrees of resources and training. The strengthening of police forces is often a topic in the public agenda, since
they constitute a keystone element to address the security crises affecting Mexico in the last few years. For
this reason, the National Statistical Office (INEGI) conducted a survey on police officers across the country so
as to provide evidence to improve policing functions. The survey implementation had to face methodological
challenges (such as the design of representative samples as there was no comprehensive listing of police
forces) and operational obstacles to interview members of various police agencies. The National Survey of
Standards and Police Professional Training produced a vast set of statistics on the sociodemographic profile and
employment conditions of police officers, including on police tasks, officers’ training, degree of specialization
and equipment. Selected results are reported in the charts below, which demonstrate that thanks to such a
survey it is possible to assess some of the challenges faced by law enforcement authorities in discharging their
duties and to identify where policy interventions are needed.

Percentage of officers who agree additional training is needed (by subject)

The adversarial Justice System 82

Giving testimony (presenting evidence – physical and intelligence) 76

Rules and Procedures (SOP) 75

Mediation and Conflict Resolution 74

Assistance to Vulnerable Populations 73

Computer Studies 71

Personal Defense 72

Arrest and Control Tactics & Use of Force 69

Interview Techniques (Suspects and Witnesses) 68

Mechanisms for Interacting with Citizens 67

Writing & Grammar 67

Police officers who reported being victims of crimes or anti-social behavior while on duty (%)

Victim Not a victim

National Average 35.6 64.4

Federal Police 28.5 71.5

Investigative Federal Police 20.1 79.9

Preventive State Police 40.9 59.1

Investigative State Police 23.4 76.6

Preventive Municipal Police 36.1 63.9

Source: INEGI, National Survey of Standards and Police Professional Training (ENECAP) 2017.

HANDBOOK ON GOVERNANCE STATISTICS | 213

 X.3.7.2 Methodological and quality issues
Resource indicators
511. Computing resource indicators might face some methodological challenges, mainly in terms
of data availability and comparability across countries.

512. When measuring the police personnel at country level, the main problem arises from the
existence of different police forces, from the local to the national or federal level. This can have an
impact when comparing police personnel figures across countries.

513. Similar issues exist when counting and comparing personnel of other institutions in the crim-
inal justice system (prosecution, courts and prison administration). The UN-CTS provides standard
definitions for producing statistics on the personnel of all relevant institutions and that can promote
international comparisons.

514. With regard to prison statistics, specific issues exist in measuring and comparing the number
of places available, as the “official capacity” of prisons is mainly subject to national definitions, which
does not necessarily imply that certain minimum standards are taken into account.

Administrative performance indicators

515. Attrition rates can be calculated level by level and on the overall criminal justice process. Inter-
level attrition ratios become more valuable when calculated for specific offences (e.g. focusing on the
number of persons brought to court and on those convicted for rape).

516. Cross-national comparisons of police “clearance” data are recognized to be extremely com-
plex, due to different national criteria for solving crimes, different counting units for recorded crime,
differing obligations to prosecute, and different degrees of police discretion in case-handling (see for
example Smit, Meijer and Groen, 2004). Nonetheless, measures of cases solved or cleared (according
to the national definition) are crucial for the measurement of police effectiveness within the national
context.

517. Another issue concerns potential changes in the classification of a crime between the initial
recording of the event and the subsequent in-depth investigation. If data are collected for statistics at
the point of initial incident recording, then different crime numbers would be reported compared to
a situation where data are collected after investigation. There might also be a lack of harmonization
between the crime classification system used to classify the crime incident (and subsequent report-
ed crime statistics) and the charge assigned to the offender (under the criminal code). “Depending
upon the national system, these two categories may or may not be equivalent. A country may, for
example, use broad descriptive codes for classifying the crime event, but a technical, detailed list of
legal offences for the charge procedure. Where this is the case, work is required to carefully ‘translate’
legal charges as they correspond to each crime classification” (UNODC, 2010a). The International
Classification of Crime for Statistical Purposes (ICCS) provides a standard crime classification system
that can be applied across different stage of the criminal justice process, thereby ensuring compara-
bility of criminal offence categories between law enforcement and justice institutions.

214 |
X.4 Recommended key indicators
518. The 2030 Agenda for Sustainable Development calls on Member States to “significantly re-
duce all forms of violence and related death rates everywhere” (Target 16.1). The four SDG indicators
under this target constitute the core set of indicators to monitor crime and safety: indicators 16.1.1,
16.1.2, 16.1.3, and 16.1.4. In order to produce data that are suitable to address the UN-SDG principle
of “leaving no one behind”, such data should be disaggregated whenever relevant (e.g. by sex, age,
location, disability status, etc.).

519. In addition to this core set, further SDG indicators can be used to expand countries’ under-
standing of patterns and trends in safety and security. Three recommended indicators under target
16.2 refer to ending all forms of violence against children. Two indicators under target 5.2 monitor
progress towards the elimination of all forms of violence against women and girls. Under SDG 4 on ed-
ucation, there are two indicators about school safety that specifically refer to the situation of students
and their teachers: (SDG 4.a.2 – “Percentage of students experiencing bullying in the last 12 months”
and SDG 4.a.3 – “Number of attacks on students, personnel and institutions”. Under SDG 11 on safer
cities, indicator 11.7.2 refers to the “Proportion of persons victim of physical or sexual harassment, by
sex, age, disability status and place of occurrence, in the previous 12 months”. These indicators can
be considered as further measures of safety and security for specific groups and contexts. Lastly, SDG
indicator 16.4.2 refers to the seizure of illicit arms and the State response to the trafficking of weapons
and explosives, and it provides an indirect measure of security in relation to the availability of illicit and
untraced arms in the community.

520. In terms of key indicators beyond the SDG framework, Table X.2 recommends crucial process
indicators to be collected on the criminal justice process and the capacity of the criminal justice insti-
tutions, as well as an expended set of outcome indicators that can be collected through population
(mostly victimization) surveys. Annex X provides recommended survey questions for this latter group
of key indicators collected through surveys.

HANDBOOK ON GOVERNANCE STATISTICS | 215

TABLE X.2 Recommended key indicators

Sub-dimension Indicator Data source Type

Levels and patterns of crime, Number of victims of intentional homicide per 100,000 Administrative data Outcome
and perceptions of safety population, by sex and age (SDG indicator 16.1.1)

Levels and patterns of crime, Proportion of population subjected to (a) physical violence, (b) psychological Administrative data, Outcome
and perceptions of safety violence and (c) sexual violence in the previous 12 months (SDG indicator 16.1.3) Population surveys

Levels and patterns of crime, Proportion of population subjected to physical and sexual harassment Population surveys Outcome
and perceptions of safety

Levels and patterns of crime, Proportion of population that feel safe walking alone Population surveys Outcome
and perceptions of safety around the area they live (SDG indicator 16.1.4)

Levels and patterns of crime, Proportion of population that feel safe when alone at home Population surveys Outcome
and perceptions of safety

Levels and patterns of crime, Number of verified cases of killing, kidnapping, enforced disappearance, arbitrary Administrative data Outcome
human rights violations/abuses, detention and torture of journalists, associated media personnel, trade unionists
and perceptions of safety and human rights advocates in the previous 12 months (SDG indicator 16.10.1)

Casualties linked to conflicts Conflict-related deaths per 100,000 population, by sex, Administrative data, Outcome
age and cause (SDG indicator 16.1.2) Estimation techniques

Levels and patterns of crime, Proportion of children aged 1–17 who experienced any physical punishment and/ Population surveys, Outcome
and perceptions of safety or psychological aggression by caregivers in the past month (SDG indicator 16.2.1) Violence against
children surveys

Levels and patterns of crime, Number of victims of human trafficking per 100,000 population, by Population surveys, Outcome
and perceptions of safety sex, age and form of exploitation (SDG indicator 16.2.2) administrative data,
estimation techniques

Levels and patterns of crime, Proportion of young women and men aged 18-29 who experience Population surveys Outcome
and perceptions of safety sexual violence by age 18 (SDG indicator 16.2.3)

Levels and patterns of crime, Proportion of ever-partnered women and girls aged 15 and older subjected to Violence against Outcome
and perceptions of safety physical, sexual or psychological violence by a current or former intimate partner Women surveys
in the previous 12 months, by form of violence and by age (SDG indicator 5.2.1)

Levels and patterns of crime, Proportion of women and girls aged 15 and older subjected to sexual Violence against Outcome
and perceptions of safety violence by persons other than an intimate partner in the previous 12 Women surveys
months, by age and place of occurrence (SDG indicator 5.2.2)

Levels and patterns of crime, Proportion of persons victim of physical or sexual harassment, by sex, age, disability Population surveys Outcome
and perceptions of safety status and place of occurrence, in the previous 12 months (SDG indicator 11.7.2)

Levels and patterns of crime, Proportion of seized, found or surrendered arms whose illicit origin Administrative data Outcome
and perceptions of safety or context has been traced or established by a competent authority
in line with international instruments (SDG indicator 16.4.2)

Performance of law enforcement Indictment ratio (persons brought before a court per persons prosecuted) Administrative data Outcome
and criminal justice institutions

Performance of law enforcement Conviction ratio (persons convicted per persons brought before a court) Administrative data Outcome
and criminal justice institutions

Resources of law enforcement Existence of independent national human rights institutions in Administrative data Structural/
and criminal justice institutions compliance with the Paris Principles (SDG indicator 16.a.1) Process

216 |
TABLE X.2 Recommended key indicators, CONT.

Sub-dimension Indicator Data source Type

Resources of law enforcement Number of police personnel per 100,000 population, by sex and function Administrative data Process
and criminal justice institutions

Resources of law enforcement Number of prosecution personnel, per 100,000 population by sex Administrative data Process
and criminal justice institutions

Resources of law enforcement Proportions of positions (by sex, age, persons with disabilities and population Administrative data Process
and criminal justice institutions groups) in the judiciary, compared to national distributions (SDG indicator 16.7.1c)

Resources of law enforcement Number of prison staff per 100,000 prisoners, by sex and function Administrative data Process
and criminal justice institutions

Resources of law enforcement Prison overcrowding ratio (total number of prisoners divided by total number of Administrative data Process
and criminal justice institutions beds/official capacity of prisons, penal institutions or correctional institutions)

Workload of law enforcement Number of prisons, penal institutions or correctional institutions Administrative data Process
and criminal justice institutions

Workload of law enforcement Number of criminal offences per police officer and persons suspected per police officer Administrative data Process
and criminal justice institutions

Workload of law enforcement Number of persons prosecuted per prosecutor Administrative data Process
and criminal justice institutions

Workload of law enforcement Number of persons brought before a court per prosecutor Administrative data Process
and criminal justice institutions

Punitivity of the criminal Number of persons convicted per prosecutor Administrative data Process
justice system

Quality of law enforcement and Sentenced persons incarcerated per persons convicted Population surveys Process
criminal justice institutions

Quality of law enforcement and Proportion of victims of violence in the previous 12 months who Population surveys Outcome
criminal justice institutions reported their victimization to competent authorities or other officially
recognized conflict resolution mechanism (SDG indicator 16.4.2)

Quality of law enforcement and Satisfaction with the work done by criminal justice Population surveys Outcome
criminal justice institutions authorities in relation to a specific crime

Quality of law enforcement and Perceptions of effectiveness of the police and criminal justice institutions Population surveys Outcome
criminal justice institutions

Quality of law enforcement and Measures of trust in the police and criminal justice institutions Population surveys Outcome
criminal justice institutions

Quality of law enforcement and Perceptions of effectiveness of the police and criminal justice institutions Population surveys Outcome
criminal justice institutions

Quality of law enforcement and Measures of trust in the police and criminal justice institutions Population surveys Outcome
criminal justice institutions

HANDBOOK ON GOVERNANCE STATISTICS | 217

 X.5 Way forward
521. This chapter has provided a set of recommended key indicators in the area of safety and
security and has laid out some of the main methodological challenges to be taken into account. To
improve the quality and availability of data on these key indicators, data providers should first focus on
adopting a methodology and data collection process that conforms to the international best practices
and standards outlined above, and to make data on the indicators available in a regular and timely
manner.

522. Progress in the production of safety and security statistics will rest on an overall strengthening
of the crime and criminal justice statistics system. This involves a number of elements:

• The progressive implementation of the International Classification of Crime for Statistical

Purposes (ICCS) is a key requirement to enhance the consistency and (internal and exter-
nal) comparability of crime and criminal justice statistics. Implementing the ICCS involves a
series of steps, such as drawing up correspondence tables between current national crime
classifications, reviewing the definitions and concepts applied, and adapting data collection
instruments currently used to make them compliant with ICCS categories. Detailed guidance
on this process and its various steps will be available in the forthcoming UNODC Manual on
ICCS Implementation.
• Adopting and following a consistent and logically coherent statistical classification is an im-
portant aspect of data quality, but other aspects of data quality should also be taken into
account and improved. Such dimensions of data quality include aspects such as compara-
bility, completeness, timeliness, internal and external consistency of the data to name but
a few (see also the Handbook chapter on “Cross-cutting issues”).34 Putting in place a com-
prehensive data quality policy for safety and security data will greatly enhance the value
and trust in the indicators derived from such data. This should also include safety and ethical
considerations for the collection of some type of relevant safety and security data, such as
violence against women and girls.
• Involving NSOs in developing concepts, categories and classifications for collecting and
producing statistics on safety and security as well as for elaborating detailed recording and
coding rules.
• Improving data quality and availability requires an improved coordination of the relevant
data providers on crime and criminal justice at country level. It is good practice to advance
this work through an inter-institutional working group or task force with a common goal and
mandate. Based on the outputs of this task force, data on crime and criminal justice should
become more comparable and useful both between institutions within the country and be-
tween countries, thus allowing a better measurement and analysis of safety and security is-
sues. In fact, the experience of most countries that have started to work on this demonstrates
that endorsing and implementing the ICCS is an excellent starting point for such improved
coordination between institutions at the national level: to adopt and develop a uniform classi-
fication at the country level on the basis of the ICCS that can serve as a common “language”
for all crime and criminal justice institutions – police, prosecution, courts, prisons and NSOs.

34 As an example of a simple quality framework for crime and criminal justice data, the Methodological Annex to the 2019 Global Study on Homicide
provides a list of data quality scores for homicide data for 240 countries and territories based on the five named dimensions. See: https://www.unodc.org/
documents/data-and-analysis/gsh/Meth_Annex_GHS.pdf. Another important reference is the United Nations National Quality Assessment Frameworks
Manual for Official Statistics (2019), which identifies the following key dimensions of quality for statistical products: relevance, accuracy, reliability, timeli-
ness, punctuality, accessibility, clarity, coherence, consistency, comparability. See: https://unstats.un.org/unsd/methodology/dataquality

218 |
 • Better coordination among the major data producers of administrative data on crime and
criminal justice will also strengthen the case for producing complementary data on the
prevalence of crime through victimization surveys, as well as governance surveys, surveys on
corruption and access to justice and also specialized surveys on violence against women. A
common task force that brings together the relevant stakeholders can provide both expertise
on crucial data needs that are important for developing a suitable, nationally adapted, survey
instrument and a link to the major users of the data generated through surveys: policy makers
and practitioners in the field of safety and security.
• Implement the methodology for SDG indicator 16.1.2 on conflict-related deaths as approved
by the IAEG-SDGs and continue collaboration to develop a comprehensive and consistent
statistical framework integrating conflict deaths and intentional homicides.
• In the longer term, a strengthened statistical system on safety and security indicators should
be able to exploit and integrate new data sources into the analysis of safety and security. Such
new sources will be contingent on ongoing social and technical developments (e.g. spread
of geospatial data recording systems). New data sources should be discussed, evaluated and
used by a broad group of stakeholders, rather than by specialized departments within law
enforcement agencies only. Examples of new data sources include new types of reporting
and recording mechanisms (e.g. citizen-based reporting on security incidents through online
platforms) and the use of “big data” for the generation of estimates.35

35 An example is the Global Terrorism Database, which is an open-source database that uses information on terrorist events around the world. See:
https://www.start.umd.edu/gtd/

HANDBOOK ON GOVERNANCE STATISTICS | 219

 LIST OF REFERENCES
Acemoglu, D., J. Robinson and S. Johnson (2001). “The Colonial origins of comparative
development: An empirical investigation”, The American Economic Review, vol. 91,
pp. 1369-1401.

Acock, A., and H. Clarke (1990). “Alternative measures of political efficacy: Models and means”,
Quality and Quantity, vol. 24, pp. 87-105, Kluwer Academic Publishers.

Adams, K., and others (2018 and 2017). Global Insights on Access to Justice. Washington, D.C.:
World Justice Project.

Adserà, A., C. Boix and M. Payne (2003). “Are you being served? Political accountability and
quality of government”, The Journal of Law, Economics, and Organization, vol. 19, No. 2,
pp. 445-490.

Aebi, M.F. (2008). “Measuring the influence of statistical counting rules on cross-national
differences in recorded crime”, In Crime and Criminal Justice Systems in Europe and North
America 1995-2004, Kauko Aromaa and Markku Heiskanen, eds., HEUNI Publication Series,
No. 55, Helsinki: European Institute for Crime Prevention and Control, affiliated with the
United Nations, pp. 196-214.

African Union (2019). Strategy for the Harmonization of Statistics in Africa (SHaSA). Available at:
https://au.int/en/ea/statistics/shasa

African Union and United Nations Development Programme (2017). Governance, Peace and
Security (GPS), Stock-Taking report 2012-2015. Addis Adaba [Eth]: UNDP.

Alesina, A., and E. Spolaore (1997). “On the number and size of nations”, Quarterly Journal of
Economics, vol. 112, No. 4, pp. 1027–1056.

Algan Y. (2018). “Trust and social capital”, Chapter 10 in For Good Measure: Advancing
Research on Well-Being Beyond GDP, Stiglitz, Fitoussi and Durand (eds.). Paris: OECD
Publishing.

Algan, Y., and P. Cahuc (2014). “Trust, growth and well-being: New evidence and policy
implications”, IZA Discussion Paper, No. 7464.

Alt, J., D. Lassen and D. Skilling (2002). “Fiscal transparency, gubernatorial approval, and
the scale of government: evidence from the states”, State Politics & Policy Quarterly, 2(3),
230-250. Available at: http://www.jstor.org/stable/40421462

Alvazzi del Frate, A., and others (eds.) (2018). Encyclopedia of Criminology and Criminal
Justice, Springer Verlag.

220 |
Anderson, C. J., and Y. Tverdova (2003). “Corruption, political allegiances, and attitudes
towards government in contemporary democracies”, American Journal of Political Science,
vol. 42, No. 1, pp. 91-109.

Arndt, C. (2009). Governance Indicators, Doctoral dissertation, Maastricht University School of

Governance, Maastricht, June.

Arndt, C., and C. Oman (2006). “Uses and abuses of governance indicators”. Paris:
Development Centre of the Organisation for Economic Co-operation and Development.

Arrow, K.J. (1972). “Gifts and exchanges”, Philosophy and Public Affairs, vol. 1, No. 4, pp. 343-
362.

Athias, L. (2018). “TIC governo eletrônico 2017: pesquisa sobre o uso das tecnologias da
informação e comunicação no setor público brasileiro” [“Governance, transparency and
ICT use in municipal administration”], Centro Regional de Estudos para o Desenvolvimento
da Sociedade da Informação (pp. 199-211). Available at:
https://www.cetic.br/media/docs/publicacoes/2/TIC_eGOV_2017_livro_eletronico

Attström, K. (2007). “Discrimination against Native Swedes of immigrant origin in access to

employment - A research study on Stockholm”, International Migration Paper 86E, Malmö
and Gothenburg: International Labour Office. Available at: https://www.ilo.org/wcmsp5/
groups/public/---ed_protect/---protrav/---migrant/documents/publication/wcms_201428.
pdf

AUC-IRD/DIAL (2018). “Monitoring Governance, Peace and Security. The GPS-SHaSA initiative:
instruments and methodology”, DIAL Working Document, No. 2018-10. Available at:
http://en-dial.ird.fr/content/view/full/307388

Australian Government Attorney-General’s Department (2015). National Legal Assistance Data

Standards Manual. Canberra: Attorney-General’s Department.

Australian Government Productivity Commission (2014). Access to Justice Arrangements,

Volume 2. Canberra: Productivity Commission.

Banisar, D. (2011). “Whistleblowing: International standards and developments”, Corruption

and transparency: Debating the Frontiers between State, Market and Society, I. Sandoval
(ed.), World Bank-Institute for Social Research, UNAM, Washington, D.C. Available at:
https://ssrn.com/abstract=1753180

Bansal, P., and K. Roth (2017). “Why companies go green: A model of ecological
responsiveness”, Academy of Management Journal, vol. 43, No. 4.

Barendrecht, M., and others (2010). Measuring Access to Justice in a Globalising World. Den
Haag: HiiL.

Barendrecht, M., Y.P. Kamminga and J.H. Verdonschot (2008). “Priorities for the justice system:
Responding to the most urgent legal problems of individuals”, TILEC Discussion Paper No.
2008/011, TISCO Working Paper No. 001/2008. Tilburg: Tilburg University Faculty of Law.

HANDBOOK ON GOVERNANCE STATISTICS | 221

 Bauhr, M. (2017). “Need or greed? Conditions for collective action against corruption”,
Governance, August, n/a-n/a. Available at: doi:10.1111/gove.12232

Bauhr, M., and R. Carlitz (2019). “When does transparency improve public goods provision?
Street level discretion, targetability and local public service delivery”, Manuscript on
review.

Bauhr, M., and others (2018). “Lights on the Shadows of Public Procurement. Transparency in
Government Contracting as an Antidote to Corruption?”, Revise and resubmit Governance.

Bauhr, M., and M. Grimes (2017). “Transparency to curb corruption? Concepts, measures and
empirical merit”, Crime, Law and Social Change, vol. 68, pp. 431–458. Available at:
https://doi.org/10.1007/s10611-017-9695-1

Bauhr, M., and M. Grimes (2014). “Indignation or resignation: The implications of transparency
for societal accountability” Governance, vol. 27, No. 2, pp. 291-320.

Bauhr, M., and N. Nasiritousi (2012). “Resisting transparency: Corruption, legitimacy, and
the quality of global environmental policies. Global Environmental Politics, vol. 12, No. 4,
pp. 9–29.

Bearak, M., and R. Ombuor (2019). “A new census in Kenya is counting people never counted
before”, Washington Post (29 Aug). Available at: https://www.washingtonpost.com/world/
africa/kenya-census-intersex-and-indigenous-people/2019/08/28/fbeca98e-c529-11e9-
8bf7-cde2d9e09055_story.html

Beetham, D. (2004). “The quality of democracy: Freedom as the foundation”, Journal of

Democracy, Johns Hopkins University Press, vol. 15, No. 4, pp. 61-75.

Begiraj, J., S. Garahan and K. Shuttleworth (2018). Ombudsman Schemes and Effective
Access to Justice: A Study of International Practices and Trends. London: International Bar
Association.

Bell, S., and M.S. Ainsworth (1972). “Infant crying and maternal response responsiveness”,
Child Development, vol. 43, No. 4, pp. 1171-1190.

Bellver, A., and D. Kaufmann (2005). “Transparenting transparency: Initial empirics and policy
applications”, World Bank Policy Research Working Paper.

Bertot, J., P. Jaeger and J. Grimes (2012). “Promoting transparency and accountability
through ICTs, social media and collaborative e-government”, Transforming Government:
People, Process and Policy, vol. 6, No. 1, pp. 78-91. Available at: doi:https://doi.
org/10.1108/17506161211214831

Besley, T. (2006). Principled Agents? The Political Economy of Good Government. Oxford:
Oxford University Press.

222 |
Besley, T.J., and R. Burgess (2002). “The political economy of government responsiveness:
Theory and evidence from India”, Quarterly Journal of Economics, vol. 117, No. 4,
pp. 1415–1452.

Bhatnagar, S. (2003). E-government and access to information. Global Corruption Report.

Transparency International.

Big Data United Nations Global Working Group (2019). Big Data Project Inventory. Available at:
https://unstats.un.org/bigdata/inventory/

Bjarnegård, E. (2013). Gender, informal institutions and political recruitment: Examining male

dominance in parliamentary representation. New York, NY: Palgrave Macmillan.

Blair, G., and K. Imai (2012). “Statistical analysis of list experiments”, Political Analysis, vol. 47,
No. 77.

Boarini, R., and others (2012). “What makes for a better life? The determinants of subjective
well-being in OECD countries – Evidence from the Gallup World Poll”, OECD Statistics
Working Papers, No. 2012/03. Paris: OECD Publishing. Available at:
https://doi.org/10.1787/5k9b9ltjm937-en

Boda, Z., and others (2018). “Societal change and trust in institutions”, Eurofund Research
Report. Available at: http://eurofound.link/ef18036

Bonsón, E., and others (2012). “Local e-government 2.0: Social media and corporate
transparency in municipalities”, Government Information Quarterly, vol. 29, No. 2, pp. 123-
132.

Borgonovi, F., and A. Pokropek (2017). “Mind the gap: The mediating role of literacy
and education in explaining disparities in external political efficacy in 28 countries”,
Intelligence, vol. 62. Available at: http://dx.doi.org/10.1016/j.intell.2017.03.006

Bouckaert, G. (2012). “Trust and public administration”, Administration, vol. 60, No. 1, pp. 91-
115.

Bouckaert, G., and S. Van de Walle (2003). “Comparing measures of citizen trust and user
satisfaction as indicators of good governance: Difficulties in linking trust and satisfaction
indicators”, International Review of Administrative Sciences, vol. 69, No. 3, pp. 329-343.

Bowler, S., and T. Donovan (2002). “Democracy, institutions and attitudes about citizen
influence on government”, British Journal of Political Science, vol. 32, No. 2, 371–390.

Brown, A.J., and others (2014). International Handbook on Whistleblowing Research. Edward
Elgar Publishing.

Brown, T.T., and others (2006). “The empirical relationship between community social capital
and the demand for cigarettes”, Health Economics, vol. 15, No. 11, pp. 1159-1172.

HANDBOOK ON GOVERNANCE STATISTICS | 223

 Browne, E. (2015). Social Media and Governance. GSDRC. Available at: https://gsdrc.org/
publications/social-media-and-governance/

Brunetti, A., and B. Weder (2003). “A free press is bad news for corruption”, Journal of Public
Economics vol. 87, No. 7, pp. 1801-1824.

Burnham, G., Lafta, R., Doocy, S. and Roberts, L. (2006). “Mortality after the 2003 invasion
of Iraq: a cross-sectional cluster sample survey”, The Lancet, 368 (9545) October), pp.
1421–1428.

Buonanno, P., D. Montolio and P. Vanin (2009). “Does social capital reduce crime?”, The Journal
of Law and Economics, University of Chicago Press, vol. 52, No. 1, pp. 145-170.

Calvo, T., M. Razafindrakoto and F. Roubaud (2018). “Are data collected by NSOs biased? A
comparison of Afrobarometer and Governance, Peace and Security Surveys in Africa”. Paris
[Fr]: DIAL, LEDa, IRD, Université Paris-Dauphine, PSL Research University.

Campbell A., G. Gurin and W. Miller (1954). The Voter Decides. Evanston: New York.

Cediey, E., and F. Foroni (2008). “Discrimination in access to employment on grounds

of foreign origin in France: A national survey of discrimination based on the testing
methodology of the International Labour Office”, International Labour Office.

Central Statistics Office, Ireland (2017). Standard Report on Methods and Quality for Crime and
Victimisation Module. Cork: Central Statistics Office.

Chamberlain, A. (2012). “A time-series analysis of external efficacy”, Public Opinion Quarterly,

vol. 76, No. 1, pp. 117–30.

Chapman, A. R., and P. Ball (2001). “The truth of truth commissions: Comparative lessons from
Haiti, South Africa, and Guatemala”, Hum. Rts. Q., Vol. 23, 1.

Chong, A., and others (2015). Does corruption information inspire the fight or quash the
hope? A field experiment in Mexico on voter turnout, choice, and party identification”, The
Journal of Politics, vol. 77, No. 1, pp. 55–71. Available at: doi:10.1086/678766

Clarke, H., A. Kornberg and T.J. Scotto (2010). “Accentuating the negative? A political efficacy
question-wording-experiment”, Methodology, vol. 6, No. 3, pp. 107-117.

Clarke, H., and A. C. Acock (1989). “National elections and political attitudes: The case of
political efficacy”, British Journal of Political Science, vol. 19, pp. 551-562.

Collier, D., and J. E. Mahon (1993). “Conceptual ‘Stretching’ revisited: Adapting categories in
comparative analysis”, The American Political Science Review, vol. 87, No. 4, pp. 845–55.
Available at: doi:10.2307/2938818

Colquitt, J. A., and others (2001). “Justice at the Millennium: A meta-analytic review of 25 years
of organizational justice research”, Journal of Applied Psychology, vol. 86, pp. 425-445.

224 |
Condon, M., and M. Holleque (2013). “Entering politics: General self-efficacy and voting
behavior among young people”, Political Psychology, vol. 34, pp. 167–181. Available at:
doi:10.1111/pops.12019

Cornford, T. (2016). “The meaning of access to justice”, in Palmer, E. and others (eds.) Access to
Justice: Beyond the Policies and Politics of Austerity. Oxford: Hart.

Coumarelos, C., and others (2012). Legal Australia-Wide Survey: Legal Need in Australia.
Sydney: Law and Justice Foundation of New South Wales.

Craig, S., and M. Maggiotto (1982). “Measuring political efficacy”, Political Methodology, vol. 8,
pp. 85-109.

Dahlberg, S., and others (2019). The Quality of Government Basic Dataset, Version Jan 19,
University of Gothenburg: The Quality of Government Institute. Available at:
http://www.qog.pol.gu.se doi:10.18157/qogbasjan19

Dahlström, C., and others (2015). The Quality of Government Expert Survey Dataset II.
University of Gothenburg: The Quality of Government Institute.

Dandurand, Y., and J. Jahn (2017). Access to Justice Measurement Framework. Abbotsford:
International Centre for Criminal Law Reform and Criminal Justice Policy.

Daude, C., H. Gutiérrez and Á. Melguizo (2012). “What drives tax morale?”, OECD
Development Centre Working Papers, No. 315. Paris: OECD Publishing. Available at:
http://dx.doi.org/10.1787/5k8zk8m61kzqen (accessed on 7 June 2018)

Dignan, T. (2006). Northern Ireland Legal Needs Survey. Belfast: Northern Ireland Legal
Services Commission.

Dreher, A., and M. Gassebner (2013). “Greasing the wheels? The impact of regulations and
corruption on firm entry”, Public Choice, vol. 155 (3/4), pp. 413-432.

Dyck, J., and E. Lascher (2009). “Direct democracy and political efficacy reconsidered”, Political
Behavior, vol. 31, No. 3, pp. 401-427. Available at: http://www.jstor.org/stable/40587291

ENCIG 2017, “Encuesta Nacional de Calidad et Impacto Gubernamental”. Available at:

https://www.inegi.org.mx/contenidos/programas/encig/2017/doc/encig2017_principales_
resultados.pdf

Eo Alemika, E. (2007). “Quality of elections, satisfaction with democracy and political trust in
Africa”, Afrobarometer Working Paper, vol. 84.

European Commission (2018). Guidelines on improving the collection and use of equality data.
EC High Level Group on Non-discrimination, Equality and Diversity. Available at: https://
ec.europa.eu/info/sites/info/files/final_guidelines_4-10-18_without_date_july.pdf. For the
compendium of practices: https://fra.europa.eu/en/theme/racism-related-intolerances/
racism-compendium-practices; for the diagnostic mapping tool: https://fra.europa.eu/en/
theme/racism-related-intolerances/racism-compendium-practices/about-compendium

HANDBOOK ON GOVERNANCE STATISTICS | 225

 European Commission (2014). “Special Eurobarometer 397. Corruption. Wave EB79.1”,
TNS Opinion and Social. European Commission. Available at: https://ec.europa.eu/
commfrontoffice/publicopinion/archives/ebs/ebs_397_en.pdf

European Commission for the Efficiency of Justice (CEPEJ) (2018a). European Judicial Systems:
Efficiency and Quality of Justice. Strasbourg: Council of Europe.

European Commission for the Efficiency of Justice (CEPEJ) (2018b). The 2018 EU Justice
Scoreboard. Strasbourg: Council of Europe.

European Commission for the Efficiency of Justice (CEPEJ) (2016). Handbook for Conducting
Satisfaction Surveys Aimed at Court Users in Council of Europe Member States. Strasbourg:
Council of Europe.

European Commission for the Efficiency of Justice (CEPEJ) (2008). Guidelines. Strasbourg:
Council of Europe.

European Union Agency for Fundamental Rights (2008, 2012). Surveys on minorities and LGBT
people. Available at: https://fra.europa.eu/en

Farrell, H., and J. Knight. (2003). “Trust, institutions, and institutional change: industrial districts
and the social capital hypothesis”, Politics and Society, vol. 31, No. 4, pp. 537–566. Available
at: https://doi.org/10.1177/0032329203256954

Farrow, T.C.W., and others (2016). Everyday Legal Problems and the Cost of Justice in Canada.
Toronto: Canadian Forum on Civil Justice.

Fehr, E. (2009). “On the economics and biology of trust”, The Journal of the European
Economic Association, vol. 7, No. 23, pp. 235-266.

Feldstein, M. (1996). “How big should government be?”, National Tax Journal, vol. 50, No. 2,
pp. 197-213.

Fernández Rodríguez, J. J. (2018). “ODS 16: paz, justicia e instituciones Fuertes”. Madrid:
Centro Superior de Estudios de la Defensa Nacional. Available at:
http://www.ieee.es/Galerias/fichero/docs_investig/2018/DIEEEINV18-2018ODS.pdf

Ferrín, M. (2016). “An empirical assessment of satisfaction with democracy”, in How Europeans
View Democracy. Oxford: Oxford University Press.

File, T. (2018). “Characteristics of voters in the presidential election of 2016. population

characteristics”, Current Population Reports. Available at:
https://www.census.gov/content/dam/Census/library/publications/2018/demo/P20-582.pdf

Finkel, S. E. (1985). “Reciprocal effects of participation and political efficacy: A panel analysis”,
American Journal of Political Science, vol. 29, No. 4, pp. 891-913.

Flavin, P., and M.J. Keane (2011). “Life satisfaction and political participation: Evidence from the
United States”, Journal of Happiness Studies, vol. 13, No. 1, pp. 63-76.

226 |
Florini, A. (2007). The Right to Know. Transparency for an Open World. New York: Colombia
University Press.

Florini, A. (2002). “Increasing transparency in government”, International Journal on World

Peace, vol. 19, No. 3, pp. 3-37.

Fox, J. (2007). “The uncertain relationship between transparency and accountability”,

Development in Practice, vol. 17, No. 4/5, pp. 663-771.

Freedom House (2019). “Freedom and the Media: A Downward Spiral”. Available at:
https://freedomhouse.org/report/freedom-media/freedom-media-2019

Frey, B., and A. Stutzer (2005). “Happiness research: State and prospects”, The Review of Social
Economy, vol. 63, pp. 207-228.

Frey, B., M. Benz and A. Stutzer (2004). “Introducing procedural utility: Not only what, but also
how matters”, The Journal of Institutional and Theoretical Economics (JITE) / Zeitschrift für
die gesamte Staatswissenschaft, vol. 160, No. 3, pp. 377-401.

Fukuda Parr, S., and D. McNeill (2019). “Knowledge and politics in setting and measuring
SDGs: Introduction to the special issue”, Global Policy vol. 10, Supp. 1 (January), pp. 5-16.

Fukuyama, F. (2013). “What is governance?”, Center for Global Development Working Paper
314, January.

Garth, B.G., and M. Cappelletti (1978). “Access to justice: The newest wave in the worldwide
movement to make rights effective”, Buffalo Law Review, vol. 27, pp. 181-292.

Geneva Declaration Secretariat (2015). Global Burden of Armed Violence: Every Body Counts.
Available at: http://www.genevadeclaration.org/measurability/global-burden-of-armed-
violence/global-burden-of-armed-violence-2015.html

Genn, H. (1999). Paths to Justice. Oxford: Hart.

Gleditsch, N.P., and others (2002). “Armed conflict 1946-2001: A new dataset”, Journal of Peace
Research, vol. 39, No. 5, pp. 615-637.

Golub, S., and McQuay, K. (2001). “Legal empowerment: Advancing good governance and
poverty reduction”, Law and Policy Reform at the Asian Development Bank, Manila: Asian
Development Bank.

González, S. (2020), “The accuracy of public sector responsiveness measures”, OECD Public
Governance Working Papers, OECD Publishing, Paris.

González, S., and C. Smith (2017). “The accuracy of measures of institutional trust in household
surveys: Evidence from the OECD Trust database”, OECD Statistics Working Papers,
2017/11. Paris: OECD Publishing. Available at: http://dx.doi.org/10.1787/d839bd50-en

HANDBOOK ON GOVERNANCE STATISTICS | 227

 González, S., L. Fleischer and M. Mira d’Ercole (2017). “Governance statistics in OECD
countries and beyond: What exists, and what would be required to assess their quality?”,
OECD Statistics Working Papers, 2017/03. Paris: OECD Publishing. Available at:
https://doi.org/10.1787/c0d45b5e-en

Goudrian, H. (2006). Reporting Crime: Effects of Social Context on the Decision of Victims to
Notify the Police. Veenendaal: Universal Press.

Gramatikov, M.A., and others (2014). Justice Needs in Indonesia 2014: Problems, Processes and
Fairness. Den Haag: HiiL.

Gramatikov, M.A., M. Barendrecht and J.H. Verdonschot (2011). “Measuring the costs and
quality of paths to justice”, Hague Journal of the Rule of Law, vol. 3, pp. 349-379.

Green, B.A. (2016). “Access to criminal justice: Where are the prosecutors?”, Texas A&M Law
Review, vol. 3, No., 3, pp. 515-536.

Greenwald, A.G., D.E. McGhee and J.L. Schwartz (1998). “Measuring individual differences in
implicit cognition: The implicit association test”, J Pers Soc Psychol, vol. 74, No. 6, pp. 1464-
80.

Grigorescu, A. (2003). “International organizations and government transparency: Linking the

international and domestic realm”, International Studies Quarterly, vol. 47, No. 4, pp. 643-
667.

Groves, R.M., and others (2009). Survey Methodology, 2nd ed. Hoboken, New Jersey: John
Wiley and Sons.

Guha-Sapir, D., and others (2015). “Civilian deaths from weapons used in the Syrian conflict”,
Bmj, 351, p. 4736.

Hardin, R. (2004). Trust and Trustworthiness. Russel Sage Foundation, New York.

Harlow, C.W. (1985). “Reporting crimes to the police. Bureau of Justice Statistics Special
Report NC 99432”. Washington D.C.: US Government Printing Office.

Harrendorf, S., M. Heiskanen and S. Malby (2010). International Statistics on Crime and Justice.
Helsinki: European Institute for Crime Prevention and Control and UNODC.

Heiskanen, M., and others (eds.) (2014). “Recording community sanctions and measures and
assessing attrition: A methodological study on comparative data in Europe”, HEUNI,
pp. 134-142. Available at: www.heuni.fi

Helliwell, J.F., and S. Wang (2010). “Trust and well-being”, National Bureau of Economic
Research Working Paper No. 15911.

Henning, K., and others (2017). “Portland’s neighborhood involvement locations project (final
report)”. Portland, Oregon: Criminal Justice Policy and Research Institute, Portland State
University.

228 |
Her Majesty’s Treasury, UK (2000). “2000 spending review: Public service agreements”.
Norwich: HMSO.

Herrera, J., M. Razafindrakoto and F. Roubaud (2007). “Governance, democracy and poverty
reduction: Lessons drawn from household surveys in sub-Saharan Africa and Latin
America”, International Statistical Review, vol. 75, No. 1, pp. 70-95.

Himelein, K., N. Menzies and M. Woolcock (2010). “Surveying justice: A practical guide to
household surveys”, Justice and Development Working Paper, Series 11/2010. Washington,
D.C.: World Bank Justice Reform Practice Group.

Hobolt, S.V., and R. Klemmensen (2008). “Government responsiveness and political

competition in comparative perspective”, Comparative Political Studies, vol. 41, No. 3,
pp. 309-337, Sage Publications.

Hollyer, J.R., B.P. Rosendorff and J.R. Vreeland (2015). “Transparency, protest, and autocratic
instability”, American Political Science Review, vol. 109, No. 4, pp. 764–784. Available at:
doi:10.1017/S0003055415000428

Hollyer, J.R., B.P. Rosendorff and J.R. Vreeland (2014). “Measuring transparency”, Political
Analysis, vol. 22, No. 4, pp. 413–34. Available at: doi:10.1093/pan/mpu001

Hollyer, J.R., B.P. Rosendorff and J.R. Vreeland (2011). “Democracy and transparency”,
The Journal of Politics, vol. 73, No. 4, pp. 1191–1205. Available at: doi:10.1017/
S0022381611000880

Hood, C., and D. Heald, eds. (2006). “Transparency: The key to better governance?”,
Proceedings of the British Academy (135). Oxford University Press, Oxford, UK.

Howard, G., G. Newman and W.A. Pridemore (2000). “Theory, method, and data in
comparative criminology”, Criminal Justice, vol. 4, Measurement and Analysis of Crime
and Justice, pp. 139-211. Washington, D.C.: United States Department of Justice, National
Institute of Justice.

IDB (2019). “Trust measures and interventions in Latin America and the Caribbean”. Available
at: https://www.iadb.org/en/research-and-data/project-details?nid=1253

IDEA (2017). “Open data in electoral administration”, Stockholm.

IDEA (2016). “Voter turnout trends around the world”. IDEA Publishing, Stockholm.

IDEA (2014). Electoral Management Design, revised edition, Stockholm.

IMF (2010). IMF Annual Report 2010: Supporting a balanced global recovery. Available at:
https://www.imf.org/en/Publications/AREB/Issues/2016/12/31/International-Monetary-Fund-
Annual-Report-2010-Supporting-a-Balanced-Global-Recovery-23972

HANDBOOK ON GOVERNANCE STATISTICS | 229

 INEGI (2017a). Encuestas en Hogares. Encuesta Origen-Destino en Hogares de la Zona
Metropolitana del Valle de México. Available at: http://www.beta.inegi.org.mx/contenidos/
programas/eod/2017/doc/resultados_eod_2017.pdf

INEGI (2017b). National Survey of Standards and Police Professional Training (ENECAP).
Available at: https://www.inegi.org.mx/programas/enecap/2017/

INEGI (2016). Encuesta nacional de acceso a la información pública y protección de datos

personales (ENAID), Mexico.

INEGI (2012). Proceso estándar para el aprovechamiento de registros administrativos, Mexico

City: INEGI.

Injury Surveillance Workgroup (2003). “Consensus recommendations for using hospital

discharge data for injury surveillance”, Marietta (GA): State and Territorial Injury Prevention
Directors Association.

Inter-Agency and Expert Group on SDG indicators (2019). “Tier classification for global SDG
indicators”. Sustainable Development. Available at: https://unstats.un.org/sdgs/files/
Tier%20Classification%20of%20SDG%20Indicators_13%20February%202019_web.pdf

Inter-Agency and Expert Group on SDG indicators (2017). “Fourth expert meeting of the
IAEG-SDGs: Working group on geospatial information”, IAEG-SDGs activities on data
disaggregation, New York: United Nations Global Geospatial Information Management.

Inter-Agency and Expert Group on SDG indicators (2014). “A world that counts: Mobilising the
data revolution for sustainable development”, United Nations. Available at:
http://www.undatarevolution.org/wp-content/uploads/2014/12/A-World-That-Counts2.pdf

International Fund for Agricultural Development (IFAD) (2016). Governance and corruption.
Available at: https://www.ifad.org/-/governance-and-corruption

International Labour Organization (ILO) (2013). Measuring Informality: A Statistical Manual on

the Informal Sector and Informal Employment, Geneva.

IPU Parliamentary Assembly and Council of Europe (2018). “Sexism, harassment and violence
against women in parliaments in Europe”. Available at: https://www.ipu.org/resources/
publications/reports/2018-10/sexism-harassment-and-violence-against-women-in-
parliaments-in-europe

Isenring, G.L., G. Mugellini and M. Killias (2016). “The willingness to report employee offences
to the police in the business sector”, European Journal of Criminology, vol. 13, No. 3,
pp. 372–392, SAGE. Available at: DOI: 10.1177/1477370815623569

Islam, R. (2006). “Does more transparency go along with better governance?”, Economics and
Politics, vol. 18, No. 2, pp. 121-167.

230 |
Istat (2010). “Reati, vittime e percezione della sicurezza”, Istituto nazionale di statistica,
Employee theft still costing business. Available at:
http://www.inc.com/articles/1999/05/13731.html

Ivanyna, M., and A. Shah (2018). “Ranking countries for good governance using public opinion
surveys”, in D.V Malito and others (eds.), The Palgrave Handbook of Indicators in Global
Governance, Springer Nature.

Jackson, J., and others (2012). “Policing by consent: Topline results (UK) from Round 5”,
European Social Survey.

Jacobsen, R., J. Snyder and A. Saultz (2014). “Informing or shaping public opinion? The
influence of school accountability data format on public perceptions of school quality”,
American Journal of Education, vol. 121, No. 1 (Nov.), pp. 1-27.

Jain, A.K. (2001). “Corruption: A review”, Journal of Economic Surveys, 15, pp. 71-121.

Jakobsen, M., and R. Jensen (2015). “Common method bias in public management studies”,
International Public Management Journal, vol. 18, No. 1, pp. 3-30. Available at: DOI:
10.1080/10967494.2014.997906

James, O. (2009). “Evaluating the expectations disconfirmation and expectations anchoring

approaches to citizen satisfaction with local public services”, Journal of Public
Administration Research and Theory, vol. 19, No. 1, pp. 107-123.

Jehle (2012)

Junger-Tas, J., and Marshall, I.H. (2012). “Introduction to the international self-report study of
delinquency (ISRD-2)”, In Junger-Tas, J., and others (eds.), The many faces of youth crime:
Contrasting theoretical perspectives on juvenile delinquency across countries and cultures,
Springer Science+Business Media, LLC. Available at: DOI 10.1007/978-1-4419-9455-4_1

Karp, J., and S. Banducci (2008). “Political efficacy and participation in twenty-seven
democracies: How electoral systems shape political behaviour”, British Journal of Political
Science, vol. 38, No. 2, pp. 311-334. Available at: http://www.jstor.org/stable/27568347

Kaufmann, D. (2002). “Transparency, incentives and prevention (TIP) for corruption control and
good governance empirical findings, practical lessons, and strategies for action based on
international experience”, Background Paper for Qinghua University-Carnegie Conference
on Economic Reform and Good Governance: Fighting Corruption in Transition Economies,
April 11-12, Beijing.

Kaufmann, D., and A. Kray (2007). “Governance indicators: Where are we, where should we be
going?”, November. Washington D.C.: World Bank.

Kaufmann, D., G. Mehrez and T. Gurgur (2002). “Voice or public sector management?
An empirical investigation of determinants of public sector performance based on a survey
of public officials”, World Bank Research Working Paper. Available at:
http://dx.doi.org/10.2139/ssrn.316865

HANDBOOK ON GOVERNANCE STATISTICS | 231

 Kemp, V., and N.J. Balmer (2008). Criminal Defence Services: Users’ Perspectives. London:
Legal Services Commission.

Kim, S., and S. Choi (2018). Reflections and lessons from the 2018 nationwide legal needs and
access to justice survey in South Korea”, Unpublished report prepared for the Judiciary of
the Republic of Korea.

Klaming, L., and I. Giesen (2008). “Access to justice: The quality of the procedure”, TISCO
Working Paper Series on Civil Law and Conflict Resolution Systems No. 002/2008. Utrecht:
University of Utrecht.

Knack, S., and P. Keefer (1997). “Does social capital have an economic payoff? A cross-country
investigation”, The Quarterly Journal of Economics, vol. 112, No. 4, pp. 1252-1288.

Kobzin, D., and others (2011). The Level of Legal Capacity of the Ukrainian Population:
Accessibility and Effectiveness of Legal Services. Kharkov: International Renaissance
Foundation and Kharkov Institute of Social Research.

Kosack, S., and A. Fung (2014). “Does transparency improve governance?”, Annual Review
of Political Science, vol. 17, No. 1, pp. 65–87. Available at:
doi:10.1146/annurev-polisci-032210-144356

Krueger, A. O. (1993a). Political Economy of Policy Reform in Developing Countries.

Cambridge, MA: MIT Press.

Krueger, A. O. (1993b). “Virtuous and vicious circles in economic development”, American

Economic Review, vol. 83, pp. 351–55.

Kurtzman, J., G. Yago and T. Phumiwasana (2004). “The global costs of opacity — Measuring
business and investment risk worldwide”, MIT Sloan Management Review, vol. 46, No. 1,
pp. 38−44.

LaGratta, E., and E. Jensen (2015). Measuring Perceptions of Fairness: An Evaluation Toolkit,
New York: Center for Court Innovation and Bureau of Justice Assistance.

La Porta, R., and others (1999). “The quality of government”, The Journal of Law, Economics
and Organization, vol. 15, No. 1, pp. 222-279.

Lane, R.E. (1959). Political Life: Why and How People Get Involved in Politics. New York: Free
Press.

Lathrop, D., and L. Ruma (2010). Open Government: Transparency, Collaboration and

Participation in Practice, O'Reilly Media. 

Lauritsen, J.L., and others (2012). Methods for Counting High-Frequency Repeat Victimizations
in the National Crime Survey. Washington, D.C.: Bureau of Justice Statistics.

232 |
Lavallée, E., and F. Roubaud (2019). “Corruption in the informal sector: Evidence from West
Africa”, The Journal of Development Studies, vol. 55, No. 6, pp. 1067-1080. Available at:
DOI: 10.1080/00220388.2018.1438597

Lavallée E., and F. Roubaud (2014). “Does corruption matter for informal sector economic
performance? Microdata evidence from sub-Saharan Africa”, In Cling J.P. and others
(eds.), The informal economy in developing countries (Routledge Studies in Development
Economics). Oxon: Routledge (112), pp. 179-196.

Leff, N. (1964). “Economic development through bureaucratic corruption”, The American

Behavioural Scientist, vol. 8, No. 2, pp. 8–14.

Lewis, D. B., E. Callahan and T. Dworkin (2004). “Whistleblowing: Australian, UK and US

approaches to disclosure in the public interest”, Virginia Journal of International Law, vol.
44, No. 3, pp. 879-912.

Lindberg, S. I. (2013). “Mapping accountability: Core concept and subtypes”, International

Review of Administrative Sciences, vol. 79, No. 2, pp. 202–226. Available at:
doi:10.1177/0020852313477761

Lindstedt, C., and D. Naurin (2010). “Transparency is not enough: Making transparency
effective in reducing corruption”, International Political Science Review, vol. 31, No. 3,
pp. 301-322.

Lindström, M. (2005). “Social capital, the miniaturization of community and high alcohol
consumption: A population-based study”, Alcohol and Alcoholism, vol. 40, No. 6, pp. 556-
562.

Lochner, K.A., and others (2003). “Social capital and neighborhood mortality rates in Chicago”,
Social Science and Medicine, vol. 56, pp. 1797-1805.

Lubua, E. W. (2018). “Enhancing e-transparency in public governance through social media”, in

Exploring the Role of Social Media in Transnational Advocacy, pp. 136-151.

Lui, F. (1985). “An equilibrium queuing model of bribery”, Journal of Political Economy, vol. 93,
No. 4, pp. 760–781.

MacFeely, S. (2019). “The (Big) Data Bang: Opportunities and challenges for compiling SDG
indicators”, Global Policy, vol. 10, Supp. 1 (January), pp. 121-133.

MacFeely, S., and B. Nastav (2018). “You say you want a [data] revolution: A proposal to
use unofficial statistics for the SDG Global Indicator Framework”, Global Policy Watch.
Available at: https://www.globalpolicywatch.org/blog/2018/11/02/you-want-a-data-
revolution/

Malesky, E., P. Schuler and A. Tran (2012). “The adverse effects of sunshine: A field experiment
on legislative transparency in an authoritarian assembly”, American Political Science
Review, vol. 106, No. 4, pp. 762-786.

HANDBOOK ON GOVERNANCE STATISTICS | 233

 Malleson, N., and M. A. Andresen (2015). “The impact of using social media data in crime rate”,
Cartography and Geographic, vol. 42, No. 5, pp. 112-121.

Maranlou, S. (2015). Access to Justice in Iran: Women, Perception and Reality. New York:
Cambridge University Press.

Maru, V., and V. Gauri (2018). Community Paralegals and the Pursuit of Justice. Cambridge:
Cambridge University Press.

Marvel, J.D. (2016). “Unconscious bias in citizens’ evaluations of public sector performance”,
Journal of Public Administration Research and Theory, vol. 26, No. 1, pp. 143-158.

McPherson, J., S. Welch and C. Clark (1977). “The stability and reliability of political efficacy:
Using path analysis to test alternative models”, American Political Science Review, vol. 71,
No. 2, pp. 509-521. Available at: DOI: 10.1017/S0003055400267427

Merry, S. E. (2019). “The Sustainable Development Goals confront the infrastructure

movement”, Global Policy, vol. 10, Supp. 1 (January), pp. 146-148.

Michener, G. (2015). “Policy evaluation via composite indexes: Qualitative lessons from
international transparency policy indexes”, World Development, vol. 74, pp. 184–196.

Michener, G., and K. Bersch (2013). “Identifying transparency”, Information Polity, vol. 18, No. 3,
pp. 233-242. Available at: DOI 10.3233/IP-130299

Miller, A., and O. Lishtaug (1990). “Political parties and confidence in government: A
comparison of Norway, Sweden and the United States”, British Journal of Political Science,
vol. 20, No. 3, pp. 357-386.

Miller, W., A. Miller and E. Schneider (1980). American National Election Studies Data
Sourcebook, 1952-1978. Harvard University Press, Cambridge MA.

Mirlees-Black, C., and C. Byron (1999) Domestic violence: Findings from the BCS self-
completion questionnaire. London: Home Office.

Mitchell, R.B. (1998). “Sources of transparency: Information systems in international regimes”,

International Studies Quarterly, vol. 42, No. 1, pp. 109–30. Available at: doi:10.1111/0020-
8833.00071

Montinola, G.R., and R.W. Jackman (2002). “Sources of corruption: A cross-country study”,
British Journal of Political Science, vol. 32, No. 01, pp. 147-170.

Moore, J.C., L. Stitson and E. Welniak (2000). “Income measurement error in surveys: A review”,
Journal of Official Statistics, vol. 16, No. 4, pp. 331-361.

Morell, M. (2003). “Survey and experimental evidence for a reliable and valid measure of
internal political efficacy”, Public Opinion Quarterly, vol. 67, pp. 589–602. Available at:
https://academic.oup.com/poq/article/67/4/589/1836340

234 |
Mosher, C.J., T.D. Miethe and D.M. Phillips (2002). The Mismeasure of Crime. Thousand Oaks,
California: Sage Publications.

Mugellini, G. (2011). “International crime statistics: Why they are needed, how they should be
improved and what has been done so far”, in S. Chawla (ed.), Forum on Crime and Society,
vol. 7, No. 1, Special Issue: Collecting Crime Data: Indicators and Measurement. Vienna:
UNODC.

Mugellini, G. (2010). “Measuring crime against business in the EU: The problem of
comparability”, Università Cattolica del Sacro Cuore, XXII ciclo, a.a. 2008/09, Milano.
Available at: http://hdl.handle.net/10280/733

Murphy, K., T.R. Tyler and A. Curtis (2009). “Nurturing regulatory compliance: Is procedural
justice effective when people question the legitimacy of the law?”, Regulation and
Governance, vol. 3, pp. 1- 26.

Murtin, F., and others (2018). “Trust and its determinants: Evidence from the Trustlab
experiment”, OECD Statistics Working Papers, No. 2018/02. Paris: OECD Publishing.
Available at: https://doi.org/10.1787/869ef2ec-en

National Center for Access to Justice (2016). “The Justice Index”. Available at:
https://justiceindex.org/2016-findings/#site-navigation

Naurin, D. (2007). Deliberation behind closed doors: Transparency and lobbying in the
European Union, ECPR Press.

Niemi, R., S. Craig and F. Mattei (1991). “Measuring internal political efficacy in the 1988
National Election Study”, American Political Science Review, vol. 85, No. 4, pp. 1407-1413.

Nix, J., and others (2019). “Police research, officer surveys, and response rates”, Policing and
Society, vol. 29, No. 5. Available at: DOI: 10.1080/10439463.2017.1394300.

Nooteboom, B. (2007). “Social capital, institutions and trust”, Review of Social Economy, vol.
65, No. 1, pp. 29-53.

North, D.C. (1990). Institutions, Institutional Change and Economic Performance. Cambridge
University Press, Cambridge, MA.

Nussbaum, M. (2011). Creating Capabilities – The Human Development Approach. Harvard

University Press, Cambridge, Mass.

Ochrana, F., and J. Pavel (2013). “Analysis of the impact of transparency, corruption, openness
in competition and tender procedures on public procurement in the Czech Republic”,
Central European Journal of Public Policy, vol. 7. No. 2, Dec 2013.

O’Reilley, M., A. O’Suilleabhain and T. Paffenholz (2015). “Reimaging peacemaking: Women’s

roles in peace processes”, International Peace Institute, June. Available at: https://www.
ipinst.org/wp-content/uploads/2015/06/IPI-E-pub- Reimagining-Peacemaking-rev.pdf

HANDBOOK ON GOVERNANCE STATISTICS | 235

 OECD (forthcoming-a). Government at a Glance 2019. Paris: OECD Publishing.

OECD (forthcoming-b). How’s Life? 2020. Paris: OECD Publishing.

OECD (2019). Access to Justice for Inclusive Growth: Putting People at the Centre. Paris: OECD
Publishing. Available at: https://doi.org/10.1787/597f5b7f-en

OECD (2018). “Diversity statistics in the OECD: How do OECD countries collect data on ethnic,
racial and indigenous identity?”, OECD Statistics Working Paper. Paris: OECD Publishing.

OECD (2017a). How’s Life? 2017: Measuring Well-Being. Paris: OECD Publishing. Available at:
https://doi.org/10.1787/how_life-2017-en

OECD (2017b). Final report of the expert group on quality of life indicators. Paris: OECD
Publishing. Available at: https://ec.europa.eu/eurostat/documents/7870049/7960327/KS-FT-
17-004-EN-N.pdf/f29171db-e1a9-4af6-9e96-730e7e11e02f

OECD (2017c). “Trust and public policy: How better governance can help rebuild public trust”,
Chapter 6, OECD Public Governance Reviews. Paris: OECD Publishing. Available at:
https://doi.org/10.1787/9789264268920-en

OECD (2017d). “LGBTI in OECD Countries”, OECD Social, Employment and Migration Working
Papers. Paris: OECD Publishing.

OECD (2017e). Government at a Glance 2017. Paris: OECD Publishing. Available at:
https://doi.org/10.1787/gov_glance-2017-en

OECD (2017f). OECD Guidelines on Measuring Trust. Paris: OECD Publishing. Available at:
http://dx.doi.org/10.1787/9789264278219-en

OECD (2016a). Open Government: The Global Context and the Way Forward. Paris: OECD
Publishing. Available at: http://dx.doi.org/10,1787/9789264268104-en

OECD (2016b). Government at a Glance: Latin America and the Caribbean 2017. Paris: OECD
Publishing. Available at: http://dx.doi.org/10.1787/9789264265554-en

OECD (2016c). Society at a Glance 2016: OECD Social Indicators. Paris: OECD Publishing.
Available at: http://dx.doi.org/10.1787/9789264261488-en

OECD (2015). How’s Life? 2015: Measuring Well-being. Paris: OECD Publishing. Available at:
http://dx.doi.org/10.1787/how_life-2015-en

OECD (2013a). OECD Guidelines on Measuring Subjective Well-being. Paris: OECD Publishing.
Available at: https://doi.org/10.1787/9789264191655-en

OECD (2013b). “Trust in government”, Government at a Glance 2013. Paris: OECD Publishing.
Available at: http://dx.doi.org/10.1787/gov_glance-2013-7-en

236 |
OECD (2012). Quality framework and Guidelines for OECD Statistical Activities. Paris: OECD
Publishing. Available at: http://www.oecd.org/officialdocuments/publicdisplaydocumentpd
f/?cote=std/qfs(2011)1anddoclanguage=en

OECD (2011). Asset Declarations for Public Officials: A Tool to Prevent Corruption. Paris: OECD
Publishing. Available at: http://dx.doi.org/10.1787/9789264095281-en

OECD and Open Society Justice Initiative (2019). Legal Needs Surveys and Access to Justice.
Paris: OECD Publishing.

OECD, Public Governance Reviews. Paris: OECD Publishing. Available at:

https://doi.org/10.1787/9789264268920-en

OECD and Korea Development Institute (2018). Understanding the Drivers of Trust in
Government Institutions in Korea. Paris: OECD Publishing. Available at:
https://doi.org/10.1787/9789264308992-en

Office of the UN High Commissioner for Human Rights (OHCHR) (2019). “Counting groups left
behind: a stellar step for inclusion in Kenya”, Geneva. Available at:
https://www.ohchr.org/EN/NewsEvents/Pages/CensushumanrightsinclusionKenya.aspx

OHCHR (2018). “Participation”, in A Human Rights Based Approach to Data – Leaving No One
Behind in the 2030 Agenda for Sustainable Development. Available at: https://www.ohchr.
org/Documents/Issues/HRIndicators/GuidanceNoteonApproachtoData.pdf

OHCHR (2012). Human Rights Indicators: A Guide to Measurement and Implementation, Office
of the United Nations High Commissioner for Human Rights, HR/PUB/12/5. Available at:
https://www.ohchr.org/Documents/Publications/Human_rights_indicators_en.pdf

OHCHR (2008). Report on Indicators for Promoting and Monitoring the Implementation of
Human Rights, HRI/MC/2008/3*.

OHCHR (2000). What is good governance? Available at: http://www.ohchr.org/EN/Issues/

Development/Good Governance/Pp./GoodGovernanceIndex.aspx

OHCHR (N.D.). Guidelines for States on the effective implementation of the right to
participate in public affairs. Report presented by the Human Rights Council to member
states and other relevant stakeholders through resolution 39/11. A/HRC/39/28,
Geneva. Available at: https://www.ohchr.org/Documents/Issues/PublicAffairs/
GuidelinesRightParticipatePublicAffairs_web.pdf

Oliver, R. (2010). Satisfaction: A Behavioural Perspective on the Consumer. Routledge: New

York.

Office for National Statistics (ONS) (2016). “Statistics on trust for methodological
testing from the opinions survey, Oct 2015 to May 2016”, UK ONS.
Available at: www.ons.gov.uk/peoplepopulationandcommunity/wellbeing/
adhocs/006326statisticsontrustformethodologicaltestingfromtheopinionssurveyoct2015
tomay2016.

HANDBOOK ON GOVERNANCE STATISTICS | 237

 Open Society Justice Initiative (2018). Strengthening Access to Civil Justice with Legal Needs
Surveys. New York: Open Society Foundations.

Open Society Justice Initiative (2015). Strengthening Pretrial Justice: A Guide to the Effective
Use of Indicators. New York: Open Society Foundations.

Open Society Justice Initiative (2014). Presumption of Guilt: The Global Overuse of Pretrial
Detention. New York: Open Society Foundations.

Open Society Justice Initiative and United Nations Development Programme (2011). The
Socioeconomic Impact of Pretrial Detention. New York: Open Society Foundations.

Osborne, S. (2018). Crime against Businesses: Findings from the 2017 Commercial Victimisation
Survey. London: Home Office.

Paterson, A. (2012). Lawyers and the Public Good: Democracy in Action? Cambridge:
Cambridge University Press.

Petruzzellis, L., A. D'Uggento and S. Romanazzi (2006). “Student satisfaction and quality of
service in Italian universities”, Managing Service Quality, vol. 16, pp. 349-364. Available at:
DOI 10.1108/09604520610675694

Piani, A., and others (2000). “Expression of MHC class II molecules contributes to
lipopolysaccharide responsiveness”, Division of Immunology, Zurich University Children's
Hospital. Available at: https://doi.org/10.1002/1521-4141(200011)30:11<3140::AID-
IMMU3140>3.0.CO;2-O

Pleasence, P. (2006). Causes of Action: Civil Law and Social Justice, 2nd ed. Norwich: TSO.

Pleasence, P., and N.J. Balmer (2019). “Development of a general legal confidence scale: a first
implementation of the Rasch measurement model in empirical legal studies”, Journal of
Empirical Legal Studies, vol. 16, No. 1, pp. 143-174.

Pleasence, P., and N.J. Balmer (2012). “Ignorance in bliss: Modeling knowledge of rights in
marriage and cohabitation”, Law and Society Review, vol. 46, No. 2, pp. 297-333.

Pleasence, P., N.J. Balmer and C. Denvir (2017). “Wrong About rights: Public knowledge of
key areas of consumer, housing and employment law in England and Wales”, Modern Law
Review, vol. 80, No. 5, pp. 836-859.

Pleasence, P., N.J. Balmer and C. Denvir (2015). How People Understand and Interact with the
Law. London: Legal Education Foundation.

Pleasence, P., and others (2014). Reshaping Legal Assistance Services: Building on the Evidence
Base. Sydney: Law and Justice Foundation of New South Wales.

Pollock, P. (1983). “The participatory consequences of internal and external political efficacy: A
research note”, The Western Political Quarterly, vol. 36, No. 3, pp. 400–09.

238 |
Porta, D., and A. Vannucci (1997). “The ‘perverse effects’ of political corruption”, Political
Studies, vol. 45, No. 3, pp. 516-538.

Prettitore, P.S. (2015). Poverty and Legal Problems in Jordan: Defining the Relationship, MENA
Knowledge and Learning Quick Notes Series, No. 150. Washington, D.C.: World Bank.

Putnam, R.D. (2000). Bowling Alone: The Collapse and Revival of American Community. New
York: Simon and Schuster.

Putnam, R., R. Leonardi and R.Y. Nanetti, (1993). Making Democracy Work. Princeton, NJ:
Princeton University Press.

Rangel, M.F.D. (2018). “Measuring access to justice using a composite indicator”. Bogota:
Departamento Nacional de Planeacion.

Raupp, F. M., and J. Antonio Gomes de Pinho (2016). “Review of passive transparency in
Brazilian city councils”, Revista de Administração, vol. 51, No. 3, pp. 288–98.

Razafindrakoto, M., F. Roubaud and Y.A. Assany (2019): Populating the Handbook on

Governance Statistics with Empirical Evidence: Illustrations from the GPS-SHaSA survey
modules in Africa, IRD DIAL Working paper no. 2019-15. Available at:
https://en-dial.ird.fr/content/view/full/306764.

Razafindrakoto, M., and F. Roubaud (2018). “Responding to the SDG 16 measurement

challenge: The governance, peace and security survey modules in Africa”, Global Policy,
vol. 20, pp. 146-182.

Razafrindakoto, M., and Roubaud, F. (2015). “The Governance, Peace and Security modules
of the Strategy for the Harmonization of Statistics in Africa (GPS-SHaSA): Development
of an innovative statistical survey methodology”, Working Papers DT/2015/20, DIAL
(Développement, Institutions et Mondialisation). Also available in French from Statéco.

Razafrindakoto, M., and Roubaud, F. (2010). “Are international databases on corruption

reliable? A comparison of expert opinion surveys and household surveys in Sub-Saharan
Africa”, World Development, vol. 38, No. 8, pp. 1057-1069.

Razafindrakoto, M., and F. Roubaud (2007). “Corruption, institutional discredit and exclusion
of the poor: A poverty trap”, Afrobarometer Working Paper No. 86. Available at:
afrobarometer.org

Razafindrakoto, M., and F. Roubaud (2003). “Wage and corruption: The case of Madagascar”,
Global Corruption Report 2003, Transparency International.

Reinikka, R., and J. Svensson (2005). “Fighting corruption to improve schooling: Evidence from
a newspaper campaign in Uganda”, Journal of the European Economic Association, vol. 2,
No. 2–3: pp. 259-267.

HANDBOOK ON GOVERNANCE STATISTICS | 239

 Relly, J.E., and M. Sabharwal (2009). “Perceptions of transparency of government
policymaking: A cross-national study”, Government Information Quarterly, vol. 26, No. 1,
pp. 148-157.

República, C. G. (2019). Estudio de opinión acerca de la transparencia en Costa Rica.

Revilla, M., and W. Saris (2012). “Evaluation of the experiments in the supplementary
questionnaire of Round 5 of the European Social Survey”, ESS: Data for a Changing Europe.
Barcelona: RECSM, UPF.

Roberts, J., and M. Hough (2005). Understanding public attitudes to criminal justice.
Maidenhead: Open University Press.

Roch, C.H., and T. Poister (2006). “Citizens, accountability, and service satisfaction the
influence of expectations”, Urban Affairs Review, vol. 41, No. 3, pp. 292-308.

Rodriguez, M., and E.J. Zechtmeister (2018). “Media pluralism, public trust and democracy:
New evidence from Latin America and the Caribbean”, Center for International Media
Assistance (CIMA).

Rose-Ackerman, S. (1999). Corruption and Government: Causes; Consequences and Reform.

Cambridge University Press, Cambridge.

Rose-Ackerman, S. (1998). “The role of the World Bank in controlling corruption”, Law and
Policy in International Business, vol. XXIX, 93–114.

Rosenberg, M. (1957). “Misanthropy and attitudes toward international affairs”, Journal of

Conflict Resolution, pp. 340-345. Available at: https://doi.org/10.1177/002200275700100403

Sandefur, R.L. (2019). “Access to what?”, Daedalus, vol. 148, No. 1, pp. 49-55.

Saris, W., and M. Torcal (2009). “Alternative measurement procedures and models”, Political
Efficacy, Research and Expertise Centre for Survey Methodology (RECSM), RECSM
Working Paper No. 3. Barcelona: Universitat Pompeu Fabra.

Sartori, G. (1970). “Concept misformation in comparative politics”, The American Political

Science Review, vol. 64, No. 4, pp. 1033–53. Available at: doi:10.2307/1958356

Scheppele, K.L. (2013). “The rule of law and the Frankenstate: Why governance checklists do
not work”, Governance, vol. 26, No. 4, pp. 559–62. Available at: doi:10.1111/gove.12049

Schwab, K. and X. Sala-i-Martin (2015). “The global competitiveness report 2014-2015”, World
Economic Forum.

Scrivens, K., and C. Smith (2013). “Four interpretations of social capital: An agenda for
measurement”, OECD Statistics Working Papers, No. 2013/06. Paris: OECD Publishing.
Available at: https://doi.org/10.1787/5jzbcx010wmt-en

240 |
Sen, A. (2009). The Idea of Justice. Cambridge, Massachusetts: Harvard University Press.

Sen, A. (1999). Development as Freedom. Oxford: Oxford University Press.

Sen, A. (1980), “Equality of What?”, in McMurrin, S. (ed.), Tanner Lectures on Human Values,
Vol. 1. Cambridge: Cambridge University Press.

Sheatsley, B. (1983). “Chapter 6: Questionnaire construction and item writing”, in Handbook of

survey research (1st ed.) pp. 195-230. New York, NY: Academic Press.

Sherr, A., and A. Paterson (2008). “Professional competence, peer review and quality assurance
in England and Wales and in Scotland”, Alberta Law Review, vol. 45, No. 5, pp. 151-168.

Sherr, A., R. Moorhead and A. Paterson (1994). Lawyers: The Quality Agenda. London: HMSO.

Shleifer, A., and R. W. Vishny (1993). “Corruption”, The Quarterly Journal of Economics, vol.108,
No. 3, pp. 599–617. Available at: doi:10.2307/2118402

Silva, R., and P. D. Ball (2006). “The profile of human rights violations in Timor-Leste, 1974-
1999”, Benetech Initiative.

Skogan W.G. (1984). “Reporting crimes to the police: The status of world research”, Journal of
Research in Crime and Delinquency, vol. 21, No. 2, pp. 113–137.

Skogan, W.G., and others (2015). “Surveying police officers”, in Maltz, M.D., S.K. Rice and
others, ed., Envisioning criminology: researchers on research as a process of discovery. New
York, NY: Springer, pp. 109–115.

Smit, P., and S. Harrendorf (2010). “Attributes of criminal justice systems – resources,
performance and punitivity”, in Harrendorf, S., M. Heiskanen and S. Malby (eds.),
International statistics on crime and justice. Helsinki: HEUNI, pp. 113-152.

Smit, P.R., R.F. Meijer and P-P. J. Groen (2004). “Detection rates, an international comparison”,
European Journal on Criminal Policy and Research, vol. 10, pp. 225-253.

Smith, D. A., and C. J. Tolbert (2004). Educated by Initiative: The Effects of Direct Democracy
on Citizens and Political Organizations in the American States. Ann Arbor: The University of
Michigan Press.

Smith, G., and others (2004). The Value of Linked Administrative Records for Longitudinal
Analysis. Oxford: Social Disadvantage Research Centre.

Sønderskov, K.M., and P.T. Dinesen (2016). “Trusting the State, trusting each other? The effect
of institutional trust on social trust”, Political Behaviour, vol. 38, p. 179. Available at:
https://doi.org/10.1007/s11109-015-9322-8

Spagat, M. (2010). “Ethical and data-integrity problems in the second Lancet survey of
mortality in Iraq”, Defence and Peace Economics, 21:1, pp. 1-41

HANDBOOK ON GOVERNANCE STATISTICS | 241

 Srbijanko, J.K., N. Korunovska and T. Maleska (2013). Legal Need Survey in the Republic of
Macedonia: The Legal Problems People face and the Long Path to Justice. Skopje: Open
Society Foundation – Macedonia.

Stasavage, D. (2004). “Open-Door or Closed-Door? Transparency in domestic and

international bargaining”, International Organization, vol. 58, No. 4, pp. 667-703.

Statistics Netherlands (2018). The Sustainable Development Goals: The situation for the
Netherlands. The Hague/Heerlen/Bonaire [NL]: Statistics Netherlands.

Statistics Norway (2015 and 2017). “Do refugees get involved in local politics”. Available at:
https://www.ssb.no/en/valg/artikler-og-publikasjoner/do-refugees-get-involved-in-local-
politics?

Statistics South Africa (2018). Victims of Crime Survey 2017/18. Pretoria: Statistics South Africa.

Stiglitz, J., J.-P. Fitoussi and M. Durand (2018). Beyond GDP: Measuring What Counts for
Economic and Social Performance. Paris: OECD Publishing.

Stiglitz, J.E., and R. Arnott (1991). “Moral hazard and non-market institutions: Dysfunctional
crowding out or peer monitoring”, The American Economic Review, vol. 81, No. 1, pp. 179-
190.

Stiglitz, J., A. Sen and J-P. Fitoussi (2009). “The measurement of economic performance and
social progress revisited: Reflections and overview”, OFCE Working Paper. Paris.

Stiglitz, J. (2002). Transparency in government. The Right to Tell. Washington, D.C.: World
Bank.

Sundström, A., and L. Wängnerud (2016). “Corruption as an obstacle to women’s political

representation: Evidence from local councils in 18 European countries”, Party Politics, vol.
22, No. 3, pp. 354–369. Available at: https://doi.org/10.1177/1354068814549339

Sung, H.-E. (2004). “Democracy and political corruption: A cross-national comparison”,

Crime, Law and Social Change, vol. 41, No. 2, pp. 179–93. Available at:
doi:10.1023/B:CRIS.0000016225.75792.02

Task Force on Justice (2019). Justice for All – The report of the Task Force on Justice,
conference version. New York: Center on International Cooperation.

Tavits, M. (2008). “Representation, corruption and subjective well-being”, Comparative Political

Studies, vol. 41 No. 12, pp. 1607-1630.

Teehankee, J.C. (2003). “Background paper on access to justice indicators in the Asia-Pacific
region”. Manila: La Salle Institute of Governance.

Temple, J. (2000). “Growth effects of education and social capital in the OECD countries”,
OECD Economics Department Working Papers, No. 263. Paris: OECD Publishing. Available
at: https://doi.org/10.1787/882344562861

242 |
The National Academies of Sciences, Engineering and Medicine (2017). “Innovations in federal
statistics: Combining data sources while protecting privacy”. Washington D.C.: National
Academy of Sciences.

Thornberry, T.P., and M.D. Krohn (2000). “The self-report method for measuring delinquency
and crime”, Criminal Justice, vol. 4.

Tolsma J., J. Blaauw and M. Te Grotenhuis (2012). “When do people report crime to the
police? Results from a factorial survey design in the Netherlands”, Journal of Experimental
Criminology, vol. 8, No. 2, pp. 117–134.

Transparency international (2009). Transparency International Plain Guide to Language.

Transparency international (2016). How-to guide for corruption assessment tools (2nd edition).

Travis, J., B. Western and S. Redburn (eds.) (2014). The Growth of Incarceration in the United
States: Exploring Causes and Consequences. Washington, D.C.: National Academies Press.

Treisman, Daniel (2007). “What have we learned about the causes of corruption from ten years
of cross-national empirical research?”, Annual Review of Political Science, vol. 10, No. 1, pp.
211–244. Available at: doi:10.1146/annurev.polisci.10.081205.095418

United Nations (2014). Guidelines for Producing Statistics on Violence against Women:
Statistical Surveys. Available at: https://unstats.un.org/unsd/gender/docs/guidelines_
statistics_vaw.pdf

United Nations (2008). Designing Household Survey Samples: Practical Guidelines, Studies in
Methods (Ser. F), No. 98. New York: UN. Available at: https://doi.org/10.18356/f7348051-en

United Nations (1948). Universal Declaration of Human Rights. Available at:

https://www.ohchr.org/EN/UDHR/Pp./Introduction.aspx

United Nations and the World Bank (2018). Pathways for Peace: Inclusive Approaches to
Preventing Violent Conflict. Available at:
https://openknowledge.worldbank.org/handle/10986/28337

United Nations, Commission on Crime Prevention and Criminal Justice (2017). World Crime
Trends and Emerging Issues and Responses in the Field of Crime Prevention and Criminal
Justice, UN, New York. Available at: https://www.unodc.org/unodc/en/data-and-analysis/
statistics/reports-on-world-crime-trends.html

United Nations, Department of Economic and Social Affairs (UN DESA) (2007). Public
governance indicators: A literature review, ST/ESA/PAD/SER.E/100.

UN DESA (2005b). Household Sample Surveys in Developing and Transition Countries. New
York: United Nations.

UN DESA (2003). Manual for the Development of a System of Criminal Justice Statistics. New
York: United Nations, ST/ESA/STAT/SER.F/89.

HANDBOOK ON GOVERNANCE STATISTICS | 243

 United Nations, Development Programme (UNDP) (2019a). “SDG Indicator 16.7.2 Proportion
of the population who believe decision-making is inclusive and responsive, by sex, age,
disability and population group”, Methodology Development Narrative, UNDP Oslo
Governance Centre.

UNDP (2019b). “SDG Indicator 16.6.2: Proportion of the population satisfied with their
last experience of public services”, Methodology Development Narrative, UNDP Oslo
Governance Centre.

UNDP (2017a). “Voices from the field: African experiences in producing governance, peace and
security statistics — recommendations for national statistical offices for monitoring goal 16
on peaceful, just and inclusive societies”, UNDP Oslo Governance Centre.

UNDP (2017b). “A review and testing of available indicators for SDG 16.7.2: Proportion of
population who believe that decision-making is inclusive and responsive”, UNDP Oslo
Governance Centre.

UNDP (2015). “User’s guide to measuring corruption and anti-corruption”, UNDP Global Anti-
Corruption Initiative (GAIN), pp. 29-30.

UNDP (2010). “Measuring and assessing corruption: UNDP Experiences”, Global Programme
on Anti-Corruption for Development Effectiveness, 1st Open-ended Intergovernmental
Working Group on The Prevention of Corruption, Vienna, 13-15 December.

UNDP (2005), Programming for Justice: Access for All, A Practitioner’s Guide to a Human
Rights-Based Approach to Access to Justice. New York: United Nations.

UNDP (2002). Human Development Report. Deepening democracy in a fragmented world.

New York: Oxford University Press.

UNDP and UN Office on Drugs and Crime (2016). Global Study on Legal Aid. New York: United
Nations.

United Nations, Economic and Social Commission for Asia and the Pacific (2009).
What is Good Governance? July 10. Available at:
http://www.unescap.org/sites/default/files/good-governance.pdf

United Nations, Economic Commission for Europe (UNECE) (2017). Guide on Poverty
Measurement. Available at: https://www.unece.org/stats/publications/guidepm.html

UNECE (2015). Fundamental Principles of Official Statistics. Available at: http://www.unece.

org/?id=3207

UNECE and the Conference of European Statisticians (2016). In-Depth Review of Governance
Statistics in the UNECE/OECD Region. Prepared by Turkey, Mexico and OECD, October
(ECE/CES/BUR/2016/OCT/2). Available at: http://www.unece.org/fileadmin/DAM/stats/
documents/ece/ces/bur/2016/October/02_In_depth_review_Governance_final.pdfUnited
Nations General Assembly (2015) Resolution A/Res/70/1.

244 |
United Nations, Economic and Social Council (UNESC) (2006). “Definition of basic concepts
and terminologies in governance and public administration”, Note by the Secretariat,
E/C.16/2006/4, January 5. Available at:
http://unpan1.un.org/intradoc/groups/public/documents/un/unpan022332.pdf

United Nations, General Assembly (1986). Declaration on the Right to Development. Available
at: https://www.ohchr.org/en/professionalinterest/pp./righttodevelopment.aspx

United Nations, General Assembly (1966). International Covenant on Economic, Social and
Cultural Rights. Available at: https://www.ohchr.org/en/professionalinterest/pp./cescr.aspx

United Nations, Human Rights Council (2008). “The role of good governance in the promotion
and protection of human rights”. Available at:
https://ap.ohchr.org/documents/e/hrc/resolutions/a_hrc_res_7_11.pdf

United Nations, Office on Drugs and Crime (UNODC) (2019). Global Study on Homicide 2019.
Homicide trends, patterns and criminal justice response. Available at:
https://www.unodc.org/documents/data-and-analysis/gsh/Booklet2.pdf.

UNODC (2017). Corruption in Nigeria. Bribery: public experience and response. Available at:
https://www.unodc.org/documents/data-and-analysis/Crime-statistics/Nigeria/Corruption_
Nigeria_2017_07_31_web.pdf

UNODC (2016). Latin America and the Caribbean Crime Victimization Survey Initiative. Available
at: https://www.unodc.org/documents/data-and-analysis/statistics/crime/CTS/VICLAC-
LACSI_CoreQuestionnaire_EN_2017_02_20.pdf

UNODC (2015). International Classification of Crime for Statistical Purposes, Version 1.0.
Available at: https://unstats.un.org/unsd/statcom/doc15/BG-ICCS-UNODC.pdf

UNODC (2013). Global Study on Homicide 2013. Available at:

https://www.unodc.org/documents/gsh/pdfs/2014_GLOBAL_HOMICIDE_BOOK_web.pdf

UNODC (2013a). United Nations Principles and Guidelines on Access to Legal Aid in Criminal
Justice Systems. New York: United Nations.

UNODC (2013b). Corruption and integrity challenges in the public sector of Iraq. An evidence-
based study, United Nations Office on Drugs and Crime. Available at:
https://www.unodc.org/documents/data-and-analysis/statistics/corruption/2013_Report_
on_Corruption_and_Integrity_Iraq.pdf

UNODC (2011a). Corruption in the Western Balkans: Bribery as experienced by the

population, United Nations Office on Drugs and Crime. Available at:
https://www.unodc.org/documents/data-and-analysis/statistics/corruption/Western_
balkans_corruption_report_2011_web.pdf

UNODC (2011b). Global Study on Homicide. Trends, Contexts and Data, Vienna. Available at:
http://dag.un.org/handle/11176/89899

HANDBOOK ON GOVERNANCE STATISTICS | 245

 UNODC (2010a). Developing Standards in Justice and Home Affairs Statistics. International and
EU Acquis.

UNODC (2010b). “Methodologies, including evidence-based approaches, for assessing

areas of special vulnerability to corruption in the public and private sectors” (CAC/COSP/
WG.4/2010/4).

UNODC (2010c). Corruption in Afghanistan Bribery as reported by the victims. United Nations
Office on Drugs and Crime. Available at: https://www.unodc.org/documents/data-and-
analysis/Afghanistan/Afghanistan-corruption-survey2010-Eng.pdf

UNODC (2007). “Surveys on crime trends and the operations of criminal justice systems (UN-
CTS)”. Available at: https://www.unodc.org/unodc/en/data-and-analysis/United-Nations-
Surveys-on-Crime-Trends-and-the-Operations-of-Criminal-Justice-Systems.html

UNODC (2004). The Global Programme Against Corruption: An Anti-Corruption Toolkit, 2nd
edition. Available at: https://www.unodc.org/documents/corruption/Toolkit_ed2.pdf

UNODC and the INEGI Center of Excellence in Statistical Information on Government, Crime,
Victimization and Justice (2013). Inventario de Encuestas de Victimización en América Latina
y el Caribe.

UNODC and INEGI Center of Excellence (2019). Repository of Corruption Measurement Tools.
Available at: http://www.cdeunodc.inegi.org.mx/index.php/herramientas-medicion-la-
corrupcion-en-mundo-2/

UNODC and UNDP (2018). Manual on corruption surveys: Methodological guidelines on the
measurement of bribery and other forms of corruption through sample surveys. Vienna
[AU]: UNODC.

UNODC and UNECE (2010). Manual on Victimization Surveys, ECE/CES/4, Geneva. Available
at: https://www.unodc.org/documents/data-and-analysis/Crime-statistics/Manual_on_
Victimization_surveys_2009_web.pdf

United Nations, Social and Economic Council (2001). Report of the Friends of the Chair of the
Statistical Commission on an assessment of the statistical indicators derived from United
Nations summit meetings, E/CN.3/2002/26, para. 65

United Nations, Statistical Commission (2017). Update on the Work of the High-level Group
for Partnership, Coordination and Capacity-Building for Statistics for the 2030 Agenda for
Sustainable Development. Available at: https://unstats.un.org/unsd/statcom/48th-session/
documents/BG-2017-3a-HLG-E.pdf

United Nations, Statistics Division (2017). Sustainable Development Goals Report. Available at:
https://unstats.un.org/sdgs/report/2019/The-Sustainable-Development-Goals-Report-2019.
pdf

246 |
United Nations, Statistics Division (2015). The World’s Women 2015. New York: United Nations.
Available at: https://unstats.un.org/unsd/gender/aboutWW2015.html

United Nations Women (2018). Turning Promises into Action. Available at: http://www.
unwomen.org/-/media/headquarters/attachments/sections/library/publications/2018/
sdg-report-gender-equality-in-the-2030-agenda-for-sustainable-development-2018-en.
pdf?la=enandvs=4332

UN Women (2016). Framework for Measuring Access to Justice Including Specific Challenges
Facing Women. New York: UN Women.

UN Women and UNDP (2017). Preventing Violence against Women in Elections: A Programming
Guide.

UN Women and UNDP (2015). Inclusive Electoral Processes: A Guide for Electoral Management
Bodies on Promoting Gender Equality and Women’s Participation.

United States Institute of Peace (2010). Governance, Corruption, and Conflict. Available at:
https://www.usip.org/sites/default/files/ETC-D/NPEC/480021.PDF

Uslaner, E. (2002). The Moral Foundations of Trust, Cambridge University Press.

Van de Walle, S. (2018). “Explaining citizen satisfaction and dissatisfaction with public services”,
in The Palgrave Handbook of Public Administration and Management in Europe. London:
Palgrave Macmillan.

Van de Walle, S., and G. Bouckaert (2003). “Public Service performance and trust in
government: The problem of causality”, International Journal of Public Administration,
vol. 26, Nos. 8-9, pp. 891-913.

van Dijk, J. (2008). The World of Crime: Breaking the Silence on Problems of Security, Justice
and Development Across the World. Los Angeles: Sage Publications.

van Dijk, J., J. van Kesteren and P. Smit (2007). “Criminal victimisation in international
perspective. Key findings from the 2004-2005 ICVS and EU ICS”, 257 Onderzoek en beleid.
Den Haag: WODC.

Van Dijk, J.J.M., P. Mayhew and M. Killias (1990). “Experiences of crime across the world:
Key findings of the 1989 International Crime Survey”, Deventer: Kluwer Law and Taxation
Publishers.

van Dijk, J., and others (2005). “The burden of crime in the EU: Research report – A
comparative analysis of the European Crime and Safety Survey (EU ICS) 2005”, EUCIS
report. Brussels: Gallup Europe.

Van Ryzin, G.G. (2004). “Expectations, performance, and citizen satisfaction with urban
services”, Journal of Policy Analysis and Management, vol. 23, No. 3, pp. 433-448.

HANDBOOK ON GOVERNANCE STATISTICS | 247

 Vecchione, M., and G.V. Caprara (2009). “On the personality determinants of political
participation: The contribution of traits and self-efficacy beliefs”, in Personality
and Individual Differences, vol. 46, No. 4, pp. 487-492. Available at: DOI: 10.1016/j.
paid.2008.11.021

Vera Institute (2008). Measuring Progress Toward Safety and Justice – A Global Guide to the
Design of Performance Indicators Across the Justice Sector. New York: Vera Institute.

Verdonschot, J.H., and others (2008). “Measuring access to justice: The quality of outcomes,
TISCO Working Paper Series on Civil Law and Conflict Resolution Systems, No. 007/2008.
Tilburg: Tilburg University.

Vial, V., and J. Hanoteau (2010). “Corruption, manufacturing plant growth, and the Asian

paradox: Indonesian evidence”, World Development, vol. 38, No. 5, pp. 693-705.

Virtual Network for the Development of Indicators on Peaceful, Just and Inclusive Societies for
Goal 16 (2015).

Vishwanath, T., and D. Kaufmann (1999). “Towards transparency in finance and governance”,
Policy Research Working Paper. Washington D.C.: World Bank.

Von Hoffer, H. (2000). “Crime statistics as constructs: The case of Swedish rape statistics”,
European Journal on Criminal Policy and Research, vol. 8, No. 1, pp. 77-89.

Walmsley, R. (2016). “World pre-trial/remand imprisonment list”. London: Institute for Criminal
Policy Research.

Wapner, P. (1996). Environmental Activism and World Civic Politics. Albany: State University of
New York Press.

Wei, Shang-Jin (1999). “Natural openness and good government”, World Bank Policy Research
Working Paper. Available at: https://doi.org/10.1596/1813-9450-2411

WHO Department of Reproductive Health and Research, London School of Hygiene and
Tropical Medicine, South African Medical Research Council (2013). Global and Regional
Estimates of Violence against Women: Prevalence and Health Effects of Intimate Partner
Violence and Non- Partner Sexual Violence. Geneva: World Health Organization.

Wilde, T. (2011). “The democratization of governance assessments”, in Making the State

Responsive: Experience with Democratic Governance Assessments, Göran Hydén and John
Samuel, eds. New York: United Nations Development Programme.

Williams, Andrew (2015). “A global index of information transparency and accountability”,

Journal of Comparative Economics, vol. 43, No. 3, pp. 804–824. Available at: doi:10.1016/j.
jce.2014.10.004

248 |
Williamson, V., and N. Eisen (2016). “The impact of open government: Assessing the evidence”,
Brookings, Development Policy Research Unit. Available at: https://www.brookings.edu/
research/the-impact-of-open-government-assessing-the-evidence/

Winters, M.S., and R. Weitz-Shapiro (2013). “Lacking information or condoning corruption:

When do voters support corrupt politicians?”, Comparative Politics, vol. 45, No. 4,
pp. 418–36. Available at: doi:10.5129/001041513X13815259182857

Wolf, C., and others (2016). The Sage Handbook of Survey Methodology. London: Sage
Publications.

World Bank Group, Enterprise Surveys. What Businesses Experience. Available at:
https://www.enterprisesurveys.org/data.

World Bank (2015). Open Government. Available at: http://www.worldbank.org/en/topic/

governance/brief/open-government-global-solutions-group

World Bank (2011). World Development Report 2011: Conflict, Security and Development.

World Bank (2000). Designing Household Survey Questionnaires for Developing Countries:
Lessons from 15 Years of the Living Standards Measurement Study, vol. 1.

World Development Report (2002). Building Institutions for Markets. New York: World Bank.

Yang, K., and M. Holzer (2006). “the performance-trust link: Implications for performance
measurement”, Public Administration Review, vol. 66, No. 1, pp. 114-126.

Young Mok, K., J. Oliver and G. Van Ryzin (2017). “Expectations of and satisfaction with public
services”, in Experiments in Public Management Research, Cambridge University Press.

Zhao, H., and S.H. Kim (2011). “An exploratory examination of the social capital and FDI linkage
and the moderating role of regulatory quality: A cross-country study”, The Thunderbird
International Business Review, vol. 53, No. 5, pp. 629-646.

Zimbabwe Human Rights Commission (2018). ZHRC 2018 Harmonized Election Report.
Available at: http://www.zhrc.org.zw/download/zhrc-final-report-on-zimbabwe-2018-
harmonised-elections/

HANDBOOK ON GOVERNANCE STATISTICS | 249

 ANNEXES

All annexes of this Handbook can be accessed online at

http://ine.cv/praiagroup/handbook/handbookannexes.pdf

250 |