KCE REPORT 183C

BELGIAN GUIDELINES FOR ECONOMIC

EVALUATIONS AND BUDGET IMPACT ANALYSES: SECOND
EDITION

2012 www.kce.fgov.be
Belgian Health Care Knowledge Centre
The Belgian Health Care Knowledge Centre (KCE) is an organization of public interest, created on the 24th of December
2002 under the supervision of the Minister of Public Health and Social Affairs. KCE is in charge of conducting studies
that support the political decision making on health care and health insurance.
Actual Members Substitute Members
Executive Board
President Pierre Gillet
CEO - National Institute for Health and Disability Insurance (vice Jo De Cock Benoît Collin
president)
President of the Federal Public Service Health, Food Chain Safety and Dirk Cuypers Chris Decoster
Environment (vice president)
President of the Federal Public Service Social Security (vice president) Frank Van Massenhove Jan Bertels
General Administrator of the Federal Agency for Medicines and Health Xavier De Cuyper Greet Musch
Products
Representatives of the Minister of Public Health Bernard Lange François Perl
Marco Schetgen Annick Poncé
Representatives of the Minister of Social Affairs Oliver de Stexhe Karel Vermeyen
Ri De Ridder Lambert Stamatakis
Representatives of the Council of Ministers Jean-Noël Godin Frédéric Lernoux
Daniel Devos Bart Ooghe
Intermutualistic Agency Michiel Callens Frank De Smet
Patrick Verertbruggen Yolande Husden
Xavier Brenez Geert Messiaen
Professional Organisations - representatives of physicians Marc Moens Roland Lemye
Jean-Pierre Baeyens Rita Cuypers
Professional Organisations - representatives of nurses Michel Foulon Ludo Meyers
Myriam Hubinon Olivier Thonon
Hospital Federations Johan Pauwels Katrien Kesteloot
Jean-Claude Praet Pierre Smiets
Social Partners Rita Thys Leo Neels
Paul Palsterman Celien Van Moerkerke
House of Representatives Lieve Wierinck
Control Government commissioner Yves Roger

Management Chief Executive Officer Raf Mertens

Assistant Chief Executive Officer Jean-Pierre Closon
Managers Program Management Christian Léonard
Kristel De Gauquier

Contact Belgian Health Care Knowledge Centre (KCE)

Doorbuilding (10th Floor)
Boulevard du Jardin Botanique, 55
B-1000 Brussels
Belgium

T +32 [0]2 287 33 88

F +32 [0]2 287 33 85
info@kce.fgov.be
http://www.kce.fgov.be
KCE REPORT 183C
HEALTH TECHNOLOGY ASSESSMENT

BELGIAN GUIDELINES FOR ECONOMIC

EVALUATIONS AND BUDGET IMPACT ANALYSES: SECOND
EDITION

IRINA CLEEMPUT, MATTIAS NEYT, STEFAAN VAN DE SANDE, NANCY THIRY

2012 www.kce.fgov.be
COLOPHON
Title: Belgian guidelines for economic evaluations and budget impact analyses: second edition
Authors: Irina Cleemput, Mattias Neyt, Stefaan Van de Sande, Nancy Thiry.
Reviewers: Chris De Laet, Sophie Gerkens, Christian Léonard, Raf Mertens, Caroline Obyn.
External Experts: Philippe Beutels (Universiteit Antwerpen), Rémi Delépine (CTI – TRI, INAMI – RIZIV), Joeri Guillaume (IMA – AIM,
Nationaal Verbond van Socialistische Mutualiteiten), Hans Hellinckx (UNAMEC), Sophie Marbaix (Pfizer), Yves
Parmentier (TCT, INAMI – RIZIV), Hugo Robays (CTG – CRM, RIZIV – INAMI, Universitair Ziekenhuis Gent),
Markus Siebert (St. Jude Medical), Françoise Stryckman (Pharma.be), François Sumkay (AIM – IMA, Alliance
Nationale des Mutualités Chrétiennes), Marc Van De Casteele (CTG – CRM, RIZIV – INAMI), Piet Vancraeynest
(CTG – CRM, RIZIV – INAMI).
External Validators: Pierre Chevalier (INAMI – RIZIV, Université Catholique de Louvain (UCL)), Andreas Gerber (Institute for Quality and
Efficiency in Health Care (IQWiG), Deutschland), Maarten Postma (Unit of Pharmacoepidemiology and
Pharmacoeconomics, Department of Pharmacy, Universiteit Groningen, Nederland).
Conflict of interest: Owner of subscribed capital, options, share or other financial instruments: Sophie Marbaix, Markus Siebert,
Françoise Stryckman.
Fees or other compensation for writing a publication or participating in its development: Hans Hellincks, Sophie
Marbaix, Maarten Postma.
A grant, fees or funds for a member of staff or another form of compensation for the execution of research: Sophie
Marbaix, Maarten Postma, Hugo Robays.
Consultancy or employment for an organisation that may gain or lose financially due to the results of this report:
Hans Hellincks, Sophie Marbaix, Markus Siebert, Françoise Stryckman.
Payments to speak, training remuneration, subsidised travel or payment for participation at a conference:; Hans
Hellincks, Sophie Marbaix, Maarten Postma, Markus Siebert.
Any other direct or indirect relationship with a producer, distributor or healthcare institution that could be interpreted
as a conflict of interests: Françoise Stryckman, Sophie Marbaix, Hans Hellincks, Markus Siebert.
Layout: Ine Verhulst, Sophie Vaes
Disclaimer:  The external experts were consulted about a (preliminary) version of the scientific report. Their
comments were discussed during meetings. They did not co-author the scientific report and did not
necessarily agree with its content.
 Subsequently, a (final) version was submitted to the validators. The validation of the report results from
a consensus or a voting process between the validators. The validators did not co-author the scientific
report and did not necessarily all three agree with its content.
 Finally, this report has been approved by common assent by the Executive Board.
 Only the KCE is responsible for errors or omissions that could persist. The policy recommendations are
also under the full responsibility of the KCE.
Publication date: 14 December 2015 (2nd edition; 1st edition: 17 July 2012)
Update: This report is an update of the following previously published KCE report: Cleemput I, Van Wilder P, Vrijens F,
Huybrechts M, Ramaekers D. Guidelines for Pharmacoeconomic Evaluations in Belgium. Health Technology
Assessment (HTA). Brussels: Health Care Knowledge Centre (KCE); 2008. KCE Reports 78.
Domain: Health Technology Assessment (HTA)
MeSH: Economics, Pharmaceutical; Guidelines
NLM Classification: QV 736
Language: English
Format: Adobe® PDF™ (A4)
Legal depot: D/2012/10.273/54
Copyright: KCE reports are published under a “by/nc/nd” Creative Commons Licence
http://kce.fgov.be/content/about-copyrights-for-kce-reports.

How to refer to this document? Cleemput I, Neyt M, Van de Sande S, Thiry N. Belgian guidelines for economic evaluations and budget impact
analyses: second edition. Health Technology Assessment (HTA). Brussels: Belgian Health Care Knowledge
Centre(KCE). 2012. KCE Report 183C. D/2012/10.273/54
This document is available on the website of the Belgian Health Care Knowledge Centre.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 1

 TABLE OF CONTENTS
LIST OF FIGURES ....................................................................................................................................................... 5
LIST OF TABLES ......................................................................................................................................................... 5
LIST OF ABBREVIATIONS ......................................................................................................................................... 6
 SCIENTIFIC REPORT ................................................................................................................................... 8
1 BACKGROUND ............................................................................................................................................. 8
2 OBJECTIVES ................................................................................................................................................. 9
3 METHODS.................................................................................................................................................... 10
4 OVERVIEW OF THE BELGIAN GUIDELINES AND GENERAL REMARKS ............................................ 11
4.1 OVERVIEW OF THE BELGIAN GUIDELINES ............................................................................................ 11
4.2 GENERAL REMARKS ................................................................................................................................. 13
5 GUIDELINES FOR HEALTH ECONOMIC EVALUATIONS ....................................................................... 14
5.1 GUIDELINE 1: LITERATURE REVIEW ....................................................................................................... 14
5.2 GUIDELINE 2: PERSPECTIVE OF THE EVALUATION ............................................................................. 16
5.3 GUIDELINE 3: TARGET POPULATION ...................................................................................................... 17
5.4 GUIDELINE 4: COMPARATORS ................................................................................................................. 19
5.5 GUIDELINE 5: ANALYTIC TECHNIQUE ..................................................................................................... 20
5.5.1 Cost-effectiveness analysis ........................................................................................................... 21
5.5.2 Cost-utility analysis ........................................................................................................................ 21
5.5.3 Cost-minimisation analysis ............................................................................................................ 22
5.5.4 Cost-benefit analysis ...................................................................................................................... 22
5.6 GUIDELINE 6: STUDY DESIGN .................................................................................................................. 22
5.6.1 Trial-based economic evaluations ................................................................................................. 23
5.6.2 Modelling ........................................................................................................................................ 24
5.7 GUIDELINE 7: CALCULATION OF COSTS ................................................................................................ 24
5.7.1 Cost categories .............................................................................................................................. 25
5.7.2 Measurement of resource use ....................................................................................................... 25
2 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

5.7.3 Valuation of resource use .............................................................................................................. 26

5.8 GUIDELINE 8: ESTIMATION AND VALUATION OF OUTCOMES ............................................................. 36
5.8.1 Effectiveness evaluation in cost-effectiveness analysis ................................................................ 37
5.8.2 Utility assessment in cost-utility analysis ....................................................................................... 37
5.9 GUIDELINE 9: TIME HORIZON ................................................................................................................... 39
5.10 GUIDELINE 10: MODELLING ...................................................................................................................... 39
5.10.1 Need for modelling ......................................................................................................................... 39
5.10.2 Choice of the model design ........................................................................................................... 40
5.10.3 Precision of model structure and hypotheses ................................................................................ 40
5.10.4 Calibration, face validity and cross-validation of a model .............................................................. 41
5.11 GUIDELINE 11: HANDLING UNCERTAINTY AND TESTING THE ROBUSTNESS OF THE RESULTS .. 42
5.12 GUIDELINE 12: DISCOUNT RATE ............................................................................................................. 43
6 GUIDELINES FOR BUDGET IMPACT ANALYSES ................................................................................... 44
6.1 SIMILARITIES AND DIFFERENCES BETWEEN ECONOMIC EVALUATIONS AND BUDGET IMPACT
ANALYSES................................................................................................................................................... 44
6.2 GUIDELINE 13: PERSPECTIVE .................................................................................................................. 45
6.3 GUIDELINE 14: TARGET POPULATION .................................................................................................... 45
6.4 GUIDELINE 15: COMPARATORS ............................................................................................................... 46
6.5 GUIDELINE 16: COSTS AND OUTCOMES ................................................................................................ 47
6.6 GUIDELINE 17: TIME HORIZON ................................................................................................................. 48
6.7 GUIDELINE 18: MODELLING ...................................................................................................................... 48
6.8 GUIDELINE 19: HANDLING UNCERTAINTY.............................................................................................. 49
6.9 GUIDELINE 20: DISCOUNT RATE ............................................................................................................. 49
6.10 GUIDELINE 21: PRESENTING RESULTS .................................................................................................. 49
7 DISCUSSION ............................................................................................................................................... 50
8 POLICY RECOMMENDATIONS ................................................................................................................. 51
9 REPORTING GUIDELINES ......................................................................................................................... 53
9.1 EXECUTIVE SUMMARY.............................................................................................................................. 53
9.2 INTRODUCTION .......................................................................................................................................... 53
9.3 OBJECTIVES ............................................................................................................................................... 53
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 3

9.4 LITERATURE REVIEW ................................................................................................................................ 53

9.4.1 Clinical literature review ................................................................................................................. 53
9.4.2 Economic literature review ............................................................................................................. 53
9.5 BASIC ELEMENTS OF THE ECONOMIC EVALUATION ........................................................................... 54
9.5.1 Analytic technique .......................................................................................................................... 54
9.5.2 Study design .................................................................................................................................. 54
9.5.3 Methods used for valuation of costs .............................................................................................. 54
9.5.4 Methods used for outcome assessment ........................................................................................ 54
9.5.5 Method of analysis of the data ....................................................................................................... 54
9.5.6 Time horizon and discount rate...................................................................................................... 54
9.5.7 Sensitivity analysis ......................................................................................................................... 54
9.6 RESEARCH METHODS .............................................................................................................................. 54
9.6.1 Identification, measurement and valuation of costs ....................................................................... 54
9.6.2 Identification, measurement and valuation of health-related outcomes ........................................ 54
9.7 RESULTS ..................................................................................................................................................... 55
9.7.1 Basic results ................................................................................................................................... 55
9.7.2 Uncertainty analysis ....................................................................................................................... 55
9.8 DISCUSSION ............................................................................................................................................... 55
9.9 CONCLUSION.............................................................................................................................................. 55
9.10 TRANSPARENCY OF FINANCIAL SUPPORT ........................................................................................... 55
9.11 REFERENCES ............................................................................................................................................. 55
9.12 ADDENDA .................................................................................................................................................... 55
10 PRESENTATION OF A MODEL.................................................................................................................. 56
10.1 DATA ............................................................................................................................................................ 56
10.2 RESULTS ..................................................................................................................................................... 56
10.2.1 Reference case analysis ................................................................................................................ 56
10.2.2 Uncertainty analysis ....................................................................................................................... 56
11 METHODOLOGICAL REFERENCES BY TOPIC ....................................................................................... 57
11.1 COUNTRY-SPECIFIC GUIDELINES ........................................................................................................... 57
4 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

11.2 METHODS FOR ECONOMIC EVALUATIONS............................................................................................ 57

11.3 LITERATURE REVIEW ................................................................................................................................ 57
11.4 STUDY DESIGN .......................................................................................................................................... 57
11.4.1 Trial-based economic evaluations ................................................................................................. 57
11.4.2 Modelling ........................................................................................................................................ 57
11.5 CALCULATION OF COSTS ......................................................................................................................... 58
11.5.1 Measurement of resource use ....................................................................................................... 58
11.5.2 Valuation of resource use .............................................................................................................. 58
11.6 HANDLING UNCERTAINTY ........................................................................................................................ 58
11.6.1 Overview ........................................................................................................................................ 58
11.6.2 Confidence interval around the ICER ............................................................................................ 58
11.7 INDIRECT COMPARISONS ........................................................................................................................ 58
11.8 DISCOUNTING ............................................................................................................................................ 59
11.9 BUDGET IMPACT ANALYSES .................................................................................................................... 59
 APPENDICES .............................................................................................................................................. 60
APPENDIX 1. CLASSIFICATION OF ECONOMIC STUDIES ............................................................................. 60
APPENDIX 2. CHECKLIST FOR ASSESSING ECONOMIC EVALUATIONS.................................................... 61
APPENDIX 3. DATA EXTRACTION SHEET FOR ECONOMIC EVALUATIONS ............................................... 61
APPENDIX 4. TECHNICAL NOTES ..................................................................................................................... 62
APPENDIX 4.1. ASSESSMENT OF EXTERNAL VALIDITY ................................................................................... 62
APPENDIX 4.2. OUTCOME VALUATION ............................................................................................................... 63
APPENDIX 4.3. USE OF EXPERT PANELS .......................................................................................................... 64
APPENDIX 4.4. HOSPITAL PER DIEM PRICES .................................................................................................... 64
APPENDIX 4.5. INDIRECT COMPARISONS ......................................................................................................... 66
APPENDIX 5. THE COST-EFFECTIVENESS PLANE, COST-EFFECTIVENESS ACCEPTABILITY
CURVE, AND EFFICIENCY FRONTIER....................................................................................... 67
APPENDIX 6. LIST OF BELGIAN DATABASES FOR THE MEASUREMENT AND VALUATION OF
RESOURCE USE .......................................................................................................................... 70
APPENDIX 7. FLEMISH EQ-5D INDEX VALUES ............................................................................................... 73
 REFERENCES ............................................................................................................................................. 74
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 5

Figure 1 – Classification of economic studies ....................................................................................................60

LIST OF FIGURES
Figure 2 – The cost-effectiveness plane and cost-effectiveness acceptability curve .........................................68
Figure 3 – The efficiency frontier ........................................................................................................................69

Table 1 – Reference case methods for economic evaluations...........................................................................11

LIST OF TABLES
Table 2 – Reference case methods for budget impact analyses .......................................................................12
Table 3 – Included and excluded costs in the reference case analysis .............................................................25
Table 4 – Weighted average of the 100% per diem hospital prices per type of stay, Belgium (2004-2010) .....28
Table 5 – Sources of financing – Hospital drugs for patients in general acute hospitals ...................................29
Table 6 – Total RIZIV–INAMI expenses on inpatient drugs (drugs included in the prospective budget only) ...30
Table 7 – Sources of financing – Hospital laboratory testing for patients in general hospitals (2012 values) ...31
Table 8 – Total RIZIV–INAMI expenses on inpatient laboratory testing.............................................................32
Table 9 – Sources of financing – Hospital medical imaging for patients in general hospitals ............................33
Table 10 – Total RIZIV–INAMI expenses on inpatient medical imaging ............................................................33
Table 11 – Description of the variables used in a model: template ....................................................................40
Table 12 – Similarities and differences between economic evaluations and budget impact analyses. .............45
Table 13 – Presentation of the results of an economic evaluation: template ....................................................55
Table 14 – Data extraction sheet for economic evaluations: template ...............................................................61
Table 15 – Components of the hospital budget of financial means....................................................................65
Table 16 – Belgian databases for the measurement and valuation of resources use .......................................70
Table 17 – Flemish EQ-5D index values ............................................................................................................73
6 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

ABBREVIATION DEFINITION
LIST OF ABBREVIATIONS
APR-DRG All Patient Refined-Diagnosis Related Group
BCFI – CBIP Belgisch Centrum voor Farmacotherapeutische Informatie – Centre Belge d'Information
Pharmacothérapeutique
BIA Budget impact analysis
B.S. – M.B. Belgisch Staatsblad – Moniteur Belge
CEA Cost-effectiveness analysis
CI Confidence interval
CrI Credibility interval
CTG – CRM Commissie Tegemoetkoming Geneesmiddelen – Commission de Remboursement des
Médicaments (Drug Reimbursement Committee)
CUA Cost-utility analysis
DSM Diagnostic and Statistical Manual of Mental Disorders
EMA European Medicine Agency
FOD – SPF Federale Overheidsdienst – Service Publique Fédéral
HTA Health Technology Assessment
IC Incremental costs
ICD-9CM International Classification of Diseases-9th Revision, Clinical Modification
ICER Incremental cost-effectiveness ratio
IE Incremental effects
IMA – AIM Intermutualistisch Agentschap – Agence Intermutualiste
LY Life year
MFG – RFM Minimaal financiële gegevens – Résumé financier minimum
MKG – RCM Minimaal klinische gegevens – Résumé clinique minimum
OCMW – CPAS Openbaar Centrum voor Maatschappelijk Welzijn – Centre Public d’Action Sociale
OTC Over the counter
PSA Probabilistic sensitivity analysis
QALY Quality-adjusted life year
RCT Randomized controlled trial
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 7

RIZIV – INAMI Rijksinstituut voor Ziekte- en Invaliditeitsverzekering – Institut National d’Assurance

Maladie-Invalidité (National Institute for Health and Disability Insurance)
SD Standard deviation
SG Standard gamble
TCT Technische Cel voor het beheer van de MKG-MFG data – Cellule Technique pour la
gestion des données RCM-RFM
TRI – CTI Technische Raad voor Implantaten – Conseil Technique des Implants (Technical
Council of Implants)
TTO Time-trade off
RVV – BIM Rechthebbende op verhoogde verzekeringstegemoetkoming – Bénéficiaire de
l’intervention majorée
8 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

 SCIENTIFIC REPORT 1 BACKGROUND

Since 2002, a request for reimbursement of a class 1 pharmaceutical
product by a pharmaceutical company has to be accompanied by an
economic evaluation. Class 1 drugs are drugs with a therapeutic added
value compared to existing therapeutic alternatives, class 2 drugs are those
with comparable therapeutic value and class 3 drugs are mainly generics.
Reimbursement requests are evaluated by the Drug Reimbursement
Committee (CTG–CRM) based on the preparatory assessments by the
experts of the RIZIV–INAMI administration. The decision to list and
reimburse and the level of reimbursement of a class 1 drug is based on 5
criteria (art. 4 and art. 6 of the December 2001 Royal Decree).37
 The therapeutic value, taking into account the efficacy, effectiveness,
side effects, applicability and user-friendliness of the product,
 The market price of the drug,
 The clinical effectiveness and likely impact of the product, taking into
account therapeutic and social needs,
 The budgetary impact for the National Health Insurance,
 The cost-effectiveness of the product from the perspective of the
National Health Insurance.
In contrast to cost-effectiveness, budget impact is not only a reimbursement
criteria for class 1 drugs, but for all reimbursement requests.
From published data on class 1 requests in the period 2002-2004, it
appeared that the claim of ‘added therapeutic value’ was approved after
evaluation in only 48% of class 1 submissions,1 which is of particular
importance to the subsequent economic evaluation.
The definition of therapeutic value used in the Royal Decree is wider than
the notion of effectiveness or outcome, as frequently used in clinical and
economic literature. Besides morbidity, mortality and health-related quality
of life it encompasses social and practical components such as applicability
of the product and comfort of use. This larger definition has implications for
the assessment of the cost-effectiveness of a product. While usual outcome
parameters in economic evaluations are morbidity, mortality and/or health-
related quality of life, additional reflections and analysis may be necessary
to describe the therapeutic (added) value of a product.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 9

Based on an evaluation by a competent experts committee of the

reimbursement report submitted by the pharmaceutical company, the Drug
2 OBJECTIVES
Reimbursement Committee formulates a motivated advice for the Minister The objective of this study was to develop methodological and reporting
of Health & Social Affairs about the class of the product (class 1, 2 or 3), the guidelines for economic evaluations and budget impact analyses of medical
appropriateness of reimbursement, the reimbursement rate and the interventions, be it pharmaceutical, medical device or other interventions,
conditions for reimbursement. submitted to expert committees at the RIZIV–INAMI, advising the health
The legal basis for the assessment criteria for reimbursement requests of minister about reimbursement. An economic evaluation is defined as a
implants and invasive medical devices exists but is not implemented yet comparative analysis of at least two health interventions in terms of their
(Chapter III of the law of 14 July 1994 on the compulsory insurance of costs and health consequences. A budget impact analysis (BIA) is defined
healthcare (B.S.–M.B. 27/08/1994)). So far, there is no requirement to as the application of methods to estimate planned resource use and
perform an economic evaluation nor an assessment of the budgetary impact expenditure of a budget over a period of time.2 The aim of BIA is to measure
of new devices for which reimbursement is requested. Even if this is not an the impact of a (possible) policy decision concerning a certain intervention
obligation (yet), economic evaluations are nonetheless usefull for medical on the healthcare budget.
devices and other health care interventions in order to stimulate the efficient Any intervention for which a health economic evaluation or a budget impact
use of our limited resources. Therefore, these guidelines are also useful to analysis is required by the RIZIV–INAMI for reimbursement, should be
devices and other medical interventions. assessed following these guidelines. Any deviation needs a clear and
In 2008, the KCE published a set of guidelines for pharmacoeconomic detailed justification.
evaluations in Belgium. This set of guidelines has now been tested The aim of these guidelines is to increase the methodological quality,
extensively by both independent researchers and pharmaceutical transparency and uniformity of the economic evaluations and budget-impact
companies. These experiences have highlighted the need for clarification of analyses of medical interventions in Belgium. They do not relate to the
some specific guidelines, and the further development of others. Moreover, procedures for the evaluation of reimbursement request dossiers, or to the
the need for specific guidelines for budget impact analyses has been methods used to arrive at a recommendation for reimbursement. Hence,
expressed. This report presents an update and extension of the guidelines compliance with the methodological and reporting guidelines for economic
developed in 2008. Compared with the previous report, the updated evaluations and budget-impact analyses as specified in this report does not
guidelines add a new set of recommendations for budget impact analyses imply a positive reimbursement advice. The better transparency and quality
and extend to all medical interventions. All guidelines were critically of the files will help the appraisal committees in formulating a better informed
reviewed. Major amendments relate to guidelines on quality of life, costs and advice, but the advice itself remains entirely the committee’s responsibility.
the comparator. These guidelines will assist both the performers and assessors in making
and evaluating economic evaluations and BIAs. Furthermore, it may assist
researchers in identifying relevant parameters that should be gathered in
their study protocols in order to allow robust economic evaluations.
10 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

Phase three involved the updating of the guidelines based on external

3 METHODS feedback from users of the guidelines and personal experience, as well as
The development of this second version of the guidelines was done in three the development of a guideline for budget impact analysis. The update was
phases. performed by three KCE health economists and discussed with a
Phase one consisted of the development of a set of draft guidelines in 2006. multidisciplinary team consisting of external health economists, experts of
These guidelines were developed by eight health economists from Belgium the Drug Reimbursement Committee and the Technical Council for Implants
and abroad, two pharmacists, one medical doctor with training in health of the RIZIV–INAMI, representatives of Belgian databases, Pharma.be and
economics and one statistician. Existing guidelines from other countries UNAMEC. The guideline on budget impact analysis was developed by KCE
were reviewed. Only guidelines issued or updated after July 2003 were health economists. In order to collect practical experiences with budget
considered, because the field of health economics is continually evolving impact guidelines, a workshop was organized with three health economists
and regular updates are necessary. The guidelines were mainly based on from The German Institute for Quality and Efficiency in Healthcare (IQWIG,
the Dutch (College voor Zorgverzekeringen, CVZ), French (Collège des Germany). The budget impact guideline was adapted accordingly.
Économistes de la Santé, CES), Australian (Pharmaceutical Benefits For most methodological aspects, different approaches exist. To improve
Advisory Committee, PBAC) and British (National Institute for Clinical consistency in the files, we present a “reference case”, including the
Excellence, NICE) guidelines.a Other guidelines were identified, but did not essential elements for each economic evaluation or budget impact analysis
add knowledge or recommendations to the ones reviewed. together with the most appropriate methodology. The expert committees
Phase two consisted of a practical implementation of these guidelines during could request an economic evaluation and/or a budget impact analysis
a 6 to 12-month test period. This pilot phase lead to conclusions about the according to these “reference case methods” in order to enhance
practicality and usefulness of the guidelines and to potential improvements consistency between submissions. Additional analyses are allowed, but
in the guidelines. Participation in the pilot test was voluntary. One company should be distinguished from the results of the reference case analysis.
submitted an adaptation according to the draft guidelines of an earlier Variations to the reference case should be justified and well-argued. It is
submitted economic evaluation of a product for which the reimbursement then up to the committee to decide how much weight it attaches to the
decision was already taken. This approach was taken to strictly separate the additional analyses.
evaluation of the feasibility and usefulness of the guidelines from the For each guideline, a short bibliography is provided in Chapter 11. The core
procedural evaluation of the content of the reimbursement request file. text of the guidelines is deliberately kept relatively brief, especially for items
Based on the experience of this company and the extensive feedback of for which there is little discussion about the most appropriate methodology.
about 20 pharmaceutical companies through the representative The document aims to serve as an easy working document for both
organization of the pharmaceutical industry in Belgium Pharma.be, the evaluators and applicants. Therefore, the executive summary
guidelines were adapted and finalized in 2008.3 These guidelines were accompanying this report simply lists all guidelines to provide a quick
applicable to all new reimbursement request dossiers that (have to) include overview. The appendices provide supportive documents for the economic
a pharmacoeconomic evaluation. Companies were strongly recommended evaluations and BIAs, and elaborate on some technical aspects of the
to follow these guidelines for every economic evaluation submitted in the guidelines.
context of a reimbursement dossier.

a Full references to these guidelines can be found in Chapter 11.

KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 11

The guidelines for the economic evaluations and budget impact analyses
are treated in Chapters 5 and 6, repsectively. A general discussion related
4 OVERVIEW OF THE BELGIAN
to the guidelines and the use of economic evaluations and budget impact GUIDELINES AND GENERAL REMARKS
analyses is provided in Chapter 7. Recommendations for Belgian policy
makers are formulated in Chapter 8. Chapter 9 and Chapter 10 present 4.1 Overview of the Belgian guidelines
reporting guidelines for health technology assessments and models, The reviewed guidelines show very limited differences amongst each other.
respectively. Differences relate for instance to the perspective to be taken, the cost items
to be included and the discount rate for costs and outcomes.
The reference case defines the elements of an economic evaluation or
budget impact analysis and the recommended methodology for each
component. We are aware that discussion about the appropriateness of the
recommended methodology is possible. Such discussion may relate to value
judgments (e.g. the choice of the perspective or time preference for health
benefits) or technical aspects (e.g. the choice of the uncertainty analysis).
The reference cases for economic evaluations and budget impact analyses
are presented in Table 1 and Table 2, respectively.

Table 1 – Reference case methods for economic evaluations

Component of an Reference case Guideline
economic
evaluation

Literature review Systematic review of up-to-date clinical and 1

economic literature following methodological
standards: reproducible search strategy, transparent
selection criteria, critical appraisal.

Perspective of the Costs: Health care payers (federal government + 2

evaluation communities + patients).
Outcomes: Society. For health-related quality of life,
health states should be described by patients on a
generic instrument. Health state valuations for these
states should come from the general public.

Target population Consistent with the clinical file. Relevant subgroups 3

need to be defined. Post-hoc subgroup analyses only
in case of statistical proof of difference in costs or
baseline risk between the post-hoc subgroups.

Comparator Economic relevant comparisons are performed on 4

the efficiency frontier.
12 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

Analytic technique Cost-effectiveness analysis (CEA) or cost-utility 5 Table 2 – Reference case methods for budget impact analyses
analysis (CUA), choice should be justified.
Component of a Reference case Guideline
Study design Economic evaluation based as much as possible on 6 budget impact
data from head-to-head comparisons between the analysis
study product and the comparator.
Perspective of the See economic evaluation guideline. 2
Calculation of costs Health care costs paid out of the health care budget, 7 evaluation
by the federal government, the communities and the 13
patients. Target population See economic evaluation guideline. 3
Valuation of Final endpoints. 8 Calculate the yearly budget impact up to the steady 14
outcomes Cost-effectiveness analyses: life years gained for state. The size of the population may vary over time.
interventions with an impact on mortality. Comparator The current situation that might change if the 15
Cost-utility analyses: QALYs, with quality-of-life intervention under consideration is introduced in the
weights based on empirical data obtained with a healthcare system.
generic quality-of-life instrument such as the EQ-5D
for which public preference values exist. Calculation of costs See economic evaluation guideline. 7

Time horizon The appropriate time horizon for the economic 9 Costs of health The cost consequences of the treatment effect, side 16
evaluation depends on the duration of the impact of outcomes effects and other short- and long-term
the study intervention on relevant outcomes as consequences should be included in the BIA.
compared to the comparator intervention.
Time horizon The time needed to reach a steady state budget 17
Modelling Based as much as possible on data from clinical 10 impact, with a minimum time horizon of 3 years (if
studies comparing the study medication and the the steady state is already reached in the short-
comparator, data from validated databases and/or term).
data from literature. Model inputs and outputs
Modelling See economic evaluation guideline. 10
consistent with existing data. Face validity checked.
Clear presentation of structural hypotheses, Consistent with the clinical and economic 18
assumptions and sources of information. assumptions in the economic evaluation.

Handling Probabilistic sensitivity analyses for parameter 11 Handling See economic evaluation guideline. 11
uncertainty uncertainty. uncertainty 19
Scenario analyses for analyses of methodological
and structural uncertainty. Discount rate No discounting. 20
Presentation of uncertainty around the incremental
costs (IC), incremental effects (IE) and ICERs by
means of confidence or credibility intervals. Results Before starting with the actual guidelines, some general remarks are made
shown on the cost-effectiveness plane and cost-
effectiveness acceptability curve.
about economic evaluations and budget impact analyses of medical
interventions.
Discount rate 3% on costs and 1.5% on outcomes. 12
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 13

4.2 General remarks For reimbursement revisions, e.g. 1.5 to 3 years after the initial submission
for class 1 pharmaceutical products and orphan drugs, real life effectiveness
Data requirements for good economic evaluations and budget-impact
data should be made available. In cases where it has been decided to
analyses are high. However, even though good quality data for Belgium are
temporarily reimburse an intervention, however, e.g. because of uncertainty
often available at the company’s level or at the governmental level, these
about the efficacy of the intervention, it is more important to strengthen the
data are not always easily accessible. For example, companies may find it
evidence-base for the efficacy by means of an RCT.
difficult to estimate the average costs of treating a specific complication in
Belgium without access to individual data on health care expenditures; while It is clear that at the initial submission, such evidence is rarely available, as
governmental agencies may find it difficult to perform or verify economic the new intervention is not (yet) widely used. Therefore, if companies or
evaluations of medical interventions without access to complete data of all providers would start early with thinking about the organization of an
clinical trials. Confidentiality, secrecy and publication bias hamper the quality effectiveness evaluation study and the collection of economic data alongside
of economic evaluations and budget impact analyses in Belgium as well as this study (e.g. at the time of submission of the drug or device registration
their relevance for reimbursement decisions. Access to essential public data request), this kind of evidence may be available at the time of the initial
for the measuement and valuation of resource use for the people performing reimbursement request. This would strengthen the economic evaluation. If
economic evaluations, including companies, their sub-contractors for the still insufficient data are available from the study at the time of the initial
economic evaluation and other experts performing health economic submission, more data will nevertheless be available at the time of the
evaluations that serve resource allocation decisions is often limited. revision 1.5 to 3 years after the initial submission. Especially for drugs and
devices with potentially long-term effects, which would not be observed in a
Full access to relevant study results should be pursued. Currently, there is
one or two year clinical study, it may be particularly interesting to start
no legal framework that obliges manufacturers to provide all relevant
organizing an active control study at the time of registration of a product.
evidence, e.g. both positive and negative trial results. It is desirable that the
producer submits a list of all studies and provide transparency about the Each economic evaluation and budget impact analysis should be
results. This is essential to perform a balanced assessment of the accompanied by an adequate description of the disease and the therapy.
intervention. Current obligations to register trials seem not sufficient. Not all This description should provide information about the illness or health
trials are registered at onset, nor are all results presented in due time.4 It problem, including a specification of the disease area (pathology/problem),
should be explicitly stated if companies are not able or willing to provide the epidemiology (incidence and prevalence, in absolute and relative figures,
requested information. e.g. per 100 000 inhabitants), the natural evolution of the illness, its morbidity
and mortality and the current clinical practice. The information provided
Analogous to the clinical file, the document describing the model should be
should be as relevant as possible for Belgium.
signed by the author(s) taking the responsibility for the model. Their contact
details should be provided. Whenever extrapolations are performed, e.g. from Flemish, Walloon or
foreign data to Belgium, methods for extrapolation should be clearly
For reimbursement decisions, it is preferred that the outcome data used in
described.
economic evaluations reflect the interventions’ effectiveness in daily practice
(i.e. effectiveness in contrast to efficacy). Because effectiveness data are
usually not available (yet) at the time of the initial reimbursement request,
efficacy results are often transposed to the real life target population to
estimate effectiveness in a cost-effectiveness analysis. This is acceptable,
as long as adjustments are made for baseline risk in the real life target
population.5
14 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

Belgium does not use an explicit discrete threshold value for incremental
cost-effectiveness ratios below which an intervention is considered
5 GUIDELINES FOR HEALTH ECONOMIC
worthwhile and above which it is not.6, 7 Referring to such thresholds from EVALUATIONS
other countries should be avoided. A cost-effectiveness threshold -if we
assume that it exists- is very context dependent. It depends, for instance, on 5.1 Guideline 1: Literature review
the available health care budget and the interventions already financed in a
country. Therefore, it does not make sense to refer to a previously stated Each economic evaluation should be accompanied by a description of
threshold or a threshold from another country in a Belgian economic the disease and the interventions studied and a systematic review of
evaluation. Comparisons with other currently (non-)reimbursed interventions the existing relevant clinical literature. The review should reveal up-to-
are difficult to interpret, since it is not clear whether economic or other date evidence for clinical effectiveness of the intervention relative to
arguments have been considered or played a decisive role. its appropriate comparator(s). A review of economic studies is useful
Key points to identify relevant input parameters for the economic model and to
support the assessors. The medical and economic search strategies
 Access to good quality Belgian data for the measurement and should be reproducible and selection criteria and procedures clearly
valuation of resource use should be facilitated to allow for presented. The evidence should be critically appraised, its quality
economic evaluations with higher relevance for health care policy assessed and data presented in summary/evidence tables. A clear and
makers. concise synthesis, substantiated with references, should be provided.
 In order to avoid bias and formulate balanced recommendations, Ongoing studies should be mentioned.
initiatives should be taken to have access to all relevant evidence.
 Companies might consider the organisation of an active control For a full overview of the clinical effectiveness of a medical intervention, it is
study already at the time of registration of a product to increase crucial to start with a thorough and systematic literature review on the safety
the relevance of the economic evaluation either at initial and efficacy and/or effectiveness of the interventions. The review should be
submission or at revision. based on the best available up-to-date evidence for clinical effectiveness of
the intervention and the comparator(s). Besides published literature, an
 Belgium does not apply a threshold value for the incremental cost- overview of ongoing studies should be provided. The relationship with the
effectiveness ratio. Referring to threshold values applied in other clinical literature review submitted for the registration on the Belgian market
countries or based on previous decicions should be avoided. should be clear.
The value of an economic evaluation crucially depends on the value of the
evidence it is based upon. A review of economic studies is therefore useful
to identify relevant input parameters for an economic model and to support
the assessors.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 15

Unless there is evidence about their clinical efficacy and safety, “off-label” A good review starts with identification of the review questions. This includes
medical treatments are not acceptable as comparators in the formal specification of the population, the intervention, the comparator(s), the
economic evaluation. The evidence on their (cost-)effectiveness can outcomes and the study designs selected (PICOS: Patient, Intervention,
nevertheless be described in the literature review. This is not a formal Comparator, Outcome, Study Design). As for the outcomes, it is worth
requirement, but for the Drug Reimbursement Committee the existence and considering (1) disease-specific outcomes, (2) adverse events, (3) overall
current use of an “off-label” used product can sometimes be a consideration survival and (4) quality of life, for both the intervention and the comparator.
in its advice to the minister. The applicant therefore has an interest in The review should moreover contain the search strategy, study selection
presenting the evidence on off-label used products in his literature review. criteria and procedures followed for selecting studies, study quality
This increases the transparency of the dossier. assessment, data extraction sheets, and a synthesis of the evidence found.
Selective presentation of evidence is a pitfall. From the point of view of the The approach to find general HTA reports will be the same for both the
applicant it may be felt that selective presentation of evidence provides a clinical and economic search strategy, e.g. searching CRD’s HTA database
stronger case for the economic evaluation, but from the point of view of the and websites of HTA institutes. However, to retrieve primary clinical and
assessor this creates suspicion about the validity and reliability of the economic studies, different databases and search strings will be used.
economic evaluation. Therefore, it is important to even include studies in the Databases searched for clinical evidence should include at least:
review that are not directly used in the economic evaluation if they are
relevant for the topic. The reason for not using the information provided by  Medline,
these studies in the actual economic evaluation should be explained. The  Embase,
literature review forms the basis of the economic evaluation. As for  The Cochrane Controlled Trials Register,
economic models, transparency and reproducibility is the key to a good
 The Cochrane Database of Systematic reviews and
literature review.
The best available up-to-date evidence can be found following the  The NHS CRD Database of Abstracts and Reviews of Effectiveness
methodology of systematic literature reviews. Systematic reviews of clinical (DARE).
and economic literature should be carried out following the guidelines of the Next to Medline and Embase, the economic search strategy should also
Centre for Reviews and Dissemination search the public CRD’s NHS Economic Evaluation Database (NHS EED).
(http://www.york.ac.uk/inst/crd/finding_studies_systematic_reviews.htm for The KCE process note “Search for evidence and critical appraisal: Health
clinical reviews, http://www.york.ac.uk/inst/crd/econ.htm for economic Technology Assessment (HTA)” provides further details and methodological
reviewsb). A literature review is an iterative process. A first search might advices on how to perform literature searches for HTAs.c
reveal the existence of a high-quality systematic review. In that case, the
literature review can be limited to an update of the existing review with more
recent primary studies.

b The search algorithms proposed in the CRD guidelines may have to be c Formulation of the review questions, study location, study selection, critical
updated to current MeSH terms. appraisal and data extraction, https://kce.fgov.be/content/kce-processes
16 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

The methodology used for the literature search should be clear and The statements and data presented in the literature review should always
reproducible. Selection of articles is part of the review process. The selection be accompanied by the references from which they are derived. The
criteria could relate to the years of publication, population size, publication external validity of study results included in the review, and their applicability
type, language, indication, etc. The main requirement is transparency in to Belgium, should be assessed, especially if these results are afterwards
selection criteria and argumentation why certain selection criteria were used in the economic evaluation. In this context it is worth noting that clinical
applied. Therefore, exclusion of articles is not problematic per se as long as practice guidelines can be but are not necessarily evidence-based. Issues
the arguments for exclusion are well-justified. affecting external validity of RCTs are discussed by Rothwell.8 They relate
Not being from Belgian origin is not an appropriate exclusion criterion for to the setting of the trial, selection of patients, characteristics of selected
studies. Also clinical and economic studies from other countries may provide patients (e.g. baseline risk), differences between the trial protocol and
useful and relevant information for a Belgian economic evaluation. For routine practice, outcome measures, follow-up, and adverse effects of
instance, the design and assumptions of earlier published economic treatment. A full list of the issues highlighted by Rothwell8 is found in
evaluations on the same intervention may provide a good cross-check of the Appendix 4.1. The analysis of the external validity and hence the relevance
assumptions and design of the submitted economic evaluation. This does of study results for Belgium is mainly descriptive in nature.
not mean that the same design and assumptions must be used, but they The literature review will be critically appraised by the Expert Committee.
allow argumentation for or against a specific approach.
5.2 Guideline 2: Perspective of the evaluation
The search algorithm should be presented, including search terms used for
each database. A flow diagram, specifying the yield and exclusions (with the
reason for exclusion) should be presented. Both clinical and economic In economic evaluations submitted in the context of a reimbursement
literature should be critically appraised.. request, the reference case analysis should only include direct health
Data extraction sheets and/or checklists (e.g. Appendix 2) can be useful care costs from the perspective of the health care payers. This
tools to collect the relevant data from the selected literature. The submission includes payments out of the federal government’s and the
dossier must contain a synthesis of the relevant input variables, including communities’ health care budget as well as patients’ co-payments.
the uncertainty measures (e.g. 95% CI, p-value). should be provided for all Health outcomes should be measured in patients but health state
the studies retained for the synthesis. Appendix 3 provides examples for values should come from the general public.
data extraction sheets for economic studies.
In the literature, it is often recommended to use the societal viewpoint for the
If modelling is used for the primary economic evaluation, all (clinical) studies
economic analysis, i.e. costs and outcomes for society as a whole should
that served as a basis for the modelling input parameters’ valuation should
be valued. This would include costs borne outside the health care sector,
be described in detail (including methodology used, assumptions, results).
such as productivity losses and travel expenses, and stricto sensu also
Relevance and appropriateness should be discussed in detail. The use of
outcomes for patients’ family.
unpublished material in an economic evaluation is allowed but then the
material should be sufficiently described to allow evaluation of its The decision maker, however, is usually more interested in the costs of a
appropriateness. treatment from the point of view of the health care sector. This includes costs
paid out of the health care budget (be it federal or from the communities)
and patients’ out-of-pocket expenses for health care.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 17

The aim of the health care decision maker is to maximize health within the In conclusion, the base-case analysis should be performed from the
constraints of limited resources and taking into account additional decision perspective of the health care payers (federal government + communities +
elements. In the allocation of scarce health care resources, it is important to patients). Analyses from a broader perspective are allowed but should be
know how these resources can be allocated in the best possible way; in clearly distinguished from the reference case.
principle across disease areas.
5.3 Guideline 3: Target population
To be of interest to the decision maker the calculation of the incremental
cost-effectiveness ratio should be based on the aggregated costs of the
health care payers, i.e. the patients, the federal government and the The patient population to which the economic evaluation applies
communities. An incremental cost-effectiveness ratio for either the should be consistent with the patient population defined in the clinical
government or the patient only does not make much sense as its value will part of the reimbursement request submission.
depend basically on the level of reimbursement of the product. Therefore, If the intervention’s effectiveness and/or costs differ between
the cost-effectiveness ratio should be based on the aggregated costs of all subgroups, separate subgroup analyses should be performed. A
health care payers. scientific justification for subgroup analysis should always be
For the health care policy makers’ information, it is nevertheless useful to provided.
report costs for the different categories of health care payers also in
disaggregated form, i.e. as the costs borne by the different categories of Post-hoc subgroup analyses are only allowed if the safety,
payers (cfr. guideline 7). effectiveness or costs between the subgroups are proven to be
different based on appropriate statistical analyses or if the baseline
Outcomes included in the analysis should be relevant for the patient risk for events differs between subgroups of the target population.
population involved in the treatment and valued from a societal perspective. Relative effectiveness should be assumed equal across subgroups in
If health-related quality of life is used as an outcome measure, health states the latter case. The validity of this assumption should be checked.
should be described by patients but values of health-related quality of life
should be values allocated to these states by the general public. Epidemiological data for Belgium should be presented if available for
This does not mean that broader consequences of a treatment cannot or will both the total target population and the relevant subgroups.
not be taken into account in resource allocation decisions. Decisions are not
necessarily made on the basis of cost-effectiveness information alone. Other
considerations, such as substantial reductions in the absence from work, The economic evaluation should follow the clinical evidence. The target
may be important factors in determining the value of a therapy.7,9 In addition, population described in the economic file should be consistent with the
the decision maker will take other consequences into account: medical and target population identified for routine use of the intervention in the clinical
therapeutic need, equity considerations, organizational issues, population data or information provided in the reimbursement request dossier (hereafter
characteristics, budget impact, etc. If these consequences are expected to called the ‘clinical file’). The definition of the target population for routine use
be important for a specific treatment, additional analyses can be presented. of an intervention is not necessarily identical to the population included in
However, these complementary analyses cannot replace the reference case clinical trials, where selection criteria are often very strict and not applicable
analysis. to routine care (e.g. Phase I, II or III studies). This would imply that the actual
target population is larger than the population included in the trials. The
opposite is also possible, i.e. that the target population is actually smaller,
for instance if a treatment is only cost-effective in a subgroup of the patients
18 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

studied in the trial. Sometimes the implications of an intervention on the subgroups should be detailed enough to allow the evaluator to assess the
costs or effects of treatment are different between subgroups. These appropriateness and relevance of the subgroups.
subgroups may already be described and analyzed in the clinical file. In this Subgroups should be clearly defined groups that can be identified in real-
case, subgroup analyses are also indispensable in the economic evaluation. life. For the policy maker it is of utmost importance that the sub-groups are
While for the clinical file subgroup analyses are only allowed under specific identifiable based on objective criteria. If not, it is impossible to apply specific
conditions, there is more room for subgroup analyses in economic reimbursement rules to the subgroups. The application of objective selection
evaluations. The evaluations must consider the cost-effectiveness of an criteria might be difficult. In that case, this should be explicitly discussed in
intervention for different indications and characteristics of the affected the economic evaluation and budget impact analysis.
population. Even if subgroups were not analyzed in the clinical study, Outliers can therefore never be considered as a separate subgroup, as they
subgroup analyses might still be useful for the economic evaluation, e.g. if are not a clearly identifiable homogenous group of patients with specific
there are variables affecting cost-effectiveness which are different from the characteristics. Separate analyses on outliers are not acceptable.
variables affecting clinical efficacy. Such analyses should always properly
Two reasons for (post-hoc) subgroup analysis are acceptable:
be referred to as post-hoc subgroup analyses. Post-hoc subgroup analyses
are often explorative. 1. Differences in safety, effects or costs between clearly defined
subgroups, as demonstrated by appropriate statistical analyses.
This does not mean, however, that choices should not be justified. Ad hoc
data mining in search of subgroups with significant results is not acceptable. 2. Heterogeneity in baseline risk of clearly defined subgroups.
There should be a clear rationale behind the choice of subgroups and an Coincidentally observed differences in relative effectiveness between
answer should be provided to the question of why a differential effect is subgroups are not sufficient for post-hoc subgroup analyses, because it is
expected. impossible to say whether the differences observed are true differences if
Post-hoc subgroup analyses always go with certain assumptions, e.g. about the study was not designed to observe such subgroup differences in
the treatment effect in the different subgroups. It is essential to use an effectiveness. Therefore, the relative effectiveness of an intervention should
assumption of constant relative treatment effect if the subgroups were not always be assumed equal in post-hoc subgroup analyses. The validity of
defined a priori and included as such in the clinical trial design. This means this assumption should be checked by considering, for instance, the face
that the relative effectiveness in the different post-hoc subgroups is validity of the outcomes of the model when applying this assumption. For
assumed to be equal to the relative effectiveness found in the complete example, a relative mortality risk reduction in a patient group of a specific
sample of the clinical trial(s), while the baseline risks between the subgroups age with severe co-morbidities cannot lead to a lower overall mortality risk
are different.d,5 Other assumptions cannot be justified in the absence of than in the healthy population of that age.
clinical effectiveness data for the different subgroups. Epidemiological data for Belgium for the target population or relevant sub-
Again, appropriate justification should be provided for the subgroup populations is part of the clinical submission. If epidemiological data are not
analyses and uncertainty associated with assumptions related to the available for Belgium, data from other European countries should be
analyses assessed. Specification of patient characteristics in the different presented and be well described. In this case, the relevance of these data
for Belgium should be assessed.

d For example, if a clinical trial finds a 10% increase in survival due to treatment, assume a 10% increase in survival in all age groups. The relative effect is
and if it is expected that the cost-effectiveness of the treatment will differ hence the same in the different subgroups, but the absolute effect will differ,
according to the age of patients, the cost-effectiveness analysis should due to the higher baseline survival in younger patients.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 19

5.4 Guideline 4: Comparators In order to avoid the fallacious improvement of the cost-effectiveness ratio
of an intervention by comparing it to a non cost-effective intervention,
comparisons and thus calculations of incremental cost-effectiveness ratios
For the identification of the appropriate comparator, the efficiency should be performed on the efficiency frontier.10 The efficiency frontier is the
frontier should be constructed. This involves the identification of all line on the cost-effectiveness plane connecting the non-dominated
relevant treatments for the targeted indication and population, the treatment alternatives (see Appendix 5). It can be constructed as follows:
removal of dominated or extendedly dominated interventions from the
list of relevant comparators, and the calculation of the ICERs of all 1. Make a comprehensive list of all possible therapeutic strategies for the
interventions compared to the next best alternative. target indication and population.
2. Exclude interventions that are dominated by other interventions with
The comparators can be medical and/or non-medical treatments “Off- lower costs and greater therapeutic benefits.
label” used products or services should not be used as a comparator
in the reference case analysis, unless there is evidence about their 3. Exclude extendedly dominated alternatives, which means that linear
clinical safety and efficacy. combinations of other strategies can produce the same (or greater)
benefit at lower (or the same) cost.
The choice of the comparator(s) should always be justified. 4. For the remaining alternatives, calculate the cost effectiveness by
Indirect comparisons are only allowed under specific conditions. The comparing each strategy with the previous less costly and less effective
choice for an indirect rather than a direct head-to-head comparison intervention.11,12
between the study treatment and the comparator should be explained, It is possible that the current treatment approach is not cost-effective itself,
together with the limitations of the indirect comparison. e.g. because it received a positive reimbursement decision in the past for
reasons that now no longer apply (e.g. because at that time no other
treatment was available for that indication or economic considerations were
The intervention should be compared with an alternative intervention with not taken into account). Striving to make calculations on the efficiency
proven efficacy (in RCTs) that is considered a clinically recommended frontier avoids situations in which interventions are made cost-effective by
intervention for the target indication. It can be a medical or non-medical comparing them with non-cost-effective alternatives, further stimulating non-
treatment, best supportive care, watchful waiting or doing nothing. Note that efficient use of limited resources. The appropriate economic approach
the “doing nothing” approach is usually not associated with zero costs and compares every alternative with the previous most cost-effective alternative.
effects. The evidence should not be restricted to interventions supported by In practice, this means that next to the current treatment situation, alternative
the manufacturer, but should also include alternative treatment options more cost-effective comparators should be included in the analysis. This can
studied (in RCTs) by e.g. independent governments or academics. be a generic product, lifestyle adoptions (e.g. smoking cessation), or a new
evidence-based intervention that has shown to be more cost-effective than
the current treatment practice. Note that clinical practice guidelines usually
do not consider cost-effectiveness. This implies that the recommended
intervention in the guidelines is not necessarily situated on the efficiency
frontier. However, guidelines may point to the different interventions that
should be considered for constructing the efficiency frontier.
The choice of the comparator should always be justified and supported by
clear arguments. Consistency between the clinical and the economic
20 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

submission should be pursued. Off-label used pharmaceutical products can 5.5 Guideline 5: Analytic technique
be used as valid comparators in a pharmacoeconomic evaluation if evidence
is available about the clincial safety and efficacy of the off-label use, e.g.
from government sponsored trials. The value of these products should then Cost-effectiveness analysis should be used if improving life
be described in the literature review. The same applies to non- expectancy is the main objective of the treatment and also the most
pharmaceutical interventions used beyond their described indication. important outcome from the patient’s point of view. Outcomes of cost-
effectiveness analyses should be expressed in euro per life-year
At first sight, it may seem strange that an evidence-based off label use or an gained.
intervention that is currently not applied in everyday practice is included in
the analysis as a comparator. Nevertheless, this may be very relevant in Cost-utility analysis should be used if the treatment has an impact on
order to stimulate efficient use of resources. If e.g. more cost-effective health-related quality of life that is significant to the patient or if there
alternatives are currently not registered or reimbursed, than pointing this out are multiple patient-relevant clinical outcome parameters expressed in
could have an impact on regulations, reimbursement decisions, or different units that cannot be translated into one common unit in a
supporting/requiring further research. valid way. Outcomes of cost-utility analyses should be expressed in
In some cases, the choice of the comparator will be difficult due to, for euro per quality-adjusted life year gained.
instance, changes in prescription behaviour and therapeutic insights over Given the continuing controversy over the appropriate methodology
time. The comparator defined at the time of the clinical trials may no longer for cost-benefit analyses, cost-benefit analyses are not accepted as a
be the relevant comparator at the time of the economic evaluation. In this reference case for economic submissions.
case, indirect comparisons and/or modelling may be required. Indirect
comparisons are second best solutions and are only accepted if no single Results should be expressed as incremental costs, incremental effects
trial of appropriate quality or relevance to the Belgian target population has and incremental cost-effectiveness or cost-utility ratios with their
been performed and under specific conditions regarding the analyses. associated uncertainty. If a cost-utility ratio is presented as the result
Appropriate statistical techniques must be used for indirect comparisons (i.e. of a reference case analysis, the corresponding cost per life-year
adjusted indirect comparisons13,14,15). Useful reports about indirect gained should also be presented.
comparisons is available on http://www.hta.ac.uk/fullmono/mon926.pdf16
and on https://www.iqwig.de/download/General_Methods_4-0.pdf.38 The report should specify whether a cost-effectiveness or cost-utility
Comparators for which no direct or indirect evidence is available cannot be analysis is used. Justification for the choice of analytic technique should be
included in the economic evaluation. All other interventions for which provided.
evidence is available can be included in the analysis. Evidence about the Cost consequence analyses, i.e. descriptions of costs and consequences
relative effectiveness of the two treatments is indispensable for an economic without calculation of an incremental cost-effectiveness ratio, are insufficient
evaluation. Without such evidence, an economic evaluation will not be for an economic evaluation but may be considered as a logical first step
informative for the health care decision maker. towards a formal economic evaluation. A table classifying the different types
of economic studies is provided in Appendix 1. Separate reporting of
incremental costs and incremental effects (both life-years gained and
QALYs gained), besides the incremental cost-effectiveness ratio, is always
recommended.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 21

5.5.1 Cost-effectiveness analysis 5.5.2 Cost-utility analysis

In cost-effectiveness analyses the outcome should be expressed in terms of In these guidelines, the term cost-utility analysis is used for economic
life years gained. The choice of the outcome measure should be consistent evaluations that include health-related quality of life in the assessment of
with the objectives of the treatment and the impact on patient-relevant treatment outcome.e A cost-utility analysis should always complement a
outcomes. cost-effectiveness analysis if:
The result of a cost-effectiveness analysis is expressed as an incremental  the treatment has an impact on health-related quality of life that is
cost-effectiveness ratio (ICER). The ICER reflects the additional significant to patients or,
(incremental) cost per additional unit of outcome achieved. If the  the treatment is associated with multiple clinical outcomes that are
effectiveness of a drug is better and the costs lower than the comparators’, expressed in different units (e.g. side effects versus survival).
or vice versa, the ICER is negative. In that case, the intervention is either
dominant or dominated and the ICER should not be presented. Independent Cost-utility is not relevant in all disease areas or treatment situations. For
of the sign of the ICER, incremental costs and incremental effects should instance, for very serious infections, leading to a high short term mortality
always be presented in disaggregated form, with their respective credibility rate but little quality-of-life consequences in survivors, it is more important to
intervals (cfr. guideline 11). look at survival than to health-related quality of life and hence cost-
effectiveness analysis may be more appropriate.
If different patient-relevant clinical outcomes are expressed in different units
(e.g. life years gained and complications avoided), cost-effectiveness While it is easy to find at least one argument to use a cost-utility analysis,
analysis is less appropriate. For example, a cost-effectiveness analysis of a the outcome measures used in cost-utility analyses are much more subject
drug treatment that prolongs life expectancy significantly albeit at a high cost to variation according to the measurement methods than the outcome
in terms of co-morbidity should present its results in terms of quality-adjusted measures of cost-effectiveness analyses. As a consequence, the
life years that includes the impact of the drug on symptoms related to the comparability of different cost-utility analyses is limited. Validity of the utility
treatment. Although this case for cost-utility analysis is strong, the cost per values cannot be assessed because there is no golden standard for
life year gained should nevertheless be presented to provide the most measuring utility. In order to stimulate the use of generic utility instruments
complete information to the decision maker. and to promote consistency, the Belgian guidelines explicitly encourage the
use of the EQ-5D instrument. If researchers feel that an intervention will
have an impact on a patient’s quality of life, including this instrument in the
study protocol should be considered. This does not replace the use of
disease-specific instruments, but rather complements them. If the EQ-5D
instrument is not considered suitable, then the use of another generic utility
instrument or direct measurement of utilities by means of time-trade-off
(TTO) or standard gamble (SG) can be considered. This should then also
be justified.

e Note that health-related quality-of-life values do not necessarily represent these guidelines, however, we use the term “cost-utility analysis” for all
utility values. Measurement of utilities is subject to specific requirements. analyses that include quality-of-life considerations in their outcome measure,
Therefore, more strict definitions of cost-utility analysis could be used. In to contrast them with cost-effectiveness analyses where this element is not
taken into account.
22 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

To increase the usefulness of a cost-utility analysis for health care decisions, 5.5.4 Cost-benefit analysis
the applicant must provide sufficient detail about the methods used for Unlike cost-effectiveness and cost-utility analyses, cost-benefit analyses
valuing utilities. express outcomes in monetary terms. The monetary valuation of clinical and
5.5.3 Cost-minimisation analysis non-clinical outcomes has been debated since long. As a consequence,
cost-benefit analyses have not been used as frequently as cost-
Cost-minimisation analyses are used if the effects of two treatments are
effectiveness or cost-utility analyses. Given the methodological difficulties
identical. Hence, cost-minimisation analysis can only be justified by proof of
and controversies associated with this technique, cost-benefit analysis is not
equal outcome.
acceptable as a stand-alone reference case analysis, but may be presented
Pharmaceutical products for which a pharmacoeconomic evaluation is as an additional analysis to cost-effectiveness analysis or cost-utility
needed have, by definition, claimed an added therapeutic value (as defined analysis to illustrate societal benefits accruing from non-health impacts.
by the aggregate value of the 5 items mentioned in the background section).
Nevertheless, due to the multiple outcomes considered in the definition of 5.6 Guideline 6: Study design
“therapeutic value”, the outcome value in terms of life years gained (LYG) or
QALYs gained can be identical for two interventions compared in an Whenever possible, health economic evaluations should always be
economic evaluation, while other elements of the therapeutic value (e.g. based on data from randomized controlled trials comparing the study
applicability or user-friendliness), which are not captured in the QALY or intervention and a relevant comparator. Economic evaluations based
LYG-estimate, are still different. In that case, cost-minimisation analysis is on active control studies are preferred.
recommended and additional reflections on the impact of the treatment on
the other non-health outcome parameters should be provided. If modelling is needed because clinical trials provide insufficient
In practice, it is often impossible to know a priori that cost-minimisation information for the economic evaluation, the number of assumptions
analysis is appropriate. The analysis will therefore usually be preceded by a not based on clinical evidence should be reduced to a minimum and
cost-effectiveness or cost-utility approach, during which it becomes clear be fully justified.
that health outcomes are identical. In this sense, a cost-minimisation
analysis can be interpreted as a special case of cost-effectiveness or cost- Cost-effectiveness or cost-utility analyses can be performed alongside a
utility analysis with equal outcomes. clinical trial (e.g. piggy-back trial) or an observational study or can be based
on a model. Clinical studies, as defined in the Law of 7 May 2004 regarding
experiments on human beings, where a product under study is compared to
its relevant comparator, are preferred as they offer a direct comparison
between products. Each design has its peculiarities and specific caveats.
Analyses should be explicit about the limitations of the design and should
explain the methods used to overcome these.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 23

5.6.1 Trial-based economic evaluations Piggy-back studies are useful if the weaknesses are made explicit and
There are basically two types of trial-based economic evaluations: piggy- whenever possible tackled in advance:
back studies, i.e. an evaluation alongside a randomised controlled trial  either the economic evaluation is planned a priori, in which case the
(RCT), and economic evaluations alongside non-interventional trials. economic evaluation should be included in the study protocol and
The weaknesses of piggy-back studies are directly related to the purposes appropriate measures be taken to tackle the potential weaknesses of
of the RCT. RCTs are not set up for economic evaluation but rather for economic evaluations alongside RCTs,
evaluating treatment efficacy. For economic evaluations, information is  or the economic evaluation is performed retrospectively, using data
needed on the effectiveness in routine practice. As a consequence the gathered in the RCT, in which case appropriate measures to tackle the
information provided in RCTs is often insufficient for the economic weaknesses should be taken before the actual economic analysis is
evaluation. Some of the weaknesses of RCT for the purpose of economic performed. The analyst should evaluate the appropriateness of the
evaluations are: sample size for measuring differences in costs and outcomes, develop
 a potentially inappropriate comparator, methods to deal with protocol driven costs, assess the availability of an
appropriate comparator and a relevant outcome measure for the
 an inadequate sample size, economic evaluation.
 a limited time horizon, The Drug Reimbursement Committee developed guidelines for non-
 the occurrence of protocol-driven costs or outcomes, interventional studies, defined as studies where procedures are not
 inappropriate outcome measures and protocol-driven but rather by usual care. Such trials are considered
complementary to randomised controlled trials, and especially useful to
 patient selection. demonstrate the experience with the product in routine care (effectiveness
Note that these weaknesses do not apply to RCTs only. Other study designs rather than efficacy) as well as for the post-registration evaluation of the real
often have worse limitations (e.g. biases and confounding factors, lack of cost-effectiveness of the product after 1.5 to 3 years. At the time of the initial
comparator group). reimbursement request, non-interventional studies will usually not be
When using results from RCTs performed in other countries, the treatment available yet, at least not for Belgium. Therefore, they will be more important
protocol may be different from the protocol that would be followed in for the revision file submitted after 1.5 to 3 years of use of the product in
Belgium. Some weaknesses, such as the problem of protocol driven costs, routine care. However, it should be reminded that if there is no comparator
can be overcome with adequate methodology but others will require some group in a non-interventional study, the relative effectiveness of an
extent of modelling. intervention cannot be assessed. Non-interventional studies avoid some of
Besides weaknesses, piggy-back studies may also have important the weaknesses of RCTs but may nevertheless be insufficient to
strengths, which should be exploited if certain conditions are fulfilled. An demonstrate long-term (cost-)effectiveness of a product, e.g. if there is no
RCT design is the strongest design to demonstrate differences in clinical comparator group. In designing a non-interventional study, it is important to
efficacy, which can be causally linked to the treatment. Before include the specific features for the economic evaluation in the protocol.
reimbursement of a product it is often the only information available on In some cases, additional data may be necessary to support better decision
efficacy. making. For example, temporary reimbursement may be allowed while
gathering better data. In such cases, the remaining research questions
should be clear and the research design should be appropriate to tackle
these questions.
24 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

E.g. registries, without comparison group, are unable to provide reliable 5.7 Guideline 7: Calculation of costs
information on efficacy/effectiveness. On the other hand, they may be useful
to provide e.g. safety or incidence/prevalence information.
The identification, measurement and valuation of costs should be
For economic evaluations alongside RCTs or non-interventional trials, consistent with the perspective of the Belgian health care payers. The
original data should be provided to the Expert Committee evaluating the reference case should only include direct health care costs. Direct
reimbursement file upon request. costs outside the health care sector, productivity costs and health care
5.6.2 Modelling costs associated with unrelated diseases should not be included in the
reference case, but may be reported as a separate analysis. In this
Even if a trial-based economic evaluation exists, some modelling is likely to
case, the impact of the intervention on these elements should be
be needed (e.g. to extend the time horizon to longer time spans or to model
demonstrated by means of hard data. Validated sources should be
comparators which have become more relevant in practice since completion
used for the unit costs.
of the trial). Very often, already in the analysis of a piggy-back study, certain
assumptions will be made (e.g. assuming that the study population and Where products under the reference pricing system or generic
observed resource use are representative for Belgium, while only a small pharmaceutical products exist, the lowest priced product should be
portion of the study was set in Belgium), which turns it de facto into a model. used in the economic evaluation, even if the cheapest products are not
However, modelling should never be used as a substitute for a bad RCT. frequently used in Belgium.
Health economic models allow the analyst to combine information from a For co-payments, the general rule is to use the official co-payments for
variety of sources and to link these data to outcomes of interest to decision patients without preferential reimbursement. Deviations from this rule
makers. Computer based models allow the simulation of various policies. should be justified.
They are therefore distinct from statistical models such as regressions or
meta-analyses.
The perspective for the cost calculation is that of the health care payers
Models are used for different reasons: extension of time horizons, (government and patient). Health care costs borne by the government
extrapolation of intermediate outcome parameters to final outcome (RIZIV–INAMI, FOD–SPF, communities) and costs borne by the patients, as
parameters, consideration of externalities associated with a treatment, far as available, should be reported both separately and aggregated.
translation of foreign data to the Belgian context, pooling data from multiple Valuation of resource use in monetary units must be consistent with the
trials, etc. The major weakness of models is that data from different sources perspective of the analysis.
are combined and assumptions have to be made (e.g. about the
comparability of the data derived from different sources, resource use in
Belgium, etc). The arguments to use a modelling approach should be set
out clearly and sources for hypotheses should be presented.
A separate guideline is devoted to modelling (see guideline 10).
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 25

5.7.1 Cost categories 5.7.2 Measurement of resource use

Table 3 specifies the cost categories that should be included in or excluded Measurement of resource use should be done by means of observations or
from the cost analysis in the reference case. derived from literature. Observations offer the best guarantee for
appropriateness of the resource use estimates for the Belgian context.
Table 3 – Included and excluded costs in the reference case analysis Different sources can be used to obtain observational data: clinical trials,
prospective observational studies, databases and patient charts.
Health care costs Non-health care costs
Use of expert panels for resource use measurement is subject to specific
Direct costs Included. Not included. conditions (Appendix 4.3). Expert panels are preferably only used as a
complementary source of information rather than as the sole source of
e.g. health services, e.g. travel expenses to and information on resource use. If they are used, it is essential to provide a
medications, from hospital, informal desciption of the way experts are selected.
hospitalisations… care, invalidity/incapacity
allowances… Transparency in the methods used to obtain resource use estimates from
experts is crucial. If questionnaires are used, these should be provided in
Indirect Not included. Not included. appendix as well as descriptive statistics and in case of small samples (<10
costs e.g. health care costs in life e.g. productivity losses experts) individual responses. Names and affiliations of experts should be
years gained (unrelated disclosed.
health care costs) If derived from literature or studies from other countries, resource use
estimates should be validated for Belgium. This validation process must be
This report uses this classification although other cost classifications exist.
described in the submitted file.
The reference case should only include direct health care costs. These For the measurement of mean length of hospital stay per All Patient Refined
encompass costs directly related to the treatment of the disease as well as Diagnosis Related Group (APR-DRG), data can be found on the web-site of
direct health care costs related to the disease in life years gained. Indirect the “Cellule Technique pour la gestion des données RCM-RFM–Technische
health care costs – these are health care costs in future life years associated cel voor het beheer van de MKG-MFG data”f, under the heading “Feedback
with unrelated diseases – should not be included. Costs borne outside the Financier par pathologie–Financiële Feedback per pathologie”. The
health care sector should not be included in the reference case analysis. database also provides distributional parameters for each APR-DRG. This
If productivity losses, non-health care costs and/or unrelated health care information should be used in the sensitivity analysis of the economic
costs are deemed important for a specific treatment, they may be presented evaluation. Other databases can be used, provided that they are compliant
in a separate analysis. with legal requirements about privacy and provided that the data in the
database are validated, for instance against the data of the Cellule
Technique–Technische Cel. An overview of health care related databases
in Belgium can be found in Appendix 6.

f https://tct.fgov.be/etct/index.html
26 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

Each database has its weaknesses, such as for instance the cross-sectional interventions which are relevant in the economic evaluation (e.g. follow-up
nature of the data, discrepancies in the computation of length of stays, treatments) and for which no official prices are published.
imperfect registration, etc. These weaknesses can generally not be The values should reflect Belgian prices/costs for each resource input rather
remedied without some assumptions. Therefore it is recommended to than foreign prices converted to euros. Valuation of resource use by means
discuss these weaknesses and their potential impact on the cost estimates of simple currency conversion of values found in literature or in studies from
in the text rather than trying to solve them by means of ad hoc assumptions. other countries is not acceptable.
For all analyses of data, methods to handle missing data should be All costs should be expressed in values for the current (or most recent) year,
described. For longitudinal RCT or observational studies in particular, e.g. by using current prices. If this is not possible and costs from past years
information should be provided on the proportion of missing cost data, the are used, these costs should be inflated using the appropriate Health Index
reasons for these missing data, and the methods used to handle them in the figures, if relevant. In some cases, indexation will not be relevant for
analysis. particular products or services. For example, the reimbursement basis of
5.7.3 Valuation of resource use pharmaceuticals is not necessarily indexed and might even decrease for
products entering the reference price system. Index figures can be obtained
5.7.3.1 General principles of cost estimation from the web-site of the ministry of Economic Affairsg.
The principle of the cost analysis is that costs are valued at opportunity Co-payments for the regularly insured should be used and not those for
costs. In practice, the opportunity costs will be approximated by market special categories of insured citizens, such as “patients with preferential
prices or some kind of mechanism used for the reimbursement of reimbursement” (Rechthebbende op verhoogde
procedures (e.g. the Belgian per diem hospitalization price). In the absence verzekeringstegemoetkoming – Bénéficiaire de l’intervention majorée,
of a better alternative and for reasons of uniformity between analyses, it is RVV–BIM), unless there are good reasons to make the distinction. For
suggested to use these proxies in the reference case, knowing that these instance, if an intervention is targeted at specific population groups that
proxies do not always reflect real opportunity costs. Alternative cost typically belong to one of these special categories, the distinction may be
estimates, e.g. based on micro-costing approaches, can be presented in made between the groups in the cost analysis. For the calculation of the
alternative scenarios, supported with arguments of why the analyst thinks ICER, which is done from the perspective of the health care payers, the
these alternative cost estimates are more appropriate. distinction will not have an impact as the total costs, born by the patients and
the government, are the same for the different groups. Hence, the distinction
If the health intervention to be valued is the intervention under investigation
is only relevant for the disaggregated reporting of costs.
(i.e. for which a reimbursement request is introduced), there will be no official
market price publicly available yet. A plausible price should then be
estimated using alternative sources of information (contact with
manufacturer, ad hoc study, literature, micro-costing…). Details on how the
price was estimated should be clearly reported. This also applies to

http://economie.fgov.be/fr/statistiques/chiffres/economie/prix_consommation
/indice_sante/
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 27

5.7.3.2 Valuation of health care services generic products would require lots of effort for probably limited benefit. If
follow-up treatments are simulated based on hypotheses, costs of generic
Unit prices/costs for ambulatory and hospital health care services pharmaceuticals should be used. Hypothetically constructed follow-up
(honorarium fees) can be found in the Belgian reimbursement scheme scenarios are simplified versions of real-life follow-up treatments. It is
(Nomenclatuur–Nomenclature), which is publicly available on the RIZIV– therefore feasible to use generic cost in these scenarios.
INAMI website. h
Standard fees should be used for regularly insured patients. No account 5.7.3.4 Valuation of devices
should be taken of additional charges (the so-called “supplements”) for The list and prices (reimbursement basis and patients’ share) of reimbursed
specific patients (e.g. in a private hospital room). implants and invasive medical devices (per category) is to be found on the
5.7.3.3 Valuation of drugs RIZIV–INAMI website (Articles 28, 35 and 35bis of the RIZIV–INAMI
nomenclature).k They also can be found in the Belgian reimbursement
Unit prices (reimbursement basis and the patients’ share) for reimbursed scheme (Nomenclatuur–Nomenclature).l
drugs are publicly available at the RIZIV–INAMI website. i Unit prices for The list and prices of implants and invasive medical devices (per producer
reimbursed and non-reimbursed (e.g. over the counter) drugs are available and per product) listed on the so-called “limitative lists”, also published on
on the BCFI–CBIP website. j the RIZIV–INAMI website.m
Where products under the reference pricing system or generic
pharmaceutical products exist, the lowest priced product should be used in 5.7.3.5 Valuation of per diem hospitalization prices
the economic evaluation, even if it is not frequently used in Belgium. The Belgian per diem hospitalization prices (in euro) are available on the RIZIV–
rationale of this approach is that the limited use of the lowest priced product INAMI website.n The per diem prices are reported per hospital and per type
is a policy issue that is outside the scope of the economic evaluation. The of hospital stay (acute, burns, geriatrics, palliative, psychiatric and
aim of the economic evaluation is to assess the ICER relative to the specialized stays). This break-up per type of hospital stay is the one used in
appropriate comparator. If the comparator encompasses two kinds of the Belgian hospital financing law. A description on how per diem hospital
products with a different price but equal outcomes, the least costly product prices are computed in Belgium is provided in Appendix 4.4.
should be used in the evaluation, as this product is more cost-effective than
its more expensive counterpart. For follow-up treatments, a distinction is
made according to the source of the data. If secondary cost data are used,
e.g. from the Intermutualistic Agency (IMA–AIM), real-life data could be
used, even though these real-life data might also contain the costs of brand
products where generic products could have been used. Although
theoretically possible, converting all costs of brand products to costs of

h m
http://www.riziv.be/care/fr/nomenclature/index.htm http://www.inami.fgov.be/care/fr/other/implants/ information-
i http://www.riziv.fgov.be/drug/fr/index.htm topic/listart35_35bis/index.htm (“listes par type d’implant – Lijst per type-
j implantaat”).
http://www.cbip.be/ n
k http://www.riziv.fgov.be/care/nl/hospitals/specific-information/prices-
https://www.riziv.fgov.be/insurer/fr/rate/index.htm day/index.htm
l http://www.riziv.be/care/fr/nomenclature/ index.htm
28 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

Based on this published list of per diem prices, a mathematical average

across hospitals could be computed to derive Belgian average per diem
prices per type of hospital stay. However this method does not account for
the volume effect of each hospital such that big hospitals with higher
hospitalization days and per diem prices would have the same weight as
smaller hospitals with lower per diem prices. The simple mathematical
average of the per diem prices should therefore not be used.
Rather the weighted average per diem prices that account for disparities in
the case-mix (different levels of activities) of the hospitals should be used.
Data over the number (volume) of stays per hospital are available (not
publicly) from the MFG–RFM database. Data for the years 2004 to 2010
were obtained and weighted average 100% per diem prices were computed
for those years. In an update, information up to 2013 was added.

Table 4 – Weighted average of the 100% per diem hospital prices per type of stay, Belgium (2004-2013)
Type 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013*

Acute €289 €288 €308 €322 €355 €376 €390 €415 €432 €445
Burns €1 061 €1 075 €1 135 €1 155 €1 208 €1 253 €1 296 €1 411 €1 584 €1 576
Geriatrics €174 €179 €179 €196 €213 €214 €215 €244 €258 €252
Palliative €402 €402 €418 €426 €446 €461 €470 €499 €533 €535
Psychiatric €178 €177 €186 €196 €210 €215 €222 €240 €261 €273
Rehabilitation €194 €194 €208 €220 €229 €235 €247 €261 €276 €284
More recent values will be published on the KCE website when data become available.
* 2013 are preliminary results

Standard weighted average per diem prices should be used and no account
should be taken of supplements related to extra-ordinary services, such as
private room. Lump sums for drugs, medical imaging and clinical biology
should be added to the per diem price (see the methods for valuation below).
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 29

5.7.3.6 Valuation of lump sums for drugs, medical imaging and For the pharmaceuticals integrated in the prospective budget, payments to
clinical biology in hospitalized patients the hospital are two-fold:
Hospital lump sum for drugs  Fee-for-service: the hospital retrospectively charges the sickness funds
25% of the reimbursement basis of each delivered drug.
Since 1 July 2006, a prospective budget for pharmaceuticals administered  Lump sum: the hospital receives prospective lump sum allocations per
to hospitalized patients in a general acute hospital was introduced by means inpatient admission, regardless of the magnitude (even absence) of the
of lump sum allocations.17 drugs administered.
This prospective system covers reimbursed pharmaceuticals (classes A, B, Unit prices for (reimbursed and over-the-counter) drugs (100% of the
C, Cs, Cx, Fa, Fb) and prophylactic (i.e. before surgery) antibiotics, with the reimbursement basis) administered during a hospitalization are publicly
exclusion of the drugs listed on a so-called “exclusion list”. available from the RIZIV–INAMI and BCFI–CBIP websites.q
Drugs excluded from the prospective system are: Lump sums per admission are hospital-specific and depend on the case mix
 drugs highly relevant to medical practice, in terms of therapeutic needs, (APR-DRG) of each hospital, taking into account the severity of illness. In
social values and innovative character, 2011 lump sums varied from €62.11 to €170.6 per admission. Lump sums
 and whose high costs can strongly delay their administration to a per admission are published on the RIZIV–INAMI website.r
hospitalized patient if it is included in the prospective budget. The contribution of the patient is limited to a lump sum payment of €0.62 per
Other specific products are excluded by law from the prospective budget inpatient day, which is charged irrespective of his actual consumption.
(e.g. orphan drugs, cytostatics, immunoglobulins and albumins, retroviral Drugs administered in the hospital are thus financed through the following
drugs, radioisotopes, etc.). The list of excluded pharmaceuticals is updated channels :
monthly and is publicly available.o The RIZIV–INAMI database on the
pharmaceutical specialties also mentions if the drug belongs to the
prospective budget or is excluded.p Excluded pharmaceuticals are
reimbursed by a retrospective fee-for-service system.
The prospective pharmaceutical budget is limited to inpatients (patients who
stay at least one night in hospital) in acute hospitals. It is not applicable to
psychiatric or chronic hospitals, nor for one-day hospitalisations.

o q
http://www.riziv.fgov.be/care/fr/hospitals/specific-information/ http://www.riziv.fgov.be/drug/fr/index.htm; http://www.cbip.be/
forfaitarisation/index.htm, (“liste des spécialités pharmaceutiques”–“lijst van r http://www.riziv.fgov.be/care/fr/hospitals/specific-information/
farmaceutische specialiteiten” and “explications”–“uitleg”) forfaitarisation/index.htm (‘Anonieme lijst van alle forfaits’–’Liste anonyme de
p http://www.riziv.fgov.be/drug/fr/index.htm l'ensemble des forfaits’)
30 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

Table 5 – Sources of financing – Hospital drugs for patients in general over the years. This factor will be checked regularly by KCE and
acute hospitals updated if necessary (publication on KCE’s web-site).
Sickness funds Ordinary 2. If IMA–AIM population data cannot be used, the number and type of
patient drugs administered per inpatient stay should be obtained from clinical
trials, observational studies or be simulated based on literature about
Drugs included in the Lump sum per admission patient health care pathways. Drugs included in the lump sum and
lump sum (varies per hospital) drugs on the exclusion list should be valued at 100% of their
reimbursement basis. The patient share of €0.62 per inpatient stay
25% of the (taking into account an average length of stay per pathology) should be
reimbursement basis as added, as well as expenses for class D drugs.
fee-for-service €0.62 per
Reimbursement as fee- inpatient day Table 6 – Total RIZIV–INAMI expenses on inpatient drugs (drugs
Drugs outside the
lump sum (exclusion for-service (according to included in the prospective budget only)
list) the reimbursement Year Lump sum Fee-for- Total Extrapolation
categories A, B, C, Cs, expenses* service expenses factor **
Cx, Fa, Fb). expenses
Drugs in - 100% out of
2006*** €81 965 692 €21 545 320 €103 511 011 4.80
reimbursement class D pocket
2007 €258 548 716 €77 780 783 €336 329 499 4.32
In order to value the full cost of the pharmaceuticals delivered during an
inpatient stay, taking into account the dual system of financing (25% fee-for- 2008 €263 207 655 €79 711 734 €342 919 389 4.30
service and lump sums), the following procedure is suggested: 2009 €246 271 579 €77 723 673 €323 995 253 4.17
1. If IMA–AIM population data can be used (or IMA–AIM data from the
“Permanent Sample” - Echantillon Permanent–Permanente Steekproef 2010 €230 943 715 €72 597 774 €303 541 489 4.18
- which is a representative subset of the IMA–AIM population data), the 2011 €217 654 214 €70 708 529 €288 362 743 4.08
drugs reimbursed as fee-for-service (at 25% of their reimbursement
basis) should be identified in the IMA–AIM database and multiplied by * Lump sums per admission; ** Extrapolation factors from fee-for-service expenses
to total expenses (= Total expenses / Fee-for-service expenses); *** Six-months
4. As a consequence, the lump sum per admission should be identified period
and removed from the computations. The patients’ share and the drugs
on the exclusion list are also recorded in the IMA–AIM database. Their
total costs should further be added. The multiplication factor was
obtained by computing the yearly (2000-2011) RIZIV–INAMI expenses
(in lump sums, fee-for-service and in total) for inpatient drugs included
in the lump sum based on full RIZIV–INAMI accountancy records (Doc
N, “financial year–boekjaar–année comptable”), see Table 6. This
multiplication factor clearly demonstrates a trend towards the value 4
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 31

Hospital lump sum for laboratory testing Table 7 – Sources of financing – Hospital laboratory testing for patients
in general hospitals (2012 values)
Laboratory testing for patients hospitalized in a general hospital is financed
through a mixed system of fee-for service and (for the greater part) lump Sickness funds Ordinary
sum payments.17 patient
 Fee-for-service: the hospital retrospectively charges the sickness funds Laboratory tests Fee-for-service: 25% of the -
25% of the honorarium fees of each test performed. honorarium fee
 Lump sum 1: the hospital receives prospective lump sum allocations Prospective lump sum per inpatient
per inpatient day, inclusive the first day. day (varies per hospital)
 Lump sum 2: the hospital receives lump sum allocations per inpatient “Remuneration €23.56: basis lump sum per €7.44 per
admission. for the admission admission
Fee-for-service charges per laboratory test (25% and 100% of the biologists” €46.81: lump sum per admission 24h
honorarium fee) and lump sums per admission can be found on the RIZIV– service + 2 biologists
INAMI website.s €66.19: lump sum per admission 24h
The lump sums per inpatient day are hospital-specific and depend on the service + 3 biologists
case mix of each hospital (APR-DRG). In November 2011 they varied from
€0.27 to €37.94. Lump sums per day for each hospital can be found on the In order to value the full cost of the laboratory tests performed during an
RIZIV–INAMI website.t Lump sum per inpatient day are charged irrespective inpatient stay, taking into account the dual system of financing (25% fee-for-
of the actual number of performed tests (and even for days without tests service and lump sums), the following procedure is suggested:
performed).
1. If IMA–AIM population data can be used (or IMA–AIM data from the
Basic lump sums per admission are fixed at €31 (March 2012 values). If the “Permanent Sample” - Echantillon Permanent–Permanente Steekproef
laboratory works 24 hours a day and consists of 2 biologists full time, the - which is a representative subset of the IMA–AIM population data), the
lump sum reaches €54.25, and €73.63 with 3 full time biologists. Of these lump sums per day and per admission, the patients’ share, as well as
lump sums, the patient charge is €7.44, irrespective of the actual number of the tests reimbursed as fee-for-service (25%), are available in IMA–AIM
tests performed.u records (all have a RIZIV–INAMI (pseudo) code). Although there is no
Laboratory tests performed in patients hospitalized in a general hospital are link with actual consumption (as the lump sums are paid to the hospitals
thus financed through the following channels : irrespective of the actual services delivered) for simplicity we
recommend to aggregate all the costs appearing in the IMA–AIM
records for a specific hospital stay,

s https://www.riziv.fgov.be/insurer/fr/rate/index.htm (“médecins - biologie verstrekkingen klinische biologie’–’Honoraires forfaitaires par journée pour
clinique”–”Artsen - klinische biologie”). les prestations de biologie clinique’)
u
t http://www.inami.fgov.be/care/fr/hospitals/specific- https://www.riziv.fgov.be/insurer/fr/ rate/index.htm (“Médecins - Biologie
information/pseudo/index.htm (‘Forfaitair honorarium per verpleegdag voor clinique”–”Artsen - Klinische biologie”)
32 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

2. If IMA–AIM population data cannot be used, the number and type of 2010 €380 370 502 €95 455 303 €475 825 805 4.98
laboratory tests performed per inpatients stay should be obtained from 2011 €411 539 641 €88 385 028 €499 924 669 5.66
clinical trials, observational studies or be simulated based on literature
about patients health care pathways. The 25% fee-for-service charge Total €4 048 976 251 €975 713 831 €5 024 690 082 5.15
of each laboratory test performed should then be multiplied by 5 in order * Lump sums per inpatient day and lump sums “remuneration for the biologists”;
to account for the lump sum allocations prospectively paid by the **Extrapolation factors from fee-for-service expenses to total expenses (= Total
sickness funds. The patient share of €7.44 per inpatient stay should expenses / Fee-for-service expenses).
further be added from a health care payers perspective. The
multiplication factor was obtained by computing the yearly (2000-2011) Hospital lump sum for medical imaging
RIZIV–INAMI expenses (in lump sums, fee-for-service and in total) for Medical imaging for patients hospitalized in a general hospital is financed
inpatient laboratory tests based on full RIZIV–INAMI accountancy through a mixed system of fee-for service (about 70% of the hospital budget)
records (Doc N, “financial year–boekjaar–année comptable”), see and lump sum payments (about 30% of the hospital budget):17
Table 8. This factor will be checked regularly by KCE and updated if
necessary (publication on KCE’s web-site).  Fee-for-service: the hospital retrospectively charges the sickness funds
the honorarium fee of each medical imaging act performed.
Table 8 – Total RIZIV–INAMI expenses on inpatient laboratory testing  Lump sum 1: the hospital receives prospective lump sum allocations
per inpatient admission.
Year Lump sum Fee-for-service Total Extrapolation
expenses* expenses expenses factor **  Lump sum 2: the hospital charges a “consultancy” lump sum per
inpatient stay from the first day on. This “consultancy” fee is a
2000 €270 811 853 €65 754 539 €336 566 391 5.12
remuneration for the intellectual act of the performing radiologist.
2001 €298 866 969 €72 041 667 €370 908 636 5.15
Fee-for-service charges per medical imaging act and consultancy lump
2002 €308 482 633 €71 007 877 €379 490 510 5.34 sums can be found on the RIZIV–INAMI website.v Consultancy lump sums
2003 €311 901 044 €74 876 818 €386 777 862 5.17 are fixed (€15.83 and €16.83 for non-accredited and accredited radiologist
2004 €318 008 470 €79 464 354 €397 472 824 5.00 in 2012, respectively). Of this lump sum, hospitalized patients pay €6.20 per
admission, irrespective of the actual consumption.
2005 €343 076 050 €79 743 404 €422 819 454 5.30
The lump sum per admission is determined by the case mix of each hospital
2006 €310 148 434 €80 470 707 €390 619 141 4.85
(APR-DRGs and severity level). In 2011 they varied from €0.58 to €124.16
2007 €331 763 720 €84 663 406 €416 427 127 4.92 per admission. This lump sum is chargeable whether or not any medical
2008 €369 195 737 €89 113 191 €458 308 928 5.14 imaging act is performed. Lump sums per admission for each hospital can
2009 €394 811 197 €94 737 538 €489 548 736 5.17 be found on the RIZIV–INAMI website.w

v http://www.inami.fgov.be/insurer/fr/rate/index.htm (“Médecins - Imagerie d’imagerie médicale”–”Forfaitair honorarium per opneming inzake medische
médicale”–”Artsen - Medische beeldvorming”) beeldvorming)
w http://www.inami.fgov.be/care/fr/hospitals/specific-information/pseudo/
index.htm (“Honoraires forfaitaires par admission pour les prestations
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 33

Medical imaging acts performed in hospitalised patients are thus financed by 1.7 in order to account for the lump sum allocations prospectively
through the following channels : paid by the sickness funds. The patient share of €6.20 per inpatient stay
should further be added under a health care payers perspective. The
Table 9 – Sources of financing – Hospital medical imaging for patients multiplication factor was obtained by computing the yearly (2000-2011)
in general hospitals RIZIV–INAMI expenses (in lump sums, fee-for-service and in total) for
inpatient medical imaging acts based on full RIZIV–INAMI accountancy
Sickness funds Ordinary records (Doc N, “financial year–boekjaar–année comptable”), see
patient Table 10. This factor will be checked regularly by KCE and updated if
necessary (publication on KCE’s web-site).
Medical imaging Fee-for-service per act -
acts Lump sum per admission Table 10 – Total RIZIV–INAMI expenses on inpatient medical imaging
(varies per hospital)
Year Lump sum Fee-for-service Total Extrapolation
Remuneration for €9.63: lump sum per admission €6.20 per expenses* expenses expenses factor **
the radiologist for non accredited radiologist admission
“Consultancy 2000 €97 007 111 €173 769 938 €270 777 050 1.56
€10.63: lump sum per
lump sum”
admission for accredited 2001 €99 233 648 €179 041 295 €278 274 943 1.55
radiologist
2002 €98 800 265 €164 329 045 €263 129 310 1.60
In order to value the full cost of the medical imaging acts performed during
2003 €102 475 616 €170 112 091 €272 587 707 1.60
an inpatient stay, taking into account the dual system of financing (fee-for-
service and lump sums), the following procedure is suggested: 2004 €101 204 649 €182 282 412 €283 487 061 1.56
1. If IMA–AIM population data can be used (or IMA–AIM data from the 2005 €122 653 467 €176 320 522 €298 973 989 1.70
“Permanent Sample” - Echantillon Permanent–Permanente Steekproef
- which is a representative subset of the IMA–AIM population data), the 2006 €98 001 155 €175 651 862 €273 653 017 1.56
lump sums per admission and for consultancy, the patients’ share, as 2007 €104 823 792 €180 422 669 €285 246 460 1.58
well as the acts reimbursed as fee-for-service are available in IMA–AIM
records (all have a RIZIV–INAMI (pseudo) code). Although there is no 2008 €114 356 262 €184 257 405 €298 613 667 1.62
link with actual consumption (as the lump sums are paid to the hospitals 2009 €123 063 728 €190 071 715 €313 135 443 1.65
irrespective of the actual services delivered) for simplicity we
recommend to aggregate all the costs appearing in the IMA–AIM 2010 €109 876 769 €191 173 726 €301 050 495 1.57
records for a specific hospital stay, 2011 €138 703 898 €188 603 565 €327 307 464 1.74
2. If IMA–AIM population data cannot be used, the number and type of
Total €1 310 200 361 €1 967 432 678 €3 277 633 039 1.67
medical imaging acts performed per inpatients stay should be obtained
from clinical trials and/or observational studies or be simulated based * Lump sums per admission and “consultancy lump sums”; **Extrapolation factors
on literature about patients health care pathways. The fee-for-service from fee-for-service expenses to total expenses (= Total expenses / Fee-for-
charge of each medical imaging act performed should then be multiplied service expenses).
34 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

Hospital per Per diem price Number of Total cost

Example: Valuation of an inpatient stay. diem prices inpatient days
A patient is hospitalized for 6 days in an acute hospital. During his
hospitalization he undergoes the following tests: a urine test, a blood test, a Acute hospital €388 6 €2 328
chest radiograph, an abdominal echography. He also receives the following
medications: ACE inhibitors (25mg per day), aspirine (Acide acétyl
salicylique 80 mg per day ) and immunoglobulins (1 perfusion per Medicatio Drug included (Extrapolation Nbr of Cost
day). n in the lump to) 100% inpatient
sum or on the reimbursemen days
What is the cost of his hospital stay, from the perspective of the health care exclusion list? t basis
payers? (1/1/2012)
ACE inhibitors
(Captopril In lump sum €0.0715/25 mg 6 €0.43
Laboratory RIZIV–INAMI 25% Multiplication Cost Apotex)
tests nomenclature honorarium factor Acide acetyl-
fee salicylique In lump sum €0.0257/80 mg 6 €0.15
(1/1/2012) (Asa Mylan)
Urine test 543723 €1.19 5 €5.95 Immunoglobuli Exclusion €222.8 / 6 €1 336.8
Blood test 120061 €0.40 5 €2.00 n (Gammagard) list perfusion 0
Patients share - - - €7.44 Patients share - €0.62 6 €3.72
Total costs laboratory tests €15.39 Total costs medications €1 341.1
0

Medical RIZIV–INAMI Honorarium Multiplication Cost

imaging nomenclature fee factor
acts (1/1/2012) In this example, the total cost of the hospital stay is: €15.39 + €72.57 +
€1 341.10 + €2 328 = €3 757.06.
Chest
452701 €12.58 1.7 €21.39
radiograph
Abdominal
460165 €26.46 1.7 €44.98
echography
Patients share - - - €6.20
Total costs medical imaging acts €72.57
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 35

5.7.3.7 Valuation of the average transport costs to health care 5.7.3.8 Valuation of productivity costs (societal perspective)
services Productivity costs are not direct health care costs and as such should not be
Transport costs are not direct health care costs and as such should not be included in the reference case. If relevant, indirect productivity costs can be
included in the reference case. For some interventions, however it may be presented as a complementary analysis, separate from the reference case.
important to quantify the travel expenses incurred by the patient or the In any case, the impact of the medical intervention on the level of productivity
RIZIV–INAMI (e.g. dialysis) to and from the caring institution or physician. must be real and well documented.
These expenses may then be reported as a complementary analysis, Productivity costs are costs arising from production losses due to:
separate from the reference case.
 Unfitness to work/sick leave (in the case of treatment/illness),
Travel expenses belong to the direct non-health care costs category. In
 Early retirement/incapacity to work (in the case of long-term illness or
order to increase the consistency across the economic evaluations, the
disability),
following standard travel costs are suggested:
 Premature death.
 Transport costs are estimated to be €0.30 (2010 value) per kilometer,
which corresponds to the travel fee reimbursed by the RIZIV–INAMI to  Productivity costs are divided into paid and unpaid work (i.e. voluntary
patients admitted in a day care centre (KB–AR 12/10/2010). This fee is job, house keeping…).
indexed each year with the health index. No adjustement is done for the Short-term lost productivity during paid work has to be valued using the
type of transport (personal car, public transport…) and the number of Human Capital Approach. The human capital approach values the time
kilometers is limited to a maximum of 15 per journey (i.e. 30 kilometers during the whole period of work inability due to sick leave, early retirement
per day). (potentially up to retirement) or premature death (up to retirement).
Although these standard costs contribute to reduce some price difference Productivity costs in the Human Capital Approach are to be computed by
between studies, deviations from those values are allowed if the researcher multiplying the total number of days of work absenteeism by the national
can demonstrate that the values relevant for his/her study considerably average labour cost per day. Labour costs include employee wages and/or
differ. salaries and employers' social security contributions, The Belgian average
labour cost per working day was estimated at €257 (costing year 2010;
Source Eurostat, Monthly labour costs,x and assuming 18.8 working days
per months: 52 weeks * 5 working days minus 24 days (legal holidays and
agreed extra holidays) minus 10 public holidays).

x http://epp.eurostat.ec.europa.eu/
portal/page/portal/labour_market/labour_costs/database
36 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

In case of long-term absence from work or death, instead of accounting for Costs of informal care that is not related to the intervention/disease in life-
the whole period of work inactivity, it should be considered that vacant years gained (indirect non-health care costs) should not be measured nor
workplaces can be filled again within a certain period of time. In that case, valued.
only the period until the workplace is filled again by a previously unemployed
person will be valued. The Friction Cost Method should then be used. The 5.8 Guideline 8: Estimation and valuation of outcomes
friction-cost method is based on the idea that organizations need a certain
time span (the friction period) to restore the initial production level after an Outcomes in economic evaluations should be expressed in terms of
employee becomes absent from work. final endpoints instead of intermediary outcomes. Clearly defined
The amount of production lost due to disease depends on the length of this outcome measures, for which there is little debate about the
friction period. Productivity costs are then calculated by multiplying the measurement methods, are recommended.
labour costs per day (i.e. €257, see above) with the duration of the friction For cost-effectiveness analyses, outcomes should be expressed in
period. Unfortunately precise data on the length of the friction period could terms of life years gained. For cost-utility analyses, QALYs should be
not be identified for Belgium. Productivity costs should then be computed by calculated. Life expectancy should be estimated based on Belgian age-
varying the friction period from 2 to 6 months. and gender-specific life tables. Health-related quality-of-life weights
Unpaid work such as voluntary jobs or housekeeping can be considerable, should be based on empirical data, obtained in patients with a
especially in those with chronic diseases. It is recommended to present the descriptive system for health status for which corresponding
incremental number of unpaid working days. As there is currently no preference values exist from the general public such as the EQ-5D. The
consistent method to value unpaid working days, it is recommended not to use of Belgian preference values is preferred. Scenarios with disease-
include them in the cost estimates. specific measures for health-related quality of life and scenarios
including effects on caregivers’ health-related quality of life can be
5.7.3.9 Valuation of informal care costs
presented as complementary analyses but are not acceptable in the
Informal care costs are the value of the time spent, by family and relatives, reference case.
caring for a sick relative. Informal care can be important particularly for long-
term diseases requiring non-specialized care (e.g. Alzheimer disease). The aim of an economic evaluation is to assess the additional costs
Informal care costs may belong to two cost categories. Informal care costs associated with the better outcome of a health intervention. It is important to
are direct non-health care costs if they are related to the intervention/disease include all cost and outcome consequences, including those associated with
under consideration (e.g. relatives caring for a sick parent after hospital positive and negative effects of the treatment (e.g. adverse side effects).
discharge). Informal care costs are indirect non-health care costs if they are The valuation of outcomes depends on the analytic technique used. In cost-
unrelated to the disease/intervention under consideration and occur in future effectiveness analyses, outcomes are expressed in clinical units such as life
life years gained (e.g. relatives caring for the same now older parent years years gained, in cost-utility analyses outcomes are expressed in QALYs
after the intervention succeeded). gained.
If relevant, informal care related to the disease/intervention under study can
be presented as a complementary analysis, separate from the base-case.
For reasons of consistency with unpaid work, informal care should only be
measured (i.e. in number of days spent caring) but not valued.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 37

5.8.1 Effectiveness evaluation in cost-effectiveness analysis To avoid possibilities for manipulation of the quality-of-life values, it is
For cost-effectiveness analyses, the outcomes should be consistent with the strongly recommended to use the EQ-5D and the same set of utility values
results of the clinical file. If this file contains only short term outcomes and for the EQ-5D health states (so-called “EQ-5D tariff” or index values) across
long term outcomes are considered important for the economic evaluation, all economic evaluations. Moreover, it is strongly recommended to calculate
modelling may be needed (cfr. guideline 10). For an intervention with a QALYs based on original Belgian empirical data. If this is not possible, e.g.
impact on short- or long-term mortality, outcomes should be expressed in because the original clinical study was performed in another country, two
terms of “number of life years gained”. Age- and gender-specific life tables options exist:
for Belgium should be used to estimate life expectancy. These data are  either primary data on the health state descriptions should be obtained
available at the National Institute of Statistics.y and “translated” into utility values based on the Belgian tariff scores,
The estimated effectiveness should be based on all-cause mortality in the  or – in case of secondary data analysis – health-related quality of life
reference case analysis. Effectiveness estimates based on disease-specific and utility data used in the economic analysis should originate from the
mortality can be presented in complementary analyses. If the disease has a same country.
major impact on overall mortality in the population examined, all-cause Similar to the requirement for adjustment for baseline risk in estimating the
mortality figures should be corrected for the fact that they include disease- incremental effectiveness of an intervention, adjustment for baseline (age-
specific mortality (see ISPOR’s Principles of Good Practice for Decision and gender-specific) health-related quality of life is required in estimating the
Analytic Modelling in Health Care Evaluation.z All-cause mortality should be incremental utility of an intervention. Unfortunately such data are not (yet)
modelled non-parametrically based on life table data. The functional form of available for Belgium. When quality-of-life data from another country are
the chosen disease-specific mortality function should be explained and used, baseline health-related quality-of-life data should also be from that
justified in a complementary analysis. country. For treatments leading to complete remission to normal health,
5.8.2 Utility assessment in cost-utility analysis reference data on health-related quality of life from the general population
should ideally be used. However, in Belgium such data are not available as
In cost-utility analyses, the valuation methods for health-related quality of of yet. Therefore, reference data from another country should be used for
life should be equal for all comparators. Data on survival and health-related now.
quality of life should be presented separately. As no weights that represent
distributional preferences of the general public according to the populations
affected are available, QALYs should not be weighted in the economic
analysis. This means that in submitted economic evaluations a QALY is a
QALY, no matter to whom it accrues.
Quality-of-life assessment in specific health states, needed for the
calculation of QALYs, should be done in two steps. The first step is to obtain
patients’ health state description (5.8.2.1). Health states should be
described on a standardised descriptive system such as the EQ-5D. The
second step consists of assigning a value between 0 (= value for dead) to 1
(= value for perfect health) to these health states (5.8.2.2).

y z
http://www.statbel.fgov.be http://www.ispor.org/workpaper/healthscience/tfmodeling.asp
38 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

5.8.2.1 Health state description interviewed. There is evidence that expert opinions are not always close to
the descriptions given by patients. Therefore, the use of proxies to describe
In the reference case, a generic health-related quality-of-life measure should patients’ health states is only accepted if patients cannot describe their
be used for the description of health states. The health state description health state themselves (e.g. mentally ill patients, very young children,
should be made by patients on a generic descriptive system such as the unconscious patients…). The reason for using proxies for the description of
EQ-5D (for adults) and the EQ-5D-Y (for youngsters) or SF-6D.aa If justified health states should always be justified with clear arguments.
by the disease or the intervention (e.g. care for disabled persons, terminal
care, vaccination…), also informal caregivers’ health states can be 5.8.2.2 Health state valuation
considered in a complementary analysis, but not as part of the reference
Values assigned to the health state descriptions should come from (a
case. It should then be explicitly stated that the QALY estimates include the
representative sample of) the general public. If the EQ-5D is used it is
effect on caregivers’ health-related quaity of life. Other instruments than the
recommended to use the Flemish tariff values. For other instruments a
EQ-5D exist, e.g. the HUI or QWB scale, but these have not been validated
similar preference valuation set is not available yet for Belgium. If the
in Dutch or French for Belgium. Health state descriptions with the EQ-5D or
primary data are not available but only health-related quality-of-life results
SF-6D in similar patient populations in other countries may be used,
from trials from another country are used, index values from that country
provided that the criteria for valuation as explained in 5.8.2.2 are fulfilled.
should be used for consistency (cfr. supra).
The use of a generic utility instrument should be considered in the phase of
designing a clinical study when a future economic evaluation is envisaged. Mapping valuations from other health-related quality-of-life instruments (e.g.
The study protocol should specify which instrument willl be used to obtain disease specific instruments or another generic instrument) to EQ-5D or SF-
utility values and when it will be used (at prespecified endpoints and events). 6D public preference values should be avoided. The direct use of a generic
utility instrument is recommended. If primary data on health state
If it is thought that a generic instrument is insufficiently sensitive to relevant
descriptions cannot be obtained, mapping is still only allowed if mapping
changes in health in a specific disease, additional (disease-specific) quality-
functions are based on and validated with empirical data.
of-life results can be described in separate analyses. It is not acceptable
though to create an ad hoc disease-specific questionnaire for a single If no original Belgian data are collected and mapping is not possible, generic
economic evaluation and use this in the reference case analysis to estimate health state descriptions and valuations from other countries in the same
the number of QALYs gained. Such ad hoc created instruments, defined as patient population can be used, provided that the source of the valuations is
a set of alleged relevant questions about a disease state and its associated transparent and that potential problems of transferability are discussed. A
health-related quality of life, are not sufficiently validated and tested to offer basic requirement is that health states are valued from a societal
reliable and consistent results. If disease-specific instruments are used, perspective, i.e. derived from a representative sample of the general public.
references to publications that document the psychometric properties Details should be provided on the population to which the valuations refer,
should be provided. Description of health states on a disease-specific and references to publications describing the general population survey
quality-of-life instrument by proxies should be avoided as long as patients in should be given.
the target population are able to complete a survey themselves or can be

aa The SF-6D consists of a subset of the SF-36. When the SF-36 has been used necessary for calculating QALYs. The advantage of the complete SF-36
in a clinical trial, and SF-6D tariff values are available, the SF-36 data can be instead of the SF-6D is that it offers more detailed insight into the dimensions
used to calculate QALYs. In other words, the SF-36 is sufficient but not of health-related quality of life affected by an intervention, as compared to the
more limited SF-6D or EQ-5D.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 39

In the reference case, generic health state descriptions and valuations term morbidity effect between treatment options and with only short term
should be used. Newly set-up studies should therefore preferably always differential costs. If a shorter time horizon is chosen, this should be
include a generic utility instrument such as the EQ-5D (alongside a disease- substantiated with clear arguments. The potential consequences of not
specific instrument if a disease-specific instrument is considered including long term costs and outcomes should in this case be discussed.
necessary), in order to increase the potential usefulness of the data for later A particular issue that may be important for some drug treatments is the
economic evaluations. rapid evolution in development of new drugs. These innovative drugs may
Health state values from different (clinical) studies should be treated with not be formally evaluated yet, but may be expected on the market in the near
utmost caution. Only if measured with the same instrument and in a similar future, making the current drug under evaluation redundant, for instance.
patient population are the values comparable and can they be used in one This cannot be an argument for shorter time horizons, but it can be
and the same economic evaluation. Consistency in methodology for the mentioned in the discussion that certain innovations are expected in the near
valuation of utilities of different health states in the economic evaluation or distant future, which may change the results of the analysis. No formal
should be pursued. analysis can be performed on the likely effect, however, as the clinical
effectiveness of the innovations is still uncertain.
5.9 Guideline 9: Time horizon
5.10 Guideline 10: Modelling
The appropriate time horizon for the economic evaluation depends on
the duration of the impact of the study intervention on relevant costs Modelling should be applied if the available data are insufficient to
and outcomes as compared to the comparator intervention. allow a full assessment of the cost-effectiveness or cost-utility of an
intervention. Models should be based as much as possible on data
The time horizon of the economic evaluation should be in concordance with from clinical studies with the same study intervention and comparator,
the period over which the main differences in costs and health on data from validated databases and/or data from literature. Modelling
consequences between the intervention under consideration and its should always be justified. If modelling is performed, the structural
comparator are expected. Health consequences include intended as well as hypotheses, assumptions and sources of information should be
unintended consequences (e.g. side effects). justified and presented in a clear and transparent way. Modelling
Treatments for chronic diseases or acute diseases with long term sequelae inputs and outputs should be consistent with existing data and have
mostly have consequences over a patient’s lifetime. In these cases, a at least face validity. Primary data and original sources of information
lifetime time horizon should be adopted for the economic evaluation. used to define the values of input parameters as well as the original
However, even in chronic diseases, the impact of the investigated treatment computer model should be kept at the disposal of the Expert
on outcome as compared to the comparator treatment may be limited in Committee responsible for the reimbursement advice.
time. For example, a new peri-operative intervention may reduce adverse
events and improve quality of life in the first two months after surgery, after
which the risk for events and quality of life becomes equal to that of the
comparator intervention. In this case, it is sufficient (if impact on adverse
events and quality of life is the main or only outcome of interest) to focus the
economic evaluation on the first two months after surgery. Hence, a shorter
time horizon may be justified when there is no differential mortality or long
40 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

5.10.1 Need for modelling Guidelines for good general modelling practices have been developed by
Modelling may be needed for the extension of the analysis beyond observed the modelling task force of ISPOR.bb Specific guidelines for infectious
time periods, e.g. because patients are no longer followed once they have disease modelling have been developped by the WHO.cc,18 These guidelines
reached a particular clinical endpoint. In order to know the effects of a ought to be followed whenever a model is built.
treatment on long-term mortality or other long-term outcomes, extrapolation 5.10.2 Choice of the model design
modelling may be necessary.
Different types of models can be used, the major categories are decision
Another reason for modelling is the simulation of final outcomes based on trees and Markov models. The main principle is that a model should be kept
observed data from intermediate outcomes. Often in clinical trials, only as simple as possible. A model’s internal structure should be consistent with
intermediate outcome measures are included (e.g. blood pressure proven or generally accepted relationships between parameters and health
reduction). Other studies may provide information on the relationship states. The more complex the model, the less likely it is that sufficient data
between the intermediate outcome measure and a final outcome measure are available to populate the model.
(e.g. blood pressure and mortality). The relevant outcomes in economic
evaluations are the gain in life-years or quality-adjusted life years. 5.10.3 Precision of model structure and hypotheses
Modelling can also be used to simulate the real life application of an All assumptions made in the model should be explicitly documented and
intervention even if trial data are available. RCTs usually do not reflect real justified. All assumptions should be tested in the sensitivity analysis and/or
life settings. Adaptations by means of modelling may be useful to assess scenario analysis to test the robustness of the results (see guideline 11).
effectiveness instead of efficacy as presented by the RCTs. This can be If primary data or expert opinions are used, the original dataset should be
done by adjusting for differences in baseline risk between the trial population provided. The population for which outcomes are modelled should be
and the real-world target population.5 Adjustments for protocol-driven costs specified. This may be a hypothetical population, but should be consistent
or events should also be considered. with the target population for the product and the sources used for valuing
Modelling allows the inclusion of data from different sources. Meta-analysis the modelling input parameters.
of clinical trials may increase the reliability of the clinical evidence and All variables in the model and their sources must be listed and documented
thereby the validity of the economic model. Administrative data may provide in a table (Table 11):
reliable estimates of e.g. intervention costs for the health care payers.
Sometimes, modelling is needed to take externalities associated with the Table 11 – Description of the variables used in a model: template
disease or treatment into account (e.g. transmission of infections, bacterial
Variable Description Mean Distribution & Source
resistance…). Externalities may not always be captured well during clinical
parameters (e.g. SD, 95%
trials, e.g. because they were not expected and therefore measurement was
CI, α1, α2…)
not included in the study protocol.
Finally, modelling can be used to compare the intervention with the relevant
comparator. A comparisons between interventions that have never been
directly compared in a clinical trial may be modelled. SD: Standard deviation, CI: confidence interval
The decision to model should be justified in the economic submission.

bb cc
http://www.ispor.org/workpaper/healthscience/tfmodeling.asp http://whqlibdoc.who.int/hq/2008/WHO_IVB_08.14_eng.pdf, see chapter 6
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 41

Sources used for outcome assessment, valuation of costs and assessment Models should be kept as simple as possible, but without omission of
of probabilities should be presented and described in detail. Preference is important processes. The original computer model should be put at the
given to peer-reviewed publications or primary data as source for the input disposal of the Expert Committee upon request. Confidentiality will be
parameters’ values. Expert panels are not allowed for the assessment of guaranteed by the Committee. The choice of the modelling software is free.
probabilities or outcomes if data are available in literature. They are of the
lowest level of evidence. If no published evidence is available, strict 5.10.4 Calibration, face validity and cross-validation of a model
methodological criteria apply to expert panel consultation for this approach The results of the model should be logically consistent with real-life
to be an acceptable source of input (see Appendix 4.3). The use of expert observations and data (calibration). For example, if age-specific incidences
panels should always be well justified. Abstracts and oral presentations of a disease are used in a model, the total incidence generated by the model
usually provide insufficient information to assess the quality of their contents. should not considerably be higher or lower than the observed incidence in
They should be avoided as source for input values. the population, unless the difference can be explained by differences in the
Whenever input variables are based on pure assumptions, this should be population structure. Or life expectancy of a population with multiple severe
explicitly mentioned as such in the table, by putting “assumption” in the co-morbidities should not be better after extrapolating results to a lifetime
column “source”. horizon in comparison to the general population. In other words, there must
be a logical connection between inputs and outputs of a model.
For models that extrapolate to longer time periods, i.e. for interventions with
long-term sequelae, it is recommended to present different scenarios to The results of the model should be intuitively correct, that is, the model
show the impact of different extrapolation approaches on the results:19 should have face validity. The model description should be transparent
enough to allow an explanation of the differences with other models for the
 The first scenario assumes that the treatment effect disappears same interventions (cross-validation).
immediately in the extrapolated phase (stop-and-drop approach). This
is the most conservative extrapolation approach. The presentation of the results of an economic model as a point estimate
together with its appropriate uncertainty range is an absolute prerequisite.
 The second scenario assumes that the incremental treatment effect An economic model is by definition subject to uncertainty. The results are
stays the same as during the observed phase. conditional upon the input data and the assumptions applied in the model.
 The third scenario assumes that the initial treatment effect fades out in Both the uncertainty about the input data and the assumptions generate
the long term. uncertainty in the outputs. This uncertainty should be appropriately
The scenarios are all part of the reference case analysis because the choice presented, as the level of uncertainty might be an element in the decision
of an extrapolation approach is mainly a judgment. By presenting different, making process. For the recommended presentation of the results, see
sometimes extreme, scenarios, the uncertainty related to the effectiveness chapter 9.
of the therapy in the extended period can be assessed.
The presentation of scenarios is the most transparent option to show how
robust the results are to the extrapolation approach used. Each scenario
should be accompanied by appropriate sensitivity analyses on uncertain
parameters as specified in guideline 11.
42 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

5.11 Guideline 11: Handling uncertainty and testing the 2. Structural and methodological uncertainty: uncertainty coming from the
robustness of the results analytical methods chosen to perform the evaluation (e.g. health states
in the model, discount rate or extrapolation methods). This type of
uncertainty is usually handled by presenting results from a
Irrespective of the study design, the uncertainty surrounding the cost- methodological reference case and other scenarios handled through
effectiveness/cost-utility estimates should be analysed using one-way sensitivity analyses.
appropriate statistical techniques. Interval estimates should be Generalisability refers to applicability of the results to other populations (e.g.
presented for each uncertain parameter in the economic evaluation. non-trial populations with different baseline risk). Transferability refers to the
The different types of uncertainty should be addressed, i.e. parameter, applicability of the results from other countries. These two aspects should
structural and methodological uncertainty. For models, probabilistic be assessed separately. Context-specific studies are, however, preferred.
sensitivity analyses should be presented. Uncertainty around the
incremental costs, incremental effects and ICERs should be provided Parameter, methodological and structural uncertainty should be specifically
by means of confidence or credibility intervals. A cost-effectiveness addressed in the economic evaluation. State-of-the-art methods should be
plane and cost-effectiveness acceptability curve should be presented. used for the estimation of the confidence interval around the incremental
The most important contributors to the uncertainty of the estimated cost-effectiveness ratio.
incremental cost-effectiveness/cost-utility ratio should be shown. In case of modelling, sensitivity analysis can help determining the
importance of the different assumptions behind the model on the results.
Uncertainty in economic evaluations of healthcare interventions is Probabilistic sensitivity analyses should be performed on all uncertain
omnipresent, and should be properly described and accounted for in the parameters in a model; i.e. one probabilistic sensitivity analysis where all
submitted economic file. Uncertainty should be distinguished from uncertain parameters are allowed to vary according to a predefined
variability, heterogeneity,20 generalisability and transferability. distribution, e.g. by means of Monte Carlo simulations. Distributions used for
the uncertain modelling parameters should be justified. For composite
Variability refers to the variation or randomness observed within a
measures, such as total costs, the different components with their respective
homogeneous sample of patients. Variation by chance between individual
distributions should be included in the sensitivity analysis if applicable. The
patients is not the primary concern in economic evaluations that focus not
probabilistic sensitivity analysis should be performed on the reference case
on the individual but on a specific target population. Nevertheless, detailed
and the alternative scenarios such as the scenarios related to the assumed
descriptive statistics, showing the distribution and variability of costs and
effectiveness of the therapy in extended time periods. It is recommended to
effects data, should be presented.
show the most important contributors to the uncertainty of the estimated
Heterogeneity refers to observed differences between patients which can, incremental cost-effectiveness/cost-utility ratio (e.g. by means of a Tornado
in part, be explained (e.g. differences in age, sex…).20 diagram). The central estimate of the ICER results directly from the
Uncertainty is usually divided into two areas:20 probabilistic sensitivity analysis as the mean of the simulated ICERs. This is
1. Parameter uncertainty: uncertainty around input parameters. This not necessarily equal or close to the ratio of the mean incremental cost and
uncertainty is reflected in probability distributions based on a sample of mean incremental effect, which is the deterministic version of the ICER. A
data and is handled via probabilistic and one- or multiple-way sensitivity deterministic ICER can be presented if the Monte Carlo simulations fall in
analyses. different quadrants of the cost-effectiveness plane.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 43

In addition to probabilistic sensitivity analyses, a scenario or univariable The argument for doing so is to avoid a too strong penalization of
sensitivity analysis could be performed on modelling parameters that are interventions that generate most of their benefits in the future (e.g. screening
decisive for the cost-effectiveness ratio, such as the price of a and vaccination programmes). The choice of the discount rate for costs in
pharmaceutical product. Also for methodological uncertainty, arising for Belgium was previously set at 3%. This rate is maintained in order to allow
instance from the applied discount rate or the extrapolation method used in the comparison with previous economic evaluations. This guideline prefers
models, scenario analyses should be used. This is comparable to one-way consistency in the discount rate above a fluctuating one (e.g. the interest on
sensitivity analysis, where only one parameter is changed (the discount rate the short-term government bonds). The change in the value of health over
or the assumed effectiveness in the extended time period). For each time is highly uncertain. Therefore, the discount rate for outcomes is
scenario a probabilistic sensitivity analysis can be easily performed and uncertain. Awaiting further evidence on the most likely discount rate for
hence results can be presented with their 95% credibility interval. Values outcomes in Belgium, and to remain consistent with previous guidelines, we
and distributions of other parameters can be kept as in the reference case currently recommend a rate of 1.5% for discounting outcomes in the
analysis for these scenarios. There is no need to present all possible reference case analysis.
combinations of all scenarios. Hence, if one scenario changes the discount Apart from the reference case analysis with a 3% discount rate for costs and
rates and another scenario changes the price of the product, it is not 1.5% for effects, the company should present alternative scenarios to allow
necessary to present a scenario where both the price and the discount rates the decision maker to judge the relative importance of using different
have been changed compared to the reference case. discount rates for the final result. Given the prevailing advice for the base-
The applicant is free to present additional univariable sensitivity or scenario case analysis in many economic guidelines of other countries, a 3% discount
analyses if these are deemed relevant. Appropriate justification of the rate for both costs and benefits can be considered. Alternative scenarios
additional analyses should be provided. include a 0% discount rate for both costs and benefits or a 5% discount rate
for both costs and benefits. For the decision maker it is important to keep in
5.12 Guideline 12: Discount rate mind that, if he wishes to compare the ICER of a new product with the ICER
of a product for which a decision has already been taken (based on the ICER
Future costs should be discounted at a rate of 3%; future benefits at a and other elements), he should always compare the ICERs of the reference
rate of 1.5%. To assess the sensitivity of the results to the discount case analyses of both products.
rate applied, different scenarios should be presented.

Incremental cost-effectiveness ratios should be presented in present values.

This means that future costs and benefits should be discounted to reflect the
lower value given to future costs and benefits. The formula to translate future
values to their present value is: valuet / (1 + i)t, with t being the time period
and i being the discount rate. The choice of the discount rate for costs and
benefits is mainly a normative issue. Guidelines recommended an equal rate
for costs and benefits for a long time, but this approach has been debated
frequently in literature. Dutch guidelines now recommend a lower discount
rate for benefits than for costs.
44 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

6 GUIDELINES FOR BUDGET IMPACT Costs and outcomes: In budget impact analyses, it is recommended to
ANALYSES calculate both the global budget impact and consequences for the
different health care payers separately. This implies that potential
transfers of budgets between different levels of governments and/or
Economic evaluations help to assess the ”value for money” of an patients should explicitly be considered. Tariffs and prices should be
intervention (i.e. the acceptability). Budget impact analyses assess the kept constant over the years (i.e. not inflated). The cost consequences
affordability. of the treatment effect, side effects and other short- and long-term
The following guidelines for economic evaluations are also applicable consequences (e.g. follow-up treatment) should be included in the
to budget impact analyses: budget impact analysis since they will have an impact on the
healthcare budget.
• Guideline 2 (Perspective of the evaluation),
• Guideline 7 (Calculation of costs), Time horizon: The time horizon in a budget impact analysis depends
on the time needed to reach a steady state. It is recommended to
• Guideline 10 (Modelling),
present the budget impact up to the steady state, with a minimum time
• Guideline 11 (Handling uncertainty). horizon of three years.
In addition, the following guidelines apply to budget impact analyses: Discount rate: Future costs and savings should not be discounted in a
Target population: Besides the recommendations stated in guideline 3 budget impact analysis.
for economic evaluations, budget impact analyses should estimate the
potential size of the population targeted and its potential evolution 6.1 Similarities and differences between economic
over time (e.g. shifts in incidence, prevalence, disease severity). The
methods used to estimate the population size should be described and
evaluations and budget impact analyses
justified. The degree of penetration of the intervention in the targeted Economic evaluations assess the acceptability of an intervention, i.e. does
population (e.g. detection rate, compliance, market share…) needs to it offer value for money. In contrast, budget impact analyses (BIA) help to
be considered and justified. determine if we can afford a specific intervention, i.e. do we have the
budgets to implement and/or reimburse the intervention. Table 12 provides
Comparator: The budget impact analysis calculates the predicted an overview of similarities and differences between guidelines for economic
financial impact of introducing an intervention compared to the current evaluations and BIA. More details on similarities are provided in the previous
situation. sections. In the BIA guidelines we focus mainly on the differences.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 45

Table 12 – Similarities and differences between economic evaluations consequences of health outcomes (e.g. treatment cost of adverse events) are
and budget impact analyses. included in both the economic evaluations and BIA.
The similarities in Table 12 show that BIA can partly rely on the economic
CEA BIA
evaluation. In case an economic evaluation has been performed, a lot of
Research question Acceptability Affordability information is often already available: the cost of the initial treatment, the
Perspective Healthcare payers costs of avoided re-interventions, adverse event costs, exclusion of protocol-
driven costs, adjustments to make the analysis for the real-world target
Consistent with reimbursement request population, etc. Differences should however not be neglected and several
Target population
Closed * Open adaptations may be required to make the data from the economic evaluation
Comparator On the efficiency frontier Current situation useful for the BIA, such as accounting for the open population and time-
dependencies required for a BIA.21 While the BIA can be performed
Direct healthcare related costs separately, integration of the two analyses may avoid duplication of efforts.22
Costs
No transfers Transfers
6.2 Guideline 13: Perspective
Health outcomes Included Not included **
As economic evaluations, BIA should be carried out from the healthcare
As long as incremental payer’s perspective (see 5.2). Other alternatives are also possible, such as
Time horizon costs or outcomes are Up to steady state the hospital’s or patient’s perspective. Nevertheless, in the context of
generated reimbursement, the BIA calculates the impact of a policy decision on the
Modelling Decision tree, Markov model… healthcare budget. As a result, the main target of such analysis remains the
healthcare payers. Complementary analysis from other perspectives should
Probabilistic and one- or multiple-way probabilistic
Handling be separated from the reference case and justified.
sensitivity analyses, scenario and subgroup
uncertainty
analyses 6.3 Guideline 14: Target population
Costs: 3%, effects: As for economic evaluations, the target population should be consistent with
Discount rate No discounting
1.5% the population defined in the reimbursement request (see 5.3). Subgroup
Presenting results Incremental cost, Yearly budget impact, analysis can be performed if there is an appropriate justification. Similar as
incremental effect, disaggregated impact, in the economic evaluation guidelines this can be due to differences in
ICER, cost- results of the safety, treatment effect, baseline risks, or costs, which will result in different
effectiveness plane, sensitivity analyses ICERs.
CEA-curve, results of Furthermore, BIA entails some specific considerations:
the sensitivity analyses  The potential population size should be specified and the estimation
CEA-curve: cost-effectiveness acceptability curve; ICER: incremental cost- method described and justified.23 Attention should be paid to the
effectiveness ratio. evolution of the size of the target population over time with and without
* In most cases, the population in economic evaluations is closed. However, there the new technology.21 If it is impossible to make a good estimate of the
are examples where this is not the case. For example, were there are contagion population size, then it is advised to perform the BIA for a number of
effects. patients that is easy to extrapolate (e.g. 100 patients). Applying this
** Health outcomes as such are not included in the BIA. Nevertheless, the cost option should be justified.
46 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

 In this context, it is important to consider shifts in incidence and  The budget impact should be calculated for all indications of the
prevalence. For example, if the extension of survival is expected, it is intervention. This should avoid ‘salami slicing’ to minimize the potential
necessary to consider the increase of the prevalence of a given budget impact of a specific intervention. For example, a drug may be
illness.24 evaluated for the treatment of an orphan disease but also be used for a
 Shifts in disease severity should be considered. For example, a more common disease. This should be explicitly stated and the budget
screening program may detect more patients with a specific disease impact should be presented separately for each indication. This should
and may create a shift in identifying the disease at an earlier stage be done for the indication under consideration and those for which there
compared to the situation without screening. is already a reimbursed.
 The degree of implementation needs to be considered (e.g. detection  Off-label use is a complex issue. Several types of off-label use should
rate, the population percentage expected to use the technology,23 be distinguished. First, off-label use may be observed without any hard
compliance, the market share…). Justification for the estimates should underlying evidence. This aspect of BIA may especially be of
be provided. The market share refers to two concepts. First, there is the importance in a reimbursement revision or to demonstrate the
market share that indicates in how far a new intervention may replace importance of introducing preventive measures.24 Next, there is off-label
an existing alternative. For example, the introduction of a new device use because certain interventions are included in clinical practice
may replace existing treatments. This type of market share should be guidelines while being off-label (e.g. in paediatrics because the drug is
included in the BIA. One of the main factors that should be considered only licensed for adults). Finally, it may also occur that the company
in this case is the substitution of other interventions by the new does not apply for marketing authorisations for a specific indication,25
alternative.24 On the other hand, the market share can also indicate or similarly, does not apply for an alternative treatment schedule if more
which part of the market different competitors of a specific intervention profitable alternatives are at their disposal. If there is evidence that
will take. For example, if a certain intervention is reimbursed, how much these alternatives are effective and possibly more cost-effective, then
of the market will be taken by company A, B, C and others that bring the BIA should take these alternatives into account. This might support
such a product to the market? The BIA should in the first place calculate conditions for reimbursement or justify the initiation of further research.
the impact of reimbursing the intervention, and not just the companies’ As for all input variables, sources should be clearly described.
own product. Such additional information may be provided in a separate
analysis, especially if the price of the competitors is different.
 If several relevant subpopulations are distinguished, both
subpopulation-specific BIA and aggregated BIA for the general target
population should be performed and reported.24
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 47

6.4 Guideline 15: Comparators The cost consequences of the treatment effect, side effects and other short
and long term consequences (e.g. follow-up treatment) should be included
The budget impact analysis calculates the predicted financial impact of
in the BIA since they will have an impact on the healthcare budget. The data
introducing an intervention compared to the current situation. The
source for estimating the treatment effect should be based on studies with
intervention in the BIA is the same as in the economic evaluation, but the
the appropriate design (i.e. preferably RCTs). If information from the
comparator may be different since cost-effectiveness is calculated on the
underlying studies is insufficient for determining the influence of side effects
efficiency frontier (see 5.4 and Appendix 5).
on the healthcare budget, this should be noted in the analysis.24
The treatment most likely to be replaced by the new treatment can be
Protocol-driven costs should be excluded from the analysis.24
identified through market research, surveys, database analyses or patient
chart reviews. In case of an add-on treatment, the comparator is the usual The input variables for the BIA should be presented transparently. The level
daily practice without the add-on treatment. of detail should be such that the reader could duplicate all the calculations
in the model.21 An overview of these input variables should be provided in a
If it is not possible to identify the treatment most likely to be replaced, the
table (containing e.g. name of variable, mean cost, uncertainty, source).
reference treatment, as defined by Belgian clinical guidelines, should be
Assumptions should be mentioned explicitly. It is recommended to separate
used.
the measurement of resource use or volumes and the valuation or unit costs.
Several BIAs should be performed for all relevant treatment alternatives (i.e.
The valuation of cost items is related to the perspective of the BIA (see part
for those on the efficiency frontier). As such, it is not only relevant to
6.2). Researchers should strive to use Belgian real-world expenditures.
calculate the budget impact for the intervention under consideration, but also
Further details on cost calculations and an overview of databases is given
for a more cost-effective alternative.
in part 0.
6.5 Guideline 16: Costs and outcomes It is recommended to keep the tariffs and prices stable in the BIA, unless
It is obvious that the cost of the intervention under consideration should be there is a good justification for not doing so (e.g. confirmed information on
included in the BIA. It should be specified if the new intervention seems to pricing policy, implementation of an approved new policy rule in the near
replace currently used alternatives. This should be justified and the budget future or price changes after patent expiration).
impact of abandoning these alternatives should also be included.23,24 The BIA should also include (or at least discuss) the costs related to possible
The inclusion of cost items is directly related to the chosen perspective. conditions to introduce the intervention under consideration. This may
Similar to the guidelines for economic evaluations, direct healthcare related involve the need to train the personnel or the existence of specific
costs are included in the reference case. The impact on productivity and diagnostics or care facilities.23,24,26 Discussion of preconditions of effective
other items outside the health-care system costs should not routinely be introduction should focus on those conditions that are necessary for the
included in a BIA as these are not generally relevant to the budget holder.21 effective, cost-effective, and socially accepted use of the new healthcare
Nevertheless, significant direct non-healthcare related costs (e.g. transport) intervention. Financial needs for establishing these preconditions should be
or indirect non-healthcare related costs (e.g. productivity costs or costs for summarised.26
unpaid caregivers) could be included in BIA, provided that the relevance of
these costs is considered.24 This should be performed as a complementary
analysis, clearly separated from the reference case analysis.
48 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

It is recommended to calculate both the global budget impact and It is recommended to present the budget impact up to the steady state, with
consequences for the different health care payers. Transfers of budgets a minimum time horizon of three years. If the steady state is reached within
between different governments and/or patients are not considered in a shorter period of time, then it is easy to extrapolate the budget impact to
economic evaluations since they are no incremental cost to the healthcare those 3 years. In contrast, if the budget impact would further increase after
payer. This means they are included in the cost-effectiveness/utility this period of time, restricting the analysis to a shorter period will not provide
analyses without it being clear who bears the cost. In BIA, the total budget complete information to the decision makers in order to make well-informed
impact is calculated in the first place. Nevertheless, introducing an decisions.
intervention may have a different impact on the funds at different levels: It is clear that the budget impact in the long term may be a multiple amount
federal, communities, regions, municipalities or other parties. Savings for of the budget impact in the short term. First, the population examined in the
one party may result in expenditures for another. For example, savings for BIA is open, which means that particular patients enter or leave the
the federal government from accelerated rehabilitation and possible shorter population when they meet or fail to meet the defined inclusion criteria at a
hospital stays may increase expenditures for communities due to increasing given moment.23 This contrasts to the clinical efficacy/effectiveness and the
use of home care for patients that are sent home.27 economic analysis, where the examined population usually is closed (a
Another example is the shift in expenditures when starting preventive cohort of patients is defined at the start and all the included patients remain
campaigns, at the expense of communities, while this possibly creates in the examined population within a given time horizon).23 The Polish
savings at the federal level. guidelines mention that the time horizon of the analysis should correspond
to the time necessary to obtain a maximum or stable share in the market of
6.6 Guideline 17: Time horizon the drug.24 However, even if an equilibrium is reached in the market share
The time horizon of the BIA depends on the time needed to reach a steady or treated population, the budget impact may still change afterwards. For
state. The analyst should calculate the yearly country-specific budget impact example, device replacements, re-interventions or adverse events may
up to this steady state. Some guidelines recommend to estimate the budget have an additional impact on budgets several years later. For policy makers,
impact in the short and medium term and explicitly mention a time period of it is not only important to know if we can afford to fund an intervention given
2 to 3 years,28 a period of usually 5 years,29 or the cumulative impact over a the current budgets, but also to know what the long term budget impact may
period of 3–5 years.26 The ISPOR guidelines state that “BIA should be be. Therefore, an assessment of the budget impact of a given intervention
presented for the time horizons of most relevance to the budget holder. They should be performed over a time period that is sufficient to reach a steady
should accord with the budgeting process of the health system of interest, state impact on the general annual health care budget. Many factors, such
which is usually annual. The framework should allow, however, for as the diffusion rate, type of treatment and disease survival, long-term
calculating shorter and longer time horizons to provide more complete events, evolution of target population, etc. may thus have an impact on the
information of the budgetary consequences. A particularly useful extension appropriate time horizon for the BIA.
of the time horizon is to reflect the impact that might be expected when a
steady state would be achieved. This will generally be longer than the
current budget period because of costs and benefits that accrue over time.”21
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 49

6.7 Guideline 18: Modelling 6.9 Guideline 20: Discount rate

Similar for economic evaluations, modelling may be needed to calculate the The BIA provides an estimate of financial means over time. Discounting is
budget impact for several reasons: make the analysis for the real-world performed to reflect time preferences. However, the BIA calculates the
target population, take account of the appropriate comparator, include real- (yearly) budget impact of introducing an intervention, without presenting the
world costs (e.g. excluding protocol-driven costs), extend the analysis to the current value of these financial streams. Therefore, in agreement with some
appropriate time horizon, etc. In other words, bring together the best other guidelines,21,23 the Belgian guidelines recommend not to discount
available data from different sources. For further details, we refer to costs in the BIA.
part 5.10.
6.10 Guideline 21: Presenting results
If an economic evaluation was performed, the BIA model should be
consistent with the clinical and economic assumptions in this economic Presented results should be transparent, complete and understandable. In
evaluation.21 For example, assumptions on compliance should be the same order to do so, the budget impact should be presented for each year within
in both cases. In contrast, the justified comparator in an economic evaluation the relevant time horizon. This will show the evolution of expenses over time
(working on the efficiency frontier) may be different from the comparator in (inclusive the steady state). Results should also be disaggregated. The
the BIA (actual financial streams compared with the current situation). contribution of different components to the budget impact should be reported
separately from the general budget impact: e.g. the impact of the initial
6.8 Guideline 19: Handling uncertainty intervention, replacement costs, re-hospitalisations or re-interventions,
Similar to the economic evaluations, uncertainty in BIA of healthcare adverse events, follow-up costs, etc.
interventions is omnipresent, and should be properly described and In the base case, the BIA separately presents the direct healthcare related
accounted for. costs. Where it is likely to have an impact on the results of the analysis, the
Similar as in the economic evaluation guidelines, the probability of the impact of the intervention on direct non-healthcare related costs (e.g.
appearance of particular values for a range of input variables is accounted transport) or indirect costs (e.g. productivity loss) can be quantified in a
for through probabilistic sensitivity analysis (PSA). One- or multiple-way separate analysis.
sensitivity analysis can be performed on the most important variables such Depending on the approach used, and if considered relevant,
as the price of the intervention or the diffusion rate. The variability between disaggregation is possible for several other aspects:
subgroups is handled in subgroup analysis and structural or methodological  Outcomes can be presented separately for the different healthcare
uncertainty is dealt with in scenario analysis. The subgroup and scenario payers. A distinction is not only possible between government and
analyses should also be performed probabilistically. For further details, we patients, but also between e.g. the federal government and the
refer to part 5.11. communities.
 Outcomes can be presented in natural (e.g. number of unpaid working
days) and monetary units.21, 24
 Impact on the pharmaceutical budget must be presented separately
from the impact on other budgets
50 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

Results of the probabilistic sensitivity analysis should show the mean budget
impact, as well as the 95% credibility interval. The mean and 95% credibility
7 DISCUSSION
interval can also be presented for the different components. A critical These methodological and reporting guidelines are developed as a tool to
description of the obtained results and conclusions is necessary. make economic evaluations and budget impact analyses in Belgium more
Presentation of the (dis)aggregated budget impact in table format or in a relevant, transparent and consistent.
graph are encouraged. Similar as for the economic evaluation, results of the The ultimate decision to reimburse or not reimburse a medical intervention
sensitivity and/or scenario analyses can be presented in table format or will depend on the quality of the submitted reimbursement request file and
graph. The factors that determine the budget impact should be described.2 the therapeutic value of the intervention but also on other aspects that may
not be considered explicitly in the submission, e.g. equity implications,
severity of disease, patient characteristics and organisational issues. As
such, the economic evaluation and/or the BIA will be but one input in the
decision making process.9 Other information or additional analyses that may
provide relevant information to the policy maker may be presented but
should be clearly separated from the original economic evaluation and BIA.
An intervention with a relatively high incremental cost-effectiveness ratio
may still be worthwhile if other elements weighted heavily in the decision
process. Nevertheless, the economic evaluation and the BIA are very
important elements for the decision maker, as they give clues about the
efficient allocation of scarce resources and the affordability of specific
interventions. Quality and consistency in health economic submissions
might improve the extent to which such evaluations can reliably and
consistently be used in the reimbursement decision making process.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 51

of the European Public Assessment Reports for national reimbursement

8 POLICY RECOMMENDATIONS commissions. While this is an important step in the right direction for
A number of issues have been identified during the development process of pharmaceutical products, data requirements and data availability for medical
these guidelines, giving rise to some policy recommendations. devices lag behind.
First, access to data is a major problem in Belgium. It should be noted, Fifth, input data required to performed an economic evaluation or a budget
however, that since the publication of the first guidelines for impact analyses are numerous and specific. The need for such data should
pharmacoeconomic evaluations in Belgium, several initiatives have been be taken into account when setting up clinical trials, before market
taken to improve access to these data. For example, the per diem price per authorization and before revision requests.
hospital is now publicly available. These data are made available in Sixth, the analysis, reporting, evaluation and interpretation of economic
compliance with the privacy regulation by provision of aggregated rather evaluations and budget impact analyses are of utmost importance in the
than individual data. context of advising reimbursement committees. Applicants and policy
Second, the Royal Decree of December 21, 2001 would benefit from an makers should take care that sufficient resources are available to take up
integration of guideline 2 concerning the perspective of the cost calculation this responsibility. The systematic use of the economic guidelines will
in an economic evaluation. The Decree stipulates that the advice formulated increase the credibility of the evaluations and consequently their usefulness
by the Drug Reimbursement Committee should take the relative costs to the for drug reimbursement decisions.
health insurance (RIZIV–INAMI) and the relative effects into account. Finally, Belgium lacks baseline reference data on the health status of the
However, as demonstrated in guideline 2, using the costs for the health Belgian citizen in each age and sex group, collected with a generic utility
insurance in a full economic evaluation may have a perverse effect in the instrument that can be used to calculate QALYs. As a consequence,
Belgian health care reimbursement system towards other health care economic evaluations have to rely on baseline reference data from other
payers, such as the patients. It would be more appropriate to state that the countries, introducing additional uncertainty in the economic evaluation as
costs should be calculated from the perspective of the health care payers, we cannot be sure whether the reference health states from abroad are
including the government and the patient. Also, in general, the applicable to the Belgian population. There is an urgent need for data
recommendations from this report should be integrated in the legislation collected in a representative sample of the general population, allowing to
about the reimbursement of drugs, medical devices and medical judge the relative gain and/or loss in health from an intervention and/or a
interventions. disease.
Third, new evidence or reliable data may become available after a
reimbursement decision has been made, that invalidate the results of the
economic evaluation or the budget impact analysis. In that case health care
payers should be allowed to ask for a revision of the reimbursement
conditions of that intervention and should more often use this possibility.
Fourth, the problem of access to complete data from all RCTs, including all
outcomes according to the trial protocol, to governmental agencies and
other research groups remains, however, a problem. Initiatives have been
taken on the European level, to register all clinical trials set up in Europe in
a central public database; In addition, a collaboration between EMA and
HTA agencies has been set up to improve the relevance and completeness
52 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

Key points • Baseline reference data on the health state of the Belgian citizen
in all age and sex groups should be collected, using a generic utility
To the Minister of Social Affairs and Public Health:
instrument (i.e. EQ-5D) to allow more valid estimates of the level and
 Access to and provision of Belgian data for the measurement and the natural evolution of the health-related quality of life.
valuation of resource use should be further facilitated for
economic evaluations and budget impact analyses.
 The recommendations from this report should be integrated in the
legislation about the reimbursement of drugs, medical devices and
medical interventions.
 If new evidence or reliable data invalidate the results of an
economic evaluation or a budget impact analysis, also health care
payers should be allowed to ask for a revision of the
reimbursement conditions and more often use this possibility.
To the health care industry and the competent national and
international bodies:
 Access to data from RCTs available at the companies should be
facilitated for governmental agencies.
To the health care industry:
 Data requirements for economic evaluations and budget impact
analyses should be taken into account when setting up clinical
trials, before market authorization as well as for revisions.
To the healthcare industry and the Experts Committees:
 To increase the credibility and usefulness of economic
evaluations and budget impact analyses for reimbursement
decisions, both the applicants and the RIZIV–INAMI should
systematically apply these guidelines for drugs and medical
devices. The extent to which those guidelines can be integrated in
the current appraisal procedure should be assessed. For other
medical interventions, the operational implementation of these
guidelines will be made progressively and will be evaluated after 2
to 3 years.
To the sponsors of the Belgian Health Interview Survey and the WIV–
ISP:
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 53

 Target population and possible subgroups + justification for choice of

9 REPORTING GUIDELINES patients and subgroups in a Belgian context.
The recommended structure of an economic evaluation report is presented
 Based on this information: formulate a clear question in answerable
below. This structure is based on the reporting guidelines developed by the
form.
Pharmacoeconomic Committee of the Belgian Society for
Pharmacoepidemiology (BESPE).30 Some specific reporting guidelines for 9.4 Literature review
models are presented in Chapter 10.
9.4.1 Clinical literature review
9.1 Executive Summary
9.4.1.1 Methods
Includes:
 Objectives: specifying study intervention, comparator, target  Review questions.
population.  Search strategy, including search terms and databases used.
 Methods: design, analytic technique, sources for effectiveness  Selection procedures and criteria.
evaluation, cost calculation methods, time horizon, sensitivity analysis,  Quality assessment tools and procedures.
discount rate.
 Data extraction strategy.
 Results: incremental costs, incremental effects, incremental cost-
effectiveness/cost-utility ratio, sensitivity, additional results. 9.4.1.2 Results
 Conclusions.  Flow diagram.
9.2 Introduction  Evidence tables.
Information about the illness or health problem:  Synthesis of the extracted evidence.
 Disease area (pathology/problem). Data need to be accompanied by relevant measures of variability.
 Epidemiology (incidence and prevalence, in absolute and relative 9.4.1.3 Discussion and Conclusions of the clinical literature review
figures (e.g. per 100 000 inhabitants).
9.4.2 Economic literature review
 Natural evolution of the illness, morbidity and mortality.
 Current clinical practice. 9.4.2.1 Methods
9.3 Objectives  Review questions.
 Study intervention: therapeutic group, product name (+ generic name)  Search strategy, including search terms and databases use.
and galenic type if applicable, route of administration, treatment plan,  Selection procedures and criteria.
approved indications.  Quality assessment tools and procedures.
 Comparator (describe treatment and options if treatment fails) +  Data extraction strategy.
justification in a Belgian context.
54 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

9.4.2.2 Results 9.5.6 Time horizon and discount rate

 Flow diagram.  Choice of, and rationale for, the time horizon and the discount ratefor
the analysis.
 Evidence tables.
 Reasons for an extension of the analytical horizon in relation to the
 Synthesis of the extracted evidence. primary data (e.g. from clinical trials).
Data need to be accompanied by relevant measures of variability.
9.5.7 Sensitivity analysis
9.4.2.3 Discussion and Conclusions of the economic literature
 Parameters on which a sensitivity analysis is performed.
review
 Distributions used for uncertain parameters.
Data extraction sheets are provided in Appendix 3.
 Sources for distributions.
9.5 Basic elements of the economic evaluation 9.6 Research Methods
9.5.1 Analytic technique
9.6.1 Identification, measurement and valuation of costs
 Analytic technique used (CEA or CUA) + reasons for this choice.
 Which cost items were taken into account and why.
9.5.2 Study design  What natural units were used to express the selected cost items before
 Study design used (Trial-based pharmacoeconomic evaluation or they were converted into monetary units.
model) + justification for this design.  Sources consulted for the measurement of resource use.
 If modelling is used, describe the model’s structure, including the  If a number of data elements were difficult to measure, show how the
assumptions used. problem was solved.
9.5.3 Methods used for valuation of costs  Provide a table with quantities of resource use per cost item and unit
 Methods used for the identification, measurement and valuation of costs attached to the items.
costs.  The cost calculation must be reproducible.
 Methods used to validate the data, documentation on the quality control 9.6.2 Identification, measurement and valuation of health-related
of the data. outcomes
9.5.4 Methods used for outcome assessment  Which health-related outcomes were, or were not taken into
 Methods used for the measurement and valuation of outcomes. consideration and why (e.g. side effects, morbidity, mortality).
 Methods used to validate the data, documentation on the quality control  Summary of the assumptions made regarding the identification,
of the data. measurement and valuation of health outcomes.
 Possible differences in effectiveness between patient subgroups.
9.5.5 Method of analysis of the data
 Possible differences between the efficacy measured on the one hand
 Statistical analysis techniques, handling missing data, statistical and the effectiveness on the other.
techniques for the sensitivity analysis.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 55

 Methods used to described health status (instruments used). 9.7.2 Uncertainty analysis
 Methods used to measure health-related quality of life.  Present 95% confidence or credibility interval around the incremental
costs, the incremental effects and the incremental cost-effectiveness
9.7 Results ratio.
9.7.1 Basic results  Present cost-effectiveness or cost-utility plane.
 Undiscounted life expectancy should be presented for both the  Cost-effectiveness acceptability curve (see Appendix 5).
interventional and comparator group. This allows to check how realistic
results are. In cost-utility analyses, the outcomes both with and without 9.8 Discussion
QoL adjustments should be presented.  Weaknesses of the study.
 Results should be presented in a tabular form. Undiscounted outcomes  Comparison with other studies, if available .
should be presented separately for the study intervention and the
comparator(s). Point estimates subject to variability should always be 9.9 Conclusion
presented with relevant measures of this variability (e.g. 95%
9.10 Transparency of financial support
confidence or credibility intervals). The table should therefore contain
incremental costs and incremental outcomes with the 95% confidence  Disclose financing and contractual arrangements. Declaration of
or credibility interval. Incremental cost-effectiveness ratios should be interests.
presented if the treatment is not dominant (lower costs and better  Autonomy and publication rights of the researchers.
effectiveness) or dominated (higher costs and lower effectiveness).
9.11 References
Table 13 – Presentation of the results of an economic evaluation:
template
9.12 Addenda
 Detailed data tables.
Reference case Mean Lower limit of Upper limit of
the 95% CrI the 95% CrI  Interim results.
 Work sheets and registration forms used for data collection,
Incremental costs questionnaires, measuring tools, etc.
Incremental effects  A detailed description of the measuring tools, data and analysis.
*
ICER (€/LYG or €/QALY gained)
* If relevant, i.e. if the intervention is not dominated or dominant. In such cases,
incremental costs and incremental effects should still be reported separately. CrI:
credibility interval.
56 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

The contribution of each uncertain parameter to the uncertainty in the ICER

10 PRESENTATION OF A MODEL can also be presented in case of probabilistic sensitivity analysis.
10.1 Data If there are additional sources of uncertainty, e.g. regarding the model
The data used in a model should be presented in tabular form, with structure, source of input data, assumptions, separate analyses can be
references, as presented in Table 11 above. presented.
Distributions of modelling input variables should be presented with the A Tornado diagram should be presented to highlight the modelling
relevant parameters of their distribution, e.g. Beta distributions should be parameters with the largest impact on the results. Uncertainty around the
presented with alpha1 and alpha2. incremental costs (IC), incremental effects (IE) and ICERs in scenario
analyses can be presented in an analogous table format as the reference
Continuous variables should be characterized by their mean and 95% case.
confidence interval. Measures of precision should be presented. Uncertainty
around input parameters and distributions for (probabilistic) sensitivity
analyses should be presented.
For each health state used in a Markov model, the nature of the health state
should be specified (temporary, absorbing). The choice of the health states
(and the omission to avoid complexity) should be justified. Transition
probabilities should be presented, e.g. in a matrix form. It should be
indicated whether a transition probability is constant or variable. The choice
of the cycle length should be justified.
10.2 Results
10.2.1 Reference case analysis
See 9.7.1.
10.2.2 Uncertainty analysis
Parameter uncertainty should be examined using probabilistic sensitivity
analysis. Cost-effectiveness estimates should be presented on a cost-
effectiveness plane and cost-effectiveness acceptability curve. The cost-
effectiveness plane, with the results of the Monte Carlo simulations or
bootstrapping, should always be presented for the cost-per-QALY gained
and/or for the cost-per-LY gained. In addition, if simulations are spread over
different quadrants of the cost-effectiveness plane, the percentage of
simulations in each quadrant should be reported. A cost-effectiveness
acceptability curve should be presented in order to show the probability that
the treatment is cost-effective, given varying theoretical threshold values for
the cost-effectiveness ratio.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 57

11 METHODOLOGICAL REFERENCES BY 11.3 Literature review

Rothwell P, External validity of randomised controlled trials: “To whom do
TOPIC the results of this trial apply?” Lancet. 2005; 365: 82-93.
11.1 Country-specific guidelines 11.4 Study design
CVZ. Richtlijnen voor farmaco-economisch onderzoek, geactualiseerde
versie. 2006 (http://www.cvz.nl). 11.4.1 Trial-based economic evaluations
CES. French Guidelines for the Economic Evaluation of Health Care O’Sullivan AK et al. Collection of health economic data alongside clinical
Technologies. Sept. 2004. (http://www.ces-asso.org/Pages/defaut_fr.htm). trials: is there a future for piggyback evaluations ? Value in Health. 2005;
8(1): 67-79.
HAS. Choix méthodologiques pour l’évaluation économique à la HAS. Oct
2011 (http://www.has-sante.fr/portail/jcms/c_1120711/choix-methodologiq Evans C et al. Data collection methods in prospective economic evaluations:
ues-pour-l-evaluation-economique-a-la-has). how accurate are the results ? Value in Health. 2000; 3(4): 277-286.
Australian Government – Department of Health and Ageing. Guidelines for Butler NA, Schapira MM, Warren JL, Earle CC. Methodological Issues in the
the Pharmaceutical Industry on Preparation of Submissions to the Use of Administrative Claims Data to Study Surveillance After Cancer
Pharmaceutical Benefits Advisory Committee (PBAC). Sept. 2004 Treatment. Medical Care. 2002; 40(8): Supp. IV 69-74.
(www.health.gov.au/internet/wcms/publishing.nsf). 11.4.2 Modelling
NICE. Guide to the Methods of Technology Appraisal. 2008 Barton P, Bryan S, Robinson S. Modelling in the economic evaluation of
(http://www.nice.org.uk/aboutnice/howwework/devnicetech/guidetothemeth health care: selecting the appropriate approach. Journal of Health Services
odsoftechnologyappraisal.jsp) (Guide currently under review). Research & Policy. 2004;9(2):110-118.
(Updated Guide to the Methods of Technology Appraisal (status of “draft for Weinstein MC, O'Brien B, Hornberger J, Jackson J, Johannesson M,
consultation” in June 2008): http://www.nice.org.uk/aboutnice/ McCabe C, et al. Principles of good practice for decision analytic modelling
howwework/devnicetech/technologyappraisalprocessguides/GuideToMeth in health-care evaluation: report of the ISPOR Task Force on Good
odsTA201112.jsp?domedia=1&mid=B52851A3-19B9-E0B5-D48284D172B Research Practices--Modelling Studies. Value Health. 2003;6(1):9-17.
D8459.
Drummond M, Sculpher M. Common Methodological Flaws in Economic
11.2 Methods for economic evaluations Evaluations. Med Care. 2005; 43(7): II-5-II-14.
Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-effectiveness in Health
and Medicine. New York: Oxford University Press; 1996.
Drummond MF, Sculpher MJ, Torrance GW, O’Brien B, Stoddart GL.
Methods for the Economic Evaluation of Health Care Programmes. 3rd
edition. Oxford: Oxford University Press, 2005.
Gray A, Clarke P, Wolstenholme J, Wordsworth S. Applied Methods of Cost
effectiveness Analysis in Healthcare. Handbooks in Health Economic
Evaluation. Oxford: Oxford University Press; 2010.
58 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

11.5 Calculation of costs 11.6.2 Confidence interval around the ICER

Chaudhary MA, Stearns SC. Estimating confidence intervals for cost-
11.5.1 Measurement of resource use
effectiveness ratios: an example from a randomized trial. Stat Med.
Van De Sande S, De Wachter D, Swartenbroekx N, Peers J, Debruyne H, 1996;15(13):1447-58.
Moldenaers I, et al. Inventaire des bases de données de soins de santé.
Briggs AH, Mooney CZ, Wonderling DE. Constructing confidence intervals
Objective Elements - Communication (OEC). Bruxelles: Centre Fédéral
for cost-effectiveness ratios: an evaluation of parametric and non-parametric
d'Expertise des Soins de Santé (KCE); 2006 19/05/2006. KCE Reports 30B
techniques using Monte Carlo simulation. Stat Med. 1999;18(23):3245-62.
(D2006/10.273/15). Available from:
https://kce.fgov.be/fr/publication/report/inventaire-des-bases-de-données- O'Brien B, J., Briggs AH. Analysis of uncertainty in health care cost-
de-soins-de-santé. effectiveness studies: an introduction to statistical issues and methods. Stat
Methods Med Res. 2002;11:455-468.
11.5.2 Valuation of resource use
Hakkaart- van Roijen L, Tan SS, Bouwmans CAM. Handleiding voor 11.7 Indirect Comparisons
kostenonderzoek - Methoden en standaard kostprijzen voor economische Song F, Altman DG, Glenny AM, Deeks JJ. Validity of indirect comparison
evaluaties in de gezondheidszorg. College voor Zorgverzekeringen. 2010. for estimating efficacy of competing interventions: empirical evidence from
Institute for Quality and Efficiency in Health Care (IQWiG). Working Paper published meta-analyses. BMJ. 2003;326(7387):472.
Cost Estimation. Version 1.0 – 19/11/2009. Glenny AM, Altman DG, Song F, Sakarovitch C, Deeks JJ, D'Amico R, et al.
Pharmaceutical Benefits Advisory Committee. Manual of resource items and Indirect comparisons of competing interventions. Health Technology
their associated costs. Commonwealth of Australia 2002. Assessment. 2005;9(26).
Durant Guy. Le financement des hôpitaux en Belgique. Situation au 1er Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and
septembre 2011. Waterloo : Wolters Kluwer Belgium SA, 2011. indirect treatment comparisons in meta-analysis of randomized controlled
trials. J Clin Epidemiol. 1997;50(6):683-91.
11.6 Handling Uncertainty Lim E, Ali Z, Ali A, Routledge T, Edmonds L, Altman DG, et al. Indirect
11.6.1 Overview comparison meta-analysis of aspirin therapy after coronary surgery. BMJ.
2003;327(7427):1309.
Briggs AH, Gray AM. Handling uncertainty when performing economic
Thompson SG, Higgins JP. How should meta-regression analyses be
evaluation of healthcare interventions. Health Technology Assessment.
undertaken and interpreted? Stat Med. 2002;21(11):1559-73.
1999;3(2). Available from http://www.ncchta.org/execsumm/summ302.htm.
Baker SG, Kramer BS. The transitive fallacy for randomized trials: if A bests
Stinnett AA, Mullahy J. Net Health Benefits. A new framework for the
B and B bests C in separate trials, is A better than C? BMC Med Res
analysis of uncertainty in cost-effectiveness analysis. Med Decis Making.
Methodol. 2002;2:13.
1998; 18(2):S68-S80.
Institute for Quality and Efficiency in Health Care (IQWIG). General
Briggs A, Sculpher M, Claxton K. Decision modelling for health economic
Methods. Version 4.0 of 23.09.2011. Germany. Available from
evaluation. Oxford: Oxford University Press; 2006.
https://www.iqwig.de.
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 59

11.8 Discounting
Brouwer WB, Niessen LW, Postma MJ, Rutten FF. Need for differential
discounting of costs and health effects in cost effectiveness analyses. BMJ.
2005;331(7514):446-8.
Claxton K, Sculpher M, Culyer A, McCabe C, Briggs A, Akehurst R, Buxton
M and Brazier J. Discounting and cost-effectiveness in NICE: stepping back
to sort out a confusion. Health Econ. 2006; 15: 1–4.
Bos JM, Postma MJ, Annemans L. Discounting health effects in
pharmacoeconomic evaluations : current controversies.
Pharmacoeconomics. 2005;23(7):639-49.
Gravelle H., Smith D.. Discounting for Health Effects in Cost-benefit and
Cost-effectiveness Analysis. Health Econ. 2001: 587-599.
11.9 Budget impact analyses
Orlewska E, Mierzejewski P. Polish guidelines for conducting
pharmacoeconomic evaluations, (Year of publication not mentioned).
Orlewska E, Mierzejewski P. Proposal of Polish guidelines for conducting
financial analysis and their comparison to existing guidance on budget
impact in other countries. Value Health. 2004;7(1):1-10.
Mauskopf JA, Sullivan SD, Annemans L, Caro J, Mullins CD, Nuijten M, et
al. Principles of good practice for budget impact analysis: report of the
ISPOR Task Force on good research practices--budget impact analysis.
Value Health. 2007;10(5):336-47.
60 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

 APPENDICES APPENDIX 1. CLASSIFICATION OF

ECONOMIC STUDIES
Figure 1 – Classification of economic studies
Are both costs (inputs) and consequences (outputs)
of the alternatives examined?

No

Consequences Costs only Yes

only

Partial evaluation Partial evaluation

Is there a comparison of at
least two alternatives?
No Outcome Cost Cost-consequence analysis
description description

Partial evaluation Full economic evaluation

Efficacy or Cost Cost-utility analysis (CUA)

Yes effectiveness comparison Cost-benefit analysis (CBA)

evaluation
Cost-effectiveness analysis (CEA)

Cost-minimisation analysis (CMA)

Adapted from Drummond M et al.31

KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 61

APPENDIX 2. CHECKLIST FOR APPENDIX 3. DATA EXTRACTION SHEET

ASSESSING ECONOMIC EVALUATIONS FOR ECONOMIC EVALUATIONS
This section is adapted from Drummond et al.32 It should be noted that for
some questions in this checklist, the assessor will have to make value Table 14 – Data extraction sheet for economic evaluations: template
judgments. Applying this checklist will therefore not preclude critical Reference
appraisal of the study quality by the researcher.
Sponsor(s) of the study
 Is there a well defined question?
Country, currency, price year
 Is there comprehensive description of alternatives?
Research question
 Are all important and relevant costs and outcomes for each alternative
identified? Analytic technique
 Has clinical effectiveness been established? Study Design
 Are costs and outcomes measured accurately? Perspective
 Are costs and outcomes valued credibly? Time horizon
 Are costs and outcomes adjusted for differential timing? Interventions compared
 Is there an incremental analysis of costs and consequences? Population
 Were sensitivity analyses conducted to investigate uncertainty in Assumptions
estimates of cost or consequences? Data sources for costs
 How far do study results include all issues of concern to users? Data sources for outcomes
 Are the results generalisable to the setting of interest in the review? Cost items included
Outcomes parameter
Discounting (Yes/No + rate)
Results:
Costs
Outcomes
Cost-effectiveness
Sensitivity analysis
Conclusions
Remarks Specify weaknesses of the
study
62 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

APPENDIX 4. TECHNICAL NOTES Appendix 4.1.3. Characteristics of randomised patients

This section presents some technical aspects of the methods that have been  Baseline clinical characteristics
mentioned in the guidelines.  Racial group
Appendix 4.1. Assessment of external validity  Uniformity of underlying pathology
This list of issues for external validity has been derived from a paper  Stage in the natural history of their disease
published by Rothwell.8  Severity of disease
Appendix 4.1.1. Setting of the trial  Comorbidity
 Healthcare system  Absolute risks of a poor outcome in the control group
 Country Appendix 4.1.4. Differences between the trial protocol and routine
 Recruitment from primary, secondary or tertiary care practice
 Selection of participating centres  Trial intervention
 Selection of participating clinicians  Timing of treatment
Appendix 4.1.2. Selection of patients  Appropriateness/relevance of control intervention
 Methods of prerandomisation diagnosis and investigation  Adequacy of non-trial treatment – both intended and actual
 Eligibility criteria  Prohibition of certain non-trial treatments
 Exclusion criteria  Therapeutic or diagnostic advances since trial was done
 Placebo run-in period Appendix 4.1.5. Outcome measures and follow-up
 Treatment run-in period  Clinical relevance of surrogate outcomes
 Enrichment strategies  Clinical relevance, validity and reproducibility of complex scales
 Ratio of randomised patients to eligible non-randomised patients in  Effect of intervention on most relevant components of composite
participating centres outcomes
 Proportion of patients who declined randomisation  Who measured outcome
 Use of patient-centred outcomes
 Frequency of follow-up
 Adequacy of the length of follow-up
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 63

Appendix 4.1.6. Adverse effects of treatment years gained. However, apart from the EQ-5D, HUI 2/3 and the SF-6D, there
are very few profile measures for health-related quality of life that can be
 Completeness of reporting of relevant adverse effects
translated into such a utility index.
 Rates of discontinuation of treatment
The values for health-related quality of life attached to different health states
 Selection of trial centres and/or clinicians on the basis of skill or can be derived from patients, the general public, health care providers or
experience family. Including patients’ preferences in the outcome assessment seems
 Exclusion of patients at risk of complications the most logical approach. However, some caveats should be kept in mind.
If patients are asked to value their health-related quality of life directly on a
 Exclusion of patients who experienced adverse effects during a run-in
visual analogue scale and these raw data are directly used for analysis,
period
there will be a problem of comparability and aggregation. The values of one
 Intensity of trial safety procedures patient are not necessarily comparable to the values of another patient,
which makes aggregation and calculation of means, medians and spread of
Appendix 4.2. Outcome valuation little relevance. For a wide application of the utility data and for reasons of
Appendix 4.2.1. Health-related quality of life comparability across patient groups, it is important to use public preferences
Outcomes can be expressed in physical units (life years gained) or in ‘utility’ values for health states described by patients (indirect method) (see
terms. The most frequently used utility outcomes are QALYs. For the Appendix 7).
valuation of the quality weights of life years gained, different methods and There are three major direct methods for measuring health state
instruments can be used. Different possibilities exist for the assessment of preferences: the time trade-off, the standard gamble and the visual analogue
health-related quality of life, but not all are useful for economic evaluations. scale. Each method has advantages and disadvantages. The time trade-off
There are disease specific and generic health-related quality-of-life risks to be biased by time preference of the respondents, the standard
measures, profile measures or single index measures, health-related quality gamble by the risk attitude of the respondents and the visual analogue scale
of life can be assessed by patients themselves or by health care providers by the definition of the endpoints.
or family and valuation of a health state can be done by means of a Time- In order to ensure that the patient’s perspective is represented, it is crucial
Trade-Off, Standard Gamble or Rating Scale. that the health states are first described by the patients, using a generic
Disease-specific quality-of-life measures are useful to get an insight into the descriptive system for health-related quality of life (e.g. the EQ-5D, the SF-
domains of life that are affected by a disease or treatment. They are 6D). The utilities corresponding to these descriptions should be derived from
considered to be more sensitive to small changes in health-related quality of preferences for health states expressed by the general public.
life in a specific disease than generic measures. However, from a societal
point of view, it is also necessary to include a generic outcome measure in
the analysis. Decisions about the reimbursement if an intervention involves
budget allocation decisions. Therefore it is useful to be able to compare
different budget allocations in terms of the incremental cost per QALY they
involve. Only with a generic utility outcome measure, broad comparisons
across diseases are possible.
Profile measures are less useful for economic evaluations unless they allow
translation into one single utility index that can then serve as a weight for life
64 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

Appendix 4.3. Use of expert panels o the approach used to present the variability in the opinions (range,
variance).
Use of expert panels should be avoided as much as possible. Sometimes,
however, insufficient empirical data are available to estimate variables It may be useful to ask each expert to explain the reasoning behind the
needed for the economic evaluation. This relates specifically to resource expert opinion offered.
use. Expert panels can help to predict which resources will be used and how The expert opinions should be summarised and the variability in opinions
often each will be used to manage outcomes reported but not followed-up in presented. It should be clearly indicated how the opinions have been used
clinical trials. in the economic evaluation and how is dealt with the uncertainty around the
If expert opinion is used in a submission, the need for expert opinion should expert opinions.
be justified. The methods used to obtain and collate the opinions should be
described in detail. The following elements should be addressed:
Appendix 4.4. Hospital per diem prices
The financing of non-medical hospital activities (i.e. capital expenditures for
 the criteria for selecting the experts,
housing and medico-technical facilities, hotel function, nursing care, etc.) is
 the number of experts approached, based on the reforms introduced in 2002. Since 2002, Belgian hospitals
 the number and identity of experts who participated, receive an annual budget (so called “budget of financial means”) which
covers their activities for the period July to June the 30th of the next year.
 whether a declaration of potential conflict(s) of interest was sought from
The budget is adapted on January, it is computed by the FPS public health
all experts or medical specialty groups whose opinions were sought,
and its amount is different for each hospital. The budget is composed of
 whether the participants were blinded to the purpose of the study, three major parts (A-Capital costs, B-Operational costs and C-Corrective
 whether the experts were remunerated for their participation and how, measures), which are further divided into subparts. The share of each (sub-
)part in the total budget is given in parentheses in the table below.
 the background information provided and its consistency with the totality
of the evidence provided in the submission,
 the detailed method that was followed to collect the opinions,
 the medium used to collect the opinions (direct interview, telephone
interview or self-administered questionnaire…),
 the questions asked (with a copy of the questionnaire or an outline of
the interview),
 whether iteration was used in the collation of opinions and, if so, how it
was used,
 the number of responses received for each question,
 whether all experts agreed with each response, and, if not:
o the approach used to finalize the estimates. For example, a Delphi
technique could be applied; or the majority opinion, the median, or
the mean could be presented,
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 65

Table 15 – Components of the hospital budget of financial means How is this budget paid to the hospitals? The payment of the budget of
financial means to a hospital contains two parts: a fixed part and a variable
Categories Description part. The fixed part is paid by the sickness funds on the basis of monthly
advances (the so-called provisional twelfths). This part includes
A (8%) A1 Investment charges
(theoretically) 80% of subparts B1 and B2, and 100% of all other parts. The
A2 Short-term credit burdens variable part, including 20% of subparts B1 and B2, is paid, via an invoice,
A3 Investment charges for some medico-technical services according to the number of admissions (10% of the budget) and the number
which are exclusively financed via the hospital budget (not of nursing days (10% of the budget) for the general hospitals, and
via fees) exclusively according to the number of days (20% of the budget) for the other
hospitals. The invoice is submitted by the hospitals to the sickness funds for
B B1 Common operational costs (administration, maintenance,
all patients enrolled in a sickness fund.dd Specific RIZIV–INAMI tariff codes
(>90%) laundry) (30%)
(from the nomenclatuur–nomenclature) have been created to this effect).
B2 Clinical costs (personnel and medical equipment) (40–50%) The amounts per admission and per nursing day are hospital-specific and
B3 Medico-technical departments (radiotherapy, MRI and PET also depend on the type of hospital stay (e.g. acute, burned, elderly,
scans) (1%) psychiatric, palliative and chronic disease care). They are adapted twice a
year.
B4 Some specific (mostly) lump sum costs (as a result of legal
obligations), e.g. hospital hygiene, quality assessment, Note that these codes are the only ones recorded in the MFG–RFM
palliative care and recording of hospital data administrative database. It is therefore not suitable to use this database for
economic evaluations since it only records the variable part (i.e. 20%) of the
B5 Pharmacy costs (2%) hospital stay costs while the fixed part (80%) of this budget is not recorded
B6 Costs for carrying out the social agreements for personnel in this database.
not included in the budget of financial means (2%) The amounts per admission and per nursing day are published as excel files
B7 Extra costs for teaching hospitals or university function of the on the RIZIV–INAMI website, together with the 100% per diem prices (in
hospital (applied scientific research, the development of new which admission and nursing day amounts are imputed).ee The per diem
technologies and the training of specialists) (3%) prices are reported per hospital and per type of hospital stay.
B8 Specific costs for patients with a weaker socioeconomic
profile or social function of the hospital (0.5%)
B9 Extra-legal financial benefits
C Corrective measures

dd ee http://www.riziv.fgov.be/care/fr/hospitals/specific-information/prices-
For patients not enrolled in a sickness fund, the invoice (with the full day-price
covering categories A, B and C) is sent to the paying authorities (i.e. day/index.htm
OCMW/CPAS, a private health insurance or a work accident insurance).
66 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

Based on this list of 100% prices, it could be tempting then to compute a Although indirect comparison can arise in different contexts and can have
mathematical average across all hospitals to derive Belgian average per different purposes, the statistical options are the same whichever scenario
diem prices per type of hospital stay. This method however does not account applies. The simplest case is when results of 2 RCTs are available, RCT 1
for the volume effect of each hospital such that big hospitals with higher comparing treatment B with treatment A (B vs. A) and RCT 2 comparing
hospitalization days and per diem prices have the same weight as smaller treatment C versus treatment A (C vs. A), and the purpose is to compare B
hospitals with lower per diem prices. The simple mathematical average of and C (B vs. C), indirectly. Different statistical methods have recently been
the 100% per diem should therefore not be used. proposed for this purpose, and there is still a lot of research performed on
Rather the weighted average 100% per diem price that accounts for this topic. Glenny et al.16 have done an excellent overview of the literature,
disparities in the case-mix (different levels of activities) of the hospitals with some additional research to compare the different methods. A summary
should be used. These weighted averages are to be found in section 5.7.3.5 of their findings follows, focusing on the main methods.
of these guidelines. Method 1: The naïve method (Unadjusted Comparison). In the naïve
method, results from treatment arms are simply compared between each
Appendix 4.5. Indirect comparisons other as if they would come from a single trial (so the results in the treatment
 Results from direct comparisons in RCTs are the preferred method to B arm are directly compared to the results in the treatment C arm), ignoring
estimate treatment effects. If no direct comparisons are available, the fact that studies are RCTs and discarding information from control arms
indirect comparisons from RCTs can be performed. (A arm). Based on theoretical and empirical evidence, Glenny et al conclude
 Results from the naïve approach, i.e. comparing simply the treatment that “the results of such analysis are completely untrustworthy, and naïve
arm of the RCTs as if they were one single trial, are completely comparisons should never be made”.
untrustworthy. The other methods are called “adjusted”, in the sense that the indirect
 Indirect comparisons should be based on “adjusted” methods, which comparison is adjusted by the results of their direct common control group
use the common control arm of RCTs as a way to “standardize” the within each RCT (treatment A), which is used as a way to “standardize” the
comparison. Different methods of increasing complexity are available. results of the treatment arms.
The randomized controlled trial (RCT) is the most valid design for evaluating Method 2: Adjusted Indirect Comparison. This method has been discussed
the relative efficacy of competing treatments. However, in many cases, there by Bucher et al,13 for the case of binary data, but it can be generalized to
is no trial available comparing directly the treatments, interventions or any kind of data (continuous, time to event..). First, from the 2 RCTs,
technologies of interest. A common example is within a class of several estimates for treatment effects and their standard error are known. These
drugs (A and B), each of which has been studied in placebo-controlled RCT treatment effects relate to the scale on which the data would be analyzed:
(often needed to get approval of the drug), but there are very few trials in means for continuous data, log odds ratio for binary data, log hazard ratio
which the drugs have been compared directly with each other. Another for time to event data… The effect B vs. C is then estimated by the difference
example is within the setting of an active-controlled trial, where the purpose between the effects observed in the 2 trials, and the variances are summed.
is to demonstrate that a new treatment (A) is equivalent (not better nor worse
by a certain amount) to a standard treatment (C), which itself has previously
been shown to be superior to a placebo. The active-controlled trial
comparing A versus C implicitly assumes, based on an indirect comparison,
that the new treatment A is better than a placebo (i.e. is effective).
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 67

Other methods of increasing complexity exist (meta-regression methods,

generalized linear model, Bayesian methods) and are described in the HTA
APPENDIX 5. THE COST-EFFECTIVENESS
review.16 PLANE, COST-EFFECTIVENESS
Main Assumptions. The key assumption of the indirect comparison using the ACCEPTABILITY CURVE, AND EFFICIENCY
results of trials A vs. B and A vs. C is that there should be no important
difference between the 2 sets of trials with respect to aspects that could FRONTIER
influence (bias) the estimated treatment effect of B vs. C. In other words, The cost-effectiveness plane is used to present the results of a cost-
there must be no confounding of the comparison by some trial effectiveness or cost-utility analysis. Incremental effects (life-years or
characteristics. Example of confounding is that when the treatment effect is quality-adjusted life years (QALYs)) are presented on the x-axis. The y-axis
influenced by some factors that itself varies across the different treatment shows the incremental costs. The centre of the figure shows the comparator.
comparisons, such as clinical setting or length of follow up. This situation As such, there are four quadrants comparing the intervention with the
has been illustrated graphically by Baker et al.33 comparator:
1. The intervention is more effective and more costly,
2. The intervention is more effective and less costly,
3. The intervention is less effective and less costly, and
4. The intervention is less effective and more costly than the comparator.
Interventions is the 2nd quadrant dominate the comparator, while those in
the 4th quadrant are being dominated by the comparator.
The cost-effectiveness acceptability (CEA-)curve presents the probability
that a given intervention is considered cost-effective on the basis of the
value assigned to an additional life year or QALY. The shape of this curve
depends on the location and proportion of incremental costs and effects over
the four quadrants of the cost-effectiveness plane. In Figure 2, two examples
are shown. For more information we refer to Fenwick et al.34
68 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

Figure 2 – The cost-effectiveness plane and cost-effectiveness acceptability curve

CE plane (example 1) CE plane (example 2)

IC IC
120.000 € 120.000 €
100.000 € 100.000 €
80.000 € 80.000 €
60.000 € 60.000 €
40.000 € 40.000 €
20.000 € 20.000 €
0€ IE 0€ IE
-15 -10 -5 -20.000 € 0 5 10 15 -15 -10 -5 -20.000 € 0 5 10 15

-40.000 € -40.000 €
-60.000 € -60.000 €
-80.000 € -80.000 €
-100.000 € -100.000 €
-120.000 € -120.000 €

CEA curve (example 1) CEA curve (example 2)

1 1

Probability intervention is cost-effective

Probability intervention is cost-effective

0,9 0,9
0,8 0,8
0,7 0,7
0,6 0,6
0,5 0,5
0,4 0,4
0,3 0,3
0,2 0,2
0,1 0,1
0 0
0€ 5.000 € 10.000 € 15.000 € 20.000 € 0€ 20.000 € 40.000 € 60.000 € 80.000 € 100.000 €
Value of health effect (λ) Value of health effect (λ)

CE: cost-effectiveness; IC: incremental cost; IE: incremental effect

KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 69

The efficiency frontier is the curve on the cost-effectiveness plane formed dominance, i.e. there is a linear combination of other strategies (D and F)
by the incremental cost-effectiveness or cost-utility ratios of the non- that can produce the same (or greater) benefit at lower (or the same) cost.
dominated comparators. As such, the comparator of intervention F becomes alternative D. Working
Figure 3 provides an example. In this example, interventions A, B, C, D, E on the efficiency frontier can have a large impact on results, conclusions and
and F are all alternative interventions for a specific population with a certain policy recommendations.10 In this deterministic example, F would have an
disease. Intervention B (quadrant IV) is dominated by intervention A. ICER of €10 000 per QALY if compared with (the non-efficient) alternative
Intervention A is dominated by C (quadrant II). Therefore, C becomes a A. However, taking into account alternative D, this becomes €40 000 per
justified comparator for intervention D. E is excluded due to extended QALY.

Figure 3 – The efficiency frontier

CE plane (efficiency frontier, alternative 1) CE plane (efficiency frontier, alternative 2)

IC IC
F F
120.000 € E 160.000 € E
100.000 € 140.000 €
80.000 € 120.000 €
B 60.000 € B 100.000 €
40.000 € D 80.000 € D
20.000 € 60.000 €
A IE A
0€ 40.000 €
-15 -10 -5 -20.000 € 0 5 10 15 20.000 €
-40.000 € C 0€ C IE
-60.000 € -15 -10 -5 -20.000 € 0 5 10 15
-80.000 € -40.000 €
-100.000 € -60.000 €
-120.000 € -80.000 €

CE: cost-effectiveness; IC: incremental cost; IE: incremental effect.

Alternative 1 and 2 present the same interventions and ICERs. The only difference is that in alternative 2, the first non-dominated intervention is put in the centre of the cost-
effectiveness plane
70 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

APPENDIX 6. LIST OF BELGIAN DATABASES FOR THE MEASUREMENT AND VALUATION

OF RESOURCE USE
Table 16 – Belgian databases for the measurement and valuation of resource use
Owner Database and Information collected Start Access

FEDERAL LEVEL
Cellule Technique pour la Aggregated data (coupled MKG and MFG data). Classification according 1996 https://tct.fgov.be/etct/index.html
gestion des données RCM-RFM to the ICD9-CM and INAMI–RIZIV nomenclature codes: Public access
Technische cel voor het beheer  Mean length of hospital stay per APR-DRG / Severity of illness NB: Costs from the TCT do not account
van de MKG-MFG data  Distributional parameters per APR-DRG / Severity of illness for the extrapolation in lump sums for
hospital drugs, medical imaging and
 Average cost per hospital stay per APR-DRG / Severity of illness
laboratory testing. They may
nonetheless be used as such.
Bases de données Agence Individual data on health care consumption and reimbursement (health 2002 http://www.nic-ima.be/
Intermutualiste (AIM) insurance and patients’ share) for al insured patients. Both ambulatory No public access
Databanken Intermutualistisch and hospitalized patients.
Agentschap (IMA–AIM)
INAMI–RIZIV Unit prices / costs for ambulatory and hospital health care services - http://www.riziv.be/care/fr/nomenclature
/index.htm
Search engine ‘NomenSoft’, available at: https://www.riziv.fgov.be/webprd/appl/p
nomen/Search.aspx?lg=N
INAMI–RIZIV Unit prices for reimbursed drugs (reimbursement basis and the patients’ - http://www.riziv.fgov.be/drug/fr/index.ht
share including any supplement due to the reference pricing system). m
INAMI–RIZIV Farmanet–Pharmanet: consumption of reimbursed drugs delivered to non- 2004 No public access
hospitalised patients
INAMI–RIZIV Prices (reimbursement basis and patients’ share) of reimbursed implants - https://www.riziv.fgov.be/insurer/fr/rate/i
and invasive medical devices ndex.htm (Articles 28, 35 and 35bis of
the RIZIV–INAMI nomenclature)
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 71

Owner Database and Information collected Start Access

INAMI–RIZIV Per diem prices for Belgian hospitals: per hospital and per type of hospital - http://www.riziv.fgov.be/care/nl/hospital
stay (acute, burns, geriatrics, palliative, psychiatric and specialized stays). s/specific-information/prices-
Remark: the weighted averages are reported in this guideline. day/index.htm

INAMI–RIZIV List of pharmaceuticals excluded from the hospital lump sum and - http://www.riziv.fgov.be/care/fr/hospitals
reimbursed by a retrospective fee-for-service system. /specific-
information/forfaitarisation/index.htm
INAMI–RIZIV Fee-for-service charges per laboratory test (25% and 100% of the - https://www.riziv.fgov.be/insurer/fr/rate/i
ndex.htm
honorarium fee).
http://www.inami.fgov.be/care/fr/hospital
Laboratory tests lump sums per inpatient day and per hospital. s/specific-information/pseudo/index.htm
https://www.riziv.fgov.be/insurer/fr/rate/i
Laboratory tests lump sums per admission.
ndex.htm
INAMI–RIZIV Fee-for-service charges per medical imaging act consultancy lump sums. - http://www.inami.fgov.be/insurer/fr/rate/i
ndex.htm
Medical imaging act lump sum per admission http://www.inami.fgov.be/care/fr/hospital
s/specific-information/pseudo/index.htm
BCFI–CBIP Unit prices for reimbursed and non-reimbursed (e.g. over the counter) - http://www.cbip.be/
drugs Public access
SPF Santé publique RCM–MKG: Health care consumption (INAMI–RIZIV nomenclature codes) 1995 No public access
FOD Volksgezondheid per general hospital stay. Classification according to the ICD-9-CM codes,

RIM–MVG: Nursing care consumption per general (non psychiatric) 1988 No public access
hospital stay
RPM–MPG: Health care consumption per psychiatric hospital stay. 1996 No public access
Classification according to the DSM IV and ICD-9-CM codes
ISP–WIV Sentinel general practitioners: Health care consumption in ambulant 1979 https://www.wiv-isp.be/
sector Public access
72 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

Owner Database and Information collected Start Access

COMMUNITY LEVEL

Registre du cancer Aggregated data on specialized health care consumption for hospitalized 1997 http://www.kankerregister.org/
Kanker register cancer patients. Coverage: Flanders Public access
Kind & Gezin IKAROS (geIntegreerd Kind Activiteiten Regio OndersteuningsSysteem): 1999 http://www.kindengezin.be
Health care consumption (ambulant and hospitalized) for children up to 3 Public access
years. Coverage: Flanders
ONE BDMS (Banque de données medico-sociales): Health care consumption 1994 http://www.one.be/index.php?id=banqu
(ambulant and hospitalized) for children up to 3 years and for pregnant e-de-donnees-medico-sociale
women. Coverage: Wallonia Public access
Department of general practice, INTEGO (Integrated computerized network). Health care consumption in 1994 http://www.intego.be/
KU Leuven first line treatment (GPs) and ambulant care. Coverage: Flanders
Note: a thorough description of most of the databases presented here can be found in Van de Sande et al.35
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 73

APPENDIX 7. FLEMISH EQ-5D INDEX VALUES

Table 17 – Flemish EQ-5D index values
State Score State Score State Score State Score State Score State Score State Score State Score State Score
11111 1.0000 12113 0.3020 13122 0.2391 21131 0.3497 22133 0.0602 23222 0.1339 31231 0.2444 32231 0.1618 33233 -0.1277
11112 0.7444 12121 0.6815 13123 0.1357 21132 0.2463 22211 0.6599 23223 0.0305 31232 0.1410 32232 0.0584 33311 0.2157
11113 0.3847 12122 0.5781 13131 0.2588 21133 0.1429 22212 0.5565 23231 0.1536 31233 0.0376 32233 -0.0450 33312 0.1123
11121 0.7641 12123 0.2184 13132 0.1554 21211 0.7426 22213 0.1968 23232 0.0502 31311 0.3810 32311 0.2984 33313 0.0089
11122 0.6607 12131 0.3415 13133 0.0520 21212 0.6392 22221 0.5762 23233 -0.0532 31312 0.2776 32312 0.1950 33321 0.1320
11123 0.3010 12132 0.2381 13211 0.3954 21213 0.2795 22222 0.4728 23311 0.2902 31313 0.1742 32313 0.0916 33322 0.0286
11131 0.4241 12133 0.1347 13212 0.2920 21221 0.6589 22223 0.1131 23312 0.1868 31321 0.2974 32321 0.2147 33323 -0.0748
11132 0.3207 12211 0.7344 13213 0.1886 21222 0.5555 22231 0.2362 23313 0.0834 31322 0.1940 32322 0.1113 33331 0.0484
11133 0.2173 12212 0.6310 13221 0.3117 21223 0.1958 22232 0.1328 23321 0.2065 31323 0.0906 32323 0.0079 33332 -0.0550
11211 0.8170 12213 0.2713 13222 0.2083 21231 0.3189 22233 0.0294 23322 0.1031 31331 0.2137 32331 0.1310 33333 -0.1584
11212 0.7136 12221 0.6507 13223 0.1049 21232 0.2155 22311 0.3728 23323 -0.0003 31332 0.1103 32332 0.0276 Dead 0
11213 0.3539 12222 0.5473 13231 0.2280 21233 0.1121 22312 0.2694 23331 0.1228 31333 0.0069 32333 -0.0758 Uncon
-0.0163
11221 0.7333 12223 0.1876 13232 0.1246 21311 0.4555 22313 0.1660 23332 0.0194 32111 0.3599 33111 0.2773 scious
11222 0.6299 12231 0.3108 13233 0.0212 21312 0.3521 22321 0.2892 23333 -0.0840 32112 0.2565 33112 0.1739
11223 0.2702 12232 0.2073 13311 0.3646 21313 0.2487 22322 0.1858 31111 0.4426 32112 0.2565 33113 0.0705
11231 0.3934 12233 0.1039 13312 0.2612 21321 0.3718 22323 0.0824 31112 0.3392 32113 0.1531 33121 0.1936
11232 0.2900 12311 0.4473 13313 0.1578 21322 0.2684 22331 0.2055 31113 0.2358 32121 0.2762 33122 0.0902
11233 0.1866 12312 0.3439 13321 0.2810 21323 0.1650 22332 0.1021 31121 0.3589 32121 0.2762 33123 -0.0132
11311 0.5300 12313 0.2405 13322 0.1776 21331 0.2881 22333 -0.0013 31122 0.2555 32122 0.1728 33131 0.1099
11312 0.4266 12321 0.3636 13323 0.0742 21332 0.1847 23111 0.3517 31123 0.1521 32123 0.0694 33132 0.0065
11313 0.3232 12322 0.2602 13331 0.1973 21333 0.0813 23122 0.1646 31131 0.2752 32131 0.1926 33133 -0.0969
11321 0.4463 12323 0.1568 13332 0.0939 22111 0.6907 23123 0.0612 31132 0.1718 32132 0.0892 33211 0.2465
11322 0.3429 12331 0.2799 13333 -0.0095 22112 0.5873 23131 0.1844 31133 0.0684 32133 -0.0142 33212 0.1431
11323 0.2395 12332 0.1765 21111 0.7733 22113 0.2276 23132 0.0810 31211 0.4118 32211 0.3291 33213 0.0397
11331 0.3626 12333 0.0731 21112 0.6699 22121 0.6070 23133 -0.0224 31212 0.3084 32212 0.2257 33221 0.1628
11332 0.2592 13111 0.4262 21113 0.3102 22122 0.5036 23211 0.3209 31213 0.2050 32213 0.1223 33222 0.0594
11333 0.1558 13112 0.3228 21121 0.6897 22123 0.1439 23212 0.2175 31221 0.3281 32221 0.2455 33223 -0.0440
12111 0.7651 13113 0.2194 21122 0.5863 22131 0.2670 23213 0.1141 31222 0.2247 32222 0.1421 33231 0.0791 Source: Cleemput
36
et al. 2010.
12112 0.6617 13121 0.3425 21123 0.2266 22132 0.1636 23221 0.2373 31223 0.1213 32223 0.0387 33232 -0.0243
74 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

 REFERENCES 1. Van Wilder P, Dupont A. Introducing evidence based medicine (EBM)

in reimbursement procedures: does it affect the outcome? Value in
Health. 2008;11(4):784-7.
2. Canadian Agency for Drugs and Technologies in Health. Guidelines
for the economic evaluation of health technologies. Canada. Ottawa:
Canadian Agency for Drugs and Technologies in Health (CADTH);
2006. 3d ed
3. Cleemput I, Van Wilder P, Vrijens F, Huybrechts M, Ramaekers D.
Guidelines for Pharmacoeconomic Evaluations in Belgium. Health
Technology assessment (HTA). Brussels: Belgian Health Care
Knowledge Centre (KCE); 2008 10/04/2008. KCE Reports 78C
(D/2008/10.273/27) Available from:
https://kce.fgov.be/publication/report/guidelines-for-
pharmacoeconomic-evaluations-in-belgium
4. Lehman R, Loder E. Missing clinical trial data. BMJ. 2012;344:d8158.
5. Neyt M, Cleemput I, Thiry N, De Laet C. Calculating an intervention's
cost-effectiveness for the real-world target population: the potential of
combining strengths of both RCTs and observational data. Health
Policy. 2012;May 16. [Epub ahead of print].
6. Cleemput I, Neyt M, Thiry N, De Laet C, Leys M. Using threshold
values for cost per quality-adjusted life-year gained in healthcare
decisions. IJTAHC. 2011;27(1):71-6.
7. Cleemput I, Neyt M, Thiry N, De Laet C, Leys M. Threshold values for
cost-effectiveness in health care Health Technology Assessment
(HTA). Brussels: Belgian Health Care Knowledge Centre (KCE); 2008
14/01/2009. KCE Reports 100C (D/2008/10.273/96) Available from:
https://kce.fgov.be/publication/report/threshold-values-for-cost-
effectiveness-in-health-care
8. Rothwell P. External validity of randomised controlled trials: "to whom
do the results of this trial apply"? Lancet. 2005;365(9453):82-93.
9. le Polain M, Franken M, Koopmanschap M, Cleemput I. Drug
reimbursement systems: international comparison and policy
recommendations. Health Services Research (HSR). Brussels:
Belgian Health Care Knowledge Centre (KCE); 2010 2011-03-07 (2nd
KCE Report 183 Belgian guidelines for economic evaluations and budget impact analyses: second edition 75

print; 1st print: 2011-01-11). KCE Reports 147C (D/2010/10.273/90) of the ISPOR Task Force on good research practices--budget impact
Available from: https://kce.fgov.be/search/apachesolr_search/147 analysis. Value Health. 2007;10(5):336-47.
10. Neyt M, Van Brabandt H. The importance of the comparator in 22. Hill S, Garattini S, van Loenhout J, O’Brien B, de Joncheere K.
economic evaluations: working on the efficiency frontier. Technology Appraisal Programme of the National Institute for Clinical
Pharmacoeconomics. 2011;29(11):913-6. Excellence: A review by WHO. London: NICE; 2003.
11. Briggs A, Sculpher M, Claxton K. Decision modelling for health 23. Agency for Health Technology Assessment. Guidelines for
economic evaluation. Oxford: Oxford University Press; 2006. conducting Health Technology Assessment (HTA). Poland. Warsaw:
12. Goeree R, O’Brien B, Hunt R, et al. Economic evaluation of long-term 2009.
management strategies for erosive oesophagitis. 24. Orlewska E, Mierzejewski P. Proposal of Polish guidelines for
Pharmacoeconomics. 1999;16(6):679-97. conducting financial analysis and their comparison to existing
13. Bucher H, Guyatt G, Griffith L, Walter S. The results of direct and guidance on budget impact in other countries. Value Health.
indirect treatment comparisons in meta-analysis of randomized 2004;7(1):1-10.
controlled trials. J Clin Epidemiol. 1997;50(6):683-91. 25. Evans S. Blanket disapproval of all unlicensed indications is wrong.
14. Song F, Altman D, Glenny A, Deeks J. Validity of indirect comparison BMJ. 2011;343:d7726.
for estimating efficacy of competing interventions: empirical evidence 26. Szende Á, Mogyorósy Z, Muszbek N, Nagy J, Pallos G, Dózsa C.
from published meta-analyses. BMJ. 2003;326(7387):472. Methodological guidelines for conducting economic evaluation of
15. Lim E, Ali Z, Ali A, Routledge T, Edmonds L, Altman D, et al. Indirect healthcare interventions in Hungary: a Hungarian proposal for
comparison meta-analysis of aspirin therapy after coronary surgery. methodology standards. Eur J Health Econ. 2002;3:196-206.
BMJ. 2003;327(7427):1309. 27. Kristensen F, Sigmund H. Health Technology Assessment Handbook.
16. Glenny A, Altman D, Song F, Sakarovitch C, Deeks J, D'Amico R, et Copenhagen: Danish Centre for Health Technology Assessment,
al. Indirect comparisons of competing interventions. 2005. Health National Board of Health; 2007.
Technology Assessment 9(26). 28. Collège des Économistes de la Santé. French guidelines for the
17. Durant G. Le financement des hôpitaux en Belgique. Situation au 1er economic evaluation of health care technologies. Paris: 2004.
septembre 2011. Waterloo: Wolters Kluwer Belgium SA; 2011. 29. Pharmaceutical management Agency (PHARMAC). Guidelines for
18. Walker D, Hutubessy R, Beutels P. WHO Guide for standardisation of Funding Applications to PHARMAC (Version 2). New Zealand: 2009.
economic evaluations of immunization Programmes. Vaccine 30. Annemans L, Crott R, Degraeve D, Dubois D, Huybrechts M, Peys F,
2010;28:2356-9. et al. Recommended structure for reporting economic evaluation on
19. Drummond M, Sculpher M. Common Methodological Flaws in pharmaceuticals in Belgium. Pharm World Sci. 2002;24(1):5-7.
Economic Evaluations. Med Care. 2005;43(7):II-5-II-14. 31. Drummond M, O’Brien B, Stoddart G, Torrance G. Methods for the
20. Gray A, Clarke P, Wolstenholme J, Wordsworth S. Applied Methods economic evaluation of health care programmes. 3d ed. Oxford:
of Cost effectiveness Analysis in Healthcare. Handbooks in Health Oxford University Press; 2005.
Economic Evaluation. Oxford: Oxford University Press; 2010. 32. Drummond M, Jefferson T. Guidelines for authors and peer reviewers
21. Mauskopf JA, Sullivan SD, Annemans L, Caro J, Mullins CD, Nuijten of economic submissions to the BMJ. The BMJ Economic Evaluation
M, et al. Principles of good practice for budget impact analysis: report Working Party. BMJ. 1996;313(7052):275-8.
76 Belgian guidelines for economic evaluations and budget impact analyses: second edition KCE Report 183

33. Baker S, Kramer B. The transitive fallacy for randomized trials: if A

bests B and B bests C in separate trials, is A better than C? BMC Med
Res Methodol. 2002;2:13.
34. Fenwick E, O'Brien BJ, Briggs A. Cost-effectiveness acceptability
curves--facts, fallacies and frequently asked questions. Health
Economics. 2004 13(5):405-15.
35. Van De Sande S, De Wachter D, Swartenbroekx N, Peers J,
Debruyne H, Moldenaers I, et al. Inventaire des bases de données de
soins de santé. Objective Elements - Communication (OEC).
Bruxelles: Centre Fédéral d'Expertise des Soins de Santé (KCE);
2006 19/05/2006. KCE Reports 30B (D2006/10.273/15) Available
from: https://kce.fgov.be/fr/publication/report/inventaire-des-bases-
de-données-de-soins-de-santé.
36. Cleemput I. A social preference valuations set for EQ-5D health states
in Flanders, Belgium. Eur J Health Econ. 2010;11(2):205-13.
37. Koninklijk besluit van 21 december 2001 tot vaststelling van de
procedures, termijnen en voorwaarden inzake de tegemoetkoming
van de verplichte verzekering voor geneeskundige verzorging en
uitkeringen in de kosten van farmaceutische specialiteiten - Arrêté
royal du 21 décembre 2001 fixant les procédures, délais et conditions
concernant l'intervention de l'assurance obligatoire soins de santé et
indemnités dans le coût des spécialités pharmaceutiques, B.S. - M.B.
29/12/2001.
38. Institute for Quality and Efficiency in Health Care (IQWiG). "General
Methods - Version 4.0". (2011) Available from:
https://www.iqwig.de/download/General_Methods_4-0.pdf