418

testing laboratories can use this simplifi- elimination of the split German accredita- (GUM) [1] by ISO in 1993 started a revo-
cation of the accreditation system in the tion system. The efforts must, however, lution in measurement quality assurance.
area of technical goods transport for the be concentrated and concerted. The principles set out in this document
classification of materials, raw materials constitute a coherent and consistent sche-
and products or product certification. Ex- me for evaluating and processing the un-
amples of this are the testing of mineral certainty of measurement results, which
oil and fuels, material analysis of iron References has been missing for a long time. Among
and steel products, of non-iron metals others, this breakthrough is founded on
1. DIN EN 45001 (1989) General criteria
and of lacquers, paints and pigments. the uniform view of uncertainty as lack of
for the operation of testing laborato-
information, and quantification of uncer-
ries, Brussels
tainty by the standard deviation of proba-
2. ISO Guide 25 (1990) General require-
bility distributions expressing the availa-
ments for the competence of calibra-
Conclusion ble information.
tion and testing laboratories, 3rd edn.
Although the GUM methodology is,
Geneva
in principle, applicable to all types and
The accreditation and the confirmed 3. DIN EN 45004 (1995) General criteria
levels of measurement, it soon became
quality assurance of analytical work on a for the operation of different types of
apparent that for practical applications
high level is an important goal of every authorities carrying out inspections.
sectorial guidance documents were
laboratory. However, quality assurance Brussels
needed. The guide “Quantifying Uncer-
must not be done for its own sake. The 4. DIN EN 45011 (1989) General criteria
tainty in Analytical Measurement” [2]
number one goal of every laboratory for authorities certifying products.
published by EURACHEM in 1995 was
must be the “generation” of testing re- Brussels
intended to pave the way for implemen-
sults. An appropriate quality assurance
tation of the GUM methodology in
system must also make sure that the test-
(quantitative) chemical analysis.
ing results are comprehensible and under- Presented at: Analytica Conference 98,
The principles for evaluation and ex-
standable for the customer. In order to Symposium 1 : “Trends of accreditation
pression of measurement uncertainty in
guarantee this a certification of compe- and licensing of laboratories“ Munich,
chemical analysis are now well known, as
tence by independent third parties will be 21–24 April 1998
a result of the workshops organized by
necessary in the future.
EURACHEM in Graz 1994 and in Berlin
The German accreditation system W. Jäger (Y) 7 K. Gerlach
1997. In addition, these principles have
must be simplified and made more effec- Institut Dr. Jäger,
been described and explained in recent
tive in order to make it possible to sup- Ernst-Simon-Strasse 2–4
publications, e.g. by A Williams, Chair-
port both the quality of the analytical 72072 Tübingen, Germany
man of EURACHEM’s Uncertainty
work and the results. Several authorities Tel.: c49-7071-7007-0
Working Group [3], and need not be re-
and organizations are working on the FAX: c49-7071-7007-77
peated here. So far, procedures for uncer-
tainty evaluation have focused on the
step-by-step approach (also called bot-
tom-up approach), where the uncertainty
is calculated from a combination of var-
ious components, relating to building
POLICIES AND CONCEPTS blocks of the overall procedure or to sig-
nificant error sources. This approach re-
quires thorough investigation of the ana-
lytical procedure and proper assessment
Relevant accreditation policies and concepts in EU-, EFTA-, NAFTA- of component uncertainties. Its main
and other regions tools are uncertainty budgets and uncer-
tainty propagation, based on an appro-
priate mathematical model of the analyti-
Accred Qual Assur 3 : 418–422 guides published by ISO and EURA- cal process. Much less attention has been
Q Springer-Verlag 1998 CHEM, were reviewed, and the inventory paid so far to the direct approach (the
of procedures and tools for effective im- top-down approach), where the uncer-
Werner Hässelbarth plementation of these principles was ex- tainty is determined directly by investi-
amined. While the emphasis in the lec- gating the spread of results obtained on
ture was on basic principles, this article samples of known composition. This ap-
Measurement uncertainty presents a more detailed elaboration of proach requires appropriate reference
procedures revisited: one of the main practical procedures out-
lined in the lecture: direct determination
materials or reference analytical methods.
In addition the comparison measurements
Direct determination of uncertainty and handling of bias. have to be performed under appropriate
reproducibility conditions.
of uncertainty and bias Key words Uncertainty 7 Traceability 7 According to the step-by-step ap-
Validation 7 Bias handling proach the uncertainty of the results of
handling an analytical procedure is determined on
the basis of a detailed uncertainty budget
in a series of steps as follows:
Abstract This article is based on a lec- 1. Separate the analytical procedure into
ture presented at the Analytica Confer- Introduction well-designed steps, e.g. sample pre-
ence, where the basic principles for evalu- treatment, measurement and data eval-
ating and expressing the uncertainty of Publication of the “Guide to the Expres- uation.
analytical measurements, as set out in the sion of Uncertainty in Measurement”
 419

2. For each step identify and list all these steps are given in the EURA- As an obvious requirement, both ap-
sources of uncertainty, that is, all fac- CHEM Guide [2]. proaches, if properly executed, should
tors that may influence the perform- – Often the model (Eq. 1) will include lead to the same result, that is, there
ance of the step. factors 1Bu(1) and additive terms should be no significant difference be-
3. Make a preliminary estimate of the 0Bu(0) which do not contribute to the tween standard uncertainties determined
contribution of the identified uncer- value of y but do contribute to its un- directly and corresponding standard un-
tainty sources to the uncertainty of the certainty u(y). certainties calculated step-by-step from
target quantity, most often the content – Often the model (Eq. 1) will not be an uncertainty budgets. Therefore, as alrea-
of a specified analyte. algebraic equation but a computer al- dy proposed by Heydorn [6], the direct
4. Decide about which uncertainty contri- gorithm, and the sensitivity coefficients approach can (and should !) be used to
butions merit refined evaluation (sig- according to Eq. (2) will be calculated verify uncertainty budgets. If the actual
nificant contributions) and which can by numerical differentiation instead of variability (standard deviation) of repli-
be neglected (insignificant contribu- differential calculus. cate analytical results agrees with the pre-
tions). As an alternative to the step-by-step ap- dicted standard uncertainty, the uncer-
5. Taking into account the results of pre- proach, as outlined above, in the direct tainty budget is confirmed. Often, howev-
vious steps, design a mathematical approach the analytical procedure is ap- er, the predicted standard uncertainty will
model describing the target quantity of plied to appropriate reference samples, be significantly smaller than the experi-
the analytical procedure, y, as a func- and the results are compared with the mental standard deviation. Then the un-
tion of input parameters xi (ip1, 2, ..., reference values attributed to the refer- certainty budget has to be re-examined
N) associated with the significant un- ence samples. Alternatively, the proce- and corrected for identified errors. Ob-
certainty contributions, e.g. sample dure under investigation and an estab- vious possible errors are:
flow, pressure and temperature, recov- lished reference method are applied in – overlooking major uncertainty contri-
ery rates, measured instrument re- parallel to appropriate samples, and the butions
sponse, response function parameters, results are compared. This comparison – underestimation of major uncertainty
conversion factors and corrections. serves a double purpose: contributions.
(a) By comparing the mean of repeated Of course amendments of uncertainty
ypF(x1, x2, ..., xN). (1) measurements with the corresponding budgets have to be comprehensible. Com-
6. For each significant uncertainty source reference value, analytical bias is de- pletion of an uncertainty budget by an
evaluate the contribution ui(y) to the tected, if significant. In addition an empirical “recovery factor” is certainly
uncertainty of the target quantity, y, as appropriate bias correction is derived. not acceptable.
a standard uncertainty, that is, as a (b) From the spread of results of repeated
standard deviation, either of a series of measurements and the uncertainty of
repeated measurement results (type A the reference values the uncertainty of
the results of the analytical procedure, Direct determination of uncertainty
evaluation), or of a hypothetical distri-
bution expressing the available infor- including bias correction, is calculated.
mation about the respective quantity A more detailed description of this com- For determining the uncertainty of a can-
(type B evaluation). As a rule, ui(y) is parison and its evaluation is given in the didate analytical method, the method is
determined as a product of a sensitivi- next section. applied to appropriate calibration sam-
ty coefficient ci taken from the model, This approach can be applied either ples, and the results are compared with
and the standard uncertainty u(xi) of by an individual laboratory or in an inter- the reference values attributed to the cali-
the respective input quantity: laboratory study. In the first case it is es- bration samples. Alternatively, the candi-
sentially a calibration study. Then it is date method and a reference method, i.e.
ui (y)pciu (xi) where cipiF/ixi . (2) important to ensure that: a method whose uncertainty is known,
– the reference values and their uncer- are applied in parallel to appropriate
7. Consider possible correlations between tainties are well established samples, and the results of the candidate
different uncertainty contributions – the calibration samples are representa- method are compared with those of the
ui(y)and uj(y), determined before, e.g. tive for the range of samples to be ana- reference method. This comparison is
due to use of the same measurement lysed performed and evaluated in a series of
standard. If relevant, estimate the cor- – the variability of measuring conditions three steps as follows:
responding covariance uij(y): in the calibration study covers the var- 1. The measurement results and the cor-
iability in the intended applications. responding reference values are com-
uij(y)pcicj cov(xi, xj) (3) In the second case of an interlabora- pared, taking into account the relevant
8. Calculate the combined standard un- tory study the procedures described in uncertainties. Typically, the difference
certainty u(y) of the target quantity, y, the ISO 5725 series of International between corresponding values is tested
as the root of the sum of squares of Standards [4] should be followed. In this for significance against the expanded
the constituent standard uncertainties case the reproducibility standard devia- uncertainty on that difference, using
and their possible covariances: tion sR provides an estimate of the aver- an appropriate coverage factor.
age uncertainty of the analytical proce- 2. If significant bias has been found, a
dure under investigation. In applications bias correction for the specified meas-
u 2(y)p Au 2i (y)c2 A uij(y) . (4) of methods validated by interlaboratory uring range is derived. For a restricted
i i~j
studies, however, laboratories have to measuring range this can be a bias fac-
Remarks: consider additional individual uncertain- tor or an additive correction term,
– It is understood that a correction is ap- ties, see e.g. Ref. [5]. based on a single-level comparison.
plied to every identified source of sys- If neither suitable reference samples For extended measuring ranges correc-
tematic error, leaving as a contribution nor reference methods are available, the tion curves or correction functions are
to the overall uncertainty budget the comparison may be based on consensus required, based on multi-level compar-
uncertainty of the correction. values instead of reference values, as a isons.
– Generic steps of analytical procedures substitute but by no means as an equival- 3. The uncertainty of the results of future
and uncertainty sources relevant for ent. measurements, including bias correc-
420

tion, is calculated by the combination Testing for bias xcorrpf7xmeas . (8)

of two contributions: the uncertainty
of the uncorrected measurement result Basically, the difference found between The correction parameters a, or f, respec-
and the uncertainty of the correction. the mean value ~x 1 and the reference tively, are determined from the reference
The uncertainty of the correction is es- value xref is taken to constitute a signifi- sample data (the reference value and the
timated from the calibration data, cant bias, if the absolute value of this dif- mean of the measured values) as follows:
either by uncertainty propagation or ference is larger than twice the standard apxrefP~x 1 .
by statistical analysis of the residual deviation of the mean ~x 1 : (9)
scattering. xref
The main requirements for this type of 2s fp . (10)
h~x 1 Pxrefh 1 . (5) ~x 1
calibration, concerning the calibration ;n
samples and the measuring conditions, The correction can be carried out either
For low values of n (n~10) the factor 2
have already been described in the intro- by adjustment of the measuring system,
should be replaced by the Student factor
duction. referring to the zero point or to the sensi-
t for n–1 degrees of freedom and signifi-
Through this procedure the uncertain- tivity, respectively, or by recalculation.
cance level 0.95 (two-sided).
ty of the calibrated analytical method is
The significance criterion (Eq. 5) is
traceable to the reference values, includ-
based on the assumption that the stand-
ing their uncertainties, attributed to the Uncertainty evaluation
ard uncertainty of the reference value xref
calibration standards. Therefore these
is much smaller than the standard uncer-
data have to be well established by them- The standard uncertainty of the final
tainty of the mean value ~x 1 , and
selves or traceable to acknowledged ref- measurement result after bias correction,
therefore can be neglected. If this is not
erence standards. xcorr, is calculated from the uncertainty of
the case, the following significance criteri-
The procedure described in the fol- the uncorrected measurement result,
on has to be used:
lowing uses a single reference sample, xmeas, and the uncertainty of the correc-
which is selected to represent the speci-
" sn cu(x
2
tion, according to the rules of uncertainty
fied measuring range. Therefore this pro- h~x 1 Pxrefh 1 2 ref )2 . (6) propagation, as described in the GUM
cedure is only admissable, if it can be [1]. In the case of an additive correction
taken for granted that the results ob- In this case replacement of the factor 2 the result is as follows:
tained on the reference sample are valid by the Student factor t cannot be recom-
for the entire measuring range, that is, mended, because the required number of u(xcorr) 2pu(xmeas) 2cu(a) 2
for the variety of samples to be analysed degrees of freedom is often not well de- s2
pu(xmeas) 2cu(xref) 2c . (11)
in the future. If this is not the case, there fined. n
are two possibilities as follows: either the The significance test above refers to
measuring range has to be restricted ac- the customary 2s significance level, which In the case of a multiplicative correction
cordingly, or a two-level or a multi-level in the case of a normal distribution corre- an analogous equation holds for the rela-
calibration has to be used. sponds to 95% confidence, and therefore tive standard uncertainty, ur(X)pu(X)/
In the description of the procedure, makes use of the expanded uncertainty hXh.
the following quantities, relating to the with a coverage factor of kp2. However,
reference sample, are used: ur(xcorr) 2pur(xmeas) 2cur(f) 2
replacement of expanded uncertainty by sr 2
xref the reference value of the standard uncertainty is worth considering pur(xmeas) 2cur(xref) 2c . (12)
measured quantity X; for achieving compatibility with the bias n
u(xref) the standard uncertainty of handling approach in the Sect. Bias han-
the reference value; Here srps/h~x 1 h is the relative stand-
dling without correction, where significant ard deviation of the measurement results
x1, x2, ..., xn the results of repeated meas- bias is not corrected but added to the un- on the reference sample.
urements using the candidate certainty budget. These equations also apply if the bias
method;
turns out to be not significant. In this
~x 1 the mean value of x1, x2, ..., xn;
Correction of significant bias case no correction is necessary, that is,
s the standard deviation of
the correction term a and the correction
x1, x2, ..., xn.
This procedure applies if in the first step factor f are taken to be zero or unity, re-
In the case of the single-level proce-
significant bias was found. However, it is spectively. Nevertheless, also the values
dure the three steps specified above are
important to realize that a single-level zero or unity come with an uncertainty,
as follows. First, the mean value ~x 1
correction, as described in this section, is and this uncertainty has to be taken into
and the reference value xref are com-
only admissible, if it can be taken for account in the uncertainty budget.
pared, taking into account the relevant
granted that either the absolute errors of The standard uncertainty u(xmeas) is
uncertainties (Sect. Testing for bias). Sec-
ond, if significant bias is found, a bias measurement, or the relative errors of determined from the results of repeated
correction for the specified measuring measurement, are constant over the en- measurements on the test sample. For
range is derived (Sect. Correction of sig- tire measuring range. this purpose a clear definition of the
nificant bias). Third, the uncertainty of In the case of constant absolute errors quantity xmeas is important, namely
the results of future measuments, includ- significant bias is corrected by means of whether xmeas is a single measured value
ing bias correction, is calculated by the an additive correction term a as follows: or the mean of a specified number of
combination of two contributions: the un- measured values. If the measurement re-
xcorrpxmeasca . (7) sult is a single measured value, then
certainty of the uncorrected measurement
result and the uncertainty of the correc- In this equation xmeas is the raw result of u(xmeas) is given by
tion (Sect. Uncertainty evaluation). a measurement on a test sample, and xcorr u(xmeas)psmeas .
is the final result after bias correction. (13)
In the case of constant relative errors If the measurement result is the mean of
significant bias is corrected by means of a m measured values, then u(xmeas) is given
correction factor f as follows: by
 421

smeas certainty, due to expected bias, and com- single squared difference (x–m) 2 is a one-
u(xmeas) p . (14) bine it with the measurement standard shot estimate of the variance s 2. In the
;m
deviation. In this case, as no correction is special case where mp0, a single square
Here smeas is the standard deviation of the applied, we use the symbol xest instead of x 2 is a one-shot estimate of s 2. Now, as-
results of repeated measurements on the xcorr for the estimate of the true quantity sume that for the analytical method un-
test sample. Analogously, the relative under consideration, taking into account der consideration the bias varies at ran-
standard uncertainty ur(xmeas) is ex- the results of bias investigation. Then, dom over the specified range of analyte
pressed in terms of the relative standard keeping with the other notation in the concentration, matrix composition and
deviation srmeas. Sect. Direct determination of uncertainty, measuring conditions, taking values from
If it can be taken for granted that the a distribution of biases with expectation
spread of the results of repeated measur- xestpxmeas (15) zero and variance s 2bias. Then the
ements on the test sample is the same as squared difference (~x 1 –xref) 2, deter-
that on the reference sample, the stand- and the standard uncertainty on xest is
mined at a single reference point, is a
ard deviation smeas can be replaced by the given by
one-shot estimate of the expectation of
standard deviation s of the results of re- u(xest) 2pu(xmeas) 2c(~x 1 Pxref) 2 (16) the squared bias, hence of the bias var-
peated measurements on the reference iance s 2bias.
sample. Analogously, if the spread of the As an ad hoc statistical rationale, this Now it is time to compare the ap-
results of repeated measurements is equation may be justified as follows. proach described above, where significant
known to grow proportionally to the First, we note the basic fact that a single bias is not corrected but added to the un-
measured quantity, the relative standard value x from a probability distribution certainty budget, with the approach in the
deviation srmeas referring to the test sam- with expectation m and variance s 2 is a Sect. Direct determination of uncertainty,
ple, can be replaced by the relative stand- one-shot estimate (or a ”John-Wayne es- where significant bias is corrected, and
ard deviation sr referring to the reference timate”) of the expectation m. Similarly, a the uncertainty on the correction is added
sample.
Finally it has to be noted that if the
same correction is applied to independent
measurement results, the final results in-
cluding bias correction are correlated. In
the case of additive bias correction by
means of a correction term a the corre-
sponding covariance is given by u(a) 2.
Analogously, in the case of multiplicative
bias correction by means of a correction
factor f the corresponding covariance is
given by ur(f) 2.

Bias handling without correction

In practice it will frequently happen that
significant bias is observed, but the data
basis is not sufficient for deriving a mean-
ingful correction. For example, if bias was
detected using a single reference sample,
a single-level correction, as described in
the Sect. Direct determination of uncer-
tainty, is only admissable if it can be tak-
en for granted that either the absolute er-
rors of measurement, or the relative er-
rors of measurement, are constant over
the entire measuring range. Otherwise
the alternatives are:
– investigate bias more closely using ad-
ditional reference samples, and derive
a two-level or multi-level correction;
– do not correct for the observed bias,
but increase the uncertainty corre-
spondingly.
In this section we consider the second op-
tion, which will often be the only way out
due to lack of appropriate reference ma-
terials.
To the best of the author’s knowl-
edge, there is so far no rigorous statistical
approach for assigning an uncertainty to
observed, uncorrected bias. A pragmatic
solution currently under discussion is to
take the observed bias as a standard un- Fig. 1 Harmonized bias handling
422

to the uncertainty budget. As an obvious 6. Heydorn K (1993) Anal Chim Acta

requirement, bias correction should be References 283 : 494–499
rewarded by lower residual uncertainty, 7. Walker R (1997) Accred Qual Assur
that is, in case of significance the bias 2 : 40–43
should be larger than its standard uncer- 1. Guide to the Expression of Uncertain-
tainty. However, the significance criterion ty in Measurement (1993) ISO, Gene-
according to Eq. (6) only requires that va
Presented at: Analytica Conference ’98,
the bias is larger than its expanded uncer- 2. Quantifying Uncertainty in Analytical
Symposium 2: “Uncertainty budgets in
tainty, with a coverage factor of two. Measurement (1995) EURACHEM chemical measurements”, Munich,
Therefore it may be better, for the sake Laboratory of the Government Chem- 21–24 April 1998
of compatibility, to use another decision ist, London
rule in the Sect. Testing for bias, where 3. Williams A (1996) Accred Qual Assur W. Hässelbarth (Y)
the coverage factor of two is replaced by 1 : 14–17 Federal Institute for Materials Research
unity. If this is done, a harmonized sys- 4. ISO 5725 : 1994 Accuracy (trueness and and Testing (BAM),
tem of bias handling cases results as fol- precision) of measurement methods D-12200 Berlin, Germany
lows. and results. ISO, Geneva Tel.: c49-30-6392 5861
First, an estimate a of additive bias, 5. Ellison S, Williams A (1998) Accred Fax: c49-30-6392 5577
and its standard uncertainty u(a) are de-
Qual Assur 3 : 6–10 e-mail: werner.haesselbarth6bam.de
termined, and the bias estimate is tested
for significance: bias is significant if
hah 1 u(a), otherwise insignificant.
For insignificant bias, no correction is
applied, and the standard uncertainty on
the bias estimate is added to the uncer-
tainty budget. LEGISLATION AND NORMS
2 2 2
xestpxmeas, u(xest) pu(xmeas) cu(a) . (17)
Significant bias is corrected if appro-
Relevant legislation, legislation plans and related initiatives in different countries
priate, and the standard uncertainty on
and continents
the bias estimate is added to the uncer-
tainty budget.
xestpxmeasca, u(xest) 2pu(xmeas) 2cu(a) 2 . Accred Qual Assur 3 : 422–425 note is to give the reader a basic idea of
(18) Q Springer-Verlag 1998 the recent draft of ISO 17025 published
by the working group ISO/CASCO WG
If not appropriate, no correction is ap- Herfried Kohl 10/40 in February 1998.
plied, and the bias estimate is added to
the uncertainty budget.
The new ISO 17025 –
xestpxmeas , u(xest) 2pu(xmeas) 2ca 2 (19) The ISO world of standards
basic idea
A flow diagram of bias handling is given The two standards ISO 9001 (Quality sys-
in Fig. 1. tems – model for quality assurance in de-
Abstract This paper offers a concise sign, development, production, installa-
overview of the recent draft of ISO tion and servicing) and ISO 14001 (Envi-
Conclusions 17025, which is expected to replace EN ronmental management systems, specifi-
45001 in 1999. Reference is made to the cation with guidance for use) have re-
general area of related ISO standards for ceived enormous attention and accept-
As an alternative, and complement, of management systems. The main changes ance in all kinds of business. Compared
the step-by-step approach, uncertainty of compared with EN 45001 are discussed with these two standards, EN 45001 has
analytical methods may be determined di- for testing laboratories. some serious drawbacks. First, it is only a
rectly by analysis of reference samples European standard and second, some
and comparison of results with the known Key words ISO 17025 7 Quality assu- standard modules of quality management
reference values. Equivalently, the meth- rance 7 Quality management 7 Laborato- standards are lacking. Roughly speaking,
od under investigation may be applied in ry management EN 45001 focuses mainly on the technical
parallel with an established reference competence of the testing laboratory,
method to appropriate samples. As the whereas new quality management stand-
key item in this approach, handling of ards have more explicit requirements on
bias and estimation of bias-related uncer- Introduction how to deal with customers.
tainty have been discussed. Further work The new standard ISO 17025 will be
on this topic is currently in progress as a ISO 17025 (General requirements for the more in line with the philosophy of ISO
CITAC 1 project. competence of testing and calibration la- 9001 and, as a rule, laboratories now op-
boratories) designed to replace EN 45001 erating a management system conforming
(General criteria for the operation of with ISO 9001 and EN 45001 will have
testing laboratories) and ISO Guide 25 no difficulties in conforming with ISO
1
) CITAC denotes the Co-operation on (General requirements for the compe- 17025. In practice this is especially true
International Traceability in Analytical tence of calibration and testing laborato- for those laboratories which are part of
Chemistry that was established in 1993, ries) is now converging rapidly towards large organizations which run quality
see [7] for a recent account. its final version. The intent of the current management systems according to ISO