Outaki M et al. JOTCSA.

2024; 11(2): 751-764 REVIEW ARTICLE

Reference Materials: A review

Meriem Outaki1* , Sarah Loukhmas2 , Said Gmouh3 , Ebrahim Kerak4

1University Hassan I, Faculty of Sciences and Technologies Settat, Laboratory of applied chemistry and
environment, Settat, 26000, Morocco.
2University Mohamed V, Rabat, 10000, Morocco.
3University Hassan II, Faculty of sciences Ben M’Sik, Laboratory of engineering and materials (LIMAT),

Casablanca, 20000, Morocco.

4University Hassan II, Faculty of sciences and technologies Mohammedia, Laboratory of materials, catalysis

and natural resources valorization, Mohammedia, 20650, Morocco.

Abstract: Several factors have increased the use of reference materials in laboratories. This can be explained
by the fact that the reference materials have several roles, namely: the confirmation of the identity of
unknown materials and/or the determination of their properties; the calibration of measuring equipment; the
validation of methods; the realization of proficiency tests; etc. To be able to produce and use them, a set of
standards and guidelines concerning the subject of reference materials has been established. There are
several producers of reference materials in many fields, but finding the right choice is sometimes considered
difficult given the multitude of materials to be analyzed that do not correspond perfectly to the reference
material, especially in the case of matrices. This makes the market always seek new materials. To develop
them, five steps are essential: material preparation, homogeneity study, stability study, characterization,
and evaluation of measurement uncertainties. These steps are equally important; the fact of highlighting less
than one of them will imply a significant decrease in the quality of the reference material developed. This
review seeks to furnish the scientific community with a paper elucidating the functions of these materials in
research laboratories, the normative references devised to standardize their production and utilization, the
factors influencing their production, and the essential steps for their development.

Keywords: Reference material, Homogeneity study, Stability study, Characterization, Evaluation of

measurement uncertainties.

Submitted: October 15, 2023. Accepted: February 16, 2024.

Cite this: Outaki M, Loukhmas S, Gmouh S, Kerak E. Reference materials: A review. JOTCSA.
2024;11(2):751-64.

DOI: https://doi.org/10.18596/jotcsa.1361004

*Corresponding author’s E-mail: meriem.outaki@gmail.com

1. INTRODUCTION Decision-making is based on the knowledge of

information, which in most cases, in the field of
Throughout the world, the results of measurements research, rests on the results of measurements and
and testing are considered as a base to be able to testing delivered by the laboratories (1). These
control industrial production, affirm or disconfirm results, aiming mainly to answer research questions,
scientific hypotheses, make diagnoses in the medical are supposed to be of high quality (trueness,
field, assess the nature and degree of pollution in the precision) to guarantee a relevant interpretation by
environment, assess the nutritional quality of foods, the researchers. As a result, the prescribers of
etc. In this perspective, incorrect measurements and measurements and testing, who are increasingly
testing can be the source of incorrect decisions and aware, have increased their requirements about the
thus generate additional costs or risks. Therefore, reliability of the results provided by the laboratories
accurate measurements are of paramount (2). Measurements and testing results alone, not
importance for the functioning of society in all accompanied by information on their quality and
sectors: industry, health, environment, agri-food, traceability, are no longer considered sufficient and
etc. satisfactory; laboratories are obliged to prove the
reliability of their measurements and testing at all

751
 Outaki M et al. JOTCSA. 2024; 11(2): 751-764 REVIEW ARTICLE
stages, from sampling until the delivery of the final environmental, or industrial sources. They may also
report (2). be prepared by spiking the component(s) of interest
into an existing material.
In this perspective, modalities aiming at ensuring
good quality assurance of the measurements and 3. NORMATIVE REFERENCES
testing and accuracy of the produced data have been
established. This includes setting up good laboratory A panoply of standards and guidelines concerning the
practices and requirements of accreditation topic of RM are established whether by the
organizations (3) as well as participation in International Organization for Standardization (ISO)
interlaboratory comparisons (4) and the use of or by other organizations. Below is a non-exhaustive
reference materials. The use of the latter is list of these references:
considered a means for quality control of the results • ISO Guide 30:2015: Reference materials -
provided since they make it possible to join them to Selected terms and definitions (5);
known quantities (4). The certified reference • ISO Guide 31:2015: Reference materials -
materials are considered an essential link of a Contents of certificates, labels, and
metrological traceability chain to ensure that the accompanying documentation (7);
unity of the result is universal (1). Their use makes • ISO Guide 33:2015: Reference materials -
the results of measurements and testing comparable Good practice in using reference materials
to the recognized reference values and those (8);
obtained by the international scientific community. • ISO Guide 35:2017: Reference materials -
Reference materials are of paramount importance Guidance for characterization and
when the accuracy and reliability of the assessment of homogeneity and stability (9);
measurement and testing results must be • ISO 17034:2016: General requirements for
guaranteed. the competence of reference material
producers (10);
This review aims to provide the scientific community • ISO Guide 80:2014: Guidance for the in-
with a paper detailing the roles of these materials in house preparation of quality control
research laboratories, the normative references materials (QCMs) (11);
developed to standardize their production and use, • ISO/TR 16476:2016: Reference materials --
the elements that govern their production, as well as Establishing and expressing metrological
the steps necessary for their development. traceability of quantity values assigned to
reference materials (12);
2. TERMS AND DEFINITIONS • ISO/TR 79:2015: Reference materials -
Examples of reference materials for
ISO Guide 30:2015 (5) defines a reference material qualitative properties (13);
(RM) and a certified reference material (CRM) as • ISO/TR 10989:2009: Reference materials --
follows : Guidance on, and keywords used for, RM
• Reference material: “material, sufficiently categorization (14);
homogeneous and stable concerning one or • ISO 11095:1996: Linear calibration using
more specified properties, which has been reference materials (15);
established to be fit for its intended use in a • ISO/TR 11773:2013: Global distribution of
measurement process”. Properties can be reference materials (16);
quantitative or qualitative (e.g., identity of • ISO 15194:2009: In vitro diagnostic medical
substances or species). Uses may include the devices - Measurement of quantities in
calibration of a measurement system, samples of biological origin - Requirements
assessment of a measurement procedure, for certified reference materials and the
assigning values to other materials, and content of supporting documentation (17);
quality control. • ISO/FDIS 33407: Guidance for the
• Certified reference material: “RM production of pure organic substance
characterized by a metrologically valid certified reference materials (under
procedure for one or more specified development) (18);
properties, accompanied by an RM certificate • LAB MR REF 02 established by the Cofrac
that provides the value of the specified (Comité français d’accréditation) (19). It's a
property, its associated uncertainty, and a document concerning the requirements for
statement of metrological traceability”. accreditation of producers of RMs;
• APLAC TC 012: Guidelines for acceptability of
That said, the difference between an RM and a CRM chemical reference materials and commercial
is that parameters in the CRM are guaranteed by the chemicals for calibration of equipment used
producer and are known with great trueness (2). in chemical testing (20) established by the
APLAC (Asia Pacific Laboratory Accreditation
Different categories of RMs may be encountered: Cooperation).
primary RM, secondary RM, and in-house or working
RM. By going from first to last, uncertainty increases 4. ROLE OF REFERENCE MATERIALS
(6). An RM that is characteristic of a real sample is
called matrix RM (examples: soil, drinking water, Several fields, such as chemistry, biology, agri-food,
blood) according to ISO Guide 30:2015 (5). Matrix health, environment, etc., use the RMs. These latter
RM may be obtained directly from biological,

752
 Outaki M et al. JOTCSA. 2024; 11(2): 751-764 REVIEW ARTICLE
allow the realization of certain practices ensuring the matrix as the unknown sample (3).
quality of the measurements, namely: • Materials intended to test the step(s) of a method
• The determining of the properties of a material (3): the method, that contains steps where the
(e.g., quantity, hardness, etc.) and/or the sample is physically destroyed (e.g., acid
confirmation of its identity (1). digestion, fusions, dry ash) or the analyte to be
• The calibration of certain methods allows to determined is extracted from the matrix, can be
establish the substance signal-quantity verified using a CRM with a matrix (matrix CRM)
interaction (e.g., UV spectroscopy, colorimetry, similar to the unknown sample. The use of CRM
atomic emissions, chromatography, etc.). The allows us to demonstrate if there are losses or
calibration should be done by using the RMs and contaminations by applying the method. The
taking into account the nature of the matrix as presence of errors in this method can be affirmed
much as possible (21). This makes it possible to or disconfirmed by comparing the certified value
obtain quantitative results through the of the CRM and the value determined by the
comparison of the sample signals and those of the laboratory.
RM(22). The purity of the RM as well as the
uncertainty of this purity must be known (22). 5. PRODUCTION OF REFERENCE MATERIALS
• Calibration of measurement equipment (1–
3,6,23). 5.1. Producers of Reference Materials
• The development and validation of methods To meet the requirements of prescribers for reliable
(2,3,6,24) as well as the validation of the measurements, it is essential to produce RMs in a
modifications applied to the method (3): the wide variety of fields. As a result, the RMs market is
validation as one of the requirements of ISO/IEC considered a very active market (1). For the different
17025:2017 (25) should be performed whenever areas of measurements and testing, there are
non-standard methods, internally developed several RMs and CRM providers. There is a need to
methods, or methods outside their intended scope ensure the availability of sufficient quantities of these
are used. The estimation of the bias (difference materials as well as their availability in the long term,
between the measured value and the reference even if their production is long and expensive (21).
value) can be done by using the RM within the The main organizations distributing RMs are (26):
limits of the uncertainty of the certified value and • NIST: National Institute of Standards and
the method submitted to validation (1). Technology, United States;
• The internal quality control for the monitoring of • NIBSC: National Institute for Biological Standards
an analytical process (1–3,6,23,24): the and Control (WHO international laboratory for
verification of the long-term reproducibility of a biological standards), United Kingdom;
method is performed by using the RMs in • CENAM: Centro Nacional de Metrología, Mexico;
statistical control systems while creating control • IRMM: Institute for Reference Materials and
charts (a graphical representation of the results Measurements, European Union;
obtained from an RM over time, the system is out • LGC: LGC Limited, United Kingdom;
of control if the upper or lower control limit is • HECTEF: HECTEF Standard Reference Center
exceeded, among others) (3). The RMs thus allow Foundation, Japan.
monitoring of the stability of testing (2).
• The comparison of the performances of the Given the growth of the RM market and the diversity
methods in the same laboratory (3) by comparing of producers, finding the right choice has become
the testing results by using the same RM and difficult for users. Databases were developed to
several methods. assist laboratories in finding the CRM that they need
• The realization of proficiency tests (1,3,6,24): the by providing information on the available CRM.
laboratory that organizes the proficiency test However, the COMAR (Code of Reference Materials)
sends an RM to the participating laboratories. The remains the main source of information on RMs (3).
comparison of the testing results of the RM of a It was produced at the end of 1970 by the LNE
laboratory with those of other laboratories using (laboratoire national de métrologie et d’essai,
the same technique allows for evaluation of the France). Formal cooperation between LNE, NPL
performance of the laboratory with this technique (National Physical Laboratory, United Kingdom), and
(3). BAM (Bundesanstalt für Materialforschung und-
• The uncertainty estimation (1–3,6): the Prüfung, Germany) was then established to improve
realization of the design of experiments by using and propagate the database (27). The use of the
RMs is necessary for the estimation of all latter is free.
components of the measurement uncertainties
and for estimating the uncertainty of The COMAR database lists information on thousands
measurement by intra-laboratory reproducibility of CRMs produced by more than 200 producers in 26
(2). The uncertainty associated with the purity of countries.
the RM contributes to the total uncertainty of the
measurement (6). Figure 1 shows the distribution of the RMs according
• The traceability of measurement (24): CRMs are to their area of use. The biological and clinical
used for quantitative measurements, which domains have the lowest percentage. For the other
makes them "traceable" to these CRMs. However, domains, the distribution is very close, with a
this approach is not always reliable because, in dominance of the industrial sector.
many cases, the CRM does not have the same

753
 Outaki M et al. JOTCSA. 2024; 11(2): 751-764 REVIEW ARTICLE

Biological and
Quality of life,
clinical, 4%
14%
Ferrous, 12%

Physical, 15%

Industries, 19%
Organics, 9%

Inorganics, 13%
Non ferrous, 15%

Figure 1: Percentage distribution of CRM by fields of application (27).

Below are some examples of RMs in the carbohydrate oils, fat-soluble vitamins, etc.)
environmental, food, and clinical fields (3): are also taken into account during the
• Environmental analysis: river, lake, preparation of certain matrices.
estuarine, and marine sediments, as well as • Clinical analysis: RMs of serum and blood (in
fresh-, ground-, and seawater certified for lyophilized form) are most often used for the
trace elements or trace organics, are some quality control of, for example, toxic
examples of existing CRMs. elements (e.g., Cd, Pb) or main group
• Food analysis: matrices for human nutrition elements (e.g., Ca, Li, Mg) and hormones
(e.g., meats, cereals, and milk) are usually (e.g., cortisol, progesterone). Proteins or
analyzed for their content of toxic organic enzymes that are partially or highly purified
(e.g., PAHs, PCBs) or inorganic substances are also used.
(e.g., heavy metals, major elements),
nitrates, pesticides, toxin, and mycotoxins. Based on the consultation of the COMAR database, a
Nutritional quality and nutritional properties distribution of the origins of the produced RMs is
(e.g., fatty acids, sterols, total fat, proteins, presented in Figure 2.

Solid, 4% Artifacts, 2% Chips, granules,

11%
Crystals,
monocrystals
Powder, 29% and bicrystals,
1%

Cylinders,
Pellets, 0% mushrooms,
plates, discs,
23%
Miscellaneous,
Gas, 5%
1%
Liquid, 22% Ingots, rough
globules, bars
and wires, 2%

Figure 2: Origin of RMs at the level of COMAR database (27).

For each RM, the available information is the name, The selection of an RM by the user depends on his
status (available or not), year, country, producer, needs and objectives (2). The choice of an RM by the
general description, fields of application, and certified user takes into account the nature of the matrix, its
properties (27). concentration, availability, lifetime, and cost.

754
 Outaki M et al. JOTCSA. 2024; 11(2): 751-764 REVIEW ARTICLE
5.2. Constraints Related to The Production of suitable since they are not sufficiently similar to the
Reference Materials real sample (3). The RMs that are available and
The production and certification of the RMs require adapted to the complexity and multiplicity of testing
enormous costs. These latter are linked to the initial are insufficient (3). Also, new complex matrices need
investment for the purchase of the equipment, the to be analyzed, and the producers are unable to
time required for their development, the costs rapidly produce CRMs for these new samples (3). As
related to the certification procedure, as well as those a result, CRM request exceeds supply in terms of
about the production of a large stock needed to availability and nature of materials. This encourages
ensure the continuity of the availability of a specific users and accreditation organizations to understand
lot over several years (23). Having accurate the limits of the RMs used (6).
information on producer turnover and the cost price
of the RMs is not easy. Nevertheless, the cost of R&D 5.3. Accreditation of Reference Materials
in the case of an RM is weakly echoed by the cost Producers
price in the case of the national metrology The producers of RMs can be accredited by the
laboratories (1). This situation remains valid even for standard ISO 17034 which specifies the general
the big CRM producers such as NIST and IRMM (1). requirements for the competence of the producers of
RM users are confronted with the problem of RMs. To get an overview of this standard in process
identifying the appropriate material to use. Users are form, we propose a process mapping bringing
expected to look for an available RM, that also meets together all the elements covered by ISO
their needs (e.g., components, uncertainty, etc.) 17034:2016 (10) (Figure 3).
(28). In many cases, the available RMs are not

Figure 3: Process mapping bringing together all the elements covered by ISO 17034:2016.

ISO/IEC 17025 is considered indispensable for the Cofrac, an accrediting organization, also takes into
application of the ISO 17034 standard which is account the requirements of the standards ISO/IEC
aligned with the relevant requirements (10). 17025 (25) and/or ISO 15189 (29) to accredit these
Reference materials producers that comply with this producers (19). Table 1 presents common elements
international standard (ISO 17034) will also operate between ISO 17034:2016 (10), ISO/IEC 17025:2017
generally by the principles of ISO 9001 (10). The (25), and ISO 9001:2015 (30).

755
 Outaki M et al. JOTCSA. 2024; 11(2): 751-764 REVIEW ARTICLE

Table 1: Common elements between ISO 17034:2016, ISO/IEC 17025:2017, and ISO 9001:2015.
ISO 17034:2016 ISO/IEC 17025:2017 ISO 9001:2015
4.1 Contractual matters 7.1 Review of requests, tenders, -
and contract
4.2 Impartiality 4.1 Impartiality -
4.3 Confidentiality 4.2 Confidentiality -
5 Structural requirements 5 Structural requirements -
6.1 Personnel 6.2 Personnel -
6.2 Subcontracting 6.6 Externally provided -
6.3 Provision of equipment, products and services
services, and supplies
6.4 Facilities and environmental 6.3 Facilities and environmental -
conditions conditions
7.6 Measurement procedures 7.2 Selection, verification, and -
validation of methods
7.3 Sampling
7.7 Ensuring the validity of
results
7.7 Measuring equipment 6.4 Equipment -
7.8 Data integrity and evaluation 7.11 Control of data and -
information management
7.9 Metrological traceability of 6.5 Metrological traceability -
certified values
7.16 Control of quality and 7.5 Technical records -
technical records
7.17 Management of non- 7.10 non-conforming work -
conforming work
7.18 Complaints 7.9 Complaints -
8.2 Quality policy 8.2 Management system 5.2 Policy
documentation 6.2 Quality objectives and
planning to achieve them
8.3 General management system 8.3 Control of management 7.5 Documented information
documentation system documents
8.4 Control of management 8.4 Control of records
system documents
8.5 Control of records
8.6 Management review 8.9 Management review 9.3 Management review
8.7 Internal audits 8.8 Internal audits 9.2 Internal audits
8.8 Actions to address risks and 8.5 Actions to address risks and 6.1 Actions to address risks
opportunities opportunities and opportunities
8.9 Corrective actions 8.7 Corrective actions 10.2 non-conformity and
corrective actions
8.10 Improvement 8.6 Improvement 10.3 Continual improvement
8.11 Feedback from customers 8.6 Improvement 9.1.2 Customer satisfaction

Among the mandatory requirements of ISO subcontractor, the standard specifies the services
17034:2016 (10) about the production of RMs, there that may or may not be subcontracted. These
are requirements for subcontractors. If the services are presented in Table 2.
organization decides to call on the services of a

Table 2: Services that can be subcontracted or not.

Activities that can be subcontracted Processes should not be subcontracted
Sampling Production planning
Processing an RM Selection of subcontractors
Handling an RM Assignment of proper values and their uncertainties
Homogeneity and stability testing Authorization of property values and their uncertainties
Characterization Authorization of RM documents
Storage of RM
Distribution of RM

756
 Outaki M et al. JOTCSA. 2024; 11(2): 751-764 REVIEW ARTICLE

When a reference material producer calls on a For the development of RMs, a within-unit and
subcontractor to carry out part of the production, it between-unit homogeneity study is carried out (4).
must have procedures in place to ensure that the The objective is to ensure that the content is the
subcontractors' experience and technical same in one unit and from one unit to another (3).
competence are sufficient for the tasks entrusted to
them and that they comply with the relevant sections Different designs can be used for homogeneity
of ISO 17034:2016 (10) and other appropriate studies, namely: simple randomized design,
standards. randomized block design, and nested design (9).
Concerning the number of samples, it is
6. DEVELOPMENT OF REFERENCE MATERIALS: recommended to take 10 to 30 samples from the
KEY STEPS batch of material, ISO Guide 35:2017 proposes a
formula to calculate this number (9). Three
6.1. Preparation of The Material repetitions per sample are advised; it is
The RM preparation is performed in a manner that recommended to analyze them in a random order
minimizes heterogeneity and instability (31). and not in the same order (4). The analyses must be
Ensuring RM stabilization is the most sensitive and carried out following the conditions of repeatability.
difficult step (3). Many operations can be performed The types of sampling to be performed can be
to avoid material changes (e.g., to avoid chemical or systematic, random, or stratified random (4).
microbiological changes in the material, drying,
sterilization by irradiation, and/or freezing can be The study of homogeneity is carried out after placing
used, among others). The nature of the material and the material in the containers (33). The authors
the purpose of its use guide the choice of the method evaluated the between-unit homogeneity by
to be adopted (3). This choice must be made to analyzing samples whose number varied between 6
respect, as far as possible, the integrity of the and 16; this number was between 2 and 5 concerning
material (3). To avoid contamination, measures must the sub-samples used for the study of the within-unit
be taken not to cause inhomogeneity in a material homogeneity (34–43). The kind of sampling used is
that is already homogeneous (31). Also, precautions random (35–38,40,41,43) or stratified random
must be taken to prevent any negative influence (34,39). For the homogeneity study, internal (42,44)
from the environment. After homogenizing the or external (33,34) standards can be used. The
material, its storage must be done in suitable testing of samples is performed in a random order to
containers (3) (waterproof and inert (31)). distinguish between an analytical drift and a possible
tendency in the filling sequence (37).
Homogeneity and stability are two primordial
characteristics of an RM. The most important Homogeneity can be assessed using the ANOVA test
measures taken to ensure them are those taken (33,35–39,41,44–47). Other authors have used the
during the selection, preparation, and packaging of Student's t-test to evaluate significant differences
the material (32). The testing realized at a later step between averages obtained from within-unit and
over a limited period of some units of the RM can only between-unit analysis (34,48). The outliers in the
verify the success or failure of previously performed homogeneity study can be detected by visual
steps (32). inspection (9) or using the Grubbs test (9,37,38,44–
The 2017 version of ISO Guide 35 (9) leaves to the 46). Some authors used regression analysis to reject
RMs producers the judgment of the need or not to the drift of the measurement (36). The standard ISO
carry out experimental studies of homogeneity and 13528:2022 (49) recommends the use of the
stability based on several elements, namely (32): Cochran test for repetitions. Other standards provide
• The knowledge of the physical, chemical, and/or several methods for identifying outliers, namely: ISO
biological properties of the material as well as the 5725-2:2019(50) (which describes Grubbs tests)
knowledge of the variation of these properties and ISO 16269-4(51) (which describes graphical
over time based on the scientific literature; outlier tests and other tests for multiple outliers).
• The analysis of previous experimental data or
monitoring data of stability; Concerning uncertainty associated with
• The absence of anomalies detected by the heterogeneity (whom), further to ISO Guide 35:2017
producer and the users further to cumulative (9), it can be calculated by Formula:
observations on complete lifetimes;
• The tests were performed on the effect of the 2 2
𝑢ℎ𝑜𝑚 = √𝑢𝑏𝑏 + 𝑢𝑤𝑏
packaging.

6.2. Homogeneity Study With,

There are two types of homogeneity tests (5): ubb: standard uncertainty associated with between-
• Within-unit homogeneity, which represents the unit variability
uniformity of a specified property value within uwb: standard uncertainty associated with within-unit
each unit of an RM; heterogeneity
• Between-unit homogeneity represents the
uniformity of a specified property value among If a basic design and one-way analysis of variance
units of an RM. are used to evaluate homogeneity (figure 4), sbb2
(between-unit variance from a homogeneity study),
is identical to the (squared) between-unit

757
 Outaki M et al. JOTCSA. 2024; 11(2): 751-764 REVIEW ARTICLE
homogeneity contribution to the uncertainty, u bb2 homogeneity will be equivalent to swb2 (the second
(9). In the case presented in Figure 4, the sbb2 we will component of the uncertainty uhom (uwb2)).
obtain from ANOVA 1 in the case of within-unit

Within-unit homogeneity (Un)

One-way ANOVA
In this case, variance is called within-unit, between-subsample

Figure 4: Diagram of a basic plan for carrying out a between-unit and within-unit homogeneity study.

Classification of stability
studies

Classification according Classification according Designs for different

Classification study
to conditions of to stability study storage and treatment
objective
measurements duration and conditions conditions

Classical stability
Transportation or other
studies - Intermediate Real-time stability
short-term stability
conditions of studies
studies
measurement

Isochronus stability
studies - Repeatability Accelerated stability Long-term stability
conditions of studies studies
measurement
Figure 5: Classification of stability studies according to ISO Guide 35:2017.

758
 Outaki M et al. JOTCSA. 2024; 11(2): 751-764 REVIEW ARTICLE

35,37,38,46,54,55). The number of samples

analyzed at each time/temperature pair varied
Some authors (37) have used the method described between 1 (56), 2 (33,57), 3 (46), 5 (35) and 6 (37)
by Van Der Veen, Linsinger, & Pauwels (52) to samples. Certain authors use an external calibration
calculate the uncertainty due to possible for the study of stability (33). The evaluation of
inhomogeneity which can be masked by the stability is carried out by comparing the obtained
repeatability of the method. results in the initial time with those obtained in each
period (35). Some authors (34,37,57,58) used the
6.3. Stability Study isochronous approach (59) to assess the stability of
The properties of the material and the parameters the RM.
studied should remain unchanged over long periods
(3). For that purpose, a stability study is performed The stability study is performed by analyzing
to evaluate the behavior of the material under samples chosen randomly (33,46). Increasing the
various conditions. In general, a series of number of repetitions per time and increasing the
measurements realized at different times is duration of the stability study allow us to reduce the
performed to accomplish a stability study (31). influence of the analytical variation (31). To assess
Figure 5 presents the classification of stability studies stability, certain authors have used the F test (54).
according to ISO Guide 35:2107 (9). The Grubbs test is used by other authors to identify
outliers (37,44). The latter also performed linear
ISO 17034:2016 (10) requires stability to be regression analysis as a function of time and for each
assessed under the proposed conditions of transport temperature. These same authors tested the slopes
and the proposed conditions of storage. This for their significance using a t-test proposed also by
assessment can be carried out either accelerated or ISO Guide 35:2017 (9). The uncertainty of the
in real-time, using a classical or isochronous study. stability has been estimated by some authors
The latter is defined by ISO Guide 35:2017 (9) as an (37,44) “as the uncertainty of the regression line
“experimental study of reference material stability in with a slope of zero multiplied by the envisaged
which units exposed to different storage conditions shelf-life of the material”, as described by Thomas
and times are measured in a short period”. P.J. Linsinger et al. (60).

A short-term stability study is generally performed to 6.4. Characterization and Evaluation of

determine conditions during delivery Measurement Uncertainty
(3,37,41,44,46). Habitually high temperatures According to ISO Guide 30:2015 (5), the
(between 35 and 40 °C) are used to assess characterization of an RM is the “determination of the
predictable degradation under the most unfavorable property values or attributes of a reference material,
conditions, such as during transport (3). The long- as part of the production process”. It aims to obtain
term stability helps to identify optimal storage a metrologically valid estimation of the real value of
conditions for the RM (37,41,44). An accelerated the property (61).
stability study (where some samples are exposed to
extreme conditions in comparison to desired storage There are several valid characterization approaches
or transport conditions) has the advantage of (61):
reducing the total time required for the study of • A single (primary) method in a single laboratory;
experimental stability (32). Some authors suppose • Two or more independent reference methods in
that a “sample stable at + 40 °C during one year may one or several laboratories;
be stable at + 20 °C for a longer period (Arrhenius)” • One or more methods of demonstrable accuracy,
(3). Other authors judge the simulation of long-term performed by a network of competent
storage under difficult conditions as inappropriate, laboratories;
given the possibility of modification of the • An approach providing method-specific,
degradation mechanism (31). Therefore, using operationally defined property values, using a
extrapolation to estimate stability data from higher network of competent laboratories.
temperatures using the Arrhenius equation is not
recommended (31). The Consultative Committee for Amount of
Substance (CCQM) defined the primary method of
To study the stability of the RM, the authors analyzed measurement as a “method having the highest
samples of the RM stored at expected temperatures metrological qualities, whose operation can be
for storage, extreme temperatures (e.g., 40 °C, 60 completely described and understood, for which a
°C), and temperatures where it is supposed to have complete uncertainty statement can be written down
very weak or negligible variations of the RM (e.g., - in terms of SI units, and whose results are, therefore,
20 °C, -70° C, and -80° C) (3,31,33–35,37,46). The accepted without reference to a standard of the
stability study can also be performed by modifying quantity being measured” (62). The VIM (63) defined
other conditions, such as light, or darkness (53) the primary reference measurement procedure as a
and/or humidity. The short-term stability was “reference measurement procedure used to obtain a
assessed by the authors over periods ranging from measurement result without relation to a
0, 1, and 2 weeks, then over regular intervals of one measurement standard for a quantity of the same
month to 4 months (33–35,37,38). The study of kind”.
long-term stability has been carried out between 3
months and 2 years for some authors (33–

759
 Outaki M et al. JOTCSA. 2024; 11(2): 751-764 REVIEW ARTICLE
Some authors consider that the primary method is To help RM producers in preparing the
completely understood, so they judge that a single documentation that will accompany them, the ISO
value from a single laboratory is sufficient since an has established ISO Guide 31 (7) which lists the
unknown laboratory or method bias can be excluded information that must be included on a certificate of
(61). Other authors say that the laboratory may have a CRM, on the label attached to the container, and
a bias, and therefore, by using only one method in a also on the product information sheets. This
single laboratory, the certified value may be wrong information will help the users to confirm the
(3). In addition, an estimation of the uncertainty may adequacy of the chosen CRM.
not be correct by choosing the use of the results of
one method in a single laboratory (3). 7. CONCLUSION

On the other hand, the reference method is defined The importance of the reference materials has
in ISO Guide 30:2015 (5) as a “measurement pushed among others the ISO to the establishment
method, that has been shown to have the of several standards concerning the development
appropriate trueness and precision for its intended and use of the reference materials.
use and has been officially defined as reference
method by a competent body”. The VIM (63) defines The development of the reference materials consists
the reference measurement procedure as a of four steps mainly: preparation, realization of the
“measurement procedure accepted as providing homogeneity study, carrying out the stability study,
measurement results fit for their intended use in and characterization. These steps are equally
assessing the measurement trueness of measured important; the fact of highlighting less than one of
quantity values obtained from other measurement them will imply a significant decrease in the quality
procedures for quantities of the same kind, in of the reference material developed.
calibration, or in characterizing reference materials”.
The production of reference materials is governed by
The use of several independent methods is the most many different factors and may differ from one
commonly adopted approach (3,21). country to another. The introduction of quality
assurance in laboratories, the implementation of a
The choice of the characterization of the samples by quality approach by ISO/IEC 17025, the regulatory
several laboratories is within the scope of the framework which sometimes imposes thresholds for
interlaboratory comparisons. This involves sending certain elements as well as the demand of
the same entity or similar entities to the selected industrialists can be considered elements
laboratories for analysis. Further to the obtained encouraging the production of the reference
results by the different laboratories, a statistical materials. These latter are of paramount importance
analysis is planned to identify the assigned values in analysis laboratories since they have several roles,
and their associated uncertainties. ISO 13528 namely calibration of equipment, development, and
presents the statistical methods that can be used in validation of methods, and realization of
proficiency testing by interlaboratory comparison. interlaboratory comparisons among others.
Interlaboratory comparisons allow us to verify the Certainly, the laboratories need more and more
trueness of the methods of analysis (2). reference materials, however finding the appropriate
ones is not easy. In several cases, the available
The number of chosen laboratories (6, 11,24), the reference materials do not correspond 100 % to the
number of used methods (1, 2, 3), the number of sample to be analyzed because of the variety of
analyzed samples (2), the number of repetitions per compounds, concentrations, and matrices; this
sample (1, 2, 6) as well as the period of analysis (2 makes the development of new reference materials
days, 3 days) differ from one author to another a necessity. This expensive operation requiring
(4,41,44). Some authors chose to do the repetitions technical knowledge and important experience
under intermediate precision conditions (4). For the makes the price of these reference materials high,
control of the measurements, the authors have which makes their use limited in the laboratories.
chosen to send also two samples of a "standard
certified calibration" for analysis (41). The selected The outlook for the reference materials development
laboratories must use validated methods and be able market will depend on several factors, including the
to ensure the traceability of results (4). rapid development of analytical technologies using
cutting-edge analytical methods, and the expansion
Evaluation of the uncertainty is carried out following of the biotechnology, nanotechnology, and
the characterization. This assessment takes into renewable energy sectors. Demand for sustainably
account the uncertainty of the testing having allowed produced reference materials could increase as
the determining of reference values environmental concerns gain in importance.
(characterization) as well as those related to within- Increasingly stringent standards and regulations,
unit and between-unit homogeneity and stability economic change, and scientific developments could
(2,39). These last two uncertainties may not appear lead to a customization of demand for reference
in the uncertainties budget if the contribution of the materials tailored to the specific needs of certain
one related to homogeneity is shown to be negligible fields.
and if the stability of the property value of the
material can be insured (64). 8. CONFLICT OF INTEREST

Nothing to report.

760
 Outaki M et al. JOTCSA. 2024; 11(2): 751-764 REVIEW ARTICLE
15. ISO. ISO 11095:1996 Linear Calibration Using
9. ACKNOWLEDGMENTS Reference Materials. 1996.

Nothing to report. 16. ISO. ISO/TR 11773:2013 Global Distribution of

Reference Materials. 2013.

10. REFERENCES 17. ISO. ISO 15194:2009 In Vitro Diagnostic Medical

Devices - Measurement of Quantities in Samples of
1. Charlet P, Hervouët G. Nouvelles demandes en Biological Origin - Requirements for Certified
matériaux de référence Certifies cas de secteurs Reference Materials and the Content of Supporting
émergents: Environnement: Biologie et Autres. Lab Documentation. 2009.
Natl d’essai, Paris. 2001.
18. ISO. ISO/FDIS 33407 Guidance for the
2. Lafargue M, Seiller MP. Les matériaux de Production of Pure Organic Substance Certified
référence: définitions, disponibilité, sélection et Reference Materials.
utilisation. Spectra Anal. 2004;240:33–7.
19. Cofrac. Exigences pour l’accréditation des
3. Günzler H, editor. Accreditation and Quality producteurs de matériaux de référence LAB MR REF
Assurance in Analytical Chemistry [Internet]. Berlin, 02 – Révision 02. 2019.
Heidelberg: Springer Berlin Heidelberg; 1996.
20. APLAC. Guidelines for Acceptability of Chemical
Available from: <URL>.
Reference Materials and Commercial Chemicals for
4. Daugey G, Soule P, Hakim K, Hewitt A, Prunet T, Calibration of Equipment Use in Chemical Testing
Nicolas M, et al. Production d’un échantillon de APLAC TC 012. 2009.
référence certifié : du prélèvement au
21. Vialle J, Linet P, Maier EA. La métrologie en
dimensionnement des essais de certification. Cah des
chimie. Quelques principes et beaucoup
Tech l’INRA [Internet]. 2015;1(84):1–8. Available
d’incertitudes. Analusis [Internet]. 1999 Jul
from: <URL>.
1;27(6):479–90. Available from: <URL>.
5. ISO. ISO Guide 30:2015 Reference Materials -
22. Borges R, Meyer VR. The uncertainty of purity of
Selected Terms and Definitions. 2015.
reference materials must be known. J Pharm Biomed
6. King B. The selection and use of reference Anal [Internet]. 2013 Apr 15;77:40–3. Available
materials - a basic guide for laboratories and from: <URL>.
accreditation bodies. Accredit Qual Assur [Internet].
23. Jenks PJ, Henk Boekholt A, Maaskant JFN,
2003 Sep 1;8(9):429–33. Available from: <URL>.
Rucinski RD. Are certified reference materials a
7. ISO. ISO Guide 31:2015 Reference Materials - victim of quality systems? The need for working
Contents of Certificates, Labels and Accompanying matrix-certified reference materials. Fresenius J Anal
Documentation. 2015. Chem [Internet]. 1998 Feb 1;360(3–4):366–9.
Available from: <URL>.
8. ISO. ISO Guide 33:2015 Reference Materials -
Good Practice in Using Reference Materials. 2015. 24. Majcen N. A need for clearer terminology and
guidance in the role of reference materials in method
9. ISO. ISO Guide 35:2017 Reference Materials - development and validation. Accredit Qual Assur
Guidance for Characterization and Assessment of [Internet]. 2003 Apr 1;8(3–4):108–12. Available
Homogeneity and Stability. 2017. from: <URL>.

10. ISO. ISO 17034:2016 General Requirements for 25. ISO. ISO/IEC 17025 General Requirements for
the Competence of Reference Material Producers. the Competence of Testing and Calibration
2016. Laboratories. 2017.

11. ISO. ISO Guide 80:2014 Guidance for the in- 26. Vassault A, Giroud C, Arnaud J. Traceability of
House Preparation of Quality Control Materials the standards and calibrations. Ann Biol Clin (Paris).
(QCMs). 2014. 2010 Dec;68(1):295–303.

12. ISO. ISO/TR 16476:2016 Reference Materials - 27. BAM - RRR - Home [Internet]. 2017 [cited 2018
Establishing and Expressing Metrological Traceability Jul 14]. Available from: <URL>.
of Quantity Values Assigned to Reference Materials.
2016. 28. Steiger T, Pradel R. COMAR: the international
database for certified reference materials—an
13. ISO. ISO/TR 79:2015 Reference Materials - overview. Accredit Qual Assur [Internet]. 2015 Feb
Examples of Reference Materials for Qualitative 12;20(1):47–52. Available from: <URL>.
Properties. 2015.
29. ISO. ISO 15189:2022 Medical Laboratories --
14. ISO. ISO/TR 10989:2009 Reference Materials - Requirements for Quality and Competence. 2022.
Guidance on, and Keywords Used for, RM
Categorization. 2009. 30. ISO. ISO 9001:2015 Quality Management
Systems — Requirements. 2015.

761
 Outaki M et al. JOTCSA. 2024; 11(2): 751-764 REVIEW ARTICLE
31. Linsinger TPJ, Pauwels J, van der Veen AMH, 42. Kinumi T, Goto M, Eyama S, Kato M, Kasama T,
Schimmel H, Lamberty A. Homogeneity and stability Takatsu A. Development of SI-traceable C-peptide
of reference materials. Accredit Qual Assur certified reference material NMIJ CRM 6901-a using
[Internet]. 2001 Jan 9;6(1):20–5. Available from: isotope-dilution mass spectrometry-based amino
<URL>. acid analyses. Anal Bioanal Chem [Internet]. 2012
Jul 20;404(1):13–21. Available from: <URL>.
32. Ellison SLR, Botha A. Principles for the
assessment of homogeneity and stability in the new 43. Takahashi K, Matsuyama S, Kato H, Kinugasa S.
ISO Guide 35:2017. Accredit Qual Assur [Internet]. Development of the Certified Reference Material of
2018 Feb 25;23(1):47–51. Available from: <URL>. Poly(ethylene glycol) Nonylphenyl Ether by
Supercritical Fluid Chromatography. Int J Polym Anal
33. Ng SY, Dewi F, Wang J, Sim LP, Shin RYC, Lee Charact [Internet]. 2012 Apr;17(3):208–17.
TK. Development of a cosmetic cream certified Available from: <URL>.
reference material: Certification of lead, mercury and
arsenic mass fractions in cosmetic cream. Int J Mass 44. Zeleny R, Schimmel H, Ulberth F, Emons H.
Spectrom [Internet]. 2015 Oct 15;389:59–65. Development of a certified reference material for the
Available from: <URL>. content of nitroimidazole parent drugs and hydroxy
metabolites in pork meat. Anal Chim Acta [Internet].
34. Chew G, Sim LP, Ng SY, Ding Y, Shin RYC, Lee 2009 Feb 23;634(2):237–42. Available from:
TK. Development of a mushroom powder Certified <URL>.
Reference Material for calcium, arsenic, cadmium
and lead measurements. Food Chem [Internet]. 45. Kim MJ, Kim M, Kim S, Lee K, Kim Y, Kim J, et al.
2016 Jan 1;190:293–9. Available from: <URL>. Feasibility study on the homogeneity and stability of
megastigmatrienone isomers for the development of
35. Valente A, Sanches-Silva A, Albuquerque TG, certified reference material. J Food Compos Anal
Costa HS. Development of an orange juice in-house [Internet]. 2023 Oct 1;123:105573. Available from:
reference material and its application to guarantee <URL>.
the quality of vitamin C determination in fruits, juices
and fruit pulps. Food Chem [Internet]. 2014 Jul 46. Sin DWM, Chan P kwan, Cheung STC, Wong YL,
1;154:71–7. Available from: <URL>. Wong S kay, Mok C shing, et al. Development of a
candidate certified reference material of
36. Espinoza M, Azar CO, Massiff G, Villa A. cypermethrin in green tea. Anal Chim Acta
Development of a reference material for copper and [Internet]. 2012 Apr 6;721:110–4. Available from:
iron in wine. J Food Compos Anal [Internet]. 2008 <URL>.
Dec 1;21(8):684–8. Available from: <URL>.
47. Mazánová M, Kačur M. Homogeneity evaluation
37. Zeleny R, Emteborg H, Charoud-Got J, Schimmel of liquid reference material developed for nuclear
H, Nia Y, Mutel I, et al. Development of a reference decommissioning. Appl Radiat Isot [Internet]. 2023
material for Staphylococcus aureus enterotoxin A in Jul 1;197:110775. Available from: <URL>.
cheese: Feasibility study, processing, homogeneity
and stability assessment. Food Chem [Internet]. 48. Shimizu Y, Ohte Y, Kato K. Development of a
2015 Feb 1;168:241–6. Available from: <URL>. certified reference material for calibration of DSC and
DTA below room temperature: NMIJ CRM 5401-a,
38. Gong N, Zhang B, Hu F, Du H, Du G, Gao Z, et Cyclohexane for Thermal Analysis. Thermochim Acta
al. Development of a new certified reference material [Internet]. 2013 Sep 20;568:61–6. Available from:
of diosgenin using mass balance approach and <URL>.
Coulometric titration method. Steroids [Internet].
2014 Dec 15;92:25–31. Available from: <URL>. 49. ISO. ISO 13528:2022 Statistical Methods for Use
in Proficiency Testing by Interlaboratory Comparison.
39. Yarita T, Otake T, Aoyagi Y, Kuroda Y, Numata 2022.
M, Iwata H, et al. Development of soybean certified
reference material for pesticide residue analysis. 50. ISO. ISO 5725-2:2019 Accuracy (Trueness and
Talanta [Internet]. 2014 Feb 15;119:255–61. Precision) of Measurement Methods and Results —
Available from: <URL>. Part 2: Basic Method for the Determination of
Repeatability and Reproducibility of a Standard
40. Otake T, Yarita T, Aoyagi Y, Kuroda Y, Numata Measurement Method. 2019.
M, Iwata H, et al. Development of apple certified
reference material for quantification of 51. ISO. ISO 16269-4:2010 Statistical Interpretation
organophosphorus and pyrethroid pesticides. Food of Data — Part 4: Detection and Treatment of
Chem [Internet]. 2013 Jun 1;138(2–3):1243–9. Outliers. 2010.
Available from: <URL>.
52. van der Veen AMH, Linsinger TP, Pauwels J.
41. Köppen R, Bremser W, Stephan I, Klein-Hartwig Uncertainty calculations in the certification of
K, Rasenko T, Koch M. T-2 and HT-2 toxins in oat reference materials. 2. Homogeneity study. In:
flakes: development of a certified reference material. Bièvre P, Günzler H, editors. Measurement
Anal Bioanal Chem [Internet]. 2015 Apr Uncertainty in Chemical Analysis [Internet]. Berlin,
16;407(11):2997–3007. Available from: <URL>. Heidelberg: Springer Berlin Heidelberg; 2001. p. 94–
8. Available from: <URL>.

762
 Outaki M et al. JOTCSA. 2024; 11(2): 751-764 REVIEW ARTICLE
53. Cuadros-Rodríguez L, Bosque-Sendra JM, de la processing, homogeneity and stability assessment.
Mata-Espinosa AP, González-Casado A, Rodríguez- Anal Bioanal Chem [Internet]. 2015 Apr
García FP. Elaboration of Four Olive Oil Certified 28;407(11):3083–91. Available from: <URL>.
Reference Materials: InterOleo-CRM 2006
Certification Study. Food Anal Methods [Internet]. 59. Lamberty A, Schimmel H, Pauwels J. The study
2008 Dec 13;1(4):259–69. Available from: <URL>. of the stability of reference materials by isochronous
measurements. Fresenius J Anal Chem [Internet].
54. Hon PYT, Chu PWS, Cheng C ho, Lee TCL, Chan 1998 Feb 1;360(3–4):359–61. Available from:
P kwan, Cheung STC, et al. Development of <URL>.
melamine certified reference material in milk using
two different isotope dilution mass spectrometry 60. Linsinger TPJ, Pauwels J, Lamberty A, Schimmel
techniques. J Chromatogr A [Internet]. 2011 Sep HG, van der Veen AMH, Siekmann L. Estimating the
28;1218(39):6907–13. Available from: <URL>. uncertainty of stability for matrix CRMs. Fresenius J
Anal Chem [Internet]. 2001 Jun 1;370(2–3):183–8.
55. Gab-Allah MA, Getachew Lijalem Y, Yu H, Lee S, Available from: <URL>.
Baek SY, Han J, et al. Development of a certified
reference material for the accurate determination of 61. Linsinger TPJ, Emons H. Characterization of
type B trichothecenes in corn. Food Chem [Internet]. reference materials: proposal for a simplification of
2023 Mar 15;404:134542. Available from: <URL>. the options listed in ISO Guide 34. Accredit Qual
Assur [Internet]. 2013 Apr 12;18(2):149–52.
56. Ortega-Gavilán F, García-Mesa JA, Marzal- Available from: <URL>.
Fernández JC, Moreno-Ballesteros FJ, Rodríguez-
García FP, González-Casado A, et al. Innovative and 62. Milton MJT, Quinn TJ. Primary methods for the
thorough practice to certify reference materials for measurement of amount of substance. Metrologia
sensory defects of olive oil. Food Chem [Internet]. [Internet]. 2001 Aug 1;38(4):289–96. Available
2022 Jun 30;380:132195. Available from: <URL>. from: <URL>.

57. Paleari R, Muñoz A, Mosca A. Towards the 63. ISO. ISO/IEC Guide 99:2007 International
development of a certified reference material for Vocabulary of Metrology — Basic and General
hemoglobin A 2. cclm [Internet]. 2010 Nov Concepts and Associated Terms (VIM). 2007.
1;48(11):1611–8. Available from: <URL>.
64. Trapmann S, Botha A, Linsinger TPJ, Mac Curtain
58. Grimalt S, Harbeck S, Shegunova P, Seghers J, S, Emons H. The new International Standard ISO
Sejerøe-Olsen B, Emteborg H, et al. Development of 17034: general requirements for the competence of
a new cucumber reference material for pesticide reference material producers. Accredit Qual Assur
residue analysis: feasibility study for material [Internet]. 2017 Dec 26;22(6):381–7. Available
from: <URL>.

763
Outaki M et al. JOTCSA. 2024; 11(2): 751-764 REVIEW ARTICLE

764