Food Quality and Preference 13 (2002) 341353

www.elsevier.com/locate/foodqual

A comparison of sensory methods in quality control

E. Costell*
Instituto de Agroqumica y Tecnologa de Alimentos (CSIC) PO Box 73, 46100 Burjassot, Valencia, Spain
Accepted 8 February 2002

Abstract
Many dierent types of sensorial methods have been proposed and used to evaluate and control the sensory quality of foods.
However, not all of them are suitable for incorporation in to quality control programmes. To simplify comparison a distinction is
proposed between methods that can be used to dene sensory specications or to select a product quality standard and those that
can be used to check if a product complies with stated requirements. With this approach, the appropriateness and limitations of
dierent methods and their practical applicability, according to their use with or without a previously selected or developed standard (product, mental or written), are discussed. # 2002 Elsevier Science Ltd. All rights reserved.
Keywords: Sensory quality; Sensory analysis; Quality standards; Quality specications

1. Introduction
The term quality has been used so much and in so
many contexts that its meaning is frequently unclear. A
number of denitions have been proposed, always with
reference to the situation or problem to be solved in each
case. They vary widely between simple expressions such as
Fitness for use (Juran, 1974) to more detailed ones like
that proposed by Molnar (1995): The quality of food
products, in conformity with consumers requirements
and acceptance, is determined by their sensory attributes, chemical composition, physical properties, level
of microbiological and toxicological contaminants,
shelf-life, packaging and labelling. Any of these or
many other denitions could be useful in a certain context but none of them is always satisfactory. Quoting
Fisken (1990), quality is a fuzzy and relative term and
it is in a constant motion. Due to this lack of conceptual denition, any specication, method or group of
methods designed to control the quality of a certain
product may be applicable in a particular situation but
they are subject to a constant evolution. The changes
come, on the one hand in function of methodological
advances in each area (chemical analysis, microbiology,
toxicology, etc.) and on the other, of changes undergone

* Tel.: +34-96-3900022; fax: +34-96-3636301.

E-mail address: ecostell@iata.csic.es (E. Costell).

by market requirements and degree of commercial

competition for the particular food product.
Notwithstanding the lack of conceptual denition of
quality, food quality control and assurance is evidently
a top subject both in industry and in public and private
control institutions (Gould & Gould, 1988; Herschdorfer, 1984; Juran, 1974; Kramer & Twigg, 1970; Stauer,
1988) and continues to be a matter of discussion in both
academic and industrial forums. Some of the most relevant items, advances and problems related to the denition and measurement of quality of foods have been
discussed in a Special Issue of Food Quality and Preference
published in 1995.
Basically the utilisation of any type of method in food
quality control follows a common approach: rst, denition of specications or quality standards and second,
development and testing of methods to evaluate, in a
reliable manner, whether or not a product complies with
the requirements of the quality standards. Two questions arise with this approach: (1) Which food characteristics or properties should be included in the
standard? (2) Which methods are to be used for their
analysis or evaluation? The answer to the rst question
is usually conditioned by the need to establish a compromise between two extreme alternatives: either consider a large number of the food samples or a long list of
food properties, leading to a rather complete specication but dicult to apply in practice, or else, select only
those characteristics of higher incidence on quality that

0950-3293/02/$ - see front matter # 2002 Elsevier Science Ltd. All rights reserved.
PII: S0950-3293(02)00020-4

342

E. Costell / Food Quality and Preference 13 (2002) 341353

make it possible to decide if the food fulls the requirements of a certain quality grade in a simpler manner.
On answering the second question a similar situation
holds: not always the most precise and costly methods
are most suitable but, in general, the selection is based
on the capacity of the method to measure variations in
each of the characteristics that inuence product quality
with sucient precision.
The implementation of food quality control and
assurance systems, in the areas of chemical composition,
microbiological and toxicological safety, and nutritional
characteristics, brings up problems related to the selection of properties or characteristics to be measured and
to the methods to be used. These problems are much
more numerous when the system is designed to control
what is known as sensory quality. Sensory quality is
even more dicult to dene because it is linked not only
to food properties or characteristics but to the result of
an interaction between the food and the consumer.
Besides, sensory evaluation is a rather recent discipline,
as compared with others such as chemical or microbiological analysis. It was born and slowly developed its
methodology during the second half of the twentieth
century (Costell & Duran, 1981; Costell, 2000; Larmond, 1994; Moskowitz, 1993). As a consequence, not
all methods developed and used by dierent research
teams at dierent times can today be considered adequate to evaluate and control the sensory aspect of
quality.
The concept of sensory quality has changed with time
since it was dened by Kramer in 1959 as the composite of those characteristics that dierentiate among
individual units of a product and have signicance in
determining the degree of acceptability of that unit by
the user. Some authors centre their attention on the
rst part of this denition. For them sensory quality is
product oriented. Others emphasise the second part and
consider that sensory quality is consumer oriented. In
the rst case, quality is considered as a convention
developed by experts and it may therefore be considered
as constant over a limited period only (Molnar, 1995).
With the second approach, quality is mainly related to
consumer acceptance and is context dependent (Cardello, 1995). The product oriented approach may, in
some cases, render results of doubtful practical validity
since it is assumed that the opinion of a group of experts
is representative of the reaction of the potential consumers of the product in question. But the second
approach is not totally satisfactory either because if a
specication or standard has to be established in order
to dene the sensory quality of a certain food product, it
is not sucient to collect acceptability data that merely
give statistically signicant results (Booth, 1995). In
relation to the latter point it should be considered that,
according to Stone and Sidel (1993), when xing acceptable deviations of the magnitude of an attribute with

reference to a standard, one should consider not only

their direct eect on acceptability but also to what
extent such a variability may aect consumer condence
in the product. It is evident that a constantly changing
product is certain to aect consumer condence. For
example, in a recent study (Costell, unpublished data),
to determine the inuence of some sensory attributes on
the acceptance of commercial chocolate milk beverages,
no relationship was found between perceptible colour
dierences and consumer acceptance. Products of
clearly dierent colours were equally acceptable. However, it is evident that manufacturers should dene their
colour standard and control variability between lots to
avoid the negative eect that perceptible dierences will
have on the condence of regular consumers.
The recentness and slow development of the discipline
of sensory analysis is perhaps the cause of the lack of
immediate responses to the real need for methods to
measure and control sensory quality both in industry
and in control organizations. Consequently, many
methods or systems, with variable scientic base, have
been proposed for sensory quality control. Some of
them are being used at the present time simply by inertia
or habit. Every quality control technician or group has
tried to solve problems by themselves in the best possible way, as, although some books on this matter are
available (ASTM, 1996; Pangborn & Roessler, 1965;
Lawless & Heymann, 1998; Moskowitz, 1983; Stone &
Sidel, 1993), it was not until 1992 that Munoz, Civille &
Carr published the book Sensory Evaluation in Quality
Control, the rst one exclusively dedicated to this specic
topic.
For these reasons, when analysing the dierent alternatives proposed and used by control organizations and
by dierent industries, the rst impression is that there
is a great diversity of approaches, requirements, levels
of strictness and practical applicability and still today it
is generally considered that the correct application of
sensory methods requires a lot of time to carry out and
to analyse data and that the number of qualied assessors is not always available. To simplify comparison
between methods, the distinction between the sensory
methodology to be used in the development of standards and specications from that to be used to check if
a product complies with their requirements is important. On one hand, when establishing standards and
specications, collecting data sets from dierent tests
and relating them allows for powerful modelling of the
relationships between physical process and ingredient
variables and the perceived attributes from descriptive
proling, and ultimately, consumer appeal, present no
problems (Munoz & Chambers, 1993). In this case it is
neither necessary nor convenient to use fast methods or
to take quick decisions. On the other hand, in practical
quality control, fast methods using a few assessors are
needed in order to take quick decisions at a given

E. Costell / Food Quality and Preference 13 (2002) 341353

moment. For these reasons, the author proposes to differentiate between methods that can be used to dene
specications or to select a product quality standard and
those destined to decide whether a particular food item
fulls the requirements of the appropriate standard.

2. Preliminary steps
2.1. Selection or establishment of sensory quality
standards or specications
2.1.1. Sensory quality standards
The establishment or denition of the quality standards is the critical point in the implementation of a
quality control programme. In practice, each company
or institution must dene the quality level to be controlled in a certain product and then develop a standard
that ts their objectives. When dealing with foods and
with sensory quality it is dicult, and often practically
impossible, to obtain a product or a series of products
showing the same unaltered sensory characteristics during enough time to permit their use as reference items in
subsequent comparisons. Fortunately, for some attributes, such as colour or appearance, quality standards
(photographs or reproductions of the food in materials
like plastics or ceramic) have been used successfully
when the product itself cannot be used, generally for
reasons of sensory variation or alterations. In the
majority of cases even this is not possible. This problem
has traditionally been solved in two ways: either relying
on the mental standard created by one or several experts,
or developing a written standard, in which a description
of the main attributes is commonly included.
2.1.1.1. Product standard. As indicated above, the use
of the same product as a standard in the evaluation of
the quality of raw or processed foods is almost always
dicult or impossible. However it is more frequently
used in quality control of ingredients or of some raw
materials. According to Munoz et al. (1992) a control
standard selected for quality purposes is referred to as a
product that is used as a representation of certain characteristics (not necessarily the optimal) and a product
that can easily be obtained, maintained or reproduced.
The criteria for choosing a product as a control standard can be arbitrary or deliberate. In any case, before
its selection, information must be obtained on the product variability and on its incidence on the sensory
quality of the nal food item. This implies the identication and quantication of the sensory attributes of the
studied ingredient or raw material by using sensory
descriptive techniques (proles, QDA, Spectrum) and
the determination of those attributes that inuence the
nal product quality assessment by consumers. Acceptable variation limits for each of the attributes should

343

also be established. An interesting point to consider

when a real product is used as a quality standard is the
establishment of a clear methodology to substitute it
when necessary. This need arises when the standard
product is running out of stock or when the end of its
shelf-life is getting near. The new standard must be sensorially identical to the previous one. This similarity
should be ascertained by means of sensory discriminatory
or dierence tests, such as the triangle test. An important consideration here is that the objective of the sensory test is not to detect dierences between samples but
rather to establish that they are sensorially equivalent.
In this case the analyst must determine what constitutes
a meaningful dierence by selecting the proportion of
distinguishers and then select a small value of b-risk to
ensure that there is only a small chance of missing that
dierence if it really exists (Ennis, 1993; Meilgaard,
Civille & Carr, 1999; Schlich, 1993).
2.1.1.2. Mental standard. One of the most controversial
strategies in sensory quality control is to assign a quality
level to a product with reference to a mental standard
developed by one or a group of experts or panellists.
Criticisms have been based mainly on two aspects. The
lack of concordance between experts as to the mental
standard applicable to a certain product and the fallacy
of assuming that the opinion of the experts represents
that of the consumers. Mental standards should only be
criticized if each expert or panellist operates under different criteria or mental standards, and if panellists do
not evaluate products uniformly. Therefore, when a
mental standard is to be used, panellists need to be
trained on the criteria and product attributes that are to
form the mental standard. This training provides validity and reliability to the use of mental standards and
thus contributes to a sound evaluation method. In
addition, criteria and/or products that form the mental
standard should periodically be reviewed to strengthen
the principles learned and the reliability of this practice.
Munoz et al. (1992) describe the procedures to form and
teach sound mental standards to panellists for some of
the QC/sensory methods that use these standards (e.g.
in/out method)
Consumer opinion is aected by the context in which
the food is examined and by the expectations that some
external factors, such as brand or price, will exert (Cardello, 1995; Lawless, 1995). In principle, it is understood
that experts are those individuals who posses a highly
developed ability to recognize and evaluate sensory
properties and detailed technical information about
their companies products. For the most part, these
individuals have been successful and their activities have
constituted one of the earliest organised eorts in sensory
evaluation (Stone & Sidel, 1993).
One of the main problems posed by the use of experts
in the evaluation of product sensory quality is that the

344

E. Costell / Food Quality and Preference 13 (2002) 341353

qualication of expert is not well dened. As a result,

sometimes some people are considered experts when in
fact they are not and their personal opinion on a product
is erroneously identied as a valid mental standard for
assessing the quality of same. The period of education and
training is not clearly specied and the criteria used to
select experts may vary from case to case and from time to
time. Consequently the concept of individual quality,
dened by the mental standard, usually varies from one
expert to another. Recently Guinard, Yip, Cubero, and
Mazzucchelli (1999) have conrmed this fact by analysing the quality scores of dierent types of beer given by
several experts and observing that they clearly differed in their concept of quality. Perhaps one of the
reasons for these results lies in the lack of coincidence in
the training process of what the authors call brewing
experts to various degrees. As with other terms used in
sensory analysis, the ght against ambiguity should start
by clearly dening what expert means and establishing the minimum requirements in education and training that a person must possess to be qualied as such. It
is recommendable to rst consult the ISO Standard
(1994) on this subject. This Standard denes what a
selected assessor, an expert assessor and a specialised
expert assessor are (Table 1) and describes criteria for
choosing people with particular sensory skills. This
Standard oers principles and procedures for expanding
their knowledge and abilities to the levels required for
expert and for specialized expert assessors. The main
contribution of this Standard is that the experts, as
dened in it, will not only have the experience to be

considered a specialist in the product and/or the process

and/or marketing but they will also have a contrasted
physiological sensitivity and a wide knowledge of sensory evaluation techniques. Having these characteristics,
the experts are expected to use more homogeneous
quality criteria and to communicate their qualications
in more precise terms. However these advantages do not
solve the other problem, that of the lack of concordance
between the experts opinion and that of consumers.
However, in practice, the signicance of this lack of
concordance depends on the type of product and the level
of quality that is to be assessed. To rely on a mental standard to decide the quality level (acceptable or not, good
or better) of a widely consumed food product continues
to be risky. Yet, when it is a matter of dierentiating
between good quality and exceptional quality in certain
products (wine, coee, etc.), the assessment of quality by
real experts, in accordance with their mental standards, is
still considered to be a valid alternative.
2.1.1.3. Written standard. The elaboration of written
sensory standards to be used as references when determining a product quality must include denitions for
both critical attributes, the perceptible variations of
which depend on the raw materials and on the manufacturing process and the attributes that drive consumer acceptance. The type of standard will depend
on the quality level to be controlled, on the objective
of the control and on the type of product. The quality
level is important: it is not the same to develop a
standard designed to distinguish between acceptable

Table 1
Denition and characteristics of selected assesors, expert assessors and specialized expert assessorsa
Type of assessor

Denition

Selected assessor

Assessor chosen for his/her

ability to perform a sensory test

Expert assessor

Selected assessor with a high

degree of sensory sensitivity and
experience of sensory methodology,
who is able to make consistent and
repeatable sensory assesssments of
various products

Good consistency of judgements,

both within a session and from one
session to another
Good long-term sensory memory

Specialized expert
assessor

Expert assessor who has additional

experience as a specialist in the
product and/or the process and/or
marketing, and who is able to
perform sensory analysis of the
product and to evaluate or predict
eects of variations relating to raw
materials, recipes, processing,
storage, ageing, etc.

Extensive experience in the relevant

specialist eld
Highly developed ability to recognize
and evaluate sensory properties
Mental retention of reference standards
Recognition of key attributes
Deductive skills which may be applied to
problem solving
Good ability to describe and communicate
conclusions or to take appropriate action

Based on ISO 85862 (1994).

Characteristics

E. Costell / Food Quality and Preference 13 (2002) 341353

and unacceptable products as it is to do so in order to

dierentiate between two acceptable products (which
one is of higher quality) or even to set a standard
applicable to dierentiate between high quality products
and those of optimal or exceptional quality. Evidently,
the diculties increase as the quality level increases
because the number of critical attributes grows and their
selection becomes more complex as quality levels are
higher. Frequently it is dicult to locate and describe
the dierence between a good quality product and a
better quality one mainly because of the small dierences found (Powers, 1981). This problem has not yet
been satisfactorily solved but it is undoubtedly of the
highest interest. According to Cardello (1997), establishing the relationship between sensory responses and
the pleasure associated with food is one of the most
important and practical contributions that sensory
science can make to the study of food
In the development of a quality standard the specic
objective of the quality control programme should be
taken into account. It is not the same when the objective is
to design a sensory quality control within either a public
or private control organisation or to control the quality of
products to be included in a specic Designation of Origin
or to control the quality of an industrial food product to
compete with other products in the market.
In the rst case the aim of the programme is to ensure
that no inadequate products reach the consumer. Here
the term quality is equivalent to absence of defects.
Regulatory quality should reect the minimum acceptable quality and form a base from which individual
companies can develop their standards. The appropriate
standard must then include a description of the most
common defects in the product, comprising those
defects due to inadequate characteristics of the raw
materials used or to the process conditions, those
resulting from incorrect or prolonged storage or even
those derived sporadically from accidental causes. The
description of the standards used by the Inspection
Branch of the Canadian Department of Fisheries and
Oceans to determine the sensory quality of sh, as
reported by York (1995), constitutes a good example of
criteria to be applied in the development of this type of
standard. In this case, the regulatory denition of quality
should ensure food safety and should be a reection of
consumer expectation of minimum acceptable quality.
This regulatory sensory quality is reduced to three specic measurable characteristics: taint, decomposition
and unwholesomeness. As stated by York, Consumers
have other concepts of quality such as product form,
species and processing conditions (e.g. fresh vs. frozen)
which are outside the mandate of regulatory quality. A
dierent approach has been used by various European
Fisheries Research Institutes to develop an accurate and
objective method for the determination of sh freshness
considering that freshness is a critical quality parameter

345

of this product. The Quality Index Method (QIM) is

based on objective evaluation of the key sensory attributes of each sh species using a points scoring system
(from 0 to 3). The lower the total score, the fresher the
sh. QIM procedures for 12 sh species have now been
developed. As an example, a QIM scheme for cod is
shown in Fig. 1. It is expected that the QIM will become
the leading reference method for the quality assessment
of fresh sh within the European Community (www.qim.eurosh.com). Another option is that chosen by the
International Olive Oil Council (COI) to dene the
quality standard for virgin olive oil. This organization
has recently proposed a revised method for the organoleptic assessment of virgin olive oil (COI, 1996) with
the purpose of determining the criteria needed to assess
the avour characteristics of this product and developing the methodology for its classication according to
the intensity of the perceptible defects. In this case, a
prole sheet was dened including negative (defects)
and positive attributes and unstructured continuous
scales for measuring their intensity were incorporated
(Fig. 2).
When the objective is to control the sensory quality of
products of a Designation of Origin (EEC Council,
1992), the approach is dierent. According to Bertozzi
(1995), the denomination of a product marked with the
geographic name of the zone in which it is produced
includes information on the manufacturing process and
on product characteristics. In this context, it is necessary
to furnish objective methods to certify the typicity of
every production in such a way that it can be dierentiated in comparison with imitations. For these reasons,
in this case, apart from considering the possible presence of common or sporadic defects in the product, the
standard must include not only the attributes dening
its sensory prole and those aecting acceptability but
also the attributes which can establish dierences with
other similar products from other designations of origin.
The latter additional attributes may not be necessary
because frequently the dierences between designations
lie in dierences in intensity of the same attributes
rather than in dierent attributes. The development of
this type of standard involves a lot of time consuming
work including the collection and initial screening of a
great number of dierent samples, representative of the
variability of the products belonging to the designation
and also the generation and selection of the attribute
descriptors. From the results of the required descriptive
analysis a sensory prole is nally dened which serves
as the specic standard for the designation. An example
of this type of standard is the Guide to the sensory evaluation of the texture of hard and semi-hard ewes milk
cheeses (EUR, 1999). This guide includes the attributes
to be evaluated, their physical and sensory denitions,
their evaluation techniques and seven point intensity
scales, of which three points are xed by a standard

346

E. Costell / Food Quality and Preference 13 (2002) 341353

Fig. 1. Quality Index Method (QIM) scheme for cod.

reference product. Denitions, evaluation technique and

scale for friability evaluation can be seen in Fig. 3.
Finally, the development of a quality standard for
commercial food products can follow a scheme almost
similar to the above mentioned type of standard. The difference lies in the fact that in this case dening the quality
standard requires the consideration of several points such
as marketing objectives, production variability, attributes
that vary, attributes that drive consumer acceptance,
manufacturing conditions and available resources.

2.1.2. Sensory specications

Broadly speaking, a sensory specication is designed
to determine the acceptable or tolerable variation in a
product with reference to a previously selected product
or an established written standard. In the latter case the
specication denes the range of intensities accepted or
tolerated for each of the attributes or the range of
defects included in the written standard. Specications
can be set based on managements criteria alone and/or
on consumer response. The second option provides

E. Costell / Food Quality and Preference 13 (2002) 341353

347

Fig. 2. Prole sheet for the organoleptic assessment and classication of virgin olive oil.

more realistic specications because here the inuence

of the variability of the product or of each considered
attribute on consumer acceptance is taken into account.
A product test for establishing a sensory specication
includes:
1. Selection of a group of samples showing dierent
sensory properties and representing the actual
variability in the marketplace. In some cases it is
convenient to add samples showing an especially
important defect.
2. Evaluation of the perceptible dierence/s
between each sample and the standard either by
direct comparison or by means of descriptive
analyses in which the magnitude of the defects
and/or attributes are evaluated.
3. Evaluation of the acceptability of samples by a
large consumer panel.

4. Analysis of the relationship between the variability of the attributes or the product and the
variability in consumer acceptability.

The main information thus obtained will show for

which attributes, their variability inuences consumer
acceptance. It must be accepted that variability in the
intensity of some attributes may not aect acceptability.
Furthermore the extent of the variability in an attribute
is not necessarily related to the magnitude of its eect
on acceptability. With this information and the particular
criteria used by the institution or company a denite sensory specication can be established. This specication
includes not only those attributes aecting acceptability
but also all those proposed by the responsible organisation according to its particular understanding of quality
for the product studied.

348

E. Costell / Food Quality and Preference 13 (2002) 341353

Fig. 3. Denitions, evaluation technique, scale and standard reference products for friability evaluation of hard and semi-hard ewes milk cheeses.

In any case, the development of standards and specications is neither an easy nor a quick task. On many
occasions the results obtained in the rst study are not
satisfactory and the initially proposed standard or specication must be modied. On the other hand, variations produced in the market because of changes in
consumer preferences or habits, degree of exigency, new
trends, or even changes produced in the market structure when new products are introduced, must be followed. The validity of standards or specications may
vary with time and must then be periodically updated to
adapt them to market variations.
2.2. Selection of methods
Following the described procedure, the application of
sensory methods to the development of standards and
the establishment of sensory quality specications present no special problems. The objectives, the experimental designs, the testing conditions, the number of
assessors and their level of training, the criteria for the
selection of consumer panels and the statistical analysis
of data are well dened in many recent texts (ASTM,
1996, 1997; MacFie & Thomson, 1994; Moskowitz,
1994; Lawless & Heymann, 1998; Meilgaard et al.,
1999). The problem arises when, once the quality standard has been established and the specication of a
product dened, it is necessary to use sensorial methods
in order to decide if the product meets the requirements
set or not. In principle, the most suitable sensorial
methods are those which make it possible to measure
the magnitude of variability between a product and a
previously dened standard (intensity scales, quality
rating or dierence from control method) while the dif-

ference or aective tests are not appropriate for the

routine evaluation of products quality. Dierence tests
are too sensitive to relatively small dierences and do
not determine the extent of the dierence and the
determination of the preferences of a small group of
assessors does not represent the consumer population.
In practice, the selection of the method to be used will
depend on the objectives set and the characteristics of
the products to be evaluated. For example, as Munoz et
al. (1992) comment, if the variable attributes are limited
to ve to ten key attributes, the comprehensive descriptive method is feasible but when the product variability
is not easily dened by specic sensory attributes, but
can be more readily reected in the broad sensory
parameters (appearance, avor, texture) the quality rating method is a likely method of choice. In those cases,
when variation cannot be specically dened by sensory
attributes or when examples of unacceptable product
cover a multitude of sensory conditions, the in/out
method is recommended.
To sum up, in each particular case, the choice of sensorial method should be made taking the following criteria
into account:

1. The objective of the quality control programme

2. The type of standard previously established
3. Whether or not the perceptible variability of a product can be dened by specic sensory attributes
and, if so, the number of parameters or sensorial
attributes necessary to do so.
4. The magnitude of perceptible variability that
must be detected
5. The level or levels of quality to be assessed.

E. Costell / Food Quality and Preference 13 (2002) 341353

Applying these criteria it is possible to select the

most suitable sensorial method to obtain the sensorial
information needed in a timely and cost-eective way.

3. Sensory methods in quality control

As has been commented above, various publications
have proposed dierent types of sensorial methods
which can be used in sensory quality control of food
products. According to Munoz et al. (1992), they can be
classied into eight groups: Overall dierence tests;
Dierence from control; Attribute or descriptive tests;
In/out of specications; Preference and other consumers
tests; Typical measurements; Qualitative description of
typical production and Quality grading. The problems
associated with using some of these methods have
already been mentioned, such as the dierence or consumer tests but other test types also present serious
limitations in their approach as is the case, for example,
with methods which have the objective of classifying a
product as typical or atypical giving no information as
to the reason for the classication given. But apart from
the particular limitations of each of the methods, in
many cases, the lack of validity of results may not be
attributable to the method itself but to a defective realisation of the test and/or to an incorrect analysis of the
information obtained. The same sensorial method used
with a correctly established and well dened standard or
applied directly, relying on the individual criteria
regarding product quality held by a small group of
company employees or a group of selected and trained
assessors may render results of dierent validity.
3.1. Methods involving a comparison to a standard
The objective of these methods is to evaluate the differences between the product and the corresponding
standard. This involves the clear denition of terms
used and of the experimental test conditions, the design
of a score card, the selection and training of a panel and
the selection of the method to be used to analyse data
obtained.
3.1.1. Dierence from a standard or control product
There are several ways to examine the dierences from
a standard product. The simplest one is to evaluate the
overall degree of dierence using a single scale (rating,
category or unstructured) with the extremes labelled
no dierence and extreme dierence (Fig. 4a). It is
an easy and fast method, useful when the analysed product does not have complex sensory characteristics. Its
objective is to distinguish between the samples showing a
tolerable dierence from the standard and those for which
the dierence is greater than the tolerance established in
the corresponding specication. It is recommended that

349

the nal decision be taken by the person responsible for

quality control, according to the scores given by the
judges. The judges should centre their attention on the
magnitude of the perceptible dierences. The responsibility to decide may have a psychological inuence on
the evaluation of the dierences. Another source of
inuence may be the knowledge of the specication by
the judges. To compensate for the latter it is common
practice to introduce a blind sample of the standard
product to be compared with the declared standard.
This a useful method in public or government organisations, where the objective is to separate samples of
low quality. In industrial quality control, this method
lacks the capability of giving information on the nature
of the dierence, necessary to identify the cause and
correct the dierence (Aust, Gacula, Bearm, &
Washam, 1985; Munoz et al., 1992).
A more informative method consists in selecting the
most important attributes in the product sensory quality
and evaluating the dierences from the standard for each
attribute (Fig. 4b). A dierence higher than the specied
tolerance in any attribute will mean that the product is
out of specication. With this information, corrections
can be introduced when necessary.
Still, this method detects the magnitude of the dierences in attributes but not their direction. A possible
alternative, successfully used in some cases (Costell,
unpublished data) is to design a scale similar to the
just-right scale, in which the central point corresponds to the standard product (Fig. 4c). With this scale
information is obtained not only on the magnitude but
also on the direction of the dierences from the standard in the dierent attributes. This procedure may be
of interest when the objective is to evaluate the eect of
a change in the formulation of a certain product on its
sensory quality and the direction of the possible change
in any quality attribute is not predictable. Besides the
type of scale used, the quality of the obtained information depends on the degree of training of the judges and
their knowledge of the product, on the realisation conditions and on the correct analysis of data as a function
of the type of scale (ordinal or interval scale) used. A
large trained panel (3040) is appropriate when only the
degree of dierence from the control is to be evaluated.
When additional information about dierences on specic attributes is required a smaller and more highly
trained panel is recommended.
3.1.2. Dierence from a mental standard
As commented above, the use of a mental standard by
one or several experts to dene the quality of a food
product presents two serious problems, derived from the
possible dierence between the mental standards used by
the experts and from the fact that their opinions are not
representative of consumers opinion. In principle, based
on these considerations, it would not be recommendable

350

E. Costell / Food Quality and Preference 13 (2002) 341353

Fig. 4. Dierent types of scales for: (a) overall dierence from standard product ratings; (b) dierence from control for selected attributes and (c)
directional diference from control for selected attributes.

to use this method to evaluate the quality of certain

products. But this position must be reconsidered. In
cases when an expert or a group of experts, showing
recognised ability to evaluate the magnitudes of the
perceptible dierences between products and a profound knowledge of the product and its manufacturing
process is available there are situations in which their
performance is not only admissible but recommendable.
One of these situations is when the characteristics of the
product will not be directly evaluated by the consumer,
such as, when dealing with raw materials or ingredients
or when only previous information is sought on the
eects of formulation, process or storage conditions on
the product quality. Another situation where quality
evaluation by experts is appropriate is when the objective is to evaluate dierences between quality grades of
products of exceptional sensorial characteristics, such as
wine, coee or olive oil, in which small dierences
between high quality levels may be decisive in their

market price. These dierences can hardly be detected

by naive consumers.
3.1.2.1. In/Out method. This is the simplest method to
compare product quality with a mental standard by
experts. It is mainly used to identify products that show
clear deviations (presence of o-notes or other defects)
from the normal production. It can be recommended
for the evaluation of raw materials or relatively simple
nished products. Its advantages are the simplicity and
the direct use of results obtained. The main disadvantage
is its inability to provide descriptive information and
therefore its lack of direction and actionability to x
problems (Munoz et al, 1992). The validity of the
information provided depends on whether the experts
are indeed genuine experts. If this method is used by one
or a small group of people in a company who do not
possess the necessary expertise, each of them makes
decisions based on his individual experience and on his

E. Costell / Food Quality and Preference 13 (2002) 341353

product knowledge. This situation leads to highly variable

and subjective information.
3.1.2.2. Overall quality rating method. This method
consists in assessing the quality of a product according
to an established quality criteria. Samples are rated
using a single quality scale anchored very poor and
excellent. A product is rejected when the quality ratings are low. Initially it was considered that a quality
rating test represents a combination of aective and
descriptive tasks (Sidel, Stone, & Bloomquist, 1981).
Besides this duality, other problems arise in data treatment because the quality scores are not clearly based on
psychophysical measurements (Lawless, 1994). Another
approach consists in considering the product quality as
an integrated impression like acceptability or pleasure
experienced when consuming a food or drink. Using
this criterion, the evaluation of a product quality with a
unidimensional scale may appear logical. This implies
accepting that quality and acceptability are not concepts
of an exclusively sensorial character (Costell, 2001). As
stated by Cardello (1997): In psychological terms,
pleasure and displeasure, liking and disliking, are not
sensory phenomena, although they accompany most
sensory stimuli. Rather, pleasure and displeasure are
emotional experiences. They are conscious cognitions
that accompany the somatic eects of emotions. Based
on this approach it can be accepted that a group of
experts, sharing a common mental standard, may successfully judge a product quality grade. From a quality
control standpoint, this method has the disadvantage of
producing an integrated judgement, that may not be
actionable and useful for product documentation or
guidance.
In an eort to overcome the above mentioned problem, other methods have been proposed, in which a
scale to evaluate overall quality and other scales to
evaluate the attributes quality or their intensities are
included in the same scorecard (Munoz et al., 1992).
This scheme is apparently similar to that described
above to evaluate the perceptible dierences from a
control product but the situation is not the same. Even
assuming that the experts possess a solid mental standard of the product quality it is hard to make them pay
attention to the overall quality and to the quality or
intensity of attributes in the same session. In this way
they are obliged to perform functions that require different mental attitudes which can produce erroneous
results. It should not be recommended that experts use
this type of scorecard to evaluate a product quality.
3.1.3. Dierence from a written standard
These types of tests are among the most frequently used
in quality control. Essentially they consist in the evaluation of the intensity of dierent attributes and/or defects
or the evaluation of quality grade using a scorecard. The

351

information gathered during the previous development

of the standard and the establishment of the specication is collected in the scorecard, according to dierent
criteria and in dierent ways. Several alternative procedures have been used but, practically, only one of them
is in use at present: the quality grading test.
3.1.3.1. Quality grading method. This method has been
one of the most popular sensory tests used in quality
control and consists in developing a scorecard that
includes a scoring system with points assigned for each
grade and a description of sensory characteristics dening quality for each grade. The scorecard is composed
of ordinal scales using discrete numbers and contains
the description of the characteristics. The scale amplitude may be 3, 6 or 9 points. The upper third of the
scale includes a detailed description of the intensity of
each attribute corresponding to a high level of quality,
the medium third the description corresponding to an
acceptable quality and the lower third that corresponding to rejectable quality. Frequently a scale is designed
for each basic sensory attribute, e.g. appearance, colour,
avour and texture. The judges give scores to each
attribute and when a product is assigned a score in the
lower third of the scale, it should be rejected (ISO,
1987). This test allows for a rapid qualication of the
product and for the detection of the possible causes of
rejection. However this test requires a group of very well
trained judges that can correctly interpret the descriptions corresponding to each of the quality grades for
each of the selected attributes. An important problem
here is that the judges are obliged to carry out an analytical job simultaneous to a qualication which may
produce deviations in the results, as commented above.
Finally it should be taken into account that the data
from this test are of ordinal nature and this fact leads
to the use of non-parametric statistical methods for
analysing the data obtained.
3.2. Methods of evaluation without a standard
3.2.1. Descriptive method
This method consists in having a well-trained sensory
panel that provides data on a set of the products sensory attributes. During the initial development of the
sensory standard a number of attributes are selected.
Some of them have been selected because their variations aect product acceptability by consumers and
some others may be introduced by the industrial company on the basis of their interest in connection with the
identity and/or desired image of the product in the
market. In the establishment of the corresponding specication their tolerable variability is xed. Evaluation
of quality with this descriptive method consists basically
in the evaluation of the intensity of each attribute by a
trained panel using descriptive proling (conventional

352

E. Costell / Food Quality and Preference 13 (2002) 341353

prole, QDA, Spectrum). The person responsible for

quality control then studies the results obtained from
the statistical analysis of experimental data and makes
the nal decision based on the sensory specication
previously established. In this case, the specications are
represented by the range of intensities tolerated for each
attribute. Products whose intensity on any given attribute fall outside specications are considered unacceptable. For example, a company wants to assess the
quality of the virgin olive oil it produces, in accordance
with the standard proposed by the COI (Fig. 2). The
rst step should be the selection and training of the
panel. The training comprises the denition, evaluation
procedures and magnitude scoring of each of the attributes and of the defects included in the scorecard (COI,
1996). Once it has been ensured that the panel works
well, the comparison between the panel results and the
specications set for each of the product attributes can
be used to make decisions regarding the product quality. As stated by Munoz et al, (1992), the two main
advantages of this approach are the absence of any
subjectivity in the evaluation and the quality of the data
obtained. The main disadvantages are the time and cost
necessary to train and calibrate the panel and the time
necessary to perform the test and to analyse the data.
This test and the corresponding data analysis can be
simplied by using the software available (Punter,
1994). The method described is not suitable for solving
some particular problems that require an immediate
decision. In this case one possibility may be to perform
a reduced version of the prole. Once the panel is
trained on the whole prole (10 to 15 attributes), a small
group of judges may be selected to evaluate the most
important attributes (45). This simplication may
allow its use in daily quality control.
3.2.2. Other methods
Many of the merited criticisms of sensory methods
used in quality control originate from the lack of a previously developed standard or an established specication. It should also be considered that the standards and
specications are developed for a specic situation
(industry, public or private organisation, etc). The use
of some methods (In/Out, Quality Rating, etc.) without
a previous standard or specication aord results of
doubtful validity. It is especially important to note that
the development of a quality grading system without a
previous study of the relations between the variations in
attributes and product acceptance may lead to the construction of scorecards without any practical value. It is
also important to point out that the use of these methods in food research to compare products or to study
the eects of processing conditions, must be avoided.
The evaluation of the sensory quality of any product
by a group of 10 to 20 more or less trained panellists in
a laboratory without a standard or specication has

the same doubtful value as their opinion on product

acceptance.
Finally something must be said about the quality
evaluation methods based on what is known as complete scorecard. These scorecards include evaluations
for dierent sensory categories such as appearance, avour, texture, etc. as well as for some specic attributes
like sourness or viscosity and a variable number of
quality points is assigned to each one of them. The
sum of points obtained determines the product quality.
Another, alternative, method consists in assigning
scores to the intensity of dierent attributes, multiplying
them by dierent factors according to their importance
and adding them up to get a product qualication. One
of the better known methods of this type is the U.C.
Davis 20-point wine scoring system described in 1981 by
Amerine and Roessler (Lawless, 1995). These methods
were once very popular and were adopted by some
industrial rms and control organisations. They would
appear to make it possible to express the quality of a
product with a single number. But in practice they present several problems. They have been criticised on
many occasions because the weight of each attribute has
been arbitrarily assigned and the product quality is
taken as the sum of the qualications given to a limited
number of attributes. On the other hand, the scales used
to evaluate the intensity of the dierent attributes do
not always have sensorially equivalent magnitudes. This
also means that the validity of the information obtained
when each score is multiplied according to a previously
established weighting factor is questionable, even when
this factor has not been established in an arbitrary
manner. For all of these reasons this type of test is not
considered recommendable for assessing the quality of a
product.

4. Concluding remarks
In accordance with what has been stated above, we
can conclude that not all methods proposed for evaluating the sensorial quality of food products are suitable
for incorporation in quality control programmes. Difference or preference tests, typical measurement or
those methods based on complete scorecards are the
less appropriate while dierence from control methods
and descriptive methods, are the most sound sensory
tests for quality control purposes. Others methods such
as In/Out, Quality rating and Quality grading methods
may be used in particular situations. The characteristics
of each product, the degree or level of quality that it is
wished to control and the resources available condition
the choice of method to be used. On the other hand, it
should be borne in mind that designing an eective
programme for testing the sensorial quality of a product
is based on the following points: (a) The selection of the

E. Costell / Food Quality and Preference 13 (2002) 341353

sensory quality standard; (b) The establishment of the

sensory specication; (c) The selection of a method to
evaluate dierences between the product and the corresponding standard and; (d) The selection, training and
maintenance of the panel. The practical value of the
information obtained will be determined by the correct
fullment of these requirements.

Acknowledgements
To Ministerio de Ciencia y Tecnologa of Spain (Project AGL 20001590). The author acknowledges Dr.
Luis Duran for revision of the manuscript and helpful
observations and Alejandra Munoz for constructive
comments.

References
ASTM. (1996). Sensory testing methods: Second Edition. MNL 26.
Philadelphia: American Society for Testing and Materials.
ASTM. (1997). Relating consumer, descriptive and laboratory data.
MNL 30. Philadelphia: American Society for Testing and Materials.
Aust, L. B., Gacula, M. C., Beard, S. A., & Washam II, R. W. (1985).
Degree of dierence test method in sensory evaluation of heterogeneous product types. Journal Food Science, 50, 511513.
Bertozzi, L. (1995). Designation of origin: quality and specication.
Food Quality and Preference, 6, 143147.
Booth, D. A. (1995). The cognitive basis of quality. Food Quality and
Preference, 6, 201205.
Cardello, A. V. (1995). Food Quality: conceptual and sensory aspects.
Food Quality and Preference, 6, 163168.
Cardello, A. V. (1997). Pleasure from food: its nature and Role in
Sensory Science. Cereal Foods World, 42, 550552.
COI (1996). Organoleptic assessment of virgin olive oil. COI/T.20/
Doc.no 15/Rev.1. International Olive Oil Council.
Costell, E., & Duran, L. (1981). El analisis sensorial en el control de
calidad de los alimentos. Revista de Agroqumica y Tecnologa de
Alimentos, 21, 110.
Costell, E. (2000). Analisis sensorial: evolucion, situacion actual y
perspectivas. Industria y Alimentos Internacional, 2, 3439.
Costell, E. (2001). La aceptabilidad de los alimentos. Nutricion y placer. Arbor, 661, 6585.
EEC Council (1992). Council Regulation 2081/92. 14 July 1992. Ocial Journal of the European Community. Luxemburg.
Ennis, D. M. (1993). The power of sensory discrimination methods.
Journal Sensory Studies, 8, 353370.
EUR (1999). A guide to the sensory evaluation of the texture of hard
and semi-hard ewes milk cheeses. No 18829. Ocial Publications of
the European Communities. DG XII, Brussels.
Fisken, D. (1990). Sensory quality and the consumer: viewpoints and
directions. Journal Sensory Studies, 5, 203209.
Gould, W. A., & Gould, R. W. (1988). Total quality assurance for the
food industry. Baltimore: CTI Publications Inc.
Guinard, J. X., Yip, D., Cubero, E., & Mazzucchelli, R. (1999).

353

Quality ratings by experts, and relation with descriptive analysis

ratings: a case study with beer. Food Quality and Preference, 10, 59
67.
Herschdoerfer, S. M. (1986). Quality control in the food industry. New
York: Academic Press.
ISO. (1987). Sensory analysis. Methodology. Evaluation of food products by methods using scales. International Standard no. 4121.
Switzerland: International Organization for Standardization.
ISO. (1994). Sensory analysis. General guidance for selection, training
and monitoring of assessors. Part 2: Experts. International Standard
no. 85862.. Switzerland: International Organization for Standardization.
Juran, J. M. (1974). Quality Control Handbook (3rd ed.). New York:
McGraw Book Co.
Kramer, A. (1959). Glossary of some terms used in the sensory (panel)
evaluation of foods and beverages. Food Technology, 13, 733738.
Kramer, A., & Twigg, B. A. (1970). Quality control for the food industry (3rd ed.). Westport. Connecticut: The AVI Publishing Co.
Lawless, H. T. (1994). Getting results you can trust from sensory evaluation. Cereal Foods World, 39, 809814.
Lawless, H. T. (1995). Dimensions of sensory quality: a critique. Food
Quality and Preference, 6, 191199.
Lawless, H. T., & Heymann, H. (1998). Sensory evaluation of food.
New York: Chapman & Hall. International Thomson Publishing.
Lardmond, E. (1994). Is Sensory Evaluation a Science? Cereal Foods
World, 39, 804808.
MacFie, H. J. H., & Thomson, D. M. H. (1994). Measurement of food
preferences. London: Blackie Academic & Professional.
Meilgaard, M., Civille, G. V., & Carr, B. T. (1999). Sensory evaluation
techniques (3rd ed.). Boca Raton. Florida: CRC Press.
Molnar, P. J. (1995). A model for overall description of food quality.
Food Quality and Preference, 6, 185190.
Moskowitz, H. R. (1983). Product testing and sensory evaluation of
foods. Westport, Connecticut: Food & Nutrition Press.
Moskowitz, H. R. (1993). Sensory analysis procedures and viewpoints:
Intellectual history, current debates, future outlooks. Journal of
Sensory Studies, 8, 241256.
Moskowitz, H. R. (1994). Food concepts and products. Just-in time
development. Trumbull, Connecticut: Food & Nutrition Press, Inc.
Munoz, A. M., & Chambres IV, E. (1993). Relating sensory measurements to consumer acceptance of meat products. Food Technology,
47, 128131, 134.
Munoz, A. M., Civille, G. V., & Carr, B. T. (1992). Sensory evaluation
in quality control. New York: Van Nostrand Reinhold.
Powers, J. J. (1981). Multivariate Procedures in Sensory Research:
Scope and limitations. MBAA Technical Quarterly, 18, 1121.
Punter, P. H. (1994). Software for data collection and processing. In
J. R. Piggott, & A. Paterson (Eds.), Understanding natural avors
(pp. 97111). London: Blakie Academic &Professional.
Schlich, P. (1993). Risk tables for discrimination tests. Food Quality
and Preference, 4, 141151.
Stauer, J. E. (1988). Quality Assurance of Foods. Westport, Connecticut: Food & Nutrition Press, Inc.
Sidel, J. L., Stone, H., & Bloomquist, J. (1981). Use and misuse of
sensory evaluation in research and quality control. Journal Dairy
Science, 64, 22962302.
Stone, H., & Sidel, J. L. (1993). Sensory Evaluation Practices ((2nd
edition.)). New York: Academic Press, Inc.
York, R. K. (1995). Quality assessment in a regulatory environment.
Food Quality and Preference, 6, 137141.