A Comparison of Sensory Methods in Quality Control

8/10/2019 A Comparison of Sensory Methods in Quality Control

1/13

A comparison of sensory methods in quality controlE. Costell*

Instituto de Agroqu mica y Tecnolog a de Alimentos (CSIC) PO Box 73, 46100 Burjassot, Valencia, Spain

Accepted 8 February 2002

Abstract

Many different types of sensorial methods have been proposed and used to evaluate and control the sensory quality of foods.

However, not all of them are suitable for incorporation in to quality control programmes. To simplify comparison a distinction isproposed between methods that can be used to dene sensory specications or to select a product quality standard and those thatcan be used to check if a product complies with stated requirements. With this approach, the appropriateness and limitations of different methods and their practical applicability, according to their use with or without a previously selected or developed stan-dard (product, mental or written), are discussed. # 2002 Elsevier Science Ltd. All rights reserved.

Keywords: Sensory quality; Sensory analysis; Quality standards; Quality specications

1. Introduction

The term quality has been used so much and in somany contexts that its meaning is frequently unclear. A

number of denitions have been proposed, always withreference to the situation or problem to be solved in eachcase. They vary widely between simple expressions such asFitness for use (Juran, 1974) to more detailed ones likethat proposed by Molnar (1995): The quality of foodproducts, in conformity with consumers requirementsand acceptance, is determined by their sensory attri-butes, chemical composition, physical properties, levelof microbiological and toxicological contaminants,shelf-life, packaging and labelling. Any of these ormany other denitions could be useful in a certain con-text but none of them is always satisfactory. QuotingFisken (1990), quality is a fuzzy and relative term andit is in a constant motion. Due to this lack of con-ceptual denition, any specication, method or group of methods designed to control the quality of a certainproduct may be applicable in a particular situation butthey are subject to a constant evolution. The changescome, on the one hand in function of methodologicaladvances in each area (chemical analysis, microbiology,toxicology, etc.) and on the other, of changes undergone

by market requirements and degree of commercialcompetition for the particular food product.

Notwithstanding the lack of conceptual denition of quality, food quality control and assurance is evidently

a top subject both in industry and in public and privatecontrol institutions (Gould & Gould, 1988; Herschdor-fer, 1984; Juran, 1974; Kramer & Twigg, 1970; Stauffer,1988) and continues to be a matter of discussion in bothacademic and industrial forums. Some of the most rele-vant items, advances and problems related to the deni-tion and measurement of quality of foods have beendiscussed in a Special Issue of Food Quality and Preferencepublished in 1995.

Basically the utilisation of any type of method in foodquality control follows a common approach: rst, de-nition of specications or quality standards and second,development and testing of methods to evaluate, in areliable manner, whether or not a product complies withthe requirements of the quality standards. Two ques-tions arise with this approach: (1) Which food char-acteristics or properties should be included in thestandard? (2) Which methods are to be used for theiranalysis or evaluation? The answer to the rst questionis usually conditioned by the need to establish a com-promise between two extreme alternatives: either con-sider a large number of the food samples or a long list of food properties, leading to a rather complete specica-tion but difficult to apply in practice, or else, select onlythose characteristics of higher incidence on quality that

0950-3293/02/$ - see front matter # 2002 Elsevier Science Ltd. All rights reserved.P I I : S0950-3293(02 )00020-4

Food Quality and Preference 13 (2002) 341353www.elsevier.com/locate/foodqual

* Tel.: +34-96-3900022; fax: +34-96-3636301.E-mail address: [email protected] (E. Costell).


2/13

make it possible to decide if the food fulls the require-ments of a certain quality grade in a simpler manner.On answering the second question a similar situationholds: not always the most precise and costly methodsare most suitable but, in general, the selection is basedon the capacity of the method to measure variations in

each of the characteristics that inuence product qualitywith sufficient precision.The implementation of food quality control and

assurance systems, in the areas of chemical composition,microbiological and toxicological safety, and nutritionalcharacteristics, brings up problems related to the selec-tion of properties or characteristics to be measured andto the methods to be used. These problems are muchmore numerous when the system is designed to controlwhat is known as sensory quality. Sensory quality iseven more difficult to dene because it is linked not onlyto food properties or characteristics but to the result of an interaction between the food and the consumer.Besides, sensory evaluation is a rather recent discipline,as compared with others such as chemical or micro-biological analysis. It was born and slowly developed itsmethodology during the second half of the twentiethcentury (Costell & Dura n, 1981; Costell, 2000; Lar-mond, 1994; Moskowitz, 1993). As a consequence, notall methods developed and used by different researchteams at different times can today be considered ade-quate to evaluate and control the sensory aspect of quality.

The concept of sensory quality has changed with timesince it was dened by Kramer in 1959 as the compo-

site of those characteristics that differentiate amongindividual units of a product and have signicance indetermining the degree of acceptability of that unit bythe user. Some authors centre their attention on therst part of this denition. For them sensory quality isproduct oriented. Others emphasise the second part andconsider that sensory quality is consumer oriented. Inthe rst case, quality is considered as a conventiondeveloped by experts and it may therefore be consideredas constant over a limited period only (Molnar, 1995).With the second approach, quality is mainly related toconsumer acceptance and is context dependent (Car-dello, 1995). The product oriented approach may, insome cases, render results of doubtful practical validitysince it is assumed that the opinion of a group of expertsis representative of the reaction of the potential con-sumers of the product in question. But the secondapproach is not totally satisfactory either because if aspecication or standard has to be established in orderto dene the sensory quality of a certain food product, itis not sufficient to collect acceptability data that merelygive statistically signicant results (Booth, 1995). Inrelation to the latter point it should be considered that,according to Stone and Sidel (1993), when xing accep-table deviations of the magnitude of an attribute with

reference to a standard, one should consider not onlytheir direct effect on acceptability but also to whatextent such a variability may affect consumer condencein the product. It is evident that a constantly changingproduct is certain to affect consumer condence. Forexample, in a recent study (Costell, unpublished data),

to determine the inuence of some sensory attributes onthe acceptance of commercial chocolate milk beverages,no relationship was found between perceptible colourdifferences and consumer acceptance. Products of clearly different colours were equally acceptable. How-ever, it is evident that manufacturers should dene theircolour standard and control variability between lots toavoid the negative effect that perceptible differences willhave on the condence of regular consumers.

The recentness and slow development of the disciplineof sensory analysis is perhaps the cause of the lack of immediate responses to the real need for methods tomeasure and control sensory quality both in industryand in control organizations. Consequently, manymethods or systems, with variable scientic base, havebeen proposed for sensory quality control. Some of them are being used at the present time simply by inertiaor habit. Every quality control technician or group hastried to solve problems by themselves in the best possi-ble way, as, although some books on this matter areavailable (ASTM, 1996; Pangborn & Roessler, 1965;Lawless & Heymann, 1998; Moskowitz, 1983; Stone &Sidel, 1993), it was not until 1992 that Mun oz, Civille &Carr published the book Sensory Evaluation in QualityControl , the rst one exclusively dedicated to this specic

topic.For these reasons, when analysing the different alter-natives proposed and used by control organizations andby different industries, the rst impression is that thereis a great diversity of approaches, requirements, levelsof strictness and practical applicability and still today itis generally considered that the correct application of sensory methods requires a lot of time to carry out andto analyse data and that the number of qualied asses-sors is not always available. To simplify comparisonbetween methods, the distinction between the sensorymethodology to be used in the development of stan-dards and specications from that to be used to check if a product complies with their requirements is impor-tant. On one hand, when establishing standards andspecications, collecting data sets from different testsand relating them allows for powerful modelling of therelationships between physical process and ingredientvariables and the perceived attributes from descriptiveproling, and ultimately, consumer appeal, present noproblems (Mun oz & Chambers, 1993). In this case it isneither necessary nor convenient to use fast methods orto take quick decisions. On the other hand, in practicalquality control, fast methods using a few assessors areneeded in order to take quick decisions at a given

342 E. Costell/ Food Quality and Preference 13 (2002) 341353


3/13

moment. For these reasons, the author proposes to dif-ferentiate between methods that can be used to denespecications or to select a product quality standard andthose destined to decide whether a particular food itemfulls the requirements of the appropriate standard.

2. Preliminary steps

2.1. Selection or establishment of sensory qualitystandards or specications

2.1.1. Sensory quality standardsThe establishment or denition of the quality stan-

dards is the critical point in the implementation of aquality control programme. In practice, each companyor institution must dene the quality level to be con-trolled in a certain product and then develop a standardthat ts their objectives. When dealing with foods andwith sensory quality it is difficult, and often practicallyimpossible, to obtain a product or a series of productsshowing the same unaltered sensory characteristics dur-ing enough time to permit their use as reference items insubsequent comparisons. Fortunately, for some attri-butes, such as colour or appearance, quality standards(photographs or reproductions of the food in materialslike plastics or ceramic) have been used successfullywhen the product itself cannot be used, generally forreasons of sensory variation or alterations. In themajority of cases even this is not possible. This problemhas traditionally been solved in two ways: either relying

on the mental standard created by one or several experts,or developing a written standard, in which a descriptionof the main attributes is commonly included.

2.1.1.1. Product standard. As indicated above, the useof the same product as a standard in the evaluation of the quality of raw or processed foods is almost alwaysdifficult or impossible. However it is more frequentlyused in quality control of ingredients or of some rawmaterials. According to Mun oz et al. (1992) a controlstandard selected for quality purposes is referred to as aproduct that is used as a representation of certain char-acteristics (not necessarily the optimal) and a productthat can easily be obtained, maintained or reproduced.The criteria for choosing a product as a control stan-dard can be arbitrary or deliberate. In any case, beforeits selection, information must be obtained on the pro-duct variability and on its incidence on the sensoryquality of the nal food item. This implies the identi-cation and quantication of the sensory attributes of thestudied ingredient or raw material by using sensorydescriptive techniques (proles, QDA, Spectrum) andthe determination of those attributes that inuence thenal product quality assessment by consumers. Accep-table variation limits for each of the attributes should

also be established. An interesting point to considerwhen a real product is used as a quality standard is theestablishment of a clear methodology to substitute itwhen necessary. This need arises when the standardproduct is running out of stock or when the end of itsshelf-life is getting near. The new standard must be sen-

sorially identical to the previous one. This similarityshould be ascertained by means of sensory discriminatoryor difference tests, such as the triangle test. An impor-tant consideration here is that the objective of the sen-sory test is not to detect differences between samples butrather to establish that they are sensorially equivalent.In this case the analyst must determine what constitutesa meaningful difference by selecting the proportion of distinguishers and then select a small value of b-risk toensure that there is only a small chance of missing thatdifference if it really exists (Ennis, 1993; Meilgaard,Civille & Carr, 1999; Schlich, 1993).

2.1.1.2. Mental standard. One of the most controversialstrategies in sensory quality control is to assign a qualitylevel to a product with reference to a mental standarddeveloped by one or a group of experts or panellists.Criticisms have been based mainly on two aspects. Thelack of concordance between experts as to the mentalstandard applicable to a certain product and the fallacyof assuming that the opinion of the experts representsthat of the consumers. Mental standards should only becriticized if each expert or panellist operates under dif-ferent criteria or mental standards, and if panellists donot evaluate products uniformly. Therefore, when a

mental standard is to be used, panellists need to betrained on the criteria and product attributes that are toform the mental standard. This training provides valid-ity and reliability to the use of mental standards andthus contributes to a sound evaluation method. Inaddition, criteria and/or products that form the mentalstandard should periodically be reviewed to strengthenthe principles learned and the reliability of this practice.Mun oz et al. (1992) describe the procedures to form andteach sound mental standards to panellists for some of the QC/sensory methods that use these standards (e.g.in/out method)

Consumer opinion is affected by the context in whichthe food is examined and by the expectations that someexternal factors, such as brand or price, will exert (Car-dello, 1995; Lawless, 1995). In principle, it is understoodthat experts are those individuals who posses a highlydeveloped ability to recognize and evaluate sensoryproperties and detailed technical information abouttheir companies products. For the most part, theseindividuals have been successful and their activities haveconstituted one of the earliest organised efforts in sensoryevaluation (Stone & Sidel, 1993).

One of the main problems posed by the use of expertsin the evaluation of product sensory quality is that the

E. Costell / Food Quality and Preference 13 (2002) 341353 343


4/13

qualication of expert is not well dened. As a result,sometimes some people are considered experts when infact they are not and their personal opinion on a productis erroneously identied as a valid mental standard forassessing the quality of same. The period of education andtraining is not clearly specied and the criteria used to

select experts may vary from case to case and from time totime. Consequently the concept of individual quality,dened by the mental standard, usually varies from oneexpert to another. Recently Guinard, Yip, Cubero, andMazzucchelli (1999) have conrmed this fact by analys-ing the quality scores of different types of beer given byseveral experts and observing that they clearly dif-fered in their concept of quality. Perhaps one of thereasons for these results lies in the lack of coincidence inthe training process of what the authors call brewingexperts to various degrees. As with other terms used insensory analysis, the ght against ambiguity should startby clearly dening what expert means and establish-ing the minimum requirements in education and train-ing that a person must possess to be qualied as such. Itis recommendable to rst consult the ISO Standard(1994) on this subject. This Standard denes what aselected assessor , an expert assessor and a specialised expert assessor are (Table 1) and describes criteria forchoosing people with particular sensory skills. ThisStandard offers principles and procedures for expandingtheir knowledge and abilities to the levels required forexpert and for specialized expert assessors. The maincontribution of this Standard is that the experts, asdened in it, will not only have the experience to be

considered a specialist in the product and/or the processand/or marketing but they will also have a contrastedphysiological sensitivity and a wide knowledge of sen-sory evaluation techniques. Having these characteristics,the experts are expected to use more homogeneousquality criteria and to communicate their qualications

in more precise terms. However these advantages do notsolve the other problem, that of the lack of concordancebetween the experts opinion and that of consumers.However, in practice, the signicance of this lack of concordance depends on the type of product and the levelof quality that is to be assessed. To rely on a mental stan-dard to decide the quality level (acceptable or not, goodor better) of a widely consumed food product continuesto be risky. Yet, when it is a matter of differentiatingbetween good quality and exceptional quality in certainproducts (wine, coffee, etc.), the assessment of quality byreal experts, in accordance with their mental standards, isstill considered to be a valid alternative.

2.1.1.3. Written standard. The elaboration of writtensensory standards to be used as references when deter-mining a product quality must include denitions forboth critical attributes, the perceptible variations of which depend on the raw materials and on the manu-facturing process and the attributes that drive con-sumer acceptance. The type of standard will dependon the quality level to be controlled, on the objectiveof the control and on the type of product. The qualitylevel is important: it is not the same to develop astandard designed to distinguish between acceptable

Table 1Denition and characteristics of selected assesors, expert assessors and specialized expert assessors a

Type of assessor Denition Characteristics

Selected assessor Assessor chosen for his/herability to perform a sensory test

Expert assessor Selected assessor with a highdegree of sensory sensitivity andexperience of sensory methodology,who is able to make consistent andrepeatable sensory assesssments of

various products

Good consistency of judgements,both within a session and from onesession to anotherGood long-term sensory memory

Specialized expertassessor

Expert assessor who has additionalexperience as a specialist in theproduct and/or the process and/ormarketing, and who is able toperform sensory analysis of theproduct and to evaluate or predicteffects of variations relating to rawmaterials, recipes, processing,storage, ageing, etc.

Extensive experience in the relevantspecialist eldHighly developed ability to recognizeand evaluate sensory propertiesMental retention of reference standardsRecognition of key attributesDeductive skills which may be applied toproblem solvingGood ability to describe and communicateconclusions or to take appropriate action

a Based on ISO 85862 (1994).



5/13

and unacceptable products as it is to do so in order todifferentiate between two acceptable products (whichone is of higher quality) or even to set a standardapplicable to differentiate between high quality productsand those of optimal or exceptional quality. Evidently,the difficulties increase as the quality level increases

because the number of critical attributes grows and theirselection becomes more complex as quality levels arehigher. Frequently it is difficult to locate and describethe difference between a good quality product and abetter quality one mainly because of the small differ-ences found (Powers, 1981). This problem has not yetbeen satisfactorily solved but it is undoubtedly of thehighest interest. According to Cardello (1997), estab-lishing the relationship between sensory responses andthe pleasure associated with food is one of the mostimportant and practical contributions that sensoryscience can make to the study of food

In the development of a quality standard the specicobjective of the quality control programme should betaken into account. It is not the same when the objective isto design a sensory quality control within either a publicor private control organisation or to control the quality of products to be included in a specic Designation of Originor to control the quality of an industrial food product tocompete with other products in the market.

In the rst case the aim of the programme is to ensurethat no inadequate products reach the consumer. Herethe term quality is equivalent to absence of defects.Regulatory quality should reect the minimum accep-table quality and form a base from which individual

companies can develop their standards. The appropriatestandard must then include a description of the mostcommon defects in the product, comprising thosedefects due to inadequate characteristics of the rawmaterials used or to the process conditions, thoseresulting from incorrect or prolonged storage or eventhose derived sporadically from accidental causes. Thedescription of the standards used by the InspectionBranch of the Canadian Department of Fisheries andOceans to determine the sensory quality of sh, asreported by York (1995), constitutes a good example of criteria to be applied in the development of this type of standard. In this case, the regulatory denition of qualityshould ensure food safety and should be a reection of consumer expectation of minimum acceptable quality.This regulatory sensory quality is reduced to three spe-cic measurable characteristics: taint, decompositionand unwholesomeness. As stated by York, Consumershave other concepts of quality such as product form,species and processing conditions (e.g. fresh vs. frozen)which are outside the mandate of regulatory quality. Adifferent approach has been used by various EuropeanFisheries Research Institutes to develop an accurate andobjective method for the determination of sh freshnessconsidering that freshness is a critical quality parameter

of this product. The Quality Index Method (QIM) isbased on objective evaluation of the key sensory attri-butes of each sh species using a points scoring system(from 0 to 3). The lower the total score, the fresher thesh. QIM procedures for 12 sh species have now beendeveloped. As an example, a QIM scheme for cod is

shown in Fig. 1. It is expected that the QIM will becomethe leading reference method for the quality assessmentof fresh sh within the European Community (www.qi-m.eurosh.com). Another option is that chosen by theInternational Olive Oil Council (COI) to dene thequality standard for virgin olive oil. This organizationhas recently proposed a revised method for the orga-noleptic assessment of virgin olive oil (COI, 1996) withthe purpose of determining the criteria needed to assessthe avour characteristics of this product and develop-ing the methodology for its classication according tothe intensity of the perceptible defects. In this case, aprole sheet was dened including negative (defects)and positive attributes and unstructured continuousscales for measuring their intensity were incorporated(Fig. 2).

When the objective is to control the sensory quality of products of a Designation of Origin (EEC Council,1992), the approach is different. According to Bertozzi(1995), the denomination of a product marked with thegeographic name of the zone in which it is producedincludes information on the manufacturing process andon product characteristics. In this context, it is necessaryto furnish objective methods to certify the typicity of every production in such a way that it can be differ-

entiated in comparison with imitations. For these reasons,in this case, apart from considering the possible pre-sence of common or sporadic defects in the product, thestandard must include not only the attributes deningits sensory prole and those affecting acceptability butalso the attributes which can establish differences withother similar products from other designations of origin.The latter additional attributes may not be necessarybecause frequently the differences between designationslie in differences in intensity of the same attributesrather than in different attributes. The development of this type of standard involves a lot of time consumingwork including the collection and initial screening of agreat number of different samples, representative of thevariability of the products belonging to the designationand also the generation and selection of the attributedescriptors. From the results of the required descriptiveanalysis a sensory prole is nally dened which servesas the specic standard for the designation. An exampleof this type of standard is the Guide to the sensory eva-luation of the texture of hard and semi-hard ewes milkcheeses (EUR, 1999). This guide includes the attributesto be evaluated, their physical and sensory denitions,their evaluation techniques and seven point intensityscales, of which three points are xed by a standard



6/13

reference product. Denitions, evaluation technique andscale for friability evaluation can be seen in Fig. 3.

Finally, the development of a quality standard forcommercial food products can follow a scheme almostsimilar to the above mentioned type of standard. The dif-ference lies in the fact that in this case dening the qualitystandard requires the consideration of several points suchas marketing objectives, production variability, attributesthat vary, attributes that drive consumer acceptance,manufacturing conditions and available resources.

2.1.2. Sensory specicationsBroadly speaking, a sensory specication is designed

to determine the acceptable or tolerable variation in aproduct with reference to a previously selected productor an established written standard. In the latter case thespecication denes the range of intensities accepted ortolerated for each of the attributes or the range of defects included in the written standard. Specicationscan be set based on managements criteria alone and/oron consumer response. The second option provides

Fig. 1. Quality Index Method (QIM) scheme for cod.



7/13

more realistic specications because here the inuenceof the variability of the product or of each consideredattribute on consumer acceptance is taken into account.

A product test for establishing a sensory specicationincludes:

1. Selection of a group of samples showing differentsensory properties and representing the actualvariability in the marketplace. In some cases it isconvenient to add samples showing an especiallyimportant defect.

2. Evaluation of the perceptible difference/sbetween each sample and the standard either bydirect comparison or by means of descriptiveanalyses in which the magnitude of the defectsand/or attributes are evaluated.

3. Evaluation of the acceptability of samples by alarge consumer panel.

4. Analysis of the relationship between the varia-bility of the attributes or the product and thevariability in consumer acceptability.

The main information thus obtained will show forwhich attributes, their variability inuences consumeracceptance. It must be accepted that variability in theintensity of some attributes may not affect acceptability.Furthermore the extent of the variability in an attributeis not necessarily related to the magnitude of its effecton acceptability. With this information and the particularcriteria used by the institution or company a denite sen-sory specication can be established. This specicationincludes not only those attributes affecting acceptabilitybut also all those proposed by the responsible organisa-tion according to its particular understanding of qualityfor the product studied.

Fig. 2. Prole sheet for the organoleptic assessment and classication of virgin olive oil.



8/13

In any case, the development of standards and speci-cations is neither an easy nor a quick task. On manyoccasions the results obtained in the rst study are notsatisfactory and the initially proposed standard or spe-cication must be modied. On the other hand, varia-tions produced in the market because of changes inconsumer preferences or habits, degree of exigency, newtrends, or even changes produced in the market struc-ture when new products are introduced, must be fol-

lowed. The validity of standards or specications mayvary with time and must then be periodically updated toadapt them to market variations.

2.2. Selection of methods

Following the described procedure, the application of sensory methods to the development of standards andthe establishment of sensory quality specications pre-sent no special problems. The objectives, the experi-mental designs, the testing conditions, the number of assessors and their level of training, the criteria for theselection of consumer panels and the statistical analysisof data are well dened in many recent texts (ASTM,1996, 1997; MacFie & Thomson, 1994; Moskowitz,1994; Lawless & Heymann, 1998; Meilgaard et al.,1999). The problem arises when, once the quality stan-dard has been established and the specication of aproduct dened, it is necessary to use sensorial methodsin order to decide if the product meets the requirementsset or not. In principle, the most suitable sensorialmethods are those which make it possible to measurethe magnitude of variability between a product and apreviously dened standard (intensity scales, qualityrating or difference from control method) while the dif-

ference or affective tests are not appropriate for theroutine evaluation of products quality. Difference testsare too sensitive to relatively small differences and donot determine the extent of the difference and thedetermination of the preferences of a small group of assessors does not represent the consumer population.

In practice, the selection of the method to be used willdepend on the objectives set and the characteristics of the products to be evaluated. For example, as Mun oz et

al. (1992) comment, if the variable attributes are limitedto ve to ten key attributes, the comprehensive descrip-tive method is feasible but when the product variabilityis not easily dened by specic sensory attributes, butcan be more readily reected in the broad sensoryparameters (appearance, avor, texture) the quality rat-ing method is a likely method of choice. In those cases,when variation cannot be specically dened by sensoryattributes or when examples of unacceptable productcover a multitude of sensory conditions, the in/outmethod is recommended.

To sum up, in each particular case, the choice of sensor-ial method should be made taking the following criteriainto account:

1. The objective of the quality control programme2. The type of standard previously established3. Whether or not the perceptible variability of a pro-

duct can be dened by specic sensory attributesand, if so, the number of parameters or sensorialattributes necessary to do so.

4. The magnitude of perceptible variability thatmust be detected

5. The level or levels of quality to be assessed.

Fig. 3. Denitions, evaluation technique, scale and standard reference products for friability evaluation of hard and semi-hard ewes milk cheeses.



9/13

Applying these criteria it is possible to select themost suitable sensorial method to obtain the sensorialinformation needed in a timely and cost-effective way.

3. Sensory methods in quality control

As has been commented above, various publicationshave proposed different types of sensorial methodswhich can be used in sensory quality control of foodproducts. According to Mun oz et al. (1992), they can beclassied into eight groups: Overall difference tests;Difference from control; Attribute or descriptive tests;In/out of specications; Preference and other consumerstests; Typical measurements; Qualitative description of typical production and Quality grading. The problemsassociated with using some of these methods havealready been mentioned, such as the difference or con-sumer tests but other test types also present seriouslimitations in their approach as is the case, for example,with methods which have the objective of classifying aproduct as typical or atypical giving no information asto the reason for the classication given. But apart fromthe particular limitations of each of the methods, inmany cases, the lack of validity of results may not beattributable to the method itself but to a defective rea-lisation of the test and/or to an incorrect analysis of theinformation obtained. The same sensorial method usedwith a correctly established and well dened standard orapplied directly, relying on the individual criteriaregarding product quality held by a small group of

company employees or a group of selected and trainedassessors may render results of different validity.

3.1. Methods involving a comparison to a standard

The objective of these methods is to evaluate the dif-ferences between the product and the correspondingstandard. This involves the clear denition of termsused and of the experimental test conditions, the designof a score card, the selection and training of a panel andthe selection of the method to be used to analyse dataobtained.

3.1.1. Difference from a standard or control productThere are several ways to examine the differences from

a standard product. The simplest one is to evaluate theoverall degree of difference using a single scale (rating,category or unstructured) with the extremes labelledno difference and extreme difference (Fig. 4a). It isan easy and fast method, useful when the analysed pro-duct does not have complex sensory characteristics. Itsobjective is to distinguish between the samples showing atolerable difference from the standard and those for whichthe difference is greater than the tolerance established inthe corresponding specication. It is recommended that

the nal decision be taken by the person responsible forquality control, according to the scores given by the judges. The judges should centre their attention on themagnitude of the perceptible differences. The responsi-bility to decide may have a psychological inuence onthe evaluation of the differences. Another source of

inuence may be the knowledge of the specication bythe judges. To compensate for the latter it is commonpractice to introduce a blind sample of the standardproduct to be compared with the declared standard.This a useful method in public or government organi-sations, where the objective is to separate samples of low quality. In industrial quality control, this methodlacks the capability of giving information on the natureof the difference, necessary to identify the cause andcorrect the difference (Aust, Gacula, Bearm, &Washam, 1985; Mun oz et al., 1992).

A more informative method consists in selecting themost important attributes in the product sensory qualityand evaluating the differences from the standard for eachattribute (Fig. 4b). A difference higher than the speciedtolerance in any attribute will mean that the product isout of specication. With this information, correctionscan be introduced when necessary.

Still, this method detects the magnitude of the differ-ences in attributes but not their direction. A possiblealternative, successfully used in some cases (Costell,unpublished data) is to design a scale similar to thejust-right scale, in which the central point corre-sponds to the standard product (Fig. 4c). With this scaleinformation is obtained not only on the magnitude but

also on the direction of the differences from the stan-dard in the different attributes. This procedure may beof interest when the objective is to evaluate the effect of a change in the formulation of a certain product on itssensory quality and the direction of the possible changein any quality attribute is not predictable. Besides thetype of scale used, the quality of the obtained informa-tion depends on the degree of training of the judges andtheir knowledge of the product, on the realisation con-ditions and on the correct analysis of data as a functionof the type of scale (ordinal or interval scale) used. Alarge trained panel (3040) is appropriate when only thedegree of difference from the control is to be evaluated.When additional information about differences on spe-cic attributes is required a smaller and more highlytrained panel is recommended.

3.1.2. Difference from a mental standard As commented above, the use of a mental standard by

one or several experts to dene the quality of a foodproduct presents two serious problems, derived from thepossible difference between the mental standards used bythe experts and from the fact that their opinions are notrepresentative of consumers opinion. In principle, basedon these considerations, it would not be recommendable



10/13

to use this method to evaluate the quality of certainproducts. But this position must be reconsidered. Incases when an expert or a group of experts, showingrecognised ability to evaluate the magnitudes of theperceptible differences between products and a pro-found knowledge of the product and its manufacturingprocess is available there are situations in which theirperformance is not only admissible but recommendable.One of these situations is when the characteristics of theproduct will not be directly evaluated by the consumer,such as, when dealing with raw materials or ingredientsor when only previous information is sought on theeffects of formulation, process or storage conditions onthe product quality. Another situation where qualityevaluation by experts is appropriate is when the objec-tive is to evaluate differences between quality grades of products of exceptional sensorial characteristics, such aswine, coffee or olive oil, in which small differencesbetween high quality levels may be decisive in their

market price. These differences can hardly be detectedby naive consumers.

3.1.2.1. In/Out method. This is the simplest method tocompare product quality with a mental standard byexperts. It is mainly used to identify products that showclear deviations (presence of off-notes or other defects)from the normal production. It can be recommendedfor the evaluation of raw materials or relatively simplenished products. Its advantages are the simplicity andthe direct use of results obtained. The main disadvantageis its inability to provide descriptive information andtherefore its lack of direction and actionability to xproblems (Mun oz et al, 1992). The validity of theinformation provided depends on whether the expertsare indeed genuine experts. If this method is used by oneor a small group of people in a company who do notpossess the necessary expertise, each of them makesdecisions based on his individual experience and on his

Fig. 4. Different types of scales for: (a) overall difference from standard product ratings; (b) difference from control for selected attributes and (c)directional diference from control for selected attributes.



11/13

product knowledge. This situation leads to highly variableand subjective information.

3.1.2.2. Overall quality rating method. This methodconsists in assessing the quality of a product accordingto an established quality criteria. Samples are rated

using a single quality scale anchored very poor andexcellent. A product is rejected when the quality rat-ings are low. Initially it was considered that a qualityrating test represents a combination of affective anddescriptive tasks (Sidel, Stone, & Bloomquist, 1981).Besides this duality, other problems arise in data treat-ment because the quality scores are not clearly based onpsychophysical measurements (Lawless, 1994). Anotherapproach consists in considering the product quality asan integrated impression like acceptability or pleasureexperienced when consuming a food or drink. Usingthis criterion, the evaluation of a product quality with aunidimensional scale may appear logical. This impliesaccepting that quality and acceptability are not conceptsof an exclusively sensorial character (Costell, 2001). Asstated by Cardello (1997): In psychological terms,pleasure and displeasure, liking and disliking, are notsensory phenomena, although they accompany mostsensory stimuli. Rather, pleasure and displeasure areemotional experiences. They are conscious cognitionsthat accompany the somatic effects of emotions. Basedon this approach it can be accepted that a group of experts, sharing a common mental standard, may suc-cessfully judge a product quality grade. From a qualitycontrol standpoint, this method has the disadvantage of

producing an integrated judgement, that may not beactionable and useful for product documentation orguidance.

In an effort to overcome the above mentioned prob-lem, other methods have been proposed, in which ascale to evaluate overall quality and other scales toevaluate the attributes quality or their intensities areincluded in the same scorecard (Mun oz et al., 1992).This scheme is apparently similar to that describedabove to evaluate the perceptible differences from acontrol product but the situation is not the same. Evenassuming that the experts possess a solid mental stan-dard of the product quality it is hard to make them payattention to the overall quality and to the quality orintensity of attributes in the same session. In this waythey are obliged to perform functions that require dif-ferent mental attitudes which can produce erroneousresults. It should not be recommended that experts usethis type of scorecard to evaluate a product quality.

3.1.3. Difference from a written standard These types of tests are among the most frequently used

in quality control. Essentially they consist in the evalua-tion of the intensity of different attributes and/or defectsor the evaluation of quality grade using a scorecard. The

information gathered during the previous developmentof the standard and the establishment of the specica-tion is collected in the scorecard, according to differentcriteria and in different ways. Several alternative proce-dures have been used but, practically, only one of themis in use at present: the quality grading test.

3.1.3.1. Quality grading method. This method has beenone of the most popular sensory tests used in qualitycontrol and consists in developing a scorecard thatincludes a scoring system with points assigned for eachgrade and a description of sensory characteristics den-ing quality for each grade. The scorecard is composedof ordinal scales using discrete numbers and containsthe description of the characteristics. The scale ampli-tude may be 3, 6 or 9 points. The upper third of thescale includes a detailed description of the intensity of each attribute corresponding to a high level of quality,the medium third the description corresponding to anacceptable quality and the lower third that correspond-ing to rejectable quality. Frequently a scale is designedfor each basic sensory attribute, e.g. appearance, colour,avour and texture. The judges give scores to eachattribute and when a product is assigned a score in thelower third of the scale, it should be rejected (ISO,1987). This test allows for a rapid qualication of theproduct and for the detection of the possible causes of rejection. However this test requires a group of very welltrained judges that can correctly interpret the descrip-tions corresponding to each of the quality grades foreach of the selected attributes. An important problem

here is that the judges are obliged to carry out an ana-lytical job simultaneous to a qualication which mayproduce deviations in the results, as commented above.Finally it should be taken into account that the datafrom this test are of ordinal nature and this fact leadsto the use of non-parametric statistical methods foranalysing the data obtained.

3.2. Methods of evaluation without a standard

3.2.1. Descriptive method This method consists in having a well-trained sensory

panel that provides data on a set of the products sen-sory attributes. During the initial development of thesensory standard a number of attributes are selected.Some of them have been selected because their varia-tions affect product acceptability by consumers andsome others may be introduced by the industrial com-pany on the basis of their interest in connection with theidentity and/or desired image of the product in themarket. In the establishment of the corresponding spe-cication their tolerable variability is xed. Evaluationof quality with this descriptive method consists basicallyin the evaluation of the intensity of each attribute by atrained panel using descriptive proling (conventional



12/13

prole, QDA, Spectrum). The person responsible forquality control then studies the results obtained fromthe statistical analysis of experimental data and makesthe nal decision based on the sensory specicationpreviously established. In this case, the specications arerepresented by the range of intensities tolerated for each

attribute. Products whose intensity on any given attri-bute fall outside specications are considered unac-ceptable. For example, a company wants to assess thequality of the virgin olive oil it produces, in accordancewith the standard proposed by the COI (Fig. 2). Therst step should be the selection and training of thepanel. The training comprises the denition, evaluationprocedures and magnitude scoring of each of the attri-butes and of the defects included in the scorecard (COI,1996). Once it has been ensured that the panel workswell, the comparison between the panel results and thespecications set for each of the product attributes canbe used to make decisions regarding the product qual-ity. As stated by Munoz et al, (1992), the two mainadvantages of this approach are the absence of anysubjectivity in the evaluation and the quality of the dataobtained. The main disadvantages are the time and costnecessary to train and calibrate the panel and the timenecessary to perform the test and to analyse the data.This test and the corresponding data analysis can besimplied by using the software available (Punter,1994). The method described is not suitable for solvingsome particular problems that require an immediatedecision. In this case one possibility may be to performa reduced version of the prole. Once the panel is

trained on the whole prole (10 to 15 attributes), a smallgroup of judges may be selected to evaluate the mostimportant attributes (45). This simplication mayallow its use in daily quality control.

3.2.2. Other methodsMany of the merited criticisms of sensory methods

used in quality control originate from the lack of a pre-viously developed standard or an established specica-tion. It should also be considered that the standards andspecications are developed for a specic situation(industry, public or private organisation, etc). The useof some methods (In/Out, Quality Rating, etc.) withouta previous standard or specication afford results of doubtful validity. It is especially important to note thatthe development of a quality grading system without aprevious study of the relations between the variations inattributes and product acceptance may lead to the con-struction of scorecards without any practical value. It isalso important to point out that the use of these meth-ods in food research to compare products or to studythe effects of processing conditions, must be avoided.The evaluation of the sensory quality of any productby a group of 10 to 20 more or less trained panellists ina laboratory without a standard or specication has

the same doubtful value as their opinion on productacceptance.

Finally something must be said about the qualityevaluation methods based on what is known as com-plete scorecard. These scorecards include evaluationsfor different sensory categories such as appearance, a-

vour, texture, etc. as well as for some specic attributeslike sourness or viscosity and a variable number of quality points is assigned to each one of them. Thesum of points obtained determines the product quality.Another, alternative, method consists in assigningscores to the intensity of different attributes, multiplyingthem by different factors according to their importanceand adding them up to get a product qualication. Oneof the better known methods of this type is the U.C.Davis 20-point wine scoring system described in 1981 byAmerine and Roessler (Lawless, 1995). These methodswere once very popular and were adopted by someindustrial rms and control organisations. They wouldappear to make it possible to express the quality of aproduct with a single number. But in practice they pre-sent several problems. They have been criticised onmany occasions because the weight of each attribute hasbeen arbitrarily assigned and the product quality istaken as the sum of the qualications given to a limitednumber of attributes. On the other hand, the scales usedto evaluate the intensity of the different attributes donot always have sensorially equivalent magnitudes. Thisalso means that the validity of the information obtainedwhen each score is multiplied according to a previouslyestablished weighting factor is questionable, even when

this factor has not been established in an arbitrarymanner. For all of these reasons this type of test is notconsidered recommendable for assessing the quality of aproduct.

4. Concluding remarks

In accordance with what has been stated above, wecan conclude that not all methods proposed for evalu-ating the sensorial quality of food products are suitablefor incorporation in quality control programmes. Dif-ference or preference tests, typical measurement orthose methods based on complete scorecards are theless appropriate while difference from control methodsand descriptive methods, are the most sound sensorytests for quality control purposes. Others methods suchas In/Out, Quality rating and Quality grading methodsmay be used in particular situations. The characteristicsof each product, the degree or level of quality that it iswished to control and the resources available conditionthe choice of method to be used. On the other hand, itshould be borne in mind that designing an effectiveprogramme for testing the sensorial quality of a productis based on the following points: (a) The selection of the



13/13

sensory quality standard; (b) The establishment of thesensory specication; (c) The selection of a method toevaluate differences between the product and the corre-sponding standard and; (d) The selection, training andmaintenance of the panel. The practical value of theinformation obtained will be determined by the correct

fullment of these requirements.

Acknowledgements

To Ministerio de Ciencia y Tecnolog a of Spain (Pro- ject AGL 20001590). The author acknowledges Dr.Luis Dura n for revision of the manuscript and helpfulobservations and Alejandra Mun oz for constructivecomments.

References

ASTM. (1996). Sensory testing methods: Second Edition. MNL 26 .Philadelphia: American Society for Testing and Materials.

ASTM. (1997). Relating consumer, descriptive and laboratory data.MNL 30 . Philadelphia: American Society for Testing and Materials.

Aust, L. B., Gacula, M. C., Beard, S. A., & Washam II, R. W. (1985).Degree of difference test method in sensory evaluation of hetero-geneous product types. Journal Food Science , 50 , 511513.

Bertozzi, L. (1995). Designation of origin: quality and specication.Food Quality and Preference , 6, 143147.

Booth, D. A. (1995). The cognitive basis of quality. Food Quality and Preference , 6, 201205.

Cardello, A. V. (1995). Food Quality: conceptual and sensory aspects.Food Quality and Preference , 6, 163168.

Cardello, A. V. (1997). Pleasure from food: its nature and Role inSensory Science. Cereal Foods World , 42 , 550552.

COI (1996). Organoleptic assessment of virgin olive oil . COI/T.20/Doc.no 15/Rev.1. International Olive Oil Council.

Costell, E., & Dura n, L. (1981). El ana lisis sensorial en el control decalidad de los alimentos. Revista de Agroqu mica y Tecnolog a deAlimentos , 21 , 110.

Costell, E. (2000). Ana lisis sensorial: evolucio n, situacio n actual yperspectivas. Industria y Alimentos Internacional , 2, 3439.

Costell, E. (2001). La aceptabilidad de los alimentos. Nutricio n y pla-cer. Arbor , 661 , 6585.

EEC Council (1992). Council Regulation 2081/92. 14 July 1992. Offi-cial Journal of the European Community. Luxemburg.

Ennis, D. M. (1993). The power of sensory discrimination methods.Journal Sensory Studies , 8, 353370.

EUR (1999). A guide to the sensory evaluation of the texture of hardand semi-hard ewes milk cheeses. No 18829. Official Publications of the European Communities. DG XII, Brussels.

Fisken, D. (1990). Sensory quality and the consumer: viewpoints anddirections. Journal Sensory Studies , 5, 203209.

Gould, W. A., & Gould, R. W. (1988). Total quality assurance for the food industry . Baltimore: CTI Publications Inc.

Guinard, J. X., Yip, D., Cubero, E., & Mazzucchelli, R. (1999).

Quality ratings by experts, and relation with descriptive analysisratings: a case study with beer. Food Quality and Preference , 10 , 59 67.

Herschdoerfer, S. M. (1986). Quality control in the food industry . NewYork: Academic Press.

ISO. (1987). Sensory analysis. Methodology. Evaluation of food pro-ducts by methods using scales. International Standard no. 4121 .Switzerland: International Organization for Standardization.

ISO. (1994). Sensory analysis. General guidance for selection, trainingand monitoring of assessors. Part 2: Experts. International Standard no. 85862. . Switzerland: International Organization for Standardi-zation.

Juran, J. M. (1974). Quality Control Handbook (3rd ed.). New York:McGraw Book Co.

Kramer, A. (1959). Glossary of some terms used in the sensory (panel)evaluation of foods and beverages. Food Technology , 13 , 733738.

Kramer, A., & Twigg, B. A. (1970). Quality control for the food indus-try (3rd ed.). Westport. Connecticut: The AVI Publishing Co.

Lawless, H. T. (1994). Getting results you can trust from sensory eva-luation. Cereal Foods World , 39 , 809814.

Lawless, H. T. (1995). Dimensions of sensory quality: a critique. Food Quality and Preference , 6, 191199.

Lawless, H. T., & Heymann, H. (1998). Sensory evaluation of food .New York: Chapman & Hall. International Thomson Publishing.

Lardmond, E. (1994). Is Sensory Evaluation a Science? Cereal FoodsWorld , 39, 804808.

MacFie, H. J. H., & Thomson, D. M. H. (1994). Measurement of food preferences . London: Blackie Academic & Professional.

Meilgaard, M., Civille, G. V., & Carr, B. T. (1999). Sensory evaluationtechniques (3rd ed.). Boca Raton. Florida: CRC Press.

Molnar, P. J. (1995). A model for overall description of food quality.Food Quality and Preference , 6, 185190.

Moskowitz, H. R. (1983). Product testing and sensory evaluation of foods . Westport, Connecticut: Food & Nutrition Press.

Moskowitz, H. R. (1993). Sensory analysis procedures and viewpoints:Intellectual history, current debates, future outlooks. Journal of Sensory Studies , 8, 241256.

Moskowitz, H. R. (1994). Food concepts and products. Just-in timedevelopment . Trumbull, Connecticut: Food & Nutrition Press, Inc.

Mun oz, A. M., & Chambres IV, E. (1993). Relating sensory measure-ments to consumer acceptance of meat products. Food Technology ,47 , 128131, 134.

Mun oz, A. M., Civille, G. V., & Carr, B. T. (1992). Sensory evaluationin quality control . New York: Van Nostrand Reinhold.

Powers, J. J. (1981). Multivariate Procedures in Sensory Research:Scope and limitations. MBAA Technical Quarterly , 18 , 1121.

Punter, P. H. (1994). Software for data collection and processing. InJ. R. Piggott, & A. Paterson (Eds.), Understanding natural avors(pp. 97111). London: Blakie Academic &Professional.

Schlich, P. (1993). Risk tables for discrimination tests. Food Qualityand Preference , 4, 141151.

Stauffer, J. E. (1988). Quality Assurance of Foods . Westport, Con-

necticut: Food & Nutrition Press, Inc.Sidel, J. L., Stone, H., & Bloomquist, J. (1981). Use and misuse of sensory evaluation in research and quality control. Journal DairyScience , 64 , 22962302.

Stone, H., & Sidel, J. L. (1993). Sensory Evaluation Practices ((2nd

edition.)). New York: Academic Press, Inc.York, R. K. (1995). Quality assessment in a regulatory environment.

Food Quality and Preference , 6, 137141.


A Comparison of Sensory Methods in Quality Control

Documents

Transcript of A Comparison of Sensory Methods in Quality Control