Food Quality Certification an Approach for the Development of Accredited Sensory Evaluation Method

www.elsevier.com/locate/foodqual

Food Quality and Preference 18 (2007) 425–439

Food quality certification: An approach for the developmentof accredited sensory evaluation methods q

F.J. Perez Elortondo a,*, M. Ojeda a, M. Albisu a, J. Salmeron a, I. Etayo a, M. Molina b

a Laboratorio de Analisis Sensorial, Facultad de Farmacia, Universidad del Paıs Vasco/Euskal Herriko Unibertsitatea,

Paseo de la Universidad, 7, 01006 Vitoria, Gasteiz (Alava), Spainb Consejo Regulador de la Denominacion de Origen Queso Idiazabal, Granja Modelo de Arkaute, Apartado 46, Arkaute (Alava), Spain

Available online 20 May 2006

Abstract

The standardization and accreditation of sensory quality evaluation methods is a pressing need for the certification of food products,particularly for foods and beverages with specific sensory characteristics, such as those with a protected designation of origin (PDO).This study describes a proposal for the development of accredited sensory evaluation methods, showing as an example an applicationfrom a specific certified product (PDO Idiazabal cheese). A training and qualification process for expert panelists is required. In cheese,panelists score quality of overall sensory parameters (shape, rind, paste colour, eyes, odour, texture, flavour and aftertaste) on a scale,based on how close the product lies to a specific quality standard. Panelists justify the quality scores given on the basis of the absence/presence of specific characteristics in the product and/or the presence of defects. Training requires the prior establishment of referencesfor both characteristics and defects. Qualification trials determine whether or not the expert panelists (both individually and as a panel)are appropriately qualified to carry out the sensory evaluation. This work also shows the quality control maintenance of qualificationsfor the expert panellist. This approach could be generalized to any type of food and beverage as a reference for the accreditation of sen-sory quality evaluation methods according to ISO 17025. In this way, each product manufacturer would be able to define its qualitystandard and, on the basis of this standard, carry out the sensory evaluation using a panel specifically trained for this purpose.� 2006 Elsevier Ltd. All rights reserved.

Keywords: Sensory quality evaluation; ISO 17025 sensory method accreditation; Food quality certification; Protected designation of origin (PDO);Idiazabal cheese

1. Introduction

The scientific strategy most acceptable for sensory qual-ity evaluation takes into account the relation from twotypes of data: tests with consumers (affective class, hedonictype test) and trained analytical panels (descriptive class,analytic type test). The relation between the two makesit possible to determine sensory profiles best adapted tothe concept of the product quality in the target market,

0950-3293/$ - see front matter � 2006 Elsevier Ltd. All rights reserved.

doi:10.1016/j.foodqual.2006.05.002

q A poster of this paper was accepted and presented during theEuropean Conference on Sensory Science of Food and Beverages ‘‘Asense of identity’’ in Florence (Italy) from 26th to 29th September 2004.

* Corresponding author. Tel.: +34 945 013075; fax: +34 945 013014.E-mail address: [email protected] (F.J. Perez Elortondo).

enabling large companies to establish control activities,improve quality, and develop new products. This strategy,however, has limitations with respect to the possibility of itbeing applied by small producers with typical sensory char-acteristic products, such as those with protected designa-tion of origin (PDO).

The integration of the two types of information requireshigh level technical work, where multidisciplinary researchis needed (psychology, marketing, dynamic group tech-niques, sensory methodology, statistical multivariateanalysis).

There is a standardized method for the selection, basictraining and development of sensory profiles, but a lackof standards for consumer tests, this being one of the mainneeds of sensory analysis standardization.

426 F.J. Perez Elortondo et al. / Food Quality and Preference 18 (2007) 425–439

In laboratories that only use descriptive techniques, typ-ically the number of ‘‘positive’’ descriptors is limited (keyattributes) and it is doubtful that it is even possible toachieve a product more in line with a given sensory profile,but where quality is insufficient due to the presence of a‘‘defect’’.

Finally there are some products, which, although theyshould be market orientated to ensure successful sales, theirsensory attributes are influenced by certain origins or pro-cessing factors. This is the case of PDO products that needto gain the acceptance of consumers while at the same timerespecting their sensory typicalness. In many cases, theseproducers are small enterprises, where it is economicallyimpossible for them to develop and introduce systematicquality control on the end product from hedonic and ana-lytical information.

Questions arise such as the followings: Apart from theopinion of the consumer, should the influence of technical,cultural, social and environmental factors be taken intoaccount to obtain differentiated products? Would it beappropriate to propose a sensory analysis method vali-dated and recognised (accredited), allowing a direct evalu-ation of the quality previously defined for a product, bymeans of a panel of expert panelists trained specificallyfor the purpose? Using this method, should we take intoaccount the possible defects known in the product thatare important for its identity? How do we integrate saidpossible defects in the quality evaluation?

Generally, in quality certification of PDO products,regulations underline the fact that these products shouldpresent sensory characteristics in accordance with thejudgement of a ‘‘Sensory Committee’’. These panelists,who may be labelled ‘‘expert judges of a specific product’’,

Quality definition of the product

Specific development of the method

Individual qualificatio

Validation of the method

Quality control maintenance of ex

General quality m

Fig. 1. Main activities for the accreditation of a sensory q

should evaluate not only the presence of certain specificsensory characteristics in the product, but also the absenceof possible defects. The members of these panels are peoplewith a great deal of experience in tasting a specific product,but in many cases, neither use standardised methods norhave they been given systematic training. Many of thePDO products still lack a Sensory Committee, and this islargely due to the lack of references published on how toconduct such a task.

Sensory evaluation comprises a set of techniques foraccurate measurements of human responses to foods andminimizes the potentially biasing effects of brand identity(Lawless & Heymann, 1998). Sensory characteristics com-prising appearance, odour, flavour and texture are includedwithin the quality of food products. There are a limitednumber of internationally accepted standards on generalmethods in sensory analysis, such as: general guidance(ISO 6658, 1985), assessor’s selection and panel trainingprocedures (ISO 8586, 1993) and sensory tests (ISO 5495,1983; ISO 4120, 1983; ISO 6564, 1985; ISO 10399, 1991;ISO 1036, 1994). These standards permit the selection,basic training of panelists and general application of dis-criminative and descriptive sensory methods. Using thesestandards it is possible to detect and describe sensory differ-ences between food products but not to establish and eval-uate their quality or define training procedures for expertsin specific foods.

Accreditation means the demonstration of technicalcompetence. Sensory laboratory accreditation would havea future and perhaps could be realized in the next two dec-ades (Moskowitz, Munoz, & Gacula, 2003). ISO 17025(1999) was developed in order to obtain an accepted systemfor comparable evaluation of testing activities. According

Selection and basic training of panelists

Specific training of panelists

n of the expert panelists

(proficiency tests included)

pert panelists and of validated method

anagement activities

uality evaluation method on a specific food product.

F.J. Perez Elortondo et al. / Food Quality and Preference 18 (2007) 425–439 427

to the European Community regulations for PDO prod-ucts, the certifying organizations of these food productshave to comply with the European Standard EN45011(DOCE, 1992). Consulting the mentioned standard, itbecomes obvious that the analysis for certifying these prod-ucts, sensory included, should be carried out in laboratoriesaccording to the standard ISO 17025. One of the mostimportant documents nowadays for standardization of sen-sory activity is the European publication reference EA-4/09‘‘Accreditation for sensory testing’’ (ENAC, 2003) thatsupplements ISO 17025 and provides specific guidance onthe accreditation of sensory laboratories. This documentgives detailed guidance for the interpretation of ISO17025 for those undertaking sensory examinations. Accred-itation bodies will only accredit laboratories for sensorymethods that have been fully documented and validated.Some laboratories have accredited general discriminative,descriptive and consumer sensory tests, but there is no pre-vious experience in the accreditation of sensory qualityevaluation methods of specific products.

The aim of this study is to describe, for example, a sen-sory evaluation method that serves as a basis for the sen-sory quality certification of a specific product (PDOIdiazabal cheese), as well as the training and qualificationprocess for expert panelists required for implementation.The paper presents the experience of four years of workand shows a general perspective across all the activities(Fig. 1) developed in an accredited sensory laboratory(June, 2005) according to ISO 17025:http://www.enac.es/html/anx/le/UPV_EHU_LE1020_REV1.pdf.

2. Material and methods

2.1. Expert panelists for sensory quality evaluation

of a specific product

ISO standards on the selection and basic training ofpanelists may be used as a reference during the initial train-ing stages of experts, although such standards do not estab-lish the required level needed to pass these preliminarytests. Afterwards, expert judges should be trained specifi-cally on the sensory quality of the product under study(characteristics and defects) and for the specific applicationof the evaluation method.

2.2. Installations, equipment, chemical and foodstuffs‘‘specific references’’

An accredited sensory analysis laboratory (SAL) shouldhave a meeting room according to the ISO standard, stan-dardized booths and an isolated sample preparing area.The other equipment in a SAL is not sophisticated, in orderto comply with the accreditation standards. This equip-ment is basically limited to thermo hygrometers and con-tinuous read-out thermometers, refrigerators, temperedsample chambers, weighing scales, distilling equipmentand/or water deionisation and volumetric material, chemi-

cal compounds and foodstuffs used for the standardisationof references.

The harmonized use of common references by differentlaboratories represents in itself a beginning of standardiza-tion and currently it could be valid in the accreditation ofspecific sensory evaluation methods. However, as pointedout by Issanchou, Schlich, and Lesschaeve (1997) eachproduct needs its own adequate references which can becompletely different from one type of cheese to another,thus making it impossible to generalize a set of standardsfor every variety.

2.3. Specific samples of the product

During the specific training of panelists, a wide varietyof product samples is essential in order to accommodatethe highest possible quality variability in the product.

On a daily basis, the systematic evaluations make it pos-sible for the laboratory to identify interesting product sam-ples for different purposes, such as harmonize and improvethe consensus among the panel members, introduce qualitycontrol tests, and increase the experience of the panelistsfor the identification of defects.

2.4. Data treatment

A coding and internal recording system should ensuretraceability of the results and all the information fromthe activities of the SAL.

Data treatment is applied at very different levels: quali-fication and follow-up of the panelists’ performance,method validation, quality control tests, proficiency testingschemes, treatment of the results and test reports. Gener-ally, statistical descriptive analysis and average comparisontests are appropriate; more sophisticated multivariate anal-yses are not essential.

3. Results and discussion

3.1. Sensory quality definition of the product

Sensory analysis is a research discipline that shouldbecome one of the main driving forces to ensure the qualityand protection of PDO food products (Bertozzi & Panari,1993; Perez Elortondo, Barcenas, Casas, Salmeron, &Albisu, 1999). To establish the sensory quality of a PDOproduct and define a method for its evaluation, it is neces-sary to have in-depth knowledge of the sector involved(type of company, production systems, and product vari-ability) and to compile scientific and market informationrelated to the type of product under study. A workinggroup on sensory analysis should be set up to reach a con-sensus of opinion on progress, both in terms of sensoryquality definition and the evaluation method. The membersof this group are a decisive element for ensuring the successof the work. It is very important to be able to rely on peo-ple with acknowledged prestige and experience in the sector

Table 1Sensory standard definition of Idiazabal cheese used in quality evaluation(adapted from Perez Elortondo, 1993)

Variable Optimal situation (reference)

Shape Cylindrical, well-proportioned, 8–12 cm high, 10–30 cmdiameter and 0.9–3 kg weight. Markedly flat faces.Slightly convex sides. Rounded edges on small cheeses,sharp edges on larger ones

Rind Hard. Smooth, without trace of outside agents. Slightsigns of the cloths used. Homogenous colour, from apale yellow or whitish grey to a darker grayish-brownin the case of smoked cheeses. Absence or light marksfrom the trays on one of the faces

Paste colour Homogeneous. Variable (from ivory to straw yellow).Matt. Narrow and slightly darker border

Eyes Random distribution. Absence or not very numerous.Mostly of irregular shape. Smaller than a grain of rice.Absence of cracks

Odour Intense odour, ewes’ milk, clean and sharp. Slightlypungent. Variable acidity (intensity null to medium) andsweetness (intensity null to medium). Medium intensitysmokiness with smoked cheeses

Texture Slight elasticity. Medium firmness. Weak granularityFlavour Characteristic flavour, balanced and intense: character

of mature ewes’ milk, somewhat of ‘‘natural rennet’’flavour, clean and consistent. Slightly pungent. Variablesweetness and acidity. Absence of bitterness. Mediumsaltiness. Mild to medium smokiness in smoked cheeses

Aftertaste Continuity with respect to the characteristic flavour.Persistent. Pronounced


and that ‘‘have something to say’’. Members of this workinggroup should include product processors, restaurateurs,and technicians in sensory analysis, gastronomic criticsand consumers.

In order to achieve an initial sensory quality definitiondraft and a consensus over the evaluation method, experi-ence has taught us that several meetings are needed, lastingup to 3 h and held weekly over a period of approximately6–12 months, depending on the complexity of the productand on the number of companies involved. These meetingsshould include sensory evaluation sessions to discuss thesensory variability of the product, to generate and selectsensory descriptors, and to record any possible defects.At this stage, it is important to identify the essential sen-sory attributes linked to the origin and/or particular sys-tems of production. It is advisable to corroborate theidentification of these essential attributes by means of aparallel research procedure, using descriptive methods byan analytical trained panel. These key attributes distinguishthe PDO product and ensure its typicalness and protectionin terms of possible imitations.

The result of this work (sensory quality definition stan-dard) should be conveyed to the production sector foracceptance as a sensory reference for the control andimprovement of the product quality. Follow-up of adjust-ment to this initial sensory quality standard should con-tinue, because changes may occur in the consumerpreferences, in the production systems and/or new knowl-edge in research sensory characterisation may also beavailable.

As an example this procedure made it possible to estab-lish the standard sensory quality definition of a PDO prod-uct such as Idiazabal cheese (Table 1) a decade ago.Nowadays this definition is the basic reference which isused for applying the accredited sensory evaluation methoddeveloped by our SAL.

3.2. Selection and basic training of panelists

3.2.1. Selection of panelists

SAL comprises three selection stages: recruiting, selec-tion and preliminary formation. SAL includes two waysof recruiting: internal (when candidates belong to the Uni-versity), or external (when candidates are expert techni-cians, restaurateurs or consumers). Initial selection ofnew judges is carried out by questionnaire, to determinepersonal aspects, such as the aptitude for foods, communi-cation, health, availability and other factors of a generalnature.

Preliminary formation trains new judges by familiariz-ing them with the discipline of sensory analysis. Explana-tions are given based on theory, and selection tests areconducted. Test results, used as a basis for selecting panel-ists, address the following objectives: to detect inabilities,determine sensory sensibility and evaluate the potential ofthe candidates in describing and communicating theirperceptions.

Table 2 shows the tests and selection criteria for newpanelists. The type of test and concentrations of the refer-ences are based on ISO standards, although some selectioncriteria are internal to our SAL due to the absence of spec-ifications in the standards. Selection tests are of a basic andgeneral nature, regardless of the type of product to betested by the panelists. Each test is conducted twice.

SAL establishes that the judges achieving a P75% suc-cess rate out of the total number of tests conducted areto be accepted, and will proceed on to the following basictraining stage. Those obtaining a success rate between60% and 75% may repeat once the tests failed. If after thistest they achieve a 75% success rate, then they are to beaccepted; lower success rates mean rejection from partici-pating in the subsequent basic training.

3.2.2. Basic training of panelists

Table 3 shows the tests and criteria required for the pan-elists to pass the basic training tests, regardless of the typeof food being tested. The purpose of this training is two-fold. First, the training provides judges with the basic ele-ments of the standardized techniques used in sensoryanalysis, developing their aptitude to detect, recognizeand describe the sensory stimuli involved. Second, thetraining allows judges to become competent in applyingthese basic techniques to any food item. The training pro-gramme encompasses both theory and practice. As with theselection tests, the type of basic training test and the con-centrations of the references are based on ISO standards,

Table 2Tests and criteria for selection of panelists

1. Viewing colours test (Ishihara test): If the candidate views colournormally, he/she will attain certain numbers. Full points are required

2. Colour recognition test. Ranking test: Candidates are presented withfive random colour strips from the Yolk Colour Fan for colours of eggyolk. They are asked to put the strips in ascending order in terms of thedegree of colour intensity. A maximum inversion is required in adjacent

pairs

3. Taste sensitivity test (taste identification). 13 dissolutions are presentedto judges. Each judge will taste, successively, the content of eachdissolution, following the order in which they are presented, withoutgoing back to the samples previously tasted. A minimum success rate of

80% is required

4. Sapid substance recognising tests4A. Duo-trio test: Samples representing basic tastes, at concentrationshigher than the recognition threshold. A sample of each type ispresented to each candidate, and they are left to familiarize themselveswith the same. Then they are presented, at random, with two series ofthe same samples. Candidates should pair them off with the originalsamples. A minimum 80% success rate is required in the pairing off test

4B. Triangular test: The concentration of the product being testedshould be at a supraliminal level. The candidate should detect whichsample is different. Four trials will be presented. A minimum 75%

success rate is required

4C. Ranking test: Panelist are asked to classify samples in order ofgrowing intensity. The order of presentation will be the same for allcandidates. A maximum inversion is required in adjacent pairs

5. Olfactory substance recognition tests5A. Ranking test: Panelists are asked to classify samples in order ofgrowing intensity. The order of presentation will be the same for allcandidates. A maximum inversion is required in adjacent pairs

5B. Odour description test: Samples containing different smellingsubstances, at concentrations higher than the recognition threshold.Candidates must mentally classify the sensory perceptions identified andnote down the family of odours and/or descriptors. 3 points

(identification or correct description); 2 points (description in general

terms); 1 point (identification or description of an appropriate association

following discussion); 0 for no answer. Candidates must score at least 65%

6. Texture recognition tests6A. Ranking test: Solid samples (foods). Panelists are asked to orderthe samples in ascending order, in terms of elasticity and granularity. A

maximum inversion is required in adjacent pairs

6B. Test for describing texture: Solid samples are presented. Panelists areasked to describe their properties as regards texture, noting down theterm and/or terms most frequently linked with its texture. 3 points

(identification or correct description); 2 points (description in general

terms); 1 point (identification or description of an appropriate association

following discussion); 0 for no answer. Candidates must score at least 65%

Table 3Tests and criteria for basic training of panelists

1. Training tests in detecting and recognizing smell and sapid substances1A. Two duo-trio tests:(a) Samples representing odours, at concentrations higher than therecognition threshold. A sample of each type is presented to eachcandidate, and they are left to familiarize themselves with the same.Then they are presented, at random, with two series of the samesamples. Candidates should pair them off with the original samples. A

minimum 80% success rate is required in the pairing off test

(b) Four sets of three sapid samples with the combinations AAB, ABA,BAB, BBA are presented. Samples A and B correspond to two differentbranches of the same product. They are asked to examine the samples ineach set from left to right so that the reference sample (the first on theleft) is examined first, and to indicate which of the other two samples isidentical to the reference. A minimum 75% correct answer will be

required, both in the sapid and in the smell tests

1B. Pairs comparison test. Panelists are presented with two pairs ofsamples for each characteristic to be evaluated in terms of flavour andodour. Panelists must compare, within each pair of samples, which ofthem presents a higher degree of intensity in terms of a particularattribute (forced judgement). A minimum 75% correct answer will be

required, both in the sapid and in the smell tests

2. Training in the use of scales2A. Use of one-dimensional descriptors: Olfactory and sapid samplesare presented at different concentrations. To enable the panelists tobecome acquainted with the concepts of classification, with the help ofdifferent scales (structured continuous and non-structured scales,categories or discontinuous scales), they will order the odour andflavour stimuli in terms of the intensity of each particular attribute.They will be asked not to invert the order of the samples in more than one

of the characteristics evaluated, considering both the odour and flavour

scales separately

2B. Use of multidimensional descriptors. Odour, flavour and texturesamples are presented, at different concentrations. Using the samescales as in the previous sessions, panelists will put the stimuli from thesamples of odour, flavour and texture in order, in terms of the intensityof each particular attribute. They must not invert the order of the samples

by more than one of the characteristics evaluated, considering odour,

flavour and texture separately

3. Training in developing and using descriptorsThe purpose is to make the panelists aware of the concept of profile. Inorder to develop the vocabulary, 2 brands of a food are evaluated. Bymeans of the Interlinked Kelly Method, panellists will describe thedifferences and similarities between the samples. Once the test hasconcluded and following an open discussion between the panel and thepanel leader, the terms which the group considers appropriate for thesensory description of the products are chosen. Hedonic and intensityterms will be removed. Synonyms or terms presenting similar meaningsare to be grouped together3A. Sensory evaluationPanelists will evaluate the food under study in the previous test, notingpoints for intensity, using the attributes finally agreed on. Theevaluation method for the descriptors is established and references willbe used at some point on the scale. Three repetitions per brand ispresented to each panelist. The panel leader will establish the profile foreach product, using the results to illustrate the value of the descriptiveanalysis. These profiles will be shown to the panelists so that they canbecome acquainted with their useSo that the panel as a whole becomes acquainted with discriminating

parameters, a comparison of averages is applied (t-Student test) on each

attribute. In order to verify the discriminating capacity of each member of

the panel, a t-Student test will be applied to the results from each panelist

for each discriminating parameter. An individual discriminating capacity

will be required of at least 50% of the discriminating parameters

considered by the panel as a whole


although some selection criteria are internal to our SAL inview of the absence of specifications in the standards.

During the basic training stage, the panel leader carriesout a follow-up evaluation to assess the interest and dedi-cation of each panelist and on their sensory capacitiesand correct application of the sensory methodology,recording these in the laboratory record. The judges whohave a success rate of at least 75% out of the total numberof tests in duplicate are considered to have passed the basictraining stage. Those who do not achieve this success ratemay repeat once the tests not passed. Finally, the labora-

Table 4List of defects in appearancea

Shape Rind Paste colour Holes

Blowing Colouring Excess of rind halo CavernsLow Not developed White LargeConvex Cracks Dark CracksSunken Extraneous agents Irregular Badly distributedInclined Marks Internal moulds Numerous

Mouldy Crystals RoundedDirty

a References: catalogue of photographs.


tory management personnel decides, which of the candi-dates continue in future tests, based upon interest, avail-ability and training results.

SAL considers these judges (‘‘food panelists’’) to be qual-ified to conduct simple standardized sensory analysis, basi-cally discriminating tests, and to participate in thedevelopment of descriptive sensory methods for specificfoods.

3.3. Specific training of panelists: PDO Idiazabal cheese

as example

So that a panelist can evaluate the sensory quality ofIdiazabal cheese, he must be given specific training and

Table 5Definition and references for characteristics of odour, flavour and aftertaste

Characteristic Definition

Odour

Sheep’s milk Smelly sensation frequently associated with acidifiedsheep’s milk

Sharp Sensation of penetration in the nasal cavityPungent Sensation perceived inside the nasal cavity in the

form of irritation, burning or stingingAcid Sensory property of pure or mixed substances, the

olfactation which is reminiscent of an acidic sensationSweet Sensory property of pure or mixed substances, the

olfactation which is reminiscent of a sweet sensationSmoky (only smoked

cheeses)Odour produced following combustion of wood

Flavour

Sheep’s milkd Olfactive-taste sensation frequently associated withacidified sheep’s milk

‘‘Natural rennet’’d Combination of typical aromas of traditional rennetPungent Sensation perceived in the mouth as an irritation,

burning or stingingSweet Elemental taste perceived in the mouth with watery

solutions of diverse solutions such as sucrose or fructoAcid Elemental taste perceived in the mouth with watery

solutions of diverse organic substances such as citricor lactic acid

Bitter (absence) Elemental taste caused by diluted watery solutions ofdiverse solutions such as quinine or caffeine

Salty Elemental taste caused by diluted watery solutions ofdiverse solutions such as sodium chloride

Smoky (smokedcheeses)d

Sensation produced following the combustion of woo

Aftertaste

Continuity Olfactive-taste sensation maintained after removingthe product that does not differ from the sensationsperceived when it was in the mouth

Persistent Olfactive-taste sensation very close to the sensationperceived when the sample of cheese was in themouth, remaining localized a certain time

Notable Olfactive-taste sensation with high intensity perceivedafter removing the sample of cheese from the mouth

a 5 mL of base solution (BS).b 10 mL of base solution (BS).c 15 mL of base solution (BS).d References of acidified raw sheep’s milk flavour, ‘‘natural rennet’’ and s

references.

attain the qualification of ‘‘expert judge in a specific product

(Idiazabal cheese)’’.

Reference

15 mL acidified raw sheep’s milk 36 h at room temperaturec

Add 5 mL H2Oa to 40 lL propionic acidDissolve 80 lL isobutyric acid to 10 mL with H2Ob

Dissolve 50 lL acetic acid to 10 mL with H2Ob

Dissolve 0.003 g maltol to 10 mL with H2Ob

Dissolve 5 lL commercial phenolic type liquid smoke aroma to10 mL with H2Ob

15 mL acidified raw sheep’s milk 36 h at room temperaturec

Add 50 mL H2O a 0.5 g rennet paste. Stir for 15 min and filterc

Add boiling 100 mL H2O to 0.5 g cayenne pepper seeds.Leave for 5 min and filter. 1 mL supernatant is added to 100 g curd

seTo 1.5 g D(+) sucrose are added 10 mL H2O. Add 6 mL BS to100 g curdTo 75 g curd 25 g of natural yoghurt are added

Not applicable

To 2.35 g anhydrous sodium chloride add 50 mL H2O. Add 2.4 mLBS to 100 g curd

d Dissolve 5 lL commercial phenolic type liquid smoke aroma to10 mL with H2Ob

Not applicable

Not applicable

Not applicable

moky, may not be ingested. They are prepared and presented as odour


3.3.1. Training for appearance

The panel leader explains the sensory definition and theevaluation method of each of the characteristics (Table 1)

Table 6Definition and references of texture characteristics

Characteristic Definition

Elasticity Aptitude of a cheese sample to rapidly regainthickness after compression and deformation

Firmness Resistance of a cheese sample to a very slighopening and shutting of the jaws

Granularity Perception at the end of the sensation of theand shape of particles in the sample. E.g., ro(more or less hard when bitten into)

Table 7Group of defects and references for odour, flavour and aftertaste

Defects Definition

Odour

Acid Vinegar Smelly sensation associated with whi

Acid (in excess) Organoleptic sensation of pure or miwith an olfactation reminiscent of an

Rancid Butyric (in excess) Smelly sensation reminiscent of butyrRancid Smelly sensation, characteristic of fat

following deterioration reactions (ran

Animal Cowshed Smelly sensation reminiscent of an unsheep cowshed

Faecal Smelly sensation reminiscent of substfaeces and fermentation products wit

Putrid Smelly sensation reminiscent of deterrich foods caused by microbial growtunpleasant odours

Flavour/aftertaste

Acidic Acid (in excess) Elementary taste perceived in the mosolutions of diverse organic substanclactic acid

Vinegar Flavour sensation reminiscent of whi

Animal Cowshedc Flavour sensation reminiscent of a sh‘‘Natural rennet’’(in excess)c

Flavour sensation associated with lamin excess

Faecalc Flavour sensation reminiscent of subin faeces and fermenting products in

Putridc Flavour sensation associated with derich foods, caused by microbial growunpleasant odours

Bitter Elementary taste produced by dilutedof diverse substances such as quinine

Pungent Abrasive Irritating action perceived in the moumucous, possible accompanied by a s

Pungent Sensation perceived in the mouth asburning or stinging

Salty (in excess) Elementary taste produced by dilutedof diverse substances such as sodium

a 10 mL of base solution (BS).b 15 mL of base solution (BS).c References for flavour/aftertaste, ‘‘natural rennet’’, faecal and putrid, are n

and defects (Table 4) for the four appearance parameters(shape, rind, paste colour, and eyes). Theoretical training isbacked by a specific catalogue of photographs (references)

References

its initial Null: soft butter/raw carrotMedium: stuffed oliveHigh: ‘‘frankfurter’’ sausage

t Null: melted cheeseMedium: ‘‘frankfurter’’ sausageHigh: cooked carrot (5 min)

dimensionsunded grains

Null (floury): potato pureeMedium (granular): small couscous grainHigh (coarse): medium couscous grain

Reference

te wine vinegar White wine vinegar directly into 50 lL bottle,adding 10 mL H2Oa

xed substances‘‘acid’’ sensation

Dissolve 60 lL acetic acid to 10 mL with H2Oa

ic acid To 15 lL butyric acid add 10 mL vaseline oila

ty substances,cidness)

5 g D.O.P Idiazabal cheese (>12 months ripening).Slice and keep at room temperature for about6 days

ventilated 5 g rear sheep wool

ances presentingh enteric bacteria

5 g sheep dung

iorating proteinh, leading to

5 g putrid cheese. Add 3.5 mL BS to 100 g curd

uth with wateryes such as

Add 65 g curd to 35 g natural yoghurt

te wine vinegar To 100 g curd add 3.3 mL white wine vinegar

eep cowshed 5 g rear sheep woolb rennet paste To 0.5 g rennet paste add 35 mL H2O.

Stir for 15 min and filterb

stances presententeric bacteria

5 g sheep dung

teriorating proteinth, leading to

5 g putrid cheese

watery solutionsor caffeine

To 0.35 g anhydrous caffeine add 50 mL H2O.Add 4 mL BS to 100 g curd

th and pharynxensation of warmth

To 5 mL distilled alcohol add 15 mL H2O.20 mL BS

an irritation, Add 50 mL boiling H2O to 0.5 g cayennepepper seeds. Keep for 5 min and filter.2 mL floating add to 100 g curd

watery solutionschloride

To 2.35 g anhydrous sodium chlorideadd 50 mL H2O. Add 3.5 mL BS to 100 g curd

ot ingested. Prepared and presented as odour references.


representatives of each of the parameters. Once the panel-ists acquire sufficient theoretical knowledge, they partici-pate in two types of training sessions:

1. Description of appearance: Panelists describe cheeses inqualitative terms, taking into account both the optimumcharacteristics and possible defects.

2. Evaluation of appearance: quantification on scale (shape,rind, paste colour, and eyes).

3.3.2. Training for odour, flavour, aftertaste and texture

The panel leader gives a theoretical description of eachof the olfactory-taste and texture characteristics anddefects. The use of standardized references is an essentialpoint in order to ensure that the panelists memorize thesesensations (Tables 5–8).

Once the panelists have acquired the necessary knowl-edge on odour, flavour, aftertaste and texture descriptorsand defects, different samples of PDO Idiazabal cheeseare presented and two types of training session procedures

Table 8Definition and references for defects in texture

Defect Definition Reference

Elastic Sample quickly recovers its initial thicknessafter being compressed and deformed by thethumb

‘‘Frankfurter’’sausage

Soft Easily deformed when compressed betweenthe tongue and palate

Melted cheese

Hard Resilient to being cut or compressed whenteeth clenched

Raw carrot

Floury After masticating, rounded, very fine sizedparticles noted in the mouth

Potato puree

Sandy After masticating, separated, large sized,fine sized particles noted in the mouth

Breadcrumbs

Lumpy After masticating, separated, large sizedparticles are noted in the mouth that ismore or less hard on the teeth

Mediumgrainedcouscous

Crystals After masticating, angular shaped separatedparticles noted in the mouth which, whencrushed, let out an audible crunch

Sugar

Coarse Lack of smoothness and unpleasantsensation perceived as the sample comesinto contact with the surface of themouth’s soft tissues

Pippin apple

Melting Forms a paste with saliva and continuouslymelts giving a perception of presencein the mouth

Melted cheese

Moist Releases some amount of liquid followingmastication

Boiled whiteof egg

Liquid Releases a fair amount of liquid followingmastication

Granny Smithapple

Doughy Cohesive when mixed with saliva, adheresto the surface of the soft parts of the mouth

Boiled egg yolk

Plastic Slowly deforms in the mouth beforebreaking up, without recovering itsinitial shape

Soft toffee

Brittle Easily breaks into pieces when the sampleis compressed with fingers or incisors

Toast

Dry Absorbs saliva ToastSoluble Dissolves in contact with saliva and

disappears in the mouthSugar

are conducted: qualitative description of characteristicsand defects; and quantification on scale of overall parame-ters (odour, texture, flavour and aftertaste).

3.3.3. Training by harmonization tasting

Prior to each harmonization session, panelists undergo arefamiliarization (i.e. reminder) session to refresh theirknowledge of the different characteristics and/or possibledefects of PDO Idiazabal cheese using the references.Afterwards, they evaluate the different product samples.In these harmonization sessions, panelists discuss openlyamong themselves, with the intervention of the panel leaderas moderator.

3.4. Preliminary individual qualification of panelists:

PDO Idiazabal cheese as an example

As a preliminary stage of qualification for quality quan-tification on scale, a study of the discriminating capacity,repeatability and reproducibility for each panelist is carriedout in two sessions. The sensory analysis laboratory’s(SAL) criteria are the following:

• Discriminating capacity: a panelist is qualified in termsof discriminating capacity if a compatibility rate greaterthan 1 is overcome (as a value of uncertainties, the ref-erence value of standard deviations is taken) in 50% ofthe discriminating parameters of the values in the twosessions as a whole. The compatibility rate is defined as

I c ¼j�x1 � �x2jffiffiffiffiffiffiffiffiffiffiffiffiffiffi

I21 þ I2

2

q

taking as I1 and I2 the reference values.• Repeatability: the standard repeatability deviation is cal-

culated as the average of the standard deviations in thesession with each cheese. A person is considered repeat-able when the standard repeatability deviations are lessthan or equal to the reference value in 50% of the totalnumber of parameters analysed.

• Reproducibility: the standard reproducibility deviation iscalculated as the square root of the sum of the variancesbetween sessions plus the variance due to repeatabil-ity. The variance between sessions is considered as thevariance corresponding to the average of the valuesrecorded in the sessions. A panelist is considered asqualified in reproducibility when in 50% out of the totalnumber of parameters, the standard reproducibilitydeviation value is less than or equal to the referencevalue taken.

As an example, Table 9 shows the qualification results ofone of the judges. During two sessions (eight cheeses persession), panelists evaluated four different Idiazabal cheeses(ICh), where two of them were evaluated three times in eachsession. In this example, results show that this panelist isqualified to reproducibility, repeatability and discriminat-

Table 9Means, standard deviation, reproducibility, repeatability and discriminating capacity of one panelist

Shape Rind Paste colour Eyes Texture Odour Flavour Aftertaste

Panelist: reproducibility and repeatability

Day 1 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2

Mean (N = 3) 3.25 3 3.75 2 4 3 3 1 4 4 4 4 4 3.50 4 3.25S sample 0.50 0 0.50 0 0 0 0 0 0 0 0 0 0 0.58 0 0.50

Day 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2

Mean (N = 3) 3 4 4 2 3 3 3 2 5 4 4.25 3 4 2.25 4 2S sample 0 0 0 0 0 0 0 0 0 0 0.5 0 0 0.50 0 0

S session 1 0.50 0 0.50 0 0 0 0 0 0 0 0 0 0 0.58 0 0.50S session 2 0 0 0 0 0 0 0 0 0 0 0.50 0 0 0.50 0 0S repeatability 0.25 0 0.25 0 0 0 0 0 0 0 0.25 0 0 0.54 0 0.25

S between session 0.35 0 0.35 0 0 0 0 0 0 0 0.35 0 0 0.05 0 0.35S reproducibility 0.43 0 0.43 0 0 0 0 0 0 0 0.43 0 0 0.54 0 0.43

% Reproducibility 94

% Repeatability 94

Panelist: discriminating capacity

ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2

Mean 3.12 3.5 3.87 2 3.5 3 3 1.5 4.5 4 4.12 3.5 4 2.87 4 2.62Ic 0.53 2.65 0.70 2.12 0.70 0.88 1.59 1.94

% Discriminating 67

Panel as a whole: discriminating parameters

ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2 ICh 1 ICh 2

Mean (N = 10) 3.10 3.36 3.56 1.61 3.37 2.43 3.07 1.29 3.77 3.45 4.27 3.82 3.77 2.96 3.86 2.97S panelists 0.22 0.73 0.72 0.49 0.46 0.72 0.33 0.33 0.57 0.38 0.42 0.43 0.58 0.46 0.76 0.40

t calculated 1.09 7.06 3.47 11.97 1.49 2.35 3.45 3.25

t tabulated 2.23 2.10 2.10 2.10 2.10 2.10 2.10 2.14

OK OK OK OK OK OK

ICh 1: Idiazabal cheese 1; ICh 2: Idiazabal cheese 2; N: Number of times that every kind of cheese is evaluated during each session; S: Standard deviation;Ic: Compatibility race.


ing capacity. That is, 94% of standard reproducibility andrepeatability deviations are less than the 0.5 reference value;and the compatibility rate is greater than 1, form more than2/3 of the discriminating parameters. In the example (Table9), shape and texture parameters were not considered dis-criminating parameters by the panel as a whole.

3.5. Qualification of panelists as experts: PDO Idiazabalcheese as an example

Qualification of panelists as experts (quality quantifica-tion on scale and defects identification) is done on the basisof their performance history. The competence of panelistsin formation is measured after a minimum of 10 sessionswith eight cheeses each (80 cheeses).

3.5.1. Qualification for quality quantification on scale

A comparison is carried out between the panelist ratingand the average rating obtained from the qualified panel asa whole. The panelist is qualified for quality quantificationon scale if during 10 consecutive sessions the panelist showsless than 15% of deviations (difference with the average of

the qualified panel bigger than ±1) considering all theparameters evaluated in all the cheeses (eight cheeses persession · eight parameters per cheese = 64 comparisonsper session).

3.5.2. Qualification for defect identification

A comparison is carried out comparing defects identifiedby each panelist with those defects identified by the quali-fied panel. The SAL has established that a cheese presentsa defect when at least five out of the seven qualified panel-ists point this out. It is considered that a panelist is quali-fied to identify a specific defect if he/she shows acompetence according to the following criteria:

Number of cheeses with the defect that the panelist hasevaluated and has pointed out/total number of cheeseswith said defect evaluated by the panelist · 100 >66.6%.

For example, when a defect has appeared on three occa-sions, then the panellist is considered to be qualified whenhe/she has noted the defect at least twice out of the possiblethree times. When a panelist is qualified for more than


66.6% of the total number of valuable defects, he isacknowledged to have the capacity to join the PDO Idiaza-bal cheese panel as an expert. In order to reinforce theirperformance, harmonization tasting procedures are under-taken, at two-week intervals to increase the historical expe-rience in identifying defects of the previously qualifiedpanelists.

Experience has shown us that at least one year of activ-ity is needed for a panelist to demonstrate sufficient histor-ical experience to be considered qualified as an expert forevaluating the quality of this complex specific product.This long period is necessary due to the limitations in foodsensory analysis, such as the lack of stable references andsensory fatigue.

3.6. Sensory quality evaluation method: PDO Idiazabal

cheese as an example

Sensory evaluation is carried out in standardizedbooths. Sensory sessions usually starting at 10:00 h, andmay extend for a maximum of 2 h. In the case of the sen-sory quality evaluation of Idiazabal cheese, the maximumnumber of test samples to be evaluated in each session is8, being the temperature of the samples 16 ± 2 �C. Samplesare prepared according to the specific procedure publishedby Lavanchy et al. (1993).

Sample replication is not considered in routine tests.When a well-trained panel generates data with consider-ably lower variability and replications are also desirable,but not essential, and if replications cannot be accommo-dated, the data is nonetheless still valid (Moskowitzet al., 2003).

3.6.1. Tasting room preparation

Prior to each session, checks are made on the tempera-ture (21 ± 2 �C) and relative humidity (60 ± 20%) in thebooths, and on the correct working order of the taps andwarning devices. In each booth, a paper serviette, a plasticcup, a bottle of water with low mineralization and a solidfood (apple, biscuit without salt) are placed.

3.6.2. Conditions for panelists

Samples are to be analysed by seven of the expert qual-ified judges for this specific product. In general, a smallnumber of panelists are adequate if they are well-trained(Moskowitz et al., 2003).

Prior to sensory evaluation panelists should take the fol-lowing precautions into account: personal hygiene andclothing, punctuality; to arrive at least 30 min before thetest. Panelists should abstain from consuming any stronglyflavoured products and from using cosmetics with veryintense or residual odours, and should also be rested andclear-headed. Panelists should not take part in the test if:pregnant, suffering from a headache, nasal congestion orusing drugs. If judges find themselves in any of theseconditions, they should advise the panel leader 24 h inadvance.

3.6.3. Scales of evaluation

Quality evaluation is conducted by taking into accountthe extent to which each sample complies with the opti-mum state of PDO Idiazabal cheese in each of the eightparameters analysed (shape, rind, paste colour, eyes,odour, texture, flavour and aftertaste). A 7-point categoryscale is used, where ‘‘1’’ = null, meaning that the sample isconsiderably removed from the optimum state for eachparameter; ‘‘7’’ = maximum, full compliance with the opti-mum state. To carry out evaluation, each judge has docu-ments in the test booth showing: the optimum states for thedifferent parameters (Table 1); the definition, evaluationtechnique and references used in characteristics (Tables 5and 6) and defects (Tables 7 and 8) of olfactory-taste andtexture; a catalogue of photographs illustrating the opti-mum characteristics (Table 1) and possible defects (Table4) of appearance; and the evaluation criterion (Fig. 2).

3.6.4. Procedure

Prior to sensory evaluation, a harmonization tastingwith one or two samples of PDO Idiazabal cheese is carriedout. These samples are to be different from the routine sam-ples. In order to avoid problems with order and presenta-tion, this order shall be randomized and different foreach judge. In the test booths, communication betweenthe judges and the panel leader is done by a system ofwarning lights.

Panelists giving points below the average value (<4 onthe scale) mark the identified listed defect/s. Evaluatingallows noting down any defect that, despite not beinglisted, has been identified by any of the judges for any ofthe samples. These new defects will not be taken into con-sideration when drawing up the test reports for the currentyear. In order to include a new defect on the list, the num-ber of judges coinciding on the same defect, in the samecheese and at the same session, should be equal to orgreater than 5. Such defects on the list that have not beenrecorded in any test report in the course of a year areremoved from the following year’s list. Depending on theextent to which the sensory parameters appearing in Table1 are complied with, and the possible presence of defects,panelists are to give points to the samples in line with thecriteria shown in Fig. 2.

Test reports are according to ISO 17025 and they aresent to the protected designation of origin (PDO) Idiazabalcheese, including the average points given by the panel foreach parameter and any defect detected. Nowadays, theseresults permit the Regulatory Council of Idiazabal cheeseto certify, from the sensory viewpoint, the identity andquality of the products made by 104 registered cheese pro-ducers. To do so, this Certification Body establishes theminimum limits for quality in terms of points and the seri-ousness of the possible defect (not all the parameters areconsidered equally important). It is important the panelistsdo not know where the limits laid down by the Designationstand, so that they will not be influenced by factors of apsychological nature. This system enables producers to

Tree diagram for ascribing points

To be applied for each parameter and each cheese

Do you note-detect any defect to be pointed out?

No Yes

To consider all the characteristics Do you consider this/these defect/s to be slight?

To be all present and appropriate?

No Yes No Yes

7 3 One absent-inappropriate? Do you consider this/these defect/s to be medium?

No Yes No Yes

6 2Various absent-inappropriate? Do you consider this/these defect/s to be serious?

No Yes Yes

5 1Are the majority absent-inappropriate?

Yes

4

Fig. 2. Tree diagram for evaluating sensory quality (evaluation criteria).


detect points to be improved upon in the minimum sensoryquality when making their product.

Munoz, Civille, and Carr (1992) discuss differentapproaches to sensory quality assessment. A reasonablecompromise between the quality-rating method and adescriptive approach was proposed by Beckley and Kroll(1996). The centrepiece of this procedure is a scale for over-all quality. The advantage of this method is its outwardsimplicity in using an overall rating and recognizing thatthere are situations where products will not exactly matchthe gold standard but are still acceptable to ship.

In the future, in spite of the open discussion betweenthose in favour or otherwise of sensory quality assessmentby experts, accreditation will become the tool for evaluat-

ing the validity of the methods applied in sensorylaboratories.

3.7. Validation of the method: PDO Idiazabal cheeseas an example

In a similar manner to the preliminary individual quali-fication of panelists for quality quantification on scale, therepeatability, reproducibility and discrimination capacity isconsidered as the validation criteria of the method.

Maximum uncertainties of repeatability and reproduc-ibility acceptable by the SAL should be previously estab-lished. These values can be calculated from three sessionswith three different Idiazabal cheeses, where every panelist

Eye

sT

extu

reO

do

ur

Fla

vou

rA

fter

tast

e

sIC

h1

ICh

2S

sIC

h1

ICh

2S

sIC

h1

ICh

2S

sIC

h1

ICh

2S

sIC

h1

ICh

2S

s

22

03.

573.

570

3.43

3.57

0.10

3.86

3.71

0.10

3.71

3,43

0.20

2.43

2.43

03

30

2.86

2.86

03.

143

0.10

2.86

2.86

00

00

.05

0.1

00

.10

.08

0.25

0.33

0.37

0.42

0.43

.08

0.2

50

.33

0.3

80

.43

0.4

4

hse

ssio

n;

Sb

s:st

and

ard

dev

iati

on

bet

wee

nse

ssio

ns.


evaluates each Idiazabal cheese sample three times. Afterthis study, the SAL have proposed 0.5 (repeatability) and0.8 (reproducibility) as maximum uncertainties accepted.

From these criteria, validation of the method (reproduc-ibility and repeatability) is carried out in three later sessions,in the same way as the uncertainties study was done. Thediscrimination capacity is analysed in a later session whereeach panelist evaluates both a sample of Idiazabal cheeseand a sample of non-Idiazabal cheese, doing each twice.

As an example, Table 10 shows the validation results forthe parameter flavour. This parameter can be validatedbecause the repeatability and reproducibility valuesobtained by the qualified panel are less than the maximumuncertainties acceptable proposed by the SAL. Also, thestudent-t test shows the discriminating capacity of themethod for the parameter flavour.

In a similar manner a historical analysis and validation(repetition of cheeses with defects) has been carried out forthe identification of defects by the panel as a whole.

Proficiency/accreditation is a very current and pressingissue being addressed by sensory professionals worldwide.The SAL validation dossier also includes proficiency testswith other laboratories. Proficiency testing is the use ofinter-laboratory test comparisons as part of the processto assess the ability of laboratories to competently performtest for which accreditation is held. Nowadays, proficiencytesting schemes in Sensory Science are available (Mosko-witz et al., 2003). The first attempt to address sensory pro-ficiency was initiated in 1999 by the PROFISENSEuropean Union project, where technical requirementsfor conducting sensory proficiency tests and the manage-ment systems requirements are provided (Lyon, 2001).Other relevant publications of this project are the papers

Tab

le11

Qu

alit

yco

ntr

ol

for

qu

anti

fica

tio

no

nsc

ale

of

sen

sory

par

amet

ers

Sh

ape

Rin

dP

aste

colo

ur

ICh

1IC

h2

Ss

ICh

1IC

h2

Ss

ICh

1IC

h2

S

Mea

nd

ay1

3.14

3.14

04

4.57

0.40

2.86

2.86

0M

ean

day

23.

143.

430.

204.

284

0.20

2.71

2.71

0S

rep

eata

bil

ity

0.1

00

.30

0

Sb

s0.

150.

270

Sre

pro

du

cib

ilit

y0

.17

0.4

10

ICh

1:Id

iaza

bal

chee

se1;

ICh

2:Id

iaza

bal

chee

se2;

Ss:

stan

dar

dd

evia

tio

no

fea

c

Table 10Example of results of validation (mean of the seven panelists as a whole)for the parameter flavour

Repeatability and reproducibility

ICh 1 ICh 2 ICh 3 S session

Session 1 3.71 3.43 3.57 0.142Session 2 3.86 4 4 0.082Session 3 4 4.14 4 0.082

S repeatability 0.103

S betweensession

0.237

Sreproducibility

0.258

Discriminating capacity

Idiazabalcheese

Other PDO cheese

Mean (N = 14) 4.14 3.36S panelists 0.86 0.50t calculated 2.95

t tabulated 2.064

OK

ICh 1: Idiazabal cheese 1; ICh 2: Idiazabal cheese 2; S: Standard deviation;N: Number of times the sample has been evaluated.


by McEwan, Hunter, Van Germert, and Lea (2002b,2002a) that describe measuring performance panel issuesfor ranking and profile testing.

The SAL has established systematic quality controls formaintaining both method validity and the individual qual-ification of the panelists.

3.8. Quality control activities: PDO Idiazabal cheese

as an example

3.8.1. Maintaining method validity

Quality control is conducted by studying the reproduc-ibility and repeatability of the panel as a whole, in orderto maintain method validity. To do so, at two monthlyintervals, a cheese is analysed, in duplicate, at the same ses-sion and afterwards at a subsequent session. The standarddeviations for repeatability and reproducibility for eachparameter are calculated. These deviations should be lessthan the SAL’s maximum acceptable uncertainty for morethan 80% of the parameter analysed. As example, Table 11shows quality control activity for the quantification onscale carried out by the SAL in 2005. This control was posi-tive because both standard deviation for repeatability andreproducibility were less than SAL’s maximum acceptableuncertainties in 100% of the parameters analysed.

On an every two-week basis, quality control for theidentification of defects is carried out using a systematicrepetition of cheese samples, where defects had been iden-

Table 12Individual report of a panelist after a session

Data Judge

Cheese code Shape Rind Paste colour

256 4 3 4Deviation?Panel mean 4 4 4




632 6 2 2Deviation? Yes

Panel mean 4 3 3


No. cheeses: 6No. parameters: 48No. deviations: 3

Panelist % Deviations: 6.25

No. deviations: 4Panel % Deviations: 8.3

tified in previous routine sessions. If the results of repeti-tion show fewer than five members of the panel toidentify the defect, then it will be openly discussed toachieve consensus as to whether or not the specific defectis found in the cheese. To back this up, by means of harmo-nization tasting, cases where panelists were not unanimous(3–4 panelists noting down the defect) are investigated inorder to reduce the frequency of these situations as muchas possible.

3.8.2. Follow-up and monitoring the panelists

To ensure the individual qualification of the panelistsand the panel as a whole, it is essential to follow up andmonitor each one of the expert judges and the panel. Todo so, the SAL has standardized procedures:

• Follow-up and routine monitoring of the panelists. Anindividual report is drawn up comparing the resultsgiven by each the judges in each session with the averageresults from the panel as a whole. In those cases wherethe standard deviation of the panel is P1, an analysisof the data from panelists responsible of this dispersionis conducted. Panelists who stand at a distance from therounded off average obtained by the panel as a whole by± one unit are noted as a deviation for these parameters.This report is drawn up for each session, and is to beshown to the judges so that they can see where therewere deviations. Where a panelist, in a given session,

Eyes Texture Odour Flavour Aftertaste

3 3 5 4 4Yes

3 3 3 3 3

3 3 4 4 5

3 3 4 4 4

3 3 3 4 3Yes

3 3 3 2 2

3 3 4 3 4

3 3 4 3 3

3 4 4 3 4

3 3 4 3 3

2 3 3 3 4

2 3 3 3 3

DefectsDefects identified by the panel: 28Defects not identified by the panelist: 5Defects identified by the panelists: 23% Identified: 82.14 OK


presents more than 15% deviations for all the parame-ters evaluated, one of the following measures is taken:the panelist is interviewed to evaluate any possiblecauses, re-training on the characteristics affected, tempo-rary withdrawal and/or personal re-qualification. As anexample, Table 12 shows the individual (each panelist)report appearance after a session where six cheeses wereanalysed.

• Follow-up and routine monitoring of the panel. If during agiven session the panel as a whole presents 15% devia-tions for all the attributes evaluated, then the followingmeasures are to be taken: re-training of the panel and/ortemporary withdrawal from participating.

In order to ensure maintaining the qualification ofdefects, a follow-up and monitoring of the rate of defectsidentified by each panelist at each session is carried out.If for the total of cheeses in a session, the rate of defectsnoted by a panelist is less than 66.6% of the total numberof existing defects (noted by more than five out of the seven

Table 13Cross-reference table showing ISO 17025 Standard items and the qualitymanual (QM) and operative quality procedures (OQP)

ISO 17025:1999standard section

QM OQP

1. Object and field ofapplication

01

2. Standards for consulting 023. Terms and definitions 03

4. Management requirements4.1. Organization 044.2. Quality system

management4.3. Monitoring documents OQP-01: Drafting OQP

OQP-02: Drafting NWP*

OQP-03: Monitoringdocuments

4.4. Reviewing applicationsoffers and contracts

OQP-08: Acquisition andinventory of supplies andevaluation of suppliers

4.5. Subcontracting testsand calibrations

4.6. Out pooling servicesand supplies

4.7. Customer service OQP-12: Customer serviceand claims management

4.8. Claims4.9. Non-conformities

controlOQP-10: Treatment of non-conformities, correctional andpreventive actions

4.10. Corrective actions4.11. Preventive actions4.12. Records control OQP-06: Control of internal

records and test reports4.13. Internal audits OQP-11: Conducting internal

audits and management reviewingthe system

4.14. Reviews by themanagement

* NWP: normalized work procedure (technical procedure).

panelists), then he/she should be subject to: re-trainingand/or temporary withdrawal.

3.8.3. Panel re-training: specificity of the PDO Idiazabal

cheese

Idiazabal cheese is a seasonal product, made from Jan-uary to July. Sensory evaluations are also seasonal (Febru-ary–October). During the period from November toJanuary, panelists are called to carry out some of the spe-cific training activities mentioned above.

3.9. Management system and improving activities

All technical activities should be developed in line with aquality management system. The management require-ments under the ISO 17025 standard should be reflectedin a manual and/or in quality procedures (Table 13). Thissystem of documenting and recording makes it possibleto demonstrate to the accreditation body that the stan-dards are complied with in the external audits (or thirdparty audits). The system for dealing with internal non-conformities and claims by correctional and/or preventiveactions, internal and external audits and by the manage-ment reviewing the system, are the most dynamic activitiesof the quality system management and they make it possi-ble to improve the activities at SAL.

4. Conclusions

Ensuring the food quality on a sensory basis is animportant corporate goal in a competitive environment.Sensory evaluation is a powerful resource with a great vari-ety of applications in the commercial sector. The standard-ization and accreditation of sensory evaluation methodsfor the quality certification of food products is a pressingneed, particularly with regard to those with protected des-ignation of origin (PDO). This study proposes a generalapproach for the development of an accredited sensorymethod for the quality evaluation of food and beverages,taking into account specific characteristics and possibledefects in the products. The work shows the applicationwith a specific product (Idiazabal cheese) for example.Also, it shows the selection, training and qualificationrequired for expert panelists for implementation. Animportant activity is the systematic control quality toassure the maintenance of expert panelist qualification(individually) and of the previously validated method. Interms of PDO products, this manuscript would make pos-sible to respond to an existing need: to show a work meth-odology so that the members of expert panels (‘‘Sensory

Committees’’) can carry out an objective and independenttask. The proposed method allows the application of goodsensory practices and ensures consistent data.

The approach proposed may be transferred to any typeof food as a reference for the accreditation of sensory qual-ity evaluation methods according to ISO 17025, especially


for sensory certification activities. Each product shoulddefine its quality standard and, on the basis of this stan-dard, the sensory specialist should carry out the sensoryevaluation using a panel specifically trained for this pur-pose. The approach is well suited to standard commoditieswhere minimum levels of quality could be ensured, butexcellence is rarely the issue. This proposal does notexclude applying other sensory analysis methods for otherpurposes (hedonic and descriptive tests) to complement it.Quality evaluation methods should take the data derivedfrom the application of_hedonic and descriptive tests.

Acknowledgements

Financial support of the work was provided by ConsejoRegulador de la Denominacion de Origen Queso Idiazabal,the Departamento de Agricultura y Pesca del GobiernoVasco and the Universidad del Paıs Vasco/Euskal HerrikoUnibertsitatea (project 9/UPV 00042.125-15317/2003 andQuality Chair financement). Expert Members of theTasting Committee of the Regulating Council of the Desig-nation of Origin are thanked for their enthusiastic partici-pation in this study.

References

Beckley, J. P., & Kroll, D. R. (1996). Searching for sensory researchexcellence. Food Technology, 50, 61–63.

Bertozzi, L., & Panari, G. (1993). Cheeses with appellation d’OrigineControlee (AOC): Factors that affect quality. International Dairy

Journal, 3, 297–312.DOCE (1992). Reglamento 92/2081/CEE del Consejo relativo a la

proteccion de las indicaciones geograficas y de las denominacionesde origen de los productos agrıcolas y alimenticios, Diario Oficial delas Comunidades Europeas de 14 de julio de 1992.

ENAC (2003). Guıa para la acreditacion de laboratorios de analisissensorial. G-ENAC-02, Rev. 1. Octubre 2003. Entidad Nacional deAcreditacion, Madrid, Espana.

I.S.O. 5495 (1983). Sensory analysis. Methodology. Paired comparisontest. International Organization for Standardization, Geneve,Switzerland.

I.S.O. 4120 (1983). Sensory analysis. Methodology. Triangular test.International Organization for Standardization, Geneve, Switzerland.

I.S.O. 6564 (1985). Sensory analysis. Methodology. Flavour profilemethods. International Organization for Standardization, Geneve,Switzerland.

I.S.O. 6658 (1985). Sensory analysis. Methodology. General guidance.International Organization for Standardization, Geneve, Switzerland.

I.S.O. 10399 (1991). Sensory analysis. Methodology. Duo-trio test.International Organization for Standardization, Geneve, Switzerland.

I.S.O. 8586 (1993). Sensory analysis. General guidance for the selection,training and monitoring of assessors. Part 1: Selected assessors.International Organization for Standardization, Geneve, Switzerland.

I.S.O. 1036 (1994). Sensory analysis. Methodology. Texture profile.International Organization for Standardization, Geneve, Switzerland.

I.S.O. 17025 (1999). General requirements for the competence of testingand calibration. International Organization for Standardization,Geneve, Switzerland.

Issanchou, S., Schlich, P., & Lesschaeve, I. (1997). Sensory analysis:methodological aspects relevant to the study of cheese. Lait, 77, 5–12.

Lavanchy, P., Berodier, F., Zannoni, M., Noel, Y., Adamo, C., Squella,J., & Herrero, L. (1993). L’evaluation sensorielle de la texture desfromages a pate dure ou semi-dure. Etude interlaboratoires. Leben-

smittel Wissenschaft und Technology, 26, 59–68.Lawless, H. T., & Heymann, H. (1998). Sensory evaluation of food:

principles and practices. New York: Chapman and Hall.Lyon, D.H. 2001. International guidelines for proficiency testing in

sensory analysis. Guideline No. 35. CCFRA (Campden & Chorley-wood Food Research Association), Chipping Campden, GL55 6LD,United Kingdom.

McEwan, J. A., Heinio, R., Hunter, E. A., & Lea, P. (2002a). Proficiencytesting for sensory ranking panels: measuring panel performance. Food

Quality and Preference, 14, 247–256.McEwan, J. A., Hunter, E. A., Van Germert, L. J., & Lea, P. (2002b).

Proficiency testing for sensory profile panels: measuring panel perfor-mance. Food Quality and Preference, 13, 181–190.

Moskowitz, H. R., Munoz, A. M., & Gacula, M. C. (2003). In viewpoints

and controversies in sensory science and consumer product testing.Trumbull, Connecticut: Food and Nutrition Press, Inc.

Munoz, A. M., Civille, G. V., & Carr, B. T. (1992). Sensory evaluation in

quality control. New York: Van Nostrand Reinhold.Perez Elortondo, F. J. (1993). Actividades tecnicas y de control de la

Denominacion de Origen Idiazabal. Sustrai, 28, 46–51.Perez Elortondo, F. J., Barcenas, P., Casas, C., Salmeron, J., & Albisu,

M. (1999). Standardization of sensory methodologies: some applica-tions to protected designation of origin cheeses. Sciences des Aliments,

19, 543–558.

Food Quality Certification an Approach for the Development of Accredited Sensory Evaluation Method

Documents

Transcript of Food Quality Certification an Approach for the Development of Accredited Sensory Evaluation Method