Post on 22-Aug-2020
UNCERTAINTY INTERVALS IN
CERTIFIED REFERENCE MATERIALS
FOR NUTRIENTS IN FOODS
Wayne R. Wolf1, Katherine M. Phillips2
1. Food Composition Methods Development Laboratory, Beltsville Human Nutrition Research Center
United States Department of Agriculture, Beltsville, Maryland, 20705, USA
2. Biochemistry Department , Virginia Polytechnic Institute and State University, Blacksburg, VA, 24061
The Future of Reference Materials – Science and Innovation
IRMM, Geel, Belgium, 23-25 November 2010
• 30 plus years of helping to provide Food Related CRMs ----
• Initial Question was: How many Food CRMs do we need?:
• Answer is: ?
• Present Question: How Good do the Food CRMs need to be?
1987 RMs for biological RMs
exclusively for elements (BERM-2)
11987
Parr, et. al. Fresenius Z. Anal. Chem (1987) 326:601-608.
1992 Organic Components in food RMs
(BERM-5)
• Major components and major elements in five food RMs - BCR
• Determinations of vitamins in food and preparation of RMs for vitamin analysis –BCR
• Trace organics in food and agriculture materials (4 papers)
– Agrochemical residues
1994 System for defining food matrices
(BERM -6)
1997 Food CRMs characterized by
collaborative process (BERM-7)
2007 Summary of Food RMs
(BERM 10)
USDA National Food Nutrient
Analysis Program
• NFNAP was designed to update the USDA nutrient database (USDA Standard Reference, SR) by collecting foods consumed in the US according to a statistical sampling plan for analysis of key foods and nutrients
• Over 3000 samples of more than 875 food items have been collected and analyzed for more than100 nutrients and other dietary components as part of the NFNAP
Use of CRM in NFNAP• NFNAP QC system included CRM and control
samples developed specifically for the program
• Blind samples of CRM were sent to labs for
analysis to
– qualify for a NFNAP contract
– QC analysis with routine unknown samples
• 2600+ values from 9 laboratories for 77 nutrients
in 26 CRM have been obtained over 6.5 years
NFNAP CRM Data Set
• What can this set of data, generated by contract analytical laboratories, on well characterized known samples, tell us about the general state of practice of the measurement system for generating food composition data?
• This is not an interlaboratory evaluation, nor an assessment of individual methodology.
• These data can give insight of the present state of practice of the measurement system for generating food composition data.
Goals
• Overview of Food CRM availability by matrix and nutrient
• Uncertainty intervals for CRM
• Preliminary assessment of results from routine analysis by contract laboratories during NFNAP for nutrients in CRM with certified or reference values
– Nutrient level issues
– Matrix issues
• Begin dialog of how good do CRM need to be (?)
Identification of Food CRM
• Knowledge of existing suppliers
• IAEA Database for Natural Matrix Reference
Materials
• Supplier websites (selected)
– National Institute of Standards and Technology (NIST)
– Institute for Reference Materials and Measurements (IRMM)
– American Association of Cereal Chemists (AACC)
– National Research Centre for Certified Reference
Materials (China)
– National Research Council of Canada (NRC )
– LGC Promochem
Matrix Triangle
1
2
3
4
56
7 89
1
2
3
4
56 7 8
9
NIST 1563
LGC 7104
NIST 2384
NIST 2387
NIST 8415
NIST 1546
LGC 7106
LGC 7000
LGC 7150
LGC 7151
LGC 7152
100% Fat
100% Protein100% Carbohydrate
SRM 3244
NIST 8418
SRM 1974a
NIST 1946
LGC 7176
NIST 1566b
LGC 7001NIST 1570a
NIST 2385
NIST 8435
NIST 1544
NIST 1548a
NIST 1846
LGC 7105
NIST 8432
NIST 8433
NIST 8436
NIST 2383
BCR-382
BCR-381
LGC 7107
LGC QC1002
SMRD 2000
LGC 7101
LGC 7103
Nutrient / CRM Availability1= certified
2= reference
Sector RM M M Pr Ash Fat Suc Fru Glu Lac Mal Gal Fib St5 BCR 121 WholemealFlour5 BCR 189 WholemealFlour5 BCR 381 RyeFlour 1 1 15 BCR 382 WheatFlour 1 1 15 BCR 383 FreezeDriedGreenBeans 1 15 BCR 431 FreezeDriedBrusselsSprouts5 BCR 485 FreezeDriedMixedVeg5 BCR 679 FreezeDriedWhiteCabbage5 ERMBC 515 FreezeDriedCarrots 15 ERMBC 516 FreezeDriedApple 15 ERMBC 518 DriedBranBreakfastCereral 15 GBW 07605 Tea5 GBW 08508 Rice
5 LGC 7003 WhiteBreadFlour 1 15 LGC 7017 SugarConfectionery 1 1 1 15 LGC 7103 SweetDigestiveBiscuit 1 1 1 15 LGC 7107 MadeiraCake 1 1 1 1 15 NIST 1567a WheatFlour5 NIST 1568a RiceFlour5 NIST 2383 Baby Food 2 2 2 2 25 NIST 8432 CornStarch 2 2 2 25 NIST 8433 CornBran 2 2 2 25 NIST 8436 DurumWheatFlour 2 2 2 2 25 VMA 195 Cereal 1 1 1 15 VMA 399 Cereal 1 1 1 1
Proximates CHO
CRM Over All Sectors1= certif ied
2= reference
Sector RM M M Pr AshFat Suc Fru Glu Lac Mal Gal Fib St Ca K Mg Na P Cu Fe Mn Se Zn A B1 B2 Nia B6 B12Fol PA C D E Car K FA
BCR 122 Margarine
BCR 162 SoyaMaizeOil
BCR 163 BeefPorkFat Blend
BCR 164 AnhydrousMilkFat
LGC 7104 SterilizedCream
NIST 1563 FortifiedCoconutOil
2 NIST 2384 BakingChocolate
LGC 7112 PigLiverPaste
NIST 2387 Peanut Butter
GBW 08552 FreezeDriedPorkMuscle
LGC 7000 BeefPorkMeat
LGC 7101 MackerelPaste
LGC 7106 ProcessedCheese
LGC 7152 ProcessedPork
NIST 1546 MeatHomogenate
NIST 1845 WholeEggPowderNIST 8415 WholeEggPowder
BCR 121 WholemealFlourBCR 189 WholemealFlour
BCR 381 RyeFlour
BCR 382 WheatFlour
BCR 383 FreezeDriedGreenBeans
BCR 431 FreezeDriedBrusselsSprouts
BCR 485 FreezeDriedMixedVeg
BCR 679 FreezeDriedWhiteCabbage
ERMBC 515 FreezeDriedCarrots
ERMBC 516 FreezeDriedApple
ERMBC 518 DriedBranBreakfastCereral
GBW 07605 TeaGBW 08508 Rice
LGC 7003 WhiteBreadFlour
LGC 7017 SugarConfectionery
LGC 7103 SweetDigestiveBiscuit
LGC 7107 MadeiraCake
NIST 1567a WheatFlour
NIST 1568a RiceFlour
NIST 2383 Baby Food
NIST 8432 CornStarch
NIST 8433 CornBranNIST 8436 DurumWheatFlour
VMA 195 Cereal
VMA 399 Cereal
BCR 421 MilkPowder
LGC 7105 RicePudding
NIST 1544 Total Diet
NIST 1548a FreezeDriedMixedDiet
NIST 1846 InfantFormulaNIST 8435 WholeMilkPowder
LGC 7001 PorkMeat
NIST 1549 NonFatMilkPowder
NIST 1566b FreezeDriedOysterTissue
NIST 1570a FreezeDriedSpinachLeavesNIST 2385 SlurriedSpinach
LGC 7176 CannedCatFood
NIST 1946 FishTissue
BCR 422 FreezeDriedCodMuscle
BCR 487 FreezeDriedPigsLiver
LGC 7160 CrabPaste
NIST 1577b Freeze-DriedBovineLiver
NIST 8418 WheatGlutenNRCDORM 2 FreezeDriedDogfishMuscle
ERMBC 514 FreezeDriedHaricotBeans
ERMBC 517 DriedFullFatSoya
ERMCE 278 FreezeDriedMuscleTissue
LGCQC 1003 Chocolate NRCDOLT 3 FreezeDriedDogfishLiver
9
5
4
6
7
Vitamins Other
1
3
Proximates CHO Minerals Trace elements
8
Uncertainty Intervals for CRM
• Assigned uncertainty value in the CRM
• Uncertainty intervals as a % of target values
were determined
• UIP = uncertainty / target*100
Un
ce
rtain
ty a
s %
of T
arg
et V
alu
e
(UIP
)
0
10
20
30
40
50
60
70
80
90
1
51
101
151
201
251
301
351
401
451
501
551
601
651
701
751
(Uncertainty / Target value ) *100
Comparison of UIP for Vitamins and
Minerals
0
5
10
15
20
25
30
35
40
45
501
11
21
31
41
51
61
71
81
91
101
111
121
131
141
151
(Un
cert
ait
y/T
arg
et
valu
e)*
100
Vitamins
Minerals
UIP by Nutrient Class and
Concentration
0
10
20
30
40
50
60
70
80
90
1.0E
-04
1.0E
-03
1.0E
-02
1.0E
-01
1.0E
+00
1.0E
+01
1.0E
+02
1.0E
+03
1.0E
+04
1.0E
+05
Nutrient concentration (mg / 100 g)
UIP
Prox Carbo Min Trace VitWaterS VitFatS AA FA Other
UIP Summary
• Most UIP for CRM nutrients were less than 10%, but some had a far greater uncertainty
– 64% of the UIP were <10%
– 25% were 10%-20%
– 11% were >20%
• The UIP for proximates, minerals, and trace elements were most consistently <10% of the assigned value
• On average UIP were significantly higher for vitamins
Use of CRM in NFNAP• NFNAP QC system included CRM (as well as control
samples developed specifically for the program*)
• Blind QC samples, including the CRM were sent to labs for analysis
– To qualify for a NFNAP contract
– As QC samples batched with routine NFNAP foodsamples
• All values are shown, including results where labs did not qualify for specific nutrients
• 2600+ values from 9 laboratories for 77 nutrients in 26 CRM have been obtained over 6.5 years
*Phillips, K.M., Patterson, K.Y., Rasor, A.S., Exler, J., Haytowitz, D., Holden, J.M., and Pehrsson, P. (2006). Quality-control materials in the USDA National Food and Nutrient Analysis Program. Anal. Bioanal. Chem., 384(6): 1341-1355.
Evaluation of Lab Results
• Each reported nutrient value was evaluated for deviation from the target certificate value
– (Result – target value) / uncertainty interval
– (“Z Score”)
– Calculation designated as UI Unit (UIU)
• A value within +/-1 UIU lies within the assigned certificate concentration range
Vitamins
Laboratory
UIU
−20
−10
0
10
20A Thiamin Riboflavin Niacin B6 B12 Folate Panto
AcidC α Carot
A B C D E A B C D E A B C D E A B C D F A B C D F A C D HA B C DA C DA C DA C DA C D
tocoph
UIU by Both Lab and Matrix
• Overall summary of UIUs allows a preliminary
identification of nutrient analyses that may be of
inadequate accuracy and/or precision
• Further, nutrient – matrix combinations that
present analytical challenges can be identified
-18-16-14-12-10
-8-6-4-202468
101214161820
1 5 9
13
17
21
25
29
33
37
41
45
49
53
57
61
65
69
73
(Re
su
tls
-TV
) /
Un
ce
rta
inty
In
terv
al
Niacin By Lab and Matrix
Lab A �
Lab B �
Lab C �
Lab E �
Lab D �
Milk/Infant formula
Meat
Babyfood
Vegetables/Peanut butter
-8
-7
-6
-5
-4
-3
-2
-1
0
1
2
3
4
5
1 5 9
13
17
21
25
29
33
37
41
45
49
53
57
61
65
69
73
(Re
su
lts
-TV
) / U
nc
ert
ain
ty In
terv
al
Thiamin by Lab and Matrix
Lab A �
Lab B �
Lab C �
Lab E �
Lab D �
Flour/Cereal
Milk/Infant formulaMeat
Baby food
Vegetables/Peanut butter
CRM ID Year Nia UIP RiboUIP Thia UIP Pyri UIP
NIST1846 1996 12 5.7 13.8 11.9
BCR 431 1997 7
NIST 2383 1997 12 14.1 16.5 14.6
BCR 121 1998 8.4 24.9
BCR 485 1998 11.1 16.7
BCR 487 1998 5.2 12.8 15
NIST 1546 1999 10.5 29.5 20.4 46.9
NIST 8435 1999 12.2 30.9 28.9 31.4
NIST 2384 2001 16.5 13.2
NIST 2387 2003 4.2 20.2 13.3
NIST 2385 2003 14.4 6.8
NIST 3244 2006 3.3 7.7 17.6 6.5
NIST1849 2009 2.4 5.7 8.2 10.5
NIST 3280 2009 1.6 12.8 11.3 9.4
GBW 10037 2010 5.6
NFNAP Composite Control Samples
• * Composite Control food samples sent as blind QC samples with each batch for contract analysis
• Evaluation of >25,000 values assayed at multiple labs over 6.5 years for over 100 nutrients is planned to further evaluate lab performance and measurement reproducibility
• .
Pyridoxine RSD vs Concentration
Data from KM Phillips, VPI&SU Personal communication
Dietary Supplements Quality
Assurance Program- NIST/ODS
How Good do the Data Need to Be?
Fit for Purpose Factors
• Importance of the nutrient in the diet
• Concentration of the nutrient in the food
• % of nutrient requirement in a serving
• Application of the data, e.g.
– Food composition databases
– Validation of diets for clinical feeding trials
(highest precision and accuracy needed)
– Food labeling
Evaluation of
Sources of Method Bias
• Recovery - Spiked amounts
• Difference from CRM –”True Value”
– Calibration errors
– Actual Method Bias - Matrix effects
(Extraction, Detection)
– Requires Tight Uncertainty Limits on CRM
to determine extent of bias.
Conclusions For Food Related CRMs
• Uncertainty intervals need to be tighter for many food CRMs to validate accuracy of results (e.g. definitive rather than consensus certification)
• Suggested UI of < 5%
• UI for elements in foods are at this level
• UI for vitamins are approaching this level
• CRM should be renewed ca. every ten years to reflect advances in technology
Thank you
for listening
Stakeholder Panel for Infant Formula and Adult Nutritionals
8-12 November 2010, AOAC INTERNATIONAL
• Objective: To develop standard method performance requirements for priority nutrients…
• First group: Vitamins A, D, B12, Folic Acid and inositol.
• Fitness-for-Purpose statements were developed in September
Vitamin A
Codex Upper Limit 125
ug/100g, RTF Typical Label Claim 63
Codex Lower Limit 42
SMP Model Performance Required (no bias, no product uncertainty) CUL to CLL CUL to LC
Max Intermed Precision (RSDI) 8.3 5.5
Max Interlab Reproducibility(RSDR) 14.4 9.6
Max Overall Interlab RSD 16.7 11.1
Evaluation of
Sources of Method Bias
• Recovery - Spiked amounts
• Difference from CRM –”True Value”
– Calibration errors
– Actual Method Bias - Matrix effects
(Extraction, Detection)
– Requires Tight Uncertainty Limits on CRM
to determine extent of bias.
SMPR Vitamin A
SPIFAN –SMPR
• Process is developing SMPR initially for 5 components in IF/AN
• Will develop SMPR for at least 15 more in 12-24 months.
• Should be able to gain estimates of required level of UI for CRMs in IF/AN which can be a basis for these 20 components in a wider range of foods.
NFNAP Composite Control Samples
• * Composite food matrix control materials sent as blind QC samples with each batch for contract analysis
• * 25,000 data points.
How Good do the Data Need to Be?
Current Thinking and Future Plans
• CRM are needed to evaluate analytical accuracy
and to provide interlaboratory reference data
• In-house QC materials for precision:
– Retrospective inspection of NFNAP control sample results planned to evaluate lab precision
• To evaluate new methodology and changes in
methods – material challenges
• Need better definition of “Fit for Purpose” issues
2000 Food Matrix CRM development (BERM-8)
2003 Food CRMs for Vitamins available
(BERM-9)
Food and Nutrition MetrologyGeneration of High Quality Data
Measurement System Requirements
* Appropriate Validated Analytical Methods that are fit for purpose– Appropriate precision
– Adequate selectivity
– Needed sensitivity
* Accuracy assessed with appropriate external matrix standards (CRM’s)
• Proficiency of Analysts –Appropriate QC System
Vitamin D
Codex Upper Limit 1.7
ug/100g, RTF Typical Label Claim 1
Codex Lower Limit 0.7
SMP Model Performance Required (no bias, no product uncertainty) CUL to CLL CUL to LC
Max Intermed Precision (RSDI) 7.1 4.2
Max Interlab Reproducibility(RSDR) 12.4 7.2
Max Overall Interlab RSD 14.3 8.3
SMPR Vitamin A - Initial Draft