Data Quality and Measurement Quality Objectives for the National Air Toxics Trends j

Program

Dennis K MikelDennis K. MikelEPA – OAQPS

AT Workshop 2011

•Urban Sites •Rural

•E. Providence, RI•Boston (Roxbury), MA•New York, NY•Rochester, NYW hi t DC

•Chicago, IL•Houston (Deer Park), TX•St. Louis, MO•Bountiful, UTS J CA

•Underhill, VT• Grayson Lake, KY•Chesterfield, SC•Grand Junction, CO•Washington, DC

•Decatur, GA•Tampa, FL (2)•Detroit, MI•Los Angeles, CA•Rubidoux, CA

•San Jose. CA•Phoenix, AZ•Seattle WA•Richmond, VA•Portland, OR

CO•La Grande, OR•Harrison County, TX•Mayville, WI

NATTS QA ObjectiveNATTS QA Objective

Data Quality Objective (DQO) is in support of the GPRA goal of reduction of Air Toxics by 75% (1993 levels) by 2010:

“To be able to detect a 15% difference (trend) between two successive 3-year annual mean concentrations within acceptable levels of decision error.”

To meet this DQO we need:

• 1-in-6 day sampling frequency with at least an 85% quarterly completeness

• Precision controlled to a Coefficient of Variance (CV) of no more than 15%

• Detectability based on 2001 Pilot Study Minimum Detection Limits (MDLs)

• Bias for the data set of less than 25%• Bias for the data set of less than 25%

These are our Measurement Quality Objectives (MQOs)!

Measurement Quality ObjectivesMeasurement Quality Objectives

How are these determined?

• Precision: Determined by analyzing collocated/duel samples or analyzing sample twice (replicate)

• Bias: EPA sends out blind PT samples to the labs – report back results

• Completeness: Calculated from number of samples reported to AQS divided by possible samples

• Detectability: Determined by laboratories using EPA Method of 7 duplicate samples near the level of detection

NATTS QA ProgramNATTS QA Program

Measurement Quality Objectives (MQOs)Measurement Quality Objectives (MQOs)

Compound Precision Bias Detectability Completenessp(CV) (Lab)

y p

Arsenic < 15% < 25% 0 217 ng/m3 > 85%Arsenic < 15% < 25% 0.217 ng/m3 > 85%

Benzene < 15% < 25% 0.016 ug/m3 > 85%

1 3-Butadiene < 15% < 25% 0 013 ug/m3 > 85%1,3-Butadiene < 15% < 25% 0.013 ug/m3 > 85%

Formaldehyde < 15% < 25% 0.0074 ug/m3 > 85%

Meeting Objectives: Precision Results 2004 - 2008Five Year Average:

Benzene: 25%NATTS - Method Precision 2004 - 2008

1,3 Butadiene 19%

Formaldehyde: 16%

Arsenic: 12% 50

60

Only Arsenic is meeting our Precision MQO

30

40

% C

V

Benzene

1,3 Butadiene

10

20

30%

Formaldehyde

Arsenic

0

10

2004 2005 2006 2007 2008

Years

M ti Obj ti O ll P i i A l ti l P i i 2008Meeting Objectives: Overall Precision vs. Analytical Precision -2008

Arsenic

Analytical Precision t f

1 3 Butadiene

Formaldehyde

Method

Analytical

accounts for laboratory practices and instrumentation

M th d P i i

Benzene

1,3 Butadiene y Method Precision accounts for analytical precision, sample handling and fi ld ti

0 5 10 15 20 25 30 35 40 45 50 55 60

Coefficient of Variance (%)

field operations

Meeting Objectives: Overall Precision of 1,3 butadiene + Benzeneg j ,

Benzene 1,3, Butadiene

Meeting Objectives: Method Detection Limits 2004 2008

2.00

Benzene 1,3 BD Formaldehyde Arsenic

Meeting Objectives: Method Detection Limits 2004 - 2008

0.50

1.00

0.13

0.25

/m3

/m3

2004

2005

2006

2007

0.03

0.06

ug

/n

g/ 007

2008

MQO

0.01

0.02

0.00

Meeting Objectives: Method Detection Limits 2004 – 2008 eet g Object es et od etect o ts 00 008

1.2

8

1

0.6

0.8

ug

/m3

ng

/m3

Benzene

1,3, BD

0.4

u n Formaldehyde

Arsenic

0.2

0

2004 2005 2006 2007 2008

Meeting Objectives: Data Completeness 2004 – 2008

120

80

100

enes

s

60

benzene

1,3 Butadiene

Formaldehydennua

l com

plet

e

40

Arsenic

% a

n

20

0

2004 2005 2006 2007 2008

Meeting Objectives: Proficiency Testing: Arsenic 2004- 2009

5

Meeting Objectives: Proficiency Testing: Arsenic 2004- 2009

4

3

s (n

g/fil

ter)

2

Lab

Res

pons

e

0

1

L

00 1 2 3 4 5

True (ng/filter)

12

Meeting Objectives: Proficiency Testing: Benzene 2004- 2009

10

12

6

8

e pp

b-v)

4

6

Lab

Res

pons

2

00 2 4 6 8 10 12

True (ppb-v)

M ti Obj ti P fi i T ti 1 3 B t di 2004 2009

14

Meeting Objectives: Proficiency Testing: 1, 3 Butadiene 2004- 2009

10

12

v)

6

8

spon

ses

(ppb

-v

4

6

Lab

Res

0

2

0 2 4 6 8 10 120 2 4 6 8 10 12

True (ppb-v)

Meeting Objectives: Proficiency Testing: Formaldehyde 2004- 2009

4.5

5

g j y g y

3.5

4

2.5

3

es (u

g/fil

ter)

1.5

2

Lab

Res

pons

e

0.5

1

0

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

True (ug/filter)

Meeting Objectives: Acrolein PT Results 2006 - 2009

10

7

8

9

5

6

7

pons

es (p

pb-v

)

3

4

Lab

Res

p

1

2

00 1 2 3 4 5 6 7 8 9 10

True (ppb-v)

Summary: Is the Program Able to Meet the DQOs?

Yes and NoYes and No

The data completeness (2007-2008) is above the required 85% good job everyone!

Overall precision data illustrates that we are not meeting a CV of less than 15% with the exception of Arsenic

Th MDL d t ill t t th t t ti The mean MDLs data illustrates that we are not meeting our MQO for detectability with the exception of Arsenic

The laboratories are meeting the 25% bias requirement

Take Home Message: We now have a realistic view of the quality of Air Toxics data - This will guide us in the formulating new Data Quality Objectives