1 of 36

Managing Uncertainty with Systematic Planning for Environmental

Decision Making

3-Day DOE DQO Training

Day 1

2 of 36

PresenterSebastian Tindall

DQO Coordinator

Bechtel Hanford Inc.

3190 George Washington Way

MS H9-03; Room 49

Richland, WA 99352

(509) 372-9195

sctindal@mail.bhi-erc.com

http://www.doe.hanford.gov/dqo

3 of 36

History of the DQO Training

• Funding provided by DOE Headquarters in 1998 to develop the DQO training

• Development of the presentation

• Funding provided by EPA Headquarters to train Region personnel - 2002 to present

• Funding provided by other interested parties (i.e., States, USACE) for training

4 of 36

DOE

EPA

Other

London, England

5 of 36

Number of People Trained

Company Number of People

DOE - HQ 770

EPA - OSWER 542

Other 390

TOTAL 1702

As of October 6, 2004

By Funding entity:

6 of 36

Introduction to Course and Module Objectives, Agenda,

and Key Concepts

Presenter: Sebastian Tindall

15 minutes

DQO Training CourseDay 1

Module 0

7 of 36

The Problem in a Nutshell

Without systematic planning, your agency/company or your client may be subject to huge financial liability

8 of 36

Federal Facility Projects Under the Comprehensive Environmental

Response, Compensation, and Liability Act of 1980 (CERCLA)– Joint, Several, and Strict Liability– Local entities “buy” contamination FOREVER– Deep Pockets

City, County, State attorneys have issued opinions that failure to conduct an independent environmental assessment of the Site could leave them liable for the cleanup of unexpected contamination

9 of 36

Deficiencies and Inconsistencies in Environmental Decision Making

Exist because an integrated decision-making process is not being used to make environmental decisions

Result from the lack of systematic planning Result from failure to manage uncertainty Result from lack of standard documentation Result in decisions that are not scientifically, technically,

or legally defensible Result in vulnerability to third-party challenges

10 of 36

What’s In It For Me(and my Organization)?

Understand that there is inherent uncertainty in all environmental decisions that use data

Reduce possibility of third-party challenges Reduce re-work Reduce unnecessary clean up Cost-effective budget allocation in time of

diminishing budgets

11 of 36

Course Objectives

Learn the concepts behind the jargon Learn to use systematic planning to manage

uncertainty Use the FAM approach for more cost-

effective data collection Use the tools to generate the documentation

and perform calculations

12 of 36

Uncertainty is Inherent in Data and Must be Managed

We are forced to make environmental decisions based on estimates

Estimates always involve errors Errors in estimates are not mistakes If unmanaged, errors in estimates CAN lead to

decision errors which ARE MISTAKES Decision errors must be managed

– Acknowledge– Identify– Quantify

Severe consequences of decision errors mandate a statistical basis

13 of 36

Uncertainty is additive!Analytical

+ Sampling & Sub-sampling

+Natural heterogeneity of the site

=Total Uncertainty

14 of 36

Learn the Jargon

• t-test• UCL - upper

confidence limit• AL - action level• N - target population• n - population units

sampled - population mean

• x - sample mean - population

standard deviation• s - sample standard

deviation

• H0 - null hypothesis

- alpha error rate - beta error rate - width of gray

region

15 of 36

Use Systematic Planning to Manage Uncertainty

The Data Quality Objectives (DQO) Process is a type of Systematic Planning– Step 1: State the Problem– Step 2: Identify the Decisions– Step 3: Identify the Inputs– Step 4: Specify the Boundaries– Step 5: Define the Decision Rules– Step 6: Specify Error Tolerances– Step 7: Optimize Sample Design

16 of 36

The Managing UncertaintyThe Managing UncertaintyApproachApproach

Systematic Planning

Dynamic Work Plan

Real-Time Measurement Technologies

17 of 36

Insert M-cubed graphic

18 of 36

Foundation of Defensible Decisions Guidance

– EPA QA/G-4 (DQOs)– EPA QA/G-9 (Data Quality Assessment - DQA)– Complete documents on web

Training– Days 1 (foundation), 2 (how to) and 3 (how to)– Visual Sample Plan (VSP)

Standardized documentation– DQO e-Workbook (electronic template)

19 of 36

Foundation (cont.) - Tools DQO Implementation Work Process Flow

Chart Scoping Checklist DQO Web Site

– DQO tools and materials– Latest version of all of today’s slides

Visual Sample Plan– Download free software

Data Quality Assessment (coming!)

20 of 36

How Many Samples do I Need?

REMEMBER:

HETEROGENEITY

IS THE RULE!

21 of 36

The following slides show today’s agenda with the content

and times of each module

22 of 36

Module 1Evolution of the DQO Concept

Module Objective– To understand how the DQO Process has

matured over time from a qualitative concept to practical implementation

20 minutes(10 minute break)

23 of 36

Module 2EPA Inspector General Audit

Reports

Module Objective:– To highlight the general findings from

Environmental Protection Agency (EPA) Inspector General Audit Reports and the problems created when the DQO Process is not followed

20 minutes

24 of 36

Module 3How Many Samples do I Need?

Part 1 Module Objective:

– To describe the sources of uncertainty that can lead to errors when making decisions based on environmental sampling results and to discuss how these errors can be managed and controlled

60 minutes

25 of 36

Module 4 Key Concepts Underlying

DQOs and VSP

Module Objective:

– To gain a basic understanding of the key statistical concepts needed to develop defensible sampling plans

60 minutes(75 min lunch break)

26 of 36

Module 5How Many Samples do I Need?

Part 2 Module Objective:

– To describe the sources of uncertainty that can lead to errors when making decisions based on environmental sampling results and to discuss how these errors can be managed and controlled

60 minutes(5 minute “stretch” break)

27 of 36

Module 6How Many Samples do I Need?

Part 3 Module Objective:

– To describe the sources of uncertainty that can lead to errors when making decisions based on environmental sampling results and to discuss how these errors can be managed and controlled

50 minutes(5 minute “stretch” break)

28 of 36

Module 7Introduction to

The EPA 7-Step DQO ProcessSteps 1 - 4

Module Objective:– To understand the role of Steps 1-4 in the

EPA 7-Step DQO Process

30 minutes

29 of 36

Module 8Introduction to

The EPA 7-Step DQO ProcessSteps 5 - 7

Module Objective:– To understand the role of Steps 5-7 in the

EPA 7-Step DQO Process

25 minutes(5 minute “stretch” break)

30 of 36

Module 9DQO Implementation Process

Flow Chart

Module Objective:– To highlight the elements of the flow

diagram for implementing the DQO Process

30 minutes

31 of 36

• Appendix A: – Thomas Grumbly DOE DQO directive,

September 1994

• Appendix B:

– Timothy Fields EPA OSWER directive, June 1999

• Appendix C:

– Expedited Site Characterization Field Quality Assurance Meeting minutes, January 1994

Note: Appendices have been removed from the binders but can be found on our web site:

http://www.hanford.gov/dqo/training/contents1.html

32 of 36

• Appendix D: Effective Data– Dr. Bart Simmons, CAL/EPA DTSC

• Using Field Methods: Court decisions

– Dr. Al Robbat, Tufts University• Dynamic Work Plans & Field Analytics

– Sebastian Tindall, BHI

• ESC M-Cubed Approach

• Culture Change Flow Chart

33 of 36

• Appendix E: – List of Acronyms and Symbols used in course

materials

• Appendix F:

– U.S. EPA ORDER 5360.1 A2 CHG 2 May 5, 2000

• Appendix G:

– Science Advisory Board Report to EPA:

• Report

• Enclosure D

(http)://www.hanford.gov/dqo/training/contents1.html)

34 of 36

We want to help you:

Do it!Do it!(Get the job done - right)

Prove it!Prove it!(Document what/why/how)

CertifiedCertified

35 of 36

EM-3 DQO TRAINING, Part 1 – EPA Region X DAY 1 COURSE/INSTRUCTOR EVALUATION FORM

Course Title: Managing Uncertainty and Systematic Planning for Environmental

Decision-Making __________________________________________________

Instructor: Sebastian Tindall

Date: _______________ Time:________________

Please check your assessment of the following statements: Range: 5= completely agree 1 = disagree

5

4

3

2

1

1. The objectives of the module were understood and accomplished

2. There was adequate coverage of the subject matter

3. The subject matter was organized

4. The instructional tools were appropriate and organized

5. Enough time was allocated to cover module material

6. The instructor was effective

8. I would recommend the module to colleagues

If recommended, to whom:________________________________

9. What did you like best about the module? List the strong points.

_______________________________________________________________________________________

_____________________________________________________________________________________

10. What would you change? Are there any improvement opportunities?

_______________________________________________________________________________________

_____________________________________________________________________________________

11. What was one thing (insight, learning) you found of most value?

_______________________________________________________________________________________

_____________________________________________________________________________________

12. Was there enough time to cover the material? If no, please explain.

_______________________________________________________________________________________

_____________________________________________________________________________________

13. Any constructive feedback for the instructor?

_______________________________________________________________________________________

_____________________________________________________________________________________

Name (Optional):________________________________________________________________________

36 of 36

End of Module 0

Thank you

Questions?