Model Contract ImplementationModel Contract Implementationat Sandia National Laboratoriesat Sandia National Laboratories

Risk Based OversightRisk Based Oversight

Dan PellegrinoDan PellegrinoSandia Site Office Sandia Site Office

November, 2007November, 2007

NNSA Report to Congress

• NNSA will redesign the federal-contractor relationship to improve accountability.

• NNSA will streamline oversight by clarifying NNSA authorities and responsibilities, coordinating with DOE and other external overseers, evaluating systems – not transactions, and redefining federal jobs.

• NNSA will lift administrative burdens through streamlining policies, procedures, and staffing.

Goals of the 2004 Sandia Model Contract

The Model Contract was established between NNSA and SNL in FY04 with the following goals in mind:

• Redesign the Federal/ Contractor Relationship

• Capitalize on Private Sector Expertise

• Increase Contractor Accountability

• Implement a Simpler, Less Adversarial Contracting Model

Sandia is implementing the model contract and contractor assurance through the Integrated Laboratory Management System

Model Contract Objectives

1. Provide a model for implementing an operational Contractor Assurance System (CAS) with an increase in contractor accountability and a shift in federal oversight

2. Capitalize on industry standards and outside certifications3. Introduce a more disciplined Work Authorization process through

contracting officer (CO) and contracting officer representative (CORs) appointments and Performance Direction clause

4. Streamline the contract by reducing non-applicable clauses5. Provide a framework for systems oversight of non-nuclear work6. Allow investment of cost efficiencies with onsite investment as an

incentive7. Increase involvement of the parent entity and Sandia Corporation Board

of Directors8. Improve performance through benchmarking with private sector and

NNSA contractors9. Enhance coordination within the Nuclear Weapons Complex (NWC)10. Emphasize Sandia’s accountability for daily operations

Twelve Special H Clauses

• Clause H-1, Redefining the Federal/Contractor Relationship to Improve Management and Performance

• Clause H-2, Performance Direction• Clause H-3, Contractor Assurance System• Clause H-4, NNSA Oversight• Clause H-5, Accountability• Clause H-6, Standards Management• Clause H-7, Utilization of Parent Corporate Systems• Clause H-8, Contractor Reinvestment of Cost Efficiencies• Clause H-9, Contractor Multi-Year Laboratories’ Vision for Continuous

Improvement• Clause H-10, Performance Based Management• Clause H-11, Performance Incentives• Clause H-12, Award Term

H - Clause Requirements

• Clause H-1, Redefining the Federal/Contractor Relationship to Improve Management and Performance– NNSA focuses on the “what” and the contractor is responsible for the “how”.

• Clause H-3, Contractor Assurance System– The Contractor shall develop a Contractor Assurance System that at a minimum has nine key

attributes. • Clause H-4, NNSA Oversight

– Once NNSA is satisfied that the Contractor Assurance System is operating effectively, NNSA will conduct oversight of the Contractor’s Non-Nuclear Facilities operations, Programs, Projects and Business Systems at the systems level.

• Clause H-5, Accountability– The Contractor is responsible for the quality of its products and for assessing its operations,

programs, projects and business systems and identifying deficiencies and implementing needed improvements in accordance with the terms and conditions of this Contract, regardless of whether NNSA has evaluated the Contractor’s performance in any area of the Contract. The purpose of NNSA oversight is for assessing the Contractor’s performance in meeting its obligations under this Contract. NNSA oversight shall not be relied upon by the Contractor in assessing its performance.

• Clause H-6, Standards Management– The Contractor shall regularly benchmark with industry to identify best commercial standards and best

business practices that will improve site operations with the goal of improving performance where cost effective.

• Clause H-7, Utilization of Parent Corporate Systems– If the Contractor adopts or adapt its parent corporate systems or services, it will ensure that the

Government and Contractor’s data in such systems is readily transferable to a successor contractor.

H - Clause Requirements (Continued)

• Clause H-8, Contractor Reinvestment of Cost Efficiencies• Clause H-9, Contractor Multi-Year Laboratories’ Vision for Continuous Improvement • Clause H-10, Performance Based Management• Clause H-11, Performance Incentives• Clause H-12, Award Term

Clause H3: CAS Nine Attributes

• A comprehensive description of the Contractor Assurance System with risks, key activities and accountabilities clearly identified.

• A Process for notifying the Contracting Officer of significant assurance system changes.• Rigorous, risk based credible self-assessments, feedback and improvement activities,

including utilization of nationally recognized experts, and other independent reviews to assess and improve its work process and to carry out independent risk and vulnerability studies. The Contractor is encouraged to seek third party certifications (such as VPP and ISO 9001 or ISO 14001), audits, peer reviews and independent assessments with external certification or validation.

• Identification and correction of negative performance/compliance trends before they become significant issues.

• A method for validating assurance processes.• Integration of the assurance system with Contractor management systems including Integrated

Safety Management.• A process for defining performance metrics and performance targets to assess performance,

including benchmarking of key functional areas with other NNSA/DOE contractors and industry and research institutions to enhance processes and to assure development of performance metrics and performance targets that will result in achievement of best in class/industry performance where efficient and cost effective.

• Continuous feedback and performance improvement. • An implementation plan that defines a transition period for the implementation to the

Contractor Assurance System.

SNL Design of the Contractor Assurance System

1. SNL adopted Integrated Laboratory Management System (ILMS) as management system for all of SNL

2. ILMS has an Assess, Assure and Improve Component that serves as the Contractor Assurance System

3. SNL completed 7 integrated Corporate Process Requirements as Framework for CAS

a. Enterprise Risk Management

b. Corporate Quality Assurance Program

c. Corporate Work Process

d. Corporate Issues Management Program

e. Corporate Self-Assessment Process

f. Corporate Corrective Action Process

g. Corporate Comparison Activities

SSO Risk-Based Oversight • Mapped federal and contractor requirements for 113 DOE orders (except cyber) in SNL contract and determined current oversight method, establishing oversight baseline. These map into 38 Functional Areas of

oversight.• Developed a risk-based oversight procedure consistent with DOE O 226.1 and trained federal staff.• Federal staff trained in systems level oversight, DOE O 226.1, and the Sandia ILMS/CAS.• Chartered a risk-based oversight board (RBOB) to review each SSO oversight area.• Prior to the RBOB, each Assistant Manager:

– Reviews federal requirements– Evaluates and scores SNL program health for their applicable SSO oversight areas (38 total for SSO)– Evaluates and scores each Functional Area’s implementation of CAS for implementation consistent with SNL’s 7 CAS corporate policy requirements– Recommends risk level (H/M/L) for federal oversight, based on SNL program health and CAS confidence– Risk level determines SSO method of oversight--systems or transactional – Recommends tailor/alter/exemptions to federal requirements as necessary for contractor accountability or clarification of roles and responsibilities

• RBOB evaluates input and makes recommendations to the Site Manager for final approval.

Understanding Requirements of OversightFederal Requirements Oversight Group:

• Objective – Determine baseline of federal & contractor requirements and status of current oversight

• Reviewed SNL contract to identify contractor and federal oversight requirements

• 113 directives – 7,752 lines with 6,676 contractor and 3,812 federal requirements

– Security (45%), Business (23%), ES&H (23%), F&PM (3%), Program (2%), Nuclear Safety (2%)

– Interviewed SMEs for 105 of 113 directives to identify status of current federal oversight of contractor

113 directives – 7,752 lines with 6,676 contractor and 3,812 federal requirements

DOE O 226.1 -- Six Oversight Techniques: Steps and

Process Flows for SSO Performing Oversight

Is Requirement being Met?

226 Elements(6 Oversight

Types)

FEDERAL REQUIREMENTS

Appendix G (SNL Contract)

Non CRD Directive

Documented in Appendix B:

RBOP

MAS Results

SSO Self- Assessment Procedure

JPRT/JPC

MAS

SSO Procedure 803.03 (PEP/PER)

Pegasus(SSO 703.03)

PegasusDOMaster

Assessment Schedule (MAS)

Documented Risk Via Table 10

Systems Oversight

Perform

SNL PEAR/SSO PER

Transactional Oversight

NO

Yes

SelectContract Incentive

Annual Basis

External Assessments (2)

CAS (3)

SSO Assessment s (4) and

Operational Awareness (5)

Performance Evaluation Plan (PEP)

(6)

Input consideration:

SNL SA Schedule

Corporate SA Schedule

Past Performance

Other

PerformNotification of

External Assessment

Select Tools for Oversight

Annual Feedback

Self-Assessments (1)

SSO CAP as applicable

See Figure 2 (Steps, Process Flow, and Responsibilities for Reporting to the RBOB)

Report reason to RBOB and

Document in Appendix B,

RBOP

Steps, Process Flow, and Responsibilities for Reporting to the RBOB

Step 2: Determine FA CAS

Confidence by Pegasus Function/Element if Element

is stand-alone program)

Review All elements of 7

processesTable 5

Assign effectiveness

grade in Table 5Based on criteria

in table 3

Assign transparency

grade in Table 5based on criteria in

table 4

Enter CAS Confiedence

grades into table 5

Enter FA/Element CAS confidence

level(High/Med/Low)

into Table 5

Step 1: Determine

Functional Area (FA) Program

Health by Pegasus Function

Using criteria, rate FA program Health

(Tables 1 and 2)

Health Rating (Outstanding,

Good, Satisfactory,

Unsatisfactory)

Step 3: Determine Mitigated Risk of

Federal Requirements

RBOB recommends to SOM

Alter It

Waive It

Tailor It

Recommend Do, Tailor, Waive or Alter of Federal Requirements

and Transactional/

Systems Oversight by Pegasus FA

Determine consequence of occurrence

High – Medium – Low(Table 8)

Determine probability of occurrence

High – Medium – Low(Table 9)

(If federal oversight withdrawn, considering

Program Health and CAS Confidence)

Determine mitigated Risk

(Table 10)(High/Med/Low)

Systems Oversight

Transactional Oversight

Do It

Apply DOE O 251.1B and/or DOE M 251.1-1B

See Figure 1, Steps, Process Flows and Responsibilities for Performing Oversight

Continuing Performance Review through JPRT/JPC

Program Health Sample Evaluation TableTopic Scored Score Number of Years Since Last External Inspection Evaluate if there are areas that were independently assessed (e.g., IG/GAO) Has been over 3 years since last review = 1 point Has been over 2 years since last review = 2 points Has been reviewed within last year = 3 points 3

Findings from External Inspections

No Significant Findings or Recommendations = 3 points Recommendations/Findings that resulted in process changes= 2 points One to Two Findings that indicated a SNL system failure=1 point 1 Three or More Findings that indicate a SNL System Failure =0 points Internal Self Assessments Evidence of a Continuous Self-Assessment Program = 3 points 3 Evidence of Limited Self-Assessments in 3 or more areas = 2 points Evidence of a Self-Assessment in one area= 1 point No evidence of a self-assessment = 0 points PEP/PER Evaluations

Evidence of meeting negotiated requirements with no major findings that indicated a less than satisfactory and effective program = 3 points 3 Performance did not meet all negotiated requirements or there was one major finding that indicated a less than satisfactory and effective program = 2 points Performance did not meet negotiated requirements and/or there were major findings that indicated a less than satisfactory and effective program = 0 points External Certification Indicating Health of Program External Certification/Registration by a Third Party = 3 points 3 On Path for External Certification/Registration by a Third Party = 1 points Total 13

Overall Rating: 13/5 = 2.6 or Outstanding

SSO Risk Based Oversight Procedure Confidence in CAS Process (1/5)

SNL CAS Assurance Process

Process Weight

(%)

SNL CAS Assurance Process Elements

Implementation (I)

(Table 3)

Transparency (T) (Table 4)

CAS Confidence

(I+T)/2

Adjectival Rating

(Table 6)

Comments

A. RISK MANAGEMENT

30 1 Planning strategies, goals, and milestones

99 99 99 High

2 SMU approved work scope 90 90 90 High

3 LLT, SMU, Div and EPSs identified risks

89 89 89 Medium

4 understand interfaces, dependencies, trends and combinations

70 70 70 Medium

5 Identify new/increased risks 69 69 69 Low

6 Clear line of sight 80 85 83 Medium

7 LLT, SMU, Div and Pas implement controls, conduct assessment, complete CAs, reports results and metrics

80 95 88 Medium

8 Monitor at Lab level high level residual risks, quarterly lab performance annual assurance reports

70 80 75 Medium

9 Revisit Planning 80 85 83 Medium `

SSO Risk Based Oversight Procedure Confidence in CAS Process (2/5)

SNL CAS Assurance Process

Process Weight

(%)

SNL CAS Assurance Process Elements

Implementation (I)

(Table 3)

Transparency (T) (Table 4)

CAS Confidence

(I+T)/2

Adjectival Rating

(Table 6)

Comments

OVERALL SCORE FOR

RISK MANAGEMENT 83 Medium

B. SELF-ASSESSMENT

20 1 Inputs (customer requirements)

60 75 68 Low

2 Plan assessments 65 85 75 Medium

3 Implement Controls (competent personnel, adequate tools, metrics, performance trend data)

70 95 83 Medium

4 Perform assessments (collect data, analyze, report results)

75 75 75 Medium

5 Identify disclose deficiencies, non-compliances

80 85 83 Medium

6 Outputs 85 95 90 High

7 Improve work processes 90 75 83 Medium

OVERALL SCORE FOR SELF-ASSESSMENT

81 Medium

C. ISSUES MANAGEMENT

15 1 Inputs (incidents, occurrences, assessments, trend analysis)

90 60 75 Medium

2 Identify even/issues 90 60 75 Medium

3 Periodic SME group analysis 90 60 75 Medium

SSO Risk Based Oversight Procedure Confidence in CAS Process (3/5)

SNL CAS Assurance Process

Process Weight

(%)

SNL CAS Assurance Process Elements

Implementation (I)

(Table 3)

Transparency (T) (Table 4)

CAS Confidence

(I+T)/2

Adjectival Rating

(Table 6)

Comments

4 Local issue CA 90 60 75 Medium

5 Meet corporate escalation criteria

90 60 75 Medium

6 Assign decision maker 90 60 75 Medium

7 Assign CA owner 90 60 75 Medium

8 Analyze issue, casual analysis, develop CA

90 60 75 Medium

9 Enter into CATS 90 60 75 Medium

10 Track issues 90 60 75 Medium

11 Verify completion 90 60 75 Medium

OVERALL SCORE FOR ISSUES MANAGEMENT

75 Medium

D.

CORRECTIVE ACTION

5 1 Inputs (external, internal, self assessments, issues management)

80 60 70 Low

2 Initiate CA process 90 60 75 Medium

3 Analyze deficiencies, determine cause

100 60 80 Medium

4 Develop CAP 60 60 60 Low

5 Implement CAP 70 60 65 Low

SSO Risk Based Oversight Procedure Confidence in CAS Process (4/5)

SNL CAS

Assurance Process Process Weight

(%)

SNL CAS Assurance Process Elements

Implementation (I)

(Table 3)

Transparency (T) (Table 4)

CAS Confidence

(I+T)/2

Adjectival Rating

(Table 6)

Comments

6 Track status in CATS 80 60 70 Low

7 Verify completion 80 60 70 Medium

8 Complete CA 90 60 75 Medium

9 Outputs (metrics, evidence) 100 60 80 Medium

OVERALL SCORE FOR CORRECTIVE ACTION

72 Medium

E.

QUALITY ASSURANCE

10 1 Program 80 80 80 Medium

2 Personnel training and qualification

80 80 80 Medium

3 Quality improvement 80 80 80 Medium

4 Documents and records 80 80 80 Medium

5 Work processes 80 80 80 Medium

6 Design 80 80 80 Medium

7 Procurement 80 80 80 Medium

8 Inspection and acceptance testing

80 80 80 Medium

9 Management assessment 80 80 80 Medium

10 Independent assessment 80 80 80 Medium

SSO Risk Based Oversight Procedure Confidence in CAS Process (5/5)

SNL CAS

Assurance Process Process Weight

(%)

SNL CAS Assurance Process Elements

Implementation (I)

(Table 3)

Transparency (T) (Table 4)

CAS Confidence

(I+T)/2

Adjectival Rating

(Table 6)

Comments

OVERALL SCORE FOR QUALITY ASSURANCE

80 Medium

F.

WORK PROCESS

15 1 Plan work 80 100 90 High

2 Evaluate risk 80 100 90 High

3 Implement controls 80 100 90 High

4 Perform work 80 100 90 High

5 Improve process 80 100 90 High

OVERALL SCORE FOR WORK PROCESS

90 High

G.

COMPARISON ACTIVITIES

5 1 Corporate/local 80 95 88 Medium

2 Chartering (driver, objective, schedule, deliverables, type, funding

80 95 88 Medium

3 Planning, execution and tracking

80 95 88 Medium

4 Reporting and documentation

80 95 88 Medium

OVERALL SCORE FOR COMPARISON ACTIVITIES

88 Medium

OVERALL CAS SCORE

881 Medium

Expected Results from Model Contract and CAS

Capitalize on Private Sector Expertise

Incr

ease

Contra

ctor A

ccounta

bility

Balance Federal Oversight

Improve Management, Performance and

Effectiveness, Mission and Operational Efficiency and

Redirect Efficiencies to Mission Priorities

Expected Benefits from Model Contract and CAS

• Consistent with NNSA Report to Congress:

1. Streamlined and reduced oversight with minimal

workload impact

2. Clarified and simplified requirements, including both operational and programmatic guidance and reporting requirements

3. Maximum empowerment of expertise in the laboratories

4. Site contractors accountable for performance in compliance with clear expectations

5. Federal oversight based on risk

Implementation of Model Contract Summary

• SSO requests NNSA HQ partner with SSO on path forward to get approval for DOE Order/Directives exemptions from federal requirements.

• SSO advise NNSA HQ on transition from Transaction Level to Systems-Based Oversight for non-nuclear facility operations, programs, projects and business systems.

• SSO requests NNSA HQ process to get DOE Independent Oversight to recognize changes in oversight at SSO and to hold contractor accountable.

•BACK UP

Results to Date

• Facilities

– 81/133 (71%) of federal requirements exempted through tailoring, altering or waiving– 44/113 (39%) of federal requirements utilize systems oversight– 51/113 (45%) of federal requirements utilize transactional oversight– 18/113 (16%) of federal requirements are performance based

• Procurement– 8/19 (42%) of federal requirements exempted through tailoring or altering– 8/19 (42%) of federal requirements utilize systems oversight– 11/19 (58%) of federal requirements utilize transactional oversight

• Property– 10/16 (62%) of federal requirements exempted through tailoring or altering– 14/16 (94%) of federal requirements utilize systems oversight– 2/16 (6%) of federal requirements utilize transactional oversight

• Contractor Human Resources– 43/89 (48%) of federal requirements exempted through tailoring, altering or waiving– 58/132 (44%) of federal requirements utilize systems oversight– 74/132 (56%) of federal requirements utilize transactional oversight