Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main...
Transcript of Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main...
![Page 1: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/1.jpg)
Performance Testing Nuclear Security
Presented to the International Conference on Physical Protection of Nuclear Materials and
Nuclear Facilities
Richard P. Rosano, Senior Vice President
EXCEL Services Corporation
November 20171
![Page 2: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/2.jpg)
Overview• General guidelines for protection of nuclear facilities and materials
are established in State and international documents
• Requirements are set forth in State regulatory documents
• Compliance inspections are baseline reviews
• Only performance tests can measure the readiness to repel a terrorist attack on a facility
2
• Combination of compliance inspections and performance tests required
• Various methods for conducting performance tests
![Page 3: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/3.jpg)
Outline
• Compliance v. Performance
• Objectives of Performance Testing
• Testing Methodologies
• Paper Review
• Tabletop Drills
• Computer Simulations
• Limited Scope Performance Testing
• Force-on-Force Exercises
• Assessment of Findings
3
![Page 4: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/4.jpg)
Compliance v. Performance
Compliance = planning
Performance = battle
4
![Page 5: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/5.jpg)
Compliance v. Performance
“No battle plan survives contact with the enemy.”
Helmuth von Moltke, German military strategist
5
![Page 6: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/6.jpg)
Compliance v. Performance
“No battle plan survives contact with the enemy.”
Helmuth von Moltke, German military strategist
“In preparing for battle I have always found that plans are useless, but planning is indispensable.”
Dwight D. Eisenhower, American general and president
6
![Page 7: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/7.jpg)
Compliance v. Performance
• Compliance
• Design Basis Threat
• State regulatory requirements
• “Baseline” inspections – appropriate
systems, appropriately implemented
7
![Page 8: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/8.jpg)
Compliance v. Performance
• Compliance
• Design Basis Threat
• State regulatory requirements
• “Baseline” inspections – appropriate
systems, appropriately implemented
• Performance
• Activation of all systems
• Mobilization of forces
• Engagement and results
8
![Page 9: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/9.jpg)
Objectives of Performance Testing
• Final, true test of the protective strategy
• Enhances training techniques
• Provides evidence to regulators
• Validates the planning
• Confirms whether the security force can:
• perform the right tasks
• at the right time
• with sufficient force to counter the adversary attack
9
![Page 10: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/10.jpg)
Testing Methodologies
• Paper Review – desktop review of commitments and past actions
• Tabletop Drills – uses plant drawings or 3-dimensional mockups of facility to conduct “tactical chess” game for opposing forces
• Computer simulations – allow multiple consecutive tests in short period of time
• Limited Scope Performance Testing – isolated skills tests based on specific posts, timelines, and portions of strategy
• Force-on-Force Exercises – full-field deployment of “shadow” force to repel an adversary attack
10
![Page 11: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/11.jpg)
Paper Review
• Review of documents
• design basis threat statement
• current physical security plan
• past results from tabletop drills and force-on-force exercises
• Conceptual testing
• Consideration of protective strategy modifications
• Written evaluation of results
11
![Page 12: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/12.jpg)
Tabletop Drills
• Plant drawings or 3-dimensional mockup of plant
• Design basis threat
• Players to represent “shadow” force and mock adversary
• List of adversary “tool kit” – weapons, tools, tactics
• Clock management
• Written evaluation of results
• Computer simulations
12
![Page 13: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/13.jpg)
Computer Simulations
• Advantages
• multiple computer tests of same strategy
• more accurate documentation of results
• allows rapid modifications of assets and re-test
• avoids human error in observations
• Disadvantages
• requires modeling of plant and security assets
• requires onsite knowledge of program and process
13
![Page 14: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/14.jpg)
Limited Scope Performance Testing
• Plant drawings or 3-dimensional mockup of plant
• at least the portion being tested
• Design basis threat
• as represented by adversary force at point of engagement
• Players to represent “shadow” force and mock adversary
• for the portion that will be tested
• List of adversary “tool kit” – weapons, tools, and tactics
• Clock management
• Written evaluation of results14
![Page 15: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/15.jpg)
Force-on-Force Exercises
• Identification of Teams
• Collection of Information
• Identification / Elimination of Artificialities
• Preparation for the Exercise
• Conduct of the FOF Exercise
• Time Management
• Documenting Observations
15
![Page 16: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/16.jpg)
FOF – Identification of Teams
• Mock adversary team
• “Shadow” security force
• Controllers and event judges
• Record-keepers and exercise managers
16
![Page 17: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/17.jpg)
FOF – Collection of Information
• All participants cleared for sensitive information
• Information includes:
• physical security plan, procedures, and post orders
• contingency plan(s)
• past results of FOF tests
• Exercise event sheets and records
• Comments/observations from participants
• Time records and neutralization patterns
17
![Page 18: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/18.jpg)
FOF – Identification of Artificialities
• Stopwatch
• Use of smoke or small explosives
• Climbing
• Engagement systems
• Explosive breaching of physical barriers
• Radio frequency jamming equipment
18
![Page 19: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/19.jpg)
FOF – Preparation for the Exercise
• Training – controllers, judges, timekeepers, participants
• Proper forms and paperwork to record events
• Placement of assets in best positions
• Safety training
• Steps to avoid confusion between real force and shadow force
• Communication equipment and protocol
• Time management
19
![Page 20: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/20.jpg)
FOF – Conduct of the Exercise
• Proper placement of participants
• Pre-exercise warning – “This is a drill” – repeated as necessary
• Recording of actions and engagements – with time stamps
• Flagging neutralized participants
• Leave equipment at spot of neutralization
• Clock stoppages clearly announced
20
![Page 21: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/21.jpg)
FOF – Time Management
• Clock stoppages clearly announced
• Eliminate actions during clock stoppages
• Record time “in” and “out”
• Note all actions according to time stamp
21
![Page 22: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/22.jpg)
FOF – Documenting Observations
• Collection of documents
• Time sheets
• Controller forms
• Notes and comments from participants
• Organization of notes
• Post-exercise out-briefings with all participants
22
![Page 23: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/23.jpg)
FOF - Assessment of Findings
• Brief intermission to allow exercise managers to collect and organize time sheets, controller forms, etc.
• Prompt post-brief to allow specific memory to contribute to findings
• Discussion can focus on:
• results of exercise
• appropriateness of exercise game plan
• effectiveness of the protective strategy
23
![Page 24: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/24.jpg)
Additional Information from NUSAM
NUSAM – Nuclear Security Assessment Methodologies
• Main objective of testing program is:
• risk-informed, performance-based methodological framework
• systematic, structured, comprehensive, and transparent
• Secondary objective of testing program is:
• sharing knowledge and experience
• providing guidance
• illustrating best practices
24
![Page 25: Performance Testing Nuclear Security · NUSAM –Nuclear Security Assessment Methodologies • Main objective of testing program is: • risk-informed, performance-based methodological](https://reader035.fdocuments.net/reader035/viewer/2022071019/5fd2aacd6487754be3108932/html5/thumbnails/25.jpg)
Contact Information
Richard P. Rosano
Senior Vice President, Nuclear Security
EXCEL Services Corporation
11921 Rockville Pike, Suite 100
Rockville, Maryland 20854 USA
office: +1 301-984-4400
cell: +1 240-888-8877
25