Thar’s Gold in Them Thar AAR/ IPs
description
Transcript of Thar’s Gold in Them Thar AAR/ IPs
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
Thar’s Gold in Them Thar AAR/IPs
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
WELCOME AND AGENDA• Purpose and Value of After Action Reports• How to develop an Improvement Plan
Esther Corwin, MEPExercise Training Officer, Texas Division of Emergency [email protected] Info is on handout
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
• A consolidation of information gathered during the testing and evaluation of a community’s emergency operations plan through an exercise
• Provides feedback to executive leadership, participating entities and governing agencies in the achievement of the exercise objectives and overall capabilities of the community
• Information gleaned from this process, identifies and guides future improvement actions
WHAT IS AN AAR
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
WHY WRITE AN AAR• Creates a multi-agency process to review an
exercise or incident• General exercise identification information,
but also includes evaluation information• Facilitates honest and critical appraisal of
actions based on plans and policies
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
WHERE DO WE LOOK• Exercises to simulate a stressful realistic scene• Exercises to evaluate a new or seldom used
equipment or skill• Incidents presenting unexpected challenges• Incidents having “less than expected” results
OR• Incidents employing innovative actions• Incidents with unexpected positive results
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
EXERCISE EVALUATION IS BASED ON• Community policies, Emergency Operations
Plan, agency SOPs and SOGs, MOUs, etc.• Exercise Evaluation Methodology
– S.MA.R.T. objectives– Exercise Evaluation Guides
• Observational Data• Hotwash• Participant Feedback
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
INCIDENT EVALUATION IS BASED ON• Community policies, Emergency Operations
Plan, agency SOPs and SOGs, MOUs, etc.• Incident Action Plans• Briefings, De-Briefings• Observational Data• Participant Feedback
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
TYPES OF OBSERVATIONAL DATA• Descriptive reporting
– Typically yields reliable data.• Inferential reporting
– Includes terms like “adequate” or “timely”– Yields inconsistent data
• Evaluative reporting– Includes terms like “efficient” or “successful”– Difficult to reliably to collect
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
THREE LEVELS OF ANALYSIS
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
THREE LEVELS OF ANALYSIS
Critical Resource Logistics & Distribution
Onsite Incident Command
Capability Being Evaluated
Capability-Level Analysis
Activity-Level Analysis
Task-Level Analysis
Resource Management – Identify, dispatch, mobilize,
track, demobilize and pay for resources
Direct/Control incident management activities.
Identify the resources based on incident need
Develop all necessary components of an IAP and
obtain approvals
Inventory resources by type/ category that are available to
support this incident
Establish incident objectives and priorities and operational
periods
Leve
l of E
valu
atio
n
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
EXERCISE EVALUATION GUIDES• Identify the activities, tasks, and performance
measures to be observed• Developed and customized before the exercise • Evaluators complete the EEG by:
– Logging times and actions accurately– Decision making processes and participants– Player roles and responsibilities– Coordination and cooperation– How actions were performed, resources involved– Documenting issues and recommendations
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
DATA ANALYSIS
The goal is to evaluate the ability of involved functions to perform
target capabilities
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
PRELIMINARY ANALYSIS• Evaluators organize observations into key
issues and a chronological narrative• At a minimum, should include:
– Description of the assigned function or operation, analyzed by capability, activity, and task
– Documented record of significant observed actions
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
• Review exercise objectives– What was the intent of the objective?– What would demonstrate the successful
performance of the objective?– If the objective was not met, what factors
contributed to this result?
• Review incident operational objectives
• Review exercise objectives– What was the intent of the objective?– What would demonstrate the successful
performance of the objective?– If the objective was not met, what factors
contributed to this result?
• Review incident operational objectives
REVIEW OBJECTIVES
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
4 STEPS OF DATA ANALYSIS
1. Identify issues– Compare performance data to standards– Identify differences– Determine consequences
2. Determine root cause– Discuss conditions leading to observed behavior– The “Why” staircase
Fool’s Gold
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
WHY STAIRCASE
Each step should explain the step above …
… down to the underlying root cause.
Why did it happen?
Why was that?
And why was that?
ROOT CAUSE
And why was that?
Why did that happen?
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
4 STEPS OF DATA ANALYSIS3. Develop recommendations
– Sustain and improve– Short and long term– What needs to be changed and how– Give references
4. Capture lessons learned– Innovative practice or a piece of knowledge gained from
experience– Provides guidance for approaching a similar problem in
the future– Save time, conserve money, and accelerate preparedness
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
DRAFT AAR• Provides feedback to participating entities and
governing agencies regarding the achievement of objectives and overall capabilities
• Records recommendations for improvement• Establish consensus and buy-in on next steps
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
THE AAR• Suggested AAR format includes:
– Executive Summary– Overview– Exercise Goals and Objectives– Analysis of Capabilities Demonstrated– Conclusion
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
• Observations• References• Analysis
– For improvements, list consequences of action or inaction
• Recommendations
• Observations• References• Analysis
– For improvements, list consequences of action or inaction
• Recommendations
• Observations• References• Analysis
– For improvements, list consequences of action or inaction
• Recommendations
• Observations• References• Analysis
– For improvements, list consequences of action or inaction
• Recommendations
OBSERVATIONS AND RECOMMENDATIONS
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
AFTER ACTION CONFERENCE• Planning Team, evaluators, and stakeholders
meet to review and refine draft AAR• Evaluation leads to a disciplined process for
implementing improvement actions and continually strengthening preparedness
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
AFTER ACTION CONFERENCE• Address identified issues• Develop specific improvement actions• Prioritizes action items• Assigns responsibility to track
implementation
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
These questions aid development:• What changes need to be made to plans and
procedures to improve performance?• What changes need to be made to
organizational structures?• What changes need to be made to leadership
and management processes?
DEVELOPING IMPROVEMENT ACTIONS
?
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
More questions to aid development:• What training is needed?• What changes to, or additional
equipment are needed?• What lessons can be learned that will
direct how to approach a similar problem in the future?
DEVELOPING IMPROVEMENT ACTIONS
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
IMPROVEMENT BENCHMARKS• Must be clearly defined and attainable• Examples include:
– Number of personnel trained in a task– Percentage of equipment that is up-to-date– Finalization of an interagency agreement within a
given amount of time• Include concrete deadlines to track progress
toward full implementation
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
ACTIVITY – IMPROVEMENT DEVELOPMENT
• Older elementary school in a rural area just before dismissal on a spring day, parents are arriving to take students home
• NWS issues tornado warning estimating less than 10 minutes before impact
• Principal activates “bell” for sheltering, but students/parents confuse it with dismissal
• Some parents enter school to take child and “run” from the storm
As a large group, given a scenario and issue, discuss
possible recommendations and develop a simple improvement
action.
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
Issue:• Student SafetyBasic Recommendation:• Develop something to
assist the principle in communicating emergency information
• Changes to plans and procedures
• Changes organizational structures
• Changes to leadership/ management processes
• What training is needed• Changes or additional
equipment• Lessons learned for the
future
ACTIVITY – DEVELOP IMPROVEMENT
Jarrell Tornado May 27, 1997
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
SUMMARY• Objectives and Evaluation Methodology are the
foundation of Improvement Planning• Root Cause analysis helps find the key policy,
procedure, training or equipment to which an improvement could be made
• Applies to Exercises and actual Incidents
Improvements make things better than before
Texas Department of Public SafetyDIVISION OF EMERGENCY MANAGEMENT
QUESTIONS ???
Any question can lead to a
good idea.
Synergy is the Energy of People.
Esther Corwin, MEPExercise Training Officer, [email protected]
512-424-2198