CAPP 163, September 2019 OPR: CAP/SE

CAPP 163 30 September 2019

Safety Assurance & Continuous Improvement

_____________________________________________________________________________________________________________________

Third Pillar of the Safety Management System

NATIONAL HEADQUARTERS CIVIL AIR PATROL Maxwell Air Force Base, Alabama

2

Contents

Preface ………………………………………………………………………………………………. 3 Safety Assurance ………………………………………………………………………………… 3 Continuous Improvement ………………………………………………………………….. 4 Risk Management ………………………………………………………………………………. 5 Plan, Do, Check, Adjust (PDCA) …………………………………………………………… 6 The Corrective Action (CA) Process …………………………………………………….. 9 Root Cause Analysis – The Basics ………………………………………………………. 11 Root Causes …………………………………………………………………………………………... 11 The “Five Whys” …………………………………………………………………………………….. 12 Cause Mapping ……………………………………………………………………………………… 12 The “Five Ms” ………………………………………………………………………………………… 13 Measures of Effectiveness (MOEs) ……………………………………………………. 14 Quantitative MOEs ………………………………………………………………………………… 14 Qualitative MOEs ………………………………………………………………………………….. 14 The Commander’s Role ……………………………………………………………………. 16 The Annual SMS Program Review ……………………………………………………. 17

3

PREFACE

This pamphlet expands on the Safety Assurance concepts and processes outlined in CAPR 160-1, Civil Air Patrol Safety Program, Chapter 4. It includes easy to follow methods to ensure your safety efforts are successful, and to help improve risk controls and processes at every level. These methods are meant to be easy to understand and easy for members to apply. In the spirit of continuous improvement, we need your feedback as part of our own continuous improvement efforts. Let us know what is easy to understand and what isn’t; what is easy to use and what isn’t; and what you would change if you had a chance. safety@capnhq.gov

SAFETY ASSURANCE

Perhaps the most important and least understood pillar of the CAP Safety Management System is Safety Assurance. In Safety Assurance, members take an objective look at the risk controls and safety enhancements they have put in place. This Safety Assurance pamphlet is like a toolbox of processes that can be used to give you the confidence that those risk controls are really working. If they aren’t working as planned, you will find the tools and skills to help you improve your processes and make sure your safety enhancements and risk controls are truly effective.

“Safety” Cannot Be “Solved.” There is no single final solution to any “safety” issue. No matter how thorough you are you will never be able to remove all risk or put controls in place that will cover every eventuality. That is why risk management must be a continuous process to control risk. Likewise, Safety Assurance provides processes and tools we can methodically use to make sure we are continuously improving our risk controls and daily processes.

The final piece of the Safety Assurance pillar is a method for chronicling your improvement efforts. The annual program review allows you report on your safety enhancements for the previous year, outline your methods for assessing their effectiveness, and plot the course for further improvements in areas of concern. A standard approach to this program review will allow for sharing and benchmarking of successful improvement efforts, and ultimately the ability to show that lessons learned have been incorporated into our day-to-day processes.

The key to an effective Safety Management System

is Continuous Improvement.

4

CONTINUOUS IMPROVEMENT

The entire concept of continuous improvement begins with the desire and commitment to improve. Regardless of the mission or activity, we want to do it successfully, efficiently and safely. If there is room for improvement in the way we are doing it, we want to identify those areas and determine what we can do to improve. Once we take action to improve, we want to see if our actions are effective. If they aren’t as effective as we’d like, we determine why and then take actions to further improve. It is a continuous process.

There are lots of acceptable methods for going through the continuous improvement process. An entire civilian industry has grown around the concept of defining, branding, selling and teaching methods for continuous improvement.

There is even an Air Force Policy on Management Improvement (AFPD 38-4). It is implemented through AFI 38-401, Continuous Process Improvement, outlining the required use of various process improvement methodologies. The USAF’s commitment to continuous improvement is stated very clearly in AFI 1-2, Commander’s Responsibilities, where the Air Force “establishes broad responsibilities and expectations” of its commanders. Put succinctly, one of the responsibilities of an Air Force Commander is:

“Improve the Unit. Continuous process improvement is a hallmark of highly successful organizations. Wasteful, ineffective or unsafe ways of doing business cannot be tolerated. Commanders must foster a culture of innovation and challenge inefficiencies. A process for identifying and fixing deficiencies should be established and followed… Commanders should strive to leave their unit better than they found it.” - AFI 1-2, paragraph 3.4

The CAP SMS shares this commitment to continuous improvement. We have attempted to take industry standard processes, gleaned some common methods from each, and melded them into a few easy-to-understand CAP-standard approaches to continuous improvement. This pamphlet offers some simple tools to guide you through the process.

When you set out to “improve” you don’t solve; you evolve.

5

Risk Management The 5-step process of risk management, outlined in CAPR 160-1, Chapter 3, is designed to reduce risk but it can also be a very effective problem-solving and planning tool. Faced with a mission or activity we want to pursue, we ask ourselves how we can effectively accomplish that mission while minimizing the risks.

Once you determine the scope and goals of the event and come up with a basic plan, you can use the five steps of risk management to help determine how to successfully complete the event. To put it in very simple terms, risk management allows you to ask “what can go wrong” with this plan. Then it guides you through the process of determining “what can I do to keep it from going wrong.” Finally, throughout the event and after the event is complete, it guides you through the process of determining how effective you have been in reducing risk and keeping things from “going wrong.” CAPF 160, Deliberate Risk Assessment Worksheet, can help lead you through that process.

The 5 steps of risk management lend themselves to the planning phases of a mission or activity, then provide guidance on how to assess and improve the effectiveness of risk controls. Most of the other problem-solving tools in safety assurance are best suited to existing programs or processes, allowing us to look at why our processes might not be working as well as intended. The goal in all these tools is continuous improvement.

6

Plan, Do, Check, Adjust (PDCA)

CAPR 160-1, Chapter 4, describes the basic steps of PDCA. Here we will go into more detail on how to actually apply those steps in your improvement efforts. Walk through this guide for a good understanding of each step of the PDCA process.

STEP 1 - PLAN. This step is the most important (and most time-consuming) step of the PDCA process. Regardless of the problem you are trying to solve, or the process you are trying to improve, it is imperative that you take the time to fully define the problem at hand before you can determine the improvements that need to be made. Follow these steps to ensure your plan results in the improvements you desire.

- Assemble a Team. The size of the team may vary depending on the size and scope of the problem. Whether you decide on a large team or just one or two dedicated members, make sure you consider the following:

-- Pick people with expertise in the area you are trying to improve

-- Make sure everyone understands the PDCA process before getting started

-- Identify roles, responsibilities and timelines to keep the process moving

- Set Goals. There a couple fundamental questions that need to be answered before you get started.

-- What are we trying to accomplish? What is it you are trying to fix? What is the “end game” for the process you are trying to improve? This might be anything from reducing a certain type of mishap, to expanding the size or length of your activity while controlling risks.

-- What is the scope of effort? As you examine your process or area for improvement, limit your plan to the things you can control. For example, if you are using the PDCA process at the squadron level, you should limit your goals to improving squadron-level processes. However, make note of those higher echelon restrictions or “barriers” that you might want to point out to leadership. All levels of the organization can and should benefit from grass roots efforts to improve. Even when PDCA is used on issues at the national level, the team must work within the constraints of the laws and policies that control their efforts.

- Examine the Current Process. Before you can determine what is “wrong” with a current process, or why things just don’t work as smoothly as you would like in a certain

7

program or activity, you need to fully understand how the current process works (or is supposed to work).

-- What is the current “plan?” Is it written down or is it just something you “do” based on common knowledge and experience? What written guidance is there that might apply to the mission or activity?

-- How do you actually do it? Sometimes the way you perform a mission or complete an activity doesn’t necessarily follow the “plan.” Shortcuts or work-arounds can develop over time. Gather the team and map out how you really get “the job” done.

-- Where does your current process fall short of your goals? That will help you define the problem.

- What’s the Problem? Take the current process you mapped in the previous step. Compare it to the goals you set. You’re likely to see differences between the way you want it to be done, and the way it is currently being done. Figure out what is causing those differences, and you will be able to determine exactly what it is that you need to “fix.” To help in that process, you can use some of the “Root Cause Analysis” techniques outlined later in this pamphlet.

- Design Your Plan of Action. A good team will be able to come up with quite of few options for how to “fix” or improve on those areas you just identified. It is important to select the most promising and most cost-effective solutions for your plan of action. Trying to do too much all at once with limited resources can frustrate your efforts and leave you short of your goal.

-- Develop the Plan. The plan must be specific about the tasks to be performed, responsibilities for those tasks, approval authorities for implementing the plan, the timeline for implementing the plan, and how you are going to measure the success of that plan. This is where you outline a step-by-step plan on what you will change and how you will change it. Don’t forget to plan how you will communicate the plan and educate members on what is changing. Your plan needs to build the team that will get it done.

-- Develop Measures of Effectiveness (MOEs). One of the most overlooked parts of Continuous Improvement is the effort to make sure the improvements you have put in place are actually working. If you have done a good job of defining exactly what you want to accomplish, it should be relatively easy to determine how you will measure your success. Part of that process is to make sure the improvements you put in place are working as designed, and then determine if they are effective. Check out page 14 for tips on developing good MOEs.

IMPORTANT TIP:

Several small easy-to-implement solutions will ensure a successful improvement effort. Success will reinforce the process that brought that success, inspiring wider use of the process.

Success is contagious!

8

STEP 2 – DO. This is the most straightforward step. If you have done a thorough job in the “Plan” step, the “Do” step is as easy as putting the plan in motion and gathering the data you need to measure the effectiveness of the plan. Remember that every new plan will need some oversight to make sure each step of the plan goes according to schedule. However, it is important to remember that part of what you are checking is how well the plan is communicated and how well it is understood. Once it’s in place the plan needs stand on its own. Make observations and gather data so you can use your MOEs to judge the success of your efforts.

STEP 3 – CHECK. Once the plan has been in place for a pre-determined period, it is time to assess its effectiveness. Look at the data you gathered during the “Do” step. Use your MOEs to answer a few important questions.

- Did your changes result in an improvement? If so, how much? - Are the changes sustainable? Were they cost effective? - Did any of the changes have unintended consequences? Did anything get worse? - If changes were effective, did you see ways they could be adjusted … … so they are easier to understand? … so they would be easier to do? … so they would make an even greater improvement?

STEP 4 – ADJUST. This is the step that defines “continuous improvement.” You worked hard to develop a plan to make an improvement, then put it in place and measured its effectiveness. Now, you can improve on your improvement plan based on how well it worked. You may find that the plan itself needs adjustment, or the way the plan is explained needs adjustment, or you need to improve the training that goes along with the plan. Those types of adjustments are normal. This final step in the plan makes it a continuous process. Pick a time down the road to come back and use the same MOEs to assess whether your plan fits the current operations or landscape. Make more adjustments, and make the commitment to continuously reassess, adjust and improve.

9

The “Corrective Action” (CA) Process The International Organization for Standards (ISO) has developed a widely used process for reviewing and solving quality control problems. These Corrective Action (CA) steps are very similar to the PDCA process. You will also note some similarities to CAP mishap reviews where we use some of the same techniques to find the contributing factors and recommend corrective actions. Some members may find this process easier to use, as it breaks out the numerous steps of the “Plan” part of PDCA and provides an easy 7-step problem solving process.

STEP 1 - Define the Problem. The steps for defining the problem are similar to the steps laid out in PDCA. It is still important to go through the process of determining what you want your outcome to be (your goal) and comparing that with what might be wrong with the current process … what is it you want to improve to better achieve your goal? A good way to word your problem statement, to make sure you have adequately defined it, is to say, “This is how the process CURRENTLY works and this is how the process SHOULD work.”

STEP 2 – Define the Scope. As described in a similar PDCA step, it is important to limit your efforts to those things which you can control. It is easy to let the “scope” get out of hand as members of the team add on other related issues that need to be addressed. Make sure you keep your efforts focused on a problem you have the time and resources to solve. It is better to focus on one issue and improve it rather than let the scope grow too large, resulting in results that fall short of your goals.

STEP 3 – Containment Actions. As you begin the process of defining the problem, you may find that you need to make an immediate correction to stop the problem from occurring NOW. If, for example, you have an issue with unexplained malfunctions in a particular type of aircraft, your best course of action may be to ground those aircraft until you can fully define the problem. This is like to the “Knock it Off” described in CAP risk management training. You can also think of this as the “put out the fire” step … get the situation under control immediately so you will have time to take a thorough look. There will be plenty of time to analyze the situation and properly define the problem once the immediate situation is under control and everyone is safe.

IMPORTANT NOTE:

If you find a problem with aircraft, vehicles, equipment, training, or processes that may affect members in other wings and regions, consider immediately informing your chain of command and

NHQ so others may also take the proper containment actions!

CRITICAL FIRST STEPS?

1) Knock it Off!!

2) Put out the “fire”

3) Go back to Step 1

10

STEP 4 – Find the Root Cause. “Root cause” is a term that is widely used, but not always understood. The fallacy in the idea of finding THE root cause is that there is rarely (if ever) a single root cause for any mishap or event. In short, root cause analysis helps you go from knowing what happened, to knowing how it happened, then continuing to investigate until you find out why it happened. Then, and only then, will you be able to take corrective action that will prevent this type of event from occurring again. The web is full of companies selling week-long courses and elaborate matrixes to teach a person all about root cause analysis. For our problem-solving purposes, we can keep it relatively simple. Check out the next page of this pamphlet for some guidelines on finding the root cause.

STEP 5 – Plan the Corrective Action. This is very similar to the “design your plan of action” step in the PDCA process. If you methodically worked to find the root cause(s), you probably identified numerous areas where improvements could be made and some specific recommendations to address the root causes you found. Trying to fix everything at once will probably lead to frustration. It is important to prioritize your recommendations, focusing on the CAs that will provide the best results with the greatest impact based on cost, effort and time. Remember, you will still need to come up with a plan for measuring the effectiveness of the corrective action. Those measures should include how well the CA is understood by your members, how you can ensure it is being used correctly, and whether it was successful in correcting or improving the situation you identified as a root cause. See page 14 for help developing your Measures of Effectiveness.

STEP 6 – Implement the Corrective Action. This is the same as the “Do” step in PDCA. It is as simple as following through to make sure the corrective action is implemented as planned and monitored using the MOEs you developed.

STEP 7 – Follow Up! This is the key to continuous improvement. This is where your Measures of Effectiveness are so important. Did your plan work the way you expected? If it didn’t work according to plan, then “Adjust” it. If it did work, then benchmark it, share it, and see if it will help others in similar situations.

11

Root Cause Analysis – The Basics

The most important step in starting any improvement effort is defining the problem. What is the issue that you are trying to “solve” with your improvement effort? Too often, people set about trying to fix a “symptom” without taking the time to determining the cause of that symptom. That’s where root cause analysis fits in.

If a process doesn’t work well, we must ask “why” is doesn’t work well. If we are trying to reduce the frequency of a certain type of mishap, we must know “why” those mishaps are happening. Put simply, “Root Cause Analysis” (RCA) leads you to ask “why” until you can determine what it is you need to “fix.” Like most of the other continuous improvement processes, there are a lot of resources online (most of them will cost you money). Here’s a simple approach that everyone can use.

Before we go into how we “dig” to find the root cause, let’s take a quick look at what a root cause looks like. After all, you won’t know you’ve found it, unless you know what you’re looking for. For our purposes we can look at three primary criteria for defining “root cause.”

Root causes …

… are the underlying causes of a problem. In mishap reviews, we use the term “contributing factors” when we refer to the things that led up to the event, occurrence, or situation, and the same idea applies here. The more specific we can be, the easier it is to make a recommendation to fix that underlying cause or contributing factor.

… are things that management and members can control. Keep asking “why” until you get to something that you can “fix.” Find something that you can change or improve. For example, “severe weather” would not be a root cause of a mishap. However, the guidance, checklists, warning systems, training, and other factors that ensure we minimize the risks associated with severe weather are things we can control, so we look to see if those were factors.

… can be addressed with specific effective recommendations. For example, let’s say you determine that a certain mishap occurred in part because a regulation was ignored. A recommendation to “follow regulations” would be too vague; that probably means you haven’t done enough digging. Ask “why” that regulation wasn’t followed, and you may find that training needs to be improved AND the requirement needs to be more clearly written. NOW, you have enough detail to make recommendations that address the need for enhanced training AND a clarification of the regulation.

12

We talked about asking “why” until we get to the root cause. Let’s take a look at how that works…

The “Five Whys”

The “Five Whys” strategy is generally traced back to Sakichi Toyoda, founder of Toyota Industries. It was developed to find out “why” something happened, or “why” a process wasn’t working, so the process could be improved. This strategy leads you to ask a series of questions about “why” something happened and can help steer you to your root cause(s). The number “five” is just a rule of thumb; sometimes it will take more and sometimes you’ll find that root cause in fewer steps.

Put very simply, if a mishap occurs, or a process isn’t working, your team should ask, “why?” That question might give you the immediate cause, or what happened right before the mishap, but you still don’t know the root cause. Ask “why” again. “It happened because of this.” Then ask why this happened. Continue that process until you come up with a “cause” that meets the three criteria for defining a root cause.

Cause Mapping

In a technique known as “cause mapping,” you can actually draw out your series of “why” questions to show how you arrived at the final root causes that you will address with your improvement efforts. The diagram below shows what the “cause mapping” may look like and provides a reminder that sometimes the “why” question will have a couple answers and you’ll find you have a couple different contributing factors that worked together to “cause” the problem.

As an example, let’s say you are charged with finding out how to prevent a certain type of mishap that has become a trend in your unit. You may find that one of your “why” questions comes up with an answer that the members don’t know how to do a certain task. Ask “why” again and you’ll discover they weren’t trained the right way to do it. Ask “why” again and you may find two answers … the only available training is outdated and doesn’t really address what went wrong, AND the regulation doesn’t require that training. Let’s look back at the criteria for root causes and see if the two issues you found (inadequate training and no requirement for

13

training) meet the criteria… they are underlying causes of the problem you’ve defined, they are something management and members can control, and they can be improved with specific recommendations! Good job!

The “Five M’s”

If you are taking the time to go through a root cause analysis process to define your problem, you want to be confident you’ve done a thorough job. How will you know? It helps to use the “Five M’s” to make sure you’ve looked at all the things that can affect a process or activity or event.

We use the “Five M’s” when performing risk assessments, and reviewing mishaps, and they can be a good tool here. By examining how each of these areas may affect the problem you are addressing, you can be sure you identify all the things …

… that are underlying causes of the problem, … that management and members can control, and … that can be addressed with specific recommendations.

Member: Take a look at the people involved. A few examples: - Were the members trained for the mission or activity? - Were they physically able to perform the task?

Medium: This refers to the environment and what effect it might have. - Did the weather affect the plan? Did the plan account for changing weather? - How about the terrain? Was the “playing surface” appropriate for the activity? - What was the lighting? Was it noisy? Distractions?

Machine: This looks at the airplanes, the vehicles, and ALL the equipment that is used - Is the equipment well-suited to the task or mission? - Is the equipment well-maintained? Well designed? Did something break? Why?

Mission: This refers to how the actual mission or activity was planned and executed. - Was the activity well-planned? Did everyone know the plan? - Was a Risk Assessment accomplished before the activity and were risk controls actively used? - Was the mission or activity too complex for the members involved? - What was the chain of events that led to the problem; what went wrong? Why?

Management: This refers to the organizational factors that influence our activities and missions. - What do the regulations and written guidance say about the activity? -- Are they clear? Easy to understand? Were they followed? - Is there any other informal guidance or local standards or “the way we’ve always done it?” - Who was in charge? Was there adequate supervision? - Did the plan define everyone’s role, and did everyone perform that role?

14

Measures of Effectiveness (MOEs)

A key element of all the Continuous Improvement processes we’ve discussed is the final step where you assess the effectiveness of the initiatives or “fixes” you put in place. If those improvements aren’t as good as you thought they’d be, you can improve them. Well-constructed MOEs will help you determine if they are working, how well they are working, and where they could use some adjustments.

If you did a good job of coming up with specific changes, or fixes, or safety initiatives to address your problem, it should be relatively easy to measure how well they are working. When you’re developing the MOEs, remember that your measurements should be both quantitative as well as qualitative. Let’s talk about each…

Quantitative MOEs

As the name implies, quantitative MOEs involve counting something. If your objective is to “reduce” the number of a certain type of mishap, or “increase” the completion rate of a certain process, you need to “count” how many, so you can compare the results of your efforts with the way things used to be. There are some traps associated with “numbers” and “rates” in safety. Here are some tips:

- Keep it specific. Make sure you are only measuring the results of your specific initiative. If you determined that low-tire pressure was the root cause of some of your vehicle mishaps, and you strengthened your risk controls to make sure vehicle tires were always properly inflated, then you want to see if that specific effort was effective. Compare how many low-tire-pressure mishaps you had before your efforts, with how many low-tire-pressure mishaps you had after your efforts. THAT will tell you how successful your corrective action was.

- Measure on a scale – it’s not “all or nothing.” If your goal is to “reduce” mishaps, or “increase” efficiency, don’t fall into the trap of saying that “zero mishaps” or “100% participation” is your MOE. What is the level of success that will satisfy you based on the effort and resources you’ve expended? Remember, you acted based on what you could control with your resources. Measure your improvement, then keep the improvement process working.

Qualitative MOEs

Qualitative MOEs don’t look at the numbers as much as they measure whether or not your initiatives worked the way they were intended to work. Here are a few things to consider when you are building your MOEs:

15

- Were the changes delivered? All your hard work is for naught if members aren’t aware of the changes you’ve made. Are members accessing the new training? Are the new checklists being used? You need to have some way of assessing whether the improvements you’ve made are reaching their target audience.

- Were the changes understood? Once you’ve determined that the members received the new guidance, how can you be sure they understand it? Is it confusing? Delivery and understanding can be assessed by interviewing or surveying or observing the target audience.

- Do the improvements work as intended? If everyone is aware of the changes, and they understand the changes, are they working the way you intended? An example can be seen in new or improved training. After members take the new training, are they exhibiting the skills the training was designed to impart? This can be assessed through post-training interviews or surveys, or through observation of the members after training.

- Are the improvements sustainable? Even if the changes bring positive results, you need to be able to measure whether you’ve made a long-term change. Is the change affordable for the long term? Can members commit to the change or are you seeing a whole lot of effort for just a little bit of improvement? Remember that improvements must be cost-effective in both money and effort if they are going to last

16

The Commander’s Role

Lead. Like so many other programs and processes within CAP, commanders don’t need to be the process experts, and they don’t need to lead each of these improvement efforts. However, members need to see their commanders’ commitment to improvement and their support of the processes that bring that improvement.

Steer. Commanders can and should steer their members toward the continuous improvement processes in this pamphlet. If a member approaches and says they “have an idea” on how to improve something, the commander could ask, “What’s the problem?” By asking what problem that “idea” is meant to solve, they can steer the members toward the PDCA or Corrective Action steps of defining the problem then developing and implementing solutions. Remember that their “idea” might help point out a problem, but you may find their “idea” isn’t the best solution.

Challenge. Our members love a challenge. They work as a team. They succeed. Challenge them to improve something that needs improving. Form a team of those affected by the issue you’ve chosen. Let them keep you updated on their progress and applaud their successes no matter how small. In this case the goal is as simple as starting the process, so it can become “contagious” and “continuous.”

Share. The goal of the Safety Assurance pillar of the SMS, and this pamphlet, is to provide a CAP-wide institutionalized approach to risk management and continuous improvement, that allows our member experts to work together to find lasting solutions through common processes. A huge step in the institutionalization of that process is the Annual SMS Program Review … commanders sharing their improvement efforts with a summary of benchmarks, proven best practices, and the challenges their leadership and their teams have met head on.

“Commanders should strive to leave their unit

better than they found it.” AFI 1-2

17

The Annual SMS Program Review As described in CAPR 160-1, the Annual SMS Program Review provides wing commanders with a single standardized way to assess the effectiveness of their safety programs. With minimum effort the annual review compiles information from already-existing safety assurance tools. The process is meant to highlight their successes, identify areas for further improvement, and allow for the sharing of this information.

Objectives The Annual SMS Program Review has several objectives, all geared towards process improvements leading to reduced mishaps.

- It is intended to be a tool to help commanders focus their safety program attention on long-term solutions rather than reactionary one-time fixes to problems. - It provides a single balanced review of all the safety assurance elements of the SMS. - It provides wing and region commanders and the National Command Team a standardized look at their safety programs. - It provides a standardized process for determining safety best practices and risk management benchmarks that can be shared and institutionalized throughout CAP.

Commander Support This pamphlet, and CAPR 160-1, will be in place for over a year before wing commanders are expected to turn in their first Annual Program Reviews. During that time, as resources improve, and processes mature, additional guidance and examples will be offered. Commanders are encouraged to embrace the processes described in this pamphlet and visibly support the continuous improvement philosophy. While the wing and region directors of safety will be expected to play a large role in assembling these reviews, the best efforts will reflect strong involvement from all mission and support areas of the commanders’ staff. A concerted effort by commanders and their staffs to support this annual review process will speed the maturation of the system through the sharing of suggestions and best-practices as part of their annual reviews. Elements of the Annual SMS Program Review There is some flexibility as to what is included in the Annual SMS Program Review. Requirements will mature and may change from year to year based on available data and

18

Command direction or areas of emphasis. As a minimum, all commanders should consider including the following elements in their annual reviews. Summary of Reportable Events. Provide a summary of mishaps or reportable events in each category (i.e., aircraft, vehicle, bodily injury, etc.). Special attention should be given to the following:

- Trends. Note any trends or multiple cases of any specific type of mishap, or any common causes or contributing factors. In addition to any physical hazards identified, note any observed weaknesses in guidance, training, risk management practices, or the adequacy and clarity of prescribed processes and plans. - Controls. Note what corrective actions or process improvements were made based on the trends you observed, and how you will assess the effectiveness of those changes.

Safety Surveys. Summarize any safety surveys that were used during the previous year. Include the following:

- The reason for the survey and a general description of the type of questions asked. - What method/website/software was used for the survey? - Describe the target audience, and the actual rate of participation. - What was learned from the survey and how will the information be used?

Safety Program Assistance to Subordinate Units. Summarize the types of visits or communication efforts accomplished for the previous year. The summary should include, but is not limited to, the following.

- Note how many of each type of assistance (visits, phone calls, on-line meetings, etc.) occurred during the previous year. Note whether actual efforts exceeded, met, or fell short of planned goals. - Note any significant lessons learned, improvement opportunities, or benchmarks discovered through the subordinate unit assistance efforts. Attach examples of especially noteworthy plans, schedules, or programs. - Note any adjustments made to your upcoming assistance plan based on lessons learned from the previous year.

19

Previous Command Emphasis Items. Following the PDCA (or equivalent) format, list the Command Emphasis Items from the previous year. For each emphasis item, include the following:

- Plan. Briefly outline the issue or problem that was addressed, along with the process used for outlining objectives and controlling risks. Then discuss the planned initiative and what MOEs were used to gauge success.

- Do. Briefly provide an update on the implementation of the plan and the data gathering.

- Check. Briefly explain the results of the initiative, to include how the MOEs reflected the successes along with areas where expectations were not met.

- Adjust. Explain what changes have been made to the original plan, and how the improvements will be sustained in the future.

New Command Emphasis Items. Commanders should develop new emphasis items each year, based on already existing safety assurance measures, or the presence of newly identified risks or missions. The following elements of the “Plan” step of the PDCA process (or Corrective Action process if preferred) should be included.

- What is the actual emphasis item, issue, or problem being addressed?

- Briefly outline the plan for dealing with this initiative in the coming year.

- Explain how you will measure the effectiveness of the plan (MOEs). Miscellaneous. The annual review process provides a means for commanders to provide safety program feedback up through the chain of command. To that end, wings are encouraged to add additional items they would like to share with leadership. Suggested items include, but are not limited to:

- Benchmark processes the commander feels have contributed to the effectiveness of their wing safety program.

- Innovative ways of delivering safety and risk management training to members. - Any areas where the wing feels it needs increased support, funding, training, or other assistance to effectively integrate risk management into all their missions and activities. - Highlight the wing’s safety recognition programs and the awards given/received under the auspices of CAPR 160-1, paragraph 5.3. - Highlight any new safety education or training initiatives, along with their goals and a summary of their effectiveness.

- Highlight any newsletters or other innovative methods of communicating safety program updates and safety reminders to members.