Rational and Steps for Quality Rational and Steps for Quality ImprovementImprovement

Presented to Presented to Dr. Betty T. Polido Dr. Betty T. Polido

Faculty of the College of Nursing Faculty of the College of Nursing Central Philippine UniversityCentral Philippine University

In Partial FulfillmentIn Partial Fulfillmentfor the Requirement in the coursefor the Requirement in the course

N 414- FN 414- F

By By REGINE MAE SOMONGCADREGINE MAE SOMONGCAD

BRIAN SOMOSIERRABRIAN SOMOSIERRAJuly 10, 2012July 10, 2012

ObjectivesObjectives

11. To be able to understand the steps of . To be able to understand the steps of performance improvement.performance improvement.

2. To be able to learn the steps and its 2. To be able to learn the steps and its rationale.rationale.

3. To be able to identify the barriers to 3. To be able to identify the barriers to successful performance improvement.successful performance improvement.

4. To be able to apply strategies to build a 4. To be able to apply strategies to build a positive work climate.positive work climate.

5. To be able to discuss approaches for 5. To be able to discuss approaches for successful negotiation.successful negotiation.

INTRODUCTIONINTRODUCTION

Performance refers to the way people do their jobs and the results Performance refers to the way people do their jobs and the results of their work.of their work.

Organizations seeking to solve a performance problem Organizations seeking to solve a performance problem frequently implement a specific intervention, such as training, frequently implement a specific intervention, such as training, without fully understanding the nature of the problem or without fully understanding the nature of the problem or determining whether or not the chosen intervention is likely to determining whether or not the chosen intervention is likely to succeed. Just as often, professionals with a high level of succeed. Just as often, professionals with a high level of expertise in a specific intervention area see every problem as expertise in a specific intervention area see every problem as an opportunity to ply their trade. As Abraham Maslow once an opportunity to ply their trade. As Abraham Maslow once said, “To the person who only has a hammer in the toolkit, said, “To the person who only has a hammer in the toolkit, every problem looks like a nail.” In fact, there are a number of every problem looks like a nail.” In fact, there are a number of methods for improving the performance of organizations, methods for improving the performance of organizations, teams and individuals. Organizational development, industrial teams and individuals. Organizational development, industrial engineering, training and development, quality assurance, and engineering, training and development, quality assurance, and human resources development address performance gaps in human resources development address performance gaps in particular ways. Performance Improvement differs from these particular ways. Performance Improvement differs from these approaches by using a systematic methodology to find the root approaches by using a systematic methodology to find the root causes of a performance problem and then implement an causes of a performance problem and then implement an intervention (or “fix”) that applies to that specific performance intervention (or “fix”) that applies to that specific performance deficit.deficit.

What is Performance What is Performance Improvement?Improvement?

Performance Improvement (PI) is a method Performance Improvement (PI) is a method for analyzing performance problems and for analyzing performance problems and setting up systems to ensure good setting up systems to ensure good performance. PI is applied most effectively performance. PI is applied most effectively to groups of workers within the same to groups of workers within the same organization or performing similar jobs. organization or performing similar jobs. While PI principles are relevant to workers While PI principles are relevant to workers in any field, this publication focuses on in any field, this publication focuses on primary providers of family planning and primary providers of family planning and reproductive health (FP/RH) care services.reproductive health (FP/RH) care services.

Factors That Affect PerformanceFactors That Affect Performance

Certain factors need to be in place for Certain factors need to be in place for workers to be able to perform well on their workers to be able to perform well on their jobs:jobs:

Clear job expectationsClear job expectations Clear and immediate performance feedbackClear and immediate performance feedback Adequate physical environment, including Adequate physical environment, including

proper tools, supplies and workspaceproper tools, supplies and workspace Motivation and incentives to perform as Motivation and incentives to perform as

expectedexpected Skills and knowledge required for the job.Skills and knowledge required for the job.

Rational and Steps for Rational and Steps for Performance ImprovementPerformance Improvement

Stage 1: Stage 1: Consider Institutional ContextConsider Institutional ContextThe goal of this stage is to have the PI The goal of this stage is to have the PI

facilitator and team members understand the facilitator and team members understand the institutional and cultural context of the institutional and cultural context of the organization you are working with. In considering organization you are working with. In considering the institutional context it is necessary, from the the institutional context it is necessary, from the beginning, to pay attention to the culture within beginning, to pay attention to the culture within which the work will take place as well as the goals which the work will take place as well as the goals and mission of the institution with which you are and mission of the institution with which you are working. If at all possible, it is also important to working. If at all possible, it is also important to incorporate the perspective of the clients and incorporate the perspective of the clients and communities served by the organization, communities served by the organization, particularly during the implementation stage of PIparticularly during the implementation stage of PI..

StepsSteps There are no specific steps for this stage. Rather, use the There are no specific steps for this stage. Rather, use the

above list as a guide for your information gathering. If your above list as a guide for your information gathering. If your project team is large, you may want to summarize your project team is large, you may want to summarize your findings in a meeting and/or written communication.findings in a meeting and/or written communication.

Stage 2: Obtain and Maintain Stage 2: Obtain and Maintain Stakeholder AgreementStakeholder Agreement

STEP 1:STEP 1: Recognize the opportunity to apply PIRecognize the opportunity to apply PI

Awareness of an opportunity to apply Awareness of an opportunity to apply the PI methodology can come through a the PI methodology can come through a request for assistance from someone request for assistance from someone within your organization or from another within your organization or from another organization.organization.

STEP 2: Gather preliminary project STEP 2: Gather preliminary project informationinformation

Meet with the key decision-makers to Meet with the key decision-makers to gather the preliminary information you gather the preliminary information you need to get started. You may want to need to get started. You may want to approach this meeting with the stated goal approach this meeting with the stated goal of “making sure we have all the elements of “making sure we have all the elements we need to ensure that training is we need to ensure that training is successful.” Find out about sources of successful.” Find out about sources of valuable information, such as existing valuable information, such as existing research, trip reports or documentation of research, trip reports or documentation of previous attempts to improve performance, previous attempts to improve performance, and be sure to review those materials and be sure to review those materials before conducting interviews.before conducting interviews.

STEP 3: Conduct interviews with STEP 3: Conduct interviews with representative stakeholdersrepresentative stakeholders

Your goal is to identify the major Your goal is to identify the major performance issues facing the organization performance issues facing the organization and gather opinions about what might be and gather opinions about what might be causing the problems. If a key decision-maker causing the problems. If a key decision-maker has all the answers, this step may take only has all the answers, this step may take only one meeting. However, it is best to talk to a one meeting. However, it is best to talk to a representative sample of stakeholders who representative sample of stakeholders who are familiar with the organization. Ask the are familiar with the organization. Ask the key decision-makers to identify other key decision-makers to identify other stakeholders you should consult with. You stakeholders you should consult with. You can talk to them one-on-one, in focus groups, can talk to them one-on-one, in focus groups, meetings or whatever forum suits your style meetings or whatever forum suits your style and that of the organization.and that of the organization.

STEP 4:Review your findings with the key STEP 4:Review your findings with the key decision-makers; prepare for Project decision-makers; prepare for Project Agreement MeetingAgreement Meeting

Review your findings with the key decision-Review your findings with the key decision-makers. Sometimes the interviewees you talked makers. Sometimes the interviewees you talked with in Step 3 will disagree on some points. with in Step 3 will disagree on some points. Discuss these disagreements with the key Discuss these disagreements with the key decision-makers and try to reach a consensus decision-makers and try to reach a consensus before proceeding to the Project Agreement before proceeding to the Project Agreement Meeting. Summarize your findings in a handout Meeting. Summarize your findings in a handout for use at the meeting. The handout should for use at the meeting. The handout should contain the same headings as the Project contain the same headings as the Project Agreement Letter described in Step 6. At the Agreement Letter described in Step 6. At the Project Agreement Meeting, when all the Project Agreement Meeting, when all the stakeholders are gathered together, the goal will stakeholders are gathered together, the goal will be to reach consensus on the project. Decide who be to reach consensus on the project. Decide who will facilitate this important meeting.will facilitate this important meeting.

STEP 5: Conduct the Project Agreement STEP 5: Conduct the Project Agreement MeetingMeeting

Conduct the Project Agreement Meeting. Conduct the Project Agreement Meeting. Using the handout you prepared in Step 4, lead Using the handout you prepared in Step 4, lead the group through a discussion of the summarized the group through a discussion of the summarized findings from the Step 3 interviews.findings from the Step 3 interviews.

STEP 6:Prepare the Project Agreement Letter STEP 6:Prepare the Project Agreement Letter and facilitate necessary approvalsand facilitate necessary approvals

Summarize the findings of the Project Summarize the findings of the Project Agreement Meeting, describing the consensus that Agreement Meeting, describing the consensus that was reached. Detail your findings in the Project was reached. Detail your findings in the Project Agreement Letter. Meet with the key decision-Agreement Letter. Meet with the key decision-makers and review the letter.makers and review the letter.

When you have a signed Project Agreement Letter, When you have a signed Project Agreement Letter, you are ready to define desired performance.you are ready to define desired performance.

STAGE 3: Define Desired PerformanceSTAGE 3: Define Desired Performance

In order to gain consensus on exactly what In order to gain consensus on exactly what is desired in terms of performance, you will work is desired in terms of performance, you will work with the stakeholder group to state desired with the stakeholder group to state desired performance in results-based, measurable terms. performance in results-based, measurable terms. This focus will help every other step of the PI This focus will help every other step of the PI process (including evaluation) become more process (including evaluation) become more precise, clear and targeted. Indeed, without a precise, clear and targeted. Indeed, without a careful definition of desired performance, the rest careful definition of desired performance, the rest of the process carries little meaning. To of the process carries little meaning. To paraphrase the March Hare, if you don’t know paraphrase the March Hare, if you don’t know your destination, any path will do.your destination, any path will do.

STEP 1: Write desired performance statementsSTEP 1: Write desired performance statements Define performance in specific and measurable terms. Define performance in specific and measurable terms.

For the definition to be valid, you must ensure that all key For the definition to be valid, you must ensure that all key stakeholders have input and/or agree on the performance stakeholders have input and/or agree on the performance statements. This process will take time, as individual statements. This process will take time, as individual stakeholders may have opinions that differ, especially when stakeholders may have opinions that differ, especially when

they are forced to be specific.they are forced to be specific.

STEP 2: Attach measures to each desired STEP 2: Attach measures to each desired performance statementperformance statement

For each statement of desired performance For each statement of desired performance you should arrive at a performance indicator that you should arrive at a performance indicator that describes a quality, quantity, time or cost. This step describes a quality, quantity, time or cost. This step will help the group decide what really matters about will help the group decide what really matters about the performance in question: Is it how well it is the performance in question: Is it how well it is done, how often it happens, on how timely a basis done, how often it happens, on how timely a basis or all three?or all three?

STEP 3: Set targets for desired levels of STEP 3: Set targets for desired levels of performanceperformance

Once you have decided on the measurable Once you have decided on the measurable indicators for the performance in question, help the indicators for the performance in question, help the group set targets for each indicator. For example, group set targets for each indicator. For example, should the performance follow the standard 100% should the performance follow the standard 100% of the time, or is 90% acceptable to start with? of the time, or is 90% acceptable to start with? Should the providers always arrive by 9:00? Or is it Should the providers always arrive by 9:00? Or is it OK to arrive occasionally at 9:15? Remember that OK to arrive occasionally at 9:15? Remember that the targets you set now may be revised as time the targets you set now may be revised as time goes by. As with “ideal” performance, setting goes by. As with “ideal” performance, setting targets that are unrealistic can be a detractor for targets that are unrealistic can be a detractor for performance. Revising targets is a good technique performance. Revising targets is a good technique when performance levels are initially very low and when performance levels are initially very low and you need to set interim goals for performers.you need to set interim goals for performers.

STAGE 4: Describe Actual PerformanceSTAGE 4: Describe Actual Performance In the previous stage, the team defined In the previous stage, the team defined

desired performance in specific and measurable desired performance in specific and measurable terms. Using those same indicators and measures, terms. Using those same indicators and measures, describe the current, or actual, performance. describe the current, or actual, performance. Possible sources of performance data include clinic Possible sources of performance data include clinic records, ministry of health statistics, and previous records, ministry of health statistics, and previous projects and studies completed in the same area. projects and studies completed in the same area. Many times, however, existing data on current Many times, however, existing data on current performance will be insufficient, and you will have performance will be insufficient, and you will have to collect data on your own.The data gathered for to collect data on your own.The data gathered for this stage will serve as the baseline for this stage will serve as the baseline for determining the effectiveness of the interventions. determining the effectiveness of the interventions. After the interventions have been implemented for After the interventions have been implemented for a designated time period, you will compare a designated time period, you will compare performance data to this baseline data to performance data to this baseline data to determine whether or how much performance has determine whether or how much performance has changed. changed.

STEP 1: Decide on data collection methodsSTEP 1: Decide on data collection methods The team must decide which methods to The team must decide which methods to

use to collect current performance data linked to use to collect current performance data linked to the desired performance statements that were the desired performance statements that were written earlier. For each statement of desired written earlier. For each statement of desired performance, determine the best method or performance, determine the best method or combination of methods for gathering reliable, combination of methods for gathering reliable, valid data. For example, to gather data on the valid data. For example, to gather data on the quality of medical treatment, a combination of quality of medical treatment, a combination of client exit interviews and direct observation may client exit interviews and direct observation may be optimal. In another case, simulations may be be optimal. In another case, simulations may be necessary. As part of data-collection necessary. As part of data-collection methodology, you will decide on the most methodology, you will decide on the most adequate sample size and strategy (e.g., adequate sample size and strategy (e.g., systematic cluster) for your project.systematic cluster) for your project.

STEP 2: Design data collection instrumentsSTEP 2: Design data collection instruments With the team or a subset of the team, With the team or a subset of the team,

design data collection instruments. The forms design data collection instruments. The forms should simplify data compilation and be easy to should simplify data compilation and be easy to use in the field. Data collection instruments often use in the field. Data collection instruments often include interview guides, observation checklists, include interview guides, observation checklists, focus group discussion guides, questionnaires focus group discussion guides, questionnaires and survey forms.and survey forms.

STEP 3: Identify and equip data collectorsSTEP 3: Identify and equip data collectors Identify the data collectors and prepare Identify the data collectors and prepare

them for activity in the field. At times, the data them for activity in the field. At times, the data collectors will come from your immediate team or collectors will come from your immediate team or organization. Other times, you will need to hire organization. Other times, you will need to hire data collectors. In any case, once the team is data collectors. In any case, once the team is identified, they must be equipped with everything identified, they must be equipped with everything they need to collect baseline data. Readying they need to collect baseline data. Readying them may include training, supplying vehicles or them may include training, supplying vehicles or obtaining lodging in the field.obtaining lodging in the field.

STEP 4: Compile and analyze dataSTEP 4: Compile and analyze data Compile the data that have arrived from the data Compile the data that have arrived from the data

collectors in the field. For smaller projects, this compilation collectors in the field. For smaller projects, this compilation may be done by hand on tables and check sheets. For larger may be done by hand on tables and check sheets. For larger efforts, statistical analysis software may be used. In either efforts, statistical analysis software may be used. In either case, the data should be compiled in a way that case, the data should be compiled in a way that communicates the current level of performance to the team communicates the current level of performance to the team and to others outside the immediate team. For example, if and to others outside the immediate team. For example, if the desired level of performance is that 90% of eligible the desired level of performance is that 90% of eligible clients will receive treatment, what is the actual clients will receive treatment, what is the actual performance? 40%? 50%? Keep the audience in mind when performance? 40%? 50%? Keep the audience in mind when choosing presentation formats.choosing presentation formats.

STEP 5: Make statements about actual STEP 5: Make statements about actual performanceperformance

Using the Performance Improvement Using the Performance Improvement Specification form, take each statement of desired Specification form, take each statement of desired performance and make a corresponding entry that performance and make a corresponding entry that shows the actual (current) performance. Be able to shows the actual (current) performance. Be able to back up the statements with appropriate data back up the statements with appropriate data presentations (charts, graphs, etc).presentations (charts, graphs, etc).

STAGE 5:Describe Performance GapsSTAGE 5:Describe Performance Gaps If the previous two stages are completed If the previous two stages are completed

thoroughly, and performance statements are thoroughly, and performance statements are given observable and measurable indicators, given observable and measurable indicators, describing performance gaps becomes a simple describing performance gaps becomes a simple arithmetic exercise.For example, consider a case arithmetic exercise.For example, consider a case where desired performance states that providers where desired performance states that providers should offer HIV/AIDS counseling to 100% of should offer HIV/AIDS counseling to 100% of eligible clients who appear at the clinic, and eligible clients who appear at the clinic, and baseline data show that providers currently baseline data show that providers currently counsel only 25% of eligible clients. The counsel only 25% of eligible clients. The statement of the performance gap is that statement of the performance gap is that providers do not counsel 75% of eligible clients providers do not counsel 75% of eligible clients who appear at the clinic.who appear at the clinic.

STEP 1: Describe the performance gapsSTEP 1: Describe the performance gaps Using the same measures as were used to Using the same measures as were used to

describe desired and actual performance, describe describe desired and actual performance, describe the performance gaps.the performance gaps.

STEP 2: Decide whether to work on each gapSTEP 2: Decide whether to work on each gap Depending on your project, the list of gaps may be Depending on your project, the list of gaps may be

quite long. It is important to pause and decide as a group quite long. It is important to pause and decide as a group whether each gap is worth the effort that will be required to whether each gap is worth the effort that will be required to close it. Consider whether the gap is wide or narrow. Take close it. Consider whether the gap is wide or narrow. Take into account whether the gap seriously affects the potential into account whether the gap seriously affects the potential of the organization to reach its goals. Contemplate how the of the organization to reach its goals. Contemplate how the gap affects client outcomes. Finally, weigh the amount of gap affects client outcomes. Finally, weigh the amount of effort necessary to close the gap. Some gaps may not effort necessary to close the gap. Some gaps may not deserve further attention. Efforts to close some gaps may deserve further attention. Efforts to close some gaps may need to be postponed while you work on more serious or need to be postponed while you work on more serious or urgent gaps.urgent gaps.

STAGE 6: Find Root CausesSTAGE 6: Find Root Causes Perhaps the most critical step in the PI process is Perhaps the most critical step in the PI process is

identifying the root causes you will address with identifying the root causes you will address with appropriate interventions. Root-cause analysis should take appropriate interventions. Root-cause analysis should take place immediately following the review of performance place immediately following the review of performance gaps with the same stakeholder group that helped to define gaps with the same stakeholder group that helped to define desired performance. The outcome should be the desired performance. The outcome should be the identification of a root cause for each gap observed.identification of a root cause for each gap observed.

STAGE 7: Select and Design InterventionsSTAGE 7: Select and Design Interventions The goal of this stage is to select The goal of this stage is to select

interventions that will close the performance gaps interventions that will close the performance gaps identified during the previous stages. One or identified during the previous stages. One or more interventions should address each root more interventions should address each root cause. All of the interventions in your package cause. All of the interventions in your package should address at least one of the root causes. should address at least one of the root causes. When you have described the root causes in When you have described the root causes in terms of missing performance factors, certain terms of missing performance factors, certain interventions will naturally suggest themselves. interventions will naturally suggest themselves. For example, if knowledge and skills are lacking, For example, if knowledge and skills are lacking, some kind of training intervention will be some kind of training intervention will be appropriate. appropriate.

STEP 1: Propose and select interventionsSTEP 1: Propose and select interventions The aim of this step is to agree on the general The aim of this step is to agree on the general

interventions, not to design each intervention (which may interventions, not to design each intervention (which may require additional expertise). The team should describe the require additional expertise). The team should describe the proposed interventions in sufficient detail to allow the team proposed interventions in sufficient detail to allow the team leader to approach the stakeholders with a clear leader to approach the stakeholders with a clear explanation of how interventions will address important explanation of how interventions will address important

causescauses..

STEP 2: Develop a design plan for each STEP 2: Develop a design plan for each interventionintervention

If there are multiple interventions, each If there are multiple interventions, each type of intervention may require a smaller team type of intervention may require a smaller team to work semi-independently. These teams can be to work semi-independently. These teams can be formed at the end of the large-group planning formed at the end of the large-group planning meeting.meeting.

STEP 4: Develop, field test and produce the final STEP 4: Develop, field test and produce the final version of the interventionsversion of the interventions

The nature and extent of testing will depend The nature and extent of testing will depend on the stakes involved, the type of intervention, and on the stakes involved, the type of intervention, and the time and resources available. For example, you the time and resources available. For example, you would not pre-test a strategic planning intervention, would not pre-test a strategic planning intervention, but if an intervention requires a major production or but if an intervention requires a major production or management effort, or significant costs, then the management effort, or significant costs, then the development team should produce and test a development team should produce and test a prototype version of the intervention and/or prototype version of the intervention and/or materials before moving to final production and materials before moving to final production and implementation. Testing includes reviews with implementation. Testing includes reviews with users, clients and subject-matter experts, or actual users, clients and subject-matter experts, or actual trials with members of the target audience in the trials with members of the target audience in the environment in which the intervention will take environment in which the intervention will take place. Testing may reveal major weaknesses in the place. Testing may reveal major weaknesses in the intervention design or materials and provide the intervention design or materials and provide the necessary feedback to make revisions before final necessary feedback to make revisions before final production. Based on the test data and feedback, production. Based on the test data and feedback, revise the workplan as necessary.revise the workplan as necessary.

STAGE 8: Implement InterventionsSTAGE 8: Implement Interventions The goal of this stage is to execute the intervention package The goal of this stage is to execute the intervention package

designed and developed in the previous stage. The PI facilitator designed and developed in the previous stage. The PI facilitator and various colleagues create and manage the implementation and various colleagues create and manage the implementation team, manage the overall implementation, and oversee team, manage the overall implementation, and oversee organizational change processes. The most important outcome of organizational change processes. The most important outcome of this stage will be implemented interventions that close the this stage will be implemented interventions that close the performance gaps. performance gaps.

STEP 1: Build the implementation teamSTEP 1: Build the implementation team The first step is to reconfirm the PI team including the PI facilitator The first step is to reconfirm the PI team including the PI facilitator

and client organization staff. The “PI facilitator” may be a program and client organization staff. The “PI facilitator” may be a program manager. The PI team then identifies and contracts with the manager. The PI team then identifies and contracts with the necessary professionals to implement the interventions. The best necessary professionals to implement the interventions. The best place to start locating these team members is with the people who place to start locating these team members is with the people who helped develop the interventions. Often, the best team to helped develop the interventions. Often, the best team to implement an intervention is the one that designed it. There are, implement an intervention is the one that designed it. There are, however, occasions when developers may not be the ideal however, occasions when developers may not be the ideal implementers. Examples include:implementers. Examples include:

An instructional designer who is not the best presenter of An instructional designer who is not the best presenter of classroom trainingclassroom training

A scriptwriter who designed the training video but is not a member A scriptwriter who designed the training video but is not a member of the production company you want to hire to produce the video.of the production company you want to hire to produce the video.

STEP 2: Develop a detailed implementation planSTEP 2: Develop a detailed implementation plan Because implementation projects can become large and Because implementation projects can become large and

include many individuals and organizations with their own include many individuals and organizations with their own responsibilities, the PI team should construct an overall responsibilities, the PI team should construct an overall plan detailing the entire implementation structure and the plan detailing the entire implementation structure and the role of each individual. Because PI projects focus on results, role of each individual. Because PI projects focus on results, it is also important to maintain clarity about the it is also important to maintain clarity about the performance expectations of each intervention.performance expectations of each intervention.

STEP 3: Conduct monitoring activitiesSTEP 3: Conduct monitoring activities Once the implementation is under way, the role of Once the implementation is under way, the role of

the PI facilitator (or project manager) switches to the PI facilitator (or project manager) switches to monitoring and the performance of interim evaluations. At monitoring and the performance of interim evaluations. At the milestone points in the implementation plan, the PI the milestone points in the implementation plan, the PI facilitator or delegate should assess milestone goals, facilitator or delegate should assess milestone goals, provide feedback and solve problems. If deadlines slip, the provide feedback and solve problems. If deadlines slip, the PI facilitator will recalculate the project timeline and inform PI facilitator will recalculate the project timeline and inform other implementation team members of any necessary other implementation team members of any necessary changes in their deadlines.changes in their deadlines.

STAGE 9: EvaluationSTAGE 9: Evaluation The goal of this stage is to assess the The goal of this stage is to assess the

effects of PI project interventions on provider effects of PI project interventions on provider performance and to judge the extent to which the performance and to judge the extent to which the performance gaps have closed. The evaluation performance gaps have closed. The evaluation may also examine organizational performance or may also examine organizational performance or the capacity of an institution to support and the capacity of an institution to support and sustain improved provider performance.sustain improved provider performance.

The evaluation stage is the culmination of a The evaluation stage is the culmination of a series of evaluative exercises conducted at each series of evaluative exercises conducted at each stage in the life of the project. The end-of-project stage in the life of the project. The end-of-project evaluation is based on a foundation laid during evaluation is based on a foundation laid during the definition of desired performance and fed by the definition of desired performance and fed by assessments conducted during intervention assessments conducted during intervention selection and implementation.selection and implementation.

STEP 1: Implement the evaluation planSTEP 1: Implement the evaluation plan

An evaluation plan will:An evaluation plan will: Identify the purpose, users, resources and Identify the purpose, users, resources and

timelines of the evaluationtimelines of the evaluation Select the key evaluation questions and Select the key evaluation questions and

indicators and the best design to measure indicators and the best design to measure intended resultsintended results

Sequence evaluation activities (such as Sequence evaluation activities (such as completing baseline documentation, conducting completing baseline documentation, conducting participant follow-ups, materials pre-tests, project participant follow-ups, materials pre-tests, project reviews and special studies)reviews and special studies)

Prepare data collection and data analysis plans, Prepare data collection and data analysis plans, including cost as well as results or program dataincluding cost as well as results or program data

Plan for communication, dissemination and use of Plan for communication, dissemination and use of evaluation resultsevaluation results

Identify the technical competencies needed on Identify the technical competencies needed on the evaluation team(s).the evaluation team(s).

STEP 2: Conduct data collection, analysis and STEP 2: Conduct data collection, analysis and interpretationinterpretation

This step includes:This step includes: Gathering and organizing data in a systematic Gathering and organizing data in a systematic

way to reduce sources of bias and increase way to reduce sources of bias and increase validityvalidity

Discerning patterns, trends and comparisons from Discerning patterns, trends and comparisons from qualitative and quantitative dataqualitative and quantitative data

Linking cost and results data to have a Linking cost and results data to have a retrospective Cost and Results Analysis (CRA)retrospective Cost and Results Analysis (CRA)

Involving client and stakeholders in interpreting Involving client and stakeholders in interpreting datadata

Employing standards to arrive at conclusions.Employing standards to arrive at conclusions.

STEP 3: Write a report and communicate evaluation resultsSTEP 3: Write a report and communicate evaluation results

This step includes:This step includes: Writing a report describing the methodology, findings and Writing a report describing the methodology, findings and

conclusionsconclusions Selecting appropriate graphics to communicate summary findingsSelecting appropriate graphics to communicate summary findings Formulating recommendations based on conclusions and Formulating recommendations based on conclusions and

consultations with the client.consultations with the client. The report should present findings so an audience can clearly see:The report should present findings so an audience can clearly see: Changes in performanceChanges in performance How these changes can be attributed to the interventionsHow these changes can be attributed to the interventions Cost of the interventions.Cost of the interventions. If the evaluation design warrants, the report should also present If the evaluation design warrants, the report should also present

the effects (if any) of alternative interventions or the absence of the effects (if any) of alternative interventions or the absence of interventions in control areas and discuss differences between interventions in control areas and discuss differences between those areas and the “case” area.those areas and the “case” area.

Cost and Results Analysis (CRA) is especially useful in comparing Cost and Results Analysis (CRA) is especially useful in comparing alternatives and showing which offers the best value.alternatives and showing which offers the best value.

The ultimate goal of evaluation is the use of results to:The ultimate goal of evaluation is the use of results to: Demonstrate the validity of a new approach (i.e., “what works”)Demonstrate the validity of a new approach (i.e., “what works”) Identify areas to be strengthened in future project designs (i.e., Identify areas to be strengthened in future project designs (i.e.,

“what didn’t work and what to do differently next time”).“what didn’t work and what to do differently next time”).

When Presented with Organizational ProblemsWhen Presented with Organizational Problems All performance gaps can eventually be All performance gaps can eventually be

traced to performers or the absence of traced to performers or the absence of performers. During this stage, the objective is to performers. During this stage, the objective is to identify who those performer(s) are when an identify who those performer(s) are when an organizational problem is broad and does not organizational problem is broad and does not readily implicate particular performers. For readily implicate particular performers. For example, "Infection rates are skyrocketing" or example, "Infection rates are skyrocketing" or "Postnatal maternal deaths are on the rise." The "Postnatal maternal deaths are on the rise." The key to successfully addressing such problems is key to successfully addressing such problems is to understand the situation thoroughly enough to to understand the situation thoroughly enough to identify all of the performer(s) related to the identify all of the performer(s) related to the gaps. As these problems are often complex, gaps. As these problems are often complex, solving them may require additional investigation solving them may require additional investigation upfront before the traditional PI process can upfront before the traditional PI process can begin.begin.

STEP 1: Understand the problemSTEP 1: Understand the problem

The key stakeholder(s) will have opinions of what the The key stakeholder(s) will have opinions of what the problem is — and these opinions will be important to the problem is — and these opinions will be important to the success of the process (whether or not they are correct). success of the process (whether or not they are correct). You will need to conduct an interview, or several You will need to conduct an interview, or several interviews, to answer the following questions:interviews, to answer the following questions:

What is the problem?What is the problem? What is the desired outcome? What would the situation What is the desired outcome? What would the situation

look life if there was no problem?look life if there was no problem? What are possible causes? (Note: care should be taken not What are possible causes? (Note: care should be taken not

to accept any of these causes as 'truth' until further to accept any of these causes as 'truth' until further investigation is done.)investigation is done.)

What does the client see as possible solutions?What does the client see as possible solutions? What potential constraints are there? (political, financial, What potential constraints are there? (political, financial,

etc.)etc.) Is the problem worth solving? Discuss with the client, Is the problem worth solving? Discuss with the client,

making comparisons if possible with similar organizations making comparisons if possible with similar organizations or benchmarks.or benchmarks.

Who will be involved in the PI process from the Who will be involved in the PI process from the organization? Identify local individuals to join the PI team.organization? Identify local individuals to join the PI team.

STEP 2: Understand the organizationSTEP 2: Understand the organization

Now that you have asked your clients their opinions of the Now that you have asked your clients their opinions of the situation, it's time to explore the organization with your situation, it's time to explore the organization with your own eyes. The objective is to see how things work, start to own eyes. The objective is to see how things work, start to fill in the picture and locate possible problematic areas. fill in the picture and locate possible problematic areas. Don't stop when you've found one problem — the whole Don't stop when you've found one problem — the whole process should be understood and mapped out. By the end process should be understood and mapped out. By the end of this step, you should be able to answer the following of this step, you should be able to answer the following questions:questions:

What are the overall goals of the organization?What are the overall goals of the organization? How is the organization structured? What does it look like?How is the organization structured? What does it look like? Which components (or units) of the organization affect the Which components (or units) of the organization affect the

problem area?problem area? What are the processes and functions of these What are the processes and functions of these

components? How do they go about their work?components? How do they go about their work? What are the products or accomplishments that correspond What are the products or accomplishments that correspond

to each process?to each process? How do the components/processes/functions relate? Where How do the components/processes/functions relate? Where

do they interact or intersect?do they interact or intersect? Can you identify possible problem areas already?Can you identify possible problem areas already?

STEP 3: What should the situation look like?STEP 3: What should the situation look like? What should the organization, systems, processes What should the organization, systems, processes

look like (think in terms of "what is possible")? look like (think in terms of "what is possible")? This is an opportunity to use your experience and This is an opportunity to use your experience and guide the discussion. As an outsider to the guide the discussion. As an outsider to the situation, it will be important for you to help the situation, it will be important for you to help the client see 'the whole picture'. Don't limit yourself client see 'the whole picture'. Don't limit yourself too soon — a great deal might be possible if you too soon — a great deal might be possible if you have the backing and cooperation of key have the backing and cooperation of key stakeholders. At the very least, this step will allow stakeholders. At the very least, this step will allow you to discuss issues such as alignment of you to discuss issues such as alignment of objectives and performance indicators with the objectives and performance indicators with the client.client.

STEP 4: Identify the gaps and the performers on which STEP 4: Identify the gaps and the performers on which to focus the PI processto focus the PI process

With the PI Team, develop a set of preliminary findings and With the PI Team, develop a set of preliminary findings and conclusions, and report to the client and other key conclusions, and report to the client and other key stakeholders. Questions to be answered include:stakeholders. Questions to be answered include:

Where are the major gaps?Where are the major gaps? Do gaps correspond to pieces that are missing from the Do gaps correspond to pieces that are missing from the

organizational structure (e.g., a new area/performance will organizational structure (e.g., a new area/performance will be required to close a gap)?be required to close a gap)?

Can you identify the performers who are responsible for the Can you identify the performers who are responsible for the gaps?gaps?

Process StepsProcess Steps (These can be done with a larger stakeholder meeting or (These can be done with a larger stakeholder meeting or

with smaller meetings with key stakeholders).with smaller meetings with key stakeholders). Present maps of desired and current status of Present maps of desired and current status of

organization/processes.organization/processes. Identify and prioritize gap areas.Identify and prioritize gap areas. Develop/propose a working plan to address most critical Develop/propose a working plan to address most critical

area(s) first.area(s) first. Prepare MOE with conclusions, recommendations.Prepare MOE with conclusions, recommendations.

Lessons LearnedLessons LearnedPerformance developmentPerformance development steps have steps have

proven effective and now widely use, there has proven effective and now widely use, there has been increased interest and experience in been increased interest and experience in applying these same approaches. Recognizing applying these same approaches. Recognizing that successful performance improvement that successful performance improvement processes require adoption and tailoring in given processes require adoption and tailoring in given situation, interesting patterns of challenges and situation, interesting patterns of challenges and opportunities particularly to health service opportunities particularly to health service delivery. For instance, the growing practices of delivery. For instance, the growing practices of integration and intersectoral collaboration have integration and intersectoral collaboration have reinforced expectations to broader range of reinforced expectations to broader range of stakeholders will be involved in decision making stakeholders will be involved in decision making and planning process. and planning process.

BibliographyBibliography

BooksBooksMoore, K., et al. (1992). Moore, K., et al. (1992). Principles and vales of continuous Principles and vales of continuous

Improvement healthcare. Improvement healthcare. Paper presented at Management Paper presented at Management Conference, St. Luke’s Episcopal Hospital , Huston Texas: p. Conference, St. Luke’s Episcopal Hospital , Huston Texas: p. 156-159156-159

Cavaleri Steven ACavaleri Steven A., ., Fearon David S. (2006). Inside Knowledge: Fearon David S. (2006). Inside Knowledge: Rediscovering the Source of Performance Improvement. Rediscovering the Source of Performance Improvement.

The McGraw-Hill Companies, Inc., 1221 Avenue of the The McGraw-Hill Companies, Inc., 1221 Avenue of the Americas, New York,NY,10020: p. 678-679Americas, New York,NY,10020: p. 678-679

Electronic ResourcesElectronic Resourceshttp://www.intrahealth.org/sst/stage0.htmlhttp://www.intrahealth.org/sst/stage0.html (07-7-2012)(07-7-2012)

www.intrahealth.orgwww.intrahealth.org (07-7-2012)(07-7-2012)

http://www.insightperformance.com/ (2012)http://www.insightperformance.com/ (2012)

END