Evaluation Plan: Nonviolent Crisis Intervention Tammy …...CPI offers a variety of products,...
Transcript of Evaluation Plan: Nonviolent Crisis Intervention Tammy …...CPI offers a variety of products,...
Running head: EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 1
Evaluation Plan:
Nonviolent Crisis Intervention
Tammy Randall
Purdue University
EDCI 577
October 11, 2014
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 2
Identification of Evaluation Goals and Scope
Instructional Product Overview: Nonviolent Crisis Intervention
The Crisis Prevention Institute (CPI) is a resource for staff training that provides training in safe,
respectful, noninvasive methods for managing disruptive and assaultive behavior in a way that is
compatible with staff's duty to provide the best possible care.
CPI offers a variety of products, including the Nonviolent Crisis Intervention Training program
(CPI, 2013), which is considered the standard for crisis prevention and intervention. The hybrid
course examined here consists of two main modules that blend online learning with in-class
instruction and simulations. The first module is a self-paced online presentation that provides the
background information for all learners, including instructional videos. This module contains a
quiz at the end of each unit to ensure that each learner has mastered the basic concepts of the
program, and a participant may not advance to the next unit without a score of at least 80%. The
second module is an in-class presentation that provides learners with more hands-on instruction,
including a review of Module 1, simulations of crisis situations, and a final post-test.
The table below summarizes the content of the Hybrid course.
Unit Module Location Title
1 1 Online The CPI Crisis Development Model
2 1 Online Nonverbal Behavior
3 1 Online Paraverbal Behavior
4 1 Online Verbal Intervention
5 1 Online Precipitating Factors, Rational Detachment, Integrated
Experience
6 1 Online Staff Fear and Anxiety
7 1 Online CPI’s Personal Safety Techniques
8 1 Online Nonviolent Physical Crisis Intervention and Team Intervention
9 1 Online Custom unit on Indiana State Law updates
10 2 In class In-class overview of Units 1 - 9
11 2 In class Situational Role-Plays
12 2 In class In-class post-test
The goal of this training is to change the culture of the target organization, to modify the
professional behavior of the participants, and to change the relationships between the learners
and those they serve. Specifically, all participants should learn how to defuse disruptive behavior
before it becomes a crisis.
The course presented will include a special section for Indiana schools to help ensure compliance
with recent changes to Indiana state law concerning the use of restraint and seclusion in schools.
Under Senate Bill 345, all Indiana schools must have restraint and seclusion plans in place for
the 2014-15 school year. The law requires the use of preventive techniques, Positive Behavior
Intervention and Supports (PBIS), and conflict de-escalation to minimize or eliminate the use of
restraint and seclusion. The law also requires each school to adopt a plan to implement these
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 3
goals, as well as recurring training of the school’s staff in that school’s adopted plan and
documentation of the plan and of all incident reports (CPI, 2013).
The objectives of this course are summarized below (CPI, 2013):
Reduce the risk of injury due to physical interventions to staff and to those they serve.
Comply with any legal requirements and minimize the risk of legal liability.
Meet standards for accreditation.
Improve staff retention by providing critical skills.
Improve communication by establishing a common vocabulary.
Reduce stress and anxiety that may result from uncertainty in a crisis.
Create a safe, caring, and respectful environment for all staff members and those they serve.
Audience & Context
The target for this training is any group of professionals who must interact with clients in such a
way that crisis situations may arise. This includes members of a variety of professions, including
education, law enforcement, correctional staff, and medical and mental health staff. It is expected
that these learners have a clear understanding of their own professions and the specific situations
that might arise in their workplace. For some participants, certification in crisis intervention and
prevention is required for their employment. In most cases, the training is sponsored by the
employer, who provides the facilities for the in-class portion of the training and provides
payment.
There are two levels of learners that are targeted by this training:
Novice learners who have not had crisis prevention / intervention training in the past, and
whose goal is to gain the knowledge and skills to deal with crisis situations in their
workplace.
Experienced learners who have taken this or similar courses and need to complete the
course to refresh their skills or to reestablish their certification. In some environments,
the re-certification is required periodically, in some cases as often as yearly.
For this presentation of the course, participants will be teachers, administrators, and staff from
the XYZ School District in Indiana. During the course of their work day, these individuals may
encounter students or visitors in disruptive or crisis situations, and they must be prepared to deal
with any situation in a way that minimizes injury to all participants and that complies with school
policy and Indiana law.
Participants must complete an online module before the presentation of the in-class instruction.
They will need to have access to an internet-capable computer running a recent version of one of
the popular web browsers. The online course materials will be available for one month before the
scheduled class date, and all units must be completed with a passing grade of 80% before the
start of the in-class portion of the course. This portion of the course consists of nine units, each
of which may be completed in approximately one half hour.
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 4
The in-class module (Module 2) will be presented at the client’s location. This module will
require one full day, with the morning dedicated to review of the Module 1 concepts and the
afternoon dedicated to simulations, with approximately one hour reserved for the post-test.
Access to a computer lab will be provided in order to automate the testing. One half hour will be
reserved for completing the Level 1 evaluation.
All participants who complete the course with a passing grade and complete the level 1
evaluation will receive a “Blue Card” certification and will be granted access to an online
account that provides access to ongoing trends and review materials.
Description of Process
A detailed schedule, including a Gantt chart, is included in Appendix F. The key dates are
summarized in the table below:
Task Name Start Finish
Discovery and Planning 09/08/14 09/10/14
Identify training goals of the XYZ School District 09/08/14 09/08/14
Present training overview to stakeholders 09/09/14 09/09/14
Get buy-in / sign agreement 09/10/14 09/10/14
Perform Learner Analysis 09/11/14 09/12/14
Preliminary analysis of trainees 09/11/14 09/11/14
Group trainees 09/11/14 09/12/14
Evaluation Preparation 09/15/14 09/18/14
CPI Learner Survey 09/15/14 09/15/14
Assess and Customize 09/15/14 09/15/14
CPI Post-test 09/16/14 09/16/14
Assess and Customize 09/16/14 09/16/14
CPI Behavior Survey 09/17/14 09/17/14
Assess and Customize 09/17/14 09/17/14
CPI Facility Survey 09/18/14 09/18/14
Assess and Customize 09/18/14 09/18/14
Implement hybrid training 09/19/14 10/24/14
Online course will be available for five weeks prior to
the start of the in-house training.
09/19/14 10/23/14
In-house training 10/24/14 10/24/14
Evaluation Implementation (Data Collection) 10/24/14 10/26/15
CPI Learner Survey (level 1) 10/24/14 10/24/14
CPI Post-test (level 2) 10/24/14 10/24/14
CPI Behavior Survey (level 3) 04/27/15 04/27/15
CPI Facility Survey (level 4) 04/27/15 04/27/15
CPI Behavior Survey (level 3) 10/26/15 10/26/15
CPI Facility Survey (level 4) 10/26/15 10/26/15
Data Analysis 10/27/14 11/16/15
CPI Learner Survey (level 1) 10/27/14 10/31/14
CPI Post-test (level 2) 11/03/14 11/07/14
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 5
CPI Behavior Survey (level 3) 05/12/15 05/18/15
CPI Facility Survey (level 4) 05/12/15 05/18/15
CPI Behavior Survey (level 3) 11/10/15 11/16/15
CPI Facility Survey (level 4) 11/10/15 11/16/15
Feedback / Presentation 12/08/14 12/22/15
Prepare initial report 12/08/14 12/12/14
Present initial report results 12/15/14 12/15/14
Prepare six month assessment 06/16/15 06/22/15
Present six month results 06/23/15 06/23/15
Prepare 12 month assessment 12/15/15 12/21/15
Present 12 month results 12/22/15 12/22/15
As shown in the schedule, the analysis of the special needs of the XYZ School District, as well
as the target audience for this module, have been analyzed and all required updates have been
incorporated into the standard CPI training modules. During Discovery and Planning, CPI
representatives met with the XYZ District School Board to identify the goals of the training, to
plan the training schedule, and to discuss the simulation requirements and the required facilities.
These events are reflected in the schedule summary above, and in Appendix F. At this point, the
corporation also provided summaries of all crisis reports from the previous school-year to use as
a baseline against which to compare future results.
The evaluation strategy has been agreed upon by CPI and the school board. Evaluation
instruments have been customized to reflect the changes required by the XYZ School district.
Level 1 and 2 evaluations will be performed immediately following completion of the course,
and an initial report will be generated and presented based on these instruments. At six and
twelve month intervals, Level 3 evaluations will be performed to allow participants to assess
their own behavior and how it has changed as a result of the course. Along with those
evaluations, administrators will complete the Facility survey to assess the overall value of the
course in meeting the goals of the district. The results of the six and twelve month evaluations
will be presented on June 23, 2015, and December 22, 2015, respectively.
Instruments & Instrument Descriptions
The evaluation strategy will follow levels 1 and 2 of the Kirkpatrick and Kirkpatrick model.
Level 1 Evaluation
Instrument Description
According to Kirkpatrick & Kirkpatrick (2006, p. 27), Level 1 evaluations focus on learner
reactions to the training, and they liken it to “measuring customer satisfaction” with the training.
There are two main uses for the level 1 data that is collected. First, the information will be used
to evaluate the training program, and as feedback to improve the quality of the course materials
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 6
and presentation in future programs. It also serves as a way to inform the learners that their input
is valued and will be taken into account (Kirkpatrick & Kirkpatrick, 2006, p. 27).
The level 1 evaluation will be performed using a reaction questionnaire containing questions
assessing the learners’ reactions to the quality of the material and presentation, the relevance of
the training to their own workplace, and their overall satisfaction with the hybrid course. The
reaction questionnaire is included in Appendix A. The initial reaction questionnaire must be
completed at the end of the in-class module.
Data Collection
The client has provided computer facilities with web access, and the online version of the
reaction questionnaire will be used, allowing the data collection and analysis to be done more
quickly. The results of the reaction surveys will be aggregated and the statistical results will be
shared with the client and fed back within CPI to improve future courses.
The reaction questionnaire contains three types of information:
Identifying information about the participant, which should be recorded for comparison
to other individuals or institutions .
A number of questions that measure the participant’s reactions using a five-point Likert
scale. For these questions, only the numerical value must be recorded.
Two questions provide the ability to provide free-form answers. For these questions, the
complete text provided by the participant should be recorded.
Data Analysis
The same evaluation tools will be used at all clients, allowing data to be analyzed and compared
across multiple client sites. For each client, information will be requested regarding the type of
business (e.g. an elementary school, a mental hospital, etc.) and the profession of each
participant. All the data will be compiled and the statistics will be anonymously presented to the
management of both the client and CPI. This will provide an opportunity to CPI to continuously
improve the course materials and presentation, and will provide the client with an assessment of
how well the course met the objectives of the training. Since the presentation to the client will
include data for three categories (participants from the client company, participants in similar
professions or work environments from other CPI presentations of this course, and all
participants from all CPI presentations of this course), each client will be able to assess how well
their participants did compared to those in similar training situations.
Data Reporting
For the level 1 Reaction Assessment, the following table will be provided to the client and to CPI
management, along with an anonymous compilation of all free-form responses. Each value in the
table will contain an average value from 1 to 5 based on the user responses to the reaction
questionnaire.
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 7
Client Average Industry Average Global CPI Average
Overall Satisfaction
Quality of presentation
/ materials
Relevance to
employment
Level 2 Evaluation
Instrument Description
A level 2 (learning) evaluation of the training attempts to determine the extent to which learners
gain new knowledge, acquire or improve skills, or change their attitude as a result of the training.
Evaluating learning is critical for this training, since it will result in a certification permitting the
participant to work in potentially dangerous crisis situations. A level 2 evaluation should focus
on these three questions (Kirkpatrick & Kirkpatrick, 2006, p. 42):
What knowledge was acquired?
What skills were developed or enhanced?
What attitudes were changed?
All three of these questions are important to the objectives of this training. The participant must
gain an understanding of the CPI methodology and the common vocabulary associated with it,
develop the skills to provide maximum safety in crisis situations, and develop the attitudes to
employ the acquired knowledge and skills safely and effectively.
Both qualitative and quantitative data will be gathered during the learning process using three
evaluation tools:
During Module 1 training, quizzes will be presented after each unit, and the learner will
need to pass all unit quizzes in order to advance to the next unit, and ultimately to the
Module 2 training.
During Module 2 training, simulations will be presented to assess the knowledge and
skills of the participants and to allow the facilitator to correct and further instruct the
learners.
Following completion of the course, a post-test will be given that will assess both
knowledge and attitude. Successful completion of the post-test with a score of at least
80% will be required in order to achieve “Blue Card” certification. See Appendices B
and C for the Crisis Prevention and Intervention Post-test and the answer key.
Note that no pre-test will be given, since the degree of change in knowledge, skills, and attitude
is not as important as each learner achieving at least the minimum level in each of these areas.
While Kirkpatrick and Kirkpatrick (2006, p. 43) recommend using a control group whenever
practical, doing so in this case would not be practical since all possible participants will be
completing the training at approximately the same time.
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 8
Data Collection
The client has provided computer facilities with web access, and the online version of the post-
test (see Appendix B) will be used. All participants must pass the post-test with a score of 80%
or higher in order to successfully complete the course.
The post-test contains two types of information:
Identifying information about the participant, which should be recorded for comparison
to other individuals or institutions.
A number of questions that measure the participant’s knowledge using true/false and
free-form questions. For these questions, only the percentage correct, as checked against
the answer key in Appendix C, needs to be recorded.
Data Analysis
The same evaluation tools will be used at all clients, allowing data to be analyzed and compared
across multiple client sites. For each client, information will be requested regarding the type of
business (e.g. an elementary school, a mental hospital, etc.) and the profession of each
participant. All the data will be compiled and the statistics will be anonymously presented to the
management of both the client and CPI. This will provide an opportunity to CPI to continuously
improve the course materials and presentation based on the areas in which participants scored
poorly. It will also provide the client with an assessment of how well the course met the
objectives of the training. Since the presentation to the client will include data for three
categories (participants from the client company, participants in similar professions or work
environments from other CPI presentations of this course, and all participants from all CPI
presentations of this course), each client will be able to assess how well their participants did
compared to those in similar training situations.
Data Reporting
For the level 2 Learning Assessment, the following table will be provided to the client and to CPI
management. The first value reported will be the percentage of participants in the training who
successfully completed the course and passed the post-test. The second value will report the raw
average of the post-test scores as a percentage.
Client Average Industry Average Global CPI Average
Percentage of
participants receiving
“Blue Card”
certification
Average post-test score
Level 3 Evaluation
Instrument Description
A level 3 (behavior) evaluation of the training attempts to determine the extent to which learners
modify their behavior as a result of the training. This evaluation attempts to determine whether
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 9
the knowledge, skills, and attitudes learned in the course have been transferred to the job
(Kirkpatrick & Kirkpatrick, 2006, p. 61). Evaluating behavior is critical for this training, since it
will determine whether the participants are working in compliance with current law, and if they
are behaving in a manner that will minimize injury to everyone.
Data Collection
The Crisis Prevention and Intervention Behavior Survey (see Appendix D) will be made
available online to all participants six months and twelve months after course completion, and all
participants will be required by the school district to complete the surveys. The surveys will give
the participants an opportunity to describe if and how they have used the training, and whether
they were able to apply the learned behaviors in a crisis situation. As Kirkpatrick and Kirkpatrick
(2006, p. 62) note, the tabulation of results from questionnaires can provide a good indication of
changes in behavior.
The behavior survey contains three types of information:
Identifying information about the participant, which should be recorded for comparison
to other individuals or institutions.
A number of questions that request feedback on the use of the training in job-related
situations.
o For questions 1 and 2, record the answer for each question.
o For question 3-11, record the total number of yes and no responses.
Question 12 provides the ability to provide free-form answers. For this question, the
complete text provided by the participant should be recorded.
Data Analysis
The level 3 evaluation instrument is customized for Indiana schools, but comparison data may be
used across all schools that use the CPI training. All the data will be compiled and the statistics
will be presented to the administration and school board with no personal identifying
information.
The statistics that will be calculated will be the following:
Percentage of participants with no prior crisis prevention training / some training /
extensive training.
Average response (1-5) for question 1 regarding confidence to act appropriately.
Percentage of participants choosing each of the answers to question 2.
Aggregate percentage of yes answers to questions 3-11.
Data Reporting
There are two purposes for the level three data: to measure all participants against the minimum
standards of the state of Indiana and of the school’s policy for crisis situations, and to compare
participants from the participating school against other schools where CPI training has been
presented.
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 10
For the level 3 Behavior Survey, the following table will be provided to the client and to CPI
management. This will provide all relevant information, and will permit comparison to other
schools in Indiana who have completed the CPI training. Ideally, the data should reflect
improvement from the six-month to the twelve-month survey, as participants have more
opportunities to utilize the new skills, and both values should compare favorably to the overall
average. If the results of the surveys do not indicate proficiency and confidence in the new skills,
then students may review all course materials online. In some cases, additional training may be
recommended for some participants.
6-month Survey 12-month
Survey
Average (All
Schools)
No Prior training
Some Prior training
Extensive Prior training
Average confidence level (question 1)
No opportunity to use training
(percentage)
Not confident to use training
(percentage)
Have seldom used training
Have frequently used training
Average number of yes responses (3-
11)
Level 4 Evaluation
Instrument Description
A level 4 (results) evaluation of the training attempts to establish the tangible value of the
training to the stakeholders (Kirkpatrick & Kirkpatrick, 2006, p. 69). The results of this training
should be the accomplishment of the goals of the school district: stricter compliance with school
policy in accordance with state law, reduced injury to everyone in crisis situations, and fewer
escalations of situations to the point where restraint or seclusion is used.
The Crisis Prevention and Intervention Facility Survey (see Appendix E) will be used to perform
the level 4 evaluation. This survey will gather information on the number of incidents, the use of
restraint or seclusion, and appropriateness of the participants’ actions in response to the situation.
Since Indiana State Law requires the documentation of all incidents involving restraint or
seclusion, these surveys will simply collect the data already available to the administrators. In
order to determine what part of any improvements may be attributed to the CPI training course,
the survey will also collect aggregate statistics about the school’s staff to determine what portion
of the staff have completed the CPI training, and what portion had other prior crisis management
training. This will permit a more quantifiable assessment that the goals have been met, rather
than relying solely on self-assessments of behavior in level 3. However, as Kirkpatrick and
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 11
Kirkpatrick (2006, p. 69-70) point out, proof is sometimes impossible to get, so we must be
satisfied with evidence of improvement.
Data Collection
The Crisis Prevention and Intervention Facility Survey will be made available online to school
administrators six months and twelve months after course completion, as shown in the schedule
in Appendix F. The survey information provided during course planning will provide a baseline
against which to compare the later evaluations. Kirkpatrick and Kirkpatrick (2006, p. 66) note
that it is important to allow time for results to be achieved, and most teachers and staff members
will rarely face crisis situations. For this reason, the level 4 surveys will be conducted six and
twelve months after course completion, allowing time for new behaviors to be used.
The facility survey contains four types of information:
Identifying information about the facility.
Questions determining the percentage of staff members with prior training, and who have
taken the CPI training. For these two questions, only the percentages need to be recorded.
A number of questions that request information on crisis incidents and how they were
handled. The provided values should be recorded for each question.
Question 7 provides the ability to provide free-form answers. For this question, the
complete text provided by the participant should be recorded.
Data Analysis
The level 4 evaluation instrument is customized for Indiana schools, but comparison data may be
used across all schools that use the CPI training. All the data will be compiled and the statistics
will be presented to the administration and school board with no personal identifying
information.
The statistics that will be calculated will be the following:
Percentage of staff members with prior crisis prevention training.
Percentage of staff members who have completed the CPI training.
The raw data for questions 1-6, normalized to a six month period.
Data Reporting
For the level 4 Facility Survey, the following table will be provided to the client and to CPI
management. For comparison, the incident information is presented that was obtained during the
planning period, along with that obtained in the six- and twelve-month facility surveys. Data
from all schools that have taken the CPI training is also presented. Note that all values are
normalized to a six-month period for meaningful comparison.
Note that the information provided regarding the percentage of the staff who had prior crisis
management training, and those who have taken the CPI training, is meant to make the data more
meaningful by exposing influences that might bias the presented values.
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 12
Prior to
training
6-month
survey
12-month
survey
All CPI
Schools
Prior training (percentage)
Completed CPI training
(percentage)
Number of incidents
Number of incidents resulting in
injury
Number of incidents resulting in
restraint
Number of incidents resulting in
seclusion
Number of incidents where state
law was not followed
Number of incidents in which
school policy was not followed
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 13
Evaluation Plan Executive Summary
Discussion of Need
School administrators and staff are faced with the daunting task of keeping all students and staff
members safe and to control all students to prevent crisis situations from arising and escalating.
Recent changes to Indiana state law concerning the use of restraint and seclusion in schools will
require policy changes for many schools, and training to ensure that the law and adopted school
policy is followed.
This CPI course is a hybrid of online learning and on-site training and simulation, and is
designed to meet the requirements of Indiana law. This course will provide all staff members
with the tools to effectively handle crisis situations, to help avoid escalation and to affect de-
escalation, and to help prevent injury to all participants. This training will help schools comply
with Indiana law, and will help provide a safe, secure environment for both students and staff.
Most schools must carefully schedule training to minimize time away from students and to fit
into tight staffing budgets. By permitting most of the course to be completed online in the
privacy of staff members’ homes, the time away from the classroom will be minimized, and the
course may be completed in a single day of on-site training.
Objective of Evaluation
The goals of this training are to increase safety and to reduce the number of incidents that
escalate to the use of restraint or seclusion. By the end of the training, all participants should feel
confident that they have the knowledge and skills to handle crisis situations effectively and
safely. The purpose of the evaluation is to ensure that these goals are met to the satisfaction of all
stakeholders.
Summary of Evaluation Plan
A four-level evaluation will be performed. Level 1 evaluations are designed to ensure that
participants are satisfied with the training, and level 2 evaluations ensure that participants have
learned all the knowledge, skills, and attitudes required in the training. Level 1 and 2 evaluations
will be performed immediately following completion of the course, and an initial report will be
generated and presented based on these instruments. Level 3 evaluations determine if the
participants have modified their behavior in response to the training. At six and twelve month
intervals, Level 3 evaluations will be performed using the Behavior Survey to allow participants
to assess their own behavior and how it has changed as a result of the course. Findings from
these surveys will be collected and presented to all stakeholders. Level 4 evaluations assess the
results of the training to determine its value. At six and twelve month intervals, administrators
will complete the Facility Survey to assess the overall value of the course in meeting the goals of
the district. Findings from these surveys will be presented along with the Level 3 findings.
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 14
Key Findings
The online portion of the course is currently available to all participants, and the on-site training
is scheduled to take place on October 24. Reaction and learning data will be collected during the
training, and the initial report based on this data will be presented on December 15. The
behavior and results data will be collected six months and twelve months after the course has
been completed, and additional reports will be presented on June 23, 2015, and December 22,
2015.
Some of the results from level 1 evaluations will include the following:
Percentage of participants whose overall reaction was positive.
Percentage of participants who found the training relevant to their jobs.
Percentage of participants who found the training engaging.
Percentage of participants who found the training content sufficient.
Some of the results from level 2 evaluations will include the following:
Percentage of participants who have mastered the key information and techniques
required by the training.
The average score for participants in the training.
Some of the results from level 3 evaluations will include the following:
Percentage of participants who had an opportunity to use their training in their jobs.
Percentage of participants who have transferred the knowledge and techniques to their
jobs.
Some examples of how the training was used in real crisis situations.
Some of the results from level 4 evaluations will include the following:
To what extent have the objectives of increased safety and increased adherence to school
policy been achieved.
To what extent can the CPI training be shown to be responsible for any increase, taking
into account prior training and level of training of current staff.
Recommendations
Some possible recommendations to the school district may include the following:
Based on the results of the level 2, 3, and 4 evaluations, further training may be
recommended for some or all participants.
If the level 3 evaluation indicates excessive non-compliance with the policy, then policy
changes may be recommended to add enforcement or to prevent individuals from acting
alone in crisis situations.
If the level 4 evaluation indicates that safety and compliance have not increased to the
standards of other Indiana schools, then policy changes may be recommended.
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 15
References
Crisis Prevention Institute, Inc. (CPI). (2013). CPI: Training and consulting in
behavior management & dementia care. Retrieved from
http://www.crisisprevention.com/.
Kirkpatrick, D. and Kirkpatrick, J. (2006). Evaluating training programs: The four levels (3th
ed). San Francisco, CA: Berrett-Koehler Publishers, Inc.
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 16
Appendices
Appendix A: Crisis Prevention and Intervention Learner Survey .........................................17
Appendix B: Crisis Prevention and Intervention Post-test ....................................................19
Appendix C: Crisis Prevention and Intervention Post-test Answer Key ...............................20
Appendix D: Crisis Prevention and Intervention Behavior Survey .......................................21
Appendix E: Crisis Prevention and Intervention Facility Survey ..........................................24
Appendix F: Crisis Prevention and Intervention Project Plan and Timeline .........................26
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 17
Appendix A: Crisis Prevention and Intervention Learner Survey
Instructions: Your impressions of the Crisis Intervention course are important to help us create
the best possible learner experience. Please answer each question below so that we can derive
they clearest possible picture of your impressions of this course.
Using the five-point scale shown, please choose the number that best represents your opinion.
Student Registration Number: _________________________________________________
Type of business (e.g. educational facility): ______________________________________
Your Profession (e.g. elementary teacher): ______________________________________
Do you have prior experience with crisis prevention training before this course?
None Some Extensive
Strongly
Disagree
Disagree Neutral Agree Strongly
Agree
1. The videos presented in Module 1
helped me understand the concepts
presented in the course.
1 2 3 4 5
2. The online quizzes for Module 1 were a
reasonable test of the prerequisite
knowledge for the Module 2 training.
1 2 3 4 5
3. After completing this training I feel I
have the tools and knowledge to be
well equipped to manage a difficult
situation to help keep all stakeholders
safer.
1 2 3 4 5
4. I would recommend this training to
other colleagues in my profession. 1 2 3 4 5
5. Please share something new you
learned or of which you gained better
understanding:
6. I found the nonviolent crisis
intervention training to be applicable
for my professional needs.
1 2 3 4 5
7. I believe this training course in
nonviolent crisis intervention has
reduced my risk of injury.
1 2 3 4 5
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 18
8. The material presented was well
organized, understandable and there
were good examples of the content in
useful situations.
1 2 3 4 5
9. This course has improved my
confidence in handling a crisis
situation.
1 2 3 4 5
10. The group simulations in Module 2
added understanding to the Module 1
self-paced online training and group
class lecture presentation.
1 2 3 4 5
11. The group leader handled the
simulations in a professional manner
(e.g. learners were not made to feel
unnecessarily uncomfortable when
performing paired simulations).
1 2 3 4 5
12. I feel this course was a good fit for
blended learning, with the right amount
of online self-paced modules, group
lecture presentation, and group
simulations for optimal understanding.
1 2 3 4 5
13. Please add additional comments
providing reasons why, in your opinion,
this course was a good fit or not for
blended learning.
14. I felt that taking a qualifying post-test
immediately after completing the
Module 2 course was a reasonable
requirement (e.g. I felt prepared to take
the test at that point, and the timing was
appropriate).
1 2 3 4 5
15. After completing this course, I would
handle a crisis substantially differently
than I would have before this training.
1 2 3 4 5
16. The speed of the class was about right
for me to master the material. 1 2 3 4 5
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 19
Appendix B: Crisis Prevention and Intervention Post-test
Please answer each question below based on what you have learned from this course.
Student Registration Number: _________________________________________________
Type of business (e.g. educational facility): ______________________________________
Your Profession (e.g. elementary teacher): ______________________________________
Do you have prior experience with crisis prevention training before this course?
None Some Extensive
1. List two ways in which a hostile person may vent aggression or hostility.
2. List the four distinct and identifiable behavior levels present in every crisis development
situation.
3. (True / False) Individuals in crisis always exhibit the behaviors in the exact order described
in the CPI Crisis Development Model.
4. What approach is characterized by reassuring the client and using positive verbal and non-
verbal approaches to determine what is upsetting the client?
5. What approach is characterized by offering the client choices and setting reasonable limits
with reasonable consequences?
6. What approach is appropriate as a last resort when others have been placed in imminent
danger?
7. What approach involves attempting to re-establish communication once the client has begun
to calm down and show remorse?
8. Under what condition is setting limits the preferred approach?
9. (True / False) Physical intervention may sometimes be used as a punitive measure.
10. Describe the CPI supportive stance.
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 20
Appendix C: Crisis Prevention and Intervention Post-test Answer Key
1. Verbal and physical
2. Anxiety; defensive; Acting-out Person; Tension Reduction
3. False
4. Supportive
5. Directive
6. Nonviolent Physical Crisis Intervention
7. Therapeutic Rapport
8. Only when the individual begins to lose rationality.
9. False
10. Place your body at an angle to the person and keep a distance of at least a leg length.
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 21
Appendix D: Crisis Prevention and Intervention Behavior Survey
Instructions: Please answer each question below regarding your behavior since taking the CPI
training. Please note that only a summary of statistical information will be presented, and that
your comments will be anonymous.
Student Registration Number: _________________________________________________
Type of business (e.g. educational facility): ______________________________________
Your Profession (e.g. elementary teacher): ______________________________________
Your Role: Support Staff Professional / Certified Management / Executive
Do you have prior experience with crisis prevention training before this course?
None Some Extensive
1. How confident are you in your ability to correctly use the conflict resolution techniques
learned in the CPI course?
a. Not at all
b. Somewhat unconfident
c. Neutral
d. Somewhat confident
e. Extremely confident
2. Have you had an opportunity to use your CPI training since attending the course?
a. No, I have not had the opportunity (go to question 12)
b. No, I did not feel confident enough when the opportunity arose (go to question 12)
c. Yes, seldom
d. Yes, frequently
3. If you have used the CPI training, did you feel that the training prepared you to handle the
situation confidently and safely?
a. No
b. Yes
4. If you have used the CPI training, did you consciously consider your position relative to the
acting-out person and take a supportive stance?
a. No
b. Yes
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 22
5. If you have used the CPI training, did you use paraverbal communication to consciously
control the tone, volume, and cadence of your voice to create a calm, non-threatening
atmosphere?
a. No
b. Yes
6. If you have used the CPI training with a person in the defensive stage, did you use the
directive approach to attempt to take control of the situation by setting limits?
a. No
b. Yes
7. If you have used the CPI training, did you attempt to establish a therapeutic rapport with the
acting-out person?
a. No
b. Yes
8. If you have used the CPI training, was the situation effectively resolved without injury?
a. No
b. Yes
9. If you have used the CPI training, was the situation effectively resolved without using
seclusion or restraint?
a. No
b. Yes
10. If you have used seclusion or restraint, were you able to remember and employ the personal
safety techniques?
a. No
b. Yes
11. Overall, among your co-workers who have taken the CPI training, do you feel more
confident in the ability of the group to work together in partnership using the CPI techniques
for conflict resolution?
a. No
b. Yes
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 23
12. Please share a description of an incident in which crisis intervention was required, and your
specific actions in that situation. Please omit names and identifying information.
__________________________________________________________________________
__________________________________________________________________________
__________________________________________________________________________
Thank you for sharing!
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 24
Appendix E: Crisis Prevention and Intervention Facility Survey
Instructions: Please answer each question below regarding the number and types of incidents in
your facility over the past 6 months, and the ways in which those incidents were handled. Please
note that only a summary of statistical information will be presented, and that your comments
will be anonymous.
Facility Registration Number: _________________________________________________
Type of business (e.g. educational facility): ______________________________________
Does your staff have prior experience with crisis prevention training before this course?
a. None
b. Fewer than 25% have prior training
c. Fewer than 50% have prior training
d. More than 50% have prior training
e. Most or all have prior training
What percentage of your current staff have taken the CPI training?
a. None
b. Fewer than 25% have prior training
c. Fewer than 50% have prior training
d. More than 50% have prior training
e. Most or all have prior training
1. During the evaluation period, how many incidents have occurred in your facility requiring
crisis intervention? ____________
2. During the evaluation period, how many incidents have occurred in your facility in which
injuries were sustained by staff or students? ____________
3. During the evaluation period, how many incidents have occurred in your facility in which a
student needed to be restrained? ____________
4. During the evaluation period, how many incidents have occurred in your facility in which a
student needed to be secluded? ____________
5. During the evaluation period, how many incidents have occurred in your facility in which the
state recommendations for restraint and seclusion were not followed? ____________
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 25
6. During the evaluation period, how many incidents have occurred in your facility in which the
school policy for restraint and seclusion were not followed? ____________
7. Please note any intangible benefits that you have seen as a result of this training (e.g. greater
cooperation among staff members).
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 26
Appendix F: Crisis Prevention and Intervention Project Plan and Timeline (Gantt Chart)
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 27
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 28
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 29
EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 30
Task Name Duration Start Finish Discovery and Planning 3 09/08/14 09/10/14
Identify training goals 1 09/08/14 09/08/14
Present training overview 1 09/09/14 09/09/14
Get buy-in / sign agreement 1 09/10/14 09/10/14
Perform Learner Analysis 2 09/11/14 09/12/14
Preliminary analysis of trainees 1 09/11/14 09/11/14
Group trainees 2 09/11/14 09/12/14
Evaluation Preparation 4 09/15/14 09/18/14
CPI Learner Survey 1 09/15/14 09/15/14
Assess and Customize 1 09/15/14 09/15/14
CPI Post-test 1 09/16/14 09/16/14
Assess and Customize 1 09/16/14 09/16/14
CPI Behavior Survey 1 09/17/14 09/17/14
Assess and Customize 1 09/17/14 09/17/14
CPI Facility Survey 1 09/18/14 09/18/14
Assess and Customize 1 09/18/14 09/18/14
Implement hybrid training 26 09/19/14 10/24/14
Online course available 25 09/19/14 10/23/14
In-house training 1 10/24/14 10/24/14
Evaluation Implementation 262 10/24/14 10/26/15
CPI Learner Survey (level 1) 0 10/24/14 10/24/14
CPI Post-test (level 2) 0 10/24/14 10/24/14
CPI Behavior Survey (level 3) 0 04/27/15 04/27/15
CPI Facility Survey (level 4) 0 04/27/15 04/27/15
CPI Behavior Survey (level 3) 0 10/26/15 10/26/15
CPI Facility Survey (level 4) 0 10/26/15 10/26/15
Data Analysis 276 10/27/14 11/16/15
CPI Learner Survey (level 1) 5 10/27/14 10/31/14
CPI Post-test (level 2) 5 11/03/14 11/07/14
CPI Behavior Survey (level 3) 5 05/12/15 05/18/15
CPI Facility Survey (level 4) 5 05/12/15 05/18/15
CPI Behavior Survey (level 3) 5 11/10/15 11/16/15
CPI Facility Survey (level 4) 5 11/10/15 11/16/15
Feedback / Presentation 272 12/08/14 12/22/15
Prepare initial report 5 12/08/14 12/12/14
Present results 1 12/15/14 12/15/14
Prepare six month assessment 5 06/16/15 06/22/15
Present six month results 1 06/23/15 06/23/15
Prepare 12 month assessment 5 12/15/15 12/21/15
Present 12 month results 1 12/22/15 12/22/15