Evaluation of Interprofessional Competency (Practice ... · PDF fileEvaluation of...
-
Upload
vuongthuan -
Category
Documents
-
view
226 -
download
0
Transcript of Evaluation of Interprofessional Competency (Practice ... · PDF fileEvaluation of...
Evaluation of Interprofessional Competency (Practice) tools for Internationally Educated Health Professionals
May 06, 2016
Prepared by:
Mubashir Arain, Shelanne Hepp, Esther Suter*, Shyama Nanayakkara, Siegrid Deutschlander*, Liz
Harrison*, Grace Mickelson*, Lesley Bainbridge* and Ruby Grymonpre*
*Western Canadian Interprofessional Health Collaborative (WCIHC) core team members
Copyright © 2016. Province of British Columbia. All rights reserved. This material is owned by the Government of British Columbia and protected by copyright law. It may not be reproduced or redistributed without the prior written permission of the Province of British Columbia. Production of this document has been undertaken with the support and collaboration of the Western and Northern Health Human Resources Planning Forum and its member jurisdictions, and has been made possible through a financial contribution from Health Canada. Reference to the material contained in this document should be accompanied by appropriate acknowledgement to the source document and its sponsors. The views expressed herein do not necessarily represent the views of Health Canada. Please cite this report as Arain M, Hepp S, Suter E, Deutschlander S, Harrison L, Mickelson G, Bainbridge L, Nanayakkara S, and Grymonpre R. 2016. Evaluation of Interprofessional Competency (Practice) tools for Internationally Educated Health Professionals. This report was commissioned by the Government of British Columbia with a grant from Health Canada.
i | P a g e
Contents ACRONYMS ................................................................................................................................................... ii SUMMARY ..................................................................................................................................................... 1 BACKGROUND ............................................................................................................................................... 3 METHODOLOGY ............................................................................................................................................ 3 RESULTS ........................................................................................................................................................ 5
1. Knowledge improvement of the participants 8
Pre and Post assessments ..................................................................................................................... 8 Reflections ........................................................................................................................................... 12
2. Feedback on the modules 14
INTRODUCTORY MODULE................................................................................................................... 16 COMPETENCY-BASED MODULES ........................................................................................................ 17
Interprofessional Conflict Management......................................................................................... 17
Team Functioning ............................................................................................................................ 19
Interprofessional Communication .................................................................................................. 20
Patient-Centred Care ....................................................................................................................... 21
Collaborative Leadership ................................................................................................................ 22
Role Clarification ............................................................................................................................. 23
TIME TAKEN TO REVIEW ICT FOR IEHPS ............................................................................................. 24 LANGUAGE .......................................................................................................................................... 25 CULTURAL ASPECTS ............................................................................................................................. 25 SCENARIOS .......................................................................................................................................... 26 ASSESSMENTS ..................................................................................................................................... 27 LICENSING EXAM ................................................................................................................................. 28 TECHNICAL ISSUES ............................................................................................................................... 28
CONCLUSIONS ............................................................................................................................................. 29 LIMITATIONS ............................................................................................................................................... 29 RECOMMENDATIONS ................................................................................................................................. 30 APPENDIX I. IEHP PRE AND POST ASSESSMENT TOOL ................................................................................ 32 APPENDIX II. SURVEY FOR IEHPS ................................................................................................................. 35 APPENDIX III. INTERVIEW GUIDES ............................................................................................................... 46 APPENDIX IV: REFLECTIVE QUESTIONS ....................................................................................................... 48 APPENDIX V: ETHICS EXEMPTION FROM UNIVERSITY OF SASKATCHEWAN .............................................. 49
ii | P a g e
ACRONYMS LPN = Licensed Practical Nurse RN = Registered Nurse RPN = Registered Psychiatric Nurse OT = Occupational Therapist PT = Physical Therapist ICT = Interprofessional (Practice) Competency Toolkit IEHP = Internationally Educated Health Professional IP = Interprofessional IMG= International Medical Graduates OSCEs – Objective Structured Clinical Skills Examinations CIHC– Canadian Interprofessional Health Collaborative
1 | P a g e
SUMMARY
Purpose
The Interprofessional Competency (Practice) Toolkit for Internationally Educated Health
Professionals (ICT for IEHPs) consists of several collaborative training modules designed specially for
IEHPs. The WCIHC study team has a diverse experience and expertise to direct and oversee curriculum
development for health care providers trained outside of Canada (WCIHC website:
http://www.inspirenet.ca/resources/western-canadian-interprofessional-health-collaborative-wcihc). We
pilot tested the modules with a cohort of IEHPs and their educators to evaluate whether the these tools
are useful and appropriate for leaners who were trained outside of Canada. We tested the online format
and any knowledge and attitude changes among IEHPs toward IP collaboration. Their feedback will be
used to improve the quality and relevance of the materials.
Methods
We conducted surveys and follow-up interviews with IEHPs and educators from four provinces
(British Columbia, Alberta, Saskatchewan, and Manitoba). We aimed to include seven different
healthcare professionals: Licensed Practical Nurses (LPN), Registered Nurses (RN), Registered Psychiatric
Nurses (RPN), Pharmacists, Physicians, Occupational Therapists (OT), and Physical Therapists (PT). We
also used a pre-post assessment tool and reflective questions to measure the effectiveness of the ICT for
IEHPs.
Findings
Thirty IEHPs and five educators reviewed the ICT for IEHPs modules and provided their feedback
in this evaluation.
Two educators were from Alberta, two from Manitoba and one from Saskatchewan. IEHP
participants were 48% from British Columbia, 33% from Alberta, 10% from Manitoba, and 7% from
Saskatchewan. The majority of IEHPs were RNs (36%), followed by pharmacists (27%); other IEHPs were
LPNs (13%), OTs (10%), PTs (7%), and Physicians (7%). At the time of the survey, 60% of participants
(n=18) were preparing for their professional licensing exam and 40% (n=12) had completed their
licensure requirements (post-licensure).
We compared the scores of the participants who completed assessment before and after
reviewing the competency-based modules using Wilcoxon signed-rank test for pairwise comparison.
Pre- versus post- intervention confidence scores for all items under the role clarification and patient-
centred care domains increased significantly (p value <0.01). The scores increased for most questions in
2 | P a g e
Team Functioning, Collaborative Leadership, and Communication domains. In contrast, the post
assessments score only increased for half of the questions in Conflict Management domains.
Most of the participants found the content of the competency-based mini modules very useful.
IEHPs also found the introductory module a good overview of IP collaboration. However, educators
found the introductory module less useful than the competency-based modules. Both IEHPs and
educators reported that the introductory module was lengthy and suggested moving some of its content
to the competency-based mini modules.
The ICT for IEHPs covered a range of cultural aspects in the modules and most IEHPs were able
to relate to the scenarios presented. Pharmacy IEHPs recommended adding some scenarios of IP
collaboration in a community setting.
Participants found the pre and post assessments useful for measuring changes in their
confidence levels for each IP competency after reviewing the ICT for IEHPs modules. Participants
suggested adding more multiple choice style questions.
Nine participants (7 IEHPs and two educators) reported one or more technical issues with
accessing the modules online.
Conclusions
Overall, the ICT for IEHPs is a very useful resource and these modules significantly improve the
knowledge of IEHPs about IP collaborative practice. Some minor amendments are required before
launching the final ICT for IEHPs modules.
Key recommendations
Based on the evaluation, we recommend the following:
Reduce the length of the introductory module by moving some of its content to the
competency-based mini modules
Add more objective assessment methods such as multiple choice style questions
Address all technical issues before the final launch of ICT for IEHPs
3 | P a g e
BACKGROUND The ICT for IEHPs is a learning resource for building collaborative practice competencies among IEHPs. In
collaboration with the project curriculum working group, the curriculum designer Fluid Learning Design
was contracted to develop the modules for this project. The module content is based on the Canadian
Interprofessional Health Collaborative (CIHC) National Competency Framework (1). Fluid Learning
Design has previously developed a similar set of modules (IPC on the RUN) for Canadian healthcare
professionals (2).
Although the project targeted seven different health professions for logistical reasons, the content of
the content of ICT for IEHPs is not specific to any particular healthcare discipline. IEHPs from any
healthcare discipline can benefit from these modules. The ICT for IEHPs comprise an introductory
module followed by six competency-based mini modules; each mini module speaks to one of the six
competency domains: conflict management, team functioning, interprofessional communication,
patient-centred care, collaborative leadership, and role clarification.
This evaluation aimed to ascertain
1) What aspects of the ICT for IEHPs (e.g. format, content, language, appropriateness, and
cultural aspects) do IEHPs and educators find most helpful?
2) What do they find least helpful? And
3) Did the ICT for IEHPs improve knowledge and attitudes towards IP collaboration?
We will use the feedback from educators and IEHPs to improve the quality and relevance of the
modules.
METHODOLOGY
We conducted this evaluation from March 2016 to May 2016 in the four Western Canadian Provinces
(Alberta, Manitoba, Saskatchewan, and British Columbia). We aimed to test the ICT for IEHPs modules
with IEHPs from seven health professions: OTs, PTs, RNs, RPNs, LPNs, pharmacists, and physicians. Our
aim was to recruit seven IEHPs and two educators from each province (36 total). We defined ‘educator’
as an individual who is responsible for educating and assessing IEHPs in preparation for the IEHPs
profession specific licensing exam. The WCIHC core team approached stakeholders to connect with
educators (bridging program instructors) and asked to assist with IEHP recruitment. We used our
‘Stakeholder Directory’ to identify regulators and educators who could approach potential participants
and send them an invitation to participant in this evaluation. The ‘Stakeholder Directory’ is a list of
4 | P a g e
relevant stakeholders from each jurisdiction and was developed specifically for this project through an
environmental scan, a web search, and word-of-mouth recommendations.
Both IEHPs and educators were asked to complete the following activities:
• Review the ICTs for IEHP modules and complete the assessment toolkit (pre- and post-
assessment questionnaire; Appendix I)
• Complete a survey (Appendix II) about the appropriateness and relevance of the content for
IEHPs
• Participate in a follow-up telephone interview (interview guide; Appendix III)
We developed a survey tool and assessment questionnaires for this study. For the face validity of the
tools, we circulated it to all core team members and received feedback.
One research technician and two research consultants approached the participants to complete
questionnaires and conduct the follow-up interviews across four provinces. We also offered gift cards of
a nominal amount to IEHP participants only. IEHP participants who successfully completed all three
evaluation components (reviewing the module, complete the survey and participate in the follow-up
interview) received a $50 gift card. Participants only reviewing the module and completing the survey
received $25 gift card.
We also asked IEHPs to send their reflections on each module to get a better sense whether the content
improved their understanding about interprofessional collaboration (Appendix IV). The introductory
module had eight reflective questions including three multiple choice questions. All competency-based
modules had two reflective questions: one in the beginning after a scenario to get reflections on the
scenario and a second question near the end of the module to get reflections on the overall module
content. The Patient-Centred Care module had an additional question asking IEHPs to reflect on their
healthcare practice back home.
The pre- and post-assessment questionnaire had the same set of questions that measured confidence
level on a Likert scale from ’not confident”, “somewhat confident”, “confident”, to “very confident”. We
scored these confidence levels from 1-4. We compared the average pre-assessment scores with post-
assessment scores separately for each question within the six competency domains. We embedded the
pre-assessment questionnaire in the introductory module while the post-assessment questionnaire was
embedded after the last competency-based mini module. We also graphically compared the percentage
of participants who reported ‘strongly agree’ in pre- and post-assessments.
In the questionnaire, we collected responses on the format and content of the ICTs for IEHPs, time spent
to review each module, cultural appropriateness, and language. We also collected demographic
5 | P a g e
information of IEHPs such as age, gender, country of origin, professional background, and the province
of residence in Canada. In the follow-up interviews, we asked questions about the usefulness of the
content, method of delivery, and relevance to IEHPs.
We analyzed the questionnaires and pre-post assessments in SPSS using descriptive and inferential
statistics. We reported median with inter-quartile range (IQR) where data had extreme values. For the
Likert scale questions in the survey, we dichotomized responses into ‘agree’ or ‘strongly agree’ versus all
other responses. Due to the small sample, we used non-parametric statistical tests for comparing
module scores between educators and IEHPs (Fisher's Exact Test) and pre-post assessments (Wilcoxon
signed-rank test). We performed content analysis for interviews and open-ended responses in the
survey.
We obtained ethical approval for the study from the Research Ethics Board, University of Manitoba and
the Conjoint Health Research Ethics Board, University of Calgary. Ethics boards from University of
British Columbia and University of Saskatchewan both considered this project an evaluation and waived
the need for HREB approval (Appendix V).
RESULTS
Thirty out of 55 IEHPs (response rate 55%) and five out of 13 educators completed the questionnaire
(response rate 38%). Nineteen IEHPs and three educators also participated in the follow-up interviews.
Of the IEHP participants, 18 provided their reflections and 16 also completed pre and post-assessment
questionnaires.
Demographics of educators
The five educators resided in Alberta (n= 2), Saskatchewan (n=1), and Manitoba (n=2); two of them were
involved in teaching internationally educated physicians, the others were associated with programs for
internationally educated pharmacists (n=1), nurses (n=1), and physical therapist (n=1). The age of
educators ranged from 51 to 58 years. All educators were female.
Demographics of IEHPs
The age of the IEHPs ranged from 23 years to 48 years (Figure 1). Most participants (63%) were female
(n=19). Of the total sample, 50% participants were from British Columbia (n=15), 33% from Alberta
(n=10), 10% from Manitoba (n=3), and 7% from Saskatchewan (n=2) (Figure 2).
6 | P a g e
Figure 1: Age distribution of the IEHP survey participants (n=30)
Figure 2: Provinces of the IEHPs (n=30)
Most IEHP survey respondents (n=10) received their professional degree from the Philippines
(34%). Seventeen percent (n=5) of participants received their professional degree from Nigeria. Other
common countries were Egypt, India, and South Africa, with 10% each (n=3). The remaining countries
were distributed equally at 3% (n=1) including Iran, Australia, Singapore, Taiwan, USA, and South Africa
(Figure 3).
0
5
10
15
20-25 26-30 31-35 36-40 41-45 46-50
Age in years
Age Distribution
7 | P a g e
Figure 3: Home countries of the IEHPs where they completed their professional degrees (n=30)
The majority of the participants were RNs (36%; n=11) and pharmacists (27%; n=8); the others were
LPNs (13%; n=4), OTs (10%; n=3), PTs (7%; n=2), and Physicians (7%; n=2) (Figure 4). We did not have any
representation from RPNs.
Figure 4: Healthcare profession of the IEHPs (n=30)
8 | P a g e
At the time of the survey, 60% of participants (n=18) were preparing for their professional licensing
exam and 40% (n=12) had completed their licensure requirements (post-licensure). The majority (67%)
had attended a bridging program. Of the 12 participants who completed their licensure, only six had
started practicing in Canada: three had two years of experience and the other three had one year of
experience. The number of years participants practised in their home country varied from none to 18
years (mean=6.47; SD=4.68) (Figure 5).
Figure 5: Experience of healthcare professions in home country (n=30)
MODULE EVALUATION
The evaluation of ICT for IEHPs modules provided information on two main aspects:
1. Usefulness of these modules for improving knowledge and understanding of IEHPs about
interprofessional practice (knowledge improvement)
2. Improvement of the quality and relevance of modules to IEHPs (feedback on the content)
We only measured the knowledge improvement of IEHPs. We obtained feedback on modules from both
IEHPs and educators.
1. Knowledge improvement of the participants
We measured the knowledge and understanding of the IEHPs with a set of pre- and post-assessments
and reflective questions embedded within the ICT for IEHPs modules.
Pre- and post-assessments
Tables 1a and 1b show the scores of the participants who completed assessments before and after
reviewing the competency-based modules for the six interprofessional competency domains.
The confidence scores increased significantly for all questions in Role Clarification and Patient-Centred
Care domains after reviewing the module. The scores increased for most questions in Team Functioning,
1
13
7
9
0
2
4
6
8
10
12
14
No experience <1-4 years >5-9 years >10 or more years
9 | P a g e
Collaborative Leadership, and Communication domains [Table 1a & 1b]. In contrast, the post
assessments score only increased for half of the questions in Conflict Management domains; there was
no significant improvement in the confidence scores for other questions in this domain. This may
indicate that more intense education and coaching may be required to increase IEHP’s confidence in
their conflict management skills.
We also graphically compared the percentage of participants who reported their ratings as ‘very
confident’ in the pre and post assessments (Figure 6). We averaged the number of participants providing
ratings across survey items within each domain and calculated the percentage of people providing
different ratings. Each bar represented the average percentage of people providing the ratings within
each domain. There was a shift towards higher confidence level after reviewing ICT for IEHPs.
10 | P a g e
Table 1a Pre- and post-assessment scores of participants (n=16); Likert scale (1=Not Confident to 4=Very Confident)
Questions Before Mean (SD)
After Mean (SD)
p-value
Role Clarification 1. Described own role and that of others 2. Recognized and respected the diversity of other health and social care roles, responsibilities, and competencies 3. Performed own roles in a culturally respectful way 4. Communicated roles, knowledge, skills, and attitudes using appropriate language 5. Considered the roles of others in determining own professional and interprofessional roles 6. Accessed others’ skills and knowledge appropriately through consultation 7. Integrated competencies/roles seamlessly into models of service delivery
2.87 (0.9) 3.00 (0.7)
2.88 (1.0) 3.00 (0.9) 2.63 (0.8) 2.50 (0.9) 2.63 (1.0)
3.47 (0.5) 3.81 (0.4)
3.50 (0.6) 3.53 (0.6) 3.63 (0.5) 3.31 (0.7) 3.31 (0.6)
0.02*
<0.01*
0.01* 0.02*
<0.01* 0.02*
<0.01*
Patient-Centred Care 1. Supported the participation of patients/clients, their families 2. Shared information with patients/clients in a respectful manner 3. Ensured that appropriate education and support is provided to patients/family members 4. Listened respectively to the expressed needs of all parties in delivering care
2.94 (0.9) 2.94 (0.9) 2.88 (1.0) 2.80 (0.9)
3.81 (0.4) 3.69 (0.6) 3.56 (0.5) 3.80 (0.4)
0.01*
<0.01* 0.02*
<0.01*
Team Functioning 1. Understood the process of team development 2. Developed principles for working together 3. Effectively facilitated discussions and interactions among team members 4. Participate, and be respectful of all members in collaborative decision-making 5. Regularly reflected on functioning with team learners/practitioners 6. Established and maintained effective and healthy working relationships with learners/practitioners, patients/clients, and families 7. Respected team ethics
2.47 (0.9) 2.73 (0.8) 2.47 (1.0) 3.00 (0.9) 2.47 (0.9) 2.79 (0.9)
3.20 (1.0)
3.27 (0.5) 3.20 (0.8) 3.33 (0.6) 3.67 (0.4) 3.47 (0.5) 3.50 (0.5)
3.67 (0.4)
<0.01*
0.14 0.01* 0.03*
<0.01* 0.05
0.08
* Significant differences at 95% confidence level; p values were calculated using Wilcoxon signed-rank test for pairwise comparison
11 | P a g e
Table 1b Pre- and post-assessment scores of participants (n=16); Likert scale (1=Not Confident to 4=Very Confident)
Questions Before Mean (SD)
After Mean (SD)
p-value
Collaborative Leadership 1.Worked with others to enable effective patient/client outcomes 2. Advanced interdependent working relationships among all participants 3. Facilitated effective team processes 4. Facilitated effective decision-making 5. Established a climate for collaborative practice among all participants 6. Co-created a climate for shared leadership and collaborative practice 7. Applied collaborative decision-making principles 8. Integrated the principles of continuous quality improvement to work processes
2.80 (0.6) 2.40 (0.9) 2.13 (0.9) 2.33 (0.9) 2.47 (1.0) 2.40 (1.1) 2.47 (0.9) 2.60 (0.9)
3.40 (0.5) 3.07 (0.7) 3.33 (0.6) 3.33 (0.6) 3.40 (0.6) 3.47 (0.5) 3.47 (0.5) 3.40 (0.6)
<0.01*
0.06 <0.01* <0.01* 0.01* 0.01*
<0.01* 0.02*
Interprofessional Communication 1. Established team work communication principles 2. Actively listened to other team members including patients/clients/families 3. Communicated to ensure common understanding of care decisions 4. Developed trusting relationships with patients/families and other team members 5. Effectively used information and communication technology
2.47 (0.9) 3.13 (0.8) 2.67 (0.9) 3.27 (0.7) 2.67 (0.8)
3.27 (0.7) 3.87 (0.3) 3.53 (0.5) 3.53 (0.5) 3.33 (0.6)
0.01*
<0.01* <0.01*
0.23 0.02*
Interprofessional Conflict Management 1. Valued the potential positive nature of conflict 2. Recognized the potential for conflict to occur; taking constructive steps to address 3. Identified common situations that are likely to lead to disagreements or conflicts 4. Knew and understood strategies to deal with conflict 5. Set guidelines for addressing disagreements 6. Effectively worked to address and resolve disagreements 7. Established a safe environment in which to express diverse opinions 8. Developed a level of consensus among those with differing views; allowing all members to feel their viewpoints have been heard no matter what the outcome
2.62 (1.0) 2.77 (1.0) 2.67 (1.0) 2.46 (1.1) 2.46 (1.1 2.69 (1.0) 2.83 (1.1) 2.46 (1.0)
3.31 (0.7) 3.38 (0.6) 3.42 (0.6) 3.31 (0.6) 3.23 (0.7) 3.46 (0.6) 3.42 (0.7) 3.31 (0.6)
0.03* 0.07 0.06 0.06
0.02* 0.03* 0.16
<0.01*
* Significant differences at 95% confidence level; p values were calculated using Wilcoxon signed-rank test for pairwise comparison
12 | P a g e
Figure 6: Pre and post comparison of participants reporting “Very Confident” in each IP competency domain
Reflections
Reflections of IEHPs showed the improvement in the understanding of IEHPs about IP collaboration in all
six competency domains.
Introductory Module
In the introductory module, IEHPs expressed their understanding of effective team meetings, sharing
common goals, effective communication between healthcare professionals, respecting other team
members, negotiating responsibilities, and providing the best approach to patient care.
My understanding of interprofessional collaborative practice has changed a bit in terms of having additional knowledge of how it works. Interprofessional collaborative practice is working together with a team of health care professionals, including the patient and
0%
10%
20%
30%
40%
50%
60%
70%
80%
Average percentage of respondents reporting 'Very Confident'
Pre
Post
13 | P a g e
his or her family to create a common goal and having an effective communication in order to achieve them.
Interprofessional Conflict Management
IEHPs reflected on factors contributing to conflicts and how a conflict could be constructive or
destructive. The constructive conflict was described as using effective conflict management to build
strong teams, listening to all team members and coming to a compromise, sharing the leadership role,
and respecting team members opinions prior to making final decisions. The destructive conflict was
described as trying to win an argument instead of reaching consensus and lack of understanding among
team members. Some IEHPs identified strategies on how they would deal with conflict after reviewing
this module such as being understanding, compromising, providing alternative solutions, listening,
sharing leadership, and focusing on best patient care.
Now I understood that not all conflicts are destructive and conflicts can be beneficial towards a patient centred goal if we know about the role of other professionals and respect other members’ opinion.
Collaborative Leadership
IEHPs had two main reflections at the end of this module. First, IEHPs reported their understanding that
patients and families are the experts and leaders in decision making about care. Second, participants
reported that team leadership should be based on who has the most knowledge of the patient and
advocate more for the patient.
I will be more comfortable to share my knowledge because I know that it will be respected by my colleagues. I will also keep in mind that I need to give a chance for other team members to express and share their expertise and come up with a better treatment goal for the patient.
Interprofessional Communication
Participants reflected on how this module improved their understanding of communicating with other
healthcare professionals by recognizing communication styles, recognizing cultural and professional
diversity, and understanding needs and preferences of others.
Through this module I realize that I am from collectivist high context and multicultural communicative culture and now fully aware of intercultural communication challenges.
Patient-Centred Care
Participants reflected on the scenario in the module and provided their thoughts about the meaning of
patient-centred care. The module helped participants realize that other health care professionals,
patients, and families should collaborate to find common goals.
The values and principles (described in this module) will have a great impact as every client is different and the health practitioner should know the client and their family that is involved with the care.
14 | P a g e
Team Functioning
Participants reflected on their understanding that how teams of different healthcare professionals
function effectively.
Understanding how the interprofessional collaborative team functions in a different context will help me become a better part of teams. In this way, I will be able to clarify how they communicate and work given the differences in our professions.
2. Feedback on the modules
We collected feedback on ICT for IEHPs through surveys and interviews. The survey had a similar set of
questions for each module including some Likert scale questions. We compared scores of Likert scale
questions between the modules and also between educators and IEHPs.
The high average scores of Likert scale questions for each of the ICT for IEHPs modules indicated
that the participants strongly agreed with language, the flow of the modules, assessment tools, and
reflective questions in the modules. The comparison of the scores of each Likert scale question between
all ICT for IEHPs modules showed no significant difference between these modules [Table 2]. We
analyzed their verbal and written feedback to generate key themes and subthemes. We categorized
participants by their home country and profession.
15 | P a g e
Table 2 Comparing Likert scores (1=Strongly Disagree to 5=Strongly Agree) among all ICT for IEHPs modules for each question (n=35)
Modules
Introductory Conflict Management
Team Functioning
Interprofessional Communication
Patient-Centred care
Collaborative Leadership
Role Clarification
P values
Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD)
Language was appropriate
Easy to follow Assessment tools
Improved IP understanding
Improved IP knowledge
Improved IP attitudes
4.56 (0.78)
4.52 (0.66) 4.31 (0.75) 4.26 (0.82)
4.40 (0.77)
4.40 (0.84)
4.52 (0.66)
4.64 (0.48) 4.30 (0.63) 4.27 (0.76)
4.55 (0.56)
4.52 (0.61)
4.52 (0.56)
4.55 (0.56) 4.31 (0.64) 4.33 (0.73)
4.48 (0.50)
4.55 (0.56)
4.55 (0.56)
4.58 (0.50) 4.33 (0.64) 4.33 (0.81)
4.56 (0.56)
4.55 (0.61)
4.58 (0.50)
4.58 (0.50) 4.41 (0.56) 4.30 (0.81)
4.52 (0.66)
4.48 (0.71)
4.55 (0.50)
4.61 (0.55) 4.33 (0.64) 4.34 (0.82)
4.45 (0.71)
4.39 (0.74)
4.55 (0.50)
4.59 (0.49) 4.41 (0.61) 4.45 (0.61)
4.58 (0.50)
4.55 (0.56)
0.99
0.98 0.97 0.96
0.88
0.88
* p values were calculated using one-way ANOVA
16 | P a g e
INTRODUCTORY MODULE We found variations in responses from educators and IEHPs on the content in the introductory module.
IEHPs were significantly more likely to agree that the language in the module was appropriate than
educators. IEHPs were also significantly more likely to agree that the introductory module improved
their understanding and attitude towards IP practice than the educators [Table 3].
IEHPs reported that the introductory module was very important because it provided the context of the
competency modules. The introduction helped IEHPs to understand why collaborative practice is
important and anticipate learning expectations in the competency-based modules.
I liked the introduction provided baseline knowledge and know what to expect from the other module. It was a good overview. (LPN, Nigeria)
I think I felt that the intro module set a good context and touched on many of the things that were brought up in the mini modules. The ideas that are presented. If I just did the mini modules I wouldn’t have understood why this is important and how to apply in the workplace. (USA, RN)
Educators found the scenario slightly confusing and suggested more clarity to demonstrate
interprofessional aspects.
I am not sure all people would naturally make the case or relationship in the presented case of Esther story. It is a case with a poor outcome. If you were going to use the case, there should be a bit more about Esther and what went wrong. We didn’t know what went wrong. I think it is an ok example, but went off into patient safety and not necessarily IP collaboration (Educator, PT Program)
Both IEHPs and educators perceived that the introductory module was lengthy and could be shortened
to allow more time on the competencies.
The first module was the introduction – it went too long and could be more precise. I think the length should be the same, but I think you should dedicate more time to the six modules. (Pharmacist, Egypt)
The first introductory module was too long. Overall, the introduction is too long and I lost the direction of where that was going or what you are trying to do. I am afraid that you might lose people in the introduction and do not go on to the modules. I thought the modules were better learning than the initial introduction. So if that could be shorter and not lose people and put more context in the modules instead of them just listening would be better. (Educator, IMG Program)
17 | P a g e
Table 3. Comparing responses between IEHPs and educators to Likert scale questions in the introductory module
Modules
Strongly Disagree
Disagree Neither Agree nor Disagree
Agree Strongly Agree
Difference
% (n) % (n) % (n) % (n) % (n) p-values
Language was appropriate IEHP
Educators
Easy to follow IEHP
Educators
Assessment tools were relevant IEHP
Educators
Understanding my scores IEHP
Educators
Improved my IP knowledge IEHP
Educators
Improved my IP attitudes IEHP
Educators
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0)
40 (2)
0 (0) 20.0 (1)
3.3 (1) 0 (0)
3.4 (1) 0 (0)
3.3 (1) 0 (0)
0 (0) 20.0 (1)
0 (0) 0 (0)
0 (0) 0 (0)
6.7 (2) 20.0 (1)
6.9 (2) 60.0 (3)
6.7 (2) 20.0 (1)
10.0 (3) 40.0 (2)
27.6 (8) 20.0 (1)
39.3 (11) 40.0 (2)
43.3 (13) 40.0 (2)
37.9 (11) 20.0 (1)
33.3 (10) 40.0 (2)
26.7 (8) 0 (0)
72.4 (21) 40.0 (2)
60.7 (17) 40.0 (2)
46.7 (14) 40.0 (2)
51.7 (15) 20.0 (1)
56.7 (17) 40.0 (2)
63.3 (19) 40.0 (2)
0.01*
0.15
0.47
0.03*
0.47
0.02*
* Significant differences at 95% confidence level; p values were calculated using Fisher's Exact Test; NA where all participants agreed or strongly agreed
COMPETENCY-BASED MODULES Participants found the competency-based mini-modules very useful to learn the overview of each IP
competency. However, participants reported that some information was repetitive from the first
module.
In the mini-modules, some of the content there is already shown in the introductory one. (RN, Philippines)
We have summarized participants’ feedback on each competency-based module.
Interprofessional Conflict Management Most of the participants strongly agreed or agreed with most of the questions about the usefulness of
the modules without any significant difference between educators and IEHPs [Table 4].
18 | P a g e
Many IEHPs identified that managing and dealing with conflicts is important when working in a team-
based environment. Health care team members must trust, respect, and listen to input from each other
and resolve conflicts constructively. The module provided adequate knowledge on resolving and
avoiding conflicts among team members.
I learned how to resolve the conflict by listening to them and collectively resolving. (PT, Nigeria)
Overall, educators also agreed that the module content was very useful. One educator suggested adding
videos to test IEHPs’ understanding.
There could be more videos and within the videos the students were asked to comment on the situation. There could be some things on negative or positive conflict. Maybe add a module that has more videos of real encounters and scenarios and questions like what kind of conflict was the nurse portraying here? (Educator, PT Program)
Table 4. Comparing responses between IEHPs and educators to Likert scale questions in conflict management module
Modules
Strongly Disagree
Disagree Neither Agree nor Disagree
Agree Strongly Agree
Difference
% (n) % (n) % (n) % (n) % (n) p-values
Language was appropriate IEHP
Educators
Easy to follow IEHP
Educators
Assessment tools were relevant IEHP
Educators
Understanding my scores IEHP
Educators
Improved my IP knowledge IEHP
Educators
Improved my IP attitudes IEHP
Educators
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0)
20.0 (1)
0 (0) 0 (0)
0 (0) 0 (0)
3.6 (1) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
7.1 (2) 20.0 (1)
3.6 (1) 40.0 (2)
3.6 (1) 0 (0)
3.6 (1) 20.0 (1)
39.3 (11) 40.0 (2)
32.1 (9) 60.0 (3)
53.6 (15) 40.0 (2)
46.4 (13) 40.0 (2)
35.7 (10) 60.0 (3)
35.7 (10) 40.0 (2)
60.7 (17) 40.0 (2)
67.9 (19) 40.0 (2)
39.3 (11) 40.0 (2)
46.4 (13) 20.0 (1)
60.7 (17) 40.0 (2)
60.7 (17) 40.0 (2)
0.15
NA
0.40
0.09
0.85
0.28
*p values were calculated using Fisher's Exact Test; NA where all participants agreed or strongly agreed
19 | P a g e
Team Functioning Three IEHPs were unsure whether the assessments related to the module were relevant [Table 5]. Both
IEHPs and educators strongly agreed with all other aspects of the module. IEHPs learned about the
variations in the functioning of different teams and what to consider when entering a new team.
Team functioning and role clarification were two interesting areas that were lacking or not properly addressed during my training or practice due to the different practice system in Nigeria. This is a huge opportunity to learn them. (Pharmacist, Nigeria)
Educators believed that many IEHPs worked in hierarchical health systems and transitioning to a new
system may require mentorship at the front line to help them with the process.
I think it is so different from the systems they may be used to. Many of my students worked in private health care centres in the Philippines where the physician hires and fires the RNs. So the nurses do not question the physician. It is different here where there is an expectation to speak up. (Educator, Nursing Program)
Table 5. Comparing responses between IEHPs and educators to Likert scale questions in Team Functioning module
Modules
Strongly Disagree
Disagree Neither Agree nor Disagree
Agree Strongly Agree
Difference
% (n) % (n) % (n) % (n) % (n) p-values
Language was appropriate IEHP
Educators
Easy to follow IEHP
Educators
Assessment tools were relevant IEHP
Educators
Understanding my scores IEHP
Educators
Improved my IP knowledge IEHP
Educators
Improved my IP attitudes IEHP
Educators
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
3.6 (1) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0)
20.0 (1)
0 (0) 20.0 (1)
11.1 (3) 0 (0)
3.6 (1) 20.0 (1)
0 (0) 0 (0)
0 (0) 20.0 (1)
42.9 (12) 40.0 (2)
39.3 (11) 40.0 (2)
48.1 (13) 60.0 (3)
46.4 (13) 40.0 (2)
50.0 (14) 60.0 (3)
39.3 (11) 40.0 (2)
57.1 (16) 40.0 (2)
60.7. (17) 40.0 (2)
40.7 (11) 40.0 (2)
46.4 (13) 40.0 (2)
50.0 (14) 40.0 (2)
60.7 (17) 40.0 (2)
0.15
0.15
0.59
0.40
NA
0.15
Note: p values were calculated using Fisher's Exact Test;NA where all participants agreed or strongly agreed
20 | P a g e
Interprofessional Communication IEHPs were more likely to agree that the IP communication module improved their attitude towards IP
practice than educators [Table 6]. We found no significant difference between educators and IEHPs for
the other aspects of the IP communication module.
This module helped me to understand that different people from different cultures communicate in different ways. Therefore I ought to exercise patience and respect especially when working with multi generations. It also helped to learn about SBAR and its use to unify and simplify communication. (RN, Nigeria)
Participants identified an error in this module; the low context and high context pictures in the scenario
were interchanged.
Table 6. Comparing responses between IEHPs and educators to Likert scale questions in Interprofessional Communication module
Modules
Strongly Disagree
Disagree Neither Agree nor Disagree
Agree Strongly Agree
Difference
% (n) % (n) % (n) % (n) % (n) p-values
Language was appropriate IEHP
Educators
Easy to follow IEHP
Educators
Assessment tools were relevant IEHP
Educators
Understanding my scores IEHP
Educators
Improved my IP knowledge IEHP
Educators
Improved my IP attitudes IEHP
Educators
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
3.6 (1) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
7.1 (2) 20.0 (1)
10.7 (3) 20.0 (1)
0 (0) 20.0 (1)
0 (0) 40.0 (2)
42.9 (12) 60.0 (3)
42.3 (11) 40.0 (2)
50.0 (14) 40.0 (2)
32.1 (9) 40.0 (2)
35.0 (10) 40.0 (2)
35.7 (10) 20.0 (1)
57.1 (16) 40.0 (2)
57.9 (15) 60.0 (3)
42.9 (12) 40.0 (2)
53.6 (15) 40.0 (2)
63.0 (17) 40.0 (2)
64.3 (18) 40.0 (2)
NA
NA
0.40
0.58
0.16
0.01*
* Significant differences at 95% confidence level; p values were calculated using Fisher's Exact Test; NA where all participants agreed or strongly agreed
21 | P a g e
Patient-Centred Care The module highlighted the need for collecting and sharing information among the team for a better
patient care. Participants found the module very helpful in understanding patients’ needs, such as,
including patients in setting up care goals and advocating for patients.
I liked the module on patient centred care. I came from a medical system where we make goals for patients and don’t include the patients in making goals; we just assess and provide goals. (OT, Philippines)
Table 7 shows that all participants, both IEHPs and educators, either agreed or strongly agreed that the
language and flow of the module were appropriate.
Table 7. Comparing responses between IEHPs and educators to Likert scale questions in Patient-Centred Care module
Modules
Strongly Disagree
Disagree Neither Agree nor Disagree
Agree Strongly Agree
Difference
% (n) % (n) % (n) % (n) % (n) p-values
Language was appropriate IEHP
Educators
Easy to follow IEHP
Educators
Assessment tools were relevant IEHP
Educators
Understanding my scores IEHP
Educators
Improved my IP knowledge IEHP
Educators
Improved my IP attitudes IEHP
Educators
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
3.6 (1) 0 (0)
3.6 (1) 0 (0)
3.6 (1) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
3.7 (1) 0 (0)
7.1 (2) 40.0 (2)
0 (0) 0 (0)
0 (0) 20.0 (1)
39.3 (11) 60.0 (3)
39.3 (11) 60.0 (3)
51.9 (14) 60.0 (3)
39.3 (11) 20.0 (1)
39.3 (11) 40.0 (2)
35.7 (10) 40.0 (2)
60.7 (17) 40.0 (2)
60.7 (17) 40.0 (2)
44.4 (12) 40.0 (2)
50.0 (14) 40.0 (2)
57.1 (16) 60.0 (3)
60.7 (17) 40.0 (2)
NA
NA
0.84
0.15
0.85
0.28
Note: p values were calculated using Fisher's Exact Test; NA where all participants agreed or strongly agreed
22 | P a g e
Collaborative Leadership This was the most interesting module as this was a new concept for many IEHPs. They learned that
everyone on the team could take the leadership role depending on the circumstances.
The strategies to be an effective leader were exceptionally helpful in preparation to be a potential leader of the interprofessional team. Because many times, an individual may lack the confidence or unsure how to lead the team effectively. The leader or 'expert' will change according to the present needs and goals of the patient. (OT, Singapore)
Many IEHPs reported that they worked in a hierarchical system, so it was useful to learn the concept of
shared leadership.
The most interesting part is the collaborative leadership. I found it fascinating that the true leaders are actually the patient and any health care professional who has the best relationship with the patient. This removes hierarchies between health care professions. The new aspect for me is the discussion on conflict management. (Pharmacist, Nigeria)
Physician IEHPs also found it useful to know that leadership in healthcare is not limited to physicians
only in the Canadian health care system.
The module explains that in some areas you can be a leader but in another area, other professional can take a leadership role. This module changed my mind. (Physician, Iran)
IEHPs also said that the module offered practical strategies, and this would allow them to more openly
share and communicate.
This module helped me understand and look at all health care professionals equally; thus help me become more proactive and confident in sharing what I know that will make the team goals successful. (OT, Philippines)
Both IEHPs and educators strongly agreed or agreed with all aspects of module content [Table 8].
Educators also believed that the content would help IEHPs become familiar with the Canadian
healthcare system.
This module will help to guide IEHP's in the direction of shared leadership and provide them with a better idea of what direction the Canadian Health Care system is going (Educator, Pharmacy Program)
One educator suggested presenting a difficult situation to support better learning of IEHPs.
A less confident IEHP can be shown who may have difficulty understanding local jargon or phrases, from a high context system and professional hierarchies. (Educator, IMG Program)
23 | P a g e
Table 8. Comparing responses between IEHPs and educators to Likert scale questions in Collaborative Leadership module
Modules
Strongly Disagree
Disagree Neither Agree nor Disagree
Agree Strongly Agree
Difference
% (n) % (n) % (n) % (n) % (n) p-values
Language was appropriate IEHP
Educators
Easy to follow IEHP
Educators
Assessment tools were relevant IEHP
Educators
Understanding my scores IEHP
Educators
Improved my IP knowledge IEHP
Educators
Improved my IP attitudes IEHP
Educators
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
3.6 (1) 0 (0)
3.6 (1) 0 (0)
3.6 (1) 0 (0)
0 (0) 0 (0)
0 (0) 20.0 (1)
7.1 (2) 20.0 (1)
7.1 (2) 40.0 (2)
3.6 (1) 0 (0)
3.6 (1) 20.0 (1)
42.9 (12) 60.0 (3)
32.1 (9) 40.0 (2)
50.0 (14) 40.0 (2)
33.3 (9) 20.0 (1)
39.3 (11) 40.0 (2)
39.3 (11) 40.0 (2)
57.1 (16) 40.0 (2)
67.9 (19) 40.0 (2)
42.9 (12) 40.0 (2)
55.6 (15) 40.0 (2)
53.6 (15) 60.0 (3)
53.6 (15) 40.0 (2)
NA
0.15
0.40
0.16
0.72
0.40
Note p values were calculated using Fisher's Exact Test; NA where all participants agreed or strongly agreed
Role Clarification The participants learned the importance of understanding the roles, scopes of practice, and
competencies of other professionals in the team.
I learned how to approach other health care professionals and asking their scope because others could misinterpret it when asked about their scope/responsibility. (RN, Philippines)
I will be able to better work in a team knowing my role and their role hence avoiding any duplication. (PT, India)
Educators also reported that the module was helpful for IEHPs to learn more about the role of health
care professionals working in a team.
This module will help them to understand that there are a number of people involved in patient care and hopefully encourage them to determine the roles of those they will work with (Educator, Pharmacy Program)
24 | P a g e
All IEHPs as well as educators agreed or strongly agreed with the language appropriateness and flow of
the role clarification module [Table 9]. All participants also agreed or strongly agreed that the content
improved their knowledge about IP collaborative practice. However, educators were significantly less
likely to agree that understanding assessment scores in the module would improve how an IEHP would
work with other professionals.
Table 9. Comparing responses between IEHPs and educators to Likert scale questions in Role Clarification Care module
Modules
Strongly Disagree
Disagree Neither Agree nor Disagree
Agree Strongly Agree
Difference
% (n) % (n) % (n) % (n) % (n) p-values
Language was appropriate IEHP
Educators
Easy to follow IEHP
Educators
Assessment tools were relevant IEHP
Educators
Understanding my scores IEHP
Educators
Improved my IP knowledge IEHP
Educators
Improved my IP attitudes IEHP
Educators
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
0 (0) 0 (0)
3.7 (1) 20.0 (1)
0 (0) 40.0 (2)
0 (0) 0 (0)
0 (0) 20.0 (1)
42.9 (12) 60.0 (3)
37.0 (10) 60.0 (3)
48.1 (13) 40.0 (2)
46.4 (13) 20.0 (1)
42.9 (12) 20.0 (2)
42.9 (12) 20.0 (1)
57.1 (16) 40.0 (2)
63.0 (17) 40.0 (2)
48.1 (13) 40.0 (2)
53.6 (15) 40.0 (2)
57.1 (16) 60.0 (3)
57.1 (16) 60.0 (3)
NA
NA
0.29
0.02*
NA
0.15
* Significant differences at 95% confidence level; p values were calculated using Fisher's Exact Test; NA where all participants agreed or strongly agreed
TIME TAKEN TO REVIEW ICT FOR IEHPS The median time to complete each competency-based mini-module was 15-20 minutes without any
significant difference between IEHPs and educators, which aligns with our estimates in the module
development process [Table 10]. The average time was not an appropriate measure due to some
extreme values in the data set.
25 | P a g e
The median time to complete the introductory module was 1 hour for IEHPs and 2 hours for educators,
which was longer than what we aimed for during the module development.
LANGUAGE The majority of IEHPs said that the language in the modules was fairly easy to understand.
I understand it perfectly fine. It was just right. I find I learned it faster because they were to the point. (RN, Philippines)
Some IEHP participants had difficulty in following videos.
When the person is trying to explain something, for example they are speaking about function of a team, somehow it is very fast, and I need to come back or maybe need to concentrate more. (Physician, Iran)
The language is appropriate but the tone of speaking is a little bit fast. (Pharmacist, Egypt)
Educators also perceived that the language in the modules is the appropriate level for internationally
educated providers to understand. However, educators were less sure about whether or not the written
text would be too difficult.
I think the verbal language the people can understand okay. Some of the written text, I am not sure about. There will be some examples where text was required to type some of the concepts. I saw that was complicated. (Educator, IMG Program)
CULTURAL ASPECTS IEHPs reported that the content captured a range of cultural variation to understand and be respectful
of multiculturalism. IEHPs were aware that Canada has many diverse cultures so examples from each
Table 10. Comparing time spent to complete each ICT for IEHPs module among IEHPs (n=28) and educators (n=4)
Modules
Reported time to complete each module
(in minutes)
IEHPs
Reported time to complete each module
(in minutes)
Educators
Median (IQR) Median (IQR)
Introductory module 60.0 (30 – 180) 120.0 (98 – 143)
Conflict management 20.0 (15 – 30) 15.0 (13 – 50)
Team functioning 20.0 (15 – 30) 15.0 (15 – 26)
Communication 21.0 (15 – 30) 17.5 (11 – 28)
Patient-centred care 21.5 (15 – 30) 17.5 (11 – 28)
Collaborative leadership 20.0 (16 – 30) 15.0 (15 – 26)
Role clarification 20.0 (15 – 30) 15.0 (15 – 15)
IQR=interquartile range
26 | P a g e
could not be included. However, some IEHPs had suggestions to improve the cultural aspect of ICT for
IEHPs.
You need to capture more about aboriginal culture. Sometimes you need to know more about their culture, the way they are thinking, the way we need to address their health issues. (Physician, Iran)
The module did not address when is the time to quit, when is the time to withdraw, when is the time to say ok this is against my religion. (Pharmacist, Nigeria)
Working in the Middle East for many years, there are also many cultural nuances that need to be mentioned (e.g., limiting physical and eye contact, handshakes with those of the opposite sex). (RN, South Africa)
Educators reported that the modules described various health care providers from other
countries, which should resonate with IEHPs. Educators also mentioned that IEHPs often are not aware
of diversity and inclusiveness in care and how to interact with patients or providers from various
cultures. When dealing with different cultures, a shift in communication and practice may be required to
effectively care for certain populations. For many, being culturally sensitive is not something they have
been asked to do in previous health care environments.
I think being aware is the first part. I think the modules made them more aware of other cultures. Some of the audio described different healthcare providers from other countries. (Educator, Nursing Program)
One educator believed that transitioning to a collaborative patient-centred health system versus a
hierarchical physician driven system may cause uncertainty for some IEHPs. IEHPs may have a lot of
questions when trying to navigate this new environment, which may cause them stress. Recognizing or
acknowledging that cultural conflicts may be difficult to resolve was not addressed in the modules.
My experience has been that the international physicians… causes a lot of stress and uncertainty and I am not sure that that is addressed that it is okay and normal. The piece missing is that the two characters were confident and resolved their problems easily, but in reality the cultural conflicts transitioned here aren’t that easily addressed and it causes a lot of people stress. (Educator, IMG Program)
SCENARIOS Overall, participants perceived that the scenarios presented in the modules were very useful and most
of the IEHPs were able to relate their on situations to them. The scenarios made the contextual
information more practical.
I can relate well with some of them because being IEHP; I can compare where I worked before coming to Canada. (RN, Nigeria)
The scenarios are eye openers and relevant for my practice. I believe the colleges of each discipline can present specific case studies relevant to their jurisdiction's practice. (RN, Philippines)
27 | P a g e
However, some IEHPs, particularly pharmacy IEHPs, mentioned that the module should have some
scenarios of health care professionals working in a community setting.
Pharmacists are mostly working outside the hospital and all scenarios in the modules are hospital-based. 99% of internationally educated pharmacists are working in community pharmacy. (Pharmacist, Egypt)
Most of these scenarios are hospital-based while we work in community pharmacy. (Russia, Pharmacist)
Some of the scenarios can be based on a community-based setting type of health care team, whereby there is a good mix of regulated as well as non-regulated professions (e.g. housing workers, program planners) in the team. (PT, Nigeria)
Another participant mentioned that labeling the health care professional in scenarios would help them
understanding these scenarios.
During the scenarios, it was not clear which professional was talking. (RN, Nigeria)
ASSESSMENTS Most IEHPs found that the assessments were helpful in their understanding of the modules. Many
appreciated an opportunity to think and reflect about experiences. The reflections offered an
opportunity for IEHPs to think about applying the competencies to practical situations. However, some
IEHPs said that the assessments could be improved and offered the following suggestions:
Objectively measure competency learnings instead of subjective assessment
Prompt IEHPs to identify errors or less satisfactory collaboration practices
Measure learning only through multiple choice options as some IEHPs find writing a challenge
Use a different response rating as it may not be accurate for IEHPs to say they “strongly agree”
May be it is not accurate to say I strongly agree. For some parts you can give 1 – 10 rating instead. (Physician, Iran)
May be some of the questions you can add like if IEHPs really understood it (module content). Some sort of question that make you aware what they learned in the module. (OT, Philippines)
It will be nice to have some short quizzes, not just reflections. (LPN, Philippines)
Educators also suggested adding some objective assessment to assess competencies.
I would have them do assignments that are marked in the course or activities with which they are given feedback and maybe a quiz to see how they demonstrate it. (Educator, PT Program)
Participants identified that one question appeared twice in both pre- and post-assessment tools under
the role clarification domain.
28 | P a g e
LICENSING EXAM IEHPs had mixed opinions whether the content provided in the modules was relevant to the licensing
exams for their professions. Some believed that the content is useful for objective structured clinical
examination (OSCE) exams while others believed that the content in these modules may not be directly
relevant to the licensing exam.
It will particularly help in OSCE exam, we have real scenarios. So this module is very important for people going for OSCE exam to know how to collaborate with other health professionals. It will not help much in MCQs. (Pharmacist, Egypt)
If I have to take the exam on OT then I will just focus on subject. This module is good for practice but not relevant to licensing exam. (OT, Philippines)
Our license exam is more about subject knowledge. I don’t know if this is relevant to licensing exam. (RN, Nigeria)
TECHNICAL ISSUES The majority of participants did not report technical issues with the online modules. Nine participants
(seven IEHPs and two educators) reported one or more technical issues when reviewing the modules.
Four participants reported having trouble with sign on after creating the login account. The modules will
also need some adjustments to address other issues such as confusing forward arrows on slides and
sound issues with some videos. The summary of other key technical issues is provided in Box 1.
Box 1 Key technical issues mentioned by the survey participants
1. Slow buffering and it won’t move forward sometimes 2. Figuring out how to do the 'pop ups'. Arrows on the screen shot were confusing. 3. Sometimes there was not forward arrow which meant I had to go back to the mouse. Is it
possible to have either all forward arrow buttons on each slide or be able to use the arrow keys on the keyboard rather than the mouse
4. Kept getting a pop up stating that the internet connection was not secure and kept restarting
5. I had to use internet explorer because the video were not uploading using google chrome 6. I cannot use my IPAD because it does not have a Java software to run the program 7. My system was slowed down by the scorn software package even when I used the
recommended browser. Some of the videos did not have sound and the software sometimes refreshed by itself and I lost some of my answers to the quiz. These were very stressful to me and time consuming. It took me about 12 hours to complete the entire course.
29 | P a g e
CONCLUSIONS
The ICT for IEHPs is a very useful resource and IEHPs across different health care professions in
Canada may benefit from this resource. The ICT for IEHPs modules enhance the knowledge and
understanding of IEHPs of interprofessional practice. The content of these modules is also useful for
some professional licensing exams that involve an OSCE, such as pharmacy and medicine.
The use of terminology, contexts to practice, and varied cultural backgrounds in scenarios made
the modules relatable to IEHPs. The language in the modules seems to be appropriate for the most part.
The content captures a range of cultural variation to understand and be respectful of multiculturalism.
Further scenarios can be added to demonstrate what is acceptable and what is not acceptable in a
multicultural environment. IEHPs may require additional training on cultural competencies to apply to
both patients (e.g., First Nation’s populations) and healthcare providers.
The pre- and post-assessment tools and reflective questions are embedded in the introductory
and competency-based modules to measure their effectiveness. Although reflection exercises are
subjective and have no right or wrong answers, the questions prompt IEHPs to apply their learnings to
similar situations.
Overall, the competency-based mini modules were considered very useful and appropriate and
do not need any major change. The introductory module needs to be shortened.
Currently, the modules are available on a temporary host website. We also need to address all technical
issues (e.g., slow buffering) prior to launching them on a large-scale.
LIMITATIONS
We identified the following limitations of this evaluation:
Respondent burden was high so the response rate was low, particular from educators. There
were also variations in the response rate among provinces and professional groups.
We were only able to measure knowledge and attitude of the participants; we could not
measure the actual impact on clinical practice which would need a long-term follow up.
Another important limitation was the small number of participants, particularly the number of
educators. Our sample was limited so our findings may not have captured the true
representation of IEHPs in Western Canada.
Some potential participants who initially showed interest in the study were not able to
participate due to very tight timelines.
30 | P a g e
RECOMMENDATIONS
Based on our evaluation, we identified some areas for further improvements of the ICT for IEHPs
modules prior to the final launch.
Reduce the length of the introductory module
Clarify the patient safety scenario in the introductory module to ensure the scenario is linked to
IP collaborative practice
Add more objective assessment methods such as quizzes and multiple choice questions
Delete repeated questions in pre and post assessment tool under role clarification domain
Label the profession of healthcare providers in scenarios
Consider adding more scenarios where IEHPs communicate respectfully when dealing with
cultural issues and sensitive patient information or situations
Consider adding some scenarios of interprofessional teams working outside the hospital
environment
Consider adding videos on the examples of negative and positive conflicts
Make correction in low context and high context pictures in the communication module; the
pictures are interchanged
Address all technical issues before the final launch of ICT for IEHP
We also recommend future evaluation of ICT for IEHPs with a larger sample and increased
representation from different healthcare professions.
31 | P a g e
REFERENCES
1. A National Interprofessional Competency Framework. (2010). Canadian Interprofessional Health
Collaborative, College of Health Disciplines, University of British Columbia. URL
http://www.cihc.ca/files/CIHC_IPCompetencies_Feb1210.pdf (accessed on 02 Sep 2015)
2. IPC on the Run modules, College of Health Disciplines, University of British Columbia. URL
http://www.ipcontherun.ca (accessed on 02 Sep 2015).
32 | P a g e
APPENDIX I. IEHP PRE AND POST ASSESSMENT TOOL
After completing the interprofessional competency-based modules for internationally educated health care providers, think about how confident you feel NOW to enact the competencies for collaborative practice in your future practice. Enter your first name here This will be used to track your responses, which will be anonymized prior to analysis.
I. Role Clarification Understanding your own role and the roles of those in other professions, and using this knowledge appropriately to establish and achieve patient/client/family and community goals. Not
Confident Somewhat Confident
Confident Very Confident
Describe own role and that of others Recognize and respect the diversity of other health and social care roles, responsibilities, and competencies
Perform own roles in a culturally respectful way Communicate roles, knowledge, skills, and attitudes using appropriate language
Consider the roles of others in determining own professional and interprofessional roles
Access others’ skills and knowledge appropriately through consultation
Consider the roles of other in determining own professional and interprofessional roles
Integrate competencies/roles seamlessly into models of service delivery
II. Patient/Client/Family/Community-Centred Care Seek out, integrate and value, as a partner, the input, and the engagement of the patient/client/family/community in designing and implementing care/services. Not
Confident Somewhat Confident
Confident Very Confident
Support the participation of patients/clients, their families, and/or community representatives as integral partners alongside with health care personnel
Share information with patients/clients, (or family and community)in a respectful manner and in such a way that it is understandable, encourages discussion, and enhances participation in decision-making
Ensure that appropriate education and support is
33 | P a g e
provided to patients/clients, family members and others involved with care or service Listen respectively to the expressed needs of all parties in shaping and delivering care or services
III. Team Functioning Understand the principles of team work dynamics and group/team processes to enable effective interprofessional collaboration. Not
Confident Somewhat Confident
Confident Very Confident
Understand the process of team development Develop a set of principles for working together that respects the ethical values of members
Effectively facilitate discussions and interactions among team members
Participate, and be respectful of all members’ participation, in collaborative decision-making
Regularly reflect on functioning with team learners/practitioners and patients/clients/families
Establish and maintain effective and healthy working relationships with learners/practitioners, patients/clients, and families, whether or not a formalized team exists
Respect team ethics, including confidentiality, resource allocation, and professionalism
IV. Collaborative Leadership Understand and can apply leadership principles that support a collaborative practice model. Not
Confident Somewhat Confident
Confident Very Confident
Work with others to enable effective patient/client outcomes
Advance interdependent working relationships among all participants
Facilitate effective team processes Facilitate effective decision-making Establish a climate for collaborative practice among all participants
Co-create a climate for shared leadership and collaborative practice
Apply collaborative decision-making principles Integrate the principles of continuous quality improvement to work processes and outcomes
34 | P a g e
V. Interprofessional Communication Communicate with others in a collaborative, responsive and responsible manner. Not
Confident Somewhat Confident
Confident Very Confident
Establish team work communication principles Actively listen to other team members including patients/clients/families
Communicate to ensure common understanding of care decisions
Develop trusting relationships with patients/clients/families and other team members
Effectively use information and communication technology to improve interprofessional patient/client/community-centred care
VI. Interprofessional Conflict Management Actively engage self and others, including the client/patient/family, in positively and constructively addressing disagreements as they arise. Not
Confident Somewhat Confident
Confident Very Confident
Value the potential positive nature of conflict Recognize the potential for conflict to occur and taking constructive steps to address it
Identify common situations that are likely to lead to disagreements or conflicts, including role ambiguity, power gradients, and differences in goals
Know and understand strategies to deal with conflict
Set guidelines for addressing disagreements Effectively work to address and resolve disagreements, including analyzing the causes of conflict and working to reach an acceptable solution
Establish a safe environment in which to express diverse opinions
Develop a level of consensus among those with differing views; allowing all members to feel their viewpoints have been heard no matter what the outcome
Thank You! Hopefully this survey served as a valuable reflection on where you currently stand with regards to your collaborative competencies and how you will practice in the future.
35 | P a g e
APPENDIX II. SURVEY FOR IEHPS
36 | P a g e
37 | P a g e
38 | P a g e
39 | P a g e
40 | P a g e
41 | P a g e
42 | P a g e
43 | P a g e
44 | P a g e
45 | P a g e
46 | P a g e
APPENDIX III. INTERVIEW GUIDES IEHPs
Project title: Interprofessional (Practice) Competency Tools for Internationally Educated Health Professionals
This interview will take about 30 minutes
1. How useful were these modules in improving your knowledge about working with
health care professionals from other disciplines?
2. How well did you relate to the scenarios/situations shown in the modules?
3. How easy to understand was the language in the modules?
4. What content in the modules is most important or meaningful for you and why?
5. What content in the module is not important to you and why?
6. Of the two modules that you reviewed which is more important and why?
7. Of the six mini-modules in the Competency-Based Module which are more important and why?
8. How useful did you find the assessments in the module?
9. How can the pre and post assessments in these modules be improved?
10. What content appears to be missing from each module?
11. What cultural aspects were not covered in each module?
12. How can we make the components of the module culturally respectful and sensitive?
13. How useful do you think the module is useful for you in your preparation for licensure?
14. Do you like the current format/style of teaching the module or do you want to see a different format/content style?
If you are interested of receiving a report of evaluation results, may I have your email address please?
Thank you for your participation
47 | P a g e
Educators
Project title: Interprofessional (Practice) Competency Tools for Internationally Educated Health Professionals
This interview will take about 30 minutes
1. How useful do you believe these modules will be in improving the IEHPs
knowledge about interprofessional collaboration?
2. How well do you believe the IEHPs will relate to the scenarios/situations illustrated in the modules?
3. How easy do you think the language in the modules will be for IEHPs to understand?
4. What content in the modules do you believe is most important or meaningful for IEHPs and why?
5. What content in the module do you believe is not important for IEHPs and why?
6. Of the two modules that you reviewed which is more important and why?
7. Of the six mini-modules in the Competency-Based Module that you reviewed, which are
more important and why?
8. How useful did you find the assessments in the module?
9. How can the pre and post assessments in these modules be improved?
10. What content appears to be missing from each module?
11. What cultural aspects were not covered in each module?
12. How can we make the components of the module culturally respectful and sensitive?
13. Do you think the modules will be useful to IEHPs in their preparation for licensure?
14. Do you like the current format/style of teaching the module or do you want to see a different format/content style?
If you are interested of receiving a report of evaluation results, may I have your email address please?
Thank you for your participation
48 | P a g e
APPENDIX IV: REFLECTIVE QUESTIONS Introductory module
Interprofessional Collaboration
1. Describe your current understanding of interprofessional collaborative practice. You will have
the opportunity to revisit your description at the end of the module to see how your
understanding has changed.
2. What percentage of medical errors are due to communication and teamwork issues?
3. Communication failures are the leading cause of inadvertent patient harm.
4. Ineffective communication across professions results in medical errors.
5. What made this team meeting effective?
6. What might effective collaboration look like in the context in which you work?
7. What are some of the challenges to interprofessional collaboration?
8. Your goal for future learning.
Competency-based module
Conflict Management/Resolution
9. What happened?
10. How does an appreciation of other personalities cultures, generations, and, professions impact
the way you will collaborate?
Team Functioning
11. What happened?
12. How does an understanding of how interprofessional collaborative teams function in different
contexts impact the way you will collaborate?
Interprofessional Communication
13. What happened?
14. How does an appreciation of other personalities, cultures, generations, and professions impact
the way you will collaborate?
Patient Centred Care
15. What happened?
16. In previous practice, how have you involved patients and their families in their care?
17. How does appreciation of the values and principles of interprofessional patient- centred care
within the Canadian context impact the way you will collaborate?
Collaborative Leadership
18. What happened?
19. How does your knowledge about collaborative leadership impact the way you will collaborate?
Role Clarification
20. What happened?
21. How does respect for the diversity of other health and social care roles responsibilities and
Competencies impact the way you will collaborate?
49 | P a g e
APPENDIX V: ETHICS EXEMPTION FROM UNIVERSITY OF SASKATCHEWAN