The Athletics Review Process — Are We Doing What We Say We Are Doing?
Manual Evaluation: Are we doing what we think we are doing? Associate Professor Dee Mangin Director...
-
Upload
alejandro-anderson -
Category
Documents
-
view
217 -
download
0
Transcript of Manual Evaluation: Are we doing what we think we are doing? Associate Professor Dee Mangin Director...
Manual Evaluation:Are we doing what we think we are doing?
Associate Professor Dee ManginDirector Primary Care Research Unit
Department of Public Health and General PracticeChristchruch School of Medicine,
University of Otago
20 May 2009WHA
Overview
Rationale for testing the manual in a pilot study
Protocol for testing
Timelines
Background
WHO/HAI have developed a comprehensive teaching package
comprising a manual and accompanying resource material as well as
training for teachers implementing and this
Background
It is planned to evaluate this in a pilot study to
look at its effectiveness and at areas for
improvement in the final manual
WHY DO THIS STUDY?
Background
Many educational resources and techniques
are implemented without ever having any
assessment of their effectiveness
So we don’t know whether our efforts are effective
Overview
Our wish to think that they are can be misleading……
Many strategies for medical education have little effect on prescribing when they are tested in studies
Evidence for prescribing education strategies that have been found effective
The effect of dissemination only strategies such as didactic lectures and written information are small at best and of unknown clinical significanceFarmer et al. Cochrane Database of Systematic Reviews 2008;3
Facilitated teaching with evidence based resources and audit and feedback have been found to have a moderate effect Richards et al Family Practice 2000
A previous WHO educational resource (WHO Guide to Good Prescribing) using facilitated teaching for medical students on pharmacotherapy showed a significant effect on students skills in a randomised controlled trial Impact of a short course in pharmacotherapy for undergraduate medical students: An international randomised controlled study. de Vries T Henning R Hogerzeil H Bapna J et al The Lancet 1995
Study Design
• Randomised controlled trial
10 volunteer sites
5 intervention sites 5 control sites
• Quantitative and qualitative methods to assess change in students knowledge attitudes and skills
Questions
• What is the effect on students– knowledge, attitudes and skills of students in
understanding and responding to pharmaceutical promotion
• What are the strengths and weaknesses of the content of the manual and resources provided– assessed across a variety of settings
Data collection
•Before and after assessment of knowledge, attitudes and skills of students
• Questionnaire• Focus groups
•Key informant interviews and content evaluation by teachers in the intervention group
Two comparisons between these randomised intervention and control groups
The difference between the before and after manual assessments at the beginning and end of the year compared to the control sites
The difference between their assessments at the end of the year and students at the end of the year in the previous year groupAgain compared to the differences at the control sites
Comparison 1
Change in scores at intervention sites compared to control sites
5 5
intervention sites control sites
manual teaching as usual
Before Before
compared to
After After
difference difference
Comparison 2
• Difference in scores between study groups at the end of the year and students at the end of the previous year 5 5 intervention sites control sites
Students from previous year tested for comparison
Students from previous year tested for comparison
Students at end of study year
Students at end of study year
Study start Study start
Manual implemented Teaching as usual
Volunteer sites
• Medical or pharmacy students• Site prepared to teach all manual chapters• Site able to teach within one academic year (12
months)• Ethics approval will be gained where required
Volunteer sites
• Training, travel and accommodation will be provided for a teacher from each site
• The evaluation assessments and analysis will be organised and co-ordinated by the study team based at HAI
• These involve the before and after questionnaires and focus groups for students/interviews for teachers at each site
Volunteer sites: Timeline
• Plan to implement as academic years begin from later in 2009 through to 2010
• Training for teachers from sites randomised to intervention in Amsterdam Sept 2009
• ‘Before’ assessments as academic years start• ‘After’ assessments completed late 2010 as years
end• Training for teachers from sites randomised to
control in Amsterdam in Sept 2010
It is not new
We need to ensure medical and pharmacy graduates are well equipped to understand and deal with the effects of promotion on their prescribing decisions
It is important that we not just have good ideas
But that we know whether our collective efforts are effective
Volunteers!- Pilot study- Testing the manual in your setting- Expression of interest forms at the back
of the room OR- Email: [email protected]