Mentor support program - FINAL REPORT - 6CPA6cpa.com.au/wp-content/uploads/Mentor-Support... ·...

Mentor Support Program for Pharmacists Gaining Accreditation

to Conduct Medication Reviews

Together with

Accredited Pharmacist Services in Rural and Remote Areas

– A Mentor Support Program for Pharmacists

Final Report

October 2003

These projects were funded by the Commonwealth Department of Health and

Ageing, as part of the Third Community Pharmacy Agreement

. 3rd Community Pharmacy Agreement Mentor Support Projects - Final Report October 2003

2

Acknowledgements

The Australian Association of Consultant Pharmacy and the Australian College of Pharmacy Practice would like to acknowledge the support and contribution of the Pharmacy Guild of Australia through the work of the Third Community Pharmacy Agreement Research and Development Grants Program and the Rural and Remote Pharmacy Workforce Development Program. This report addresses the work undertaken by the projects approved through the Research and Development Grants Program Investigator Initiated Projects and the Rural and Remote Pharmacy Infrastructure Grants Scheme respectively. Grant 2001 – 052

Mentor Support Program or Pharmacists Gaining Accreditation to Conduct Medication Reviews Grant 2001 / 813 Accredited Pharmacist Services in Rural and Remote Areas – A Mentor Support Program for Pharmacists

These projects were funded by the Commonwealth Department of Health and Ageing, as part of the Third Community Pharmacy Agreement. Contributors The success of these projects can be attributed to the invaluable contribution of the following:

The mentor pharmacists who participated. A list of mentors is included in the Results section.

The Drug and Therapeutic Advisory Service (DATIS) in the development of the mentor training program

Staff at the AACP and ACPP secretariats Chief Investigator(s) Leone Coper (Dec 2001 to Jan 2002) Jessica Graves (May 2002 to May 2003) Bill Kelly (Aug 2003 to Oct 2003) Australian Association of Consultant Pharmacy PO Box 7071 Canberra Mail Centre ACT 2610 Phone: (02) 6270 1850 Fax: (02) 6273 8160 Email: [email protected] Principal Investigator Professor Peter Carroll Australian College of Pharmacy Practice

PO Box 7007 Canberra Mail Centre ACT 2610 Phone: (02) 6273 8989 Fax: (02) 6273 8988 Email: [email protected] Project Officer Pharmacist Lee Sadler Australian Association of Consultant Pharmacy PO Box 7071 Canberra Mail Centre ACT 2610 Phone: (02) 6270 1850 Fax: (02) 6273 8160 Email: [email protected]

mailto:[email protected]




3

Table of Contents

Acknowledgements.................................................................................. 2 Contributors........................................................................................................... 2

Abbreviations................................................................................................ 5

Executive Summary .........................................6

Summary of Recommendations..................................................... 8

Introduction................................................................10 Project Proposal .................................................................................................. 10 Project Rationale ................................................................................................. 11 Project Aims and Deliverables............................................................................. 11

Project Management .....................................13

Methodology .............................................................14 Outline................................................................................................................. 14 Identifying and Recruiting Suitable Mentors......................................................... 14 Developing and Delivering Mentor Training Program........................................... 15 Developing Mentor Standards and Guidelines..................................................... 15 Allocation of Mentors to Candidates .................................................................... 16 Advertising and Recruiting Suitable Candidates .................................................. 16 Contact Between Mentor and Candidate to Review Case Study/ies.................... 16 Developing Evaluation Forms.............................................................................. 17 Administration...................................................................................................... 17 Assessing Success of Mentor Support Program.................................................. 18

Results...............................................................................19 Mentor Selection ................................................................................................. 19 Mentor Training ................................................................................................... 19 Mentor Agreed Guidelines and Standards ........................................................... 20 Participation in Project......................................................................................... 21 Evaluation of Project............................................................................................ 21 Mentor / Candidate contact.................................................................................. 22 Evaluations.......................................................................................................... 24 Future Programs ................................................................................................. 44


4

Extent of Payment ............................................................................................... 45 Participation in Project......................................................................................... 48 Results Summary ................................................................................................ 49

Conclusion and Recommendations...........................................52

Mentors ............................................................................................................... 52 Mentor Training ................................................................................................... 52 Agreed Mentor Standards and Guidelines ........................................................... 53 Mentor / Candidate Contact................................................................................. 53 Evaluations.......................................................................................................... 54 Impact on confidence and time taken to complete reviews .................................. 56 Future Programs ................................................................................................. 56 Number of Participants and Accredited Pharmacists ........................................... 57

Appendices.................................................................58

Appendix 1. Mentor Agreed Guidelines and Standards for the Mentor Support Project ..................................................... 59

Appendix 2. Mentor Support Project Evaluation ............. 62 Appendix 2.1.1: Candidate Evaluation Form – Examination Phase ..................... 63 Appendix 2.1.2: Candidate Evaluation Form – Examination Phase Results......... 66 Appendix 2.2.1: Candidate Evaluation Form – First Review ................................ 74 Appendix 2.2.2: Candidate Evaluation Form – First Review Results.................... 77 Appendix 2.3.1: Mentor Evaluation Form – Examination Phase .......................... 83 Appendix 2.3.2: Mentor Evaluation Form – Examination Phase Results.............. 85 Appendix 2.4.1: Mentor Evaluation Form – First Reviews.................................... 91 Appendix 2.4.2: Mentor Evaluation Form – First Reviews Results....................... 93 Appendix 2.5.1: Mentor Evaluation Form – Overall Program ............................... 96 Appendix 2.5.2: Mentor Evaluation Form – Overall Program Results .................. 98


5

Abbreviations

AACP Australian Association of Consultant Pharmacy ACPP Australian College of Pharmacy Practice CPA R&D Community Pharmacy Agreement Research and Development DATIS Drug and Therapeutic Information Service DMMR Domiciliary Medication Management Review (also known as HMR) DOHA Commonwealth Department of Health and Ageing Guild Pharmacy Guild of Australia GP General Practitioner HMR Home Medicines Review (also known as DMMR) MMR Medication Management Review MRN Medication Review Number MSP Mentor Support Program PSA Pharmaceutical Society of Australia RCF Residential Care Facility RRPIGS Rural and Remote Pharmacy Infrastructure Grants Scheme Notes:

The terms Domiciliary Medication Management Review (DMMR and Home Medicines Review (HMR) are often used interchangeably in practice. This report endeavours to use the term DMMR consistently, however, the utilisation of either term will be evident when reviewing the comments of participants.

The term “mentee” was coined during the early stages of the project and refers to the mentor’s candidate.


6

Executive Summary

Two Mentor Support projects were undertaken by the Australian Association of Consultant Pharmacy and the Australian College of Pharmacy Practice from funds provided through the Third Community Pharmacy Agreement Research and Development Grants Program and the Rural and Remote Pharmacy Workforce Development Program. The research projects were based on the extensive utilisation of mentor services in the development and assessment of the various models of Domiciliary Medication Management Review that were subsequently used to define and articulate the DMMR service now funded by the Commonwealth Department of Health and Ageing. It was clear in the results of these research projects that collegiate and clinical support were valued and contributed to both the quality and efficiency of the medication management review services provided. In the context of this project, the notion of a mentor was aligned to the benefits gained by sharing experience, rather than the provision of theoretical clinical information. (The clinical information utilised in any medication management review is dependent on the skill of the pharmacist to interpret the information in the context of the patient. This is the key to the success of many Quality Use of Medicines initiatives, and is especially true of medication management review.) In this way, the notion of peer review and collegiate support and guidance were encouraged, and the results indicate that this was and effective strategy, with the both mentors and candidates commenting on the enjoyment and benefit of this style of mentoring. The Mentor Support Program developed eight distinct elements:

Identify and recruit suitable mentors Develop and provide mentor training Develop agreed mentor standards and guidelines Advertise and recruit candidates Develop evaluation forms Mentor / candidate contact to consider case study(s) Evaluate program

o Examination phase o First reviews phase

Administration The main area of activity of the Mentor Support Program involved the mentoring service provided to the candidates. This required administrative support to allocate and track the progress of the mentors and their candidates through the process and the provision of support and guidance as needed, in order to facilitate the “work” of the projects as undertaken by the mentors and candidates. In the examination phase of the project, the mentors provided guidance and advice to the candidate about the first case study in their assessment examination. They were not to provide the “answers”, but to indicate where the review may be improved, what resources may be useful to achieve this and to provide comment on the usefulness and acceptability of the final report, as it would be received by a doctor.


7

In the first reviews phase of the project, the mentors provided guidance and advice about “real” medication management reviews undertaken by the newly accredited pharmacist, providing comment and advice on content, layout and style. In addition to this they frequently provided advice pertaining to useful resources, templates of forms and other practical advice about the provision of medication management review services. The activities of the Mentor Support Program can be summarised:

536 pharmacists registering their interest (95 rural + 441 non-rural) 358 pharmacists sent in case studies and were allocated mentors (65 rural +

293 non-rural) 613.50 hours of mentor services were provided 309 candidates received mentor services An average of 2 hours mentoring was received by each candidate 190 pharmacists gained their accreditation as a result of the mentor support

program (33 rural + 157 non-rural) – this represents 13% of the accredited pharmacist in Australia at the time of this report

An overall pass rate of 91% was achieved (this compares quite favourably with a pass rate ~ 60% of those who attempt the case studies assessment without the support of a mentor)

These statistics demonstrate the success of the Mentor Support projects in terms of increased number of accredited pharmacists available to be involved in medication management review services. Together with the number of accredited pharmacists resulting from the Mentor Support Program, the other main indicator of success identified in the proposals related to the confidence of the candidate to become accredited and then to provide medication management reviews. In this regard, the Mentor Support projects were very successful, with:

72% of candidates indicating a significant increase in confidence as a result of the mentoring they received

79% of mentors indicating they anticipated a significant increase in the candidates confidence as a result of their input

Testimonial The following testimonial received, aptly describes the value of Mentor Support.

Dear Mentor (name deleted), Well it's almost been a year since I have become accredited, (I know because I just received my reminder to pay the fees!) I just wanted to say a big thank you to you for all your mentoring support; it is really good to know that if I needed a hand you were there. The HMR are coming through steadily, enough to keep me busy. I do find the lack of communication with some of the doctors frustrating however working on it..... Hope to catch you at another function soon Once again THANKS ! Regards Candidate (name deleted)


8

Summary of Recommendations

The following recommendations are included here in the order they appear in the chapter titled Conclusion and Recommendations, reflecting the order in which they are developed in the text (rather than a list of priorities). It is suggested that those pharmacy organisations involved in relevant areas or member services, be asked to consider how they may be involved in progressing some of the recommendations that have developed through the activity of the Mentor Support Program. In this way, it is hoped that the benefit and momentum of these projects can be maintained, as others seek to integrate some of the key features of this work into their current or future activities, and perhaps seek funding to ensure that it is sustainable in the long term.

Recommendation One: That any future mentor support program provided to assist pharmacists through the assessment and early reviews phase of their involvement in medication management review, utilise the services of mentors who have considerable experience and expertise in this area.

Recommendation Two: That training be provided for mentors involved in this area, to address the issues of:

expectation of the examiners in the assessment process principles of adult learning distance communication use of references time management skills

Recommendation Three: That any future mentor support program provided to assist pharmacists through the assessment and early reviews phase of their involvement in medication management review, develop an agreed standard and guidelines to ensure a consistency of quality across the program, and that these guidelines be reviewed periodically to ensure currency and suitability.

Recommendation Four: That any future mentor support program provided to assist pharmacists through the assessment and early reviews phase of their involvement in medication management review, encourage the use of face-to-face meetings between the candidate and their mentor whenever possible, and that all other means of communication be utilised as appropriate and convenient for both parties.


9

Recommendation Five: That the providers of the Stage I courses designed to prepare pharmacists for the examination assessment for accreditation, while designing their courses to fulfil any current or future required competencies that pertain to the provision of MMR, be encouraged to adapt their courses to more fully address the practicalities of providing medication management review services. This would include issues relating to:

The use of the relevant forms Report writing requirements How to prioritise problems and recommendations How to use clinical judgement

Recommendation Six: That utilisation of a mentor support program be strongly encouraged (or mandated?) for the first reviews phase of an accredited pharmacists involvement in this area, to ensure consistency of quality and the clinical integrity of the service.

Recommendation Seven: That the appreciation of collegiate support identified in this project be promulgated to those pharmacy organisations who may be able to develop a mechanism to encourage and facilitate this further, eg Australian Association of Consultant Pharmacy, The Australian College of Pharmacy Practice, the Pharmaceutical Society of Australia, The Pharmacy Guild of Australia through its MMR Facilitator program, and the Alumni Associations of the respective Pharmacy Schools etc.

Recommendation Eight: That a Mentor Support Program be developed and made available to those pharmacists undergoing their assessment examination and first reviews phase, and that the structure of this program facilitate one to one meetings whenever possible.

Recommendation Nine: That financial support be identified to provide a Mentor Support Service that minimises the payment required of the candidates.


10

Introduction

This report combines the results of two projects made possible through funding provided by the Commonwealth Department of Health and Ageing as part of the Third Community Pharmacy Agreement. It is reasonable to expect that the introduction of the collaborative Domiciliary Medication Management Review service will require every pharmacy in Australia to have access to an accredited pharmacist. In light of this, and with the experience of the research projects to provide guidance about the scope and nature of a suitable mentor service, the proposals to provide a Mentor Support service to pharmacists undertaking the assessment phase of accreditation and for their first few reviews in the “real world” were developed and submitted. Project Proposal In administering the equitable and appropriate distribution of funds available as part of the Third Community Pharmacy Agreement, the Pharmacy Guild of Australia has designated two areas to address this: the Third Community Pharmacy Agreement Research and Development Grants Program and the Rural and Remote Pharmacy Infrastructure Grants Program which is part of the Rural and Remote Pharmacy Workforce Development Program. The Australian Association of Consultant Pharmacy, together with the Australian College of Pharmacy Practice, made successful application to develop and administer a Mentor Support Program for pharmacists, to the Research and Development Grants Program Investigator Initiated Projects and the Rural and Remote Pharmacy Infrastructure Grants Scheme respectively. As the principal objective of both grant applications was to facilitate the accreditation of pharmacists to provide medication management review services, the proposed structure of both projects was identical. The need for accredited pharmacists in all areas of Australia is recognised at all levels and the project proposals received strong support from stakeholders.

When assessing the data generated during the evaluation phase of the projects, it became apparent that the modest number of pharmacists participating from rural and remote areas may not as effectively demonstrate the trends evident from a larger data set. It was suggested, in consultation with management at AACP, ACPP and the Guild, that the reports of the respective grants be presented as a single document in order to more fully demonstrate the results found. By presenting the results as “rural”, “non-rural” and “combined”, they are able to be compared, allowing for the identification of common trends and those areas that reflect differences attributed to geographic location. From an administrative perspective, both projects were managed by the AACP office staff and used the services of the one project pharmacist. The only point of difference involved the record keeping at the office, in order to identify which of the pharmacist candidates were drawn from rural and remote areas and those who were not. In every other respect, the projects were conducted as though they were one, with neither the mentors nor the candidates being made aware of any arbitrary difference generated by the respective sources of funding for each project.


11

Consequently, this report presents the results from the projects titled:

Mentor Support Program for Pharmacists Gaining Accreditation to Conduct Medication Reviews

Accredited Pharmacist Services in Rural and Remote Areas – Mentor Support Program for Pharmacists

Project Rationale The introduction of Domiciliary Medication Management Reviews has highlighted the need for every community pharmacy to have access to the services of an accredited pharmacist. Prior to the approval of these Mentor Support projects, it was postulated that a significant uptake of DMMRs by GPs may precipitate a workforce crisis in this important area of practice. It has also been acknowledged that the accreditation process is quite challenging, and that the disincentives most often identified include:

The cost of education and the accreditation process Pharmacists lack of confidence in attaining the standard required to gain

accreditation The Mentor Support proposals addressed one of these barriers to accreditation, by providing mentor support to pharmacists to assist them in their accreditation and post accreditation endeavours. Project Aims and Deliverables The aims of the project, as outlined in the initial project proposal are:

To raise the confidence and ability of community pharmacists to perform comprehensive medication management reviews.

To increase the number of accredited pharmacists, thereby ensuring access by consumers to needed professional pharmacy services.

To ensure the quality of medication management reviews provided by community pharmacists, thus producing best outcomes for consumers and the most useful service to other health professionals.

To inform and assist the development of future support programs delivered through the MMR Facilitator network in the divisions of general practice around Australia.

When applying to the RRPIGS, the proposal identified the following priority areas that would be addressed by the project:

Priority Area B. Pharmacist services Developing pharmacist involvement in assisting GPs and patients manage specific disease state

Priority Area C. Support for rural pharmacists National or regional mentoring systems

Priority Area D. Multi-disciplinary models Developing coordinated models of service delivery in regional areas

The expected benefits and outcomes of the project were to:

Increase the number of accredited pharmacists in rural and remote areas of Australia, thereby improving access by consumers to needed professional pharmacy services.


12

Raise the confidence and ability of community pharmacists in rural and

remote areas of Australia to provide comprehensive medication management review services.

Ensure the quality of medication management review services provided by these community pharmacists, thereby assisting in the best outcomes for the consumer and the most useful service to other health professionals.

The project proposals envisaged that the combined mentor services of 20 mentors across Australia would facilitate the accreditation of:

100 rural pharmacists 460 non-rural pharmacists

It was anticipated that the candidates would utilise the advice and guidance of their mentor for their first few medication management reviews in the “real world”, once accredited. The evaluations received from the mentors and candidates were collated to ascertain the value of the Mentor Support Program and to enquire how, from their perspective, future support programs for pharmacists may be improved.


13

Project Management

The Mentor Support Program was supervised by the Chief Investigator(s), the National Director of the Australian Association of Consultant Pharmacy and the Principal Investigator, the Dean of the Australian College of Pharmacy Practice. The Project Officer Pharmacist and the administrative staff of AACP and ACPP, together with the National Director and relevant staff of the Drug and Therapeutic Information Service, were responsible for the development and delivery of the mentor training. The administrative and financial management staff of AACP were involved in the tracking of the mentor allocation and the distribution of funds associated with the mentor activity, and the collection of the evaluation forms as they were returned to the office. The Project Officer Pharmacist provided ongoing guidance and support to the mentors during the course of the project and was responsible for the collation of the results from the evaluation forms and the preparation of much of the material developed during the course of the project. Chief Investigator(s) Leone Coper (Dec 2001 to Jan 2002) Jessica Graves (May 2002 to May 2003) Bill Kelly (Aug 2003 to Oct 2003) Australian Association of Consultant Pharmacy PO Box 7071 Canberra Mail Centre ACT 2610 Phone: (02) 6270 1850 Fax: (02) 6273 8160 Email: [email protected] Principal Investigator Professor Peter Carroll Australian College of Pharmacy Practice PO Box 7007 Canberra Mail Centre ACT 2610 Phone: (02) 6273 8989 Fax: (02) 6273 8988 Email: [email protected]

Project Officer Pharmacist Lee Sadler Australian Association of Consultant Pharmacy PO Box 7071 Canberra Mail Centre ACT 2610 Phone: (02) 6270 1850 Fax: (02) 6273 8160 Email: [email protected]





14

Methodology

Outline The methodology utilised in the projects involved:

the selection and recruitment of suitably experienced accredited pharmacists to be mentors;

the development and delivery of a suitable training program for the mentors; the development of agreed mentor guidelines to ensure a consistency of

approach and uniformity of standards throughout the projects; the development of suitable evaluation forms; the promotion of the Mentor Support Program to those pharmacists

undertaking the accreditation process, to recruit suitable candidates; the contact between a candidate and their allocated mentor to work through

the case study/ies; and the evaluation of the Mentor Support Program from the perspective of both

the candidate and the mentors, to assess the benefit and acceptability of the service, as well as ascertaining how a future support service for pharmacists may be improved.

As evident in the project proposal and the summary of the methodology outlined above, the role of the mentors has been critical to the success of the projects. For the purposes of the projects, the duties of a mentor included:

receiving the referral for mentoring from AACP; reviewing the candidate’s response to the first case study in the examination

book; telephoning the candidate and discuss their approach to the case and their

rating against the standard required; suggesting areas where the candidate may require further study or a change

in approach and offer assistance to help overcome any identified deficiencies; being available for further discussion of issues on 1-2 occasions throughout

the period of completion of the examination book by the candidate; on request, reviewing and discussing the first five actual medication reviews

completed by the newly accredited pharmacist; contributing comments from their experience and expertise to problems

posted on the AACP Forum (a restricted, adjudicated web site posting of specific problems identified during reviews); and

contributing to the evaluation of the mentor support service. Identifying and Recruiting Suitable Mentors Though the use of the term “mentor” often has connotations of a revered teacher, source of wise counsel, or a learned seer or guru, the term has developed a slightly different meaning when used in conjunction with the training for, or provision of medication management review services by pharmacists. Depending on the relevant experience and expectations of the pharmacist to be “mentored”, the exercise may range from peer review to educator. The process used to recruit suitable pharmacists to be mentors for the projects involved interrogation of the database at AACP to identify those accredited pharmacists who:


15

had at least one year of experience in conducting medication management

reviews; had gained an “excellent” or “very good” rating in their re-accreditation

assessment; had access to email; agreed to a timely response to candidates; and were available to attend the mentor training.

Suitably qualified MMR Facilitators in Divisions of General Practice were also invited to participate in the project as mentors. As the Mentor Support Program was intended to be available to pharmacists anywhere in Australia, and effort was made to ensure that the mentors reflected a wide geographic distribution. Although face-to-face meetings are acknowledged as facilitating the most effective communication, it was envisaged that the exchange between the mentors and candidates may utilise a number of communication mediums. The mentors and candidates were encouraged to use whichever mean of communication optimised their time to consider the case studies. The range of communication medium used during the project was ascertained in the evaluations. Developing and Delivering Mentor Training Program In conjunction with the Australian College of Pharmacy Practice and pharmacists from the Drug and Therapeutic Information Service who have experience in the practice and training of educational visiting, the training program for the mentors was provided in Adelaide on Tuesday 22nd (6pm to 11pm) and Wednesday 23rd (8.30am to 4.30pm) January 2002. The mentors were briefed about the nature and purpose of the project, and were also encouraged to use each other as collegiate support during the project. Acknowledging that the clinical skills of the mentors were deemed to be excellent, the learning objectives of the program included:

a review of the accreditation assessment process and the expectations and experience of the examiners (the ACPP contract examiners were present and were able to share their experience in this area);

adult learning skills to build knowledge and confidence; the use of case studies to gain exposure to the range of responses received

and the opportunity to develop a strategy to address some of the issues raised;

effective communication by phone and email – how to “listen” without “non-verbal cues”;

the use of references – which ones and why; and time management skills.

Developing Mentor Standards and Guidelines As one of the objectives of the mentor training day, the standards and guidelines to which the mentors would work, was developed and agreed by them. The issues addressed included:

candidate preparedness;


16

definition of “timely response”; approach to candidate; resources; use of forms; and importance of effective report writing.

Allocation of Mentors to Candidates Wherever possible, the allocation of mentors to candidates was determined by their geographic proximity, in order to facilitate face-to-face meetings. Similarly, those mentors in rural areas were allocated candidates from rural areas when possible, in order to provide for any particular considerations or difficulties that may be encountered in this setting. Advertising and Recruiting Suitable Candidates In order to fulfil the project objective to increase the number of accredited pharmacists in Australia, the Mentor Support Program was promoted to potential candidates in several ways. Those pharmacists who had already commenced the accreditation process, but had not completed the examination phase, each received a letter of invitation to be involved in the project. Similarly, those accredited pharmacists who were not actively involved in the provision on medication management reviews were invited to participate in the second phase of the project, if they were reticent or nervous to embark on the provision of cognitive services such as Domiciliary Medication Management Reviews. This process was repeated through the course of the project to remind and encourage participation. The Mentor Support Program was also promoted in the AACP Newsletter and on the AACP Website. As new candidates enrolled in the accreditation process by AACP or training providers, either when undertaking the Stage 1 training or when receiving the case study assessment book, they received an enrolment form to be included in the project. During the course of the project, articles were also written for inclusion in pharmacy newsletters and journals. Contact Between Mentor and Candidate to Review Case Study/ies The principal activity of the Mentor Support Program involved the contact between the mentor and the candidate. Every effort had been made to provide support for this process to be effective, with communication, training and administrative strategies in place to optimise the outcome of this important element of the project. As the results indicate, the success of the project can be attributed to the invaluable contribution of the mentors.


17

Developing Evaluation Forms As described in the project methodology outline above, the evaluation of the Mentor Support Program would occur at three levels:

administration; mentor perspective; and candidate perspective.

In order to collect and collate the impressions of the mentors and the candidates involved in the project, five evaluations forms (two for candidates and three for mentors) were designed to ascertain their experience and level of support for the program, and to request feedback that would assist the design of future support programs for pharmacists. Data collected from mentors included:

time involved in each candidate referral for review of the first case study; issues addressed; areas where remedial action was recommended; actions suggested to overcome any deficiencies; nature and number of additional candidate requests for help during the time

the candidate completed the remainder of the case studies in the examination book;

time taken in helping with these additional requests; opinion of the mentor regarding the overall worth and impact of the program;

and fee and structure the mentor believes would be appropriate for the program to

be ongoing. Data collected from candidates included:

value for candidate in having their first case study reviewed; nature and extent of the feedback received; extent to which candidate acted on the advice given by the mentor; value and nature of any additional contacts with the mentor; contribution of mentor in helping them gain insight into the issues involved in

undertaking medication reviews; contribution of mentor to helping them understand the knowledge base and

standard required to undertake medication reviews; whether the program provided confidence to undertake medication reviews; whether they would have been able to complete the examination case studies

without the help of the mentor; and fee and structure the candidate believes would be appropriate for the program

to be ongoing. Draft evaluation forms were developed prior to the mentor training day and provided to the mentors who were invited to provide comment or suggestions to improve the information gathered, in order to fulfil the stated objectives of the project. Administration The administration for the project comprised:

a project officer/pharmacist coordinating the projects;


18

contact between mentor and candidate; financial tracking (mentor fees); collecting and collating evaluation forms; preparing reports and financial statements.

The services of a project officer/pharmacist, experienced in medication management review and quality use of medicines and with high standing in the profession was secured for this project, together with the administrative support of the AACP office staff. The project officer, in conjunction with the Chief Investigator and Principal Investigator, was responsible for the development of the training program, the evaluation forms and providing ongoing support to the mentors. The interrogation and interpretation of the resulting data, and the preparation of project reports was also the responsibility of the project officer. The AACP office staff responsibilities included the allocation of candidates to mentors, the contracts between them, the processing of the claim forms from the mentors and the collection of the evaluation forms. (Due to the installation of a new computer system at the AACP office during the course of the project, it was necessary to transfer the responsibility for the collation and recording of the feedback received on the evaluation forms, from the administrative staff at AACP to the project pharmacist.) The preparation of the financial statements associated with the administration of the project was the responsibility of the AACP financial controller. Assessing Success of Mentor Support Program It was anticipated that the results of the Mentor Support Project/s would be compiled from the five evaluation forms utilised to ascertain the degree of usefulness and acceptability of the service from the perspective of both the mentors and their candidates, as well as comparing the overall pass rate of those pharmacists who had participated in the Program compared to those who had undertaken the case study assessment process without the support of a mentor. This pass rate would be a pooled result drawn from the AACP database, and would include those pharmacists who had undertaken accreditation assessment both prior to and concurrent with the Mentor Support Program, but without participating in the Mentor Support Program.


19

Results

Mentor Selection The recruitment and selection of appropriate pharmacists to be involved in the Mentor Support Projects resulted in the inclusion of the following 21 mentors.

Ben Basger Katherine Baverstock Shane Britnell Anne Buxton Alison Clark Andrew Clayton Brigitte Cusack Susan Edwards Kerry Feilding Anne Marie Fiumara Stephen Griffiths

Kaye Hazel Jeff Hughes Anthony Klinkatsis Kay Knight Anna McIntyre Howard Punchard Karen Rees Debbie Rigby Carlene Smith Andriana Vamvakinos

The distribution of mentors selected covered most of Australia, in both rural and metropolitan areas, though it was not possible to secure a mentor for Tasmania or the Northern Territory (though one of the mentors was located in NT during part of the project). This group of mentors also includes 2 state MMR co-ordinators, at least 4 MMR facilitators in divisions of general practice, 2 academic pharmacists and 1 drug information pharmacist. The selection of these pharmacists as mentors though, was based solely on their experience in the practical provision of medication management reviews in their own practice. The diversity of their areas of experience and expertise indicated the range of pharmacists that are involved in medication management review. During the course of the projects, the mentors were provided with administrative support from the AACP office staff and other enquiries and requests for support were fielded by the project pharmacist. As evident in the evaluations, the success of the mentor program can be attributed to the contribution of the mentors. Mentor Training The mentor training program was developed and delivered in collaboration with ACPP and DATIS and was held in Adelaide on Tuesday 22nd and Wednesday 23rd January 2002. The Australian College of Pharmacy Practice played a key role in the development and delivery of the mentor training program as they are:

a member of the project team ; a key provider of the Stage I training courses (both face-to-face workshop and

distance learning options) that lead to accreditation; and the providers of the contract examiners for ACPP of the Stage II

(examination) phase of the accreditation process


20

The Drug and Therapeutic Information Service were invited to contribute to the mentor training because of their experience and expertise in the area of educational visiting, principally using the skills of one health care professional to contribute to the education and learning of another. This experience in inter-professional relationships based on the exchange of useful and relevant information, and the training programs that have been developed to support and refine this skill, together with their experience in the development of adult learning programs, were used in the development of the mentor training program. As part of the evaluation of the overall program, the mentors were asked to rate the “value of the training day”, with these results: Table 1 Value of mentor training day

1 Very poor

2 3 4 5 Excellent

Mentor response

0 0 0 11 9

0

5

10

15

1 verypoor

2 3 4 5 excellent

Mentor response

Graph 1 Value of mentor training day Mentor Agreed Guidelines and Standards During the course of the training, the mentors were required to develop and agree a set of standards and guidelines for their work in the project. It was understood that there would need to be consistency of standard and approach during the project, and this needed to be agreed at the outset. The areas addressed in the Agreed Standard included:

candidate preparedness; definition of timely; determine a suitable time; general approach to candidate; resources

o references o forms o the “report”;

key points for case study 1 (the first case study in the current assessment book and the one to be encountered most frequently by the mentors); and

preferred print references – a collation of the preferred texts of the mentors The Agreed Standard, as developed during the mentor training, is included in the Appendices of this report.


21

Participation in Project During the course of the Mentor Support projects, the participation by candidates can be summarised as:

536 candidates (95 rural and 441 non-rural) registered to participate in the program; of these

358 candidates (65 rural and 293 non-rural) submitted their case study for the examination phase of the assessment process

many of this group also participated in the second phase of the program once they received accreditation and submitted their first “real world” case studies for mentor support

4 candidates (2 rural and 2 non-rural) were already accredited when they registered to participate and submitted their first case studies from the “real world” for mentor support as part of the program

Table 2 Participation in MSP Candidates

registered in MSP Cases received –

examination phase Cases received

– 1st reviews phase only

RURAL 95 65 2 NON-RURAL 441 293 2 COMBINED 536 358 4 Evaluation of Project The evaluation of the success and impact of the Mentor Support Program was drawn from the feedback received on five evaluation forms:

Candidate Evaluation Form 1 – Examination Phase Candidate Evaluation Form 2 – First Reviews Phase Mentor Evaluation Form 1 – Examination Phase Mentor Evaluation Form 2 – First Reviews Phase Mentor Evaluation Form 3 – Overall Program

An evaluation form accompanied each mentor support service, and the results were drawn from those returned to the AACP office. At several points during the course of the projects, letters were written to those candidates who had received a mentor service, but from whom their evaluation form has not been received. Similarly, regular correspondence with the mentors included prompts to forward their evaluation forms at the conclusion of work with each candidate (together with their claim for payment).


22

The following table represents the number and distribution of evaluation forms received at the AACP office. Table 3 Number of evaluation forms received RURAL NON-RURAL COMBINED Candidate Form 1 32 137 169

Candidate Form 2 10 41 51

Mentor Form 1 42 245 287

Mentor Form 2 7 39 46

Mentor Form 3 n/a n/a 19

0

50

100

150

200

250

300

CandidateForm 1

CandidateForm 2

MentorForm 1

MentorForm 2

MentorForm 3

RuralNon-ruralCombined

Graph 2 Number of evaluation forms received When reviewing the forms received, it is interesting that they do not always evaluate the same service. This is particularly evident in the First Reviews Phase of the project, where the candidates contacted the mentor of their own volition to seek support and guidance, and it may not have been clear to the mentor when the need for assistance had concluded (precipitating a claim for payment and completion of the relevant evaluation form). This is evident both in the variation between the number of Form 2s received, and in the spread of candidates it covers. In this instance:

51 Candidate Forms were received 46 Mentor Forms were received 73 candidates received a First Reviews Phase mentor service 24 of these received evaluations from both candidates and mentors

Mentor / Candidate contact The communication training that formed part of the mentor training emphasised that the benefit of the program was dependent on effective communication between the mentor and their candidate. Whilst acknowledging this, it was conceded that a significant amount of this communication may not be possible in a face-to-face setting, based on issues of time and/or distance. The allocation of the mentor to the candidate was determined by their geographic proximity, in order to facilitate face-to-face meetings whenever possible.


23

It is evident from the evaluations, that many of the mentor /candidate discussions did take place as face-to-face meetings and that there was also extensive use of phone, fax, email, and post to facilitate the exchange of information. When considering mentor / candidate contact:

613.5 hours of mentor service were provided (this was calculated from the invoices received from the mentors)

309 candidates participated in the service (again, as calculated from the mentors’ invoices)

resulting in 1.985 hours/candidate of mentor service The time required to provide the mentor service varied enormously, with the range recorded from 10 minutes to 7 hours. It is interesting that although this averages to two hours per candidate, as suggested in the initial model, the aberrations from this are quite infrequent, with the vast majority of services being provided within a 1 to 3 hour interval. Aside from extensive use of telephone and email both to arrange mutually convenient times to work together on the case/s and to provide the mentor service, the range of communication mediums used during the mentor program, as recorded on the evaluation forms, comprised: Examination Phase (drawn from Candidate and Mentor Evaluation Form 1): Table 4 Alternative communication mediums used in examination phase RURAL NON-RURAL COMBINED Candidate

Mentor Candidate Mentor Candidate Mentor

Face-to-face

4 3 33 62 37 65

Letter 1 4 1 19 2 23

Fax 5 15 8 50 13 65

Materials sent

0 6 6 30 6 36

First Reviews Phase (drawn from Candidate and Mentor Evaluation Form 2): Table 5 Alternative communication mediums used in 1st reviews phase RURAL NON-RURAL COMBINED Candidate

Mentor Candidate Mentor Candidate Mentor

Face-to-face

0 1 9 11 9 12

Letter 0 0 3 1 3 1

Fax 0 1 9 5 9 6

Materials sent

0 0 9 8 9 8


24

This is the main instance where there is a difference between the rural and the non-rural components of the respective projects. Not surprisingly, there is greater utilisation of distance communication mediums when the mentor service is provided in rural and remote areas compared to their non-rural counterparts as seen in Tables 4 and 5 above. Evaluations The results of the benefits and efficiencies of the Mentor Support Program were ascertained through the feedback received on the evaluation forms. Examination Phase - Candidate Evaluation From the perspective of the candidate commencing the examination phase of their accreditation, the main difficulties they encountered with the accreditation process are outlined below. Table 6 Candidates’ main difficulties with accreditation process Rural Non-rural Combined Time taken 20 102 122

Forms difficult to complete 17 73 90

Knowledge not up to date 7 35 42

Didn’t have a process to approach doing medication reviews

15 47 62

Lack of knowledge about which texts may assist

7 9 16

Lack of access to relevant texts 3 10 13

Other 8 31 39

020406080

100120140

Time taken Forms dif f icult tocomplete

Know ledge notup to date

Didn’t have aprocess to

approach doingmedication

review s

Lack ofknow ledge aboutw hich texts may

assist

Rural

Non-rural

Combined

Graph 3 Candidate’s main difficulties with accreditation process


25

The assistance provided by the mentor as evaluated by the candidates is detailed in the following table (these results are presented in a ranked format): Table 7 Candidates’ view of mentors’ suggestions Rural Non-rural Combined Prioritise problems found 20 82 102

More clearly state specific recommendations for management

19 78 97

Provide supporting evidence 18 72 90

Change approach of report to doctor 15 56 71

Find problems missed 10 53 63

Bring appropriate problems to doctor’s attention

13 49 62

Apply clinical judgement 12 30 42

Address compliance issues 4 34 38

Provide more detail 9 29 38

Change layout of report 4 23 27

Other 4 23 27

Advise monitoring 4 20 24

Provide less detail 6 14 20

Add patient details 1 18 19

Improve medication history 3 12 15

Include disclaimer about lesser knowledge of patient’s full circumstances

3 5 8

Provide contact details 0 2 2

The impact of the mentors’ contributions to the examination phase was ascertained using the following questions:

To what extent were you able to act on the advice provided by the mentor? Please rate the extent to which your mentor helped you gain insight into the

issues involved in conducting medication reviews? Please rate the extent to which your mentor helped you understand the

knowledge base and standard required to undertake medication reviews? Please rate the extent to which your mentor increased your confidence in

undertaking medication reviews? Please rate the likely impact of your mentor on the time taken for you to

complete your accreditation examination? Administrative support during the program. Willingness of mentor to address your concerns. Success of mentor in addressing your problems. Overall rating of the value to you of the mentor program.


26

The results of these questions are detailed in the following tables.

Table 8 Extent to which you were able to act on advice of mentor 1

Not at all 2 3 4 5

Followed all advice exactly

Rural 0 0 2 19 7

Non-rural 2 2 8 68 57

Combined 2 2 10 87 64

0102030405060708090

1 not at all 2 3 4 5 followedall advice

RURALNON-RURALCOMBINED

Graph 4 Extent to which you were able to act on advice of mentor Table 9 Insight into issues involved in medication reviews

1 No use

2 3 4 5 Excellent

Rural 0 3 5 9 15

Non-rural 1 6 15 48 63

Combined 1 9 20 57 68

010203040506070

1 n o u s e 2 3 4 5 ex ce lle n t

R U R A LN O N -R U R A LC O M B IN E D

Graph 5 Insight into issues involved in medication reviews


27

Table 10 Understand knowledge base and standard required 1

Very poor 2 3 4 5

Excellent Rural 0 2 7 12 11

Non-rural 3 4 21 55 50

Combined 3 6 28 67 61

1 very poor 2 3 4 5 excellent


Graph 6 Understand knowledge base and standard required Table 11 Increased confidence

1 Greatly reduced

2 3 4 5 Greatly

increased Rural 0 2 6 13 11

Non-rural 2 4 33 57 39

Combined 2 6 39 70 50

010203040506070

1 greatlyreduced

2 3 4 5 greatlyincreased


Graph 7 Increased confidence


28

Table 12 Impact on time taken to complete accreditation 1

Greatly increased

2 3 4 5 Greatly reduced

Rural 3 2 14 6 5

Non-rural 5 14 68 26 18

Combined 8 16 82 32 23

0

20

40

60

80

100

1 greatlyincre as ed

3 5 greatlyre duced

RURALNON-RURAL

COM BINED

Graph 8 Impact on time to complete accreditation Table13 Administrative support

1 Very poor

2 3 4 5 Excellent

Rural 1 2 15 8 5

Non-rural 7 12 69 24 17

Combined 8 14 84 32 22

0

20

40

60

80

100



Graph 9 Administrative support


29

Table 14 Willingness of mentor to address concerns 1

Very poor 2 3 4 5


Non-rural 1 2 14 30 86

Combined 1 4 19 37 104

020406080

100120



Graph 10 Willingness of mentor to address concerns Table 15 Success in addressing problems

1 Very poor

2 3 4 5 Excellent

Rural 0 2 2 11 17

Non-rural 1 2 13 51 67

Combined 1 4 15 62 84

0

20

40

60

80

100



Graph 11 Success in addressing problems


30

Table 16 Overall rating of mentor program 1

No value 2 3 4 5

Very valuable

Rural 0 2 3 7 20

Non-rural 3 5 11 29 88

Combined 3 7 14 36 108

020406080

100120

1 no value 2 3 4 5 veryvaluable


Graph 12 Overall rating of mentor program Examination Phase – Mentor Evaluation From the perspective of the mentor, the main issues addressed during the mentoring of the examination phase are detailed in the following table (these results are presented in a ranked format): Table 17 Main issues addressed by mentors Rural Non-rural Combined Communication 31 179 210

Medication review process 25 176 201

Application of clinical judgement 29 170 199

Problem identification 20 170 190

Use of references 20 152 182

Clinical knowledge 20 157 177

Other 3 67 70


31

The main strategies suggested by the mentors to candidates are described in the following table (these results are presented in a ranked format): Table18 Strategies suggested by mentors

Rural Non-rural Combined Prioritise problems found 21 201 222

Change tone of report to doctor 27 151 178

Further reading 17 119 136

Be more patient focused 31 150 191


Other 9 60 69


Further study 2 49 51

The mentors’ perception of the candidates’ response was ascertained using the following questions:

1. How receptive was the candidate to your suggestions? 2. Estimate the probable impact of your mentoring on the candidate’s

confidence? 3. Estimate the probable impact of your mentoring on the time taken by the

candidate to complete the examination?

These are the results from this section of the evaluation:

Table 19 How receptive was the candidate? 1

Very negative

2 3 4 5 Very

positive Rural 0 2 3 12 22

Non-rural 1 3 23 79 130

Combined 1 5 26 91 152

020406080

100120140160

1 verynegative

2 3 4 5 verypositive


Graph 13 How receptive was the candidate?


32

Table 20 Probable impact on candidate’s confidence 1

Greatly reduced

2 3 4 5 Greatly


Non-rural 2 4 46 128 55

Combined 2 5 50 149 68

020406080

100120140160

1 greatlyreduced



Graph 14 Probable impact on candidate’s confidence Table 21 Probable impact on time taken to complete the examination 1

Greatly increased


Rural 1 7 9 16 6

Non-rural 8 33 88 80 25

Combined 9 40 97 96 31

0

20

40

60

80

100

1 greatlyincreased

2 3 4 5 greatlyreduced


Graph 15 Probable impact on time taken to complete the examination


33

First Reviews Phase – Candidate Evaluation From the perspective of the candidate undertaking their first few medication management reviews in the “real world”, the main difficulties they encountered included (these results are presented in a ranked format): Table 22 Candidates’ main difficulties encountered in 1st reviews phase Rural Non-rural Combined Time taken 4 24 28

Constructing an effective report to doctor 4 19 23

Problems getting all the information needed

2 12 14

Identification of significant issues 0 13 13

Other 3 8 11

Lack of process for conducting reviews 0 10 10

Patient interview 0 5 5

The assistance provided by the mentor was evaluated as (these results are presented in a ranked format): Table 23 Candidates’ view of mentor’s suggestions Rural Non-rural Combined Prioritise problems found 1 22 23


2 20 22



1 17 18







0 5 5


Other 3 2 5






34


The candidates were also asked to assess the significance of their mentor’s suggestion/s, with the following results:

Table 24 Clinical significance of mentors’ suggestions Rural Non-rural Combined Detrimental 0 0 0

None 0 0 0

Minor 2 6 8

Optimising therapy 6 25 31

Major 0 8 8

Potentially life saving 0 0 0

0

5

10

15

20

25

30

35

Detrimental None Minor Optimisingtherapy

Major Potentiallylife saving


Graph 16 Clinical significance of mentor’s suggestions The impact of the mentors’ contribution to the first reviews phase was ascertained using the following questions:

1. Please rate the extent to which your mentor increased your confidence in undertaking medication reviews?

2. Extent to which the input of your mentor allowed you to improve the quality of the review

3. Willingness of mentor to address your concerns 4. Successfulness of mentor in addressing your problems 5. Overall rating of the value to you of having your initial reviews assessed


35

These are the results of this section of the evaluation: Table 25 Increased confidence 1

Greatly reduced

2 3 4 5 Greatly


Non-rural 0 0 4 21 17

Combined 0 0 5 24 21

0

5

10

15

20

25

1 greatlyreduced



Graph 17 Increased confidence Table 26 Improved quality of review 1

Very poor 2 3 4 5


Non-rural 0 1 8 18 13

Combined 0 1 11 19 17

0

5

10

15

20



Graph 18 Improved quality of review


36

Table 27 Willingness of mentor to address your concerns 1

Very poor 2 3 4 5


Non-rural 0 0 1 8 32

Combined 0 0 2 9 38

0

10

20

30

40



Graph 19 Willingness of mentor to address your concerns Table 28 Success of mentor to address problems 1

Very poor 2 3 4 5

Excellent Rural 0 0 2 6

Non-rural 0 0 1 12 27


0

10

20

30

40



Graph 20 Success of mentor to address problems


37

Table 29 Overall rating of initial reviews mentoring 1

Very poor 2 3 4 5

Excellent Rural 0 0 2 6

Non-rural 0 0 1 13 26


05

101520253035



Graph 21 Overall rating of initial reviews mentoring


38

First Reviews Phase – Mentor Evaluation From the perspective of the mentor, the main input needed during the mentoring of the First Reviews phase was (these results are presented in a ranked format): Table 30 Main mentor input needed in 1st reviews phase Rural Non-rural Combined




2 17 19


2 16 18




Other 2 12 14

Prioritise problems found 3 10 13





1 7 8






39

The mentors assessed the significance of their suggestions with respect to the candidates’ reviews as:

Table 31 Clinical significance of mentors’ suggestions Rural Non-rural Combined Detrimental 0 0 0

None 1 0 1

Minor 2 2 4


Major 2 9 11


05

1015202530354045

RURAL NON-RURAL COMBINED

DetrimentalNoneMinorOptimising therapyMajorPotentially life saving

Graph 22 Clinical significance of mentor’s suggestions The mentors assessed the quality of the reviews as submitted to them as: Table 32 Quality of reviews submitted to mentor 1

Very poor 2 3 4 5




0

5

10

15

20



Graph 23 Quality of reviews submitted to mentor


40

Overall Program – Mentor Evaluation The mentors were asked to provide feedback on the Mentor Support Program as a whole, including the following areas:

Your experience o Administrative support received during the program o Value of training day o Overall willingness of candidates to attend to your advice o Overall enjoyment of your interaction with candidates

Impact o Overall impact of your service on candidates’ confidence o Overall impact of your service on time taken by candidates to

complete their examination o Overall impact of your service on quality of candidates’ initial reviews

This is the feedback received from the mentors: Administrative support received during the program Table 33 Administrative support

1 Very poor

2 3 4 5 Excellent

0 3 5 7 4

01234567

1 v e ry p o o r 2 3 4 5 e x c e l le n t

Graph 24 Administrative support Table 34 Value of training day

1 Very poor

2 3 4 5 Excellent

0 0 0 11 9

02468

1 01 2

1 v e ry p o o r 2 3 4 5 e x c e l le n t

Graph 25 Value of training day


41

Overall willingness of candidates to attend to your advice Table 35 Willingness of candidates

1 Very poor

2 3 4 5 Excellent

0 0 0 10 10

1 v e r y p o o r 2 3 4 5 e x c e l le n t

Graph 26 Willingness of candidates Overall enjoyment of your interaction with candidates Table 36 Overall enjoyment of interaction with candidates

1 Very poor

2 3 4 5 Excellent

0 0 1 6 13

0

5

1 0

1 5

1 v e r y p o o r 2 3 4 5 e x c e l le n t

Graph 27 Overall enjoyment of interaction with candidates


42

Overall impact of your service on candidates’ confidence Table 37 Overall impact on confidence

1 Greatly reduced

2 3 4 5 Greatly

increased 0 0 1 11 7

02468

1 01 2

1 g r e a t lyr e d u c e d

2 3 4 5 g r e a t lyin c r e a s e d

Graph 28 Overall impact on confidence Overall impact of your service on time taken by candidates to complete their examination Table 38 Overall impact on time taken to complete examination

1 Greatly increased


0 2 6 9 2

02468

1 0

1 g re a t lyin c re a s e d

2 3 4 5 g re a t lyre d u c e d

Graph 29 Overall impact on time taken to complete examination


43

Overall impact of your service on quality of candidates’ initial reviews Table 39 Overall impact on quality of initial reviews

1 Greatly reduced

2 3 4 5 Greatly


02468

1 01 21 4

1 g r e a t lyr e d u c e d

2 3 4 5 g re a t lyin c r e a s e d

Graph 30 Overall impact on quality of initial reviews Overall impact on confidence and time taken to complete assessment examination One of the indicators of success identified in the project proposals was the potential impact of the Mentor Support program on the confidence of the candidates and the anticipated time that would be required to complete the assessment examination. It is pleasing to note that the Mentor Support program achieved both these objectives: Impact of mentoring on confidence (where C = candidate; M = mentor 1 = greatly reduced; 5 = greatly increased): Table 40 Summary table of impact of mentoring on confidence

C1 M1 C2 M2 C3 M3 C4 M4 C5 M5 Rural 0 0 2 1 6 4 13 21 11 13

Non-rural 2 2 4 4 32 46 57 126 39 55

Combined 2 2 6 5 38 50 70 147 50 68

020406080

100120140160

C1 M1 C2 M2 C3 M3 C4 M4 C5 M5


Graph 31 Summary table of impact of mentoring on confidence


44

Impact of mentoring on anticipated time required to complete assessment examination (where C = candidate; M = mentor 1 = greatly increased; 5 = greatly reduced): Table 41 Summary table of impact of mentoring on time required to

complete assessment examination C1 M1 C2 M2 C3 M3 C4 M4 C5 M5

Rural 3 1 2 7 14 9 6 16 5 6

Non-rural 5 8 14 33 67 86 26 80 18 25

Combined 8 9 16 40 81 95 32 96 23 31

0

20

40

60

80

100

C1 M1 C2 M2 C3 M3 C4 M4 C5 M5


Graph 32 Summary table of impact of mentoring on time required to

complete assessment examination Future Programs Included in the evaluations, both the candidates and the mentors were asked to provide comment about the structure of Mentor Support Program, including questions about a suitable fee (if external funding support were not available) and requests for suggestions about how the program may be improved in the future. The following is a summary of the responses received from the candidates:


45

Do you believe future candidates for accreditation would be prepared to pay for a mentor support service? Table 42 Future candidates prepared to pay for mentor service?

Exam – Yes

Exam – No

1st reviews – Yes

1st reviews – No

Rural 21 5 5 2

Non-rural 75 51 26 12

Combined 96 56 31 14

0

20

40

60

80

100

Exam – YES Exam – NO 1st reviews –YES

1st reviews – NO


Graph 33 Future candidates prepare to pay for mentor services? Extent of Payment When the candidates were asked to identify a reasonable fee for a mentor support service, the following range of responses was received:

$20/hr to $80/hr $30 to $200 (flat fee) $30/case study to $50/case study

The comments that accompanied the nominated fee are more instructive, with many volunteering that “accreditation is too expensive” and that “the mentor service should be included in the fee”.


46

When the mentors were asked to identify a reasonable fee for a mentor support service, the following response was received: Table 43 Mentors’ reasonable fee for mentor service

$50/hr $60/hr $70/hr $75/hr $80/hr $100/hr Examination Phase 7 2 3 4 1 0

First Reviews Phase 6 2 2 5 2 1

01234567

$50/hr $60/hr $70/hr $75/hr $80/hr $100/hr

EXAMINATION PHASEFIRST REVIEWS PHASE

Graph 34 Mentors’ reasonable fee for mentor service When asked about potential alternate structures, the candidates in the examination phase responded: Extent to which the service would be improved by having mentors situated in the same geographic area of the candidate, to allow face-to-face meetings. Table 44 Potential benefit of face-to-face meetings

1 Very

detrimental

2 3 4 5 Extremely important

Rural 0 0 11 12 9

Non-rural 0 3 30 42 58

Combined 0 3 41 54 67

0

20

4060

80

1 verydetrimental

3 5 extremelyimportant

RURAL

NON-RURAL

COMBINED

Graph 35 Potential benefit of face-to-face meetings


47

Extent to which the program would be affected by replacing expert experienced mentors (as used in this project) with general peer support from other accredited pharmacists whose specific expertise may vary Table 45 Potential impact of general peer support from other accredited

pharmacists 1

Very detrimental

2 3 4 5 Extremely positive

Rural 2 11 15 2 2

Non-rural 8 34 53 28 8

Combined 10 45 68 30 10

010203040506070

1 verydetrimental

2 3 4 5 extremelypositive


Graph 36 Potential impact of general peer support from other accredited pharmacists To what extent would the program be effected by replacing the one to one service your were provided, with local meetings with other pharmacists, facilitated by pharmacists and/or doctors? Table 46 Potential impact of using local meetings

1 Very

detrimental


Rural 1 7 11 10 3

Non-rural 4 23 46 42 15

Combined 5 30 57 52 18

0102030405060

1 verydetrimental

2 3 4 5 extremelypositive


Graph 37 Potential impact of using local meetings


48

Participation in Project During the course of the mentor projects: Table 47 Number of participants and resultant accredited pharmacists

Candidates registered in MSP

Cases received & mentors allocated

Accredited as result of mentor service

Rural 95 65 33

Non-rural 441 293 157

Combined 536 358 190

0

100

200

300

400

500

600

Registered candidates Mentors allocated Accredited as result ofMSP


Graph 38 Number of participants and resultant accredited pharmacists In addition to this:

8 pharmacists registered for the Mentor Support program and received accreditation without using the mentor service

4 accredited pharmacists registered and used the Mentor Support program for the First Reviews phase only


49

Results Summary In summary, the activities of the mentor support project/s can be quantified as:

536 (95 rural + 441 non-rural) pharmacists registered their interest in the Mentor Support Program

358 (65 rural + 293 non-rural) sent their case studies in for the examination phase and were allocated mentors

210 (33 rural + 177 non-rural) completed and submitted their case study assessment, of these: 174 (26 rural + 149 non-rural) passed – pass rate 83% 12 (5 rural + 7 non-rural) were asked to re-submit and were

subsequently passed 4 (2 rural + 2 non-rural) joined the Mentor Support Program after

initially failing themselves and were subsequently passed after their involvement in the program

this represents a total of 190 (33 rural + 157 non-rural) accredited pharmacists as a result of the Mentor Support Program – overall pass rate 91%

Of those candidates enrolled in the Mentor Support Program but yet to become accredited, 20 of these have attempted their accreditation unsuccessfully, of these:

16 (3 rural + 13 non-rural) have been asked to re-submit their cases and have been involved in the mentor program

4 (2 rural + 2 non-rural) joined the mentor program after initially failing themselves and have requested the support of a mentor to gain accreditation

Based on the evaluations received from mentors and their candidates, the qualitative results of the mentor support project/s can be summarised as: Examination Phase The candidates rated the following areas as the main difficulties encountered in completing the examination:

Time taken (72%) Forms difficult to complete (53%) Didn’t have a process approach doing medication reviews (37%) Knowledge not up to date (25%)

They identified the following areas as those prompting suggestions from their mentor:

Prioritise problems (36%) More clearly state specific recommendations for management (31%) Provide supporting evidence (31%) Change approach of report to doctor (25%) Bring appropriate problems to doctor’s attention (22%) Find problems missed (22%)

The mentors nominated the following areas as the main issues dealt with:

Communication (72%) Medication review process (69%) Application of clinical judgement (69%) Use of references (63%) Clinical knowledge (61%)


50

They made the following suggestions to the candidates: Prioritise problems found (77%) Be more patient focused (66%) Apply clinical judgement (65%) Change tone of report to doctor (61%) Further reading (47%)

In considering how these findings may inform future decisions, it is interesting to note that the main areas of concern and comment pertain to the functionality of the medication review process, and seldom to deficiencies in clinical knowledge. Anecdotally, this has generally been accepted, and these findings verify that it is the administrative “how to’s” that have impeded the accreditation process, rather than a lack of clinical knowledge. First Reviews Phase The candidates rated the following areas as the main difficulties in completing their first reviews:

Time taken (55%) Constructing an effective report to doctor (45%) Problems getting all the information needed (27%) Lack of process for conducting reviews (20%)


Prioritise problems (45%) Bring appropriate problems to doctor’s attention (43%) Provide supporting evidence (35%) More clearly state specific recommendations for management (35%) Apply clinical judgement (27%) Change approach of report to doctor (22%) Change layout of report to doctor (20%)

They made the following suggestions to the candidates:

Provide supporting evidence (48%) More clearly state specific recommendations for management (41%) Bring appropriate problems to doctor’s attention (39%) Change layout of report (39%) Apply clinical judgement (35%) Find problems missed (30%) Prioritise problems found (28%)

As with the examination phase, it is of interest to note that the principal areas of concern and comment from candidates and mentors alike pertain to those of process, rather than clinical knowledge. When assessing the significance of the mentor’s contribution to the first reviews phase, the following feedback was received:

Potentially life saving (x1) Major significance (x19) Optimising therapy (x73) Minor significance (x12)


51

This demonstrates the importance of suitable mentor support for this stage in an accredited pharmacist’s involvement in medication management review. The identification of a potentially life saving intervention highlights the importance of making this type of service available (even mandated?) for accredited pharmacists as they embark on their own reviews in the “real world”. The initial aims of the Mentor Support Project/s included the utilisation of a mentor service to increase the number of accredited pharmacists, together with an endeavour to raise the confidence and ability of community pharmacists to participate in medication management reviews. When comparing the pass rate of those pharmacists who have undertaken the case study assessment process without the support of a mentor to the results obtained during the Projects, it is clear that there has been a significance impact on the pass rate for those pharmacists who have attempted their accreditation assessment with the support of a mentor: 91% pass rate compared to ~60% pass rate (this is drawn all those pharmacists who have attempted the case study assessment without participating in the Mentor Support Program, thus representing an historical and concurrent comparison. In effect, this group of pharmacists provides a control group against which the results of the Mentor Support Program can be compared.)


52

Conclusion and Recommendations

This section of the report follows the sequence of the material included in the Results chapter. Many of the conclusions are clear from the results presented and are summarised here. The consequent recommendations are included in the relevant area and may also make reference to other areas or results. Mentors As previously mentioned, the success of the Mentor Support Program can be attributed to the contribution and commitment of the mentors involved. The task requested of the mentors was to provide guidance and advice about the case study prepared for the assessment examination and/or the candidate’s first few case studies in the “real world”. For this purpose, the mentors selected were chosen because of their experience in the provision of medication management reviews rather than clinical knowledge base or their teaching experience. Although some of the mentors are also drug information, hospital or academic pharmacists, this is not the reason they were invited to be included in the project. The majority of mentors involved were invited to contribute because of their experience and expertise in the area of medication management review. The feedback received from candidates and mentors alike, indicates this to be a suitable criteria of selection for this type of mentor service.

Recommendation One: That any future mentor support program provided to assist pharmacists through the assessment and early reviews phase of their involvement in medication management review, utilise the services of mentors who have considerable experience and expertise in this area.

Mentor Training The purpose of the mentor training was to establish an agreed standard and guidelines for the mentor service in an endeavour to provide a consistent quality of service across the duration of the project. It was also the only opportunity to provide the mentors with training in some of the more challenging areas of a mentor service, including the principles of adult learning, hints and alerts for distance communication, a review of the assessment process and the expectations of the examiners, the use of references and time management skills. They were also encouraged to use each other for collegiate support during the course of the project.

Recommendation Two: That training be provided for mentors involved in this area to address the issues of:

expectation of the examiners in the assessment process;


53

principles of adult learning; distance communication; use of references; and time management skills.

Agreed Mentor Standards and Guidelines The development of the agreed standards and guidelines at the mentor training proved an important element in ensuring a consistent quality of mentor service was provided. Though infrequent, when feedback about the service was not favourable, it usually pertained to a breach in the agreed standards, most commonly the use of a timely response (this can often be attributed to holidays or other factors that caused a delay in the mentor’s response to receiving the candidate’s case study). It was anticipated that the development of an agreed standard would be a means of moderating the service provide by the mentors and the results indicate this to have been achieved.

Recommendation Three: That any future mentor support program provided to assist pharmacists through the assessment and early reviews phase of their involvement in medication management review, develop an agreed standard and guidelines to ensure a consistency of quality across the program, and that these guidelines be reviewed periodically to ensure currency and suitability.

Mentor / Candidate Contact Although both the nature and extent of the mentor / candidate contact was extremely varied, the feedback received indicates it to have been a success. The flexibility in the medium of communication and the amount of time utilised to provide the service, seems to have been able to accommodate the very varied needs and expectations of the candidates. The preferred means of communication for the mentoring service was a face-to-face meeting. This is to be expected when considering the opportunities for enhanced communication that are available in this setting. When this option was not available, normally through considerations of time and/or distance, a range of other communication mediums were employed at the discretion of the candidates and their mentors. These included telephone, email, fax, and post. These avenues were also used for the transfer of information both before and after the main mentor / candidate contact, regardless of the communication medium used for this element of the service.

Recommendation Four: That any future mentor support program provided to assist pharmacists through the assessment and early reviews phase of their involvement in medication management review, encourage the use of face- to-face meetings between the candidate and their mentor whenever possible, and that all other means of communication be utilised as appropriate and convenient for both parties.


54

Evaluations Examination Phase The candidates rated the following areas as the main difficulties encountered in completing the examination:

Time taken (72%) Forms difficult to complete (53%) Didn’t have a process approach doing medication reviews (37%) Knowledge not up to date (25%)


Prioritise problems (36%) More clearly state specific recommendations for management (31%) Provide supporting evidence (31%) Change approach of report to doctor (25%) Bring appropriate problems to doctor’s attention (22%) Find problems missed (22%)

The mentors nominated the following areas as the main issues dealt with:

Communication (72%) Medication review process (69%) Application of clinical judgement (69%) Use of references (63%) Clinical knowledge (61%)


Prioritise problems found (77%) Be more patient focused (66%) Apply clinical judgement (65%) Change tone of report to doctor (61%) Further reading (47%)

In considering how these findings may inform future decisions, it is interesting to note that the main areas of concern and comment pertain to the functionality of the medication review process, and seldom to deficiencies in clinical knowledge. Anecdotally, this has generally been accepted, and these findings verify that it is the administrative “how to’s” that impeded the accreditation process, rather than a lack of clinical knowledge.

Recommendation Five: That the providers of the Stage I courses designed to prepare pharmacists for the examination assessment for accreditation, while designing their courses to fulfil any current or future required competencies that pertain to the provision of MMR, be encouraged to adapt their courses to more fully address the practicalities of providing medication management review services. This would include issues relating to:

The use of the relevant forms Report writing requirements How to prioritise problems and recommendations How to use clinical judgement


55

First Reviews Phase The candidates rated the following areas as the main difficulties in completing their first reviews:

Time taken (55%) Constructing an effective report to doctor (45%) Problems getting all the information needed (27%) Lack of process for conducting reviews (20%)


Prioritise problems (45%) Bring appropriate problems to doctor’s attention (43%) Provide supporting evidence (35%) More clearly state specific recommendations for management (35%) Apply clinical judgement (27%) Change approach of report to doctor (22%) Change layout of report to doctor (20%)


Provide supporting evidence (48%) More clearly state specific recommendations for management (41%) Bring appropriate problems to doctor’s attention (39%) Change layout of report (39%) Apply clinical judgement (35%) Find problems missed (30%) Prioritise problems found (28%)

As with the examination phase, it is of interest to note that the principal areas of concern and comment from candidates and mentors alike pertain to those of process, rather than clinical knowledge. When assessing the significance of the mentor’s contribution to the first reviews phase, the following feedback was received:

Potentially life saving (x1) Major significance (x19) Optimising therapy (x73) Minor significance (x12)

This demonstrates the importance of suitable mentor support for this stage in an accredited pharmacist’s involvement in medication management review. The identification of a potentially life saving intervention highlights the importance of making this type of service available (even mandated?) for accredited pharmacists as they embark on their own reviews in the “real world”. To forego the opportunity of this degree of support for a pharmacist and scrutiny of their report, as they embark on this area of practice has the potential to compromise the quality of the service provided. A mentor service for the first reviews phase of a pharmacist’s involvement in this area, is consistent with the reasoning behind the pre-registration training that is required after graduation.

Recommendation Six: That utilisation of a mentor support program be strongly encouraged (or mandated?) for the first reviews phase of an accredited pharmacists


56

involvement in this area, to ensure consistency of quality and the clinical integrity of the service.

Overall Evaluation from Mentors The overall feedback from the mentors about the Mentor Support projects indicates the provision of a valuable program from every perspective. The indicators assessed include:

willingness of candidates to attend to mentor advice; enjoyment of interaction with candidates; impact on candidates confidence; and impact on anticipated time for candidate to complete assessment

examination. The many comments received during the course of the project, from mentors and candidates alike, that indicated an appreciation of the opportunity to interact with another pharmacist, was remarkable. Many community pharmacists work in relative professional isolation. This is regardless of the number of pharmacists they may work “with”, as the workload often precludes meaningful discussion about clinical matters, or the exploration of Quality Use of Medicines issues for particular patients or situations. The Mentor Support Program has provided the opportunity for many pharmacists to interact at a professional level, and for many this may have been a new experience.

Recommendation Seven: That the appreciation of collegiate support identified in this project be promulgated to those pharmacy organisations who may be able to develop a mechanism to encourage and facilitate this further, eg Australian Association of Consultant Pharmacy, The Australian College of Pharmacy Practice, the Pharmaceutical Society of Australia, The Pharmacy Guild of Australia through its MMR facilitator program, and the Alumni Associations of the respective Pharmacy Schools etc.

Impact on confidence and time taken to complete reviews As one of the disincentives to accreditation is the pharmacists’ lack of confidence in attaining the standards required, it is pleasing to note that the results of the Mentor Support Program indicated a positive result in this area. The anticipated impact on the time required to complete the assessment examination was also impacted on favourably. Future Programs The potential structure of future support programs for pharmacists was explored in the evaluation phase of this project with a varied response. The strongest support was for a structure that maximised the opportunity for the mentor and candidate to meet face-to-face. The other structures proposed included small groups and other peer support structures, where the possibility of “learning from each other” is more readily achieved. It can be anticipated that as pharmacists become more experienced in the provision of quality medication management reviews, that the nature of mentor / peer support


57

may change, and that alternate structures may be more suited to this stage and type of practice need. However, in the accreditation and first reviews phase, it is preferred that a more formalised Mentor Support service be provided, to ensure consistency of quality and the integrity of clinical standards. This would not preclude a candidate from participating in other forms of peer support, but the clinical significance of the mentor involvement in the first reviews phase of this project is quite compelling.

Recommendation Eight: That a Mentor Support Program be developed and made available to those pharmacists undergoing their assessment examination and first reviews phase, and that the structure of this program facilitate one to one meetings whenever possible.

The issue of payment for a future mentor service was canvassed in the evaluations. Although the majority of candidates indicated they would be prepared to pay for a mentor service, they also indicated that the cost involved in undertaking the accreditation process was already quite high and that the mentor service should be part of the fees already paid. To provide a mentor service without charging the candidate for their involvement would require considerable financial support. It may be possible to identify a degree of support for a mentor support program if the candidates were to contribute a co-payment (eg Third Community Pharmacy Agreement funds may be available for this area of activity).

Recommendation Nine: That financial support be identified to provide a Mentor Support Service that minimises the payment required of the candidates.

Number of Participants and Accredited Pharmacists The Mentor Support Program has provided mentor services to 358 pharmacists and resulted in 190 pharmacists gaining their accreditation to provide medication management reviews during the course of the project. It is hard to predict how many of those yet to submit their assessment examination will be successful, as a result of the mentor service they have already received. Of the 1440 accredited pharmacists in Australia at the time of this report, the Mentor Support Program has assisted the accreditation of approximately 13% of these, with the potential impact to continue beyond the scope of this project. The feedback received, indicates that many of the candidates have a requested an ongoing association with their mentor on the basis of a private business arrangement, and this will further increase the benefit of the program.


58

Appendices Appendix 1. Mentor Agreed Guidelines and Standards

for the Mentor Support Project Appendix 2. Mentor Support Project Evaluation Appendix 2.1.1: Candidate Evaluation Form – Examination Phase

Appendix 2.1.2: Candidate Evaluation Form – Examination Phase Results

Appendix 2.2.1: Candidate Evaluation Form – First Review

Appendix 2.2.2: Candidate Evaluation Form – First Review Results

Appendix 2.3.1: Mentor Evaluation Form – Examination Phase

Appendix 2.3.2: Mentor Evaluation Form – Examination Phase Results

Appendix 2.4.1: Mentor Evaluation Form – First Review

Appendix 2.4.2: Mentor Evaluation Form – First Review Results

Appendix 2.5.1: Mentor Evaluation Form – Overall Program

Appendix 2.5.2: Mentor Evaluation Form – Overall Program Results


59

Appendix 1. Mentor Agreed Guidelines and Standards

for the Mentor Support Project Candidate preparedness Although medication management is a core pharmacist activity, it is acknowledged that sometimes your advice may be that the candidate is not yet ready to sit their accreditation examination and that specific update courses may assist their preparation. Definition of timely Timely = on receipt of case 1 from AACP, contact your mentee, preferably within 1 week – don’t forget the candidate may allow further progress on their examination to lapse until you contact them – but definitely within 2 weeks. Determine a suitable time Ring (or email if candidate has nominated this preference) to organise a mutually suitable time. Probably two serious attempts to set up a time are enough effort on the mentor’s part. But let AACP know if you are giving up, so another candidate can be offered their place. Introduction phase Establish your credibility. You probably won’t find this difficult. It may be some time since they enrolled as a candidate, but they will be keenly expecting your call, as they will have just been informed by AACP you will be their mentor (this will occur at the same time that AACP sends you the candidate’s case 1). Establish the candidate’s background, pharmacy related experience/s and the training course they did. Remember that candidates enrolled in early years may have gone straight into the examination stage based on their pharmacy experience – which may not actually have prepared them for the process of medication management reviews. Ask how they felt they went with the case – what were their issues/concerns. Explain that your role is not to give them the answers, but to guide their approach and suggest resources that will assist them to become accredited pharmacists. General approach of candidate The focus should be on the patient/consumer, not on the individual drugs – the pharmacist may ’forget’ this is the way they normally approach their pharmacy practice, overwhelmed by the examination situation. Use their day-to-day experience with consumer medication issues to anchor them. The patient is in a complete situation – taking medications is just one small part of their total health care, let alone their total situation. Reinforce that no doctor has deliberately prescribed a medication that they do not believe to be the best treatment for their particular patient, based on their knowledge and understanding. As it is for us all, it is extremely difficult to remember all new


60

relevant research and incorporate it immediately into practice and relate it to a specific patient. Resources See Preferred Print / CD ROM Resources list attached See Preferred Internet resources you will develop Disease states: Internet Medscape UQ library subscription Interventions Advise candidates to re read the examination instructions carefully. Constipation, sleep disorders, falls are important issues to deal with – consider non pharmacological therapy Forms Ensure the person understands the forms eg the ‘Red Form’ is the brainstorming area (document thoughts, possibilities etc), the ‘Blue Form’ is the final report to the facility, nursing staff and doctor (it is to be specific and concise and reflects clinical judgement, priorities, suggestions/recommendations etc). Stress that they do not have to complete the outcomes section of the final report or the B, C, D, & E sections on the Red Form. You may have to clarify the difference between the requirements of an examination (where you have to demonstrate your competence, using this non individualised method) and the real world (where you would combine written with verbal communication built on an established relationship). Remind the candidate that the examiner needs insight into their thinking and that they can’t assume the examiner will realise they will do a great job in the real world, if they haven’t demonstrated this on paper. The Report If the report language and approach are not optimal, suggest the candidate completes (or re reads) the Communication and Concordance Module. Stress that they should consider their review from the doctor’s perspective. Is the language appropriate from one health professional to another health professional? The candidate should be advised not to use phrases such as ‘cease’ or’ stop’. Encourage the use of phrases such as ‘may I suggest’ or ‘would it be appropriate to’ Statement (incontrovertible fact) – because (why believe drug related) – for example (suggested solution)


61

By applying their clinical judgement, candidates should prioritise the problems and issues they identified and listed on the ‘Red Form’, appropriately grouped, and include these in their final report, the Blue Form. Candidates’ recommendations should be clearly based on sound reasoning, indicating the evidence using as appropriate, doctor-credible references/evidence – NPS; MJA; Therapeutic Guidelines; Australian Family Physician; AMH. It may assist to go through the thought process of the candidate – how they got to their recommendation. Finally After discussion is completed, determine if they would like you to forward your written comments on the case.


62

Appendix 2. Mentor Support Project Evaluation Appendix 2.1.1: Candidate Evaluation Form – Examination Phase


Appendix 2.2.1: Candidate Evaluation Form – First Review




Appendix 2.4.1: Mentor Evaluation Form – First Review

Appendix 2.4.2: Mentor Evaluation Form – First Review Results




63

Appendix 2.1.1: Candidate Evaluation Form – Examination Phase

Candidate Name: ________________________________ MRN: ________

Please complete the following evaluation after your first case study has been

discussed with your mentor. Please return to:

Mentor Support Program Australian Association of Consultant Pharmacy

PO Box 7071 Canberra Business Centre ACT 2610

Question Response Mentor contact Number of telephone discussions with mentor

Number of emails exchanged with mentor

Other mentor contact

Estimated total time involved in mentor contact

Mentor input What were the main difficulties you found in completing the examination?

Time taken Forms difficult to complete Knowledge not up to date Didn’t have a process to approach doing

medication reviews Lack of knowledge about which texts may

assist Lack of access to relevant texts Other (please specify):

What did your mentor suggest you do? Add patient details Improve medication history Bring appropriate problems to doctor’s

attention Provide supporting evidence More clearly state specific

recommendations for management Find problems missed Provide less detail


64

Provide more detail Prioritise problems found Apply clinical judgment Address compliance issues Advise monitoring Change approach of report to doctor Change layout of report Include disclaimer about lesser knowledge

of patient’s full circumstances Provide contact details Other (please specify:

Impact To what extent were you able to act on the advice provided by the mentor?

1= not at all 5= followed all advice exactly 1 2 3 4 5

If you did not utilise the mentor’s advice, why not? Did not agree

No time Advice impractical Couldn’t understand Other (please specify):

If you had more than one discussion with your mentor, to what extent did the further discussion add additional value for you?

1 = no extra 5= very valuable 1 2 3 4 5

Please rate the extent to which your mentor helped you gain insight into the issues involved in conducting medication reviews

1 = no use 5= excellent 1 2 3 4 5

Please rate the extent to which your mentor helped you understand the knowledge base and standard required to undertake medication reviews

1= very poor 5= excellent 1 2 3 4 5

Please rate the extent to which your mentor increased your confidence in undertaking medication reviews

1 = greatly reduced 5= greatly increased 1 2 3 4 5

Please rate the likely impact of your mentor on the time taken for you to complete your accreditation examination

1 = greatly increased 5= greatly reduced 1 2 3 4 5

Administrative support received during the program 1 = very poor 5= excellent 1 2 3 4 5


65

Willingness of mentor to address your concerns 1 = very poor 5= excellent

1 2 3 4 5

Success of mentor in addressing your problems 1 = very poor 5= excellent 1 2 3 4 5

Overall rating of the value to you of the mentor program

1 = no value 5= very valuable 1 2 3 4 5

Potential value of the program to other pharmacists 1 = no value 5= very valuable 1 2 3 4 5

Payment Do you believe future candidates for accreditation would be prepared to pay for such a service?

□Yes □No

What would be a reasonable fee for such a service as you received?

$

Structure of future programs Extent to which the service would be improved by having mentors situated in the same geographical

area of the candidate, to allow face-to-face meetings

1= very detrimental …… … 5 = extremely important

1 2 3 4 5

Extent to which the program would be effected by replacing expert experienced mentors (as used in this project) with general peer support from other accredited pharmacists whose specific expertise may vary

1= very detrimental ……… 5 = extremely positive 1 2 3 4 5

To what extent would the program be affected by replacing the one to one service you were provided, with local meetings with other pharmacists, facilitated by pharmacists and/or doctors?


Should the Australian Association of Consultant Pharmacy (AACP) continue to conduct a mentor support program?

□Yes □No

Is there a more suitable structure for administering a mentor program?

□Yes (please specify): □No

Question Response Other suggestions for the structure of a mentor program in the future


66


Mentor Input What were the main difficulties you found in completing the examination?

Rural Non-rural Combined Time taken 20 102 122

Forms difficult to complete 17 73 90

Knowledge not up to date 7 35 42

Didn’t have a process to approach doing medication reviews

15 47 62

Lack of knowledge about which texts may assist

7 9 16

Lack of access to relevant texts 3 10 13

Other 8 31 39

My first case study did not seem to have much wrong with it Lack of practical clinical knowledge Knowledge about the cases from not only a pharmacist’s point of view,

general geriatric issues in a nursing home environment. I tended to look at the cases only from a medication point of view, not the case as a whole

There really wasn’t anywhere to go (eg book, website etc) that gave me a real idea of what was expected. I didn’t know how much detail was required re info, references etc. This makes it very time-consuming and very, very frustrating.

Difficult to evaluate info from pathology tests i.e. their meaning Some confusion about what to put where – especially in the HMR forms. I

could design my own – they are only there to organise info and to help present in the letter etc

Personal lack of self confidence Although attended the weekend workshop, nothing was taught about filling in

the medication review forms I needed encouragement to go on more than anything else


67

What did your mentor suggest you do?

Rural Non-rural Combined Add patient details 1 18 19



13 49 62



19 78 97











3 5 8


Other 4 23 27

Confirmed I was on the right track with clinical assessment Generally the mentor was very encouraging and gave advice on some

extension of the problems & recommendations Look at the circumstance of the case, not only from a drug perspective.

Consider patient’s lifestyle, compliance, mental status etc Number the points and issues and follow it through the case study sheets Use of forms needed improving / detection of problems / references Focus on the social aspects of the patient as well as the drug related issues

eg. is the patient new to the nursing home ? etc Look at situation as whole person not just drugs / suggested other texts

Impact To what extent were you able to act on the advice provided by the mentor?

1 Not at all

2 3 4 5 Followed all advice exactly

Rural 0 0 2 19 7

Non-rural 2 2 8 68 57


68

Combined 2 2 10 87 64

If you did not utilise the mentor’s advice, why not?

Rural Non-rural Combined Did not agree 0 3 3

No time 0 2 2

Advice impractical 0 0 0

Couldn’t understand 0 2 2

Other 1 5 6

If you had more than one discussion with your mentor, to what extent did the further discussion add additional value for you?

1 No extra

2 3 4 5 Very

valuable Rural 2 0 1 8 8

Non-rural 8 2 9 23 31

Combined 10 2 10 31 39

Please rate the extent to which your mentor helped you gain insight into the issues involved in conducting medication reviews?

1 No use

2 3 4 5 Excellent

Rural 0 3 5 9 15

Non-rural 1 6 15 48 63

Combined 1 9 20 57 68


69

Please rate the extent to which your mentor helped you understand the knowledge base and standard required to undertake medication reviews?

1 Very poor

2 3 4 5 Excellent

Rural 0 2 7 12 11

Non-rural 3 4 21 55 50

Combined 3 6 28 67 61

Please rate the extent to which your mentor increased your confidence in undertaking mediation reviews?

1 Greatly reduced

2 3 4 5 Greatly


Non-rural 2 4 33 57 39

Combined 2 6 39 70 50

Please rate the likely impact of your mentor on the time taken for you to complete your accreditation examination.

1 Greatly

increased


Rural 3 2 14 6 5

Non-rural 5 14 68 26 18

Combined 8 16 82 32 23

Administrative support received during the program

1 Very poor

2 3 4 5 Excellent

Rural 1 2 15 8 5

Non-rural 7 12 69 24 17

Combined 8 14 84 32 22

Willingness of mentor to address your concerns

1 Very poor

2 3 4 5 Excellent

Rural 0 2 5 7 18

Non-rural 1 2 14 30 86

Combined 1 4 19 37 104


70

Success of mentor in addressing your problems

1 Very poor

2 3 4 5 Excellent

Rural 0 2 2 11 17

Non-rural 1 2 13 51 67

Combined 1 4 15 62 84

Overall rating of the value to you of the mentor program

1 No value

2 3 4 5 Very


Non-rural 3 5 11 29 88

Combined 3 7 14 36 108

better initial training would be more beneficial but would be improved with practical ward rounds

Potential value of the program to other pharmacists

1 No value

2 3 4 5 Very


Non-rural 1 0 7 25 97

Combined 1 0 9 33 119

Payment Do you believe future candidates for accreditation would be prepared to pay for such a service?

Rural Non-rural Combined Yes 21 75 96

No 5 52 57

already paying a great deal for our accreditation, with very little from AACP it could be part of the $500 fee already paid I would pay for a practical session of a DMMR or nursing home review / I find

the course very expensive and I am not a shop owner I think this should be included in the membership of AACP. A fee maybe

charged if the candidate choose not to be a member of AACP Cost of exam should cover this I found accreditation process very expensive


71

I didn’t feel a desperate need for help – but it was valuable to get feedback

that I was on track Structure of future programs Extent to which the service would be improved by having mentors situated in the same geographic area of the candidate, to allow face to face meetings.

1 Very

detrimental


Rural 0 0 11 12 9

Non-rural 0 3 30 43 58

Combined 0 3 41 55 67

I travelled more than 130km each way to see a mentor

Extent to which the program would be affected by replacing expert experienced mentors (as used in this project) with general peer support from other accredited pharmacists whose specific expertise may vary

1 Very

detrimental


Rural 2 11 15 2 2

Non-rural 8 34 53 28 9

Combined 10 45 68 30 11

Mentors as convenors of small groups

To what extent would the program be effected by replacing the one to one service your were provided, with local meetings with other pharmacists, facilitated by pharmacists and/or doctors?

1 Very

detrimental


Rural 1 7 11 10 3

Non-rural 4 23 46 42 16

Combined 5 30 57 52 19


72



No 0 2 2



No 19 69 88

Providing mentor for more hours Group meetings A ward round at a nursing home and a home visit to watch a DMMR would be

more beneficial than a pone call and could be shared with other DMMR students

Through PSA branches I would prefer one-on-one program left as it is & in addition to it, to have

access local meetings as well More frequent but shorter time contact to go aver a few examples

Other suggestions for the structure of a mentor program in the future

Encourage it more. Previously I attended local meeting with other pharmacists & did as many PSA workshops as I could, but no-one ever spelt out the format of writing up the report in the way my mentor did. I felt I had the points & references, but hadn’t “caught on” to what was wanted on the forms. I feel the example case history given in the examination booklet to be a poor example. It is not referenced at all, & the recommendations don’t seem to be worded in a way that would be acceptable to all mentors/examiners.

A one on one situation with a mentor or other accredited pharmacist would be more appropriate in a face to face situation, approaching the first case study together. To complete a weekend course (in which the extent of analysis of case histories was no where near what is now required in the assessments); complete 10 case histories on that basis and fail; then have to do another case history having lost all confidence & face more criticism leads to very frustrating experience!!!

Face to face would be good / discussion group via email with a mentor facilitator as the leader would be good

It would be great to have a person all the time to bounce ideas off as each case is completed (if needed)

It would be good if we were given eg a contact email that we can initiate contact with the mentor rather than relying on them to contact us first

CPE point allocation? Study groups & case discussion would be valuable & also a proper

introduction on how to write & what is expected from the case studies (should be included when receive case studies)


73

On line access would be good for rural and remote pharmacists Mentoring would be more useful if it encompassed the discussion of more

than one case study Mentor made himself very available and approachable. I appreciate the fact

that he spoke to me as a peer pharmacist and made himself readily available. Great program.

I would prefer a brief discussion with a mentor prior to the first case study for preliminary difficulties to be solved. Otherwise I found the process very beneficial and would like to see it continue in the future.

Very happy with the program so far Issues to be addressed:

o Practical issues – what ar4e we really looking for, where to find it, how to sieve through the large amount of info available etc

o Clinical issues – identifying and solving problems eg how to actually adjust dosages in renal/hepatic impairment, real life situations, to be able to differentiate between clinically significant and not significant implications

o Process issues – how to write up the case studies eg correlation between different steps enormous amount of paperwork and very confusing!

o Communication issues – how should the data gathering form and the report form look, what can be written, what can be left out etc

Open panels where others could listen to how cases are worked out Please stress to new candidates how fantastic mentor program is!!! I should

have used it the first time around. Rather than the nominated case study, I would have preferred to submit the

case study of my choice – the one I might have had more trouble with My mentor made herself available at a time suitable to both of us for one solid

face to face meeting, which was invaluable, and on which I have based my answers. She has made herself available for further follow-up as required by me. Thank you.

I do not believe I would have been able to successfully complete pre-accreditation requirements without the aid of my mentor; I failed the first time when I attempted to do it without a mentor

Well worth doing / may consider a phone conversation interview as a part of the accreditation process

Small local groups to facilitate o Ongoing CE o Audits of MMRs for quality o Interdisciplinary liaison

Give all candidates a “gold standard” review so they know what is required. This was not done at the training weekend and resulted in a great waste of energy.

Initially, the on-on-one support is extremely useful, for completing the assignments/case studies and getting a feel fro using the specific forms. You are able to have your specific “stumbling blocks” addressed. Later, group meetings would be beneficial, once your confidence has improved.


74

Appendix 2.2.1: Candidate Evaluation Form – First Review Candidate Name: ________________________________ MRN: _________

Please complete the following evaluation after your first medication review has been

discussed with your mentor. Please return to:



Question Response Number of reviews the mentor commented on?

Number of telephone discussions with mentor about reviews

Number of emails exchanged with mentor about reviews


Face to face meeting Letter Fax Materials sent Other (please specify):

Number of postings made to AACP forum posing questions

How many initial reviews did you discuss with your mentor?

Total time involved in mentor contact

Mentor input What were the main difficulties you found in completing your first reviews?

Time taken Problems getting all the information needed Lack of process for conducting reviews Patient interview Identification of significant issues Constructing an effective report to doctor Other (please specify):


75

What did the mentor suggest you needed to do?

Add patient details Improve medication history Bring appropriate problems to doctor’s


recommendations for management Find problems missed Provide less detail Provide more detail Prioritise problems found Apply clinical judgment Address compliance issues Advise monitoring Change approach of report to doctor Change layout of report Include disclaimer about lesser knowledge

of patient’s full circumstances Provide contact details Other (please specify:

Number of clinically significant medication related problems you identified in these initial reviews

Number of additional clinically significant problems your mentor identified

What do you judge the significance of the mentor’s suggestions were, with respect to your reviews (more

than one category may apply)

Detrimental None Minor Optimised therapy Major Potentially life saving

Number of actions and recommendations you

suggested in these initial reviews

Number of additional actions and recommendations your mentor identified

To what extent did you incorporate the advice provided by the mentor into our final review?

1= not at all 5= followed all advice exactly 1 2 3 4 5


76

If not, why not?

Did not agree No time Advice impractical Couldn’t understand Other

Please rate the extent to which your mentor increased your confidence in undertaking medication reviews


Extent to which the input of your mentor allowed you to improve the quality of the review?

1 = = very poor 5= excellent 1 2 3 4 5


1 = very poor 5= excellent 1 2 3 4 5

Successfulness of mentor in addressing your problems 1 = very poor 5= excellent 1 2 3 4 5

Overall rating of the value to you of having your initial reviews assessed


Payment Do you believe future candidates for accreditation would be prepared to pay to have their first few reviews assessed by a mentor?

□Yes □No

What would be a reasonable fee for such a service?

$

Future programs Do you have any suggestions for improving this part of the program (mentoring of your first reviews) in the future?

Potential value of having initial reviews mentored, to other pharmacists


Overall program With respect to the overall program (examination support and initial reviews mentoring), please assess the impact on your confidence in conducting medication reviews


With respect to the overall program (examination support and initial reviews mentoring), please assess the impact on the quality of your reviews



77


Number of reviews the mentor commented on?

1 2 3 4 5 6 10 13 Rural 4 1 0 1 0 1 0 0

Non-rural 25 5 5 3 0 1 2 1

Combined 29 6 5 4 0 2 2 1

Number of telephone discussions with mentor about reviews

1 2 3 4 5 10 Rural 1 4 1 0 0 0

Non-rural 23 9 4 0 1 1

Combined 24 13 5 0 1 1

Number of emails exchanged with mentor about reviews

1 2 3 4 5 6 Rural 0 1 0 0 0 1

Non-rural 3 7 0 1 1 0

Combined 3 8 0 1 1 1


Rural Non-rural Combined Face to face meeting 0 9 9

Letter 0 3 3

Fax 0 9 9

Materials sent 0 9 9

Other 0 1 1


78

Mentor input What were the main difficulties you found in completing your first reviews?

Rural Non-rural Combined Time taken 4 24 28

Problems getting all the information needed 2 12 14

Lack of process for conducting reviews 0 10 10

Patient interview 0 5 5

Identification of significant issues 0 13 13

Constructing an effective report to doctor 4 19 23

Other 3 8 11

Using the formats provided Interpretation of lab results in relation to drug therapy & patient disease state Confidence Time to get feedback Lack of access to new forms (only had old draft forms) Adapting to different levels of GP awareness of process Discussed the general administration and set up required to provide mentor

service to nursing homes – how to get started etc Format of AACP report forms Confidence in having done the right thing

What did the mentor suggest you needed to do?




2 20 22



1 17 18











79


0 5 5


Other 3 2 5

Say what was discussed during interview including compliance tips etc Ask for full electrolytes where available Re-phrasing things to be more acceptable to GP To state that “no significant drug interactions were found” if this is applicable Mentor very supportive and thought that review under discussion had no

problems What do you judge the significance of the mentor’s suggestions were, with respect to your reviews (more than one category may apply)

Rural Non-rural Combined Detrimental 0 0 0

None 0 0 0

Minor 2 6 8


Major 0 8 8


To what extent did you incorporate the advice provided by the mentor into your final report?

1 Not at all

2 3 4 5 Followed all advice exactly

Rural 0 0 0 5 2

Non-rural 0 4 6 11 18


If not, why not

Rural Non-rural Combined Did not agree 0 0 0

No time 0 0 0

Advice impractical 0 0 0

Couldn’t understand 0 0 0

Other 0 0 0


80

Please rate the extent to which your mentor increased your confidence in undertaking medication reviews 1

Greatly reduced

2 3 4 5 Greatly


Non-rural 0 0 4 21 17


Extent to which the input of your mentor allowed you to improve the quality of the review?

1 Very poor

2 3 4 5 Excellent

Rural 0 0 3 1 4

Non-rural 0 1 8 18 13

Combined 0 1 11 19 17


1 Very poor

2 3 4 5 Excellent

Rural 0 0 1 1 6


Combined 0 0 2 9 38

Successfulness of mentor in addressing your problems

1 Very poor

2 3 4 5 Excellent

Rural 0 0 2 6

Non-rural 0 0 1 12 27


Overall rating of the value to you of having your initial reviews assessed

1 Very poor

2 3 4 5 Excellent

Rural 0 0 2 6

Non-rural 0 0 1 13 26



81

Payment Do you believe future candidates for accreditation would be prepared to pay to have their first few reviews assessed by a mentor?


No 2 12 14

I think the whole process is expensive and we should be offered a free service of advice

It should be part of the whole course Should be part of exam fee Course is costly enough as it is Could be part of initial fee paid to AACP

Future programs Do you have any suggestions for improving this part of the program (mentoring your first reviews) in the future?

No, I found this very useful & supportive Best to be in contact with mentor as soon as interview is done Face to face is a huge help. In country areas this may not be easily accessed Initially mentoring is very useful, however after that, small group peer reviews

would be the way to go Candidate should be able to chose mentor of their choice I believe it should be compulsory to have mentoring of first reviews before the

exam stage to have input and comments on process and ways to improve Practice at formatting the letter to the GP – wording & paraphrasing that is

non-confrontational No, it was excellent

Potential value of having initial reviews mentored, to other pharmacists

1 Very poor

2 3 4 5 Excellent

Rural 0 0 0 3 3

Non-rural 0 0 3 12 23



82

Overall program With respect to the overall program (examination support and initial reviews mentoring), please assess the impact on your confidence in conducting reviews

1 Greatly reduced

2 3 4 5 Greatly


Non-rural 0 0 4 18 19


With respect to the overall program (examination support and initial reviews mentoring), please assess the impact on the quality of your reviews

1 Greatly reduced

2 3 4 5 Greatly


Non-rural 0 0 5 21 15



83


Mentor Name: ______________________________

Candidate Name: ________________________________ MRN: _________

Date of initial mentoring contact: ____________________

Please complete the following evaluation after the first case study has been

discussed with the mentee. Please return to:



Question Response Candidate contact Number of telephone discussions with candidate

Number of emails exchanged with candidate

Other candidate contact Face to face meeting Letter Fax Materials sent Other (please specify):

Estimated total time involved in candidate contact

Nature of feedback What were the main issues you dealt with? Clinical knowledge

Medication reviews process Problem identification Application of clinical judgment Communication Use of references Other (please specify):

Question Response What did you suggest the candidate do to improve?

Further study Further reading


84

Change tone of report to doctor Provide less detail Prioritise problems found Apply clinical judgment Be more patient focused Other (please specify):

Candidate response How receptive was candidate to your suggestions?

1= very negative 5= very positive 1 2 3 4 5

Estimate the probable impact of your mentoring on the candidate’s confidence

1= greatly reduced 5= greatly increased 1 2 3 4 5

Estimate the probable impact of your mentoring on the time taken by the candidate to complete the examination

1= greatly increased 5= greatly reduced 1 2 3 4 5

Contact time record


85


Candidate Contact Other candidate contact


Letter 4 19 23

Fax 15 50 65


Other 1 3 4

Nature of feedback What were the main issues you dealt with?

Rural Non-rural Combined Clinical knowledge 20 157 177

Medication review process 25 176 201

Problem identification 20 170 190

Application of clinical judgement 29 170 199

Communication 31 179 210

Use of references 20 152 182

Other 3 67 70

Very long interview / clinical knowledge poor / little knowledge of how to use

info obtained from references / millions of questions / very unsure of what or how to write review findings / no medical condition type books or recommendations

Best references from our best reference list provided (developed during mentor training)

Capable community pharmacist needing reassurance and reinforcement To develop ongoing support network Specific references suggested for this case English language and grammar Recommended relevant references Lots of missing info / review was all drug drug drug / disease states not

researched adequately No medical reference book / very defensive / very narrow viewpoint on med

rev’s until he started to get the idea Unhappy with being corrected / admits needs disease state knowledge NPS TAIS line / PSA case review evenings Obtaining consent from residents / patients Lack of confidence Sentence structure


86

Candidate did not really understand or consider the need for an in-depth

clinical knowledge or input. Just wanted to be accredited fro HMR process – which he felt was more superficial & simple

Capturing appropriate detail on form Language difficulties and expression Prioritisation / avoided the main issues (as being too difficult!!) Confidence Disease state knowledge Use of forms Alternative treatments / tests Very low self esteem / low confidence / no idea what reference books to use

(was using the internet) Total lack of confidence Knowledge is out of date Prioritising issues Patient as main focus / form completion Approach to a problem Overcomplicated approach Indecisive comments / over concern with interaction causing non-existent

problems / inappropriate recommendations / non-prioritised recommendations / missing info re disease state control

Reasonably good considerations of problems – a few omissions and errors but very poor recommendations to doctor

Some good points but a lot of missing info gathering / report to doctor poor Wording / HMR process Questions to ask at HMR interview Identification of gaps in information & documenting this (in actions &

interventions) How to go about writing report / format / computer shortcuts / how to get in

touch with other accredited pharmacists in area Confidentiality & privacy Presentation of report Inappropriate recommendation / mentoring ongoing (after conclusion of

project time) Tendency to diagnose Care with details Attention to detail Use of forms Had difficulty with the forms MMR forms PSA med review evenings / NPS TAIS info line Completion of red form Consideration of non-pharmacological issues / report writing (from

brainstorming final report) / discussed additional information required to make decisions & how we go about finding this eg involve nurses is necessary

Difference between examiners need and mentoring process Candidate had difficulty with filling on all the forms (despite completing it quite

credibly) / candidate was quite negative about the whole process, while realising the importance of medication reviews / internet sites & reference books


87

Design of report form / minor details Development of support network Difference between findings, actions & intervention, recommendations Use of medication review programs (commercially available) Do more research Read case study / attention to detail Consider what other info you need & involve nurses and patient if necessary /

documentation discussed – show examiner what other info you would like to base recommendations on / discussed purpose of forms

Need to prioritise Prioritising drug related and health issues Did not understand the DMMR form at all – I had to explain all the categories

to him Too detailed review – needed to be streamlined Types of references Presentation of report Completing forms

What did you suggest the candidate do to improve?

Rural Non-rural Combined Further study 2 49 51

Further reading 17 119 136

Change tone of report to doctor 27 151 178




Be more patient focused 31 160 191

Other 9 60 69

Attend PSA case study evening / contact MMR facilitator / identify a study

partner Set out of letter to doctor References / trouble with writing recommendations Purchase texts / internet contact / develop library / study group development Almost every aspect needed re-jigging, but candidate is very intelligent and

capable Relatively minor issues became the main issues to the exclusion of

everything else Fill in forms correctly Use of references Identify drug interactions Write whole sentences and explain why problem identified and thus why

recommendation given Language; how to word reports to encourage open communication Reassurance Significant language problems


88

Apply commonsense to solutions Needs lots more work Try not to jump to conclusions Think more laterally about problems eg measures to increase analgesia More detail / reasons for recommendations He needed to have the procedure explained – he had the knowledge but did

not know how to apply it How to phrase recommendations better Needs to be more concise / avoid going off on tangents and be more focused Most problems were seen in terms of interactions only List of reference books & journals / repeat case study & resubmit / send other

cases Take more time and care when doing reviews – take them seriously Be specific and concise / be able to back up recommendations with

references & not ‘common usage’ (i.e. not necessarily reference everything, but be correct in what is said)

Candidate tends to be vague on paper but specific when talking so I suggested candidate specify more eg detailed adverse reaction possible

Separate the review findings from the recommendations Provide more follow through with report writing Needed more systems process to identify issues This student was fantastic! Provide specifics eg why a drug should be avoided Moe detail in recommendation / reasons for recommendation Tell doctor what he doesn’t know – don’t waste time repeating things This candidate showed very little clinical knowledge or understanding of the

medication review process Re-do case study & re-submit before mentoring could continue / too many

points not identified Do not assume anything / need to look at issues in more detail Discuss aspects of patient care with nursing staff Change style of final report – avoid asking questions Not to get defeated by the forms involved Correct use of forms / follow through on suggestions The candidate was very competent however didn’t feel her report was good. I

thought it very good, concise legible etc Documentation State references & reasons for recommendations Include actions and interventions in report Re-read communication and concordance module Read case closely / fill out forms differently Use the forms to help follow through & complete process Work through each problem on both sides of first 2 sheets, then prioritise

what needs to go on sheet 3 Make suggestions of alternative therapies rather than just saying “don’t use” Made incorrect assumption / was not patient focused Provide greater detail outlining considerations of the case Increase knowledge of nursing home and hostel conditions with regard to

background to the medication reviews / candidate already very competent Read instructions A little more depth


89

Tone to doctor ok but need to provide solutions and suggestions and

document reasons or provide references more detail in recommendations Communication Pay more attention to detail (had patient name wrong x2!) Where to find information for recommendations and clinical content Source of information / accessing information

Candidate response How receptive was the candidate to your suggestion?

1 Very

negative

2 3 4 5 Very

positive Rural 2 3 12 22

Non-rural 1 3 23 79 130

Combined 1 5 26 91 152

Estimate the probable impact of your mentoring on the candidate’s confidence

1 Greatly reduced

2 3 4 5 Greatly

increased Rural 1 4 21 13

Non-rural 2 4 46 128 55

Combined 2 5 50 149 68

Estimate the probable impact of your mentoring on the time taken by the candidate to complete the examination

1 Greatly

increased


Rural 1 7 9 16 6

Non-rural 8 33 88 80 25

Combined 9 40 97 96 31

Other comments:

Still some work required before cases are ready to submit – recently looked at 2 other cases he has done which are not of a standard to pass

Requires significantly more clinical knowledge & application skills to complete cases

Is still a long long way off the mark in terms of case studies / very difficult to understand / means well but not sure if she has the skills required (this candidate has subsequently gained their accreditation)


90

Candidate has learned a lot through this process. He has significant gaps in

his clinical knowledge, however has learnt a lot through this mentoring. I have not seen his final assessment, but have provided feedback on a case-by-case basis. He did not agree with some of my directions or suggestions. Despite significant mentoring, I would not expect him to pass. (After extensive mentoring (~7 hour) this candidate has subsequently gained their accreditation and is actively providing HMRs and continuing to use his mentor for these.)

This candidate realised there were some large gaps in her knowledge and became quite despondent despite my encouragement & suggestions on professional development

She designed form herself as she could not download AACP forms more time, but better outcome (easier to check spelling and change wording etc) / she was very angry as she was a hospital pharmacist being mentored by a community pharmacist (!

The candidate did not understand the mentoring concept. Subsequent to first case, he continued to send case studies – while I didn’t offer any help, I did indicate he was still supplying too much detail & not applying any judgement to the mass of facts provided.


91

Appendix 2.4.1: Mentor Evaluation Form – First Reviews

Mentor Name: __________________________________

Candidate Name: ________________________________ MRN: __________

Date of mentoring: _______________________________

Please complete the following evaluation after the first medication review has been discussed with the mentee. Please return to:

Mentor Support Program

Australian Association of Consultant Pharmacy PO Box 7071

Canberra Business Centre ACT 2610

Question Response Candidate contact Number of reviews you commented on

Number of telephone discussions with candidate about reviews

Number of emails exchanged with candidate about reviews

Other candidate contact Face to face meeting Letter Fax Materials sent Other (please specify):

Estimated total time involved in candidate contact

Your input What did you suggest the candidate do to improve?

Add patient details Improve medication history Bring appropriate problems to doctor’s


recommendations for management Find problems missed


92

Provide less detail Provide more detail Prioritise problems found Apply clinical judgment Address compliance issues Advise monitoring Change approach of report to doctor Change layout of report Include disclaimer about lesser

knowledge of patient’s full circumstances Provide contact details Other (please specify:

Number of clinically significant medication related problems candidate found

Number of additional clinically significant problems you identified

What do you judge the significance of your suggestions were, with respect to the candidate’s reviews (more than one category may apply

Detrimental None Minor Optimised therapy Major Potentially life saving

Number of actions and recommendations candidate suggested

Number of additional actions and recommendations you identified

How would you assess the quality of the reviews as submitted to you?


Contact time record


93

Appendix 2.4.2: Mentor Evaluation Form – First Reviews Results

Candidate contact Number of reviews you commented on

1 2 3 4 5 6 7 Rural 3 1 1 0 0 0 0

Non-rural 16 13 4 1 1 1 1

Combined 19 14 5 1 1 1 1

Number of telephone discussions with candidate about reviews

1 2 3 4 10 Rural 1 2 2 0 0

Non-rural 13 14 4 1 1


Number of emails exchanged with candidate about reviews

1 2 3 4 5 6 7 Rural 0 1 1 0 1 0 0

Non-rural 2 9 4 1 1 1 1

Combined 2 11 5 1 2 1 1

Other candidate contact


Letter 0 1 1

Fax 1 5 6


Other 1 3 4

Assisted in setting up process at a nursing home and suggested how to

approach GPS Visited GPS in vicinity together Invited to a HMR meeting at local division Wrote first reports together Continued to provide support privately Went to visit HMR patient with candidate How to do a HMR I provided templates of report forms and covering letters I use


94

Your input What did you suggest the candidate do to improve?




2 16 18



2 17 19



Provide more detail 8 8








1 7 8


Other 2 12 14

Candidate simply required help with reference materials Ongoing support provided privately and also involved in local division HMR

activities Provided template for report Wrote report together & provided template via email Sent information pack from local division re HMR process Provide more detail re interventions at patient interview Format of report and interviews – mainly practical advice, not clinical Most assistance related to set out & tone of report Address patient concerns as a high priority GP made changes during HMR process – candidate needed advice on what

to do Total process and how to claim Discussed process, documentation & report forms Suggested to document actions taken by pharmacist Candidate didn’t document all his actions and considerations which were

relevant and should have been in the report to the GP Showed how to do a HMR (as candidate had only used medication review

forms for nursing homes) Optimised wording & report


95

What do you judge the significance of you suggestions were, with respect to the candidate’s reviews (more than one category may apply)

Rural Non-rural Combined Detrimental None 1 0 1

Minor 2 2 4


Major 2 9 11


How would you assess the quality of the reviews as submitted to you?

1 Very poor

2 3 4 5 Excellent

Rural 0 0 2 3 0



Additional comments:

Thank you email from candidate: “many thanks for your prompt response and input, this certainly boosts my confidence.”

Candidate was not confident he could do the report & just needed some practical advice

Documentation of changes & recommendations were the main actions, instead of just leaving it to the GP!

Enabled “mentee” to have confidence to contract herself to do HMRs


96


Mentor Name: ______________________________

Please complete the following evaluation at the conclusion of the Mentor Support Program. Please return to:

Mentor Support Program

Australian Association of Consultant Pharmacy PO Box 7071

Canberra Business Centre ACT 2610

Question Response

Your experience Administrative support received during the program 1= very poor 5= excellent

1 2 3 4 5

Value of the training day 1= very poor 5= excellent 1 2 3 4 5

Overall willingness of candidates to attend to your

advice 1= very poor 5= excellent 1 2 3 4 5

Overall enjoyment of your interaction with candidates 1= very poor 5= excellent

1 2 3 4 5

Estimated number of postings you made to AACP forum, addressing medication review issues, during

the project

Payment If you provided such a service outside of this project, what do you think would be a reasonable fee?

a. examination phase service: $____________

b. initial reviews phase service: $__________

Impact

Overall impact of your service on candidates’ confidence


Overall impact of your service on time taken by

candidates to complete their examination 1= greatly increased 5= greatly reduced

1 2 3 4 5

Overall impact of your service on quality of candidates’ initial reviews



97

Question Response

Structure of future programs Extent to which the service would be improved by having mentors situated in the same geographical

area of the candidate, to allow face-to-face meetings

1= very detrimental ……… 5 = extremely important

1 2 3 4 5

Extent to which the program would be effected by replacing expert experienced mentors (as in this

project), with general peer support from other accredited pharmacists whose specific expertise

may vary


To what extent would the program be affected by replacing the one to one service you provided with local meetings with other pharmacists, facilitated by pharmacists and/or doctors?



□Yes □No


□Yes (please specify): □No

Other suggestions for the structure of a mentor program in the future

Suggestions for the content of a mentor support program in the future


98


Your experience Administrative support received during the program

1 Very poor

2 3 4 5 Excellent

0 3 5 7 4 Value of training day

1 Very poor

2 3 4 5 Excellent

0 0 0 11 9 Overall willingness of candidates to attend to your advice

1 Very poor

2 3 4 5 Excellent

0 0 0 10 10 Overall enjoyment of your interaction with candidates

1 Very poor

2 3 4 5 Excellent

1 6 13 Estimated number of postings made to AACP forum, addressing medication review issues, during the project: 6 Payment If you provided such a service outside the project, what do you think would be a reasonable fee?

a. Examination phase service: $50/hr x 7 $60/hr x 2 $70/hr x 3 $75/hr x 4 $80/hr x 2

b. Initial review phase service:

$50/hr x 6 $60/hr x 2 $70/hr x 2 $75/hr x 5 $80/hr x 2 $100/hr x 1

These are more complicated and unpredictable


99

Impact Overall impact of your service on candidates’ confidence

1 Greatly reduced

2 3 4 5 Greatly

increased 0 0 1 11 7 Overall impact of your service on time taken by candidates to complete their examination

1 Greatly

increased


0 2 6 9 2 Overall impact of your service on quality of candidates’ initial reviews

1 Greatly reduced

2 3 4 5 Greatly


Structure of future programs Extent to which the service would be improved by having mentors situated in the same geographic are of the candidate, to allow face to face meetings

1 Very

detrimental


0 2 8 7 3 Extent to which the program would be affected by replacing expert experienced mentors (as in this project), with general peer support from other accredited pharmacists whose specific expertise may vary

1 Very

detrimental


1 9 8 2 0 To what extent would the program be affected by replacing the one to one service you provided with local meetings with other pharmacists, facilitated by pharmacists and/or doctors?

1 Very

detrimental


1 9 5 3 1


100

Should the Australian association of consultant pharmacy (AACP) continue to conduct a mentor support program? Yes 16 No 0 Is there a more suitable structure for administering a mentor program?

Workload needs to be organised so that no more than one candidates papers arrive per week

Via PSA Direct contact of the candidate with the mentor Needs to be through to AACP to ensure consistency of message Could also consider contracting a few mentors on a “retainer” eg $5000 for 1

year service (work out estimated time and $ based on actual time interval) Other suggestions for the structure of a mentor program in the future

Make it mandatory that the first reviews are assessed by candidate’s mentor (or examiner) prior to sending them to doctor

Candidate to present more than one case study during initial contact (examination phase) eg to do the very first case study together, then for the candidate to submit 2 other cases for comment, before completing the balance for assessment

Face to face meetings desirable wherever possible (need to provide funding to encourage this if necessary)

Possibly in conjunction with other organisations with clinical expertise (eg Queensland rural medical support agency has 5 clinical pharmacists visiting rural and remote pharmacies and includes the MMR facilitator scheme)

Incorporation of program on-line Useful to use divisional facilitators (+ PSA) to develop “peer group support” I like the one to one to start, then peer support so people can learn from each

other Maybe AACP and PSA should get together and use divisional facilitators to

mentor support It would be good to get feedback from AACP when the mentored candidate

gained their accreditation Peer support suggestions seem appropriate for pharmacists who have some

experience and are seeking peer support and review of their practices. In the pre-accreditation and immediately post-accreditation stages, the one on one service seems more appropriate.

The suggestion of peer support is valuable but only one the initial reviews are complete and the candidate has sufficient base knowledge to build on. The process should therefore be well established in the candidate’s mind before they more to a less structured peer support group.

Some pharmacists might like the online mentoring – however this may not be as effective as speaking with the candidate as it is only a one-dimensional mentoring i.e. Can’t pick up on verbal clues

Besides offering one to one, could also offer networking meeting of pharmacists pre and post accreditation examination


101

The pre-examination phase mentoring is very important as it identifies issues

which need to be addressed eg clinical knowledge, written communication, explanation of forms etc before the rest of the case studies are attempted / completed

If candidates unhappy with their mentor, they should know they can be allocated another, as not all personalities are compatible

Feedback to mentors when mentee passes / fails It would be good to have candidates spread evenly over a period (I seemed to

get a bunch followed by a gap, then another bunch). I realise it depends on applications though!

Local meetings would be useful in conjunction with the mentor program – some mentee’s were getting together with other pharmacists but were still struggling. Some people had their case studies checked over by accredited pharmacists they knew before sending them in and had still failed.

I would suggest an on-line notice board of mentors with and alert available / unavailable (need to be careful that the mentor is not overworked)

Well conducted program – I enjoyed all interactions with pharmacists! Perhaps organise meetings with other mentors once a year for interactive

discussions on other mentor’s experiences +/- mistakes. It would also serve as a forum to keep the mentors abreast of political / financial decisions that may affect this invaluable program.

Review and get together of mentors on an annual basis A combination of group sessions as well as 1 to 1 Most candidates initially require more than 2 hours of help Consider having candidates accompany accredited pharmacists to nursing

home / HMR to observe Would like to have some component of one-on-one home visits or visits to

RCFS Work in groups and one to one Active web-site chat room with automatic emails to participants (with option to

opt out) Suggestions for the content of a mentor support program in the future

Additional meetings to ensure standard approach amongst mentors and to meet again with examiners following training day

Once accredited, make a mentor available to accompany them on their first visit for DMMR. This may encourage pharmacists to become actively involved early on and boost their confidence, which may be lacking. It may also ensure standards are achieved. (I appreciate this may be more difficult in more rural/remote areas.)

More on communication issues – how to build confidence – support without reducing self-esteem

More direction given to participants sending initial reviews for mentoring. Information is often missing, the report not written completely, making mentoring more difficult. It is difficult to be constructive when info / report is missing

More feedback to mentors on their candidates’ eg o Whether the candidate found the intervention useful o What the candidate liked / disliked o Opportunity for mentors to share experiences and learn from each

other


102

Mentors need to have the “answers” to the case studies (eg gleaned at the

Jan 2002 mentor training day that every mentor suggested different clinical solutions to the same case study!)

Mentors need to meet say once a year to “debrief” (eg I had a bad experience with 1”mentee”, would have been good to have the chance to share and learn from this with others)

Good to offer getting the mentor and mentee’s together after accreditation so everyone can meet

Am happy with the content in general Candidates should be made aware that it is “mentoring” and that the answers

will not be given for them to copy down An example of a completed review (both nursing home and HMR) available

on the AACP website Greater knowledge of other case studies in the workbooks – very repetitive

having to deal with olive Harvey and bill Smyth so often! I think the first part of the course (stage I) needs to be changed. Case studies

are discussed but not to the extent they should be, eg looking at disease states and the patient as a whole situation, not just a list of drugs. Most candidates were not aware that consideration of non-medication alternatives (eg diet, exercise, sleep hygiene etc) was also important. Also need to discuss report-writing styles in the course. Some people are confused about what happens after they become accredited.

Overall, my mentee’s did not appear to have adequate clinical knowledge to tackle their case studies, nor did they have the concept of what the doctor would expect in a medication review. I think the mentor support program in future may need to address the pre-examination phase training with more workshops and more written guidelines

Found lack of transparency by ACPP marking to be very frustrating – clarity on what is required to pass would be very helpful. Ongoing info re when candidates pass / fail would be helpful re knowing when to recontact candidates

Necessary for case study stage and initial reviews. While unlikely to happen, suggest mandatory peer review for first 10 cases, with aim of improving clinical quality and process/report writing skills

Mentor support program - FINAL REPORT - 6CPA6cpa.com.au/wp-content/uploads/Mentor-Support... ·...

Documents

Transcript of Mentor support program - FINAL REPORT - 6CPA6cpa.com.au/wp-content/uploads/Mentor-Support... ·...