Andrew ConnollyChair, MCNZ
The outcomes & effectiveness of revalidation in New Zealand for non-certified general practitioners
Definitions• “Non-certified” are those general registrants
who have not achieved vocational (specialist) registration and are not actively in an accredited specialist training program– Not restricted to General Practice, but vast
majority are working in this field• Does not apply to PGY-1• Will not apply to PGY-2 from 2016
Why do it?• Council legally responsible for ensuring
practicing doctors are competent• Long-established CPD requirements but a
lack of standardisation, consistency• College programs all accredited via
Australian Medical Council & MCNZ• Council recognised the lack of a general
registrant mandated program was a risk
Council decisions:• Electronic format (“e-portfolio”)• Mandatory unless “grandfathered” by existing
successful participation in an accredited scheme
• Mix of annual and triennial requirements– RPR & MSF three-yearly
• Prospective research evaluation by independent company
The Program• Introduced in May 2012
– Essentials quiz– Collegial relationship with meetings mandated– Professional development plan– Audit– CPD record– Regular practice review: began July 2013– Multi-source feedback
The numbers• Currently some 1800 participants• Very few non-compliant but Council has
escalation policy including suspension - has been necessary in a very small number
• Most radical has been the Regular Practice Review (RPR)
• RPR is prime focus of research• Numbers are small, but knowledge is growing
• A formative assessment via the Best Practice Advocacy Centre’s Inpractice programme
• Focus is on quality improvement by helping individual doctors identify areas of improvement and build them into professional development plan- A mandatory requirement for doctors registered in a
general scope of practice- Reviewers visit the participating doctors in their
practices for a one day visit involving review of notes, observation and discussion
• Will be prospectively studied
The Regular Practice Review (RPR) Programme
The Regular Practice Review (RPR) Programme
• Spans July 2014 to 2020 and
focuses on the outcomes of RPR• Collects data from participating
doctors and the reviewers
Methodology• Survey at 2 weeks post RPR• Repeated at 12 months post RPR
– Early cohort of follow-ups– Later cohort of follow-ups
• “Early” and “Later” cohorts refer to RPR pre or post July 2014– Early cohort did not have 2 week survey– Considerable on-going communication with the
profession about purpose & importance– Later cohort also saw emphasis on reviewer skills
Doctors participating in the evaluation
• Surveys of participants two-weeks and twelve-months post-RPR
Later cohort17 of 31 doctors (55% response)
Early cohort:32 of 45 doctors (71% response)
Early cohortCompleted
RPR Jan-May 2014
Later cohortCompleting
RPR from June 2014 - ongoing
Two-week post-RPR survey Twelve-month post-RPR survey
Did not complete RPR post survey - not yet
implemented
Completed twelve-month survey
Closed
Completed two-weeksurvey
159 of 236 doctors (67%) - Ongoing
Completed twelve-month survey
Ongoing
The Participants
Doctor characteristicsTwo-week
respondents(n = 159)
12 month early cohort
(n = 32)
12 monthlater cohort
(n = 17)
RoleGeneral practice 68% 91% 76%
Other 32% 9% 24%
Trained inNew Zealand 36% 34% 35%
Other 64% 66% 65%
Years in practice
0-10 45% 47% 18%
11-20 43% 38% 76%30+ 12% 16% 6%
English first language
Yes 77% 84% 100%No 23% 16% -
Improvements for patients: 2 weeks
• Just under half strongly agreed or agreed that participating in RPR had improved the care they deliver to their patients (44%) and improved their practice in other ways (49%)
Changes to practice: Two weeks after RPR
• Nearly half (46%) said they had already made changes to their practice as a result of RPR
• A further 13% said they intended to make changes
Overall views on RPR: 2 weeks
• The majority (71%) agreed that the practice visit was a positive experience
• Over half found the report useful (63%) and would positively recommend RPR to colleagues (56%)
Examples of immediate changes• Improved records and note taking – most commonly mentioned
Ensuring appropriate documentation of clinical notes. Going deeper into patient history beyond presenting complaint.
• The consultation – style and interaction with patientsTried to change consultation style, trying to prioritise patient questions.Communicating more effectively with patients
• Review of prescribing and ordering lab testsI am a bit more critical about which lab tests I order.•Improving cultural competenceTaking specific interest in Māori and Pacific cultural aspects of patients and trying to integrate them in consultations.
Changes to professional development plans (PDPs) at two weeks
• Doctors evenly divided between those who had and had not made changes to their PDP
• One-third (34%) had discussed their PDP with the reviewer
• Just over half (52%) agreed that their professional development plan is a useful tool to improve their practice
Changes to practice: Twelve months after RPR
59%
19%
I have made changes to my practice
Early cohort
Later cohort
• Early group lacked knowledge of the process & of the importance of the program; later reviewers were more skilled
• Improved relationship with collegial supervisor• Small numbers, but encouraging
Improvements for patients: Twelve months
• The later cohort were much more likely to agree that RPR had improved care (53% compared to 18%)
Changes to PDP: 12 months – Early cohort
• Similar proportions – about one-fifth – reported making changes to their PDP, the way they manage it, and that changes have made it more useful
Changes to PDP: 12 months – later cohort
• Later cohort was much more likely to agree that RPR led them to make changes to their PDPs and that those changes made their PDPs more useful
Overall views on RPR: Twelve months
• Later cohort much more likely to view RPR as useful or very useful - 53% compared to 31%
• Later cohort much more likely to recommend RPR to their colleagues – 65% compared to 28%
What about the RPR reviewers?
• Almost every reviewer provided an example of a benefit to themselves. Some benefits were personal:
Seeing a variety of different GPs in their practices and consulting has been extremely interesting.
• Online survey and interviews in 2014 – to be repeated in 2015/16. All nineteen reviewers responded in 2014.
Reviewer comments• I learn from the positives I observe in sitting through
consults of my colleagues and use it in my own practice.
• It has also been an opportunity to chew the fat over both ordinary and contentious issues and to hear fresh and diverse perspectives. I too feel under the looking glass by those I review so it puts pressure on me to ensure I am really up to date so I appear credible.
Summary• Early experience is encouraging• Better knowledge of the purpose and
importance of revalidation appears crucial to sustained success
• Reviewer skills and understanding are important
Other Groups• Obstetrics & Gynaecology – voluntary• General Practice specialists – voluntary• Orthopaedics- mandatory
– 95% pre-RPR were opposed– 95% post-RPR were in favour
• Greatest benefits in work/life balance, scope of practice
Conclusions to Date• Early indications of value of RPR
– higher proportions reporting sustained benefits• Positive influence on professional
development• Improved collegial relationships• Positive influence on practice • Positive influence on patient care
Next Steps• Continue to highlight the importance of revalidation• Continued 2-week and 12 month follow-ups• As numbers grow we will be able to firm our
conclusions– Including begin to look at competence concerns
• Further survey of reviewers• Map progress into vocational training• Ultimately see processes adopted by other
vocational groups
Top Related