Perceptions of how well graduates are prepared for the role of pre-registration house officer: a...
-
Upload
alison-jones -
Category
Documents
-
view
218 -
download
3
Transcript of Perceptions of how well graduates are prepared for the role of pre-registration house officer: a...
Perceptions of how well graduates are prepared for the roleof pre-registration house of®cer: a comparison of outcomesfrom a traditional and an integrated PBL curriculum
Alison Jones, Patricia J McArdle & Paul A O'Neill
Introduction Most medical schools in the UK have been
engaged in major curriculum reform based on their
premises of what might improve undergraduate medical
education. In 1994 the course at the medical school of
the University of Manchester changed to an integrated
course using problem-based learning throughout and
with increased emphasis on community-based medical
education. This study explores whether the new curri-
culum has produced any differences in perceptions of
how well graduates are prepared for the role of pre-
registration house of®cer.
Methods A postal questionnaire was used to survey 1998
Manchester graduates (traditional course) and 1999
Manchester graduates (new course), three months into
their ®rst pre-registration house of®cer placement. A
similar questionnaire was sent to the educational
supervisors who were supervising the graduates. The
questionnaire was designed to measure perceptions of
levels of preparedness for the role of pre-registration
house of®cer, using a list of broad areas of competence
and speci®c skills listed in the General Medical Council's
`The New Doctor'.1
Results Graduates rated the new course signi®cantly
more effective for 12 of the 19 broad competences and
eight of the 13 speci®c skills that were listed. The `new'
graduates rated their understanding of disease pro-
cesses lower than the `traditional' graduates, but there
was no difference in the ratings given by the educational
supervisors for this. Overall the educational supervisors
rated the new course as better preparing graduates in
®ve of the competences.
Conclusions Overall, the evaluation shows that a major
change in curriculum approach has changed the pro®le
of the perceived preparedness of graduates for entering
professional practice.
Keywords Comparative study; Curriculum; Education,
medical, undergraduate/*standards; Great Britain;
problem-based learning/methods; professional compe-
tence; questionnaires.
Medical Education 2002;36:16±25
Introduction
During the last decade, recommendations on medical
education have been published that have strongly
in¯uenced undergraduate curriculum development
in the UK.1±3 The course at Manchester has been
reformed to re¯ect these proposals and also to re¯ect the
body of research on the best ways to facilitate under-
graduate learning. In 1994 the new undergraduates at
Manchester embarked on a course that was markedly
different to that of their predecessors. The traditional
pre-clinical/clinical curriculum was replaced by one that
is integrated across disciplines, more learner-centred,
with increased emphasis on the delivery of medical
education in the community rather than in the hospital
setting, and uses problem-based learning (PBL) as the
method for delivering core knowledge and under-
standing. Students work in small groups to study written
descriptions of clinical situations, working with a sys-
tem-based rather than discipline-based approach. They
use a series of steps as a guide to their discussion of the
clinical scenarios, meeting two or three times each week
in small groups facilitated by a tutor. In between tuto-
rials, the students use a variety of resources to meet their
learning objectives; in year three onwards these
resources include clinical experience.
There is limited evidence in the literature on differ-
ences in outcomes between traditional and PBL courses
at the level of newly graduated doctor. The ®ndings from
Medical Education Unit, University of Manchester, Faculty of
Medicine, Dentistry, Nursing and Pharmacy
Correspondence: Alison Jones, Medical Education Unit, Ist Floor,
Rusholme Health Centre, Walmer Street, Manchester M14 5NP, UK.
Tel.: 0161 256 3015 x 289; Fax: 0161 256 1070; E-mail: alisonj@
fs1.with.man.ac.uk
Preparing for the job
16 Ó Blackwell Science Ltd MEDICAL EDUCATION 2002;36:16±25
published studies have varied from better `clinical
functioning' by PBL students, better performance in
knowledge tests by traditional course students, greater
awareness of recent guidelines by PBL students, through
to little or marginal bene®ts.4±6 In these studies, the
schools using PBL have mostly con®ned the use of this
method of learning to the early part of the curriculum.
The use of clinical supervisor rating forms to com-
pare perceptions of the performance of graduates from
traditional and PBL schools has shown that PBL
graduates perform better in respect of interpersonal
relationships, reliability and self-directed learning. It
was found that graduates from one of the two tradi-
tional schools were rated higher for teaching, diagnostic
skills and understanding of basic mechanisms.7
A common element among such studies is that the
comparisons are between students or graduates of dif-
ferent schools and in doing so there are dif®culties in
drawing conclusions on separating out the effect of
curriculum design from the overall context of the
school.5,7,8 At Manchester, with the radical change in
methods of curriculum delivery, we had the oppor-
tunity to compare graduates of two different courses
within the same institution.
Our hypothesis was that there would be differences
in perceptions of graduates' levels of competence in
some of the speci®c skills or more broad areas as de-
®ned by the General Medical Council as the aims of
general clinical training for the `New Doctor'.1
Amongst other things, our study looked at whether
changing the methods of teaching and learning could
improve the preparedness of graduates for important
areas such as communication and team-working,
which are of topical interest given the media interest in
the performance of doctors and emphasis on
performance review.
Undergraduate medical educationat Manchester
Manchester is a large UK medical school admitting
(in 1994) over 240 students in the ®rst year, with an
additional 90 students joining the course in the third
year from St Andrews University. Up until 1994 the
students followed a traditional ®ve year course with two
years of pre-clinical and three years of almost entirely
hospital-based clinical teaching. For the 1994 intake
onwards, the school introduced PBL in small groups as
the major method for delivering core knowledge and
understanding, with the curriculum integrated across
disciplines and the learning being systems-based.
We have described the new curriculum in detail
elsewhere, thus only a brief description is given.9,10 The
outline of the new ®ve-year curriculum is shown in
Figure 1. SSMs are `special study modules' in which
students have wide choice in what they study or gain
experience in.2 Assessment has also changed from
being departmental to being integrated across disci-
plines and has been designed to match the course
objectives closely.
Distinct from almost all of schools using PBL, the
students continue to use this method of learning when
they move from the medical school to a hospital base
for years three and four.9 The PBL approach used in
Manchester is high on the hierarchy set out by
Barrows.11 In years three and four, a group of students
meet for one hour with a tutor to discuss a case using a
series of steps. The students set group learning objec-
tives (questions) at the end of the ®rst discussion ses-
sion and then meet again after one week for 90 minutes
to discuss their ®ndings.
As the students in year three onwards are working in
a clinical environment, the students can seek out clin-
ical experience as well using, for example, books, arti-
cles and lectures to answer their learning questions. We
encourage the students in group discussions to make
connections between the paper case and the patients
they have seen.
The PBL cases consist of clinical vignettes that
integrate diagnosis, investigation and management as
well as the relevant basic and clinical sciences and the
wider psychosocial aspects of clinical medicine. Over
the four years, the cases are organised into modules
lasting for one semester each, and each module has an
overarching theme. During these semesters/modules
the students cover the core curriculum based on a list of
`index clinical situations' (ICSs ± consisting of 61
presentations and 154 diseases or syndromes) drawn
from repeated consultations with hospital consultants
and general practitioners in the region.9
Key learning points
Surveying house of®cers and educational super-
visors was used for generating outcome evaluation
data in order to compare `products' of different
undergraduate courses.
Graduate ratings of their levels of preparedness
were affected by a change in curriculum, with
innovative course graduates providing more
favourable ratings for several items.
Educational supervisors perceived some change
in the preparedness of house of®cers in the set
of competences.
Ó Blackwell Science Ltd MEDICAL EDUCATION 2002;36:16±25
How well prepared are graduates for the role of pre-registration house of®cer? · A Jones et al. 17
In addition to the use of PBL, there are other major
differences between the new and the old curriculum. In
the new course, whilst students are attached to clinical
®rms in hospitals, there is also a much greater emphasis
on using the community as part of their experience.2
We wanted to give students a better appreciation of the
Figure 1 Diagram of course.
How well prepared are graduates for the role of pre-registration house of®cer? · A Jones et al.18
Ó Blackwell Science Ltd MEDICAL EDUCATION 2002;36:16±25
context of medical care, including the importance of
broad health determinants and the relevance of the
interactions between health problems and social issues.
In order to achieve these aims, one day per week in
the core modules in the third and fourth year is set
aside for students to make use of community res-
ources,which are usually based around a particular GP
practice.12
In year ®ve, the students rotate through teaching
hospital, district general hospital, community and elec-
tive placements. In the ®rst three of these there is weekly
group work in which the approach to PBL is extended by
the students setting the agenda through discussing
paients that they have encountered in their clinical
attachments rather than having set paper cases. The ®nal
few weeks in the ®nal year on the new course are spent
shadowing the pre-registration house of®cer whose post
they will be ®lling.
After graduating from the course, students take up a
pre-registration house of®cer position, usually staying in
the Northwest region for their ®rst post. During their ®rst
year they have an allocated educational supervisor who is
responsible for monitoring their progress. These super-
visors are NHS consultants who have been on an
appropriate development course. Supervisors are res-
ponsible for a varying number of graduates and some will
have more contact with their PRHOs than others.
Methods
In order to obtain comparative data on the perceptions
of the knowledge, skills and attitudes of the Manchester
graduates we surveyed the graduates and their educa-
tional supervisors over a two-year period.
We surveyed the last group of students to graduate
from the traditional course (in 1998) and the ®rst group
to graduate from the new PBL course (in 1999). For
these two cohorts selection to the medical course was
based on their `A' level grades or equivalent and a
reference from their school. Manchester did not routi-
nely interview its applicants until 1995, so neither
cohort in the present study had been interviewed before
admission. For ease of tracking non-respondents, gra-
duates whose home address was overseas were excluded
from the study. The two groups of graduates were
surveyed at the same point, i.e. three months into their
®rst pre-registration house of®cer placement.
Educational supervisors were selected for the survey
if they were working in the North-west region and were
supervising Manchester graduates. Again, two cohorts
were surveyed; they comprised those supervising the
last group of graduates from the traditional course and
those supervising the ®rst group of graduates from the
new course. Most educational supervisors of house
of®cers carry out the role for several years and so almost
all of those in the second group would have been the
same as those surveyed the previous year.
In order to obtain data from a large sample, a ques-
tionnaire was designed. We used the New Doctor1 as
the basis for de®ning the competencies required of a
graduating medical student rather than using the
objectives of the new Manchester curriculum, as we
considered that these might have biased the results in
favour of the new course. We also wanted to evaluate
the curriculum against national recommendations. The
New Doctor lists broad areas of competence, e.g.
`communicating effectively' as well as speci®c skills e.g.
`suturing', which in turn represent composites of
knowledge, skills and attitudes that should be built on
in `general clinical training'.
There were two sections on the questionnaire, the
®rst asking graduates to consider `How well did the course
prepare you for¼?' followed by a list of broad areas of
competence as de®ned in the New Doctor. In relation
to competence in more speci®c procedures, the ques-
tion was `How well did the course provide you with a
competence in¼?'. Respondents were required to rate
their answers on a ®ve point scale, with `very well
prepared/competent' and `not at all well prepared/
competent' as the range; the mid-point label was `quite
well prepared/competent'.
A similar questionnaire was sent to the educational
supervisors. The question asked of educational super-
visors was `Please rate the Manchester PRHOs on their
competence in the following¼', with the same list of items
as those on the graduate questionnaire. As educational
supervisors tend to supervise more than one PRHO at a
time they were asked to respond on the basis of a
general perception of graduates rather than on ratings
of an individual. Supervisors were asked to leave an
item blank if they felt unable to provide a rating.
Follow up letters and copies of the questionnaire
were sent to non-respondents, both graduates (four
follow-ups) and supervisors (one follow-up). Graduate
non-respondents were followed up ®rst at their home
address and then at their PRHO placement.
Analysis of the data was done using SPSS for
Windows version 7. Raw data were used for the ana-
lyses of variances between groups, although for ease of
presentation some results have been combined into
fewer groups. Non-parametric tests (Mann±Whitney
(U) were used to explore any signi®cant differences
between groups. As slightly different wordings of
questions were used for graduates and supervisors, it
was not appropriate to test for statistically signi®cant
differences between the two groups.
How well prepared are graduates for the role of pre-registration house of®cer? · A Jones et al. 19
Ó Blackwell Science Ltd MEDICAL EDUCATION 2002;36:16±25
Results
Graduates
The response from the traditional graduates was 171/256
(67%) and 163/267 (61%) from the graduates of the new
course. For the purpose of comparing traditional and
new course outcomes those graduates who had taken
more than ®ve years to complete their studies, for
example those who had done an intercalated degree,
were excluded. This was because those who graduated in
1999 and had commenced before 1994 would have done
part traditional and part new course. In order to avoid
any bias from reducing the new course cohort, the same
rule was adopted to exclude respondents from the tradi-
tional cohort. Consequently, the analysis was carried out
on the data from 123 (48%) traditional course graduates
and 138 (52%) new course graduates.
Solely for the purpose of presentation, the data were
recoded into the number rating above the mid-point on
the ®ve-point scale, those rating at the mid-point and
those below the mid-point, i.e. reduced to three groups.
All statistical tests were based on the raw data. Statis-
tical signi®cance was set at P ³ 0á01 because of the
multiple comparisons.
Table 1 shows the comparative data for graduate
responses to how well they felt prepared in broad
areas of competence. Responses to 12 of the 19 items
were signi®cantly different between the two groups.
All but one of these (`understanding disease pro-
cesses') were rated more favourably by the new course
graduates.
Table 1 Graduate ratings of broad competencies. `How well did the course prepare you for¼?'
% rating themselves as
% responding
to item
More than quite
well prepared
Quite well
prepared
Less than quite
well prepared
Item 1999 1998 1999 1998 1999 1998 1999 1998 P-value
History taking, clinical examination &
selection & interpretation of diagnostic tests
99á3 99á2 66á4 77á9 29á9 22á1 3á6 ± ns
Understanding disease processes 99á3 99á2 40á1 77á9 35á8 21á3 24á1 0á8 0á000
Communicating effectively 99á3 99á2 83á2 58á2 13á9 32á8 2á9 9á0 0á000
Being aware of your own limitations 99á3 99á2 69á3 57á4 25á5 38á5 5á1 4á1 ns
Working in a team 99á3 99á2 83á2 50á8 12á4 34á4 4á4 14á8 0á000
Recognition of the social & emotional factors
in illness and treatment
97á8 99á2 68á9 43á4 25á2 37á7 5á9 18á9 0á000
Keeping accurate records 98á6 99á2 37á5 37á7 45á6 36á1 16á9 26á2 ns
Using opportunities for disease prevention &
health promotion
98á6 99á2 33á8 33á6 52á9 41á0 13á2 25á4 ns
Managing time effectively 97á8 99á2 36á3 28á7 43á7 41á8 20á0 29á5 ns
Making the best use of laboratory & other
diagnostic services
98á6 99á2 30á9 32á8 49á3 49á2 19á9 18á0 ns
Understanding the relationship between
primary & social care & hospital care
97á8 99á2 68á1 28á7 28á1 45á9 3á7 25á4 0á000
Developing appropriate attitudes towards
personal health & wellbeing
97á1 99á2 43á3 27á0 35á8 41á8 20á9 31á1 0á006
Understanding the principles of evidence-
based medicine
97á8 97á6 53á3 26á7 33á3 35á8 13á3 37á5 0á000
Diagnosis, decision making & the provision
of treatment including prescribing
98á6 99á2 35á3 25á4 42á6 43á4 22á1 31á1 ns
Coping with uncertainty 98á6 99á2 50á0 23á8 28á7 35á2 21á3 41á0 0á000
Understanding the purpose & practice of
audit, peer review and appraisal
98á6 99á2 41á9 22á1 39á7 35á2 18á4 42á6 0á000
Providing appropriate care for people of
different cultures
99á3 99á2 51á1 18á9 30á7 31á1 18á2 50á0 0á000
Using informatics as a tool in medical practice 97á1 95á9 48á5 19á5 32á8 25á4 18á7 55á1 0á000
Being aware of legal & ethical issues 98á6 99á2 46á3 18á0 44á1 32á8 9á6 49á2 0á000
ns � not signi®cant.
How well prepared are graduates for the role of pre-registration house of®cer? · A Jones et al.20
Ó Blackwell Science Ltd MEDICAL EDUCATION 2002;36:16±25
The responses from the two cohorts of graduates
in response to the question about preparedness for
performing speci®c skills are shown in Table 2.
Responses to eight of the 13 items were signi®cantly
different between the two groups, with responses from
the new course graduates being more favourable.
Educational supervisors
There were 194 educational supervisors in the North-
west region who were supervising Manchester gradu-
ates from the traditional course and 218 supervisors of
the new course graduates. All were sent questionnaires,
but ®ve supervisors of the traditional course graduates
and 12 of the new course graduates responded stating
that they could not complete a questionnaire either
because they were no longer supervising graduates or
felt unable to generalise having had too few graduates
under their supervision. In total, 151 (80%) completed
questionnaires were received from supervisors of tradi-
tional course graduates and 169 (82%) from new course
graduates.
As with the graduate questionnaire, data were
recoded into just three groups for clarity of presenta-
tion, with statistical tests based on the raw data and
signi®cance set at P ³ 0á01.
The responses of the supervisors as to how well they
thought graduates were prepared in broad areas of
competence are shown in Table 3. There was a statis-
tically signi®cant difference between the ratings of the
traditional and new course graduates for ®ve of the
18 listed competencies, with the graduates from the
new course being rated as more prepared.
There were no signi®cant differences for any of the
speci®c skills (Table 4).
Discussion
Results from graduates tended to favour the new course,
although the less favourable rating for one of the broad
areas of competence, understanding disease processes,
is of some concern. The results from supervisors were
similar for the two courses in terms of speci®c skills, but
were supportive of the new course for broad areas of
competence. These data suggest that in part the aims of
the innovative course have been met although the per-
ceived reduction in preparedness for `understanding
disease processes' needs more exploration.
The measurement of outcomes of undergraduate
medical education raises speci®c methodological issues.
This study attempts to measure perceptions of the
preparedness of graduates and compare those percep-
tions across courses. We do not claim to have measured
or reported differences in performance once graduates
are working as pre-registration house of®cers. It is
known that self-assessment may not be closely corre-
lated to actual performance.13
In our comparative analysis, we picked one point in
the graduate's career at which to collect the data on
perceptions of preparedness. It could be argued that
Table 2 Graduate ratings of speci®c skills. `How well did the course provide you with a competence in¼?'
% rating themselves as
% responding
to item
More than quite
competent
Quite
competent
Less than quite
competent
Item 1999 1998 1999 1998 1999 1998 1999 1998 P-value
Venepuncture 98á6 99á2 89á7 70á5 7á4 23á0 2.9 6.6 0á001
Basic CPR 99á3 99á2 70á8 65á6 24á8 23.0 4á4 11á5 ns
Arterial blood sampling 99á3 99á2 54á0 50á0 21á2 30á3 24á8 19á7 ns
Administering oxygen therapy safely 98á6 99á2 30á9 27á0 29á4 29á5 39á7 43á4 ns
Urinary catheterisation 99á3 99á2 58á4 32á8 21á2 25á4 20á4 41á8 0á000
Obtaining valid consent 98á6 97á6 44á1 25á0 26á5 24á2 29á4 50á8 0á000
Performing an ECG 99á3 99á2 71á5 26á2 16á8 20á5 11á7 53á3 0á000
Writing a prescription 99á3 98á4 39á4 19á0 33á6 24á0 27á0 57á0 0á000
Control of haemorrhage 99á3 99á2 14á6 14á8 35á0 28á7 50á4 56á6 ns
Calculating accurate drug dosages 99á3 98á4 24á8 14á9 27á0 25á6 48á2 59á5 ns
Correctly using a nebuliser 99á3 99á2 21á2 10á7 25á5 23á0 53á3 66á4 0á002
Suturing 99á3 99á2 11á7 9á8 21á2 8á2 67á2 82á0 0á010
Inserting a nasogastric tube 99á3 99á2 8á8 4á1 16á8 8á2 74á5 87á7 0á005
ns � not signi®cant.
How well prepared are graduates for the role of pre-registration house of®cer? · A Jones et al. 21
Ó Blackwell Science Ltd MEDICAL EDUCATION 2002;36:16±25
differences in outcomes may be more or less apparent
at a later stage in their career and so we do not have a
true picture of the long-term effect of either the tradi-
tional or new course.
A further potential drawback is the effect of removing
from the data set the responses of graduates who had
taken more than ®ve years to complete their course. We
could not identify the educational supervisors of these
house of®cers so the supervisor ratings were based on
all Manchester graduates. However, the numbers
involved were small so it is unlikely that this had a
major effect on the results.
The pattern of responses may also have been in¯u-
enced by the educational supervisors of the traditional
course graduates basing their perceptions on Man-
chester graduates observed over a number of years,
whereas the new graduate supervisors, who may have
also replied to the ®rst questionnaire, would have
based their ratings on a small number of PRHOs they
had worked with in the previous few months. How-
ever, it is known that in assessment, supervisors base
their ratings on very recent experience of the trainee
and are heavily in¯uenced by more recent inter-
actions14 so even with the traditional course, supervi-
sors were probably basing their ratings on their house
of®cers at that time.
Interpretation of the results of this study should also
be made in the knowledge that supervisors tend to be
more lenient in their ratings than trainees and that there
is a tendency for supervisors to rate on the basis of their
overall impression of a graduate rather than speci®c
aspects of performance.15 However, others have sug-
gested that this possible `halo effect' is unlikely to cause
signi®cant bias.7
Table 3 Educational supervisor ratings of broad competencies. `Please rate the Manchester PRHOs on their competence in the following¼'
% rating PRHOs as
% responding
to item
More than quite
competent
Quite
competent
Less than quite
competent
Item 1999 1998 1999 1998 1999 1998 1999 1998 P-value
Understanding disease processes 98á8 99á3 40á1 36á0 46á7 48á7 13á2 15á3 ns
Communicating effectively 99á4 98á7 63á1 48á3 31á5 47á0 5á4 4á7 0á006
Being aware of their own limitations 97á0 97á4 64á6 59á2 29á3 38á1 6á1 2á7 ns
Working in a team 99á4 99á3 65á5 56á7 28á6 38á7 6á0 4á7 ns
Recognition of the social & emotional factors
in illness and treatment
92á9 94á0 41á4 40á1 49á0 45á8 9á6 14á1 ns
Keeping accurate records 92á3 95á4 51á9 56á9 35á9 34á7 12á2 8á3 ns
Using opportunities for disease prevention &
health promotion
81á7 81á5 21á7 16á3 52á9 45á5 25á4 38á2 ns
Managing time effectively 97á6 97á4 41á8 30á6 40á6 50á3 17á6 19á0 ns
Making the best use of laboratory &
other diagnostic services
99á4 98á0 35á1 23á6 42á9 51á4 22á0 25á0 ns
Understanding the relationship between
primary & social care & hospital care
91á7 90á7 29á7 21á9 52á3 54á7 18á1 23á4 ns
Developing appropriate attitudes towards
personal health & wellbeing
87á6 86á8 35á1 42á7 58á8 48á1 6á1 9á2 ns
Understanding the principles of evidence-
based medicine
94á1 93á4 35á2 27á7 53á5 56á0 11á3 16á3 ns
Diagnosis, decision making & the provision of
treatment including prescribing
98á8 99á3 33á5 34á7 48á5 48á7 18á0 16á7 ns
Coping with uncertainty 93á5 93á4 34á2 17á0 44á3 54á6 21á5 28á4 0á002
Understanding the purpose & practice of
audit, peer review and appraisal
96á4 92á1 30á7 20á1 46á6 40á3 22á7 39á6 0á001
Providing appropriate care for people of
different cultures
86á4 90á7 55á5 39á4 41á8 50á4 2á7 10á2 0á001
Using informatics as a tool in medical practice 84á0 77á5 38á0 23á9 45á1 51á3 16á9 24á8 ns
Being aware of legal & ethical issues 91á1 92á1 25á3 17á3 53á9 43á9 20á8 38á8 0á001
ns � not signi®cant.
NB `History taking, clinical examination and the selection and interpretation of diagnostic tests' was not included as an item on the 1998
supervisor questionnaire.
How well prepared are graduates for the role of pre-registration house of®cer? · A Jones et al.22
Ó Blackwell Science Ltd MEDICAL EDUCATION 2002;36:16±25
In addition to the general aspects of supervisor rat-
ings, it is also likely that some supervisors may not have
been in a position to comment accurately on some of the
items as they may not have seen graduates performing
certain skills or demonstrating certain broad compe-
tencies. This is supported by the lower response rates
by supervisors to some items on the questionnaire,
particularly in relation to some of the speci®c skills
(Tables 3 and 4). Other supervisors may have simply
rated around the mid-point. It could be argued that
junior medical staff or nurses working with the gradu-
ates may have been better placed to comment on their
performance in speci®c areas.
Given these limitations to the methods, what are the
main ®ndings from the study? Overall, the ratings for
the new course in preparing graduates for practice may
be attributed to the substantial change in methods of
teaching and learning that occurred with curriculum
reform. For example, differences in favour of the new
course for the items `communicating effectively' and
`working in a team', may be linked to the PBL group
work. Others have reported similar results. One study
found that supervisor ratings favoured PBL graduates
in respect of communication skills.15 Similarly, the PBL
graduates in another study were rated signi®cantly
better than traditional course graduates for their rela-
tionships with patients and their families and relation-
ships with other health professionals.7
We also found that the new graduates were better
prepared in terms of `understanding the relationship
between primary and social care and hospital care' and
`providing appropriate care for people of different cul-
tures'. As encouraged by the GMC,2 we have placed a
much greater emphasis on learning in a community
setting as well as integrating this with hospital experi-
ence. Similarly, the graduates from the `New Pathway'
from Harvard medical school expressed more con®d-
ence in the wider aspects of medicine.16
We have previously reported a preliminary study
looking at the attitudes of the new course students in
Manchester compared to those of students from tradi-
tional courses.17 As well as having a more holistic
approach to care and a commitment to continued
learning, the year three students were more positive
about `coping with uncertainty'. This last attitude has
been maintained into practice and was perceived by the
educational supervisors. It has probably been rein-
forced by the nature of studying using PBL, in which
students are encouraged to discuss where sources of
information are contradictory or where there is no
reliable evidence. We believe that this approach will be
helpful to the graduates as they continue with their
postgraduate development.
The difference in ratings for `being aware of legal and
ethical issues' is likely to be a re¯ection of the increased
emphasis on this area in the new course. Others have
reported that medical ethics and law were amongst the
subjects that graduates felt there had not been enough
`factual content' in their training.18 More recently, the
teachers of ethics and law have called for these to be
Table 4 Educational supervisor ratings of speci®c skills. `Please rate the Manchester PRHOs on their competence in the following¼'
% rating PRHOs as
% responding
to item
More than quite
competent
Quite
competent
Less than quite
competent
Item 1999 1998 1999 1998 1999 1998 1999 1998 P-value
Venepuncture 92á3 94á7 50á6 60á1 38á5 33á6 10á9 6á3 ns
Basic CPR 76á9 79á5 33á8 41á7 46á2 43á3 20á0 15á0 ns
Arterial blood sampling 85á8 89á4 41á4 41á5 36á6 32á6 22á1 25á9 ns
Administering oxygen therapy safely 78á7 86á1 39á8 40á0 41á4 41á5 18á8 18á5 ns
Urinary catheterisation 75á7 82á8 39á1 31á2 45á3 45á6 15á6 23á2 ns
Obtaining valid consent 87á0 92á1 42á9 48á9 48á3 40á3 8á8 10á8 ns
Performing an ECG 75á1 78á1 40á2 42á4 44á9 40á7 15á0 16á9 ns
Writing a prescription 91á1 94á7 48á7 51á7 43á5 42á0 7á8 6á3 ns
Control of haemorrhage 61á5 71á5 28á8 22á2 51á0 50á0 20á2 27á8 ns
Calculating accurate drug dosages 85á8 91á4 41á4 47á1 48á3 49á3 10á3 3á6 ns
Correctly using a nebuliser 74á6 78á8 34á1 36á1 46á8 43á7 19á0 20á2 ns
Suturing 50á9 56á3 14á0 15á3 25á6 30á6 60á5 54á1 ns
Inserting a nasogastric tube 53á3 69á5 11á1 18á1 35á6 33á3 53á3 48á6 ns
ns � not signi®cant.
How well prepared are graduates for the role of pre-registration house of®cer? · A Jones et al. 23
Ó Blackwell Science Ltd MEDICAL EDUCATION 2002;36:16±25
core subjects in the undergraduate curriculum.19 As we
have an integrated course, we do not have a speci®c
ethics and law module, but we believe that the inclusion
of ethical and legal dilemmas in the PBL cases20 has
gone some way to developing awareness of these in the
Manchester graduates.
Within the broad competences, we found that the
new course graduates did not feel as well prepared for
`understanding disease processes'. However, in pub-
lished reviews, consistent differences in the knowledge
of graduates from the two types of courses have not
been found.4,21 A more recent study compared stu-
dents' performance on knowledge tests between those
who had followed a traditional course and those from a
PBL course. They concluded that the use of PBL
methods during the pre-clinical phase does not dis-
advantage students when they are tested on their basic
science and clinical knowledge.22
In relation to applying knowledge, it has been
reported that students from a PBL curriculum had
greater diagnostic accuracy.8 Similarly PBL students at
Rush Medical College were able to give better patho-
physiological explanations for patients' problems.23 In
Manchester, we found that in the ®nal examination, the
graduates from the new course scored signi®cantly
higher in identical questions in a patient management
problem paper than those from the last two years of the
traditional course (data on ®le: New course 1999: mean
score 62á3% [s.d. 8á8]; Traditional course 1998: mean
score 59á8% [s.d. 9á1]; Traditional course 1997: mean
score 58á0% [s.d. 7á7], P < 0á001).
From our data and the literature, we do not have
evidence that the difference in rating of `understanding
disease processes' does relate to a real gap in prepar-
edness. The educational supervisors in Manchester,
unlike the nurse raters15 or the supervisors of interns
surveyed elsewhere,7 did not rate the new graduates as
being less well prepared in this area. One possible
explanation for our results may be similar to the
`growing pains' or anxieties inherent in the ®rst year of a
new course discussed by Mann and Kaufman.24 They
contrasted the perceptions of the ®rst cohort of
graduates from an innovative Canadian course with
their performance in Part 1 of the Canadian Qualifying
exam and found that their perceptions that their
knowledge base was weak were unsupported.
The second part of the survey concerned speci®c
skills. Others have reported that PRHOs acquire the
necessary skills by the end of the pre-registration house
of®cer year, but that having had those skills to start with
would have made the transition easier.25 Our results
suggest that the development of a core set of skills has
had a bene®cial effect on perceptions of levels of
competence by the new course graduates, but not by
the supervisors. Other probable reasons for the higher
ratings from graduates were the implementation of
skills laboratories and the ®nal year being based entirely
in a clinical environment. At the end of this ®fth year,
the ®nal year students `shadow' the PRHO that they
will be taking over from after graduation and conse-
quently have a good opportunity to improve their skills
where necessary. Our skills curriculum continues
to develop so we hope that supervisors will perceive
improved competence in the future.
Conclusion
Overall, our study shows graduates from an innovative
course feel better prepared for the role of PRHO.
Considering the ®ndings of other studies and our recent
data, there is still a need for further research into the
effectiveness of non-traditional curricula to establish
the extent to which teaching and learning methods can
affect outcomes. The research presented here has
focused on perceptions and there is very little literature
on the actual behaviours of graduates from innovative
courses.5 Given the energy and resources that have
been put into curriculum reform in the UK since the
publication of `Tomorrow's Doctors',2 it is very
important that we seek evidence for any differences
produced. The current study is part of a much larger
programme of evaluation using quantitative and qual-
itative methods, which can be used to build a more
comprehensive picture of the outcomes of medical
teaching at Manchester.
Acknowledgements
We would like to thank the graduates and their
educational supervisors who took part in the survey,
and Professors Carl Whitehouse, Roger Green and
David Gordon for their helpful comments on early
drafts of this paper.
Con¯ict of interest
None declared.
Contributors
AJ and PO'N jointly wrote the paper in consultation
with PM. PO'N initiated, supervised and guided the
project and AJ was responsible for designing the
research instruments and analysing the results. PM
acted as external consultant to the project and assisted
with revision of the manuscript.
How well prepared are graduates for the role of pre-registration house of®cer? · A Jones et al.24
Ó Blackwell Science Ltd MEDICAL EDUCATION 2002;36:16±25
Funding
The project is funded by the NHS, Northwest Region.
References
1 General Medical Council. The New Doctor. London: GMC;
1997.
2 General Medical Council. Tomorrow's doctors. London: GMC;
1993.
3 British Medical Association. Report of the working party on
medical education. London: BMA; 1995.
4 Vernon DTA, Blake RL. Does Problem-based Learning
Work? A meta-analysis of evaluative research. Ac Med
1993;68:550±63.
5 Shin JH, Haynes BH, Johnston ME. Effect of problem-based,
self-directed undergraduate education on life-long learning.
Can Med Assoc J 1993;148:969±76.
6 Colliver J. Effectiveness of problem based learning curricula.
Ac Med 2000;75:259±66.
7 Rolfe IE, Andren JM, Pearson S, Hensley MJ, Gordon JJ.
Clinical competence of interns. Med Educ 1995;29:225±30.
8 Schmidt HG, Machiels-Bongaerts M, Hermans H, ten
Cate TJ, Venekamp R, Boshuizen HPA. The development
of diagnostic competence: comparison of a problem-based,
an integrated and a conventional medical curriculum. Ac Med
1996;71:658±64.
9 O'Neill PA. Problem-based learning alongside clinical
experience: Reform of the Manchester curriculum. Educ for
Health 1998;11:37±48.
10 Whitehouse C. Planning for community-oriented medical
education in Manchester. Educ for Health 1996;9:45±59.
11 Barrows HS. A taxonomy of problem-based learning methods.
Med Educ 1986;20:481±6.
12 Whitehouse CR. The community experience in the new
Manchester medical undergraduate curriculum: reactions
to the ®rst module. Educ for Health 1998;11:173±82.
13 Gordon MJ. A review of the validity and accuracy of self-
assessments in health professional training. Ac Med
1991;66:762±9.
14 Ross M. Relation of implicit theories to the construction of
personal histories. Psychol Rev 1989;96:341±57.
15 Santos-Gomez L, Kalishman S, Rezler A, Skipper B, Mennin
SP. Residency performance of graduates from a problem-
based and a conventional curriculum. Med Educ
1990;24:366±75.
16 Peters AS, Greenberger-Rosovsky R, Crowder C, Block SD,
Moore GT. Long-term outcomes of the New Pathway pro-
gram at Harvard Medical School: a randomized controlled
trial. Ac Med 2000;75:470±9.
17 Silverstone Z. Tomorrow's Doctors ± Tomorrow's Attitudes?
Presented at the Association for the Study of Medical
Education Annual Scienti®c Meeting;1997.
18 Clack GB. Medical graduates evaluate the effectiveness of
their education. Med Educ 1994;28:418±31.
19 Consensus group of teachers of medical ethics and law in
UK medical schools. Teaching medical ethics and law within
medical education. A model for the UK core curriculum.
J Med Ethics 1998;24:188±92.
20 Singer PA. Medical ethics. BMJ 2000;321:282±5.
21 Albanese MA, Mitchell SM. Problem-based learning: a review
of literature on its outcomes and implementation issues.
Ac Med 1993;68:52±81.
22 Blake RL, Hosokawa MC, Riley SL. Student performances on
Step 1 and Step 2 of the United States Medical Licensing
Examination following implementation of a problem-based
learning curriculum. Ac Med 2000;75:66±70.
23 Hmelo CE. Cognitive consequences of problem-based learn-
ing for the early development of medical expertise. Teaching
Learning Med 1998;10:92±100.
24 Mann KV, Kaufman DM. A comparative study of problem-
based and conventional undergraduate curricula in preparing
students for graduate medical education. Ac Med 1999;74:
S4±S6.
25 Calman KC, Donaldson M. The pre-registration house of®cer
year: a critical incident study. Med Educ 1991;25:51±9.
Received 23 March 2001; accepted for publication 29 March 2001
How well prepared are graduates for the role of pre-registration house of®cer? · A Jones et al. 25
Ó Blackwell Science Ltd MEDICAL EDUCATION 2002;36:16±25