EVALUATION OF MEDICAL CURRICULA

2
1014 prescriptions for the training of specialists ", and part 11 of the report consists of synopses of recommended training programmes. These so far avoid the charge of rigidity as to be almost infinitely plastic, but the introduction assures us that a more detailed programme will be sent to trainees as they enrol in the specialty(ies) of their choice. The joint committee adheres to the colleges’ view of the M.R.C.P. examination as a basic qualification for entry to specialist training which will begin when the M.R.c.p.-holder obtains " an approved post in the specialty of his choice ". There will not usually be any further examination to be taken for accreditation as a specialist. This will be based largely on reports from regional postgraduate committees and the like. Here the joint committee demonstrates its realism by pointing out sadly that some trainees may become consultants in " shortage specialties " before completing a recommended training period. Under these circumstances it proposes to accept the inevit- able and grant the necessary accreditation. It is the recognition of training posts which gives the Joint Higher Committee its true purpose. For the first time the physicians demonstrate their intent to maintain a list of approved posts, through the specialist advisory committees which constitute the active branches of the joint committee. The composition of visiting teams, and the guidelines they will follow when inspecting hospitals, are all set out. These are the teeth which all previous published guidance has lacked. It is intended to start the process of recognition of posts " later in the year ", and we must wait to see how sharp the teeth prove to be. The visiting teams must search vigorously if they are to find out the truth, and they must not be afraid of exposing skeletons in some distinguished respectable cupboards. The joint committee will be judged by the standards which it demands. All over the country consultants, medical cogwheel divisions, and H.M.c.s should now start to examine themselves carefully in preparation for visitation, lest they be found wanting. EVALUATION OF MEDICAL CURRICULA PHARMACEUTICAL companies rightly spend a sizeable fraction of their income on research and development- on investigating the nature and modes of action of drugs, on developing new and better drugs, and on evaluating methods of production. Medical schools are likewise aware of scientific methodology, but it is unusual for a school to spend substantial amounts of time or money on product research. Lately, numerous medical schools have made major changes in their curricula and most have made many minor changes, yet there have been few carefully planned attempts to evaluate the effects of such changes. Before changing the curriculum the first question to ask is, What are we trying to do ? The answer will embrace behaviourally defined aims and objectives, including the acquisition and retention of medical knowledge and techniques, and above all the ability to use them. Allied to this will be the second question, What should we be trying to do ? Here society’s health needs and the intellectual and personal needs of the student will receive attention. The next question, Levit 1 suggests, should be, How are we doing now ? The answer will entail recognition of the processes and effects of the existing curriculum. Any medical school has several varieties of curriculum: the " ostensible curriculum " (which is published but seldom directly followed) usually implies a degree of completeness and omniscience unattainable in prac- tice ; the " intended curricula " represent the ambitions of individual departments and teachers, varying widely in extent and in vector, and seldom coordinated; and the " individual curricula " are those actually followed by students-sometimes differing widely even within an officially non-elective programme, and certainly differing from the published and officially acknowledged curricula. 2 When these questions are answered, the last is, What curriculum changes are really needed ? Jason,3 Levit,l and Salzman et al. at Rochester have discussed the rationale of curricular evaluation and the development of a " minitest ", in conjunction with the American National Board of Medical Examiners, to assess the effects of changes. Jason emphasises the need to look at three main phases-the identifiable characteristics of the students entering the course, the effects and outcomes of the course (not only in cognitive gain, but also in changes in attitudes, perceptions, and career plans); and the final career success of graduates. The Tripod Test,5 a reliable and internationally validated multiple-choice (M.c.Q.) test available in two parallel forms, has been used with considerable success 6-8 to investigate cognitive gain and loss in separate subjects over various stages of the curriculum and to examine the effects of specific changes. Gilbert used it to show a 22 % overall gain during the final two years in medical school in Edmonton, Canada; van Dorp,9 in Leiden, has demonstrated very little cognitive gain during the phase of clinical clerkships. Palva,lO in Oulu, Finland, has reported similar findings using another M.c.Q. test. With careful planning, using variations of the basic pre-test/post-test design, the data are remarkably precise: one can show convincingly that on certain questions students do less well after receiving teaching than they did before. Now Sinclair," at Aberdeen, has reported on a specially designed monitoring examination to assess the effects of the new curriculum there. Multiple- choice examinations in pharmacology and anatomy, and a series of patient-management problems, 12 were 1. Levit, E. J. J. med. Educ. 1967, 42, 930. 2. Simpson, M. A., van den Bussche, R. Descriptive Profile of Indi- vidual Curriculum Variables. Institut für Ausbildungsforschung, Munich, 1970 (see ref. 7, p. 179). 3. Jason, H. J. med. Educ. 1966, 41, suppl. p. 18. 4. Salzman, L. F., Donovan, J. C., Allen, P. Z. ibid. 1972, 47, 631. 5. Kapuste, H. C., Noack, H., Götz, A. Tripod Test, an International Test for Medical Students. Institut für Ausbildungsforschung, Munich, 1969. 6. Kapuste, H. C. The Tripod Test—an Instrument for International Research in Medical Schools. Institut für Ausbildungsforschung, Munich, 1969. 7. Simpson, M. A. Medical Education: a Critical Approach; p. 148. London, 1972.2 8. Gilbert, J. A. L. in Tripod Test Document Résumés. Institut fur Ausbildungsforschung, Munich, 1969. 9. van Dorp, C. I.R.M.E. Newsletter, 1972, 4, 11. 10. Palva, I. P. ibid. 1972, 5, 10. 11. Sinclair, D. Br. J. med. Educ. 1972, 6, 176. 12. McGuire, C., Solomon, L. M. Clinical Simulations. New York, 1971.

Transcript of EVALUATION OF MEDICAL CURRICULA

Page 1: EVALUATION OF MEDICAL CURRICULA

1014

prescriptions for the training of specialists ", and part11 of the report consists of synopses of recommendedtraining programmes. These so far avoid the charge ofrigidity as to be almost infinitely plastic, but theintroduction assures us that a more detailed programmewill be sent to trainees as they enrol in the specialty(ies)of their choice. The joint committee adheres to thecolleges’ view of the M.R.C.P. examination as a basicqualification for entry to specialist training which willbegin when the M.R.c.p.-holder obtains " an approvedpost in the specialty of his choice ". There will notusually be any further examination to be taken foraccreditation as a specialist. This will be based largelyon reports from regional postgraduate committees andthe like. Here the joint committee demonstrates itsrealism by pointing out sadly that some trainees maybecome consultants in " shortage specialties " beforecompleting a recommended training period. Underthese circumstances it proposes to accept the inevit-able and grant the necessary accreditation.

It is the recognition of training posts which gives theJoint Higher Committee its true purpose. For thefirst time the physicians demonstrate their intent tomaintain a list of approved posts, through the specialistadvisory committees which constitute the activebranches of the joint committee. The composition ofvisiting teams, and the guidelines they will follow wheninspecting hospitals, are all set out. These are theteeth which all previous published guidance has lacked.It is intended to start the process of recognition ofposts " later in the year ", and we must wait to see howsharp the teeth prove to be. The visiting teams mustsearch vigorously if they are to find out the truth, andthey must not be afraid of exposing skeletons in somedistinguished respectable cupboards. The jointcommittee will be judged by the standards which itdemands. All over the country consultants, medicalcogwheel divisions, and H.M.c.s should now start toexamine themselves carefully in preparation forvisitation, lest they be found wanting.

EVALUATION OF MEDICAL CURRICULA

PHARMACEUTICAL companies rightly spend a sizeablefraction of their income on research and development-on investigating the nature and modes of action ofdrugs, on developing new and better drugs, and onevaluating methods of production. Medical schoolsare likewise aware of scientific methodology, but it isunusual for a school to spend substantial amounts oftime or money on product research. Lately, numerousmedical schools have made major changes in theircurricula and most have made many minor changes,yet there have been few carefully planned attempts toevaluate the effects of such changes.

Before changing the curriculum the first questionto ask is, What are we trying to do ? The answer willembrace behaviourally defined aims and objectives,including the acquisition and retention of medicalknowledge and techniques, and above all the abilityto use them. Allied to this will be the second question,What should we be trying to do ? Here society’shealth needs and the intellectual and personal needsof the student will receive attention. The next question,

Levit 1 suggests, should be, How are we doing now ?The answer will entail recognition of the processesand effects of the existing curriculum. Any medicalschool has several varieties of curriculum: the" ostensible curriculum " (which is published butseldom directly followed) usually implies a degree ofcompleteness and omniscience unattainable in prac-tice ; the " intended curricula " represent the ambitionsof individual departments and teachers, varyingwidely in extent and in vector, and seldom coordinated;and the " individual curricula " are those actuallyfollowed by students-sometimes differing widelyeven within an officially non-elective programme, andcertainly differing from the published and officiallyacknowledged curricula. 2 When these questions areanswered, the last is, What curriculum changes arereally needed ?

Jason,3 Levit,l and Salzman et al. at Rochesterhave discussed the rationale of curricular evaluationand the development of a " minitest ", in conjunctionwith the American National Board of MedicalExaminers, to assess the effects of changes. Jasonemphasises the need to look at three main phases-theidentifiable characteristics of the students enteringthe course, the effects and outcomes of the course(not only in cognitive gain, but also in changes inattitudes, perceptions, and career plans); and thefinal career success of graduates. The Tripod Test,5 areliable and internationally validated multiple-choice(M.c.Q.) test available in two parallel forms, has beenused with considerable success 6-8 to investigatecognitive gain and loss in separate subjects over

various stages of the curriculum and to examine theeffects of specific changes. Gilbert used it to show a22 % overall gain during the final two years in medicalschool in Edmonton, Canada; van Dorp,9 in Leiden,has demonstrated very little cognitive gain during thephase of clinical clerkships. Palva,lO in Oulu, Finland,has reported similar findings using another M.c.Q.test. With careful planning, using variations of thebasic pre-test/post-test design, the data are remarkablyprecise: one can show convincingly that on certainquestions students do less well after receiving teachingthan they did before.Now Sinclair," at Aberdeen, has reported on a

specially designed monitoring examination to assessthe effects of the new curriculum there. Multiple-choice examinations in pharmacology and anatomy,and a series of patient-management problems, 12 were

1. Levit, E. J. J. med. Educ. 1967, 42, 930.2. Simpson, M. A., van den Bussche, R. Descriptive Profile of Indi-

vidual Curriculum Variables. Institut für Ausbildungsforschung,Munich, 1970 (see ref. 7, p. 179).

3. Jason, H. J. med. Educ. 1966, 41, suppl. p. 18.4. Salzman, L. F., Donovan, J. C., Allen, P. Z. ibid. 1972, 47, 631.5. Kapuste, H. C., Noack, H., Götz, A. Tripod Test, an International

Test for Medical Students. Institut für Ausbildungsforschung,Munich, 1969.

6. Kapuste, H. C. The Tripod Test—an Instrument for InternationalResearch in Medical Schools. Institut für Ausbildungsforschung,Munich, 1969.

7. Simpson, M. A. Medical Education: a Critical Approach; p. 148.London, 1972.2

8. Gilbert, J. A. L. in Tripod Test Document Résumés. Institut furAusbildungsforschung, Munich, 1969.

9. van Dorp, C. I.R.M.E. Newsletter, 1972, 4, 11.10. Palva, I. P. ibid. 1972, 5, 10.11. Sinclair, D. Br. J. med. Educ. 1972, 6, 176.12. McGuire, C., Solomon, L. M. Clinical Simulations. New York,

1971.

Page 2: EVALUATION OF MEDICAL CURRICULA

1015

used to study two cohorts of students educated underthe old and the new systems. Despite major changesin the final year, no significant difference was foundbetween the groups in their ability to deal withsimulated clinical problems, or in their pharmacologyscores, though the large reduction in anatomy teachinghours seemed to produce a small decrease in anatomyscores. The Aberdeen investigation certainly indicatesthe feasibility of objective curricular evaluation inBritain, but the experimental design was not perfect.The objectives of the curriculum changes had notbeen clearly stated, so the hypotheses tested by themonitoring examination were necessarily vague. Onlya limited number of questions were used to test verylarge subject areas, which is unfortunate in view ofthe potential flexibility of the techniques. Then, too,the experiment relied entirely on the good will ofvolunteer students, so the numbers in some groupswere small and possibly unrepresentative. Studentscan be highly motivated to take part in such experi-ments, as Kapuste 6 has shown: when the test usedprovides useful, detailed feedback, some 77% said

they had definitely enjoyed taking part, and 87%stated that they would participate in a similar experi-ment again without hesitation. Finally, it is note-

worthy how the clinical problems used by Sinclairrevealed differences between ability to recall factsand ability to use them. " A candidate who obtaineda distinction in surgery allowed the patient (in oneproblem) to die through a total lack of any attempt atresuscitation, even though this was suggested to himseveral times in the course of the problem. Again, astudent with excellent marks in medicine allowed the

patient to die without any attempt to examine eitherthe patient or her urine; he sent for the psychiatristright away.""’ The implication that one can graduatein medicine with distinction while unable to avertdisaster in practical clinical situations is thought-provoking.

CHICKEN DINNERS

IN the September Journal of Hygiene,18 DianeRoberts reports the results of an investigation on thethawing and spit-roasting of frozen dressed chickenin relation to the hazards of food-poisoning. Salmonel-losis after consumption of chicken dishes has beenascribed to inadequate thawing and cooking.114 In alimited survey on frozen raw chickens from fourpacking stations, rates of isolations of salmonellasvaried from a low figure of 3% to 62% (mean 35%).All nine serotypes isolated were known to cause

salmonellosis in man. Clostridium welchii and Staphylo-coccus aureus were found in 62 % of birds. When

fully thawed chickens were cooked until they looked" done ", salmonellx and Staph. aureus were invari-ably killed but Cl. welchii frequently survived. Whenchickens were not fully thawed, cooking did not alwaysdestroy all salmonellae, but the survivors would beunlikely to cause disease if the meat was eaten im-mediately. When the organisms had a chance to grow(stored at the temperature of a warm kitchen, or

13. Roberts, D. J. Hyg., Camb. 1972, 70, 565.14. Semple, A. B., Turner, G. C., Lowry, D. M. O. Br. med. J. 1968,

iv, 801.

at that of a warm cabinet at too low a temperature-setting), they increased to numbers which could causeclinical disease.

Careful thawing and cooking may lessen the dangersof food-poisoning, but much more important is whathappens after cooking. The organisms in raw birdsmust be prevented from reaching cooked birds.Cross-contamination can be reduced by: havingseparate areas, surfaces, utensils, and even personnel,for handling raw and cooked birds; thorough cleaningof all surfaces and utensils as soon as the chickens gointo the oven; and hand-washing between handlingraw and cooked chickens. Staph. aureus is commonlypresent on human skin, and is not readily removed bywashing; cooked chickens should be handled as littleas possible-they should be dissected with meat

secateurs, and carved just before they are served.If poultry is not to be eaten as soon as it has beencooked, correct storage is most important. Warmcabinets should be maintained at temperatures hotenough to prevent survival and multiplication of food-poisoning organisms-particularly Cl. welchii, the

spores of which can survive cooking. A minimumwarm storage temperature of65°C (149°F) is suggested.When cooked poultry is to be eaten cold, it should becooled rapidly and refrigerated for a limited time.

So long as raw chicken and other raw meats arecontaminated with salmonella: the danger of food-poisoning will always be there. Much of the answerlies in careful attention to breeding-stock, goodanimal husbandry, and use of salmonella-free feedingmeals. A programme for control of poultry diseasehas been started in Denmark, and salmonellosis inpoultry has declined. The number of differentsalmonella serotypes isolated has also decreased, untilthe only serotype which remains as a problem isS. typhimurium. Legislation in Denmark to ensuresalmonella-free feeding-meals has eliminated the widerange of serotypes which can reach poultry. Untilsimilar measures can be adopted in Britain, caterers willhave to pay more attention to prevention of cross-con-tamination and to correct storage of the cooked birds.

THYROTROPHIN-RELEASING HORMONE ANDDEPRESSION

THE report by Professor Prange and his co-workers 1

(and on p. 999 this week) that thyrotrophin-releasinghormone (T.R.H.) has an antidepressant action hasbeen confirmed by other investigators. 2 The thera-

peutic action is remarkably rapid, being observedwithin a few hours of the intravenous infusion of thehormone. The effect lasts 2-3 days, after which thepatient relapses.The findings raise several important questions. Is

the action of T.R.H. on mood a pharmacological effectconfined to depressive patients or does T.R.H., andpossibly other hypothalamic hormones, have a physio-logical role in the control of mood and behaviour ?At present we have no data of the effect of T.R.H. onnormal man or animals. The mode of action ofT.R.H. on mood is also unclear. Prange et al.l found

1. Prange, A. J., Wilson, I. C. Psychopharmacologia, 1972, 26, suppl. 82.2. Kastin, A. J., Ehrensing, R. H., Schalch, D. S., Anderson, M. S.

Lancet, Oct. 7, 1972, p. 740.