Evidence-Based MedicineA New Approach to Teaching the Practice of MedicineEvidence-Based Medicine Working Group

A NEW paradigm for medical practiceis emerging. Evidence-based medicinede-emphasizes intuition, unsystematicclinical experience, and pathophysiolog-ic rationale as sufficient grounds for clin-ical decision making and stresses theexamination ofevidence from clinical re-search. Evidence-based medicine re-

quires new skills of the physician, in-cluding efficient literature searching andthe application of formal rules of evi-dence evaluating the clinical literature.An important goal of our medical res-

idency program is to educate physiciansin the practice of evidence-based med-icine. Strategies include a weekly, for-mal academic half-day for residents, de-voted to learning the necessary skills;recruitment into teaching roles of phy-sicians who practice evidence-basedmedicine; sharing among faculty of ap-proaches to teaching evidence-basedmedicine; and providing faculty withfeedback on their performance as rolemodels and teachers of evidence-basedmedicine. The influence of evidence-based medicine on clinical practice andmedical education is increasing.CLINICAL SCENARIOA junior medical resident working in

a teaching hospital admits a 43-year-oldpreviously well man who experienced awitnessed grand mal seizure. He hadnever had a seizure before and had nothad any recent head trauma. He drankalcohol once or twice aweek and had nothad alcohol on the day of the seizure.Findings on physical examination arenormal. The patient is given a loading

A complete list of members of the Evidenced-BasedMedicine Working Group appears at the end of this ar-ticle.

Reprint requests to McMaster University Health Sci-ences Centre, Room 3W10,1200 Main St W, Hamilton,Ontario, Canada L8N 3Z5 (Gordon Guyatt, MD).

dose ofphenytoin intravenously and thedrug is continued orally. A computedtomographic head scan is completely nor¬mal, and an electroencephalogram showsonly nonspecific findings. The patient isvery concerned about his risk of seizurerecurrence. How might the residentproceed?The Way of the PastFaced with this situation as a clinical

clerk, the resident was told by her se¬nior resident (who was supported in hisview by the attending physician) thatthe risk of seizure recurrence is high(though he could not put an exact num¬ber on it) and that was the informationthat should be conveyed to the patient.She now follows this path, emphasizingto the patient not to drive, to continuehis medication, and to see his familyphysician in follow-up. The patient leavesin a state of vague trepidation about hisrisk of subsequent seizure.The Way of the FutureThe resident asks herselfwhether she

knows the prognosis of a first seizureand realizes she does not. She proceedsto the library and, using the GratefulMed program,1 conducts a computerizedliterature search. She enters the Med¬ical Subject Headings terms epilepsy,prognosis, and recurrence, and the pro¬gram retrieves 25 relevant articles. Sur¬veying the titles, one2 appears directlyrelevant. She reviews the paper, findsthat it meets criteria she has previouslylearned for a valid investigation ofprog¬nosis,3 and determines that the resultsare applicable to her patient. The searchcosts the resident $2.68, and the entireprocess (including the trip to the libraryand the time to make a photocopy of thearticle) took half an hour.The results of the relevant study show

that the patient risk of recurrence at 1

year is between 43% and 51%, and at 3years the risk is between 51% and 60%.After a seizure-free period of 18 monthshis risk of recurrence would likely beless than 20%. She conveys this infor¬mation to the patient, along with a rec¬ommendation that he take his medica¬tion, see his family doctor regularly, andhave a review ofhis need for medicationifhe remains seizure-free for 18 months.The patient leaves with a clear idea ofhis likely prognosis.A PARADIGM SHIFTThomas Kuhn has described scientific

paradigms as ways of looking at theworld that define both the problems thatcan legitimately be addressed and therange of admissible evidence that maybear on their solution.4 When defects inan existing paradigm accumulate to theextent that the paradigm is no longertenable, the paradigm is challenged andreplaced by a new way of looking at theworld. Medical practice is changing, andthe change, which involves using themedical literature more effectively inguiding medical practice, is profoundenough that it can appropriately be calleda paradigm shift.The foundations of the paradigm shift

lie in developments in clinical researchover the last 30 years. In 1960, the ran¬domized clinical trial was an oddity. It isnow accepted that virtually no drug canenter clinical practice without a demon¬stration of its efficacy in clinical trials.Moreover, the same randomized trialmethod increasingly is being applied tosurgical therapies6 and diagnostic tests.6Meta-analysis is gaining increasing ac¬

ceptance as amethod ofsummarizing theresults of a number of randomized trials,and ultimately may have as profound aneffect on setting treatment policy as haverandomized trials themselves.7 Whileless dramatic, crucial methodological ad-

at Vrije Universiteit on August 22, 2009 www.jama.comDownloaded from

vanees have also been made in other ar¬eas, such as the assessment ofdiagnostictests8·9 and prognosis.2A new philosophy ofmedical practice

and teaching has followed these meth¬odological advances. This paradigm shiftis manifested in a number of ways. Aprofusion of articles has been publishedinstructing clinicians on how to access,10evaluate,11 and interpret12 the medicalliterature. Proposals to apply the prin¬ciples of clinical epidemiology to day-to-day clinical practice have been putforward.3 A number of major medicaljournals have adopted a more informa¬tive structured abstract format, whichincorporates issues of methods and de¬sign into the portion of an article thereader sees first.13 The American Col¬lege of Physicians has launched a jour¬nal, ACP Journal Club, that summa¬rizes new publications ofhigh relevanceand methodological rigor.14 Textbooksthat provide a rigorous review of avail¬able evidence, including a methods sec¬tion describing both the methodologicalcriteria used to systematically evaluatethe validity of the clinical evidence andthe quantitative techniques used forsummarizing the evidence, have begunto appear.1516 Practice guidelines basedon rigorous methodological review of theavailable evidence are increasingly com¬mon.17 A final manifestation is the grow¬ing demand for courses and seminarsthat instruct physicians on how to makemore effective use of the medical liter¬ature in their day-to-day patient care.3We call the new paradigm "evidence-

based medicine."18 In this article, we de¬scribe how this approach differs fromprior practice and briefly outline howwe are building a residency program inwhich a key goal is to practice, act as arole model, teach, and help residentsbecome highly adept in evidence-basedmedicine. We also describe some of theproblems educators and medical prac¬titioners face in implementing the new

paradigm.The Former ParadigmThe former paradigm was based on

the following assumptions about theknowledge required to guide clinicalpractice.

1. Unsystematic observations fromclinical experience are a valid way ofbuilding and maintaining one's knowl¬edge about patient prognosis, the valueof diagnostic tests, and the efficacy oftreatment.

2. The study and understanding ofbasic mechanisms of disease and patho-physiologic principles are a sufficientguide for clinical practice.

3. A combination of thorough tradi¬tional medical training and common

sense is sufficient to allow one to eval¬uate new tests and treatments.

4. Content expertise and clinical ex¬perience are a sufficient base from whichto generate valid guidelines for clinicalpractice.According to this paradigm clinicians

have a number ofoptions for sorting outclinical problems they face. They canreflect on their own clinical experience,reflect on the underlying biology, go toa textbook, or ask a local expert. Read¬ing the introduction and discussion sec¬tions of a paper could be considered an

appropriate way ofgaining the relevantinformation from a current journal.This paradigm puts a high value on

traditional scientific authority and ad¬herence to standard approaches, and an¬swers are frequently sought from directcontact with local experts or referenceto thewritings ofinternational experts.19The New ParadigmThe assumptions of the new paradigm

are as follows:1. Clinical experience and the devel¬

opment of clinical instincts (particularlywith respect to diagnosis) are a crucialand necessary part of becoming a com¬

petent physician. Many aspects of clin¬ical practice cannot, or will not, ever beadequately tested. Clinical experienceand its lessons are particularly impor¬tant in these situations. At the same

time, systematic attempts to record ob¬servations in a reproducible and unbi¬ased fashion markedly increase the con¬fidence one can have in knowledge aboutpatient prognosis, the value of diagnos¬tic tests, and the efficacy of treatment.In the absence of systematic observa¬tion one must be cautious in the inter¬pretation of information derived fromclinical experience and intuition, for itmay at times be misleading.

2. The study and understanding ofbasic mechanisms of disease are neces¬

sary but insufficient guides for clinicalpractice. The rationales for diagnosis andtreatment, which follow from basicpathophysiologic principles, may in factbe incorrect, leading to inaccurate pre¬dictions about the performance of diag¬nostic tests and the efficacy of treat¬ments.

3. Understanding certain rules ofevidence is necessary to correctly in¬terpret literature on causation, progno¬sis, diagnostic tests, and treatmentstrategy.It follows that clinicians should reg¬

ularly consult the original literature (andbe able to critically appraise the meth¬ods and results sections) in solving clin¬ical problems and providing optimal pa¬tient care. It also follows that cliniciansmust be ready to accept and live with

uncertainty and to acknowledge thatmanagement decisions are often madein the face of relative ignorance of theirtrue impact.The new paradigm puts amuch lower

value on authority.20 The underlying be¬lief is that physicians can gain the skillsto make independent assessments ofev¬idence and thus evaluate the credibilityof opinions being offered by experts.The decreased emphasis on authoritydoes not imply a rejection of what onecan learn from colleagues and teachers,whose years of experience have provid¬ed them with insight into methods ofhistory taking, physical examination, anddiagnostic strategies. This knowledgecan never be gained from formal scien¬tific investigation. A final assumption ofthe new paradigm is that physicianswhose practice is based on an under¬standing ofthe underlying evidence willprovide superior patient care.REQUIREMENTS FOR THEPRACTICE OF EVIDENCE-BASEDMEDICINEThe rolemodeling, practice, and teach¬

ing ofevidence-based medicine requiresskills that are not traditionally part ofmedical training. These include precise¬ly defining a patient problem, and whatinformation is required to resolve theproblem; conducting an efficient searchof the literature; selecting the best ofthe relevant studies and applying rulesof evidence to determine their validity3;being able to present to colleagues in asuccinct fashion the content of the ar¬ticle and its strengths and weaknesses;and extracting the clinical message andapplying it to the patient problem. Wewill refer to this process as the criticalappraisal exercise.Evidence-based medicine also involves

applying traditional skills of medicaltraining. A sound understanding ofpathophysiology is necessary to inter¬pret and apply the results of clinical re¬search. For instance, most patients towhom we would like to generalize theresults of randomized trials would, forone reason or another, not have beenenrolled in the most relevant study. Thepatient may be too old, be too sick, haveother underlying illnesses, or be unco¬

operative. Understanding the underly¬ing pathophysiology allows the clinicianto better judge whether the results areapplicable to the patient at hand andalso has a crucial role as a conceptualand memory aid.Another traditional skill required of

the evidence-based physician is a sen¬

sitivity to patients' emotional needs. Un¬derstanding patients' suffering21 and howthat suffering can be ameliorated by thecaring and compassionate physician are

at Vrije Universiteit on August 22, 2009 www.jama.comDownloaded from

Evaluation Form for Clinical Teaching Unit Attending PhysiciansRating

Domain Unsatisfactory Needs Improvement Satisfactory Good ExcellentRole model of practiceof evidence-basedmedicine

Seldom cites evidenceto support clinicaldecisions

Often fails to substantiatedecisions with evidence

Usually substantiatesdecisions withevidence

Substantiates decisions;is aware ofmethodological issues

Always substantiatesdecisions or

acknowledgeslimitations of evidence

Leads practice ofevidence-basedmedicine

Never assigns problemsto be resolvedthrough literature

Produces suboptimal volumeor follow-through of problemresolution through literature

Assigns problems andfollows through withdiscussion, includingmethodology

Discusses literatureretrieval, methodologyof papers, applicationto individual patient

Same as "Good" rating,and makes it excitingand fun

fundamental requirements for medicalpractice. These skills can be acquiredthrough careful observation of patientsand of physician role models. Here too,though, the need for systematic studyand the limitations of the present evi¬dence must be considered. The new par¬adigmwould call forusing the techniquesofbehavioral science to determine whatpatients are really looking for from theirphysicians22 and how physician and pa¬tient behavior affects the outcome ofcare.23 Ultimately, randomized trialsusing different strategies for interact¬ing with patients (such as the random¬ized trial conducted by Greenfield andcolleagues24 that demonstrated the pos¬itive effects of increasing patients' in¬volvement with their care) may beappropriate.Since evidence-based medicine in¬

volves skills ofproblem defining, search¬ing, evaluating, and applying originalmedical literature, it is incumbent on

residency programs to teach these skills.Understanding the barriers to educat¬ing physicians-in-training in evidence-based medicine can lead to more effec¬tive teaching strategies.EVIDENCE-BASED MEDICINE IN AMEDICAL RESIDENCYThe Internal Medicine Residency Pro¬

gram at McMaster University has an

explicit commitment to producing prac¬titioners of evidence-based medicine.While other clinical departments atMcMaster have devoted themselves toteaching evidence-based medicine, thecommitment is strongest in the Depart¬ment of Medicine. We will therefore fo¬cus on the Internal Medicine Residencyin our discussion and briefly outline someof the strategies we are using in imple¬menting the paradigm shift.

1. The residents spend each Wednes¬day afternoon at an academic half-day.At the beginning of each new academicyear, the rules of evidence that relate toarticles concerning therapy, diagnosis,prognosis, and overviews are reviewed.In subsequent sessions, the discussionis built around a clinical case, and twooriginal articles that bear on the prob¬lem are presented. The residents are

responsible for critically appraising the

articles and arriving at bottom lines re¬

garding the strength of evidence andhow it bears on the clinical problem.They learn to present the methods andresults in a succinct fashion, emphasiz¬ing only the key points. A wide-rangingdiscussion, including issues of underly¬ing pathophysiology and related ques¬tions of diagnosis and management, fol¬lows presentation of the articles.The second part of the half-day is de¬

voted to the physical examination. Clin¬ical teachers present optimal techniquesof examination with attention to what isknown about their reproducibility andaccuracy.

2. Facilities for computerized litera¬ture searchingare available on the teach¬ing medical ward in each of the fourteaching hospitals. Costs of searchingare absorbed by the residency program.Residents not familiar with computersearching, or the Grateful Med programwe use, are instructed at the beginningof the rotation. Research in our insti¬tution has shown that MEDLINEsearching from clinical settings is fea¬sible with brief training.26 A subsequentinvestigation demonstrated that inter¬nal medicine house staffwho have com¬

puter access on the ward and feedbackconcerning their searching do an aver¬

age of more than 3.6 searches permonth.26 House staff believe that morethan 90% oftheir searches that are stim¬ulated by a patient problem lead to some

improvement in patient care.253. Assessment of searching and crit¬

ical appraisal skills is being incorporat¬ed into the evaluation of residents.

4. We believe that the new paradigmwill remain an academic mirage withlittle relation to the world ofday-to-dayclinical practice unless physicians-in-training are exposed to role models whopractice evidence-based medicine. As a

result, the residency program hasplaced major emphasis on ensuring thisexposure.First, a focus of recruitment for our

Department of Medicine faculty hasbeen internists with training in clinicalepidemiology. These individuals have theskills and commitment to practice evi¬dence-based medicine. The residencyprogram works to ensure they have clin-

ical teaching roles available to them.Second, a program of more rigorous

evaluation of attending physicians hasbeen instituted. One of the areas eval¬uated is the extent to which attendingphysicians are effective in teaching ev¬idence-based medicine. The relevantitems from the evaluation form are re¬

produced in the Table.Third, because it is new to both teach¬

ers and learners, and because most clin¬ical teachers have observed few rolemodels and have not received formaltraining, teaching evidence-based med¬icine is not easy. To help attending phy¬sicians improve their skills in this area,we have encouraged them to form part¬nerships, which involve attending thepartner's clinical rounds, making obser¬vations, and providing formal feedback.One learns through observation andthrough criticisms ofone's performance.A number of facultymembers have par¬ticipated in this program.To further facilitate attending physi¬

cians' improving their skills, the De¬partment of Medicine held a retreat de¬voted to sharing strategies for effectiveclinical teaching. Part of the workshop,attended by more than 30 faculty mem¬

bers, was devoted to teaching evidence-based medicine. Some of the strategiesthat were adduced are briefly summa¬rized in the next section.

EFFECTIVE TEACHING OFEVIDENCE-BASED MEDICINERole ModelingAttending physicians must be enthu¬

siastic, effective role models for the prac¬tice ofevidence-based medicine (even inhigh-pressure clinical settings, such asintensive care units). Providing a modelgoes a long way toward inculcating at¬titudes that lead learners to developskills in critical appraisal. Acting as arole model involves specifying thestrength of evidence that supports clin¬ical decisions. In one case, the teachercan point to a number of large random¬ized trials, rigorously reviewed and in¬cluded in a meta-analysis, which allowsone to say how many patients one musttreat to prevent a death. In other cases,the best evidence may come from ac-

at Vrije Universiteit on August 22, 2009 www.jama.comDownloaded from

cepted practice or one's clinical experi¬ence and instincts. The clinical teachershould make it clear to learners on whatbasis decisions are beingmade. This canbe done efficiently. For instance:Prospective studies suggest that Mr Jones'risk of a major vascular event in the firstyear after his infarct is 4%; a meta-analysisof randomized trials of aspirin in this situa¬tion suggests a risk reduction of 25%; wewould have to treat 100 such patients to pre¬vent an event21; given the minimal expenseand toxicity of low-dose, enteric-coated as¬pirin, treatingMr Jones is clearlywarranted.Or:

How longto treat a patientwith antibiotics fol¬lowingpneumonia has not been systematicallystudied; so, my recommendation that we giveMrs Smith 3 days of intravenous antibioticsand treat her for a total of 10 days is arbitrary;somewhat shorter or longer courses of treat¬ment would be equally reasonable.In the latter type of situation, dog¬

matic or rigid insistence on following a

particular course of action would not beappropriate.Critical AppraisalIt is crucial that critical appraisal is¬

sues arise from patient problems thatthe learner is currently confronting,demonstrating that critical appraisal isa pragmatic and central aspect, not anacademic or tangential element of op¬timal patient care. The problem select¬ed for critical appraisalmust be one thatthe learners recognize as important, feeluncertain, and do not fully trust expertopinion; in other words, they must feelit is worth the effort to find out what theliterature says on a topic. The likeliestcandidate topics are common problemswhere learners have been exposed todivergent opinions (and thus there isdisagreement and/or uncertainty amongthe learners). The clinical teacher shouldkeep these requirements in mind whenconsidering questions to encourage thelearners to address. It can be useful toask all members of the group their opin¬ion about the clinical problem at hand.One can then ensure that the problem isappropriate for a critical appraisal ex¬ercise by asking the group the followingquestions:

1. It seems the group is uncertainabout the optimal approach. Is thatright?

2. Do you feel it is important for us tosort out this question by going to theoriginal literature?Methodological CriteriaCriteria formethodological rigormust

be few and simple. Most published crite¬ria can be overwhelming for the novice.Suggested criteria for studies of diagno¬sis, treatment, and review articles follow:

Diagnosis.—Has the diagnostic testbeen evaluated in a patient sample thatincluded an appropriate spectrum ofmildand severe, treated and untreated dis¬ease, plus individuals with different butcommonly confused disorders?28 Wasthere an independent, blind comparisonwith a "gold standard" of diagnosis?28Treatment.—Was the assignment of

patients to treatments randomized?29Were all patients who entered the studyaccounted for at its conclusion?29Review Articles.—Were explicit

methods used to determine which arti¬cles to include in the review?30As learners becomemore sophisticat¬

ed, additional criteria can be introduced.The criteria should not be presented insuch a way that fosters nihilism (if thestudy is not randomized, it is uselessand provides no valuable information),but as a way of helping arrive at thestrength of inference associated with aclinical decision. Teachers can point outinstances in which criteria can be vio¬lated without reducing the strength ofinference.

METHODS FOR SCALING THEBARRIERS TO THE DISSEMINATIONOF EVIDENCE-BASED MEDICINEMisapprehensions AboutEvidence-Based MedicineIn developing the practice and teach¬

ing of evidence-based medicine at ourinstitution, we have found that the na¬ture of the new paradigm is sometimesmisinterpreted. Recognizing the limita¬tions of intuition, experience, and un¬

derstanding of pathophysiology in per¬mitting strong inferences may be mis¬interpreted as rejecting these routes toknowledge. Specific misinterpretationsofevidence-based medicine and their cor¬rections follow:Misinterpretation 1.—Evidence-

based medicine ignores clinical experi¬ence and clinical intuition.Correction.—On the contrary, it is

important to expose learners to excep¬tional clinicians who have a gift for in¬tuitive diagnosis, a talent for preciseobservation, and excellent judgment inmaking difficult management decisions.Untested signs and symptoms shouldnot be rejected out of hand. They mayprove extremely useful and ultimatelybe proved valid through rigorous test¬ing. Themore the experienced clinicianscan dissect the process they use in di¬agnosis,31 and clearly present it to learn¬ers, the greater the benefit. Similarly,the gain for students will be greatestwhen clues to optimal diagnosis andtreatment are culled from the barrageof clinical information in a systematicand reproducible fashion.

Institutional experience can also pro¬vide important insights. Diagnostic testsmay differ in their accuracy dependingon the skill of the practitioner. A localexpert in, for instance, diagnostic ultra¬sound may produce far better resultsthan the average from the published lit¬erature. The effectiveness and compli¬cations associated with therapeutic in¬terventions, particularly surgical pro¬cedures, may also differ among institu¬tions. When optimal care is taken toboth record observations reproduciblyand avoid bias, clinical and institutionalexperience evolves into the systematicsearch for knowledge that forms the coreof evidence-based medicine.32Misinterpretation 2.—Understand¬

ing ofbasic investigation and pathophys-iology plays no part in evidence-basedmedicine.Correction.—The dearth ofadequate

evidence demands that clinical problemsolving must rely on an understandingof underlying pathophysiology. More¬over, a good understanding of patho¬physiology is necessary for interpretingclinical observations and for appropri¬ate interpretation of evidence (especial¬ly in deciding on its generalizability).Misinterpretation 3.—Evidence-

based medicine ignores standard aspectsof clinical training, such as the physicalexamination.Correction.—Careful history taking

and physical examination provide much,and often the best, evidence for diag¬nosis and direct treatment decisions. Theclinical teacher of evidence-based med¬icine must give considerable attentionto teaching the methods of history tak¬ing and clinical examination, with par¬ticular attention to which items havedemonstrated validity and to strategiesthat enhance observer agreement.Barriers to TeachingEvidence-Based MedicineDifficulties we have encountered in

teaching evidence-based medicine in¬clude the following:

1. Many house staff start with rudi¬mentary critical appraisal skills and thetopic may be threatening for them.

2. People like quick and easy answers.Cookbook medicine has its appeal. Crit¬ical appraisal involves additional timeand effort and may be perceived as in¬efficient and distracting from the realgoal (to provide optimal care for pa¬tients).

3. For many clinical questions, high-quality evidence is lacking. If such ques¬tions predominate in attempts to intro¬duce critical appraisal, a sense of futilitycan result.

4. The concepts of evidence-basedmedicine are met with skepticism by

at Vrije Universiteit on August 22, 2009 www.jama.comDownloaded from

many faculty members who are there¬fore unenthusiastic about modifyingtheir teaching and practice in accordancewith its dictates.These problems can be ameliorated

by use of the strategies described in theprevious section on effective teaching ofevidence-based medicine. Threat can bereduced by making a contract with theresidents, which sets out modest andachievable goals, and further reducedby the attending physician role model¬ing the practice of evidence-based med¬icine. Inefficiency can be reduced byteaching effective searching skills andsimple guidelines for assessing the va¬

lidity of the papers. In addition, one can

emphasize that critical appraisal as a

strategy for solving clinical problems ismost appropriate when the problems arecommon in one's own practice. Futilitycan be reduced by, particularly initially,targeting critical appraisal exercises toareas in which there is likely to be high-quality evidence that will affect clinicaldecisions. Skepticism of faculty mem¬bers can be reduced by the availabilityof "quick and dirty" (as well as more

sophisticated) courses on critical apprais¬al of evidence and by the teaching part¬nerships and teaching workshops de¬scribed earlier.Many problems in the practice and

teaching of evidence-based medicine re¬main. Many physicians, including bothresidents and faculty members, are stillskeptical about the tenets of the new

paradigm. A medical residency is full ofcompeting demands, and the appropri¬ate balance between goals is not alwaysevident. At the same time, we are buoyedby the number of residents and facultywho have enthusiastically adopted thenew approach and found ways to inte¬grate it into their learning and practice.Barriers to PracticingEvidence-Based MedicineEven ifour residency program is suc¬

cessful in producing graduates who en¬ter the world of clinical practice enthu¬siastic to apply what they have learnedabout evidence-basedmedicine, theywillface difficult challenges. Economic con¬straints and counterproductive incen¬tives may compete with the dictates ofevidence as determinants of clinical de¬cisions; the relevant literature may notbe readily accessible; and the time avail¬able may be insufficient to carefully re¬view the evidence (which may be volu¬minous) relevant to a pressing clinicalproblem.

Some solutions to these problems arealready available. Optimal integrationofcomputer technology into clinical prac¬tice facilitates finding and accessing ev¬idence. Reference to literature over-

views meeting scientific principles30,33and collections ofmethodologically soundand highly relevant articles14 can mark¬edly increase efficiency. Other solutionswill emerge over time. Health educa¬tors will continue to find better ways ofrole modeling and teaching evidence-based medicine. Standards in writingreviews and texts are likely to change,with a greater focus on methodologicalrigor.15·16 Evidence-based summaries willtherefore become increasingly available.Practical approaches to making evi¬dence-based summaries easier to applyin clinical practice, many based on com¬

puter technology, will be developed andexpanded. As described earlier, we are

already using computer searching on theward. In the future, the results of di¬agnostic tests may be providedwith theassociated sensitivity, specificity, andlikelihood ratios. Health policymakersmay find that the structure of medicalpracticemust be shifted in basic ways tofacilitate the practice of evidence-basedmedicine. Increasingly, scientific over¬views will be systematically integratedwith information regarding toxicity andside effects, cost, and the consequencesof alternative courses of action to de¬velop clinical policy guidelines.34 Theprospects for these developments areboth bright and exciting.DOES TEACHING AND LEARNINGEVIDENCE-BASED MEDICINEIMPROVE PATIENT OUTCOMES?The proof of the pudding of evidence-

based medicine lies in whether patientscared for in this fashion enjoy betterhealth. This proof is no more achievablefor the new paradigm than it is for theold, for no long-term randomized trialsof traditional and evidence-based med¬ical education are likely to be carriedout. What we do have are a number ofshort-term studies which confirm thatthe skills ofevidence-based medicine canbe taught to medical students35 and med¬ical residents.36 In addition, a study com¬

pared the graduates of a medical schoolthat operates under the new paradigm(McMaster) with the graduates of a tra¬ditional school. A random sample ofMcMaster graduates who had chosencareers in family medicine were more

knowledgeable with respect to currenttherapeutic guidelines in the treatmentof hypertension than were the gradu¬ates of the traditional school.37 Theseresults suggest that the teaching of ev¬idence-based medicine may help grad¬uates stay up-to-date. Further evalua¬tion of the evidence-based medicine ap¬proach is necessary.Our advocating evidence-based med¬

icine in the absence of definitive evi¬dence of its superiority in improving pa-

tient outcomes may appear to be an in¬ternal contradiction. As has been point¬ed out, however, evidence-basedmedicine does not advocate a rejectionof all innovations in the absence of de¬finitive evidence. When definitive evi¬dence is not available, onemust fall backon weaker evidence (such as the com¬

parison of graduates of two medicalschools that use different approachescited above) and on biologic rationale.The rationale in this case is that physi¬cians who are up-to-date as a function oftheir ability to read the current litera¬ture critically, and are able to distin¬guish strong from weaker evidence, arelikely to be more judicious in the ther¬apy they recommend. Physicians whounderstand the properties ofdiagnostictests and are able to use a quantitativeapproach to those tests are likely to makemore accurate diagnoses. While this ra¬tionale appears compelling to us, com¬pelling rationale has often proved mis¬leading. Until more definitive evidenceis adduced, adoption of evidence-basedmedicine should appropriately be re¬stricted to two groups. One group com¬

prises those who find the rationale com¬

pelling, and thus believe that use of theevidence-based medicine approach islikely to improve clinical care. A secondgroup comprises those who, while skep¬tical of improvements in patient out¬come, believe it is very unlikely thatdeterioration in care results from theevidence-based approach and who findthat the practice of medicine in the new

paradigm is more exciting and fun.

CONCLUSIONBased on an awareness of the limita¬

tions of traditional determinants of clin¬ical decisions, a new paradigm for med¬ical practice has arisen. Evidence-basedmedicine deals directly with the uncer¬tainties of clinical medicine and has thepotential for transforming the educa¬tion and practice of the next generationofphysicians. These physicianswill con¬tinue to face an exploding volume ofliterature, rapid introduction of new

technologies, deepening concern aboutburgeoning medical costs, and increas¬ing attention to the quality and outcomesof medical care. The likelihood that ev¬idence-based medicine can help amelio¬rate these problems should encourageits dissemination.Evidence-based medicine will require

new skills for the physician, skills thatresidency programs should be equippedto teach. While strategies for inculcat¬ing the principles ofevidence-based med¬icine remain to be refined, initial expe¬rience has revealed a number of effec¬tive approaches. Incorporating thesepractices into postgraduate medical ed-

at Vrije Universiteit on August 22, 2009 www.jama.comDownloaded from

ucation and continuing to work on theirfurther development will result in more

rapid dissemination and integration ofthe new paradigm into medical practice.The Evidence-Based Medicine Working Group

comprised the following: Gordon Guyatt (chair),MD, MSc, John Cairns, MD, David Churchill, MD,MSc, Deborah Cook, MD, MSc, Brian Haynes, MD,MSc, PhD, Jack Hirsh, MD, Jan Irvine, MD, MSc,Mark Levine, MD, MSc, Mitchell Levine, MD, MSc,Jim Nishikawa, MD, and David Sackett, MD, MSc,Departments of Medicine and Clinical Epidemiol¬ogy and Biostatistics, McMaster University,Hamilton, Ontario; Patrick Brill-Edwards, MD,Hertzel Gerstein, MD, MSc, Jim Gibson, MD, Ro¬man Jaeschke, MD, MSc, Anthony Kerigan, MD,MSc, Alan Neville, MD, and Akbar Panju, MD, De-

References

partment ofMedicine, McMaster University; AllanDetsky, MD, PhD, Department of Clinical Epide¬miology and Biostatistics, McMaster Universityand Departments of Health Administration andMedicine, University of Toronto (Ontario); MurrayEnkin, MD, Departments of Clinical Epidemiologyand Biostatistics and Obstetrics and Gynaecology,McMaster University; Pamela Frid, MD, Depart¬ment of Pediatrics, Queen's University, Kingston,Ontario; MarthaGerrity, MD, Department of Med¬icine, University of North Carolina, Chapel Hill;Andreas Laupacis, MD, MSc, Department of Clin¬ical Epidemiology and Biostatistics, McMasterUniversity and Department of Medicine, Univer¬sity of Ottawa (Ontario); Valerie Lawrence, MD,Department of Medicine, University of TexasHealth Science Center at San Antonio and Audie L.MurphyMemorial Veterans Hospital, San Antonio,Tex; Joel Menard, MD, Centre de Medicine Trezen-

tize Cardio-Vasculaires, Paris, France; VirginiaMoyer, MD, Department of Pediatrics, Universityof Texas, Houston; Cynthia Mulrow, MD, Depart¬ment of Medicine, University of Texas, San Anto¬nio; Paul Links, MD, MSc, Department of Psychi¬atry, McMaster University; Andrew Oxman, MD,MSc, Departments of Clinical Epidemiology andBiostatistics and Family Medicine, McMaster Uni¬versity; Jack Sinclair, MD, Departments of ClinicalEpidemiology and Biostatistics and Pediatrics,McMaster University; and Peter Tugwell, MD,MSc, Department of Medicine, University of Ot¬tawa (Ontario).

Drs Cook and Guyatt are Career Scientists ofthe OntarioMinistry of Health. Dr Haynes is a Na¬tional Health Scientist, National Health Researchand Development Program, Canada. Drs Jaeschkeand Cook are Scholars of the St Joseph's HospitalFoundation, Hamilton, Ontario.

1. Lindberg DA. Information systems to supportmedical practice and scientific discovery. MethodsInform Med. 1989;28:202-206.2. Hart YM, Sander JW, Johnson AL, Shorvon SD.National general practice study of epilepsy: recur-rence after a first seizure. Lancet. 1990;336:1271\x=req-\1274.3. Sackett DL, Haynes RB, Guyatt GH, Tugwell P.Clinical Epidemiology: A Basic Science for Clini-cial Medicine. 2nd ed. Boston, Mass: Little Brown& Co Inc; 1991:173-186.4. Kuhn TS. The Structure of Scientific Revolu-tions. Chicago, Ill: University of Chicago Press;1970.5. European Carotid Surgery Trialists' Collabora-tive Group. MRC European Carotid Surgery Trial:interim results for symptomatic patients with se-vere (70-99%) orwith mild (0-29%) carotid stenosis.Lancet. 1991;337:1235-1243.6. Larsen ML, Horder M, Mogensen EF. Effect oflong-term monitoring of glycosylated hemoglobinlevels in insulin-dependent diabetesmellitus. NEnglJ Med. 1990;323:1021-1025.7. L'Abbe KA, Detsky AS, O'Rourke K. Meta-anal-ysis in clinical research. Ann Intern Med. 1987;107:224-233.8. RansohoffDF, Feinstein AR. Problems of spec-trum and bias in evaluating the efficacy of diag-nostic tests. N Engl J Med. 1978;299:926-930.9. NierenbergAA, Feinstein AR. How to evaluatea diagnostic market test: lessons from the rise andfall ofdexamethasone suppression test. JAMA. 1988;259:1699-1702.10. Haynes RB, McKibbon KA, Fitzgerald D,Guyatt GH, Walker CJ, Sackett DL. How to keepup with the medical literature, V: access by per-sonal computer to the medical literature. Ann In-tern Med. 1986;105:810-816.11. Bulpitt CJ. Confidence intervals. Lancet. 1987;1:494-497.12. Godfrey K. Simple linear regression in medicalresearch. N Engl J Med. 1985;313:1629-1636.

13. Haynes RB,Mulrow CD, Huth EJ, Altman DG,Gardner MJ. More informative abstracts revisited.Ann Intern Med. 1990;113:69-76.14. Haynes RB. The origins and aspirations ofACPJournal Club. Ann Intern Med. 1991;114(ACP JClub. suppl 1):A18.15. Chalmers I, Enkin M, Keirse MJNC, eds. Ef-fective Care in Pregnancy and Childbirth. NewYork, NY: Oxford University Press Inc; 1989.16. Sinclair JC, Braken MB, eds. Effective Care ofthe Newborn Infant. New York, NY: Oxford Uni-versity Press Inc; 1992.17. AudetAM, Greenfield S, Field M. Medical prac-tice guidelines: current activities and future direc-tions. Ann Intern Med. 1990;113:709-714.18. Guyatt GH. Evidence-based medicine. Ann In-tern Med. 1991;114(ACP J Club. suppl 2):A-16.19. Light DW. Uncertainty and control in profes-sional training. J Health Soc Behav. 1979;20:310\x=req-\322.20. Chalmers I. Scientific inquiry and authoritar-ianism in perinatal care and education. Birth. 1983;10:151-164.21. Cassell EJ. The nature of suffering and thegoals ofmedicine. NEngl J Med. 1982;306:639-645.22. Ende J, Kazis L, Ash A, Moskowitz MA. Mea-suring patients' desire for autonomy: decision-mak-ing and information-seeking preferences amongmedical patients. J Gen Intern Med. 1989;4:23-30.23. Carter WB, Inui TS, Kukull WA, Haigh VH.Outcome-based doctor-patient interaction analysis.Med Care. 1982;20:550-556.24. Greenfield S, Kaplan S,Ware JE Jr. Expandingpatient involvement in care: effects on patient out-comes. Ann Intern Med. 1985;102:520-528.25. Haynes RB, McKibbon A, Walker CJ, Ryan N,Fitzgerald D, Ramsden ME. Online access toMED-LINE in clinical settings: a study of use and use-fulness. Ann Intern Med. 1990;112:78-84.26. McKibbon KA, Haynes RB, Johnston ME,Walker CJ. A study to enhance clinical end-userMEDLINE search skills: design and baseline find-

ings. In: Clayton PD, ed. Proceedings of the FifthAnnual Symposium on Computer Applications inMedical Care, November 1991; Washington, DC.New York, NY: McGraw-Hill International BookCo; 1992:73-77.27. Laupacis A, Sackett DL, Roberts RS. An as-

sessment of clinically useful measures of the con-

sequences of treatment. N Engl J Med. 1988;318:1728-1733.28. Department of Clinical Epidemiology and Bio-statistics, McMaster University. How to read clin-ical journals, II: to learn about a diagnostic test.Can Med Assoc J. 1981;124:703-710.29. Department of Clinical Epidemiology and Bio-statistics, McMaster University. How to read clin-ical journals, V: to distinguish useful from uselessor even harmful therapy. Can Med Assoc J. 1981;124:1156-1162.30. Oxman AD, Guyatt GH. Guidelines for readingliterature reviews. Can Med Assoc J. 1988;138:697\x=req-\703.31. Campbell EJM. The diagnosing mind. Lancet.1987;1:849-851.32. Feinstein AR. What kind of basic science forclinical medicine? N Engl J Med. 1970;283:847-853.33. Mulrow CD. The medical review article: stateof the science. Ann Intern Med. 1987;106:485-488.34. Eddy DM. Guidelines for policy statements: theexplicit approach. JAMA. 1990;263:2239-2243.35. Bennett KJ, Sackett DL, Haynes RB, et al. Acontrolled trial of teaching critical appraisal of theclinical literature tomedical students. JAMA. 1987;257:2451-2454.36. Kitchens JM, Pfeifer MP. Teaching residents toread the medical literature: a controlled trial of acurriculum in critical appraisal/clinical epidemiol-ogy. J Gen Intern Med. 1989;4:384-387.37. Shin J, Haynes RB. Does a problem-based, self-directed undergraduate medical curriculum pro-mote continuing clinical competence? Clin Res. 1991;39:143A.

at Vrije Universiteit on August 22, 2009 www.jama.comDownloaded from