Successes and Failures in the Implementation ofEvidence-Based Guidelines for Clinical Practice

RICHARD GROL, PHD

BACKGROUND. The development and imple-mentation of (evidence-based) clinical practiceguidelines is one of the promising and effec-tive tools for improving the quality of care.However, many guidelines are not used afterdissemination. Implementation activities fre-quently produce only moderate improvement.It is important to study specific guideline pro-grams in detail to learn from their successesand failures.

OBJECTIVES. Experiences with more than 10years of development and dissemination ofclinical guidelines for family medicine in theNetherlands are presented in this paper.

RESULTS. More than 70 evidence-basedguidelines have been set in a rigorous proce-dure and have been spread via a variety ofstrategies. Knowledge and acceptance of theguidelines in the target group is high. In par-ticular, a multifaceted approach with written(scientific journal, support materials) and per-sonal approaches (local consensus discussions,contact with colleagues, outreach visits by

peers) seems to be effective in the dissemina-tion. The guideline recommendations are fol-lowed in on average 67% of the decisions, butthere is a large variation between differentphysicians and between different guidelines.Specific strategies designed to handle possibleobstacles to implementation are needed to im-prove adherence. Specific implementationprojects showed the importance of a “diagnos-tic analysis” of the target group and targetsetting before the start of the implementation.

CONCLUSIONS. A program to implement aguideline should be well designed, well pre-pared, and preferably pilot tested before use.Such a program should be built into the nor-mal channels and structures for improvingcare. More research into the details of imple-mentation is needed to better understand thecritical determinants of change in practice.

Key words: Clinical practice guidelines;family practice; implementation; change.

(Med Care 2001;39:II-46–II-54)

Despite many years of efforts to improve thequality of patient care, research in most countriescontinues to show that many patients receiveinappropriate or even detrimental care. Americanfigures point at the “overuse,”“underuse,”or “mis-use” of care.1,2 The situation in Europe cannot beexpected to be much better. Many different ap-proaches to the improvement of clinical practicehave been proposed and used in the past decades,such as professional education and development,audit and feedback, evidence-based guidelines,

total quality management, economic incentives,and organizational changes. Although the numberof well-designed studies examining attempts tomodify clinical practices is rising, just which ap-proaches are most effective in which settingsremains unclear.3–5 The development and imple-mentation of (evidence-based) clinical-practiceguidelines appears to be one of the most promis-ing and effective tools for improving the quality ofcare: that is, reviews of the scientific literaturecombined with insights from clinical practice to

From the Center for Quality of Care Research, Universitiesof Nijmegen and Maastricht, Nijmegen, The Netherlands.

Address correspondence to: Richard Grol, PhD, Di-rector, Center for Quality of Care Research, Universities

of Nijmegen and Maastricht, Nijmegen, The Nether-lands. E-mail: r.grol@hsv.kun.nl

MEDICAL CAREVolume 39, Number 8, Supplement 2, pp II-46–II-54©2001 Lippincott Williams & Wilkins, Inc.

II-46

generate concrete recommendations can helphealth care providers and patients decide on ap-propriate care, promote education and improve-ment of care processes, reduce unwanted variationin the delivery of health care, and perhaps helpcontain costs.6 The development of such recom-mendations or guidelines is nevertheless laborintensive and expensive. Another problem is thatsuch guidelines do not implement themselves;they are often not used after dissemination, andimplementation activities frequently produce onlymoderate improvement.3–6 These findings raisethe question of how to gain greater insight intothose factors that appear to be decisive for theeffective implementation of clinical guidelines.The controlled trials currently being used withinthe Cochrane Collaboration on Effective Practiceand Organization of Care7 do not provide all theanswers. It is therefore important to study specificprograms for guideline development and imple-mentation to learn from their actual successes andfailures.8

This paper reports experiences with the devel-opment and implementation of a program fornational evidence-based guidelines for familymedicine in the Netherlands. Approximately 7,000family physicians care for approximately 16 millioninhabitants in the Netherlands. Most of the familyphysicians work alone in office-based practices(approximately 50%) or with one or more partnersin small group practices. Family physicians play acentral gate-keeping role in the Dutch health caresystem because they have to approve referrals tospecialists working in outpatient clinics and hos-pitals. After a period of regional guideline settingat the beginning of the 1980s, which led to con-flicting guidelines and confusion among physi-cians, the Dutch College of Family Physicians tookthe initiative to develop national guidelines. Thefirst set of guidelines (diabetes mellitus type 2) waspublished in 1987, and since that time more than70 guidelines have been developed and dissemi-nated. In the present paper, the lessons to belearned from more than 10 years of the dissemi-nation of guidelines will be considered.

Guideline Development andDissemination Program

The aim of the guideline development anddissemination program is to provide family physi-cians with a point of reference for their daily work

and a basis for education and local protocol set-ting. A good balance between the use of scientificevidence and the establishment of guidelines thatare clearly feasible and acceptable for normalpractice is considered crucial. A systematic andrigorous procedure with the following steps wasgradually developed over time.

First, a relevant topic is selected by an indepen-dent advisory board comprising 11 experiencedfamily physicians. The advisory board further de-fines the topic, outlines the objectives of eachguideline, and draws up a detailed working plan.

Second, a working party is composed of 4 to 8family physicians with a mixture of scientific andpractical experience. A survey of the members of52 working parties (n 5 243, response rate 96%)showed 50% of the participants to have an aca-demic affiliation and 50% to be ordinary familydoctors; 39% base their expertise on research onthe topic and 12% have published on the topic.9,10

Participation is voluntary; a paid staff memberfrom the Dutch College of Family Physicians pro-vides support for the working party (literaturesearches, writing of draft documents, organizationof meetings). The working parties meet 10 to 15times during a period of 1 to 1.5 years to establishthe guidelines. Before the first meeting, they re-ceive a synopsis of the relevant literature on thetopic; at the second meeting, a short course oncritical reading and evidence-based literatureanalysis is provided. The different tasks are thendivided among the individual group members,who scrutinize the relevant literature and draw uptentative recommendations for practice. In light ofthe fact that the scientific evidence is often simplylacking, not applicable, or conflicting, extensiveconsensus discussions often prove necessary.

Next, the feasibility and acceptability of a spe-cific set of guidelines for normal practice is evalu-ated with the circulation of a written survey to 50randomly selected family physicians. A copy of theproposed guidelines is also sent to external re-viewers, who are usually specialists on the topic.The comments of the physicians and experts arethen used to improve the guidelines.

Then an independent scientific board providesofficial approval of a set of guidelines. To obtainsuch approval, the working party must defend theguidelines to a group of experienced physiciansand researchers. After approval, the final version ofthe guidelines is written and published in the formof a paper in a scientific journal for family physi-cians. This paper includes the relevant scientific

Vol. 39, No. 8, Supplement 2 SUCCESSES AND FAILURES

II-47

background information. A summary of the guide-lines is also printed on a plastic card accompany-ing the journal. Dissemination is further promotedby developing special educational programs andpackages for each set of guidelines. This informa-tion is sent to approximately 100 regional and localcoordinators for continuing medical education andquality improvement. Support materials in theform of summaries of the guidelines for reception-ists, leaflets and letters for patients, and consensusagreements with specialist societies are also devel-oped and disseminated among the family physi-cians. The guidelines are updated every 3 to 5years, depending on the availability of new scien-tific evidence.

This stepwise approach proved to be feasibleand is more or less standardized now. Variousstudies have been undertaken to evaluate theimpact of this comprehensive program. In thefollowing sections, some critical results will bepresented along with some lessons that have beenlearned about effective implementation.

Knowledge and Acceptance of theGuidelines

A survey among a random sample of 10% of allfamily physicians (n 5 453, response rate 70%)approximately 2 years after publication of the firstset of guidelines showed most doctors to be wellinformed about the guidelines (only 7% did notknow about them) and wide acceptance of theguidelines.11 A subsequent survey among a sam-ple of 1,531 physicians (n 5 1,007, response rate of67%) 3 years later showed increased acceptance:approximately 90% of the physicians surveyedconsidered the national guidelines very useful forlocal education and for translation to local proto-cols; only 13% of the physicians thought theguidelines could be used in contracts with healthinsurers. The most frequently reported sources ofinformation on the guidelines proved to be thescientific journal (85% of the respondents), discus-sion of the guideline within the local group offamily physicians (53%), contacts with other col-leagues (43%), and course attendance (33%). Phy-sicians reporting the scientific journal as a sourceof information on the guidelines, members of theDutch College of Family Physicians, younger doc-tors, and physicians actively involved in the teach-ing of family medicine proved to be better in-formed about the guidelines than other

physicians.12 We concluded from these results thata program of guideline development “run andowned” by the profession itself and disseminatedvia a variety of channels can increase knowledgeand acceptance of the guidelines among the targetgroup. Differentiation of the target group is nec-essary for effective dissemination: for some doc-tors, quick access to evidence and guidelines isrequired and/or preferred; for others, dissemina-tion via local networks or a more personal ap-proach is required.13

In a controlled before-and-after study, the ef-fects of two strategies for speeding the dissemina-tion of the national guidelines were compared:mailing of an information package to key peopleversus mailing of an information package to keypeople with additional outreach visits by twofamily physicians specially trained to provide ex-planation and instruction on the use of the mate-rials. The information package contained an over-view of 10 sets of national guidelines (eg,guidelines for diabetes, urinary tract infection,hypertension, acute otitis media) and additionalsupport materials, which included specially de-signed audit instruments pertaining to the keyrecommendations contained in the guidelines foruse in a performance review; a computerizedversion of the audit instruments, providing imme-diate feedback after entry of the performance data;and a description of some procedures for localeducational meetings addressing the guidelines.Family physicians in the Netherlands were in-formed about the information package via publi-cation in family medicine media; interested doc-tors could order the materials at no cost. Inaddition, the following two strategies were under-taken to promote use of the guidelines and sup-port materials: the information package wasmailed to key people, namely all organizers oflocal continuing medical education and represen-tatives of all local groups of family physicians (adistrict with 527 physicians), and it was mailed tokey people with additional outreach visits by twofamily physicians specially trained to provide ex-planation and instruction on the use of the mate-rials (a comparable district in terms of size, num-ber of physicians, degree of urbanization, andorganization of quality improvement, with 504doctors). The outreach visitors approached the keypeople initially by telephone to motivate them touse the information package and explore the needfor a visit and the provision of support for the localgroup. In such a manner, this dissemination strat-

GROL MEDICAL CARE

II-48

egy was tailored to the needs of the key peopleand groups.

The actual use of the information package wasassessed using a written survey before and after anintervention period of 1.5 years. All of the physi-cians in the two study districts and a randomsample of 500 doctors from the remainder of thecountry were asked to complete the survey. Onlythose physicians completing the survey on bothoccasions were included in the analyses. No sig-nificant differences were found between the re-spondents in the three groups with regard toresponse rate and such personal characteristics asage, type of practice, postgraduate training, andCollege membership. The overall response ratebefore intervention was 67% (1,007 doctors); theoverall response rate after intervention was 76%(762 of the initial 1,007 physicians). A randomsample of 80 nonresponders was interviewed bytelephone. Compared with the nonresponders, theresponders tended to be younger but did not differotherwise. Comparison of the two study districtswith the remainder of the country showed theoutreach visits in particular to contribute to greaterknowledge and use of the national guidelines andsupport materials (Table 1). No other factors (eg,physician characteristics) were found to influencesignificantly knowledge or use of the materials.

Use of the Guidelines

To study the actual use of and adherence to thenational guidelines, several key recommendationswere selected from the guidelines to serve asindicators and review criteria. Different studieswere performed during the past decade, with themost recent study involving a sample of 200 familyphysicians who recorded their performance toassess adherence to 30 national guidelines. The

selection of the key recommendations and trans-lation of these into indicators of adherence wasperformed with the help of a panel of five expertfamily physicians who were also members of thestaff for guideline development. Because theguidelines are intended to steer clinical decisionmaking by defining how to act under specific caseconditions, indicators were constructed in theform of a number of “if-then” algorithms relatingclinical actions to specific conditions. In such amanner, differences in case mix could also beaccounted for. For 30 guidelines, a total of 342specific adherence indicators were constructed.Recording forms were next developed for datacollection. Specific inclusion criteria were formu-lated to guide physicians in the selection of thepatients for recording performance. The recordingforms did not provide any clues to appropriateperformance with regard to concrete cases. Suchself-recording was shown to be reliable in a sep-arate study in which the agreement of self-reportdata with data derived from medical records andobservation in the consulting room for sevenfamily physicians proved to be good (k 0.80–0.90).14 Computer software was developed to re-late actual performance (decisions, actions) toclinical conditions and thereby assess adherenceto the guidelines in concrete decisions. A repre-sentative sample of 740 physicians was ap-proached; 200 (27%) agreed to participate in thestudy. They differed significantly from the nonre-spondents with respect to age (younger), gender(more female), and type of practice (fewer solopractitioners), but not with respect to practicelocation, membership in the College, and being atrainer. The family physicians recorded their per-formance for 1 or 2 contacts per guideline topic,resulting in a total of 36,242 decisions for 7,614contacts allowing key recommendations to berelated to actual performance: for 67% of the

TABLE 1. Effect of Different Strategies to Disseminate National Guidelines and Support Materials

Mailing to Key Personsand Outreach Visits

(n 5 269)

Mailing to KeyPersons Only

(n 5 244)

Rest ofCountry

(n 5 249)

Being informed on materials 66%* 25% 20%

Having materials in possession 49%* 14%* 6%

Having read the materials 35%* 9% 6%

Having used the materials 25%* 4% 2%

Difference with rest of country, *P #0.01.

Vol. 39, No. 8, Supplement 2 SUCCESSES AND FAILURES

II-49

decisions, actual performance was found to followthe relevant recommendations.14 However, cleardifferences in the adherence scores were alsoobserved across the guidelines. For instance, 6guidelines had an average adherence score lowerthan 50% and 5 had an average adherence scorehigher than 80% (Table 2). We also found cleardifferences in the adherence scores across doctors:27% scored lower than 60% on average and 5%scored higher than 80% on average. A positivebias in the adherence scores cannot be excluded,because the physicians were only partly represen-tative for the national population and they mayhave selected cases in which a high score wasmore easily achieved. However, the results confirmfindings in earlier studies on guideline adherencein the Netherlands.

Different lessons can be derived from the pre-ceding evaluations. First, valid indicators and cri-teria for guideline adherence must be developed toassess normal decision making in clinical practice.The selection of such indicators and the collectionof reliable performance data are a complex andlabor-intensive undertaking.15–17 Approximately35% of the decisions made by physicians were notin line with the key recommendations. Given thevariation in the adherence scores across the differ-ent guidelines, a failure (or predisposition) toadhere to certain recommendations may depend,at least in part, on the characteristics of theguideline recommendations themselves. In one ofour studies analyzing 47 recommendations ingreater detail, it was found that, in line with certainimplementation theories,13 recommendations thatdefine the desired performance very concretely,recommendations that are compatible with exist-ing values, and recommendations that do not havemajor consequences for the organization of healthcare were better followed than others.18 Attentionto the quality of a guideline in terms of beingformulated as a credible and feasible message fordissemination to a particular target group is thusan important prerequisite for successful guidelinedevelopment and implementation.

Another lesson is that despite comprehensiveguideline dissemination, many doctors still do notfollow the key recommendations put forth by theirown scientific organization. Better-tailored pro-grams and strategies aimed at specific perfor-mance changes are probably needed. In yet an-other series of studies, different implementationstrategies are thus being further tested and evalu-ated: for example, implementation with perfor-

mance review and feedback,19 interactive educa-tion in small groups of family physicians,20

outreach visits by trained nurses,21 organizationalchanges (eg, delegation of tasks to special nurses),and patient-mediated interventions (eg, the use ofspecial passports for diabetes patients). Some ofthe experiences with the different implementationstrategies are summarized below.

Barriers to the Implementation ofGuidelines

One of the studies comparing different imple-mentation strategies was a randomized controlledtrial with two groups of family physicians and theirpatients. The aim of the study was to assess theeffectiveness of an intensive interactive programinvolving small-group education and peer review(4 sessions of 2 h) for the implementation of thenational guidelines regarding asthma and chronicobstructive pulmonary disease.20 The interventionprogram fits the model of quality improvementmost commonly used in family practice in theNetherlands and Europe quite well: continuingeducation and performance review in local groupsof physicians.22,23 The data from 433 patient ques-tionnaires and 934 recordings of consultations,however, showed no significant differences be-tween the control and intervention groups withregard to the provision of care (adherence toguidelines) and patient outcomes (symptoms,quality of life, smoking, exacerbation).20

A similar lack of an intervention effect wasfound in a study evaluating the implementation of

TABLE 2. Adherence Scores to GuidelineRecommendation in Decision Making

Average overall clinical guidelines 67%Guidelines with Highest Adherence Scores

Micturation problems in older men 100%Diagnosis of heart failure 92.3%Cholesterol management 84.4%Food allergies in neonates 82.1%

Guidelines with Lowest Adherence ScoresOtitis externa 34.4%Fluor vaginalis 36.1%Abdominal complaints 48.7%Diagnosis of asthma and chronic obstructive

pulmonary disease in adults48.8%

Adherence scores in 36,242 decisions; n 5 200physicians.

GROL MEDICAL CARE

II-50

a national guideline for cholesterol managementat the beginning of the 1990s. The specific recom-mendations for this guideline were concentratedon the appropriate testing of cholesterol levels inpatients with a “positive risk profile.”Twenty prac-tices with 32 physicians were allocated to eitherthe intervention or the control group using strat-ified randomization (with or without a computersystem, solo or group practice, and practice size asstrata). To facilitate implementation of the guide-line, the intervention group received a compre-hensive multifaceted program, which includedsmall-group education, regular feedback on gapsin performance, various support tools (eg, decisiontrees), and outreach visits by the researcher tostimulate use of the guideline. Nevertheless, in-spection of 3,950 patient records (ie, a randomsample of 10% of all patients 18 years of age orolder) showed no changes whatsoever in appro-priate cholesterol testing after an interventionperiod of 1 year.24 This finding of no changeprompted more detailed analysis of the obstaclesto implementation for this particular guideline.Interviews with the family physicians revealed avariety of problems25: doubts about the value andscientific grounds for the key recommendations;resistance to a proactive preventive approach andefforts to motivate patients to change their life-styles; reports that the algorithm for diagnosis andtreatment determination was overly complex, dif-ficult to understand, and not easy to use; extraworkload (eg, extra testing, diet advice); and thedemand for unnecessary tests by many patients.These results show the need to understand thefactors playing a role in the implementation of aguideline to develop appropriate interventionsand support strategies. The results are also in linewith an increasing number of authors who em-phasize the importance of a “diagnostic analysis”of the target group and target setting beforeinitiating implementation activities.5 In most ofour implementation projects, in fact, we nowperform such a preliminary analysis.

One example of the importance of a preliminaryanalysis of the target group and setting is a projecton the implementation of the national Low BackPain guideline for family practice. A prospectivestudy on the performance of 57 family physiciansin 1,640 back pain contacts showed that failure toadhere to this guideline concerned in particular alack of patient education on staying active, pre-scribing nonsteroidal anti-inflammatories, and re-ferral to a physiotherapist in the acute phase.

Reasons for lack of adherence provided by thephysicians were perceived preferences of patientsand following fixed routines.26 To explore thefactors determining lack of adherence to theguideline in more detail, in-depth interviews wereundertaken with patients consulting their familyphysician for low back pain and the physiciansthey consulted. Forty physicians were invited toparticipate in this study (a heterogeneous group);31 (77%) agreed to participate and 20 of themrecruited an eligible patient for an interview dur-ing the study time. These interviews were audio-taped, fully transcribed, and analyzed qualitativelyby two researchers using a gradually developedclassification scheme. Most of the patients re-ported consultation of the physician for diagnosisand/or simple advice.27 Among the main reasonsfor lack of adherence to the recommendationsregarding referral to physical therapists and theprescription of nonsteroidal anti-inflammatorieswas a tendency on the part of the family physi-cians to “give in” to the medically inappropriatedemands of patients to avoid conflict. The conclu-sion drawn from these interviews was that theimplementation may be further promoted bytraining doctors to communicate better with theirpatients and educating patients on low back pain.

Although the studies of the cholesterol and lowback pain guidelines were performed with selected(motivated) groups of physicians, the results showthat a variety of factors determine successful im-plementation in many cases. Different groups mayalso experience different problems with the use ofthe guidelines, which means that well-designedprograms with different strategies addressing dif-ferent factors at different levels may be requiredfor successful and effective implementation. Anexample of such a program is presented below.

National Program for theImplementation of Preventive

Guidelines

One of the current implementation programs isconcerned with guidelines for preventive care infamily practice, more specifically the flu vaccina-tion for people older than 65 years and special riskgroups, cervical cancer screening, and the detec-tion and reduction of cardiovascular risk. Fosteringprevention in family medical practice in the Neth-erlands has been viewed as a particularly difficultundertaking for many years. Various surveys28

Vol. 39, No. 8, Supplement 2 SUCCESSES AND FAILURES

II-51

have shown a reluctance among family physicianstoward more proactive, population-based ap-proaches to preventive care and inadequate orga-nization of the practices. A national program withdifferent strategies and arrangements linked tothese problems was developed and initiated in1995. At the national level, evidence-based guide-lines were determined and disseminated via thescientific journal; specific educational packagesand computer software to support the identifica-tion and monitoring of patients at risk were de-signed. In addition, models to improve the orga-nization of prevention were developed. Financialarrangements to reimburse doctors for the extrawork involved in the prevention efforts were ne-gotiated with the Dutch National Health Service.At the district level, educational sessions and localsmall-group meetings were organized to intro-duce the guidelines, the associated computer soft-ware, and the models for improving the organiza-tion of the prevention. Arrangements with otherparties with a vested interest in prevention (eg,local health authorities) were also made. Trainedoutreach visitors provided support for the organi-zation of the prevention and use of the computersoftware at those family practices expressing aninterest. Continuous evaluation of the impact ofthe program was undertaken in surveys of ran-domly selected practices and extraction of datafrom electronic medical records of a smaller sam-ple of practices.

The program started with the flu vaccinationguideline in 1995, the cervical cancer screeningguideline was introduced in 1996, and the guide-line for cardiovascular risk was introduced in 1998.The flu vaccination rate has increased considerablyin recent years: from 10.5% in 1994 to 16.5% in1997.29 Two sources of data were used: the na-tional health surveys from the Central Office ofStatistics and the data from a representative groupof 58 monitoring practices. For people older than65 years, the rate of vaccination was more than80% in 1999. Guidelines for cervical cancer screen-ing aim at giving family practice complete respon-sibility for this screening, because it has beenproven to increase the attendance rate of eligiblewomen with 10% to 15%.30 National surveysamong a random sample of 1,586 practices (re-sponse rate 62%; no differences between nonre-sponders and national population) were per-formed to study changes in screening routinesbefore and after the start of the program. Thepercentage of family practices completely respon-

sible for the screening (contacting the women,reminders, taking the smears, and follow-up) in-creased from 7% in 1995 to 30% in 1997.30 Thesurvey data from 1997 showed that 30% of thepractices participated in local or regional educa-tion with regard to the guidelines for cervicalcancer screening. One third of the surveyed prac-tices had at least one visit by a trained nursevisitor, and almost half of the practices made useof the computer software to identify women atrisk. The education of the doctors had a limitedeffect on their involvement in cervical screening.However, using the computer software and receiv-ing more than two outreach visits was significantlyrelated to following the guidelines for cervicalcancer screening and active involvement of thefamily practice in this preventive activity (oddsratios 1.5–8.7).31

In this national project, we have learned aboutthe importance of a well-designed and well-prepared implementation plan with a variety ofinterventions based on the evidence and experi-ences accumulated in different pilot projects. Asfound in other studies,32 outreach visits are foundto be very useful along with the provision and useof computer software; in addition, financial reim-bursement for the extra work involved in using theguidelines appears to be an important ingredientfor program success. The value of involving allparties with a vested interest in prevention isanother important insight. Finally, building theguideline implementation program into the nor-mal channels and structures for contact with thephysicians appears particularly important

Conclusions

Different projects on the implementation ofnational guidelines have been conducted in theNetherlands during the past decade with differentresults. To our knowledge, the present program isone of the most comprehensive programs forevidence-based guideline development and im-plementation in the world. Studying the accep-tance and impact of such a program over timeprovides critical insights for the provision of ade-quate care. Based on the evidence accumulated todate and our experiences with the different exper-iments, some lessons from more than 10 years ofguideline development and implementation canbe presented.

GROL MEDICAL CARE

II-52

First, the rigorous development of clinicalguidelines at a national level is both feasible andwell accepted by the target group when it is“owned and operated”by the profession itself. TheDutch program appeared to give family physiciansgreater self-confidence in relation to medical spe-cialists on the one hand and to enhance the statusof family medicine in the health care field on theother hand. The program also stimulated supportfrom the governmental and insurance agencieswithout losing the support of ordinary doctors.

Second, a comprehensive strategy to dissemi-nate the guidelines via various channels, bothwritten and personal, appears to be very impor-tant: scientific journals, local networks of peers,and colleagues trained to explain the guidelinesshould be part of such a strategy. In addition to ageneral strategy to promote the implementation ofthe guidelines, specific strategies designed to han-dle possible obstacles to adherence should also beused. This requires a “diagnostic analysis” of thetarget group and target setting: Who is interestedand/or involved in implementation, and who arethe stakeholders? Which aspects of care should beaddressed, and which recommendations are notfollowed? Which subgroups of the target groupappear to experience particular problems withchanging practice procedures, and what are theseproblems?

Third, a program to implement a guidelineshould be well designed, well prepared, and pref-erably pilot tested before use. All of those with avested interest in the program should also beinvolved in the set-up of the program, and theprogram should be built into the normal channelsand structures for monitoring and improving thequality of care in a specific profession or specificsetting.

As already stated, these lessons are only partlybased on concrete evidence. More research intothe details of the different implementation activi-ties is needed to understand the critical determi-nants of change in clinical practice

References1. Bodenheimer T. The American health care sys-

tem. The movement for improved quality in health care.N Engl J Med 1999;340:488–92.

2. Schuster M, McGlynn E, Brook R. How good isthe quality of health care in the United States? MilbankQ 1998;76:517–63.

3. Grol R. Beliefs and evidence in changing clinicalpractice. Br Med J 1997;315:418–21.

4. Grol R, Grimshaw J. Evidence-based imple-mentation of evidence-based medicine. Jt Commission JQuality Improvement 1999;25:503–13.

5. NHS Centre for Reviews and Dissemination.Getting evidence into practice. Effective Health Care1999;5:1–15.

6. Woolf S, Grol R, Hutchinson A, et al. Potentialbenefits, limitations, and harms of clinical guidelines. BrMed J 1999;318:527–30.

7. Bero L, Grilli R, Grimshaw J, et al. Closing thegap between research and practice: an overview ofsystematic reviews of interventions to promote imple-mentation of research findings by health care profession-als. Br Med J 1998;317:465–8.

8. Kanouse D, Kallich J, Kahan J. Disseminationof effectiveness and outcomes research. Health Policy1995;34:167–92.

9. Grol R, Thomas S, Roberts R. Developmentand implementation of guidelines for family practice:lessons from the Netherlands. J Fam Pract1995;40:435–9.

10. Grol R, Woolf S, Eccles M, Maisonneuve H.Developing clinical practice guidelines: European expe-riences. Diasease Management and Health Outcomes1998;4:255–66.

11. Grol R. National standard setting for quality ofcare in general practice: attitudes of general practitionersand response to a set of standards. Br J Gen Pract1990;40:361–4.

12. Grol R, Zwaard A, Mokkink H, et al. Dissem-ination of guidelines: which sources do physicians use inorder to be informed? Int J Qual HealthCare1998;10:135–40.

13. Rogers E. Diffusion of innovations. New York:Free Press; 1995.

14. Spies T, Mokkink H, Grol R. Assessing theuse of national evidence-based clinical guidelines. (Inpress)

15. Brook R, McGlynn E, Cleary P. Quality ofCare 2: Measuring quality of care. N Engl J Med1996;335:966–70.

16. Eddy D. Performance measurement: problemsand solutions. Health Aff 1998;17:7–25.

17. McColl A, Roderick P, Gabbay J, et al. Per-formance indicators for primary care groups: anevidence-based approach. Br Med J 1998;317:1354–60.

18. Grol R, Dalhuijsen J, Thomas S, et al. At-tributes of clinical guidelines that influence use of guide-lines in general practice: observational study. Br Med J1998;317:858–61.

Vol. 39, No. 8, Supplement 2 SUCCESSES AND FAILURES

II-53

19. Winkens R, Ament A, Pop P, et al. Effects ofroutine individual feedback over nine years on generalpractitioners’ requests for tests. Br Med J 1996;312:490.

20. Smeele I, Grol R, Van Schayck C, et al. Cansmall group education and peer review improve care forpatients with asthma/chronic obstructive pulmonary dis-ease? Qual Health Care 1999;8:92–8.

21. Hulscher M, Van Drenth B, Van der WoudenJ, et al. Changing preventive care: a controlled trial onthe effects of outreach visits to organise prevention ofcardiovascular disease. Qual Health Care 1997;6:19–24.

22. Grol R, Lawrence M. Quality improvement bypeer review. Oxford: Oxford University Press; 1995.

23. Grol R. Quality improvement by peer review inprimary care: a practical guide. Qual Health Care1994;3:147–52.

24. Van der Weijden T, Grol R, Knottnerus J.Feasibility of a national cholesterol guideline in dailypractice. A randomized controlled trial in 20 generalpractices. Int J Qual Health Care 1999;11:131–7.

25. Van der Weijden T, Grol R, Schouten B,Knottnerus A. Barriers to working according to choles-terol guidelines. A randomized controlled trial imple-mentation of national guidelines in 20 general practices.Eur J Public Health 1998;8:113–8.

26. Schers H, Braspenning, Drijver R, et al. Lowback pain in general practice: reported management andreasons for not adhering to the guidelines in the Neth-erlands. Br J Gen Pract 2000;50:640–4.

27. Schers H, Wensing M, Huysman Z, et al. Lowback trouble: a qualitative study on barriers for guidelineimplementation. (In press)

28. Hulscher M, Van Drenth B, Mokkink H, etal. Barriers to preventive care in general practice: the roleof organizational and attitudinal factors. Br J Gen Pract1997;4:711–4.

29. Tacken M, Braspenning J, Van Paasen J, et al.Nine years of flu vaccination in family practice. Huisartsen Wetenschap 2000;43:566–7.

30. Hermens R, Tacken M, Hulscher M, et al.Cervical cancer screening in family practices in TheNetherlands. Prev Med 2000;30:35–42.

31. Hermens R, Hak E, Hulscher M, et al. Adher-ence to guidelines on cervical cancer screening in generalpractice: program elements of successful implementa-tion. (In press)

32. Thomson MA, Oxman A, Davis D, et al.Outreach visits to impose health professional practiceand health care outcomes. In: The Cochrane Library,Issue 1, 1999. Oxford: Update Software.

GROL MEDICAL CARE

II-54