J7ournal of Epidemiology and Community Health 1993; 47: 127-133

Maintaining evaluation designs in long termcommunity based health promotion programmes:Heartbeat Wales case study

Don Nutbeam, Christopher Smith, Simon Murphy, John Catford

AbstractStudy objective-To examine the difficultiesof developing and maintaining outcomeevaluation designs in long term, communitybased health promotion programmes.Design-Semistructured interviews ofhealth promotion managers.Setting-Wales and two reference healthregions in England.Participants-Nine health promotion man-agers in Wales and 18 in England.Measurements and main results-Informa-tion on selected heart health promotionactivity undertaken or coordinated by healthauthorities from 1985-90 was collected. TheHeartbeat Wales coronary heart disease pre-vention programme was set up in 1985, and aresearch and evaluation strategy was estab-lished to complement the intervention. Asubstantial increase in the budget occurredover the period. In the reference healthregions in England this initiative was notedand rapidly taken up, thus compromisingtheir use as control areas.Conclusion-Information on large scale,community based health promotion pro-grammes can disseminate quickly and inter-fere with classic intervention/evaluationcontrol designs through contamination.Alternative experinental designs for assess-ing the effectiveness of long term interven-tion programmes need to be considered.These should not rely solely on the use ofreference populations, but should balancethe measurement ofoutcome with an assess-ment of the process of change in com-munities. The development and use ofintervention exposure measures togetherwith well structured and comprehensive pro-cess evaluation in both the intervention andreference areas is recommended.

Health PromotionAuthority for Wales,CardiffD NutbeamC SmithS MurphyInstitute for HealthPromotion, Universityof Wales Coliege ofMedicine, CardiffJ Catford

Correspondence to:Professor D Nutbeam,Department of PublicHealth, University ofSydney, Sydney, 2006,Australia.

Accepted for publicationAugust 1992

Epidemiol Community Health 1993; 47: 127-133

The late 1970s and early 1980s saw the develop-ment of a number oflarge scale, community basedhealth promotion programmes in the UnitedStates and Europe, most of which were directedtowards reducing modifiable risk factors for coro-

nary heart disease. -5 These community basedprogrammes were born out of a growing con-

sensus on the content, strategy, and methods forthe prevention of coronary heart disease whichemphasised the importance of lifestyle andbehavioural change on a population wide basis.6 7

Fundamental to the achievement and mainten-ance of lifestyle and behavioural change is success

in influencing a range of predisposing factors.These include those relevant to the individual(such as knowledge, attitudes, and skills), thoserelevant to the wider social group (social andcultural norms), and factors in the wider environ-ment (such as regulation of exposure to hazards,and access to goods and services). Influencingsocial norms and changing the wider environmentgenerally requires a total community response andhence the rationale for a community basedapproach. More practical reasons include (i) theopportunities to utilise the existing resources andnetworks within communities to provide a struc-ture for programme delivery; (ii) the enhancedimpact from combining complementary indi-vidual and environmental approaches to achievingchange; and (iii) the greater potential for creatingreproducible and self sustaining programmes in'real life' settings than those from narrowlydefined studies of volunteer populations in con-trolled settings.8These new intervention approaches have posed

special challenges for developing appropriate andmanageable evaluation designs that can be main-tained throughout the life of a health promotionprogramme. This paper uses the evaluation of theHeartbeat Wales programme as a case study toassess the sorts of difficulties that can beencountered and how they may be overcome.

Heartbeat WalesDuring the early 1 980s several health bodies in theUnited Kingdom were becoming conscious of theneed to respond to the high level of cardiovasculardisease in the country through preventive stra-tegies. The UK government's Welsh Office andthe then national agency for health education, theHealth Education Council, agreed to establish acommunity based demonstration project inWales. This decision was supported in 1983 by aconsensus conference, sponsored by the majorUK health organisations, which recommended acomprehensive range of actions by government,agriculture, education, health services, and themedia.9The 'Welsh Heart Programme' was at first

administered through the University of WalesCollege ofMedicine and then subsequently by theHealth Promotion Authority for Wales. Thedirectorate responsible for the management of theprogramme was recruited at the end of 1984 andthe programme was launched publicy as 'Heart-beat Wales' in March 1985 for an initial five yearperiod. The long term goal of the programme wasto develop and evaluate as a demonstration pro-ject, a regional strategy that would contribute to asustained reduction in coronary heart disease

on May 26, 2021 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.47.2.127 on 1 April 1993. D

ownloaded from

Don Nutbeam, Christopher Snzith, Simon Murphy, 3tohn Catford

incidence, morbidity, and mortality in the generalpopulation of Wales, particularly among thoseunder the age of 65. Desired health outcomes forthe first five years were defined as improvements innon-smoking, healthy nutrition, physical exercise,hypertension control, and cardiopulmonary resus-citation. 10When the programme was established there

were three strategic aims namely:(i) Leadership to coordinate, support, initiate,

and monitor action at local and regional levelswhich would encourage improvements in modifi-able risk factors and behaviours for coronary heartdisease prevention.

(ii) Demonstration-to stimulate, disseminate,and assist the development of strategies andprogrammes to promote health and prevent coro-nary heart disease throughout the UK andoverseas.

(iii) Experimentation to research, develop,and evaluate a range of new projects and iniativesfor heart health promotion, and provide feedbackon their feasibility and impact;These aims gave direction to the development ofthe evaluation strategy, which also took intoaccount the essential elements of the interventionthat sought changes in both personal behavioursand environmental factors.'0 11 Measures tomonitor both these changes were built into theevaluation design, which also recognised the needto balance measurement of outcome with investi-gation of the process of achieving change.'2

EVALUATION DESIGN PROBLEMS AND SOLUTIONSAt the time of constructing the evaluation designthere were relatively few well tested models for theevaluation of community based programmes.Those that existed had been developed fromrelatively small and more tightly defined interven-tions, in individual communities, towns, and cities(populations 100-200 000). Wales is a compara-tively large country, however, with a substantiallygreater population (2-8 million), defined by ninehealth districts. The experience provided by theseother projects had already indicated a number ofbasic problems in applying experimental designsto community based programmes. These includedthe impracticality of artificially assigning indivi-duals into intervention and control groups withincommunities because ofthe constraints this placeson the development of an intervention strategy.These constraints include effectively preventingthe use of the media, and the use of existingcommunity networks and infrastructures-all ofwhich cannot be controlled to reach definedindividuals within a community, but representpart of the attraction of adopting a communitybased approach. Further problems associated withthe difficulties of tracing the causal pathways incommunities (which are generally longer andharder to trace than in studies on volunteerindividuals), and with achieving high levels ofpenetration and participation from the wholepopulation had also been identified.8 13 15

A number of solutions to these problems havebeen proposed and tested. The commonly advo-cated solution in the mid 1980s was the use of aquasi-experimental design based on comparisonof change between matched populations in inter-vention area(s) and separate reference area(s).

These were generally supported by additionalevaluation studies to improve confidence in thecausal nature of observed net differences betweenthe populations.8 '3 In most examples thesedesigns were non-random, as the interventionareas were generally chosen on an opportunisticbasis. The supporting studies usually examinedthe impact on targeted community networks andthe process of diffusion ofthe intervention. As partof this approach, measures for use in populationsurveys were developed to determine programmeexposure (that is, awareness and participation)among the target population.813The evaluation ofthe North Karelia programme

was based on comparisons between a single inter-vention and single reference community, as too isthe Pawtucket Heart Health programme.' 16 TheStanford and Minnesota programmes have usedseveral intervention and reference commu-

2 317nities. The basic evaluation design for theHeartbeat Wales programme was establishedusing this quasi-experimental approach. It wasconstructed in such a way as to allow both internalcomparison of differences in outcome between thenine health districts (each ofwhich was to receive acontrasting intervention) and external compa-risons between Wales and a large single matchedreference area of similar size. At its simplest, theintention was to measure change in the populationrisks for coronary heart disease (and associatedpredisposing factors) after the 'input' of a coordi-nated and relatively well resourced communitybased intervention, and to compare this withchange in the reference area with no correspond-ing input, conducting 'business as usual'.The reference area was selected after examin-

ation of available health, social, and economicdata as being closest in profile to Wales at baseline.Full details have been published previously. 18 As aresult of the difficulties experienced in the NorthKarelia programme, which shared a commonboundary with its chosen reference area, a geo-graphically separate area was deliberately selectedto reduce problems of media overspill and moregeneral contamination that might occur along ashared border. TIhe reference area consisted of 16health districts which fell within two healthregions.The basic outcome evaluation design was based

on three population surveys in 1985, 1988, and1990 in both Wales and the reference area. Theintervention in Wales was expected to developduring 1986 and reach an optimal level of invest-ment and impact during 1987-89, after which itwas anticipated that investment might decline ifthe programme was not funded beyond its initialperiod. A range of indicators was used at baselinein Wales and the reference area to enable thetracking of changes in knowledge, attitudes, andbehaviour relating to coronary heart disease risk.Measures to identify exposure to and involvementin the programme were also developed for thefollow up studies. In Wales, a clinical survey wasalso undertaken to extend the range of dataavailable to include blood pressure, serum choles-terol, and physical fitness and to validate selfreported behaviour. In addition, a range of separ-ate studies to examine the impact of the pro-gramme on community systems (such as thehealth and education services) and among tar-

128

on May 26, 2021 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.47.2.127 on 1 April 1993. D

ownloaded from

Long term evaluation of health programmes

geted professional networks (such as GPs) wasinitiated in Wales, to help identify supportivechanges in the health promotion infrastructureand to enable a better understanding of theobserved health outcomes.'2The Health Education Council had indicated

that there would be no major provision ofadditional resources for heart disease preventionin the rest of the country (including the referencearea) pending the outcome of the Welsh 'demon-stration' programme. Furthermore, no substantialnew coordination of existing efforts in thereference community was envisaged at the time.Nevertheless, as Heartbeat Wales had been estab-lished as a national demonstration project, somediffusion of new ideas was anticipated as theintervention developed. This, it was hypothesised,would dilute, but not substantially compromise,any observable outcomes. Further details of thesurvey methods used in the baseline and follow upstudies are available, together with an overview ofpreviously published results and separate studiesto examine the reliablity and validity of the datacollected. 18-22When Heartbeat Wales was established, a

record keeping framework and a range of specificstudies were set up to document the programme'sprogress towards its stated objectives. ' 12 Most ofthese studies have been completed as planned, anda basic system of record keeping has been estab-lished through a structured annual planning andreporting process at both national (all Wales) anddistrict levels. Separate studies have documentedprogress in the development of the overall nutri-tion strategy;23 the catering accreditation projectentitled Heartbeat Awards,24 the introduction ofhealthy catering services in hospitals;25 the costsand benefits of the smoking education pro-gramme;26 collaboration with local and nationalbodies in promoting exercise in the community;27the development of heart health programmes indifferent settings, such as worksites28 andschools;29 and developments within professionalgroups, such as general practitioners30 and healthvisitors.3' No attempt was planned or made toestablish such detailed record keeping or to under-take comparable studies in the reference com-munity. This was not considered necessary norwas it affordable at the time Heartbeat Wales wasestablished.The outcome evaluation for the Heartbeat

Wales programme was thus based on the premisethat there would be no substanital additionalinvestment in heart disease prevention in the restof the UK for a period of five years. It becameapparent as Heartbeat Wales developed, however,that the level of interest was considerable and thetransfer of ideas and projects was occurring earlierthan was anticipated at the initial planning phase.It was realised that the specific transfer of com-ponents ofthe Heartbeat Wales intervention to thereference area could have a considerable impacton the validity of the basic Heartbeat Walesevaluation design. For this reason a study wasundertaken during 1990-91 to assess the extent towhich key inputs of the Heartbeat Wales pro-gramme had also occurred in the reference com-munity. The implications for the assessment ofdifferences in health behaviour and risk factorchanges between Wales and the reference area, the

basis for the outcome evaluation, would also beconsidered.

MethodsIn the UK, only health authorities have a specificresponsibility to coordinate a range of healthpromotion programmes through different settingsinlcuding community health centres, worksites,the media, and schools. Most have thereforeestablished a discrete health promotion serviceand employ staff to provide local leadership andcoordination. Given this central role, a reasonableway of assessing the extent of heart disease pre-vention activity is to carry out a review of relevantactivity either directly undertaken or coordinatedby the health authorities. This was the approachadopted in this study for both Wales and thereference area.A series of semistrucured interviews was con-

ducted with each of the nine district healthpromotion managers in Wales and their 16counterparts in the reference area by a trainedinterviewer (SM). In addition, the two relevantregional health promotion managers were asked tosupply information on activities relating to hearthealth promotion for their respective regions in thereference area between 1985-90. The HealthEducation Authority in England also provideddetails of their Look After Your Heart pro-gramme. The Heartbeat Wales interventiondatabase was also scrutinised for regional typeactivities across Wales.The questionnaire which formed the basis for

the interviews was developed and piloted in dis-tricts outside the study area. It covered threesubstantive areas ofthe Heartbeat Wales interven-tion, as follow:

(i) Coordinating, monitoring, and com-municating: This covered records of the planningprocess for heart health interventions, and ofresearch and evaluation activity. It also includedrecords ofthe adoption ofprogrammes or projectswith distinctive identities and logos, as well as theproactive use of the media.

(ii) Creating new resources: This covered twomajor resource inputs-money and staff. Itincluded reference to records of financial allo-cations from district health authorities, includingmoney earmarked for specific coronary heartdisease prevention projects, as well as externalfunding obtained for coronary heart disease pro-jects. It also included records of staffing levels,both for 'general' health promotion activities, andof staff specifically dedicated to heart healthpromotion.

(iii) Developing supportive environments: Thisincluded records of special projects directedtowards influencing the environment in its widestsense and details of relevant supportive policydevelopment.Although these three themes account only par-

tially for the wide range of programme activitieswhich operated at both national and local level inWales, they are recognisable as forming key inputsthat would distinguish the programme in Walesfrom 'business as usual' in the reference area. Asinputs, they can be quantified in such a way as toallow for a comparison between Wales and thereference area.

129

on May 26, 2021 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.47.2.127 on 1 April 1993. D

ownloaded from

Don Nutbeanm, Chnistopher Smith, Simon Murphy, J7ohn Catford

The respondents were interviewed face to faceand were given guidance on what constituted hearthealth activity. This included project or pro-gramme work on smoking, exercise, bloodpressure screening, cardiopulmonary resuscita-tion, and nutrition. Respondents were told thatthe purpose of the interview was to obtain adetailed account of heart health promotion withintheir district or regional boundaries over theprevious five years. No reference was made toHeartbeat Wales before the interview, and allquestions about respondents' knowledge andviews of the programme were asked towards theend of the interview. Although every effort wasmade to obtain answers to all questions, anexplanatory background and adjustments toinformation were often needed to qualify theresponses. This required supplementary qualita-tive notes to be taken, and their subsequentinterpretation in the construction of Tables I-III.Such a method of gathering information has

certain difficulties. It relies heavily on gainingaccess to the relevant people and records over afive year period. Although there were some con-sistencies in financial record keeping across thedistricts, the level of detail in these records variedconsiderably between health authorities. Severalofthose interviewed, six ofnine in Wales and 10 of18 in the reference area, had been employed eitherin their present managerial position or within theregional or district health authority throughout theperiod 1985-90. Where the current manager hadnot been in the district for that period of time, itwas necessary to involve other staff who had

Table I Coordinating, monitorzng, and conmmunticating in Wales and the reference area

Wales Referenice area(9 districts) (16 districts)

(1) Regional coordination

(2) Creation of intersectionalteams for heart health pro-motion

(3) Providing health statusinformation to decision makersand the media

(4) Collaborative projects withthe media

Establishment of HeartbeatWales as regional focus1985-87Subsumed within HealthPromotion Authority forWales 1988-90

Six created by end of 1985All districts by end of 1986

No equivalent regional to(calpoint 1985-86

Creation of UK Look AfterYour Heart Programme 1987-90

Seven created 1985-88Twelve were in operation by1990

I~~a I

Comprehensive risk factor Comprehensiveprofiles published 1986-87 research in threefor each district. one region 1988

Wide range of integrated Ad hoc reports tresearch studies planned andcarried out

BBC Wales, Don't Break UK networkedYour Heart (1985)BBC 1/2, Food and Health UK networkedCampaign (1986) (multipleprogrammes)BBC 1, Go For It (1987- UK networked89)HTV, When the Chips are Not networkedDown (1986)BBC, Save a Life (1986-87) UK networked

HTV, Fit for Life (1987-88) Not networked

BBC Wales, BBC Diet UK networked(1988)Not available

Substantial range of TV,radio, and newspaper cover-

age throughout 1985-90

Strong corporate image, andhigh level of public aware-

ness and support for theHeartbeat Wales programmeby 1986

Look After Youadvertising (198

and integratede districts and3-1990elsewhere

(1987)

(1986)

(1987- 89)

(1986-87)

(1988-89)

ir Heart TV87-90)

Mostly ad hoc irregular cover-

age in local TV, radio, andnewspapers. In one districtand one region this was more

substantial and coordinated

Two districts and both regionsdeveloped logo profile by 1990

knowledge of the developments during thatperiod.These problems of access were compounded by

the difficulty of collecting comparative data fromdistricts that organise programmes in radicallydifferent ways. The standard format adopted forcollecting this information helped to overcomesome of these difficulties but could not com-pensate for missing or inaccurate data. Only asummary of the full range of information collectedis presented in the results section.

ResultsThe results of the enquiries into interventionsmediated through health authorities are presentedunder the three aspects discussed above.

COORDINATING, MONITORING AND

COMMUNICATINGTable I provides summary information concern-ing a range of indicators of relevance to leadershipand coordination. Heartbeat Wales was launchedas a uniquely dedicated heart disease preventionprogramme in March 1985, and subsequentlybecame absorbed within the Health PromotionAuthority for Wales during 1987 and 1988.During the corresponding period in England, theHealth Education Council (and subsequently theHealth Education Authority) launched, in 1987,the Look After Your Heart (LAYH) programmefor England in conjunction with the UKgovernment's Department of Health. Althoughthis programme was not specifically targetted atthe reference area, it offered a focal point foractivity and an important new source of fundingfor heart disease prevention, including thereference area. In addition to this England-wideactivity, one of the two regions within thereference area (including four districts) estab-lished their own regional strategy for coronaryheart disease prevention in 1988 using the name'heartbeat' in the title.One important objective in establishing Heart-

beat Wales was to generate intersectoral supportfor coronary heart disease prevention. Table I(section 2) shows that the intersectoral teamsneeded to facilitate such an approach were estab-lished in all districts in Wales by the end of 1986.In the reference area, seven of 16 districts hadestablished comparable groups between 1985-88and 12 of 16 by 1990.As intended, Heartbeat Wales established a

comprehensive research and evaluation strategy tocomplement the intervention. This provided awide range of information to support the planningand management of projects. No comparableinformation was available in the reference area in1985, although three districts and one region nowhave reasonably well funded and properlyintegrated research programmes in operation. Inthe case of one district and one regional healthauthority, these were reported to be 'modelled onHeartbeat Wales'.As a large scale programme designed to reach

three million people, Heartbeat Wales needed tocollaborate with the Wales based and UK widemedia. As indicted in table I section 4, several ofthe television programmes that were developed inWales to support the project were subsequently

130

on May 26, 2021 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.47.2.127 on 1 April 1993. D

ownloaded from

Long termn evaluation of health programmnes

modified and networked thoroughout the UK,and therefore into the reference area. Consider-able resources were also invested by HeartbeatWales in creating a strong corporate image for theprogramme. As a consequence of this, awarenessand recall of the Heartbeat Wales programmewere high in Wales from 1986 onwards. In 1986,53% of adults had heard ofHeartbeat Wales whenprompted,32 a figure which rose to 71% by 1989.33Although no equivalent programme image existedfor the reference area as whole, two of the districtsand both regions had developed coordinated pro-grammes with distinctive identities and logos by1990. Interviews with respondents clearly estab-lished that Heartbeat Wales had been an impor-tant influence in both stimulating theestablishment of heart health activities, as well asguiding selection of content.

All of the 18 health promotion managers in thereference area had heard of and read aboutHeartbeat Wales. Thirteen stated that they hadhad direct personal contact with the programmethrough such things as conferences, visits or tours,professional relationships, and the use of Heart-beat Wales publications and resources.

CREATING NEW RESOURCESTable II indicates the resources invested directlyin coronary heart disease prevention within threecategories-dedicated staff, operational budget atdistrict level for heart health from externalsources, and operational budget at regional level.The data are presented in absolute terms, and thenper 100 000 population (in brackets). In Wales thestaff devoted to heart disease prevention and thecash available at district level rose substantiallybetween 1985 and 1988, but declined between1989 and 1990. At the regional level in Walesresources increased between 1985 and 1988, fellin 1989, and rose again in 1990. In the referencearea, a similar pattern of resource deployment canbe observed at district level. There seems to havebeen a sustained increase in budgets at theregional level, however, reflecting HealthEducation Authority/Department of Healthexpenditure since 1987 on their LAYH pro-gramme, and a specific one off grant in 1990 insupport of a special, year long initiative. Overall,cash investment in heart health promotion seemsgreater in the reference area than in Wales for 1989and 1990.

Table II Creating niew resourcesWales Referenice area(2 88 miiillion people) (3 miiillioni people)

(1) Changes in staff devoted 1985-88 1 from 1-5 (0 05) 1985-88 1 from 1 (0to heart health at district level to 14 (0 49) 7-5 (0-25) persons

1989-90 1 14 4 (0 5) to 8 1989-90 1 from 6 5(0 28) 3-5 (0 12) persons

(2) Changes in budget for 1985-88 1 from £16 000 1985-88 T £9600 (£health from external sources at (£556) to £239 000 £184 000 (£6130)district level (£8300)

1989-90 1 from £165 000 1989-90 1 £166 900(£5730) to £92 800 to £97 000 (£3230)(£3220)

(3) Changes in budget for (HBW) 1985-88 T from 1985-88 1 from 0 toregional heart health pro- £193 060 (£6720) to £254 050 (£8470)motion (excluding research £406 820 (£14 160)and evaluation, salaries, travel (HBW) 1989-90 1 from 1989-90 1 from £30and rent)* £189 500 (£6600) to (£10 010) to £1 078

£261 430 (£9100) (£35 960)LAYH=Look After Your Heart; HBW=Heartbeat WalesBudgets relate to National Health Service financial vears. Figures in brackets represennumber of staff or budget per 100 000 population.*Data for the reference area include an apportionment of the LAYH national budgeting overhead costs and direct regional allocations) on a population pro-rata basis, butmake allowance for other expenditure by the HEA on coronarv heart diseae preventioactivity, for example through its Smoking and Nutrition education programmes.tThis figure includes a special one year grant of £750 000.

Table III Developing supportive environments

(1) Food labelling and nutri-tion education with TescoSupermarkets(2) Lean meat merchandisingscheme with Meat and Live-stock Commission throughbutchers(3) 'Super Ted' fruit and veg-etables promotion throughsupermarkets(4) Heartbeat Award Schemefor restaurant and workscanteens(5) Make Health Your Busi-ness (joint Heartbeaat Wales/CBI initiative) for commerceand industry(6) Introduction of policies torestrict smoking in NHSpremises

(7) Introduction of policies topromote healthy catering inNHS premises(8) Healthy Hospital AwardScheme for NHS

Wales(9 districts)

Developed initially in Wales1985 and then maintained1985-90Developed and piloted inWales 1988-90

Developed in Wales 1989-90

Reference area(16 districts)

Applied UK wide 198

Operating in three disthe reference area 198

CREATING SUPPORTIVE ENVIRONMENTSTable III indicates a range of special initiatives,largely outside of the health sector, directed

103) to towards influencing the environment. Theseinclude those intended to improve the availability

(0 22) to and labelling of food (section 1-3), the availability320) to of healthy choices in restaurants (section 4), and

changes in worksite health promotion (section 5).Table III indicates how four of these special

t(£5560) initiatives have also reached the reference area,either fully (sections 1 and 3) or with partialcoverage (sections 2 and 4). The workplace pro-gramme, as conceived in Wales, was at least partly

0 400 matched through the LAYH programme in nine of750t the districts in the reference area.

Table III also highlights progress towardsit the creating supportive environments within the(exclud- health system (sections 6-8). Through policydoes not development work and the Healthy Hospital

Award Scheme, developed in Wales in 1987, theobjectives of securing a policy commitment fromthe health system in relation to smoking, nutrition,and catering were met within Wales for all districtsby 1988. In the reference community, all districtshad achieved similar progress by 1989 in relation

35-90 to smoking, and most (14 out of 16) had policiesrelating to nutrition and catering by 1990. The

,tricts in Healthy Hospital Award Scheme, however, had38-90 not been transferred to the reference area up until

1990.Applied UK wide 1990

Developed and operating in Operating in eight districtsWales 1985-90 (all districts) (1988-90)

Developed and operating in Look After Your Heart work-Wales 1988-90 (all districts) place programme operating in

nine districts 1989-90.

One district by 1985 Three districts bv 1985All districts by 1988 14 districts by 1988

All districts by 1989Four districts by 1985 One district by 1985All districts by 1988 Nine districts by 1988

14 districts by 1990Developed and operated in Not transferred up until 1990Wales from 1987

DiscussionThe results need careful interpretation since therange of indicators selected are partial in theirability to describe the intervention in Wales andrelated activities in the reference area. The study isalso retrospective, and relies on the frailties ofhuman recall and health service oriented recordsystems. There may also be differences in interpre-tation as to what consitutes heart disease pre-ventive programmes and in the classification ofpeople as being 'dedicated' heart disease pre-vention workers. Although the information for

131

on May 26, 2021 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.47.2.127 on 1 April 1993. D

ownloaded from

Don Nutbeanm, Christopher Snith, Simon Murphy, John Catford

Wales was verified using the previously mentionedrecord keeping system, this was not possible in thereference area.The available information indicates that the

Heartbeat Wales programme was largely suc-cessful in providing the programme 'input'described in its original planning documents. Theprogramme seems to have led to a clear increase inresources for coronary heart disease prevention,important public education programmes, andobservable changes in related policy and infra-structure in Wales. Assessing whether or not this'input' has led to improved population health andreduced cardiovascular risks will be the subject offurther reports detailing findings from the popula-tion surveys conducted in Wales and the referencearea in 1985, 1988, and 1990.The study has also shown a rapid uptake ofheart

disease prevention activities in the reference area.

This uptake occurred through a number of ways,including.

(a) The pilot projects which were set up inWales and very quickly became networkednationally

(b) The England-wide intervention programmeLAYH promoted by the Health EducationAuthority and Department of Health and SocialSecurity.

(c) A general growth in interest in coronaryheart disease prevention encouraged, for example,through national media initiatives some of whichemanated from Wales.

(d) Special one off events (such as the LifestyleHeart Health exhibition in 1989), which may havefurther 'legitimised' action already occuring in thereference area.Such substantial developments in the reference

area may undermine the usefulness of the basicquasi-experimental design established in 1985 to

assess the health outcomes of Heartbeat Wales.There is clearly much less of a contrast in 'input',particularly in the latter years, between Wales andthe reference area than was envisaged in theplanning of the evaluation design. As a result, infuture analysis of Heartbeat Wales data, greateremphasis will need to be placed on the range ofprocess evaluation studies that have examined theprogramme's development and impact in Wales.These could provide useful evidence demon-strating the links between Heartbeat Wales activityand changes in the infrastructure, systems, andservices that influence health behaviour. 12 Greateremphasis will also need to be placed on using themeasures ofprogramme exposure contained in thepopulation lifestyle surveys in Wales and in thereference area. These may help discriminatebetween observed changes in health status amongthose individuals who have been heavily exposedto the various programmes and those who have not

been so exposed.More generally, these findings may have

implications for others who are planning to assess

the effectiveness of long term community basedprogrammes. In particular, they question theclassic use of reference areas in which researchcontact with such communities is minimised forfear of contamination. Since this study suggeststhat contamination of reference areas can occur

rapidly anyway, it may be preferable to undertakeprocess evaluation studies in reference com-

munities so as to establish the contrast in activitybetween them and the intervention community. Inthe Heartbeat Wales programme, process evalu-ations were conducted in Wales only and weredirected mainly towards tracing the impact of theintervention through intermediaries such as pri-mary health care and worksites to the generalpublic. Greater replication of these studies in thereference area, and more systematic collection ofdata on input would have been beneficial, andshould be carefully considered in future designs.

Alternatively, the findings suggest that in thistype of intervention programme greater con-sideration should be given to outcome evaluationdesigns which do not rely on comparing changewith a reference population. Such designs havebeen outlined elsewhere and might include, forexample, the use of a longer sequence of measure-ment points both before and after the programmein the intervention community alone.34

Finally, this experience points to the need to

develop good quality exposure measures in pop-ulation surveys. These assist analysis of outcomeby distinguishing individuals who have clearlybeen exposed to the programme (or elements ofthe programme), and those individuals withincommunities who have not. Similarly, it may alsoallow for confirmation of unexpected exposure to

programme elements in the reference community,if a reference community is to be used.

The authors thank Jo Clarkson for fieldwork support,Alison McKellar and Sue Avery for their help in pre-paring this paper, and Les Irwig, Bob Spassof, and PaulLincoln for contructive criticism.

1 Puska P, Nissinen A, Tuomilehto J, et al. The communitybased strategy to prevent coronary heart disease;conclusions from ten years of the Karelia project. Annu RevPublic Health 1985; 6: 147-63.

2 Farquhar JW, Wood PD, Breitrose H, et al. Communityeducation for cardiovascular health. Lancet 1977; i: 1192-95.

3 Farquhar JW, Fortmann SP, Flora JA, et al. Effects ofcommunitywide education of cardiovascular risk factors:the Stanford five-city project. JAMA 1990; 264 (3): 359-65.

4 Blackburn H, Luepker R, Kline FG, et al. The Minnesotaheart health program: a research and demonstration projectin cardiovascular disease prevention. In: Matarazzo JD,Weiss SM, Herd JA, Miller NE, Weiss SM, eds. Behaviouralhealth: a handbook of health enhancement and diseaseprevention. New York: John Wiley and Sons, 1984.

5 Edler J, McGraw S, Abrams D. Organizational andcommunity approaches to community-wide prevention ofheart disease: the first two years of the Pawtucket hearthealth program. Preventive Medicine 1986; 15: 107-17.

6 World Health Organization. Prevention of coronary heartdisease. Geneva: WHO 1982. Report of a WHO ExpertCommittee. Technical Report Series no 678.

7 World Health Organization. Community prevention andcontrol of cardiovascular diseases. Geneva: WHO, 1986Report of a WHO Expert Committee. Technical ReportSeries no 732.

8 Farquhar JW. The community based model of lifestyleintervention trials. Am J Epidemiol 1978; 108 (2): 103-11.

9 Rose F, Ball K, Catford JC, et al. Coronary heart disease:plans for action. London: Pitman, 1984.

10 Welsh Heart Programme Directorate. Take heart: a

consultative document on the development of community-basedheart health initiatives within Wales. (Heartbeat Report No1). Cardiff: Health Promotion Authority for Wales, 1985.

11 Catford JC, Nutbeam D. Examples of good practice in theUK: Heartbeat Wales. In: Williams K, ed. Communityprevention of coronary heart disease London: Pitman, 1992.

12 Nutbeam D, Catford JC. The Welsh heart programmeevaluation strategy: progress, plans and possibilities. HealthPromotion 1987; 2: 5-18.

13 Farquhar JW, Fortmann SP, Wood PD. Community studiesof cardiovascular disease prevention. In Kaplan NM,Stamler J eds. Prevention of coronary heart disease: practicalmanagement of risk factors. Philadelphia: WB Saunders andCo, 1983.

14 Blackburn H. Research and development projects incommunity cardiovascular disease prevention. 7 PublicHealth Policy 1983; 4: 4.

15 Altman DG. A framework for evaluating community-basedheart disease prevention programs. Soc Sci Med 1986; 22:479-87.

132

on May 26, 2021 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.47.2.127 on 1 April 1993. D

ownloaded from

Long term evaluation of health programnmes

16 Assaf A, Bonspach S, Lasater T, et al. The Pawtucket hearthealth program: evaluation strategies. Rhode Island MedicalJournal 1987; 70: 541-6.

17 Mittelmark M, Luepker R, Jacobs D, et al. Community-wideprevention of cardiovascular disease: education strategies ofthe Minnesota heart program. Prev Med 1986; 15: 1-7.

18 Welsh Heart Programme Directorate. Welsh heart healthsurvey 1985: protocol anid questioznzaire. (Heartbeat Report no

2). Cardiff: Health Promotion Authority for Wales, 1985.19 Welsh Heart Programme Directorate. Pulse of Wales:

preliminary report of the Welsh heart health survey 1985.(Heartbeat Report no 4). Cardiff: Health PromotionAuthority for Wales, 1986.

20 Welsh Heart Programme Directorate. Heart of Wales: clinicalresuilts of the Welsh heart survey, 1985. (Heartbeat Report no

20). Cardiff: Health Promotion Authority for Wales,1987.

21 Smith C, Nutbeam D. Assessing non response bias: a case

study from the 1985 Welsh heart health survey. HealthEducation Research 1990; 5.3: 381-6.

22 Pullen E, Nutbeam D, Moore L. Demographiccharacteristics and health behaviours of consentors tomedical examination: results from the Welsh heart healthsurvey. Jf Epidemiol Coionnunity Health 1992; 46: 455-9.

23 Parish R, Catford J, and Howson H. Promoting healththrough collaboration with commerce and industry-experience of the Welsh heart programme. In: Comn,nunitybased preventionz and health promlotionl-report of an

internatioonal conference. Dusseldorf: German Society ofSocial and Prophylactic Medicine and WHO, 1987.

24 Parish R, Catford J, Nutbeam D. Breathing life into Wales:progress of the Welsh heart programme. Health Trends1987; 19: 23-6.

25 Clarkson J, Nutbeam D. Introducing healthy cateringpractice into hospitals: a case studv from Wales. _ounial ofiN utrition and Health 199 1; 7: 1 0 1 -1 0.

26 Nutbeam D, Prowle M, Phillips K. The econzomiiic bencfits ofsm?ioking interventions: a case study of heartbeat Wales. Cardiff:Health Promotion Authoritv for Wales, 1990.

27 Nutbeam D, Catford JC. Promotiong physical activity, in thecomImunity: experienices from the heartbeat Wales progranon?iie.In: Oja P, Telama R eds. Sportfor all Amsterdam: Elsevier,1991: 174-84.

28 Clarkson J, Blower E, Hunter C. et al. Ani over-view ofitnnovative workplace actioni for health in the UK; WorkingPaper No WP/91/03/EN. Dublin: European Foundation forthe Improvement of Living and Working Conditions, 1991.

29 Nutbeam D, Clarkson J, Phillips K, Everett V, Hill A,Catford J. The health promoting school: organisation andpolicy development in welsh secondarv schools. Health Educ_

1987; 46: 109-15.30 Nutbeam D, Catford J. Modifiable risks for cardiovascular

disease among general practitioners in Wales. Puiblic Health1990; 104: 353-61.

31 Hayes E, Nutbeam D. Health promlotionz practice amiionig healthvisitors in Wales. Welsh Opinion Leaders Study BriefingReport No. 1 Cardiff: Health Promotion Authority forWales, 1988.

32 Welsh Heart Programme Directorate. Heartbeat Walesazvaretness a7nd recall. Cardiff: Health Promotion Authoritvfor Wales, 1986.

33 Beaufort Research Ltd. Welsh omyniibus survz,,ey,. Cardiff:Health Promotion Authority for Wales, 1989.

34 Green LW, Lewis FM. Afeasuiremiienit anid evaluation in healtheducationz anid health prom;iotioni. Palo Alto, CaliforniaMavfield Publishing Company, 1986.

133

on May 26, 2021 by guest. P

rotected by copyright.http://jech.bm

j.com/

J Epidem

iol Com

munity H

ealth: first published as 10.1136/jech.47.2.127 on 1 April 1993. D

ownloaded from