Predicting intermediate outcomes for prevention coalitions: adevelopmental perspective

Paul Florina,*, Roger Mitchellb, John Stevensona, Ilene Kleina

aDepartment of Psychology, University of Rhode Island, Kingston, RI 02881, USAbNorth Carolina State University, USA

Received 14 October 1999; received in revised form 29 February 2000; accepted 27 March 2000

Abstract

Longitudinal data from 35 substance abuse prevention coalitions were used to examine whether success in addressing initialcoalition developmental tasks predicted intermediate outcomes one year later. Organizational climate, member skill development

and coalition linkages predicted key informants' ratings of coalition e�ects on community norms, policies, and preventionresources. 7 2000 Published by Elsevier Science Ltd.

Keywords: Community coalitions; Substance abuse prevention; Coalition outcomes; Community intervention

1. Introduction

Although the use of local coalitions as a strategy forprevention and health promotion is now widespread,ensuring the success of such e�orts remains a chal-lenge. Case studies provide rich descriptions of thecomplex dynamics that occur when coalitions addresscommunity problems (Kegler, Steckler, McLeroy &Malek, 1998; Mayer, Soweid, Dabney, Brownson,Goodman & Brownson, 1998; Mizrahi & Rosenthal,1992). Such analyses suggest numerous areas in whichtechnical assistance interventions might be useful.However, current research tells us little of howimprovements in members' skills, coalition decision-making, or other processes in¯uence a coalition's ulti-mate impact. Studies assessing coalition e�ects on sub-stance abuse outcomes are limited in their ability toimprove intervention strategies if they fail to examinethe intermediate coalition processes from which e�ectsare expected to emerge (Yin, Kaftarian, Yu & Jansen,

1997). Without such information, it's di�cult to knowwhere to invest scarce technical assistance resources toimprove the odds of success.

2. Objectives

In this study, we describe a conceptual frameworkthat suggests key coalition developmental tasks (e.g.,mobilization, establishing organizational structure,building capacity) and examine whether progress inaccomplishing initial developmental tasks is associatedwith the quality of later coalition outcomes.

Coalition interventions are complex. They demandconceptual frameworks that link coalition character-istics with intermediate processes and ultimate out-comes (Saxe, Reber, Hallfors, Kadushin, Jones &Rindskopf, 1997; Weiss, 1995). We have found an or-ganizational ``open systems'' framework a useful wayof characterizing the seemingly bewildering dynamicsamong coalition stakeholders and the constituenciesthey are trying to in¯uence (Katz & Kahn, 1978). Asdisplayed in Fig. 1, ``open systems'' concepts (e.g.,input, throughput and output) can be applied to co-alition processes. Starting with varying levels of

Evaluation and Program Planning 23 (2000) 341±346

0149-7189/00/$ - see front matter 7 2000 Published by Elsevier Science Ltd.

PII: S0149-7189(00 )00022 -7

www.elsevier.com/locate/evalprogplan

* Corresponding author. Tel.: +1-401-874-4234; fax: +1-401-874-

2157.

E-mail address: p¯4616u@posto�ce.uri.edu (P. Florin).

human and material resources (e.g., funding, technical

assistance, and varied community representation), co-

alitions must develop rules and procedures for working

together. This is no small task for a coalition, by de®-

nition a voluntary group of ``equal'' organizations that

may have little history in working together. Coalitions

that do a better job with their throughputs will pre-

sumably produce more e�ective outputs.1

With the explosion in coalition formation during

the last decade, the focus has naturally been on the

early stages of coalition development. In the

absence of ``hard'' community outcomes, many co-

alition leaders pointed to successes in recruiting par-

ticipants from key community constituencies and in

establishing organizational structures that promote

cohesive and task-focused teams. In fact, much of

the initial funding for CSAP's Community Partner-

ship Program was earmarked for coalition capacity-

building (e.g., providing participants with necessary

skills, and encouraging coalition linkage with key

constituencies). Such e�orts would presumably result

in more e�ective planning and implementation (e.g.,

selecting ``best practice'' programs, reaching typically

underserved populations). Earlier ``developmental''

tasks would be mastered and set the stage for tack-

ling more intractable community outcomes.

The appeal of a developmental perspective is the

possibility of tailoring technical assistance interven-

tions to a coalition's speci®c circumstances in the same

way that stage-based theories of individual behavior

change have allowed tailored health interventions (Pro-

chaska, DiClemente & Norcross, 1992). In Table 1, we

have arranged the systems characteristics described

earlier as a series of developmental tasks (Florin,

Mitchell & Stevenson, 1993). This framework has

guided e�orts to assess training and technical assist-

ance needs of coalitions, including the development of

data-based pro®les suggesting speci®c technical assist-

ance strategies for improving coalition processes

(Dugan, 1996; Florin et al., 1993).2 However, there has

been little empirical research linking coalition charac-

teristics with intermediate outcomes (Goodman, Steck-

ler & Kegler, 1997; Kegler et al., 1998; McMillan,

Florin, Stevenson, Kerman & Mitchell, 1995). Butter-

foss, Goodman and Wandersman (1996), for example,

found that characteristics of coalition structure (e.g.,

shared decision-making, organizational climate) were

related to member participation, but not to the inter-

mediate outcome of the quality of coalition plans.

Technical assistance e�orts with coalitions for the

most part remain intuitive rather than data based.

Fig. 1. An organizational systems view of a coalition.

1 Although the ``systems model'' may appear linear, we recog-

nize the existence of multiple ``feedback loops'' and interactive in-

¯uences. For example, a coalition that does a good job at

increasing member skills may increase member commitment and

thereby generate new inputs (i.e., volunteers and additional fund-

ing).2 Although the developmental tasks are presented in a linear

fashion, a truer picture would be a spiral, with increasing e�ec-

tiveness as coalition tasks are revisited and successfully nego-

tiated. At any one time, however, certain developmental tasks

may be particularly salient.

P. Florin et al. / Evaluation and Program Planning 23 (2000) 341±346342

In this study, we examine whether success in accom-plishing initial tasks (e.g., mobilization, establishing or-ganizational structure, building capacity for action) isassociated with success in accomplishing subsequentintermediate outcomes (i.e., better planning; moree�ective implementation of programming).

3. Method

In 1988, Rhode Island passed legislation to promotecomprehensive substance abuse prevention program-ming through the formation of broadly representativemunicipal coalitions or ``task forces''. Funds were pro-vided annually to each coalition proportionate to itscommunity's population to initiate comprehensive pre-vention programming. Communities varied in theextent to which they had existing groups concernedwith substance abuse prevention issues and in thespeed with which they actually began developing thesecoalitions. Many coalitions were still negotiating ``in-itial developmental tasks'' when we gathered this datain 1994 and 1995 as part of a federal grant to establishsupport mechanisms for local coalitions. This provideda unique opportunity to examine coalition processeswith a sample relatively homogeneous in organiz-ational structure, perceived mandate, and formal start-up date.

We used a multi-method research design thatincluded a mailed survey to coalition members�n � 350), in-person interviews with coalition leaders

(coordinators) �n � 35), telephone interviews with keycommunity informants �n � 105), and examination ofarchival documents, including prevention plans sub-mitted by the coalitions to the RI Division of Sub-stance Abuse. We used the framework in Table 1 toguide the selection of ``marker'' variables indicatinghow well coalitions negotiated these developmentaltasks.

3.1. Input resource variables

Annual funding is the 1994 funding level awarded toeach coalition, as indicated in archival records.

Structured interviews were conducted during sum-mer 1994 with a coalition leader (usually the paid co-alition coordinator, but sometimes the chair of thecoalition) to assess the following dimensions: resourcesavailable, structure (e.g., o�cers, committees, formalprocedures), functions (e.g., decision making, recruit-ment), and linkages with other community organiz-ations.

Paid sta� hours is the average number of paid sta�hours per week for a coordinator, derived from theleader interview.

3.2. Initial mobilization

Number of members attending coalition meetings isthe average number of members attending coalitionmeetings within the last year, as estimated by the co-alition leaders.

Table 1

Coalition developmental tasks

Developmental task Speci®c examples

Initial mobilization Recruit critical mass of active participants

Engage key community constituencies or sectors

Establish organizational structure Establish structure that clari®es roles and procedures

Adequately address task and maintenance functions

Build capacity for action Member-level capacity: orient members to concepts and provide skill building

Organizational level capacity: establish inter-organizational linkages with key community constituencies

Planning for action Assess needs as perceived by community constituencies; prioritize and clearly state coalition goals and

objectives

Select an array of intervention strategies based upon literature about program e�ectiveness

Develop a sequential work plan that sets timelines, allocates resources and assigns responsibilities

Implementation Implement the array of intervention strategies

Engage key organizational players, networks and broad citizen participation in implementation

Re®nement Use evaluation data for speci®c program re®nements that incorporates community reactions

Identify gaps in programming and add strategies that build toward a comprehensive and coordinated

array of programming strategies across community sectors

Institutionalization Member-level institutionalization: processes for leader succession and recruitment of new members

Organizational-level institutionalization: integrate functions into ongoing missions of existing

organizations

P. Florin et al. / Evaluation and Program Planning 23 (2000) 341±346 343

Members serving on subcommittees is the percent ofmembers serving on subcommittees, as estimated bythe coalition leader.

3.3. Establishing organizational structure

Formalization is a count of 11 di�erent dimensionsof formalized rules and procedures (e.g., writtenagendas at meetings, by-laws specifying decision-mak-ing procedures) as reported by coalition leaders.

For an additional perspective on coalition organiz-ational structure, we conducted a mail survey in thesummer of 1994 of all current coalition members �n �830� to assess participation level, prevention knowledge,and social climate. Approximately 41% of members,and 69% of ``active members'' (i.e., members who hadattended half or more of the coalition's meetings)responded. The respondents were primarily male (64%),Caucasian (90%), and employed full time (71%). Wechose ``task focus'' and ``skill development'' as import-ant indicators of coalition group process.

Task-focused group climate is a 5-item social climatescale �a � :84). Members rated their agreements withstatements about order and organization (e.g., ``the co-alition was disorganized and ine�cient''). Individualscores were aggregated into a single score for each co-alition.

3.4. Building capacity for action

Perceived knowledge and skill development is a 7-itemscale �a � :91� in which members rated the extent towhich participation had increased their knowledge,beliefs, and skills (1 = no change, 4 = majorincrease). Individual scores were aggregated into asingle score for each coalition.

Inter-organizational linkages of the coalition is thecoalition leader's report of the extent of coalition con-tact during the past twelve months (on a scale from 0= ``none'' to 3 = ``weekly or more'') with each of 12di�erent community constituencies (e.g., elementaryschools, business community, police, and so on). Asingle linkage score was created for each coalition.

The correlations among these predictor variableswere low (an average r � :16), in part because wedeliberately avoided highly overlapping variables.Funding level, sta� hours, and interorganizationallinkage variables were signi®cantly correlated witheach other. One explanation is that each is signi®cantlyassociated with community size. Consistent with theframework proposed here, task focused social climatewas positively and signi®cantly related to members'knowledge and skill development.

3.5. Planning for action

To assess the impact of input resources, mobilizationand capacity building on the planning process, weselected a data source independent of both coalitionleader and member self-report: annual planning docu-ments required of coalitions in summer 1994 to be eli-gible for continued state funding.

Plan quality is a 3-item scale in which the clarity,e�ectiveness, and quality of plans were rated (1 =``poor''; 5 = ``outstanding'') (Cronbach's a � :94).Across three years of ratings, average interrater re-liability was adequate �a � :76).

Scope is the number of categorically di�erent pre-vention strategies proposed by each coalition in itsannual plan. Activities were classi®ed into one of 7major categories (e.g., awareness, skill building, policychange), the total number of di�erent kinds of strat-egies used was calculated (1±7 total). Independentraters showed adequate interrater reliability in usingthe typology (Cohen's k � :65� (see Mitchell, Steven-son & Florin, 1996).

3.6. Implementation e�ects

To assess the impact of input resources, mobiliz-ation, and capacity building on implementation, weagain selected a data source independent of both co-alition leader and member self-report: key informantsin the communities served by the coalitions. In summer1995, we conducted structured telephone interviewswith the top elected o�cial, chief of police, and super-intendent of schools in the municipalities served by thecoalitions. We sought independent con®rmation of co-alition visibility, acceptability, and perceived impact.Of the 114 targeted key informants, 106 responded(93% response rate). Data were aggregated into aver-age responses per coalition �n � 35).

Implementation e�ects is a 5-item scale �a � :87� inwhich key informants rated the e�ects of the coalition(none, some, moderate, or a great deal) on severaldimensions of community life that coalitions expectedto in¯uence (e.g., youth and parents, community atti-tudes about alcohol and other drug problems, connec-tions between agencies, policies of organizations, andresources available for prevention).

4. Findings

Table 2 presents intercorrelations among predictorand intermediate outcome variables.

4.1. Plan quality

Only two signi®cant associations involved plan qual-

P. Florin et al. / Evaluation and Program Planning 23 (2000) 341±346344

ity. Coalitions with greater paid sta� hours �r � :35,p < :05� and with more members attending meetings�r � :41, p < :05� had more highly rated plans.

4.2. Scope of prevention activities

There were no signi®cant relationships between any ofthe predictor variables and scope of prevention strat-egies. Nor were there any signi®cant relationshipsamong scope, plan quality, and implementation ratings.

4.3. Implementation ratings

Three measures of coalition processes in 1994 weresigni®cantly related to key informant-judged im-plementation e�ects in 1995. These included task-focused social climate �r � :38, p < :05), perceivedmember knowledge and skill development �r � :50,p < :01), and inter-organizational linkages �r � :48,p < :01).

These three variables from 1994 were combined in ahierarchic multiple regression to predict more success-ful implementation in 1995. Variables were entered inorder of developmental task, with task focus forced in®rst and followed by skill development and linkagesentered simultaneously. At the ®nal step of this analy-sis, both skills and linkages contributed signi®cantly tothe prediction of later success, with a multiple R of 63and an adjusted R 2 of .33 �p < :002).

5. Conclusions

Coalitions that ``take care of business'' in terms ofbuilding their own capacity are more likely to beviewed as producing community change. Speci®cally,coalitions that had done a better job in developing atask-focused social climate, in increasing their mem-bers' perceived skills, and in making more extensive

linkages with community organizations were morelikely to be rated by community leaders as producinge�ects (i.e., resources devoted to prevention, connec-tions between organizations, community attitudes con-cerning alcohol and other drugs; policies oforganizations). This ®nding is all the more strikingbecause of the diversity among coalitions in their tar-get populations and activities, and the potential``noise'' in our measurements. We expected that ouroutcomes would be relevant for much of what co-alitions did, while realizing that they would not com-pletely overlap with each coalition's idiosyncraticobjectives. Nonetheless, success in accomplishing ear-lier developmental tasks accounted for 33% of the var-iance in these e�ects.

One limitation is the self-report nature of this data.Our con®dence in the robustness of these relationshipsis bolstered, however, by the multiple methods used tocollect information. Descriptions of coalition processescame from coalition leaders and members, while rat-ings of implementation success came from multiple keyinformants in each community. Overall, these resultssupport arguments for increasing attention to how co-alitions might in¯uence overall levels of community ca-pacity (Goodman et al., 1998).

However, not all our hypotheses about coalitionprocesses were con®rmed. The lack of relationshipsbetween coalition factors and planning products wasconsistent with some previous work (Butterfoss et al.,1996). Only two of our predictor variables (i.e., paidhours from coordinators, and number of membersattending meetings) were related to the quality ofplans. It may be as Kegler et al. (1998) suggested, that``developing high-quality action plans may be a pro-fessional skill'' (p. 350) that is less in¯uenced by theoverall climate of the coalition and participation of co-alition members. Professional skills among a smallnumber of people actually developing a plan mightwell in¯uence plan ratings.

Table 2

Intercorrelations between initial developmental tasks and planning and implementation variablesa

Plan quality Scope of strategies Implementation e�ects

Input resources

Annual funding .28 ÿ.09 ÿ.02Paid sta� hours .35� .12 .33

Initial mobilization

Number of members attending meetings .41� ÿ.02 .04

Percentage of members in subcommittees ÿ.33 .10 .16

Establishing organizational structure

Formalization .12 .01 ÿ.03Task-focused social climate ÿ.10 .08 .38�

Building capacity for action

Perceived knowledge and skill development ÿ.19 .20 .50��

Inter-organizational linkages .27 .11 .48��

a For all correlations, n � 35.

P. Florin et al. / Evaluation and Program Planning 23 (2000) 341±346 345

The lack of a relationship between the plan variablesand implementation e�ects is puzzling. From a theor-etical standpoint, a central argument for coalitions isthat multiple constituencies will increase the compre-hensiveness of prevention programming (i.e., multiplestrategies targeted at multiple settings) which will thenresult in greater impact. Perhaps our scope variablewas too gross a measure of comprehensiveness. AsMitchell et al. (1996) suggest, use of a broad array ofstrategies does not by itself guarantee that these strat-egies are used in a coherent and organized way. If aprevention plan is not coherently focused around somecommon target populations (e.g., junior high schoolyouth) or speci®c outcomes (e.g., tobacco, alcohol andmarijuana as gateway drugs), it may have little impacteven if it uses a broad array of strategies. Given theimportance often attached to coalition planninge�orts, it seems important for future work to under-stand better the relationships among participationplanning processes, the quality of the ®nished plan-ning, and its relationship to implementation.

Too little attention is given to articulating and test-ing the implicit ``treatment'' theories underlying e�ortsto implement and institutionalize community-basedhealth prevention interventions. This study attempts todevelop an empirically testable model for identifyingand responding to the needs of community-based pre-vention coalitions. Although in need of replication, theresults suggest that coalitions establishing an e�ectiveorganizational structure and building both individualand organizational capacity are perceived as ®eldingmore e�ective community interventions.

Acknowledgements

This work was completed while the authors were as-sociated with the Center for Alcohol & Addiction Stu-dies, Brown University, and was supported in part bya grant from the Center for Substance Abuse Preven-tion (H86 SP 04426-01). We thank Kristine Nilsen andother members of the Community Research and Ser-vices Team for their dedication to the evaluation pro-cess. We thank members of the RI task forces and theCUP coalition for their cooperation in this work.

An earlier version of this work was presented at theAmerican Psychological Association meeting, SanFrancisco, CA; August 1998.

Correspondence concerning this work should be sentto Paul Florin, Department of Psychology, Universityof Rhode Island, Kingston, Rhode Island, 02881.

References

Butterfoss, F. D., Goodman, R. M., & Wandersman, A. (1996).

Community coalitions for prevention and health promotion: fac-

tors predicting satisfaction, participation, and planning. Health

Education Quarterly, 23(1), 65±79.

Dugan, M. A. (1996). Participatory and empowerment evaluation:

lessons learned in training and technical assistance. In D.

Fetterman, S. Kafterian, & A. Wandersman, Empowerment evalu-

ation: Knowledge and tools for self-assessment and accountability.

Beverly Hills, CA: Sage.

Florin, P., Chavis, D., Wandersman, A., & Rich, R. (1992). A sys-

tems approach to understanding and enhancing grassroots organ-

izations: the block booster project. In R. Levine, & H. Fitzgerald,

Analysis of dynamic psychological systems (pp. 215±243). New

York: Plenum.

Florin, P., Mitchell, R., & Stevenson, J. (1993). Identifying training

and technical assistance needs in coalitions: a developmental

approach. Health education research: Theory and practice, 8, 417±

432.

Goodman, R., Speers, M., McLeroy, K., Fawcett, S., Kegler, M.,

Parker, E., Smith, S., Stering, T., & Wallerstein, N. (1998). An

initial attempt at identifying and de®ning the dimensions of com-

munity capacity to provide a basis for measurement. Health

Education and Behavior, 25(3), 258±278.

Goodman, R. M., Steckler, A., & Kegler, M. (1997). Mobilizing or-

ganizations for health enhancement: theories of organizational

change. In B. K. Rimer, Health behavior and health education:

Theory, research, and practice (2nd ed.). San Francisco, CA:

Jossey-Bass.

Katz, D., & Kahn, R. L. (1978). The social psychology of organiz-

ations (2nd ed.). New York: Wiley.

Kegler, M. C., Steckler, A., McLeroy, K., & Malek, S. H. (1998).

Factors that contribute to e�ective community health promotion

coalitions: a study of 10 Project ASSIST coalitions in North

Carolina. Health Education and Behavior, 25(3), 338±353.

Mayer, J. P., Soweid, R., Dabney, S., Brownson, C., Goodman, R.

M., & Brownson, R. C. (1998). Practices of successful community

coalitions: a multiple case study. American Journal of Health

Behavior, 22(5), 368±369.

McMillan, B., Florin, P., Stevenson, J., Kerman, B., & Mitchell, R.

E. (1995). Empowerment praxis in community coalitions.

American Journal of Community Psychology, 23(5), 699±727.

Mitchell, R. E., Stevenson, J. F., & Florin, P. (1996). A typology of

prevention activities: applications to community coalitions.

Journal of Primary Prevention, 16(4), 413±436.

Mizrahi, T., & Rosenthal, B. B. (1992). Managing dynamic tension

in social change coalitions. In T. Mizrahi, & J. D. Morrison,

Community organization and social administration (pp. 11±40).

Binghamton, NY: Haworth Press.

Prochaska, J. O., DiClemente, C. C., & Norcross, J. C. (1992). In

search of how people change: applications to addictive behaviors.

American Psychologist, 47, 1102±1124.

Saxe, L., Reber, E., Hallfors, D., Kadushin, C., Jones, D., &

Rindskopf, D. (1997). Think globally, act locally: assessing the

impact of community-based substance abuse prevention.

Evaluation and Program Planning, 20(3), 357±366.

Weiss, C. H. (1995). Nothing as practical as a good theory: exploring

theory-based evaluation for comprehensive community initiatives

for children and families. In J. P. Connell, A. C. Kubisch, L. B.

Schorr, & C. H. Weiss, New approaches to evaluating community

initiatives: Concepts, methods, and contexts (pp. 65±92).

Washington, DC: The Aspen Institute.

Yin, R. K., Kaftarian, S. J., Yu, P., & Jansen, M. A. (1997).

Outcomes from CSAP's community partnership program: ®nd-

ings from the national cross-site evaluation. Evaluation and

Program Planning, 20(3), 345±355.

P. Florin et al. / Evaluation and Program Planning 23 (2000) 341±346346