Expanding Advocacy Capacity: Findings from the Evaluation of

The California Endowment Clinic Consortia Policy and Advocacy Program

Prepared by

Annette L. Gardner, PhD, MPH, Sara Geierstanger, MPH and

Claire Brindis, Dr PH

The Philip R. Lee Institute for Health Policy Studies

University of California, San Francisco

October 28, 2008

Workshop Objectives

• Describe approaches to assessing policy and advocacy activities funded under TCE Program

• Share findings for 4 of 6 Program outcomes

• Discuss theoretical and practical challenges to assessing policy and advocacy

Background:Clinic Consortia Policy and Advocacy Program

• Launched in 2001 and refunded two more times in 2004 and 2007 ($9M per Round)

• 14 local/regional clinic consortia and 4 statewide clinic associations representing 700+ primary care clinics funded by The California Endowment

• Goals:

– Increase services to underserved populations through consortia-initiated policy and advocacy activities

– Increase access through improved efficiencies within clinic organizations

Program Logic Model

Increased grantee capacity in policy advocacy

Increased policymaker awareness of safety net and clinic policy

issues

Increased policymaker support for clinic funding

Strengthened clinic operations

Increased services for the underserved and uninsured

Improved health outcomes for targeted communities and

populations

Increased grantee capacity in policy advocacy

Increased policymaker awareness of safety net and clinic policy

issues

Increased policymaker support for clinic funding

Strengthened clinic operations

Increased services for the underserved and uninsured

Improved health outcomes for targeted communities and

populations

Evaluation Data Collection Activities: 2002-2009

Round Areas of Investigation

Round 1 (2001-2003)

Funding secured (2001 - 2009)Policies targeted (2001 - 2009)Effectiveness of advocacy activities (2001 - 2006)Partnerships with clinics and consortia (2002 - 2003)Activities to strengthen clinic operations (2002, 2003)Media perceptions of media advocacy (2003)Clinic perceptions of consortia activities (2003)Grantee Implementation and Sustainability Survey (2003)Policymaker/Stakeholder awareness of consortia advocacy (2003, 2004)

Round 2 (2004 - 2006)

Newspaper coverage secured (2004 and 2005)Individual grantee indicator data (2004 and 2005)Partnerships with non-health partners (2004 - 2009)Perceptions of consortia policy advocacy by non-member clinics (2005)Media perceptions of media advocacy (2005)Clinic perceptions of grantee advocacy (2005)Three policy advocacy case studies (2006)Grantee Best Practices - achievement of Program outcomes (2006)

Round 3 (2007 - 2009)

Tracking policy achievements - comparison of 3 key policies (2007-2009)Case studies to assess longer-term Program outcomes (2009)

Evaluation Data Collection Activities by Outcome

Outcome 1: Increased grantee

advocacy capacity

Outcome 2: Increased

policymaker awareness

Outcome 3:

Increased policymaker

support for clinic funding

Outcome 4 Strengthened

clinic operations

Grantee Implementation and Sustainability Survey

2003 Policymaker and 2004 Stakeholder Surveys

Annual Funding Secured Worksheet

2004 and 2006 Clinic Focus Groups

Annual Advocacy Activities Worksheet

2003 and 2005 Interviews with Media and Media Consultants

Annual Policy Outcomes Worksheet

16 Grantee Best Practices

Partnerships Among Clinics, Among Consortia, and with Non-Health Allies

Content Analysis of Newspaper Coverage Secured

3 Policy Case Studies

Outcome 1: Expanded Grantee Advocacy Capacity

Data Collection Activity: Annual Advocacy Activities Worksheet

• Grantee self-report (n=15 grantees)• Previous year’s activities and their outcomes (n=13)• Source: Scholzman and Tierney (1986)

Activity Federal

(Y/N)

State (Y/N)

Local (Y/N)

Effectiveness (1 to 4, 4=

“very effective”

Outcome 1: Increased

clinic funding(Y/N)

Outcome 2: Achieve a

policy change(Y/N)

Outcome 3: Increased

policymaker awareness

(Y/N)

Media advocacy

Contact policymakers directly

Analysis:

1) Determine percent activities at the federal, state and local levels, and overall

2) Calculate effectiveness overall and by activity

3) Determine percent of activities that achieved each Outcome

4) Sort by grantee age (pre/post 1990 launch)

Informal contacts with elected officials

Hosting clinic tours for policymakers

Helping draft regulations, rules or guidelines

Results: Expanding and Innovating• Grantees engaged in more grant-funded activities over

time--increase from 73% to 86% of surveyed activities, between 2002 and 2006.

• Grantees consistently targeted most activities (80%) at the local and state levels and less (60%) at the federal level.

• Grantees perceived some activities to be more effective than others, e.g., helping draft rules, regulations and guidelines

• Newer (post-1990) grantees were equally effective in achieving advocacy goals by 2006.

Increase in Number of Grantee Advocacy Activities, by Level of Membership Focus, 2002-2006

0

20

40

60

80

100

120

140

160

Statewideconsortia (n=3)

Regional consortia(n= 4)

Local consortia(n= 8)

Statewide consortia(n=3)

18 19 27 24 23

Regional consortia (n= 4) 33 34 32 35 36

Local consortia (n= 8) 69 75 78 77 83

2002 2003 2004 2005 2006

Changes in Perceived Effectiveness of Key Activities Since 2002 (4=“very effective”)

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

Drafting rules,regulations

Engaging in informalcontacts withpolicymakers

Media advocacy

Drafting rules,regulations

3.14 3.60 3.60 3.70 3.50

Engaging ininformal contactswith policymakers

3.58 3.23 3.23 3.00 3.00

Media advocacy 3.31 3.25 2.57 2.57 2.75

2002 2003 2004 2005 2006

Percent of Activities That Resulted in a Policy Change, 2002-2006

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

2002 2003 2004 2005 2006

Pre-1990 Grantees(n=6)

Post-1990 Grantees(n=9)

Percent of Grantee Advocacy Activities That Achieved 3 Desired Outcomes, 2002-2006

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2002 2003 2004 2005 2006

Policymaker Awareness

Policy Change

Clinic Funding

Outcome 2:Increased policymaker awareness of

safety net and clinic policy issues

Data Collection Activities:2003 Policymaker and 2004 Stakeholder Surveys

• 2003 Policymaker Survey

– N = 86 decision makers familiar with consortia and their activities and policy issues

• 2004 Stakeholder Survey

– N = 43 stakeholders less familiar with consortia and their activities

Not at all familiar

A little familiar Somewhat familiar

Very familiar

How familiar are you with

The consortium?

Activity 1?

Yes No Not sure

Since 2001…

Has the consortium increased you awareness of…

Analysis:

1) Determine percents

Do you recall seeing….

Results:Policymaker/Stakeholder Awareness

• 2003 policymakers well-known to grantees considered policy events “very effective” (74%) compared to 2004 stakeholders less known to grantees (61%)

• Consortia media events rated “very effective” in increasing 2003 policymaker (60%) and 2004 stakeholder awareness (42%)

• Grantees reported that nearly all advocacy activities (95%) contributed to high policymaker awareness

Outcome 3:Increased Policymaker Support

for Clinic Funding

Data Collection Activity: Funding Secured Worksheet

• Grantee self-report (n=16-18 grantees)– Requires written instructions and follow-up phone calls

• Previous year’s funding secured attributable to TCE grant– Staff time (%) used to determine esimate– Use calendar year

Funding Source

$ Amount

How secured?

Write a grant?

New or Maintained funding?

Estimate the % due

to TCE grant

How much went to

consortium?

How much

went to clinics?

Federal

1. Clinic 330 grants

Analysis:

1. Calculate total funding, funding attributable to the grant, funding allocated to consortia and clinics

2. Code and county funding type - federal, state, local, private

3. Code and categorize data on how funding was secured and what funding was spent on by consortium and clinics

4. Determine whether new or maintained (carryover) funding

State

1. Tobacco Settlement $

Local

1. County contracts

Private

Results:Increased Clinic Financial Stability, 2001-2007

• Total funding secured was $2.9B, with $979M (35%) attributable to grant

• Funding due to TCE grant increased from $104M in 2001 to $226M in 2007

• Modest increase in “new” funding (29% in 2007), e.g., Information technology

• Public funding dominates:

– Federal: $344M

– County: $324M

– State: $263M

– Private: $45M

Total Funds Attributed to Grant-funded Advocacy Activities, 2001-2007

$0

$50,000,000

$100,000,000

$150,000,000

$200,000,000

$250,000,000

Total

Consortia

Total $103,993,752 $81,203,585 $84,988,466 $105,047,411 $195,028,986 $182,416,735 $226,671,718

Consortia $4,880,660 $4,721,917 $6,281,764 $9,838,452 $6,784,201 $9,219,742 $9,856,642

2001 2002 2003 2004 2005 2006 2007

Annual Funding Secured by Type, 2001-2007

$0

$10,000,000

$20,000,000

$30,000,000

$40,000,000

$50,000,000

$60,000,000

$70,000,000

$80,000,000

$90,000,000

$100,000,000

2001 2002 2003 2004 2005 2006 2007

Federal

State

Local

Private

Total Funding by Type, 2001-2007

$344,754,414

$324,438,677

$263,286,750

$45,204,640

$0

$50,000,000

$100,000,000

$150,000,000

$200,000,000

$250,000,000

$300,000,000

$350,000,000

$400,000,000

Total Funding

FederalLocalStatePrivate

Data Collection Activity: Policy Outcomes Worksheet

• Grantee self-report (n=18 grantees)• Previous year’s activities

Policy targeted

Bill Number and/or title?

Grant-funded policy advocacy activities undertaken in 2007. Choose from list: a) media advocacy; b) in-person meetings with decision makers; c) drafting of rules, regulations; d) materials development; e) grass roots activities; f) Other: list

Target populations to be affected by policy. Choose from list: a) clinics; b) uninsured; c) low-income populations; d). clinic consortia; e) other special populations; f) Other: list

Outcomes as of 12/07. Choose from list: a) Passed - signed in to law; b) Vetoed; c) Pending - still being considered by decision makers; d) funding secured; e) Other: list.

Federal

Analysis:

1. Code and categorize all policies

2. Count number of grantees that mentioned each policy

3. Count federal and state legislation

4. Code and categorize advocacy activities, target populations, and outcomes

State

Local

Results:Legislation Passed, 2001-2006

• Grantees achieved fewer wins but were involved with more legislation over time:

– 2001-2003: Involved in 58 pieces of legislation. Five (30%) federal policies passed and 26 (60%) state bills passed

– 2004-2006: Involved in 72 pieces of legislation. Six (18%) federal policies passed and 20 (55%) state bills passed.

Federal and State Legislation Targeted by Grantees: Round 1 (2001-03) vs. Round 2 (2004-06)

5 6

26

20

0

5

10

15

20

25

30

2001-03 (N=58) 2004-06 (N=72)

Fed - Passed

State- Passed

Major Policy Wins & Losses in 2006

• Federal:

– Wins: HR 5647--Health Center Appropriations

– Losses: HR 4241--Deficit Reduction Act

• State:

– Wins: SB 437--Children’s Health Coverage Expansion

– Wins: SB 2282--Safe Harbor

– Losses:SB 1427--Four Walls

– Losses:Prop 86--Tobacco Initiative

• Local:

– Wins: Mental Health Services Act (MHSA) funding, County Contracts

Methods:Developing Policy Case Studies

• Criteria for inclusion:– 1 state and 1 local policy

– Single grantee vs. multiple grantee involvement

• Policies studied:– Measure A (Alameda County)

• Local sales tax to support health care safety net

– Mental Health Services Act (MHSA)• Local funding secured for clinics in Alameda and San

Diego Counties

– State Prospective Payment System (PPS) legislation

• Move from cost-based payment to prospective payment of FQHC clinic services

Methods:Developing Policy Case Studies (cont.)

• Data collection, development and analysis:

– 30-60 minute phone interviews with:

• Grantee staff

• 1-2 decision makers per policy

• 1 clinic staff member per policy

– UCSF developed 5-7 page description of policy, advocacy activities undertaken and outcomes

– Grantees reviewed and commented on summary

– Comparison of 3 policies by advocacy activities

Analysis: Comparison of Policy Case Studies

• Variation among cases:

– Not all policies are “media-friendly”

– Challenges vary, e.g., measures requires 2/3 majority

• Common factors for success:

– Staff expertise (technical, advocacy, networking)

– Early and often involvement

– Mobilizing clinic staff and patients and building coalitions

– Leveraging partnerships with clinics

Outcome 4: Strengthened Clinic Operations

Methods:Developing Grantee Best Practices

• Criteria for inclusion:

– Exemplary activities funded under TCE Program (2004-2006)

• Types of practices:

– A particular bill or measure

– An advocacy activity or strategy

– Programs to expand access to health care

• Data collection, development and analysis:

– UCSF reviewed background materials and interviewed grantee staff

– UCSF wrote a 1-2 page description

– Grantees reviewed and commented on description

– UCSF mapped Best Practices on Program logic model

Community Clinic Association of L.A. County (CCALAC)Preserving Public-Private Partnership (PPP) Program Funding

• The Challenge:– Sustain funding for program that supports links between public and

private primary care clinics that serves 1.3M people

• The Solution:– Multi-faceted advocacy campaign to preserve and evaluate Program

• Success:– Preservation of funding - 3 year contract ($51M in 2005)

• Lessons Learned:– Work with local data

– Partnerships key

– Garner county support

• Looking Forward: – Focus on maintaining county support

Achievement of Program Outcomes:Round 2 Grantee Best Practices

Increased grantee capacity in policy and advocacy• SCCC: Creating a Voice for Change--Launching a Clinic Consortium•NSRHN: Use of Economic Data to Inform Local Decision Making

Increased grantee capacity in policy and advocacy• SCCC: Creating a Voice for Change--Launching a Clinic Consortium•NSRHN: Use of Economic Data to Inform Local Decision Making

Increased awareness of policymakers of safety net and clinic policy issues•CCC: “What’s Going Around?” on CBS Local Channel 8•COCCC: 6th Annual Bridges Conference

Increased awareness of policymakers of safety net and clinic policy issues•CCC: “What’s Going Around?” on CBS Local Channel 8•COCCC: 6th Annual Bridges Conference

Increased policymaker support for clinic funding•CFHC: Averting Cuts to Title X Funding•CPPEF: Training and Coordinating Affiliates in Grassroots Policy Advocacy•CRIHB: Develop Research Infrastructure to Inform Policymaking

Increased policymaker support for clinic funding•CFHC: Averting Cuts to Title X Funding•CPPEF: Training and Coordinating Affiliates in Grassroots Policy Advocacy•CRIHB: Develop Research Infrastructure to Inform Policymaking

Strengthened clinic operations•CPCA: Clinic Emergency Preparedness Project•CHPSCC: Patient Advocacy Training Program

Strengthened clinic operations•CPCA: Clinic Emergency Preparedness Project•CHPSCC: Patient Advocacy Training Program

Increased services for the underserved/uninsured•NCCN: Securing Health Professional Shortage Area Designations•RCHC: Expanding Enrollment in Health Care Insurance Programs

Increased services for the underserved/uninsured•NCCN: Securing Health Professional Shortage Area Designations•RCHC: Expanding Enrollment in Health Care Insurance Programs

Improved health outcomes for targeted communities and populations - TBD

Improved health outcomes for targeted communities and populations - TBD

Summary of Findings

• Achievement of first 4 Program Outcomes:

– Sustained advocacy capacity

– Increased policymaker/stakeholder awareness

– Increased funding secured

– Important policy “wins” benefiting clinics

• Early evidence of longer-term outcomes:

– Clinic services expansions

– Increased utilization by uninsured

– Consortia as agents of systems change in their communities

Challenges and Potential Solutions

• Aggregating diverse grantee achievements– Use of longitudinal tools and individual case studies

• Issue of lobbying– Identify what is funded under the grant

– Analyze legislation as measure of capacity

– Acknowledge role of lobbying

• Relying on grantee self-report– Interview member/non-member clinics

– Interview policymakers/stakeholders

– Interview partner organizations

• Attribution– Focus on what can be identified as attributable

For More Information:

Annette Gardner, PhD, MPH

Evaluation Director

Philip R. Lee Institute for Health Studies

University of California, San Francisco

3333 California Street, Suite 265

San Francisco, CA 94121

Phone: (415) 514-1543

Email: annette.gardner@ucsf.edu