Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for...
-
Upload
philip-r-lee-institute-for-health-policy-studies -
Category
Documents
-
view
198 -
download
0
description
Transcript of Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for...
![Page 1: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/1.jpg)
Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program:
Lessons for Evaluators
Annette L. Gardner, PhD, MPHSara Geierstanger, MPHClaire Brindis, DrPHPhilip R. Lee Institute for Health Policy StudiesUniversity of California, San FranciscoNovember 4, 2011
SF
University of California San Francisco
![Page 2: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/2.jpg)
Presentation Objectives
UCSF evaluation design and findings by outcomeMonitoring advocacyAssessing impact
Relevance of findings for researchers/evaluators, advocates and funders
![Page 3: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/3.jpg)
The California Endowment Clinic Consortia Policy and Advocacy
Program
Component of Community Clinics Initiative (CCI)
Launched in 2001 and refunded two more Rounds in 2004 and 2007, an investment of $28 million
14 local/regional clinic consortia and 5 statewide clinic associations representing 800+ primary care clinics
Goals:1) Increase services to underserved populations2) Increase access through improved clinic efficiencies
![Page 4: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/4.jpg)
Theory of Change:“Resources and staffing dedicated to policy advocacy and TA will increase the collective influence of clinics and strengthen a broad base of long-term support of
clinic policy issues.”
![Page 5: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/5.jpg)
Grantees – 18 State, Regional, Clinic Consortia
Alameda Health Consortium (AHC)Alliance for Rural Community Health (ARCH)California Family Health Council (CFHC)California Planned Parenthood Education Fund (CPPEF)California Primary Care Association (CPCA)California Rural Indian Health Board (CRIHB)Capitol Community Health Network (CCHN)Central Valley Health Network (CVHN)Coalition of Orange County Community Clinics (COCCC)Community Clinic Association of Los Angeles County (CCALAC)Community Clinic Consortium Serving Contra Costa and Solano Counties (3C)Community Health Partnership, Inc. (CHP)Council of Community Clinics (CCC)North Coast Clinics Network (NCCN)Northern Sierra Rural Health Network (NSRHN)Redwood Community Health Coalition (RCHC)San Francisco Community Clinic Consortium (SFCCC)Shasta Consortium of Community Health Centers (SCCHC)
Alameda Health Consortium (AHC)Alliance for Rural Community Health (ARCH)California Family Health Council (CFHC)California Planned Parenthood Education Fund (CPPEF)California Primary Care Association (CPCA)California Rural Indian Health Board (CRIHB)Capitol Community Health Network (CCHN)Central Valley Health Network (CVHN)Coalition of Orange County Community Clinics (COCCC)Community Clinic Association of Los Angeles County (CCALAC)Community Clinic Consortium Serving Contra Costa and Solano Counties (3C)Community Health Partnership, Inc. (CHP)Council of Community Clinics (CCC)North Coast Clinics Network (NCCN)Northern Sierra Rural Health Network (NSRHN)Redwood Community Health Coalition (RCHC)San Francisco Community Clinic Consortium (SFCCC)Shasta Consortium of Community Health Centers (SCCHC)
![Page 6: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/6.jpg)
The California Endowment Clinic Consortia Policy and Advocacy
Program
Advocacy capacity
Technical Assistance
Partnerships
2001 - 03 2004 - 06 2007 - 09
![Page 7: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/7.jpg)
Evaluation Design
Methods
EvaluationTA
Evaluation Partnership
2001 - 03 2004 - 06 2007 - 09
![Page 8: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/8.jpg)
Policy Change Logic Model: Measures and Tools
Expandadvocacy capacity
Increase policymaker awareness And supportfor clinic policyIssues, funding
Strengthenedclinic operations
Increasedservices forunderservedanduninsured
Improvedhealth Outcomes
* Annual Advocacy Activities Worksheet* Annual Grantee Interviews* Sustainability Survey
* Decision Maker Awareness Surveys (2)* Interviews with Media Representatives (2)* Policy Outcomes Wrksht* Funding Secured Wrksht
* Clinic Focus Groups (2)* Grantee Case Studies (17)
* Policy Case Studies (3) * Grantee Case Studies (17)
* Policy Case Studies (3) * Grantee Case Studies (17)
Outcome 1 Outcomes 2, 3 Outcome 4 Outcome 5 Outcome 6
![Page 9: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/9.jpg)
Outcome 1: Expand Advocacy Capacity
Organizational capacity gains:StaffingAbility to maintain advocacyTechnical acumen
Advocacy activities:Strategically deploy advocacy “toolkit”, e.g.,
Working with the mediaEducating/working with elected officials pre/post
passagePursuing/leveraging partnerships with stakeholdersMobilizing grassroots advocates
![Page 10: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/10.jpg)
Tool: Annual Assessment of Advocacy Capacity and Outcomes
13 grant- funded
activities
Federal (Y/N)
State (Y/N)
Local (Y/N)
Effectiveness (1 to
4, 4= “very effective”
Outcome 1: Increased
clinic funding
(Y/N)
Outcome 2: Achieve a
policy change
(Y/N)
Outcome 3: Increased
policymaker awareness
(Y/N)
Media advocacy
Contact policymakers directly
Analysis:
1) Determine percent activities at the federal, state and local levels, and overall
2) Calculate effectiveness overall and by activity
3) Determine percent of activities that achieved each Outcome
4) Sort by grantee age (pre/post 1990 launch)
Informal contacts with elected officials
Hosting clinic tours for policymakers
Helping draft regulations, rules
![Page 11: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/11.jpg)
Findings: Grantee Advocacy Capacity, 2002 to 2006
Number: engaged in more grant-funded advocacy activities--increase from 73% to 86% of surveyed activities.Target: consistently targeted most activities (80%) at the local and state levels and less (60%) at the federal level.Effectiveness: fairly constant or 3.28 to 3.36 (4 equaled “very effective” Grantee type: newer (post-1990) consortia were equally effective in achieving advocacy goals by 2006.
![Page 12: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/12.jpg)
Outcome 2: IncreasePolicymaker Awareness
Whoever controls the flow of information dictates our perceptions and perspectives; whoever controls the news shapes our destiny.
- George ClintonMusician and Funk Philosopher
![Page 13: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/13.jpg)
Evaluating Grantee Work With the Media
Effectiveness of media compared to other advocacy tactics
Advocacy Activities Worksheet: # of grantees using media at federal, state, local levels; achievement of three Program outcomes
Partnerships with media
# and type of media request for information;
Number and type of media outlets
Coverage secured # of stories in newspapers, radio
Placement of stories, e.g., page, column
Newspaper article type (op-ed, Letter to the Editor)
Presence of policy issues in the media
Inclusion of specific issues, e.g., role of clinics, patients served
Framing of issues - content analysis
Outreach to media # of press releases developed and distributed (grantee tracking)
![Page 14: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/14.jpg)
Findings-- Media Perspective:Effectiveness of Grantee Strategies
to Obtain Media Coverage
5=very effective
![Page 15: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/15.jpg)
Policymaker and Community Leader Familiarity with Grantee
and Clinic Activities
4 = high familiarity
![Page 16: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/16.jpg)
Educating AND Working with Policymakers
Advocacy contributes to policymaker awareness. The challenge is to position yourself as a credible partner and work with decision makers
Watch for opportunities, e.g., bipartisan support
Leverage the partnership with decision makers pre/post passage of policies
Maintenance of effort required - ongoing education and visibility
![Page 17: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/17.jpg)
Outcome 3: Increased Policymaker Support for Clinic for Clinic Funding
Policy targeted
Bill Number and/or title?
Grant-funded policy advocacy activities undertaken in 2007. Choose from list: a) media advocacy; b) in-person meetings with decision makers; c) drafting of rules, regulations; d) materials development; e) grass roots activities; f) Other: list
Target populations to be affected by policy. Choose from list: a) clinics; b) uninsured; c) low-income populations; d). clinic consortia; e) other special populations; f) Other: list
Outcomes as of 12/07. Choose from list: a) Passed - signed in to law; b) Vetoed; c) Pending - still being considered by decision makers; d) funding secured; e) Other: list.
Federal
Analysis:
1. Count number policies and of grantees that mentioned each policy
2. Code advocacy activities, target populations
3. Code and count outcomes, e.g., “passed”, “vetoed”, “pending”
State
Local
Tool: Annual Grantee Policy Outcomes Worksheet
![Page 18: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/18.jpg)
![Page 19: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/19.jpg)
Evidence of Policymaker Support – Funding Secured by Grantees (ROI)
![Page 20: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/20.jpg)
Funding Secured, by Source
![Page 21: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/21.jpg)
Outcomes 4 & 5: Strengthened Clinic Operations and Increased Services for The
Underserved and Uninsured
Findings: Evidence that more Californians have insurance coverage and are using the services to which they have access
![Page 22: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/22.jpg)
OUTCOME 6: Improved Health Outcomes for Targeted Populations
Achievements in three areas:
![Page 23: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/23.jpg)
Summary of Evaluation Findings
Advocacy as staffing and toolkit to be deployed strategically – gains after 2-3 years
Strategic involvement in policymaking process - desired policy change facilitated by maintenance of effort, technical acumen and credibility
Crafting working relationships with decision makers - “pays off”
Achieving a policy change – gains to member clinics and their target populations
Partnerships with member clinics – critical to achieving a workable solution
![Page 24: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/24.jpg)
Policy Change Logic Model
Expand Advocacy Capacity
Increase policymaker awareness
and support
Strengthenedclinic
operations
Increasedservices forunderservedanduninsured
Improved health
outcomes
![Page 25: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/25.jpg)
Suggestions for Evaluators
Address different information needs of advocates, funders, evaluators
Combine rigor with creativityMixed methodsTriangulateClarify what you’re evaluating re “advocacy”
and “advocacy capacity”
Create a working partnership with advocates, funder
Communications are key
![Page 26: Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators](https://reader035.fdocuments.net/reader035/viewer/2022062405/5562d5ead8b42aac778b49f5/html5/thumbnails/26.jpg)
Thank you! For More InformationAnnette Gardner, PhD, MPHPhilip R. Lee Institute for Health Policy StudiesUniversity of California, San Francisco3333 California St., Suite 265San Francisco, CA 94118Phone: (415) 514-1543Email: [email protected]://ihps.medschool.ucsf.edu/News/california_endowment.aspx
SF
University of California San Francisco