Join the conversation now: #HHSDataFest

Welcome, Purpose, and Agenda OverviewMichael Wilkening, Undersecretary, California Health and Human Services Agency Andy Krackov, Associate Director for External Engagement, The California HealthCare Foundation Daniel Stein, President, Stewards of Change Institute

Open DataFest Goals• Share progress, celebrate accomplishments, make new connections• Connect Social Determinants of Health and Wellness with Let’s Get Healthy California

and Open Data• Showcase innovative local, state, and national programs• Explore the intersection between academic research and open data• Bring together various agencies’ involved with open data to learn from one another

and avoid building more silo• Prepare recommendations to help shape California’s ongoing open data vision and

operations• Introduce concepts, tools and methods to advance information sharing and

interoperability

Where Did We Start? Where Are We Today?

Data prison Lots of data are now ‘free’

No statewide HHS Portal New CHHSA Portal with 164 data sets from all 12 departments

No regional / community based events

Multiple efforts in counties and cities across the state

No state-wide convenings HHS Open DataFest III

No data innovation focus New innovation office

Lagging the US Leading the US

Monday’s Open DataFest AgendaIntroductions

Overview of Open Data Interactive: “Questions of Consequence”

Break Exploring the Intersection between Open Data and Academic Research Data

Lunch (Affinity Table Discussion)Social Determinants - “Let’s Get Healthy California” - Innovation Challenge Winners

Interactive: The Data Hunt Break

Employing Cognitive Learning to Assist Decision Making and Program Effectiveness Achieving Community Health and Creating Person-Centered Services in San Diego

Review and WrapReception

Instructions: Questions of Consequence Purpose: To develop a rich understanding of what the people you interview think and feel about the issues raised by your question.

Method:1. Choose a partner and introduce yourselves. (OK if you have a trio)2. You and your partner each have a unique question (#1 or #2). 3. One of you will interview the other for 8 minutes, and then switch roles. 4. After both interviews are finished, table participants will discuss both questions. 5. Whole room debrief.

Join the conversation now: #HHSDataFest

California’s HHS Open Data Overview, Updates and TrendsKevin Merritt, Founder & CEO, Socrata Stuart Drown, Deputy Secretary for Innovation and Accountability, Government Operations AgencyMichael Wilkening, Undersecretary, California Health and Human Services Agency

• A PRIMER ON OPEN DATA

Kevin MerrittCEO

Socrata

15

16

Who Uses Open Data?

17

• What are the major stakeholder groups interested in government data?

Ordinary but Interested Citizens

Researchers, Scientists, Analysts, Economists & Journalists

Developers & Entrepreneurs

Government Employees!

Data Fuels the 21st Century Digital Government

18

• By becoming data-driven, governments transform themselves

Data-Driven Government

Improve Transparency & Accountability

Build & PromotePublic Trust

Citizen Engagement & Empowerment

Improve Operational Efficiency

Support Fact-Based Decision Making

Promote Economic Development

19

• “The key to progress for digital government maturity

is a singular focus on the exclusive use of data in

designing and delivering government policies and

services.”

Gartner, 2015

20What Does Open Data Look Like?

21Data Can Build Trust & Foster Community

Open Data Solves Real & Poignant Problems22

Shedding Light on Pharmaceutical Influence23

Bring Data to Where Consumers Already Are24

Let Others Extend the Work of Your Data 25

(Raw Data & APIs) Ascel Bio Built the App

Open Data Pilot Open Data Fest IIIMarch 14, 2016

Sacramento Convention Center

26

• Pilot Goals1. Create an efficient, consolidated, and reliable infrastructure for

data.ca.gov2. Build capacity in state workforce3. Make use of CalCloud to reduce cost and reach out to local

governments

• Objectives

1. Create greater efficiencies for program operations. 2. Encourage inter-agency data-sharing; Reduce duplication of

information.3. Enable data-based decision-making.4. Better serve Californians. Provide data more quickly and in a more

user-friendly format.5. Stimulate innovation.

27

Pilot Goals & Objectives

28

Surveyed 88 entities

756 “public” data sets reported

91 different formats

Data Inventory- May 2014

29

Sustainability Innovation Challenge - DGS HeadquartersOctober 24-25, 2015; Go to greengov.data.ca.gov

30

• 11 data sets published• 14 open source apps• Introduced government to new population –potential hires• Collaboration under tight deadline built mutual respect/brand• Learned to flex new muscles = morale boost• Implementing winning apps required learning new skills• Open source Innovation “sandbox”

Innovation Challenge Outcomes

31

Open Data Pilot Timeline

32

• Pilot• Platform• Standards + Governance• Federation• Automation

• Collaboration• Within government• Civic engagement• Targeted deeper access

• Cultural Change• Siloed to Open• Deeper analysis• Data-driven decision-making

More than a platform

Join the conversation now: #HHSDataFest

Interactive Activity: How Open Data Can Be Used to Advance Data-Literacy and Decision MakingFacilitators: Stewards of Change Institute Team

Join the conversation now: #HHSDataFest

Exploring the Intersection between Academic Research and Open Data to Improve Outcomes and Measure Performance; Lessons Learned from Stanford University EventMark Cullen, MD, Director, Stanford University Center for Population Health Sciences

Panelists:Phil Leaf, Ph.D., Professor, Johns Hopkins Bloomberg School of Public Health, School of Medicine, School of Education, and School of Arts and SciencesAenor Sawyer, Associate Director, Strategic Relations, Center for Digital Health Innovation UCSF; Director UCSF Skeletal Health ServiceAnne Neville, Director, California Research Bureau, California State Erika Martin, Ph.D., MPH, Senior Fellow and Director of Health Policy Studies, Rockefeller Institute of Government and Associate Professor, Department of Public Administration & Policy, State University of New York

Responder:Regan Foust, Director, Strategic Partnerships, Research Scientist, Children’s Data Network

Open Data Are ValuableJonathan Gross, BCHD

• Open data are sometimes mistaken as not being valuable

• However, the data are made public because they are valuable!

• Federal, state, and local governments are making more data available than ever before to help solve a wide array of problems.

Baltimore: 19-Year Life Expectancy Gap

www.baltimorecity.gov/healthmaps Data at BNIA: http://bniajfi.org/

Health Insurance Estimates• Tan and red census

tracks are areas with low health insurance rates

• The markers are hospitals and FQHCs

Data source: Census/American Community Survey (ACS)and Maryland State Data Center: http://www.mdp.state.md.us/msdc/S7_ACS.shtml

www.baltimorecity.gov/healthmaps

Hospital’s Proximity to Firearm Violence

Medicare: Open Opioid Prescription DataProvider-level data are also available

https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/Medicare-Provider-Charge-Data/OpioidMap.html

Be Open Data Savvy!• Start early• Be BOLD!• Open data is valuable• Search, search, search!• Find a comparison group/area• Use simple tables and colorful graphs• Work in teams• Don’t be afraid to start over

Health Dept. Map Gallery• Baltimore City

• Social determinants of health• Health insurance at census tract level• Homicide epidemic – spatial patterns• Shooting incident animation

www.baltimorecity.gov/healthmaps

• More to come…• At least one map per division• Asset mapping

• Mid/Long-term possibilities…• Health atlas• Crowdsourced, volunteered geographic

information, or social media

Youth Health Index across 55 Baltimore City Community Statistical Areas

ACADEMIC RESEARCH

OPEN DATA

@CAStateLibrary

@annenevilleCalifornia State Library

Anne Neville

MARCH 14, 2016

The Case: The National Broadband Map (2009-2015)(You are correct. This is not health-related)

• First national + open dataset of broadband availability

• Really granular – census block or road segment

• Big: 25M new records every six months

• New opportunity to study broadband access in the US

• Stakeholders: policymakers, advocates, researchers, public/private sector analysts, journalists, consumers

Stakeholder Outreach: Researchers

Successful event at National Press Club one month after launch of public data

Seven research groups: Georgia Tech/U. Toronto/ Northwestern, U. Texas, Joint Center, Murray State, Harvard, Drexel/Mich State Phoenix Center

Federal Register notice, advanced access to data for institutions with research plans

Government agency (NTIA) has existing relationship with researchers

What Happened Next? • Dedicated space for researchers on project wiki

• Outcome: Not used

• Created a Researcher Working Group

• Outcome: Initial interest, not sustained

• Considered grants to fund independent research

• Outcome: Not created

The Plan Didn’t WorkWhat Happened?

Coordinating Research Activities

• Limited staff

• No initial external funding for research on this topic

• Limited funding - would research grants provide most usefulness?

Research-Related Activities

• Regularly answered questions from researchers about metadata, data collection methods

• Reviewers for academic papers

What’s Helpful for Gov’T? What we hope you’ll say when you call

Improving Communication• If the data is problematic, let’s talk

• We appreciate a heads up, if possible

• Specific feedback leads to specific results

Contact Info: @CAStateLibrary

@anneneville

Integrating Public Health Researchers into the Open Data Ecosystem

Erika Martin, PhD MPHRockefeller Institute of Government & University at Albany

California Health and Human Services Open DataFest IIISacramento, CAMarch 14, 2016

This work was supported by the Robert Wood Johnson Foundation’s Public Health Services and Systems Research Program (grant ID #71597)

Project overview Limited guidance on tailoring open data to different users

Open data are only valuable when used

How can we improve the quality and usability of data for public health research and practice? Systematic review of health data offerings in three open data

portals (HealthData.gov, Health Data NY, NYC OpenData) Key informant interviews with practitioners publishing open health

data to understand challenges and opportunities Pilot open data linkage project to assess the feasibility of using

open data for academic research

Data characteristics(e.g. missing data, timeframe, data collection procedures, database design, data elements, population)

Data user characteristics(e.g. intended use, expertise, skills, tasks performed)

Platform promotion and user training(e.g. value propositions, financial resources, political support, information technology, regulations and data stewardship, legal interpretation of confidentiality protections)

Rockefeller Institute of Government 55

Intrinsic data quality(e.g. accuracy, reputation, confidentiality, reliability, validity, objectivity)

Contextual data quality(e.g. appropriate amount, completeness, concise representation, ease of manipulation, relevance)

Platform usability(e.g. accessibility, functionality, learnability, representational consistency, visibility)

Metadata quality(e.g. accuracy, completeness, consistency, interpretability, provenance)

Short-term impacts(e.g. availability of health information, data-driven population health planning and monitoring, mHealth development, consumer empowerment, research grants and studies)

Long-term impacts(e.g. improved population and patient health, enhanced decision-making, higher quality/value medical and public health services)

CHARACTERISTICS OF DATA USE

DATA QUALITY AND USABILITY

DIRECT AND INDIRECT HEALTH IMPACTS

Systematic review of open data offerings(HealthData.gov, Health Data NY, NYC OpenData)

Most data offerings not designed for health research Only one-quarter of open data offerings are structured datasets Most offerings do not contain demographic variables

Variation in quality and usability across platforms Health Data NY scored highest on intrinsic data quality, contextual data quality,

and adherence to Dublin Core metadata standards

Gaps in meeting “open data” deployment criteria All offerings met basic “web availability” open data standards Fewer met higher standards of being hyperlinked to other data to provide

context

Platforms enable users to discover and access data in novel ways, with areas for improvement

Technical problems limit functionality, low web visibility, HealthData.gov is primarily a search engine

Key informant interviews(Policymakers and practitioners in New York and federal agencies)

Wide range of perceived benefits Internal benefits: improved data/metadata quality, more efficient public health

operations (e.g. data silos, FOIA requests) External benefits: health literacy, data-driven improvements in healthcare delivery

and built environment, community empowerment, improved data quality, timeliness, and usefulness

New users bring innovative ideas

Numerous challenges to releasing data Critical challenges: resources, cultural resistance, legal and regulatory issues, and

data/metadata quality Other challenges: technical issues with legacy systems and data platforms,

knowledge gaps, addressing needs of diverse end-users

General optimism that open data movement will continue Yet success depends on sustained leadership, resources, cultural changes,

promoting the use of data, and establishing governance

Pilot data linkage project(Mapping childhood obesity to the built environment in New York)

Many datasets readily available for public health research Can use data creatively to evaluate multiple dimensions of the built environment

(e.g. using restaurant inspections data for fast food availability) Can synthesize data from different domains (health, agriculture, education)

Challenges consistent with findings from other study phases Lack of standard definitions for data elements severely constrains interoperability

and ability to merge by geographic identifier Incomplete metadata, e.g. missing codebooks Data quality, e.g. incomplete addresses, inconsistent location descriptions Data timeliness High level of geographic aggregation limits value Some data not easily discoverable (or available) in open data platforms Data not yet 5-star, e.g. downloadable in multiple non-proprietary formats and

with links to provide context Limited usability, e.g. advanced statistical skills required to prepare data

Questions? Contact: emartin@albany.edu

For additional project information: www.publichealthsystems.org/erika-martin-phd-mph-0

For materials from fall 2013 workshop on open health data in New York and links to open data resources:

www.rockinst.org/ohdoo

Bibliography Martin EG, Helbig N, Shah NR. Liberating data to transform healthcare:

New York’s open data experience. JAMA 2014; 311(24): 2481-2481. Martin EG, Helbig N, Birkhead GS. Opening health data: what do

researchers want? Early experiences with New York’s open health data portal. J Public Health Manag Pract 2015; 21(5): e1-7.

Martin EG, Law J, Ran W, Helbig N, Birkhead GS. Evaluating the quality and usability of open data for public health research: a systematic review of data offerings on three open data platforms. J Public Health Manag Pract. [In press; online ahead of print]

Martin EG, Begany GM. Opening government health data to the public: benefits, challenges, and lessons learned from early innovators. [Under review]

Dwicaksono A, Brisette I, Birkhead GS, Bozlak CT, Martin EG. Evaluating the contribution of the built environment to obesity among New York State students. [Working paper.]