City Forward and Open Data Standards

cityforward.org

City Forward and open data standards

Gina Cardosi and Dave Rook

3-Aug-2012

2© 2012 IBM Corporation 2© 2010 IBM Corporation© 2012 IBM Corporation

More than ever before, human life revolves around the city.

In 1900,

13%of the world’spopulationlived in cities.

In 2007, that number surpassed

50%—and it continuesto grow.*

By 2050,

70%of all people onearth will be city dwellers.

* We are adding the equivalent of seven New Yorks to the planet every year.

2

3© 2012 IBM Corporation

Our Cities Generate Vital Signs

Today, almost anything—any object, process or system—can be instrumented, interconnected and infused with intelligence.

Data is lying in archives, published on government websites, being sensed from instrumentation in the environment, deduced from aerial imagery, and built from the ground-up by citizens electronically communicating about city life.

This reality holds enormous promise for people everywhere. Nowhere is the potential for progress more evident than in the world’s cities.

4© 2012 IBM CorporationMap provided by Data.gov http://www.data.gov/community#mapanchor

Open Data is global

4

“I want us to ask ourselves everyday, how are we using technology to make a real difference in people’s lives.”- President Barack Obama

http://www.data.gov/community

5© 2012 IBM Corporation 5© 2010 IBM Corporation

Our mission: To bring together information about cities and communities in ways that lead to new insights and support decision-making.

6© 2012 IBM Corporation 6© 2010 IBM Corporation

What is Corporate Citizenship?

© 2012 IBM Corporation 6

IBM has developed a thoughtful, comprehensive approach to corporate citizenship that aligns with IBM’s values and maximizes the impact we can make as a global enterprise. We focus on specific societal issues, including the environment, community economic development, education, health, literacy, language and culture.


Adding data to City Forward

Once sources are identified there are still legal and technical steps involved in adding data to City Forward.

Load into Database

Identify Data Source

Map to Data Structure

Review Legality

Assess the Value

Data challenges

• Some cities don’t have data that’s publically available and free

• Each new source introduces additional transformation rules

Value challenges

• Data isn’t at a city level

• Data doesn’t span years to allow for patterns over time

Legal challenges

• Data is publicly available but with restrictions, must seek permission


Data Issues

Standard classifications

Integrating data from different countries introduces incompatibilities in classifications of data. For example, defining poverty and education level varies by country. In addition, defining the geography or time frame requires an administrative organization to define the city or metro area boundaries and when the reporting period starts and ends (e.g., school year).

Aggregation

Non-additive data such as indicators cannot be aggregated over geography, time, or other classification. For example, unemployment rates, poverty rates, gross domestic product per capita are indicators that must be recomputed by geographic level and time frame. Having the unemployment rate at the county level cannot be summed to compute the unemployment rate for the metro area.

Availability

Gathering data and making it available costs money. Most countries do not have data like the US Census American Community Survey (ACS) that provides annual estimates on many topics. The House of Representatives voted to eliminate funding for the ACS and the Senate has yet to vote on the issue.


Open Data Movement timeline

Source: http://visual.ly/open-data-movement

http://visual.ly/open-data-movement




Impact on everyday life

How safe is my neighborhood?

Which career is right for me?

What type of education do I need?

Sources: http://www.chicagocitycrime.com/, http://www.bls.gov/ooh/computer-and-information-technology/software-developers.htm, http://cityforward.org

12

http://www.chicagocitycrime.com/



http://www.bls.gov/ooh/computer-and-information-technology/software-developers.htm

http://cityforward.org/

http://cityforward.org/


The United States advocates agencies make data available via APIs and widgets and provides an open source version of data.gov

Source: http://www.data.gov/opengovplatform


Options for data analysis and integration are increasing

Government agencies are promoting not only data, but application programming interfaces (APIs) and interactive widgets to help developers get access to timely data

Sites like DBpedia, Freebase, and Tetherless World Constellation (TWI) are cataloging data in Resource Description Language (RDF) to make data accessible anywhere anytime.

Linked data is prominent at data.gov.uk, dublinked, and other open data sites.

Non profit organizations are proving community level data to enhance citizen engagement.


Linked Data

DBPedia http://dbpedia.org/About

Freebase http://www.freebase.com/

Data.gov.uk http://data.gov.uk/

Dublinked http://www.dublinked.ie/

Data.gov http://www.data.gov/

Sourc: http://linkeddata.org/ Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/


The Open Data Protocol (OData) is an emerging standard Based on web technology standards

HTTP, Atom Publishing Protocol (AtomPub), JSON, and REST

OData is used to access data in various data sources

Relational databases, file systems, content management, web sites, etc.

See ecosystem link on odata.org to see current list of

Consumers: Applications to process OData

Applications: Applications exposing OData

Producers: Live OData Services

Sample Services and Sample Code

Link to article about OData and RDF -> http://www.ibm.com/developerworks/webservices/library/standards-odata/

Source: http://www.odata.org/


Socrata is an open data vendor gaining market share

Growing list of customers (see list ->)

G7 cities working together on sharing applications by standardizing data formats: Chicago, Los Angeles, Boston, New York, Seattle, Washington, DC, and San Francisco.

SODA is the Socrata Open Data API

Customer Web siteCity of New Orleans data.nola.gov MetroChicagoData.com metrochicagodata.com City of Baltimore data.baltimorecity.gov City of Austin data.austintexas.gov SAMHSA info.samhsa.govCity of San Francisco data.sfgov.org U.N.D.P. data.undp.com Kenya opendata.go.ke New York City nyc.gov/data State of Illinois data.illinois.gov King County datakc.orgCook County datacatalog.cookcountyil.gov City of Edmonton data.edmonton.ca Data.gov explore.data.gov Medicare data.medicare.gov State of Oregon data.oregon.gov State of Oklahoma data.ok.gov City of Chicago data.cityofchicago.org City of Seattle data.seattle.gov State of Colorado data.colorado.gov Region of Lombardia dati.lombardia.itState of Missouri data.mo.gov Ethics.gov ethics.gov City of De Leon deleon.socrata.com

source: http://www.socrata.com/customer-spotlight/


Widgets

Government agencies are now creating interactive widgets and gadgets that can be embedded into other web sites.

Examples:US Census

Environment Protection Agency (EPA)

Sources: http://2010.census.gov/2010census/data/index.php, http://www.epa.gov/enviro/facts/widgets.html

http://2010.census.gov/2010census/data/index.php

http://www.epa.gov/enviro/facts/widgets.html

http://www.epa.gov/enviro/facts/widgets.html


Open 311

Source: http://open311.org/


For more information

Open Government Initiatives -> http://wiki.civiccommons.org/Initiatives

Global Pulse -> http://www.unglobalpulse.org/

Open Government Data Catalog (RDF) -> http://logd.tw.rpi.edu/demo/international_dataset_catalog_search

Global City Indicators -> http://www.cityindicators.org/

NNIP Best Metro Data Releases of 2011 -> http://www.theatlanticcities.com/technology/2011/12/best-metro-data-releases-of-2011/772/

US Census Developer page -> http://www.census.gov/developers/

Dept of Labor Developer web site -> http://developer.dol.gov/

EPA Developer page -> http://www.epa.gov/developer/index.html

http://wiki.civiccommons.org/Initiatives

http://www.unglobalpulse.org/

http://logd.tw.rpi.edu/demo/international_dataset_catalog_search

http://www.cityindicators.org/

http://www.theatlanticcities.com/technology/2011/12/best-metro-data-releases-of-2011/772/

cityforward.org

Thank you.

City Forward and Open Data Standards

Technology

Transcript of City Forward and Open Data Standards