1

The Shift to {Open|Big|Linked} Data Or, how I learned to stop worrying and love the data

Pia Waugh Director of Coordination and Gov 2.0

Technology and Procurement Division

Department of Finance

2 2

Great expectations

http://www.flickr.com/photos/zebble/8212264/

http://www.flickr.com/photos/advocacy_project/5810581322/ http://www.flickr.com/photos/lowfatbrains/2588450228/

Dados abertos → libertad

● Better government● Powerful citizens● Collaborative problem solving● A first step towards gov as an API

15 15

NationalMap

14 14

data.gov.au – discovering Commonwealth data

Free, cloud, scalable API enabled platform for linking/hosting data

(CKAN, Geoserver, data model register, CSW harvester)

Staged approach

1. Publishing (mid 2013 – late 2014) Improving the functionality and ease of

publishing for agencies with training and

documentation

2. Value realisation (Late 2014) Providing useful front end tools for data.gov.au

including data visualisation and analysis tools.

Publishing quality data a pre-requisite.

3. Data quality (Late 2014) Looking at ways to provide agencies the ability

to accept iterative data improvements in a

verifiable way

Done!

• Manual and automated publishing options

• API access for spatial and tabular data

• Basic data visualisation capability

• Use cases and site/data/org analytics

• Data Request Site

• Metadata mapping (DCAT, AGLS, ANZLIC)

• National Map integration

• Data model registry

In Planning

• 5 star quality plugin

• Selective crowdsourcing for updates

• Federated search for discoverability

• League Table (from eGov Policy)

Reduces bureaucratic overhead• More efficient to share data across government and with public• Proactive automated publishing for common data requests

Improves government operations• Enables collaboration and consistency across gov and with public• Improves policy analysis, development, implementation, reporting• Data APIs support mobile and digital service delivery• Improves data quality through verifiable public contributions• Improved opportunities for evidence based and iterative policy• Reduces opportunities for corruption

Innovation• Enables government to tap into internal and external innovation• Starts to change the culture of what “innovation” means & costs• Enables greater capacity for public contributions to gov

4 4

Key Benefits to Community/Industry in Opening Data

Economic

• Creates opportunities for industry to value-add to government data

• New services, systems and industries

• New opportunities and innovation in industry, research, civil society

Accountability

• Visibility to government spending, projects, effectiveness, etc

• Increases incentive to follow evidence based approach

Better policy and programs

• Enables greater participation in policy planning and implementation

• More informed public → better decision making

• Improvements to data → better policy and decisions

12 12

Data Portals

Local Portals: • City of Melbourne

13 13

Open Data Network and Discovery Model

National Map

Datavis Application development

Analysis & Policy

Value Creation

Discovery

Data

Full Discovery

New Services

FIND

National spatial index (gov, private,

research)

17 17

18 18

DSS and ATO Statistics

16 16

Spatial Case Study

http://www.finance.gov.au/blog/2014/02/24/guest-post-on-the-value-of-open-roof-prints/

1) Reduced work

2) Value adding

3) Quality improvements

4) Zero cost API access to their own data

Some useful case studies

• Publishing Budget 2014 Data Report

• Open data – Transforming the Provider / Stakeholder Paradigm

• On the Value of Open Roof Prints

• 100 years of patent and IP data released on data.gov.au

More available soon at http://toolkit.data.gov.au

Other Australian case studies/documentation

• SA Open Data Toolkit

• QLD Government Case Studies

• Victorian Government Showcase

• NSW Apps Showcase

• ACT examples

23 23

Some Challenges

• Education

• Legislative

• Culture

• Systems

• Privacy and anonymisation

• Reactive vs proactive

• Metadata/semantic context

• Too much data

• Real time vs historic

• Definitions and common references

• Limited skills and over specialisation

22 22

Open by Design – drawing a line in the sand

Building proactive publishing into:

• Systems

• Processes

• Procurement

• Planning

• Records management

Leveraging open data through:

• Public APIs

• Analysis tools and datavis

• Internal processes looking for external sources

13 13

Some building blocks for the future

From FOSS, open data, open standards

• Improving how gov does tech

• On the shoulders of giants... globally

• Models of collaboration (and healthy competition)

• Modular and interoperable design – best tool for each job

• Exposing data/APIs as a service

• Mashable government

• JFDI - scratch an itch, don’t just dance with mosquitoes

• Meritocracy

•Technical excellence and cleverness as a core tenet

http://xkcd.com/898/

The rise of the technocracy

The geek has inherited the earth?

Why do hackfests?

➔ Public engagement➔ Tap into real innovation➔ Skills development➔ Community building➔ Motivation to publish data

Civic Hacker Motivation➔ Public good➔ Fun and community➔ Fame and glory!

The history of GovHack

GovHack 2009 – the beginningGovernment run, 1 location, 40 hackers

2009–2011 - Corporate events onlyThe rise of community frustration

GovHack 2012 - volunteers2 locations, 180 hackers, 40 projects

GovHack 2013 - volunteers8 locations, 800 hackers, 130 projects

GovHack 2014 - volunteers12 locations, 1300 hackers, 200 projects

GovHack 2015 – 3-5 July!Around 16 locations across Aus/NZ, ~2000 hackers

Federal, State and Local Government engagement. Participants are students, professionals, community, public servants.

@BrisBert

25 25

Hypothetical: Government as an API

Care of fedAPI.gov

The future is here, it is already widely distributed.

Governments are learning to be part of our world

http://www.flickr.com/photos/mr_matt/3568892622/

Challenge #1: Collaborate

Challenge #2: Design the future

Challenge #3: Lead the way

Challenge #4: Have fun

Questions?

@piawaugh

@datagovau

http://pipka.org

data.gov.au

toolkit.data.gov.au