Open data is seen as one of the key building blocks of the Big Society.Opening up the countryʼs data will allow people to  “flag up waste when they see it”, and the increased scrutiny “will act as a powerful straightjacket” David Cameron

How it can helpThe goal is to provide facilities to engage and inform citizens on policy decisions to improve democracy in the UK. Allowing the taxpayers of the country to have access to the information about how their money is spent enables a bottom up approach to making better use of available funds to support our communities.

The government is expecting an “army of armchair auditors” to pay attention to each individual line of information to put this straightjacket in place. This will only come about when developers create the ‘killer apps’ that make sense of the thousands of lines of information being released monthly.

Our Media Sandbox storyWe were lucky to be commissioned by iShed to take part in their 2010 Media Sandbox scheme to undertake a project about making spending data visible – where and what money was being spent on. The idea was to create an exciting, highly visual hyper-local tool to act as an effective feedback mechanism for the council to truly understand the thoughts and needs of the community.

When the initial spending data was released, it became apparent to us that the focus of our project needed to adjust to enable councils to produce useable and contextual data, whilst highlighting the future potential of using Open Data. Our aim was to use the data as far as we could, demonstrating both the opportunities and limitations of these datasets.

This meant our research was about how spending data can be read and cross referenced to gain useful insight. It has helped us to better understand what’s required to create a killer app, for which we formulated our ‘ingredients’ list. This list is intended to help data publishers and end users such as ourselves, and help move the situation on so these exciting projects can see the light of day.

Copyright Delib Ltd, Co.#5158056, registered at 35 King Street Bristol, BS1 4DZ.

What’s the data like? We used the data from the West of England LEP partner councils: Bristol, Bath and North East Somerset (BANES), South Gloucestershire and North Somerset. The aim was to compare and contrast both the similarities and differences amongst the four organisations in the hope to identify interesting trends and narratives.

The biggest challenges we faced were the numerous discrepancies between the councils’ data structures. North Somerset and South Gloucestershire had been publishing since April 2010, whereas Bristol has data from August and BANES only produced the December data by the end of January deadline. This meant our study was limited to the December dataset. Within the 40,000 rows, Bristol have the least contextual information with 3 untitled description columns. BANES and North Somerset had the most detail but both used internal key coding systems. What this means is that it’s hard to make any meaningful, scientifically valid or accurate reading in the Bristol data that connects spending to council strategy or goals, whereas BANES has been refined so much and categorized by internal codes there is little to work with.

The only consistent, reliably structured information is supplier name and amount spent. We started our investigation here to see what trends and narratives could be found about the LEP partners from such limited information.

Because of the volume of data across the 4 councils and the manual work involved, we took a snapshot of the top 10 most frequently paid suppliers and focused on that group to see whether these interactions were with local or external suppliers. For each supplier, we had to manually search for the registered address of that supplier. We also used the total spend with the national statistics census data for population and area to see how much the council had spent per person and per square metre. Although the narratives are limited, it does give some interesting insight to the differences in spending within the LEP across the 4 councils.

Copyright Delib Ltd, Co.#5158056, registered at 35 King Street Bristol, BS1 4DZ.

This graph illustrates how the lack of consistency across the four councils spending data makes it difficult to create an

Without context, information is available, yet obscure, creating illusory transparencyOur recommendationsTo get more use from the spending datasets, the information needs to be more closely tied to the councils’ directorates, departments and goals. Right now the armchair auditors can only really comment on the quantity of money spent and with whom - this covers the ‘what’ but not the ‘why’. Moreover, without linkable data, it is constrained to being compared with a small number of other datasets.

Review. Reconfigure. Republish.We recognise that much of the data released has never been intended for publication, therefore not collected or input in ways that satisfy needs outside of the immediate requirements of the organisations involved.

We recommend each organisation reviews the purpose and opportunities for publishing this material and adjusting recording and reporting methods to support the goals of transparency.

- LEP members (or any other bodies who wish to work together) should coordinate their coding for consistent reading.

- Treat data release as strategic – departments within a council should look a what information creates a realistic depiction of their service

- Consider relevant contextual information- Work with local developers to understand what’s need to make useful repository - Amend databases accordingly to accommodate the new information with associated

metadata

Our killer app ingredients list is intended to prompt organisations to consider how data can be opened up usefully to allow the creation of exciting applications.

Copyright Delib Ltd, Co.#5158056, registered at 35 King Street Bristol, BS1 4DZ.

Ingredients for a killer appStandard mix - Standard machine readable data - CSV files - Must be regularly updated - Linkable data e.g. geolocation Desirable extras Contextual data - e.g. salary information for service provision, ground rents etc Pertinent data - e.g. tangential information that provides greater context such as usage rates, demographics, attendance etc.

About machinesMuch of the correlation has been manual as the data structures are not compatible with each other or consistently coded enough to be simple for databases to create common links. For a successful app to work, the amount of manual work needs to be minimal so it can run without human intervention. Information structures should be linear, simple and have clear boundaries.

An example of how an optimized monthly council spending CSV file might look like:

Directorate Department Cost Centre Description Location (post code what money was spent on)

Supplier Supplier postcode

-

Open data is leaving its infancy, now entering the second phase. Where next? Context, Richness, Killer Apps.We are moving towards the second stage after the initial sets have been published and some work has been done to understand the uses. This second stage needs to be about refinement and the community is gaining a clearer understanding of what information will be useful for creating those killer apps.

As the coalition government continue to press to open up and publish data, local and central government departments need undertake work before the data is rich enough to supply developers with options to making varied approaches with the information. When the revised and complementary datasets become available, developers like us will be ready with ideas to put into practice.

Across the country, as departments are opening data, there is a growing community of developers experimenting and creating tools that use this data. Most notably, initiatives such as Rewired State’s DotGovLabs and Bristol Council’s Open Data Hack Days are collaborating developers with open data.

Best practice guidelines have now been released by the Government, and a consultation on a new spending data code of practice has just concluded.

Copyright Delib Ltd, Co.#5158056, registered at 35 King Street Bristol, BS1 4DZ.

What else has been done with this data? Openly local - links up council details, spending data and supplier detail, with facility for the public to add additional information.

openlylocal.com

Armchair auditor - loads spending data into indexable web pages, makes files easier to read but no more contextual.

armchairauditor.co.uk

Spotlight on spend - cleans, reclassifies, creates machine readable files for data publishers. A paid for, hosted web application.

whatis.spotlightonspend.org.uk

Copyright Delib Ltd, Co.#5158056, registered at 35 King Street Bristol, BS1 4DZ.

Copyright Delib Ltd, Co.#5158056, registered at 35 King Street Bristol, BS1 4DZ.

Copyright Delib Ltd, Co.#5158056, registered at 35 King Street Bristol, BS1 4DZ.

Copyright Delib Ltd, Co.#5158056, registered at 35 King Street Bristol, BS1 4DZ.