Open Source, Open Data

Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)


My presentation from Florida Linux Show 2009. Find out how open source's principles are being used outside of software, and how open source and open data can work together to change the world.

Transcript of Open Source, Open Data

  • 1. Open Source, Open Data Kirrily Robert Florida Linux Show, 2009

2. From Open Sourceto Open Data 3. 1993 Me in 1993 My Linux desktop looked like this 4. 1993 I started using Linux in 1993 I was very excited by it, even though it was quite primitive at the time Other people thought I was a little crazy 5. 1999 Image: Wikipedia Image: EngadgetJar Jar in 1999 Googles servers in 1999 6. 1999 By 1999 Linux + open source was starting to take off Companies using and building services on Linux etc. We were calling it Open Source - a more marketable term for Free Software 7. Four Software Freedoms Freedom to run the program Freedom to study the program and modify it for your own use Freedom to redistribute verbatim copies Freedom to improve the program, and release your improvements 8. Free Culture A similar movement Make cultural works freely available Mostly over the Internet 9. Free Culture 10. Free Culture 11. Free Culture 12. Free Culture Freedom to use the work Freedom to study the work and to apply knowledge acquired from it Freedom to make and redistribute copies Freedom to make changes and improvements, and to distribute derivative works 13. Image: 14. What is Open Data? 15. Data 16. Image: himmelskratzer @ Flickr 17. What is data? Ones and zeroes (obviously) But also ling cabinets, research archives, and other ofine resources Its not OPEN data unless you can get at it 18. Open Data Freedoms Freedom to use the data Freedom to study the data and modify it for your own use Freedom to make and share verbatim copies Freedom to improve the data and redistribute the results 19. Data availability Digital Online Well formatted 20. Open Data Projects 21. Created 2007by Carl Malamud Making GovernmentInformation MoreAccessible 22. SEC EDGAR records Patents database Copyright database Congressional records Legal decisions Fedix 23. Founded 2008 Increase public access to high value, machine readable datasets generated by the Executive Branch of the Federal Government. 24. OpenStreetMap 25. Compare... 26. OpenStreetMap 27. Open Library Project 28. CD data Tracks, artists, releases... CC license 29. Flickr Images Metadata tags, timestamps, geolocations, etc. Range of CC licenses and permissive TOS 30. Infochimps Large data sets Various licenses Tools for transformation 31. Open data about everything 8.5m concepts CC-BY license API and data dumps 32. 2,416,683 books 33. 16,608 ships 34. 488 cheeses 35. Structured data { "name": "Asiago cheese" "id": "/en/asiago_cheese", "region": [{ "id":"/en/asiago", "name": "Asiago", "type" : "/location/location"}],"source_of_milk": [{ "id": "/en/cattle", "name": "Cow", "type" : "/biology/organism_classification"}] } 36. Open Data Apps 37. Apps for America competition Open source and open data Round 1: various data sources Round 2: 38. Legistalker 39. Filibusted 40. Where the money goes 41. Open Source for Open Data 42. What can open source do? InputScrape Processing MungeOutputVisualise 43. Scraping data APIs XML, RSS, JSON... Downloadable data sets XML, Excel, CSV, triple dumps... Beautiful Soup (Python) 44. Munging data Perl R (statistical analysis) Hadoop (parallel data processing) 45. Visualisations MIT Simile Processing 46. 47. Semantic Web Describe meaning, not markup Triples: subject, predicate, object Expression: RDF 48. Linked Open Data 49. Semantic web tools Triple stores Sesame, BigData, Virtuoso... Libraries RDFLib (Python), Redland RDf (librdf)... 50. Freebase Acre 51. Open source for open data Low barrier to entry Hooks in to Freebase data Share and clone apps Apps are BSD licensed 52. FMDB 53. Gendered names app 54. Query editor 55. Clone! 56. 57. Where next? 58. Open Data: Issues License clarity Govt + Corporate acceptance Developer literacy What do we DO with it? 59. What do we do with it? 60. What do we do with it? 10 years ago we were asking the same questions of Open Source With Open Data, we are just starting to realise its potential Please join us! 61. Keep in touch Email Freebase blog Twitter @fbase