Data Tools cosystem_for_non_programmers
-
Upload
itnig -
Category
Technology
-
view
336 -
download
0
Transcript of Data Tools cosystem_for_non_programmers
![Page 1: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/1.jpg)
Data Tools Ecosystem for non-programmers
Óscar Marín Miró@outliers_es
www.outliers.es
1
![Page 2: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/2.jpg)
Contents at a glance
‣Why mess with Data?‣A Data Pipeline
The Data Process
Data Acquisition Tools‣Data Sources‣Google Forms & Google Docs‣Scraping tools‣Network acquisition tools‣PDF tools‣Open Refine
Data Analysis‣Excel capabilities‣Beyond excel
Data Visualization Tools‣Timeline.js‣Wordle‣Plot.ly‣Google Fusion Tables‣Gephi‣CartoDB‣MapBox‣Tilemill
Developers needed‣Size‣Real-Time‣Analysis beyond Basics‣Interactivity*
Resources
2
![Page 3: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/3.jpg)
The Data Process
http://en.wikipedia.org/wiki/Data-driven_journalism
3
![Page 4: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/4.jpg)
Why mess with data?
Data
Information
Knowledge
Wisdom
•Register observations
•Structured Data•Semantics•Narrative•Spatialized Data
•Personal Absortion•Action: Information application•‘Insights’
•Non transferable•Ethic knowledge application
Cog
nitiv
e C
ompl
exity
4
![Page 5: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/5.jpg)
A Data Pipeline
AnalysisAcquisition Presentation
Data Information
•API’s•Scraping•Files•DB’s•Providers
•Text Mining•Machine Learning•Network Metrics•Stats•Filtering
Data
•Visualization•Story
5
![Page 6: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/6.jpg)
Data Acquisition Tools
6
![Page 7: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/7.jpg)
Data Formats
• JSON (Javascript Object Notation) - Dev
• CSV, XLS - Tabular
• XML - Old, but still used
• Web data
7
![Page 8: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/8.jpg)
Data Acq Techniques
• Ask the user: Forms
• Download from Data Repositories
• Web data: Scrape
• Ask the provider: API access
8
![Page 9: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/9.jpg)
Google Forms
9
![Page 10: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/10.jpg)
Google Forms
10
![Page 11: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/11.jpg)
Google Forms
11
![Page 12: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/12.jpg)
Google Forms
http://www.encuesta15m.cc/subject.html
12
![Page 13: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/13.jpg)
Google Forms
http://www.encuesta15m.cc/subject.html
13
![Page 14: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/14.jpg)
Google Forms
http://data.elperiodico.com/
14
![Page 15: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/15.jpg)
Google Spreadsheet
http://mashe.hawksey.info/2012/10/feeding-google-spreadsheets-exercises-in-import/
15
![Page 16: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/16.jpg)
Google Spreadsheet
16
![Page 17: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/17.jpg)
17
![Page 18: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/18.jpg)
Scrape w/ Chrome Scraper
https://chrome.google.com/webstore/detail/scraper/mbigbapnjcgaffohmbkdlecaccepngjd
18
![Page 19: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/19.jpg)
Scrape w/ Chrome Scraper
19
![Page 20: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/20.jpg)
Scrape w/ Chrome Scraper
20
![Page 21: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/21.jpg)
Scrape w/ Chrome Scraper
http://dataist.wordpress.com/2012/10/12/get-started-with-screenscraping-using-google-chromes-scraper-extension/
21
![Page 23: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/23.jpg)
PDF Extraction: Tabula
http://tabula.nerdpower.org/
23
![Page 24: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/24.jpg)
PDF Extraction: Tabula
http://tabula.nerdpower.org/
24
![Page 25: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/25.jpg)
PDF Extraction: Tabula
http://tabula.nerdpower.org/
25
![Page 26: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/26.jpg)
Network Data: Flocker
http://flocker.outliers.es/
26
![Page 27: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/27.jpg)
Network Data: Flocker
http://flocker.outliers.es/
27
![Page 28: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/28.jpg)
Open Refine: Cleaning Data
https://github.com/OpenRefine
https://code.google.com/p/google-refine/downloads/list
28
![Page 29: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/29.jpg)
Open Refine: Cleaning Data
https://github.com/OpenRefine
https://code.google.com/p/google-refine/downloads/list
29
![Page 30: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/30.jpg)
Open Refine: Cleaning Data
https://github.com/OpenRefine
https://code.google.com/p/google-refine/downloads/list
30
![Page 31: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/31.jpg)
Open Refine: Cleaning Data
https://github.com/OpenRefine
https://code.google.com/p/google-refine/downloads/list
31
![Page 32: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/32.jpg)
Open Refine: Cleaning Data
https://github.com/OpenRefine
https://code.google.com/p/google-refine/downloads/list
32
![Page 33: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/33.jpg)
Open Refine: Cleaning Data
33
![Page 34: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/34.jpg)
Open Refine: Cleaning Data
https://github.com/OpenRefine/OpenRefine/wiki/GREL-Functions
https://github.com/OpenRefine/OpenRefine/wiki/GREL-String-Functions
34
![Page 35: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/35.jpg)
Data Analysis Tools
35
![Page 36: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/36.jpg)
Data Analysis Tools
• Numeric analysis
• Network analysis
• Language analysis
36
![Page 37: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/37.jpg)
Data Analysis w/ MS Excel
• Sort
• Basic Stats: Mean, Median, Sum...
• Filter
• Charts
• Pivot tables
• What-if
• Data Analysis Tool
37
![Page 38: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/38.jpg)
Data Analysis:MS Excel‘Pivot Tables’
38
![Page 39: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/39.jpg)
Data Analysis:MS Excel‘What-If ’
39
![Page 40: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/40.jpg)
Data Analysis: MS Excel
Correlations, Covariance, Statistic Summary, Regression, Histogram, Factor Analysis, Fourier Analysis....
http://ire.org/media/uploads/car2013_tipsheets/excel_stats_nicar2013.pdf
40
![Page 42: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/42.jpg)
Data Analysis: NLP??
42
![Page 43: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/43.jpg)
Data Analysis: NLP??
43
![Page 44: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/44.jpg)
Data Visualization Tools
http://well-formed.eigenfactor.org/
44
![Page 45: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/45.jpg)
Data Visualization Tools
• Timelines
• Words
• Chart tools
• Networks
• Maps
45
![Page 46: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/46.jpg)
Data Visualization ToolsTimeline JS
http://timeline.knightlab.com/
46
![Page 47: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/47.jpg)
Data Visualization ToolsTimeliner (OKFN)
http://timeliner.okfnlabs.org
47
![Page 48: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/48.jpg)
Data Visualization ToolsWordle
http://www.wordle.net/create
48
![Page 49: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/49.jpg)
Data Visualization ToolsWordle
http://www.wordle.net/create
49
![Page 54: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/54.jpg)
Data Visualization ToolsGoogle Fusion Tables
https://www.google.com/fusiontables/data?docid=1-941Px73b_XWWn3pmPHKp6WhbbSVNiEmKMadwe0#rows:id=1
https://www.google.com/fusiontables/
54
![Page 55: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/55.jpg)
Data Visualization ToolsGoogle Fusion Tables
https://www.google.com/fusiontables/data?docid=1-941Px73b_XWWn3pmPHKp6WhbbSVNiEmKMadwe0#rows:id=1
https://www.google.com/fusiontables/
55
![Page 56: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/56.jpg)
Data Visualization ToolsGoogle Fusion Tables
https://www.google.com/fusiontables/data?docid=1-941Px73b_XWWn3pmPHKp6WhbbSVNiEmKMadwe0#rows:id=1
https://www.google.com/fusiontables/
56
![Page 57: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/57.jpg)
Data Visualization ToolsGoogle Fusion Tables
https://www.google.com/fusiontables/data?docid=1-941Px73b_XWWn3pmPHKp6WhbbSVNiEmKMadwe0#rows:id=1
https://www.google.com/fusiontables/
57
![Page 58: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/58.jpg)
Data Visualization ToolsGoogle Fusion Tables
https://www.google.com/fusiontables/data?docid=1-941Px73b_XWWn3pmPHKp6WhbbSVNiEmKMadwe0#rows:id=1
https://www.google.com/fusiontables/
58
![Page 59: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/59.jpg)
Data Visualization Tools
http://www.tableausoftware.com/public/
59
![Page 63: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/63.jpg)
Data Visualization ToolsCartoDB
http://cartodb.com/
http://developers.cartodb.com/tutorials/electoral_map.html
63
![Page 64: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/64.jpg)
Data Visualization ToolsCartoDB
http://cartodb.com/http://developers.cartodb.com/tutorials/electoral_map.html
“Shapefile”
64
![Page 73: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/73.jpg)
Developers needed
73
![Page 74: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/74.jpg)
When you’ll need a developer
‣Size‣Real-Time‣Analysis beyond Basics‣Interactivity‣API access
74
![Page 75: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/75.jpg)
References
75
![Page 76: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/76.jpg)
OKFN
http://okfn.org/http://schoolofdata.org/http://ckan.org/http://2014.okfestival.org/
76
![Page 78: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/78.jpg)
Books
78
![Page 80: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/80.jpg)
Books
80
![Page 81: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/81.jpg)
Other
http://itnig.net/en/events/revealing-the-power-of-data-for-non-developers
http://www.cccb.org/ca/marc-periodisme_de_dades-44600
http://www.visualisingdata.com/index.php/resources/
81
![Page 82: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/82.jpg)
Data Sources
http://ckan.org/http://www.theguardian.com/datahttp://www.google.com/publicdata/http://data.worldbank.org/http://datahub.io/http://www.quora.com/Data/Where-can-I-find-large-datasets-open-to-the-public
82
![Page 83: Data Tools cosystem_for_non_programmers](https://reader033.fdocuments.net/reader033/viewer/2022052410/554c54f6b4c905282a8b4ca9/html5/thumbnails/83.jpg)
Any questions?Thanks for your attention!
83