Exploring the Networks in Open Public Data
-
Upload
uldis-bojars -
Category
Art & Photos
-
view
1.103 -
download
0
description
Transcript of Exploring the Networks in Open Public Data
![Page 1: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/1.jpg)
Exploring the Networks in Open Public Data
Uldis Bojārs
Institute of Mathematics and Computer Science
University of Latvia
Using Open Data Workshop
Brussels, 20-Jun-2012
![Page 2: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/2.jpg)
About us
• Institute of Mathematics and Computer Science, University of Latvia– http://www.lumii.lv/resource/show/170
– Uldis Bojārs @CaptSolo– Valdis Krebs http://orgnet.com– Pēteris Ručevskis
![Page 3: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/3.jpg)
Network visualisation and analysis
Applications:• discover interesting patterns• explore data in [more] detail
Work from the Open Data Hackaton in Riga• analysis of Saeima voting patterns• http://opendata.lv
![Page 4: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/4.jpg)
Overview
• Data needs to be Open• Pre-processing and filtering the data– selecting what to show
• Data visualization– iterative process (visualize, refine, repeat)
• What’s next?
![Page 5: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/5.jpg)
Open Data needed first (!)“Open data is data that can be freely used, reused and redistributed by anyone …”
http://opendefinition.org/
Data needs to be:• open• easy to use
Still a problem in Latvia:• only a few datasets are open in
an easy-to-consume form (PDF does not count :)
![Page 6: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/6.jpg)
http://titania.saeima.lv/LIVS11/SaeimaLIVS2_DK.nsf/0/9DEA96450E79B7E5C2257944007E589D?OpenDocument
![Page 7: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/7.jpg)
Pre-processing
• Input:– raw vote data (scraped from the website)
published at http://data.opendata.lv/
• Output:– nodes (MPs)– edges (connections between them)
• What is a connection?
![Page 8: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/8.jpg)
Defining graph connections
• Connect MPs if they have voted similarly– disagreed on at most n% of decisions
• Filter out cases where almost allMPs voted the same
• Filter out trivial decisions
• Filter out noise
![Page 9: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/9.jpg)
Node colour legend
• Ruling coalition:– Zatler’s Reform Party– Unity– the National Alliance
• Opposition:– Harmony Centre– Greens / Farmers Party
• a few non-party MPs
![Page 10: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/10.jpg)
MPs who always vote the same (n = 0%)Connection criteria too narrow
![Page 11: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/11.jpg)
MPs who disagree in less than 35% of cases
Connection criteria too broad (everyone agrees, really?)
![Page 12: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/12.jpg)
Refining the visualisation
• Need to find the right cut-off values (n%)– where patterns [start to] appear– and the visualisation makes sense
• Show the results to domain experts– MPs, journalists, political researchers, …
• Experts:– help improve visualisations– can discover new things for themselves
![Page 13: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/13.jpg)
MPs who disagree in less than 11% of cases
Opposition parties [sometimes] vote the same
![Page 14: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/14.jpg)
MPs who disagree in less than 25% of casesBridges appear b/w position and opposition parties
(see slides 21, 22 re the bridging role of yellow nodes)
![Page 15: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/15.jpg)
What next?
• Improve our understanding of data
• Enhance visualisations– add clusters, etc.
• Create multiple visualisations– different topics, changes in time, etc.
• Bring in more data– explain nodes & edges
![Page 16: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/16.jpg)
Donations to political partieshttp://www.thenetworkthinkers.com/2011/12/innovation-happens-at-intersections.html
networkvisualisationexample #1
![Page 17: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/17.jpg)
Intra-company communication patterns
networkvisualisationexample #2
![Page 18: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/18.jpg)
Conclusion
• Need more, useful Open Data
• Discovering patterns, making sense of data– helping make sense = purpose of visualisations
• Looking forward to collaboration re:– Using Open Data– Data Visualisation and Analysis
![Page 19: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/19.jpg)
More info
• Uldis Bojā[email protected]
• Social Network Analysis talk / Valdis Krebshttp://www.slideshare.net/DERIGalway/valdis-krebs-social-network-analysis-19872007
• Smart Network Analyzer toolhttp://sna.lumii.lv/in development at IMCS, University of Latvia
![Page 20: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/20.jpg)
![Page 21: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/21.jpg)
![Page 22: Exploring the Networks in Open Public Data](https://reader033.fdocuments.net/reader033/viewer/2022061112/5456b038af79592b448b4f17/html5/thumbnails/22.jpg)