Geographical aspects of Big Data
-
Upload
erik-van-der-zee -
Category
Data & Analytics
-
view
148 -
download
4
description
Transcript of Geographical aspects of Big Data
![Page 1: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/1.jpg)
Presentatie 2 juni 2014
IM Themadag te GeoFort (Herwijnen) Erik van der Zee (Geodan)
![Page 2: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/2.jpg)
Drs. Erik van der Zee ◦ Physical Geographer and Business Economist
◦ Senior Consultant at Geodan (www.geodan.nl)
◦ PhD Candidate “Added value of location in the Internet of Things and Smart Cities”
◦ Subject matter expert in the Geonovum pilot Making Sense for Society (#labms4s)
E-mail [email protected]
Twitter @erikvanderzee
![Page 3: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/3.jpg)
What is Big Data?
Spatial Big Data ◦ Big Vector Data (examples)
◦ Big Raster Data (examples)
Big Data Analysis
Spatial Analysis of Big Data ◦ Big Vector Data Analysis (real-time spatial analytics)
◦ Big Raster Data Analysis
Spatial Big Data Visualization (3DJS)
Spatial Big Data Tools and Technologies
![Page 4: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/4.jpg)
Wat is Big Data?
![Page 5: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/5.jpg)
Haseeb Budhani, 2008 “a blanket term for any collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications”
Gartner “high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making”
![Page 6: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/6.jpg)
![Page 7: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/7.jpg)
![Page 8: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/8.jpg)
Big Vector Data
![Page 9: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/9.jpg)
2D3D4D datasets (XYZ+Time) “Grote” Geodatasets (BGT/BAG) Lidar Pointclouds ◦ AHN2 (aerial) ◦ Laserscanners op autos (360°)
Real-time Event Streams ◦ Sensors measurements ◦ Actuator Control information ◦ Social Media APIs (twitter, etc.) ◦ “Pasjes en poortjes” (scanners bvb OV checkins)
…
![Page 10: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/10.jpg)
![Page 12: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/12.jpg)
“Internet of Things”
Real-time sensor and control data (XYZT)
![Page 13: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/13.jpg)
Sensor-Actuator Networks (Nationaal Datawarehouse Wegverkeergegevens)
Real-time data van tienduizenden sensoren elke seconde
![Page 14: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/14.jpg)
Content can be Tekst (Twitter/Whatsapp), Photos (Instagram/Flickr), videos (YouTube), sounds (Soundcloud), …etcetera
![Page 15: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/15.jpg)
Georeferenced Social Media Content
![Page 16: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/16.jpg)
Big Raster Data
![Page 17: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/17.jpg)
Steeds hogere resoluties
Steeds hogere update frequenties (Dynamic HR satellite photos and video)
Steeds meer “banden” (hyperspectral)
Overal land / op water/onder water / in de lucht
Voorbeelden ◦ Luchtfoto’s (5cm resolutie…)
◦ Gigapans
◦ Videobeelden (PTZcams, drones, cars, bodycams)
◦ 360° Panoramas
◦ Multispectral raster data (o.a. satellite imagery)
![Page 18: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/18.jpg)
Real-time Raster data
T=16:00 T=17:00 T=18:00 T=19:00
T=20:00 T=21:00 T=22:00 T=23:00
![Page 19: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/19.jpg)
Hyperspectral data cubes (High Resolution Multispectral Rasters ) = Big Data (petabytes of data)
Hyperspectral data cubes created yearmonthweekday…
![Page 20: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/20.jpg)
![Page 21: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/21.jpg)
Zeer hoge resolutie images die grote gebieden afdekken
http://www.gigapan.com/galleries/11203/gigapans/152220 (demo)
![Page 22: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/22.jpg)
E.g. Google Streetview
Cycomedia Cyclorama/aquarama
![Page 23: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/23.jpg)
DARPA’s big eye: ARGUS-IS 1.8-gigapixel camera for air surveillance ◦ clear images of objects as small as 15 centimeters
from an altitude of six kilometers ◦ One gigapixel is equal to 1,000 megapixels. For
comparison: Modern professional digital cameras have a resolution of about 20 megapixels
Also City Wide Video Surveillance… http://youtu.be/6VkKeM-OK6g?t=8m6s
![Page 24: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/24.jpg)
![Page 25: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/25.jpg)
Past – Present - Future
![Page 26: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/26.jpg)
Types of analysis (Past – Present – Future…) Naald in enorme (BIG) hooiberg vinden (bvb 1 specifiek nummerbord) Soms Trends en patronen destilleren uit berg informatie
![Page 27: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/27.jpg)
Big Vector Data Analysis
![Page 28: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/28.jpg)
(Real-time) analyse van vector data (Event Steam Processing)
Sensing Analysis Act(uat)ing raw events meaningful
events
Data creation
Controlling / alerting /
notification / routing of
objects and people
Data usage
![Page 29: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/29.jpg)
Big Raster Data Analysis
![Page 30: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/30.jpg)
Analyse van rasterdata m.b.v. beeldherkenningsalgoritmen
Meer en meer informatie kan real-time worden afgeleid uit images and videos (creates new derived data that can be analysed in event processing engine) ◦ Face recognition (ov Rotterdam)
◦ Number plates (ANPR)
◦ Traffic signs (Google)
◦ ...
![Page 31: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/31.jpg)
Gezichtsherkenning
![Page 32: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/32.jpg)
Project Ground Truth Google Feature extractie van informatie uit panorama images t.b.v. map Generation and validation
http://youtu.be/FsbLEtS0uls?t=3m52s ◦ Verkeersborden ◦ Rijrichtingen ◦ Bedrijfslogo’s ◦ Straatnamen ◦ Huisnummers ◦ …
![Page 33: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/33.jpg)
Automated Number Plate Recognition (ANPR)
![Page 34: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/34.jpg)
Spatial Big Data Visualization
![Page 35: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/35.jpg)
Trend van visualiseren van BI (big) Data in kaarten (bvb o.b.v. postcode / adres / woonplaats / wijk) naar Ruimtelijke Analyse (spatial analysis) van gegeorefereerde BI Data
![Page 36: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/36.jpg)
Big Data needs New User Interfaces
![Page 37: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/37.jpg)
Nieuwe visualisatie mogelijkheden Voorbeelden http://d3js.org/
![Page 38: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/38.jpg)
Spatial Big Data Tooling
![Page 39: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/39.jpg)
Spatial Extensions voor NoSQL Databases ◦ Neo4J Spatial Extensions (graph database)
◦ MonetDB Spatial Extensions (column store)
◦ Spatial Hadoop http://spatialhadoop.cs.umn.edu/
Spatial Event Stream Processing engines ◦ Esri GeoEvent Processor
◦ Oracle Spatial CEP Engine
◦ Microsoft Stream Insight…
![Page 40: Geographical aspects of Big Data](https://reader034.fdocuments.net/reader034/viewer/2022042813/547e435db379596f2b8b5431/html5/thumbnails/40.jpg)
Vragen?