Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf ·...
Transcript of Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf ·...
![Page 1: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/1.jpg)
Multimodal Location Based ServicesSemantic 3D City Data as Virtual and
Augmented Reality
Jochen Wendel Alexander Simons
Andreas Koch
José Miguel Santana Agustín Trujillo
José Pablo Suárez
![Page 2: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/2.jpg)
Collaboration
2
![Page 3: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/3.jpg)
Outline• Project Goals
• Urban and energy-related data models
• Tools: The Glob3 Mobile framework
• Representing a CityGML model in 3D
• Multimodal approach
• Virtual Globe View
• Virtual Reality Mono
• Virtual Reality Stereo
• Augmented Reality
• Use case: Visualising solar irradiation
• Conclusions
3
![Page 4: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/4.jpg)
Project Goals• Prototype an interactive and mobile system for exploring:
• Semantic building / urban representational models (e.g. CityGML)
• Energy related data sets.
• Explore the possibilities of multi-LoD maps, Virtual Reality and Augmented Reality for the visualisation of 3D urban models and temporal data-series.
• Create a iOS app, that serves as demonstration of the service.
4
![Page 5: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/5.jpg)
Urban and energy-related data models• CityGML: Information model for urban objects representation.
• Extension of GML (from OGC)
• Classes, hierarchies, aggregations, relations, and properties.
• Used on 3D virtual cities, analysis, simulation…
• Extendable via Application Domain Extensions (ADE)
• Buildings represented in different levels of detail.
5
![Page 6: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/6.jpg)
• Karlsruhe Model: CityGML LoD 2
• CityStats, Heat D./L.:Energy-related data per building
• SolarB: Temporal data-series representing energy simulations
6
Urban and energy-related data models
![Page 7: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/7.jpg)
Urban and energy-related data models• Karlsruhe model
• +30.000 buildings [LoD2] (from a total set of 80.000).
• +230.000 surfaces [Grounds, Walls and Rooftops]
• Building parameters:
• Heat demand, Volume, GHG Emissions, solar gains, 50 demographic variables…
• High resolution textured models of emblematic buildings and monuments.
7
![Page 8: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/8.jpg)
Glob3 Mobile• Open-source framework for Map & Virtual Globe Applications.
• Native implementations for iOS, Android and Web.
• Developed by CTIM (ULPGC) and IGO Software.
• Available: https://github.com/glob3mobile/g3m
• Automatic Multiplatform Porting
8
![Page 9: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/9.jpg)
G3mFeatures
Manykindsofscenario
9
![Page 10: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/10.jpg)
G3mFeatures2D/2.5D/3DInteractiveSymbology
10
![Page 11: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/11.jpg)
CityGMLto3D
• OpenGL / WebGL only understand triangles!
• The data received via CityGML needs to be translated into a triangulated mesh model to be displayed.
• Model storage can be local or on a remote server.
• Karlsruhe model ~= 400 MB.
• Fetching, parsing and tessellation are executed on parallel threads.
11
![Page 12: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/12.jpg)
12
CityGML Building
Surfaces
Energy DataBounding Vol. and
Render Info
IDs
• SAX Parsing of the CityGML documents:
• CityGMLBuilding class model:
CityGMLto3D:Parsing
STATICDATA
XML SAX Parsing
JSON Parsing
CITYGMLMODEL
![Page 13: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/13.jpg)
CityGML Building
Surfaces
Energy DataBounding Vol. and
Render Info
IDs
• Creating a single triangular mesh of buildings.
• For a high number of models, batched drawing is essential [x49]
• Gouraud shading was used as illumination model.
13
VerticesTessellation
Semantical colouring
Visibility TestIndices
Normal Extraction Vertex Normals
Vertex ColorColour Palette Legend
CityGMLto3D:Tessellation
City Model Mesh
![Page 14: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/14.jpg)
• Visibility test, eliminating shared walls.
• Proximity based limiting the O(n!) complexity.
• EPSG:4326 to Cartesian World Coordinates
• Addition of DEM displacement
• Ear Clipping Algorithm for wall tessellation
• Computationally expensive algorithm [O(n2)]
• 91% of tessellated surfaces are 4 edged.
14http://www.sunshine2k.de/coding/java/Polygon/Kong/
Kong.html
CityGMLto3D:Tessellation
![Page 15: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/15.jpg)
• Classic GIS 3D map approach.
• Multitouch navigation.
• Buildings are selectable showing their associated data.
• OpenStreet Map as base layer. [Mapnik]
15
MultimodalApp:VirtualGlobe
![Page 16: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/16.jpg)
MultimodalApp:VirtualGlobe
16
VIDEO INTERACTING WITH THE MAP
SELECTING BUILDINGAND CHANGING CITY
COLOURING
![Page 17: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/17.jpg)
MultimodalApp:VirtualReality
• Sensors output of iOS, Android and Javascript.
• Control the UI rotations.
• Device location given by GPS.
17
GeolocalizedReferenceSystem
Multiplatformdevice-attituderepresentations
![Page 18: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/18.jpg)
MultimodalApp:VirtualReality
VIDEO OF VR MONO AT WORK
18
![Page 19: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/19.jpg)
MultimodalApp:VRStereo• Stereorenderingofthescene
• Low-levelOpenGLcodingfromscratch.• Stereoparallelprojection.• Usinga1:1scalefortheinteraxial.• Stereofrustumcullingtoimproveperformance.
19
![Page 20: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/20.jpg)
MultimodalApp:VRStereo
• UseofCardboard-likeVRheadsets.
20
![Page 21: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/21.jpg)
MultimodalApp:AugmentedReality
• DeviceAttitudeTracking+RealImageMerging• Phone’scamerahastobemathematicallyrepresented.• IntrinsicparametersobtainedwithOpenCV.
21
IntrinsicC.Matrix ExtrinsicCameraParameters
![Page 22: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/22.jpg)
MultimodalApp:AugmentedReality
22
MULTIFRAMEBUFFERRENDERING
![Page 23: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/23.jpg)
Usecase:SolarB–Solarirradiancemodel
• Numericalsimulationoftheamountofsunlightcapturedbythebuildingsurfaces.• BasedonŠúri,M.andHofierka,J.(2004)
• Finalresult:TemporaldataseriesofSolarradiationpertimeintervalforsurfacesandbuildings[kWh]• Surfacesrepresentedaspointsetswitharesolutionof1p/m2.
23
a)Buildingpart classification b)Pointgrid generation c)Shadowing analysis d)Aggregationof results
![Page 24: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/24.jpg)
Usecase:SolarB–Solarirradiancemodel
• Example:TheEIFERofficesinKarlsruhe(7stories)• 14surfaces[3071points]• 1yearofsolarirradiationdata[8760timestamps]
24
VIDEO SELECTING BUILDING, ACTIVATING POINT CLOUD AND ZOOMING AROUND ON MAP MODE
![Page 25: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/25.jpg)
Conclusions
• Theprototypedemonstratesfeasibletoapplymobilecomputergraphicstechnologiestothestudyofmassiveurbandatasets.
• Theappisabletodirectlyretrievesemanticbuildingdatafromenergysimulations/modelsinastandardisedway(OGC,CityGMLstructure).
• VRandARareusefultechniquesforfieldexpertsonenergyresearchandinfrastructuremaintenance.
• Formediumsizedcities,multi-level-of-detailisnotnecessary.• GPS&CompassaccuracyforVRandspeciallyforARisstillunreliableonmanyurbansituations.• Alternativepositioningtechniquescouldachievebetterresults.
25
![Page 26: Multimodal Location Based Serviceslbs2016.lbsconference.org/wp-content/uploads/2016/09/p11_1.pdf · Project Goals • Prototype an interactive and mobile system for exploring: •](https://reader034.fdocuments.net/reader034/viewer/2022042223/5ec9aed862518453642ce1d4/html5/thumbnails/26.jpg)
Thanksforyourattention
26
Jochen Wendel Alexander Simons
Andreas Koch
José Miguel Santana Agustín Trujillo
José Pablo Suarez