Post on 21-Dec-2015
Virtual Worlds Lab
Testbed for Mobile Augmented Testbed for Mobile Augmented Battlefield VisualizationBattlefield Visualization
September, 2003September, 2003
William Ribarsky and Nickolas FaustGVU Center and GIS Center
Georgia Institute of Technology
Virtual Worlds Lab
Matrix of Proposed Activities and ResultsProposedTasks
What Was Done Done This Year Future Work
Multimodal3D Interaction
Development andevaluation of gestureand voice interface;testing of otherinterfaces
Implementation and use ofnew interface in mobileenvironment
MobileVisualization
Define and initialimplementation ofmobile situationalvisualization
More completeimplementation,evaluation, and use
Extensive use andsharing of system,data
4D Modeling Initial automated treemodeling
Modeling large collectionsof accurately placedbuildings
Joint multisourcemodeling efforts
Dynamic,UniversalDataStructures
Initial paged object datastructure for simplebuildings
Dynamic, scalablestructure for handlingobjects from multiplesources
Completescalability withefficient access
InteractiveRenderingandVisualization
Initial work on simplebuilding LODs andintegration with terrain,texture
New view-dependent,appearance preservingrendering techniques
New techniques forlarge collections ofobjects and newmanipulation
CollaborationandIntegration
Development of initialLOD approach. Flexibledevelopment of VGISenvironment
Initial implementation of atestbed for mobileaugmented
Multilevelintegration andsharing
TechnologyTransfer
Work with Army,Sarnoff Corp., NRL
Homeland Defense andother outreach efforts
Army, DARPA,Homeland Defense
Virtual Worlds Lab
Mobile Situational VisualizationAn extension of situation awareness that exploits and integrates
interactive visualization, mobile computing, wireless networking, and multiple sensors:
• Mobile users with GPS, orientation sensing, cameras, wireless• User carries own 3D database• Servers that store and disseminate information from/to multiple
clients (location, object/event, weather/NBC servers)• Location server to manage communications between users and
areas of interest for both servers and users• Ability to see weather, chem/bio clouds, and positions of other
users• Accurate overviews of terrain with accurately placed 3D buildings• Ability to mark, annotate, and share positions, directions, speed,
and uncertainties of moving vehicles or people• Ability to access and playback histories of movement• Placement of multiresolution models from MURI team members into
environment
Results of real-time collection of GPS path at night (left); screen shot with annotated path in red (right).
New lightweight wearable system
GPS and orientation tracker
Virtual Worlds Lab
Weather/Atmospheric Server
Annotation ServerAnnotated views with updated user location and orientation
Spread of dynamic Sarin gas cloud with positions of first responders
Mobile Situational Visualization
Accurate Shared Locations
Virtual Worlds Lab
Mobile Situational Visualization System
Drawing Area
Buttons Pen Tool
Mobile Team
Collaboration Example
collaborators
Shared observations of vehicle location, direction, speed
Virtual Worlds Lab
Collaborative Environment
• Everybody has a location in space and time in the Virtual World
• Geographic server lookup approach– Users– Location Servers– Data Servers
Weather Server
UserUser
User
Location Server
Traffic ServerAnnotation Server
GeoData Server
Virtual Worlds Lab
• Everybody has a location in space and time in the Virtual World
• Geographic server lookup approach– Users– Location Servers– Data Servers
UserUser
User
Location Server
Traffic ServerAnnotation Server
GeoData Server
Weather Server
Collaborative Environment
Virtual Worlds Lab
What is Novel and Compelling AboutMobile Situational Visualization?
Mobile battlefield visualization was an original proposed (and accepted) task. That’s pretty compelling!
But, beyond thatInstant placement of environmental activity information within the geospatial environment combined with fast sharing and use are novel and compelling.
-Fast, accurate, and specific annotation of activity information (both user-controlled and automated logging)
-Immediate updates of databases with this information-Server structure for sharing this with collaborators or commanders in the area of interest
-Use in computations and simulations (some launched automatically)
Virtual Worlds Lab
Matrix of Proposed Activities and ResultsProposedTasks
What Was Done Done This Year Future Work
Multimodal3D Interaction
Development andevaluation of gestureand voice interface;testing of otherinterfaces
Implementation and use ofnew interface in mobileenvironment
MobileVisualization
Define and initialimplementation ofmobile situationalvisualization
More completeimplementation,evaluation, and use
Extensive use andsharing of system,data
4D Modeling Initial automated treemodeling
Modeling large collectionsof accurately placedbuildings
Joint multisourcemodeling efforts
Dynamic,UniversalDataStructures
Initial paged object datastructure for simplebuildings
Dynamic, scalablestructure for handlingobjects from multiplesources
Completescalability withefficient access
InteractiveRenderingandVisualization
Initial work on simplebuilding LODs andintegration with terrain,texture
New view-dependent,appearance preservingrendering techniques
New techniques forlarge collections ofobjects and newmanipulation
CollaborationandIntegration
Development of initialLOD approach. Flexibledevelopment of VGISenvironment
Initial implementation of atestbed for mobileaugmented
Multilevelintegration andsharing
TechnologyTransfer
Work with Army,Sarnoff Corp., NRL
Homeland Defense andother outreach efforts
Army, DARPA,Homeland Defense
Virtual Worlds Lab
Integrated, Comprehensive Modeling
•To build comprehensive models, we need a range of modeling techniques.
•We also should combine techniques for richer and more complete models.
Model Detail
Geo
-acc
urac
y
Low Mid High
Low
Mid
Hig
h
Georgia Tech
Thousands to tens of thousands of buildings and treesHundreds of semi-automatically modeled buildings
USCTens to Hundreds of semi-automatically modeled buildings
BerkeleyHundreds of automatically modeled buildings
Virtual Worlds Lab
Integrated, Comprehensive Modeling
•To build comprehensive models, we need a range of modeling techniques.
•We also should combine techniques for richer and more complete models.
Detail
Geo
-acc
urac
y
Low Mid High
Low
Mid
Hig
h
Integrated, comprehensive models with combined techniques
Virtual Worlds Lab
New Results on Modeling Large Collections
•Generic models extruded from accurate footprints with accurate locations. (11,000 automatically generated from insurance GIS databases).
-Complete models with roofs-Generic façade textures-Databases available for automatically generating whole city (hundreds of thousands)
•Automatic generation of accurately located tree models (thousands) from high-res imagery.
•Creation of hundreds of specific buildings using commercial or self-developed (semi-automatic) software.
(individual 3D buildings have brown roofs)
3D CAD modeled objects on high resolution terrain
Virtual Worlds Lab
Automatic Identification and Placementof Trees, Shrubs, and Foliage
This can be used with Ulrich Neuman’s or Avideh Zakhor’s results to automatically identify, remove, and model foliage.
Virtual Worlds Lab
Automated identification and modeling of trees
Application to Tree Modeling
Accurate placement of 3D modeled trees
Virtual Worlds Lab
Matrix of Proposed Activities and ResultsProposedTasks
What Was Done Done This Year Future Work
Multimodal3D Interaction
Development andevaluation of gestureand voice interface;testing of otherinterfaces
Implementation and use ofnew interface in mobileenvironment
MobileVisualization
Define and initialimplementation ofmobile situationalvisualization
More completeimplementation,evaluation, and use
Extensive use andsharing of system,data
4D Modeling Initial automated treemodeling
Modeling large collectionsof accurately placedbuildings
Joint multisourcemodeling efforts
Dynamic,UniversalDataStructures
Initial paged object datastructure for simplebuildings
Dynamic, scalablestructure for handlingobjects from multiplesources
Completescalability withefficient access
InteractiveRenderingandVisualization
Initial work on simplebuilding LODs andintegration with terrain,texture
New view-dependent,appearance preservingrendering techniques
New techniques forlarge collections ofobjects and newmanipulation
CollaborationandIntegration
Development of initialLOD approach. Flexibledevelopment of VGISenvironment
Initial implementation of atestbed for mobileaugmented
Multilevelintegration andsharing
TechnologyTransfer
Work with Army,Sarnoff Corp., NRL
Homeland Defense andother outreach efforts
Army, DARPA,Homeland Defense
Virtual Worlds Lab
Organizing Large Collections of 3D Models for Interactive Display
•Merging of different types and formats•Automated replacement of structures for overlapping areas
Common format and organization for different types
Q
Q QQ Q
Q
Q QQ Q
Q
Q QQ Q
Q QQ Q
Linked Global Quadtrees
Virtual Worlds Lab
Paging, Culling, and Fast Rendering
Quadcell
Block
Block
QQQQ
Linked global quadtree
Block
BlockBlock
Block
Out-of core Storage
Virtual Worlds Lab
Matrix of Proposed Activities and ResultsProposedTasks
What Was Done Done This Year Future Work
Multimodal3D Interaction
Development andevaluation of gestureand voice interface;testing of otherinterfaces
Implementation and use ofnew interface in mobileenvironment
MobileVisualization
Define and initialimplementation ofmobile situationalvisualization
More completeimplementation,evaluation, and use
Extensive use andsharing of system,data
4D Modeling Initial automated treemodeling
Modeling large collectionsof accurately placedbuildings
Joint multisourcemodeling efforts
Dynamic,UniversalDataStructures
Initial paged object datastructure for simplebuildings
Dynamic, scalablestructure for handlingobjects from multiplesources
Completescalability withefficient access
InteractiveRenderingandVisualization
Initial work on simplebuilding LODs andintegration with terrain,texture
New view-dependent,appearance preservingrendering techniques
New techniques forlarge collections ofobjects and newmanipulation
CollaborationandIntegration
Development of initialLOD approach. Flexibledevelopment of VGISenvironment
Initial implementation of atestbed for mobileaugmented
Multilevelintegration andsharing
TechnologyTransfer
Work with Army,Sarnoff Corp., NRL
Homeland Defense andother outreach efforts
Army, DARPA,Homeland Defense
Virtual Worlds Lab
Handling Complicated Models
Bounding box
Selected LOD
View-Dependent LOD for large collections of complicated models
Q
Q QQ Q
Q
Q QQ Q
Q
Q QQ Q
Q
N LevelsLinked Global Quadtrees
Results of view-dependent simplification. The blue box is the viewing window; fully textured models with and without meshes displayed are shown on the left and right, respectively. (Top) Full resolution mesh and textures within the window. (Bottom) Significantly reduced resolution mesh and textures within the window without reduction in visual quality.
Viewpoint
Virtual Worlds Lab
Quadric Error Approach to Simplification
•Initial development Garland and Heckbert, SIGGRAPH, 1997•Quadric approach yields “optimal” simplification by permitting generalized contractions between vertices and keeping track of the deviation from the original mesh
v1 v2
contractionv
)v(
vQvT Use quadric matrix to find a vertex with error within ε; Δ is the surface at error
value ε.
v1 v2
generalcontraction
Non-topological simplification
v
Virtual Worlds Lab
Limitations on Basic Quadric Approach
•No concept of view-dependence and continuous LOD•No structure for large collections of objects•Geometry error metric; no appearance-preserving metric (e.g., for textures, shading, lighting). A combined metric is best.
full resolution w/o appearance metric
with appearance metric
Application of appearance-preserving metric to a textured object (Cohen et al., SIGGRAPH 98)
[ ]
Virtual Worlds Lab
View-Dependent Continuous LOD Tree
The vertex front is circled. Green nodes are active-interior, blue nodes are active-boundary, and orange nodes are inactive. Here, vertex V7 is split and vertices V10 and V11 are merged.
V10 V11
V5
V8 V9
V4
V2
V1
V14 V15
V7V6
V3
Simplification Pass:split: V7; merge: V10, V11
V10 V11
V5
V8 V9
V4
V2
V1
V14 V15
V7V6
V3
V10V11
V14
V15V8 V9
V6
V4
V10V11
V7
V6
V4
V5
V14
V15
V6
(a)
(b) (c)
Simp Pass
The pink, purple, and dark gray triangles are subfaces of V7, V5, and V4, respectively. (a) Full mesh. (b) Tree on left. (c) Tree on right.
Block
Façade 1 …
Façade N
LOD Hierarchy
…
…
…
Object M
…
…
…
Object 1
…
…
…
…
…
…
…
QQQQ
Linked global quadtree
Virtual Worlds Lab
View-Dependent Appearance-Preserving Simplification
Collapse Distance Deviation
possible surface Mi
current surface Mi-1
original surface M0
deviation vectors
Va Vb
Vc
PO
PC
Two-Way Incremental Distance Deviation
Va Vb
Vc
(A)
(B)
Quadric Error Deviation
Va Vb
Vc
(C)
Va Vb
Vc
P
C
One-Way Incremental Texture Deviation
(D)
Va Vb
Vc
P
A
Two-Way Incremental Texture Deviation
P
BP
C
(E)
(F)
V
a
V
b
V
c
P
O
Total Texture Deviation
Virtual Worlds Lab
Matrix of Proposed Activities and ResultsProposedTasks
What Was Done Done This Year Future Work
Multimodal3D Interaction
Development andevaluation of gestureand voice interface;testing of otherinterfaces
Implementation and use ofnew interface in mobileenvironment
MobileVisualization
Define and initialimplementation ofmobile situationalvisualization
More completeimplementation,evaluation, and use
Extensive use andsharing of system,data
4D Modeling Initial automated treemodeling
Modeling large collectionsof accurately placedbuildings
Joint multisourcemodeling efforts
Dynamic,UniversalDataStructures
Initial paged object datastructure for simplebuildings
Dynamic, scalablestructure for handlingobjects from multiplesources
Completescalability withefficient access
InteractiveRenderingandVisualization
Initial work on simplebuilding LODs andintegration with terrain,texture
New view-dependent,appearance preservingrendering techniques
New techniques forlarge collections ofobjects and newmanipulation
CollaborationandIntegration
Development of initialLOD approach. Flexibledevelopment of VGISenvironment
Initial implementation of atestbed for mobileaugmented
Multilevelintegration andsharing
TechnologyTransfer
Work with Army,Sarnoff Corp., NRL
Homeland Defense andother outreach efforts
Army, DARPA,Homeland Defense
Virtual Worlds Lab
Implementing and Using the Testbed
•Merging of tens of thousands (and more) of models from multiple sources.
•Efficient organization and culling of massive collections of 3D objects.
•Integration of view-dependent methods for accurate and efficient display of complex models.
•Deployment and use of mobile situational visualization capability.
Virtual Worlds Lab
Technology Transfer
•The VGIS visualization system with capabilities developed here (including mobile visualization) was a key part of the Georgia Tech Homeland Defense Workshop and will be part of the GT Homeland Defense Initiative with support at the State and National levels.
•The system is being used as part of the Sarnoff Raptor system, which is deployed to the Army and other military entities. In addition our visualization system is being used as part of the Raptor system at Scott Air Force Base.
•We are in discussion with the Department of the Interior on use of our mobile situational visualization capability to develop Anytime-Anywhere information system resource accessibility for countering asymmetric threats.
Virtual Worlds Lab
Plans for Next Year
•Full deployment of mobile situational visualization capability with sharing of the system and the results with team members.
•Further development of automated model building from multisource data. This will be a collaborative effort with other team members. We will move towards a robust system with ability to merge and increment model sets and update models (adding improvements to make generic models more detailed and specific as data are available).
•Development of fully scalable 3D object organization and interactive visualization capability extending to hundreds of thousands of accurately located buildings and trees (or more).
•Full integration of view-dependent capability for complex models.