Teleimmersion

Fazel Fakhrieh

Overview

• Introduction: what is teleimmersion?

• Teleimmiersion and the grid

• Application of teleimmersion

• Teleimmersion technologies

• Teleimmersion performance requirements

Definitions of Teleimmersion

• networked/distributed/collaborative virtual environments

• “[teleimmersive applications] enable the interaction between geographically remote participants within a shared, three-dimensional space.”– - Watsen & Zyda

Definitions (cont)

• “The term Tele-immersion was first used… as the title of a workshop … to bring together researchers in distributed computing, collaboration, virtual reality and networking”– - Jason Leigh

Definitions (cont)• “[teleimmersion is] a broad multidisciplinary

research area that includes all aspects of computer science, virtual reality, virtual worlds, teleoperation and telepresence. A variety of network elements are required to scale up virtual environments to arbitrary sizes, simultaneously connecting thousands of interacting players and all kinds of information objects”– - Brutzman, Macedonia & Zyda

Definitions (cont)

• “The ultimate synthesis of networking and media technologies”– - Smith & Weingarten

Definitions (cont)

• Merging Virtual Reality with Collaboration Technology

-- Rick Stevens

VR

Aud

io

Visual

Haptics

VRA

udio

Visual

Haptics

VR

Aud

io

Visual

Haptics

• a notion of some underlying virtual world• a visual (usually 3D) representation of the

world, possibly augmented with audio and haptic representations• multiple participants in the world, aware of one

another• interactions between participants, directed to

some (possibly frivolous) end--Sam Taylor

Definitions (cont)

Virtual Reality

Concept: immersing the user in

a computer generated world that

provides suspension of disbelief

(i.e. transports the user into the world).

– Immersive high-resolution real-time stereo graphics

– Point of view rendering

– High-quality (spatialized or surround) audio

– Haptics (touch and force feedback)

– Basis for creating shared work spaces

VR

Aud

io

Visual

Haptics

Examples of Teleimmersion

• scientific visualisation - big data, big compute

• cooperative design - architects and CAD

• education - distance learning, outreach

• entertainment - multi-player games

• military - training and simulation

Teleimmersion and the grid

• Symbiotic relationship

• A key is bandwidth latency ,jitter of network

• Integration of computer and data base

• Judgment how far we are from pervasive use of teleimmersion

Application of teleimmersion

Teleimmersion application classes

• Interactive scientific visualization

• Education, training, and scenario simulation

• Art and entertainment

• Industrial design, architectural, evaluation

• Interactive scientific visualization

• Education, training, and scenario simulation

• Art and entertainment

• Industrial design, architectural, evaluation

Teleimmersion application classes (cont)

• Information visualization and data mining

• Telecollaboration environments and human factors

• Telepresence for exploration, construction, and recreation

Tele-immersion case study

Teleimmertion technologies

• Information visualization and data mining

• Telecollaboration environments and human factors

• Telepresence for exploration, construction, and recreation

• Projection based system– CAVE( Cave Automatic Virtual Environment )– ImmerDesk – PowerWall

• Monitor based system– Head-mounted displays( HMDs)– Binocular Omni-Orientation Monitors( BOOMs)

Teleimmertion technologies (cont)

personal

Information infrastructure

Desktop collaborative environment

ImmersaDeskCAVE

PowerWall

Work groupsIndividuals collaborations Institutions

Teleimmertion technologies (cont)

• CAVE

• Smaller VR devices

CAVE

• Multi-person, room-sized, high-resolution, 3D video and audio environment

• Graphics are rear-projected in stereo onto three walls and the floor and are viewed with stereo glasses

CAVE Capability

• A user is presented with dynamically moving stereo full-color at multi-thousand-pixel horizontal resolution on the walls and floor.

• The primary user’s position is tracked so that the correct perspective view is generated in real time

• The primary user can navigate with variety of intuitive navigational devices currently under test and construction.

• …

Immersive Virtual Environment

Audio Synthesizer

Audio Localizer

SpeechRecognizer

DisplayElectronics

Head/Eye/HandTrackingElectronics

Haptic/Tactilekinaestheticssystem

commands

graphics

eye position

head position

hand position

VirtualEnvironmentGenerator

UserEnvironment

head-mounteddisplay orprojectiondisplaysystem

Teleimmersion performance requirements

• Teleimmersion data flows

• VR lag and communications performance

• Performance requirement

Teleimmersion data flow

• Control• Text• Audio• Video• Tracking• Data base• Simulation• Haptics• Rendering

Tele-immersion management

Synchronization of events

Audio

Video

Tracking

Database and Event Transactions

Simulation Data

Haptic Drivers

Remote Rendering

Text

Control

Type of flow in collaborative virtual reality

VR lag and communication performance

• Caused by: Rendering, display, tracking, simulation, communication, synchronization

• Communication’s latency source:– Transmission– Bandwidth– Switching or routing– Contention– Protocol

• Asynchronous teleimmersion model

Performance requirement

• Bandwidth requirement

• Latency

• Reliability

• Multicast

Bandwidth requirements

Type Latency Bandwidth Reliable Multicast Security Streaming DynQosControl < 30 ms 64Kb/s Yes No High No LowText < 100 ms 64Kb/s Yes No Medium No LowAudio < 30 ms Nx128Kb/s No Yes Medium Yes MediumVideo < 100 ms Nx5Mb/s No Yes Low Yes MediumTracking < 10 ms Nx128Kb/s No Yes Low Yes MediumDatabase < 100 ms > 1GB/s Yes Maybe Medium No HighSimulation < 30 ms > 1GB/s Mixed Maybe Medium Maybe HighHaptic < 10 ms > 1 Mb/s Mixed Maybe High Maybe HighRendering < 30 ms >1GB/s No Maybe Low Maybe Medium

Further reading

For more information on the topics covered in this chapter, see www.mkp.com/grids also following references:

• Kalawsky’s reference text on virtual environments is comprehensive

• Two national research council reports discuss open research topics in environments and user interfaces

• Burdea provides a comprehensive survey of haptics, and Minsky a wonderful haptices bibiography

• Matlin and Foley survey human sensory and perception systems, from which many of the requirments of teleimmersive and virtual environment derive