Full panoramic representation Depth information from the real world

onstruction and Presentation of a Virtual Environment Using anoramic Stereo Images of a Real Scene and Computer Graphic Models Full panoramic representation Depth information from the real world Digitizing and representing dynamic events Construction of a large-scale virtualized environment ex. urban area, natural scenes, … Approach Jun Shimamura†, Haruo Takemura‡, Naokazu Yokoya‡, Kazumasa Yamazawa‡ †NTT Cyber Space Laboratory, ‡Nara Institute of Science and Techno logy JAPAN Requirements in constructing an immersive virtualized environment • Uses a video-rate omni-directional stereo imaging r • Constructs a full panoramic 3-D model using a cyli al panoramic stereo image sequence • Constructs a prototype system of presenting a mixed environment ( virtualized, CG ) ii. Mixed environment observed from different viewpoints iii. Superimposing dynamic event layers onto a static scene layer Original viewpoint New higher viewpoint Prototype system User’s appearance in mixed environment using CYLINDRA system Bird’s-eye view of texture-mapped 3-D static scene mod Appearanc e Geometry Omni-directional stereo imaging sensor A pair of panoramic images i. Elimination of geometric distortion in images, ii. Color adjustment of camera images, iii. Concatenation of six images for completing upper and lower omni-directional images of a stereo pair Generation of panoramic stereo images i.Hardware configuration of immersive mixed reality system Background • Single viewpoint constraint • High-resolution image acquisition • Omni-directional stereoscopic imaging at video-rate majority filtering panoramic image sequence including dynamic event static image Virtualizing a dynamic real scene • Layered representation of dynamic real scene • Depth estimation from static panoramic stereo images and moving objects Static scene image generation • A panoramic image of a static scene is generated by applying a temporal mode filter to a panoramic image sequence in a time interval • Moving objects are extracted by subtracting consecutive imag frames in a sequence Extraction of moving objects subtracting epipolar line upper panoramic image lower panoramic image world cylindrical coordinate • Stereo matching used to estimate depths of static scene and moving objects Panoramic depth map of static scene i. Has a 330-degree cylindrical screen ii. Merges virtual objects with a virtualized real world scene iii. Superimposes dynamic event layers onto static scene layer Immersive mixed reality system • Generation of 3-D model 6144 x 768 pixels resolution 13,400 polygons Background: 41,340 polygons CG tree: 54,740 polygons Total: 13 frames/sec. Frame rate: Apply Delaunay’s triangulation Extracted moving object regions Composed of twelve CCD cameras and two hexagonal pyramidal mirrors 3006 x 330 pixels top view front view camera lower component upper component pyramidal mirror lens center virtual center of lens

Upload
ferris-ross
Category

Documents
view
17
download
1

TAGS:

Embed Size (px):

description

Bird’s-eye view of texture-mapped 3-D static scene model. Jun Shimamura†, Haruo Takemura‡, Naokazu Yokoya‡, Kazumasa Yamazawa‡ †NTT Cyber Space Laboratory, ‡Nara Institute of Science and Technology JAPAN. - PowerPoint PPT Presentation

Transcript of Full panoramic representation Depth information from the real world

Page 1: Full panoramic representation Depth information from the real world

Construction and Presentation of a Virtual Environment Using Panoramic Stereo Images of a Real Scene and Computer Graphics Models

Full panoramic representation Depth information from the real world Digitizing and representing dynamic events

Construction of a large-scale virtualized environment

ex. urban area, natural scenes, …

Approach

Jun Shimamura†, Haruo Takemura‡, Naokazu Yokoya‡, Kazumasa Yamazawa‡†NTT Cyber Space Laboratory, ‡Nara Institute of Science and Technology 　 JAPA

Requirements in constructing an immersive virtualized environment

• Uses a video-rate omni-directional stereo imaging sensor

• Constructs a full panoramic 3-D model using a cylindrical panoramic stereo image sequence

• Constructs a prototype system of presenting a mixed environment ( virtualized, CG )

ii. Mixed environment observed from different viewpoints

iii. Superimposing dynamic event layers onto a static scene layer

Original viewpoint

New higher viewpoint

Prototype system User’s appearance in mixed environmentusing CYLINDRA system

Bird’s-eye view of texture-mapped 3-D static scene model

AppearanceGeometry

Omni-directional stereo imaging sensor

A pair of panoramic images

i. Elimination of geometric distortion in images,ii. Color adjustment of camera images,iii. Concatenation of six images for completing upper and lower omni-directional images of a stereo pair

Generation of panoramic stereo images

i. Hardware configuration of immersive mixed reality system

Background

• Single viewpoint constraint• High-resolution image acquisition• Omni-directional stereoscopic imaging at video-rate

majority filtering

panoramic imagesequence including dynamic event static image

Virtualizing a dynamic real scene• Layered representation of dynamic real scene

• Depth estimation from static panoramic stereo images and moving objects

Static scene image generation• A panoramic image of a static scene is generated by applying a temporal mode filter to a panoramic image sequence in a time interval

• Moving objects are extracted by subtracting consecutive image frames in a sequence

Extraction of moving objects

subtracting

epipolar line

upper panoramic image

lower panoramic image

world cylindrical coordinate

• Stereo matching used to estimate depths of static scene and moving objects

Panoramic depth map of static scene

i. Has a 330-degree cylindrical screenii. Merges virtual objects with a virtualized real world sceneiii. Superimposes dynamic event layers onto static scene layer

Immersive mixed reality system

• Generation of 3-D model

6144 x 768 pixels resolution 13,400 polygonsBackground:

41,340 polygonsCG tree:

54,740 polygonsTotal:

13 frames/sec.Frame rate:

Apply Delaunay’s triangulation

Extracted moving object regions

Composed of twelve CCD cameras and two hexagonal pyramidal mirrors

3006 x 330 pixels

top viewfront view

camera