Advanced Computer Graphics CSE 190 [Winter 2016], Lecture 2 Ravi Ramamoorthi ravir.
CSE 190: Virtual Reality Technologies
Transcript of CSE 190: Virtual Reality Technologies
![Page 1: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/1.jpg)
CSE 190: Virtual Reality TechnologiesLECTURE #19: PANORAMA IMAGING
![Page 2: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/2.jpg)
Upcoming DeadlinesSunday, June 6: Project 4 due
June 7+8: Final exam Monday 11:30am – Tuesday 11:30am
2
![Page 3: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/3.jpg)
App PresentationsBaily Chen
◦ Minecraft VR
Juan Ramirez◦ Super Hot VR
3
![Page 4: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/4.jpg)
Final ExamExam window: Monday 11:30am – Tuesday 11:30am
Exam duration: 3 hours (continuous block)
Exam type: written exam (Canvas Quiz)
Open book, open internet
No consulting of other people (local or remote)
No posting or discussing exam questions or answers
![Page 5: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/5.jpg)
Panoramic Photos and Video
![Page 6: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/6.jpg)
360˚ PhotosA.k.a. panoramic photographs, surround images, image spheres
360˚ photos simulate being in the shoes of a photographer and looking around to the left, right, up and down as desired as well as sometimes zooming.
Popular example: Google Streetview◦ https://www.google.com/streetview/
![Page 7: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/7.jpg)
VR Video FormatsFixed view 3D stereo videos are typically stored side-by-side:
360 degree 3D stereo videos are stored in over-under format:
![Page 8: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/8.jpg)
360˚ Video on YoutubeYoutube VR videos can be viewed with almost any VR device:
◦ Google Cardboard, Daydream, Gear VR, Oculus, Playstation VR, HTC Vive
Youtube supports 360 degree videos◦ uses Mercator projection◦ 3D stereo in over-under format with
up to 8192 x 8192 pixels resolution
Example: City tour of Rome (monoscopic 360 degree video)
https://www.youtube.com/watch?time_continue=93&v=1ziMH_lAUW0&feature=emb_logo
![Page 9: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/9.jpg)
Panorama Capture Devices
![Page 10: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/10.jpg)
SmartphoneMost smartphones have panorama photo capture modes/apps
Photos are 360 degrees but monoscopic
![Page 11: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/11.jpg)
Samsung Gear 360Two versions: released in 2016 ($350) and 2017 ($230)
360˚ Photos:◦ Dual Lens: 25.9 MP (7200 x 3600) (2016)
or 15MP (5472 x 2736) (2017)
360˚ Video:◦ Dual Lens: up to 4096 x 2048 (24fps)
2017 version has better picture quality
Sensors:◦ Gyroscope, Accelerometer
Storage: microSD
IP53 Dust and Splash-proofV1 (2016) V2 (2017)
![Page 12: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/12.jpg)
Vuze XRPhotos: 6000 x 3000 pixels
Video: 5.7K@30 fps
Storage: microSD
Lenses: 2x F/2.4 210⁰ fisheye lenses
Sensors: 2 x Sony 12MP
Price: $400
![Page 13: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/13.jpg)
Google VR180Limited to 180 degree FOV
Advantage: much easier to shoot◦ Camera people don’t need to hide◦ Camera and audio equipment can be
used almost like in traditional video production
![Page 14: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/14.jpg)
Vuze+Spherical Resolution: 4K (per eye)Frame rate: 30fps for 3D or 60fps for 2DSensors: 8 Sony FHD image sensorsLenses: 8x F/2.4 fisheye lensesMedia FOV: 360°x180° (Full Spherical)Price: $700
![Page 15: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/15.jpg)
Nokia OzoReleased 2015
Discontinued 2017
Price: $45,000
8 lenses
3D 360 degree stitching
![Page 16: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/16.jpg)
Samsung 360 RoundPrice: $10,500
17 cameras with 2MP image sensor and F1.8 Lens
◦ 16 horizontal, 1 up camera
3D Video 3D: 4096 x 2048 at 30fps per eye
6 microphones for spatial audio
IP65 Splash and Dust Resistant
Weight: 4.3 lbs
![Page 17: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/17.jpg)
3D Video: Google Jump Yi HaloPrice: $20,000
16 horizontal cameras + 1 up camera
Sensors: Sony IMX377, 1/2.3", 12 megapixels CMOS
Lenses: F2.8 aperture / 155° wide-angle
Omni-directional microphone
Battery: ATL 93Wh high density lithium polymer battery, battery life 100 minutes in video recording
Video and photo resolution: ◦ 8192x8192 @30fps
![Page 18: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/18.jpg)
CAVECamFor full 360° by 180° PanoramasBy UCSD’s Drs. Tom DeFanti and Dan Sandin
![Page 19: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/19.jpg)
Panorama Capturing
![Page 20: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/20.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
![Page 21: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/21.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
![Page 22: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/22.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
![Page 23: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/23.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
![Page 24: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/24.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
1 center of projection!
![Page 25: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/25.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
1 center of projection!
![Page 26: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/26.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
1 center of projection!
center of projection
![Page 27: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/27.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
1 center of projection!
![Page 28: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/28.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
1 center of projection!
2 centers of projection!
![Page 29: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/29.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
1 center of projection!
2 centers of projection!
![Page 30: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/30.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
1 center of projection!
2 centers of projection!
![Page 31: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/31.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
1 center of projection!
2 centers of projection!
![Page 32: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/32.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
1 center of projection!
2 centers of projection!
![Page 33: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/33.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
1 center of projection!
2 centers of projection!
![Page 34: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/34.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
1 center of projection!
2 centers of projection!
![Page 35: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/35.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
1 center of projection!
2 centers of projection!
![Page 36: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/36.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
1 center of projection!
2 centers of projection!
![Page 37: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/37.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
1 center of projection!
2 centers of projection!
multiple centers of projection
![Page 38: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/38.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
1 center of projection!
2 centers of projection!
multiple centers of projection
![Page 39: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/39.jpg)
Panorama v Stereo Movie v Stereo Panorama
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
1 center of projection!
2 centers of projection!
multiple centers of projection
![Page 40: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/40.jpg)
Panorama v Stereo Movie v Stereo Panorama
Light Field!
Stereo Panoramastereo & head rotation
multiple centers of projection
![Page 41: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/41.jpg)
Panoramamono & head rotation
Stereostereo & no head rotation
Stereo Panoramastereo & head rotation
horizontal-only parallax
Panorama v Stereo Movie v Stereo Panorama
![Page 42: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/42.jpg)
Stitching
![Page 43: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/43.jpg)
Pan
oram
as
Slid
es fr
om M
arc
Levo
y’s
exce
llent
CS
178
cou
rse
![Page 44: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/44.jpg)
Pan
oram
as
Slid
es fr
om M
arc
Levo
y’s
exce
llent
CS
178
cou
rse
![Page 45: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/45.jpg)
Pan
oram
as
Slid
es fr
om M
arc
Levo
y’s
exce
llent
CS
178
cou
rse
![Page 46: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/46.jpg)
Omnistereo
![Page 47: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/47.jpg)
OmnistereoCan we display a 360 degree stereo panorama image in a surround screen virtual environment such as a CAVE so that multiple users can see a correct 3D image in all directions, without head tracking?
![Page 48: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/48.jpg)
Comparison: Mono and Stereo Panoramas
Central, a.k.a. Mono
Omnistereo,Multiperspective
Peleg et al. 2001side
by
Har
iLak
shm
an(E
E 3
68)
![Page 49: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/49.jpg)
A Pair of Mono Panoramas
O1 O2
Baseline = |O1-O2|in this viewing direction,
i.e., stereo possible
Baseline = 0in this viewing direction,
i.e., no stereo side
by
Har
iLak
shm
an(E
E 3
68)
![Page 50: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/50.jpg)
Head Rotation
could be IPDInter Pupillary Distance
side
by
Har
iLak
shm
an(E
E 3
68)
![Page 51: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/51.jpg)
Image Pair for Each Direction
Left im Right im Left im Right im
Store image pair for each direction Problem: Too much dataside
by
Har
iLak
shm
an(E
E 3
68)
![Page 52: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/52.jpg)
Approximation: Store only Middle Ray
Approximation: store only middle ray for L and R eyes for each direction
Omnistereo FormatPeleg et al. 2001
side
by
Har
iLak
shm
an(E
E 3
68)
![Page 53: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/53.jpg)
Omnistereo PanoramasLeft panorama
Right panorama
Omnistereo FormatPeleg et al. 2001
side
by
Har
iLak
shm
an(E
E 3
68)
![Page 54: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/54.jpg)
Omnistereo: Geometry Details
Left panorama
Right panorama
Q: scene pointd: distance to Q from originv: viewing circle radius f: imaging cylinder radiusL, R: coordinates of Q in panoramas
Q
Q
side
by
Har
iLak
shm
an(E
E 3
68)
![Page 55: CSE 190: Virtual Reality Technologies](https://reader031.fdocuments.net/reader031/viewer/2022012517/6191fa641b917216c2406c69/html5/thumbnails/55.jpg)
Left panorama
Right panorama
Disparity
Sphere-to-planedistortions
Omnistereo example
side
by
Har
iLak
shm
an(E
E 3
68)