Selecting User-Generated Videos for Augmented Reality Applications
1
Selecting User-Generated Videos for Augmented Reality Applications Hien To, Hyerim Park, Seon Ho Kim, and Cyrus Shahabi Integrated Media Systems Center University of Southern California, Los Angeles, CA BigData in Social Media IMSC Retreat 2016 Introduction Mobile cameras are everywhere! Any way to use mobile videos as AR content? Motivation • Popularity of augmented-reality (AR) AR browsers, e.g., Layar, Wikitude, and Junaio • Need content source with rich context information Scarcity of AR content Lack of user contexts causes imprecise registration Challenges • Hard to search for interesting videos for AR content Proposed Approach Spatial metadata Identify interesting videos • Sequence of FOVs that follow a particular camera shooting pattern Modeling significant video segments References • Incorporating Geo-Tagged Mobile Videos Into Context- Aware Augmented Reality Applications , IEEE BigMM 2016. • GeoUGV: User-Generated Mobile Video Dataset with Fine Granularity Spatial Metadata, MMSys 2016. Results 1. Significant Video Segments 2. Detecting Hot Spots Conclusion Proposed models and algorithms to identify interesting video segments and hotspots The algorithms are fast and able to identify interesting scenes Will consider user mobility and ranks of video segments p R Field Of View (FOV) p: camera location d: camera direction : viewable angle R: viewable distance t: timestamp Single Multiple Single Zoom Pan Multiple Track Arch Positio n Directio n 0 ¿ , > ¿ ¿ , , > ¿ ¿ , , > ¿ ¿ , > ¿ Number of scenes found TrackArch Zoom Pan 0 500 1000 1500 2000 Number of scenes Searching time (one video) Track Arch Zoom Pan 0 5 10 15 20 25 30 Runtime (ms) Tracking is the most popular way of capturing mobile videos Overall search time is fast Hotspots covered by many video frames 0 200 400 600 800 1000 Minimum FOVs per hotspot Total number of hot spots Real Dataset Statistics Value Number of videos with geo- metadata 2,397 Average length per video with content (sec) 72.14 Number of users 289 Number of FOVs 208,978 Zoom Pan Track Arch 1 d 2 d 3 d 4 d 5 d 1 d 2 d 3 d 4 d 5 d 1 d 2 d 3 d 4 d 5 d 1 d 2 d 3 d 4 d 5 d 3 4 5
-
Upload
university-of-southern-california -
Category
Science
-
view
26 -
download
1
Transcript of Selecting User-Generated Videos for Augmented Reality Applications
Selecting User-Generated Videos for Augmented Reality Applications
Hien To, Hyerim Park, Seon Ho Kim, and Cyrus ShahabiIntegrated Media Systems Center
University of Southern California, Los Angeles, CA
BigData in Social Media
IMSC Retreat 2016
IntroductionMobile cameras are everywhere!Any way to use mobile videos as AR content?
Motivation• Popularity of augmented-reality (AR)
AR browsers, e.g., Layar, Wikitude, and Junaio• Need content source with rich context information
Scarcity of AR contentLack of user contexts causes imprecise registration
Challenges• Hard to search for interesting videos for AR content
Proposed ApproachSpatial metadata
Identify interesting videos• Sequence of FOVs that follow a particular camera
shooting pattern
Modeling significant video segments
References• Incorporating Geo-Tagged Mobile Videos Into Context-Aware
Augmented Reality Applications, IEEE BigMM 2016.• GeoUGV: User-Generated Mobile Video Dataset with Fine
Granularity Spatial Metadata, MMSys 2016.
Results1. Significant Video Segments
2. Detecting Hot Spots
ConclusionProposed models and algorithms to identify
interesting video segments and hotspotsThe algorithms are fast and able to identify
interesting scenesWill consider user mobility and ranks of video
segments
�⃗��⃗�
pR𝜃
Field Of View (FOV)p: camera locationd: camera direction: viewable angleR: viewable distancet: timestamp
Single MultipleSingle
Zoom Pan
MultipleTrack Arch
PositionDirection
1d2d
3d
4d5d
1d2d 3d 4d
5d
1d
2d3d
4d
5d1d
2d3d
4d 5d
34 5
0
¿𝑡𝑚𝑖𝑛 ,𝑑𝑚𝑎𝑥>¿
¿ 𝑡𝑚𝑖𝑛 ,𝑑𝑚𝑎𝑥 ,𝑟𝑚𝑎𝑥>¿ ¿ 𝑡𝑚𝑖𝑛 ,𝑑𝑚𝑖𝑛 ,𝑟𝑚𝑎𝑥>¿
¿ 𝑡𝑚𝑖𝑛 ,𝑑𝑚𝑎𝑥>¿
Number of scenes found
Track Arch Zoom Pan0
500
1000
1500
2000
Num
ber o
f sce
nes
Searching time (one video)
Track Arch Zoom Pan0
5
10
15
20
25
30
Run
time
(ms)
Tracking is the most popular way of capturing mobile videos Overall search time is fast
Hotspots covered by many video frames
100
90 80 70 60 50 40 30 20 100
200400600800
1000
Minimum FOVs per hotspot
Tota
l num
ber o
f hot
sp
ots
Real Dataset
Statistics ValueNumber of videos with geo-metadata 2,397Average length per video with content (sec) 72.14Number of users 289Number of FOVs 208,978
Zoom Pan
Track Arch
