Self-Organized Inter-Destination Multimedia Synchronization for Adaptive Media Streaming
-
Upload
benjamin-rainer -
Category
Science
-
view
386 -
download
0
description
Transcript of Self-Organized Inter-Destination Multimedia Synchronization for Adaptive Media Streaming
SELF-ORGANIZED INTER-DESTINATION MULTIMEDIA
SYNCHRONIZATION FOR ADAPTIVE MEDIA STREAMING
Benjamin Rainer, Christian Timmerer
Alpen-Adria-Universität Klagenfurt, Institute of Information Technology
TYPES OF SYNCHRONIZATION § Intra-Stream Synchronization
§ Avoid jitter between the presentation of two consecutive media units
§ Inter-Stream Synchronization § E.g., Audio + Video + Subtitles
Network Mul$media Playback
Time (t)
Time (t)
Video
Audio
Receiver
Network
40ms
Time (t)
Receiver Video
Playback
Slide 2 of 19
MOTIVATION § Why do we need Inter-Destination Multimedia
Synchronization? § Friends are watching a soccer match and they communicate via text messaging/
phone/…
User 1 User 2 Goal! Did you see the goal?
Which goal? Thanks for the spoiler!
Slide 3 of 19
STATE OF THE ART § Most IDMS solutions
§ Extend sender/receiver reports of RTCP [4] § Assume multicast [4, 18] § Demand a centralized instance [4, 24]
• Master-/Slave- scheme • Synchronization Master
§ Push-based multimedia streaming [4, 24, 9, 18]
§ Overcoming the asynchronism § By pausing and skipping media content § Increasing or decreasing the playback rate [17]
• Adaptive Media Playout (AMP)
Slide 4 of 19
OUR APPROACH Content Provider
Application LayerPeer-to-Peer Overlay
Geographically Distributed
Clients
MPDMPD Provides MPDs
enriched with Session Information
MPD Server
Content Servers
§ Research Objectives § Adapt MPEG-DASH for IDMS § Agree on a reference in a distributed and self-organized manner § Carry out the actual synchronization more smartly than using plain AMP
Slide 5 of 19
SESSION MANAGEMENT § Inter-Destination Multimedia Synchronization Session Object
(ISO) § Time bounded entity, contains a set of peers, uniquely identifiable § (IP, port) and the type of the Network Address Translator (NAT)
• NAT covered in the paper § Allows a (unique) numbering of peers
§ ISO is identified by session key § Provided by 3rd party application or the user
§ Integrated into the MPD of MPEG-DASH
§ Server imports the corresponding ISO when requested § E.g., a peer requests the MPD with a session key
Slide 6 of 19
SYNCHRONIZATION § Two phase synchronization using non reliable communication (UDP)
§ Coarse synchronization § Fine synchronization
§ Coarse synchronization § Overlay creation § Educated guess where to start downloading
§ Fine synchronization § Distributed algorithm – Merge and Forward § Periodical sends constants sized message to neighbors § Negotiate on reference playback timestamp
§ Overcome the identified asynchronism § Dynamic Adaptive Media Playout
Slide 7 of 19
COARSE SYNCHRONIZATION § Overlay creation
§ Ask peers in the ISO for their current Playback Timestamp (PTS) + NTP TS
• If the response is received, the peer is added to the list of known peers
§ May lead to a not fully connected network • E.g., due to packet loss
§ Strategies for selecting the segment to start with § Maximum PTS from peers § Minimum PTS from peers § Weighted average PTS from peers
Slide 8 of 19
MERGE AND FORWARD - EXAMPLE 1
2
3
Initial state: {BF, LID, HID, Cnt, ATS} 1: {{1}, 1, 1, 1, P1} 2: {{2}, 2, 2, 1, P2} 3: {{3}, 3, 3, 1, P3}
1
2
3
1: {{1,2}, 1, 2, 2, (P1+P2)/2} 2.1: {{1,2}, 1, 2, 2, (P1+P2)/2} 3: {{2,3}, 2, 3, 2, (P2+P3)/2}
τ =1
BF … Bloom filter Cnt … Cumula$ve Count H/LID … Lowest/Highest Peer ID ATS … (Weighted) Average TS
Slide 9 of 19
1
2
3
1: {{1,2}, 1, 2, 2, (P1+P2)/2} 2: {{1,2,3}, 1, 3, 3, (P3+2*(P1+P2/)/2)/3} 3: {{2,3}, 2, 3, 2, (P2+P3)/2}
τ =1 1
2
3
1: {{1,2,3}, 1, 3, 3, (P1+P2+P3)/3} 2: {{1,2,3}, 1, 3, 3, (P1+P2+P3)/3} 3: {{1,2,3}, 1, 3, 3, (P1+P2+P3)/3}
τ = 2
M&F - EVALUATION § Simulation environment OMNeT++ with INET
framework § Random networks (Erdős-Rényi) § 40, 60, and 80 peers § Probabilities for creating connections between peers: 0.1
to 0.9 (uniformly distributed) § Period of 250ms § RTT of 300ms between peers
§ Compared to Aggregate § Periodically sends list of known playback timestamps and
NTP timestamps [10] § Peers extend this list and/or update their PTS and NTP TS
Slide 10 of 19
M&F - EVALUATION
§ Y-axis denotes the average traffic generated per peer in kbit § X-axis denotes the connectivity of the overlay network Slide 11 of 19
M&F - EVALUATION
§ Y-axis denotes the time required for the synchronization process § X-axis denotes the connectivity of the overlay network Slide 12 of 19
DYNAMIC ADAPTIVE MEDIA PLAYOUT § Increase/decrease playback rate
§ Pausing decreases QoE [7]
§ Overcome asynchronism by increasing or decreasing the playback rate § Select those content sections which mask the playback rate variation
§ Content features for measuring the distortion caused by AMP § Audio: spectral energy of audio frames § Video: motion intensity between consecutive video frames
§ Metrics for the distortion § Difference between the impaired and the unimpaired case for both
modalities
Slide 13 of 19
DYNAMIC ADAPTIVE MEDIA PLAYOUT § Combined metric:
§ Constrained optimization problem for finding appropriate content sections:
x1...x2...x3...ξ...L...B...bc...br...T...tmax...
X ∈ R3
(1)argminX
f (X)
(2)x2 ⋅ (x3sign(ξ ) −1) ⋅ sign(ξ ) = ξ
(3)L ≤ B− x2 ⋅ x3 + x2 ⋅bcbr
(4)x1 ≤ T(5)x2 ≤ tmax
start time
duration playback rate asynchronism
lower buffer threshold current buffer fill state
bandwidth
content bit-rate
maximum start time
maximum duration
f (X) =|| d(X) ||2, d(X)= (dv (X),da (X))T
Slide 14 of 19
DYNAMIC AMP - EVALUATION § Subjective Quality Assessment using Crowdsourcing
§ Microworkers § 15 minutes § $0.25 as reward § 80 participants
§ Sequences § Babylon A.D. for training
• {1, 0.5, 2} times the nominal playback rate § Big Buck Bunny for the main evaluation
• {0.5, 0.6, 0.8, 1,1.2, 1.4, 1.6, 1.8, 2} times the nominal playback rate • Selected content sections
Slide 15 of 19
DYNAMIC AMP - EVALUATION
§ Y-axis denotes the Mean Opinion Score, x-axis denotes the average f(X) and playback rate (µ)
§ High linear correlation between distortion metric and QoE • µ > 1: • µ < 1:
ρ = 0.975ρ = −0.995
Slide 16 of 19
Slide 17 of 19
§ Demo video available on YouTube § https://www.youtube.com/watch?v=2V9rO5SbI7A § Source Code available at: https://github.com/grishnagkh/mf
DEMO
CONCLUSION § Introduced IDMS to Adaptive Streaming
§ MPEG-DASH
§ Distributed Control Scheme that scales with the number of peers § Can be combined with any streaming protocol § Not coupled with the session management or the overlay creation
§ Dynamic AMP for carrying out the actual synchronization § (General) Optimization problem that aims on finding appropriate content
sections
§ Demo video available on YouTube § https://www.youtube.com/watch?v=2V9rO5SbI7A § Search for: MergeAndForward § Source Code available at: https://github.com/grishnagkh/mf
Slide 18 of 19
THANK YOU!
Q & A
REFERENCES [4] F. Boronat Segu, J. Guerri Cebollada, and J. Lloret Mauri. An RTP/RTCP based approach for multimedia group and inter-stream synchronization. MTAP, pp. 40:285-319, 2008. [9] C. Hesselman, D. Abbadessa, W. Van Der Beek, D. Gorgen, K. Shepherd, S. Smit, M. Gulbahar, I. Vaishnavi, J. Zoric, D. Lowet, R. De Groote, J. O'Connell, and O. Friedrich. Sharing enriched multimedia experiences across heterogeneous network infrastructures. IEEE Comm. Mag., pp. 48(6):54-65, 2010. [10] T. Hossfeld, M. Seufert, M. Hirth, T. Zinner, P. Tran-Gia, and R. Schatz. Quantification of YouTube QoE via Crowdsourcing. In IEEE ISM, pages 494{499, 2011. [17] M. Montagud and F. Boronat. On the Use of Adaptive Media Playout for Inter-Destination Synchronization. IEEE Communications Letters, pp. 15(8):863-865, 2011. [18] M. Montagud, F. Boronat, and H. Stokking. Design and Simulation of a Distributed Control Scheme for Inter-destination Media Synchronization. In IEEE 27th AINA, pp. 937-944, March 2013. [24] H. Stokking, M. Van Deventer, O. Niamut, F. Walraven, and R. Mekuria. IPTV inter-destination synchronization: A network-based approach. In 14th ICIN, pp. 1-6, 2010.
Slide 20 of 19