Model-Based Estimation of Streaming Performance draft-ko-ippm-streaming-performance Ken Ko IPPM –...
18
Model-Based Estimation of Streaming Performance draft-ko-ippm-streaming-performance Ken Ko [email protected] IPPM – Orlando – IETF86 – March 10-15, 2013
-
Upload
darleen-cain -
Category
Documents
-
view
226 -
download
0
description
Advantages of Model-Based Methodology Current state – Dedicated test generates traffic on network – Limited to one streaming rate – Result is almost binary (does it work at that rate or not?) Proposed methodology – Use (or re-use) TCP throughput results Throughput test is probably already part of performance test suite No additional network traffic required – One TCP sample metric can be run against model many times Different encoded rates Different buffer depths – Can determine the maximum supported streaming rate to a close tolerance
Transcript of Model-Based Estimation of Streaming Performance draft-ko-ippm-streaming-performance Ken Ko IPPM –...
Model-Based Estimation of Streaming Performance
draft-ko-ippm-streaming-performanceKen Ko [email protected]
IPPM – Orlando – IETF86 – March 10-15, 2013
Video streaming• Over half of all consumer Internet traffic since 20111
• Near-RT traffic– Marginal throughput can cause excessive wait time before
video starts– Throughput variation can cause interruption in video
playout (“video freeze”)• In US, FCC has measured video streaming performance
for last two years2
– Send a stream at a constant bit rate– Write received data to buffer– Read data from buffer at constant rate– Record metrics related to buffer fill1Cisco Visual Networking Index, May 30, 2012
2Methodology - Measuring Broadband America Report February 2013
Advantages of Model-Based Methodology
• Current state– Dedicated test generates traffic on network– Limited to one streaming rate– Result is almost binary (does it work at that rate or not?)
• Proposed methodology– Use (or re-use) TCP throughput results
• Throughput test is probably already part of performance test suite• No additional network traffic required
– One TCP sample metric can be run against model many times• Different encoded rates• Different buffer depths
– Can determine the maximum supported streaming rate to a close tolerance
OTT Video Characteristics
• Multiple protocols under “OTT video streaming” umbrella
• Common characteristics– Almost always over TCP– Average encoded rate, whether CBR or VBR
• Ignore adaptive rate streaming for the moment (see backup)– Usually an initial rate higher than average to fill buffer– Usually transmitted in bursts separated by idle time
• Average transmitted rate ≈ average encoded rate after initial burst
Example OTT Video Stream
• Initial streaming rate (< 400 seconds) ≈ 5.8 Mbps• Average encoded rate ≈ 2.9 Mbps
0
20
40
60
80
100
120
140
160
180
200
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
18.0
20.0
0 200 400 600 800 1000
Buffe
r (M
B)
Rate
(Mbp
s)
Time (s)
Streaming rateEst. encoded rateEst. MB buffered
Streaming model definition
Streaming Model Parameters
• Rinit: initial streaming rate– Rate at which buffer initially fills
• Ravg: average encoded rate– Rate at which the dejitter buffer is emptied– Also, rate at which buffer fills in MAINTAIN state
• Binit: initial buffer fill depth– Depth at which buffer starts to be emptied
• Btarget: Target buffer fill depth– Depth at which
Model Behavior (ideal)
1. Streaming starts at Rinit: buffer filling2. Buffer fill reaches Binit: playout starts3. Buffer fill reaches Btarget: streaming rate reduced
to Ravg
0
2
4
6
8
10
12
14
16
18
20
0
1
2
3
4
5
6
7
0 50 100 150 200
Buffe
r (M
B)
Rate
(Mbp
s)
Time (s)
Streaming rateEncoded (playout) rateBuffer fill
Ravg
Btarget
Rinit
Binit
1
2
3
Model-based methodology with examples
Model-Based Methodology
1. Perform TCP throughput test– Collect short-term TCP throughput values at regular intervals– Sample metric = series of short-term throughput values
2. Define model parameter values 3. Apply streaming model to sample metric– Generate derived sample metric showing buffer fill over time– Generate statistics based on derived sample metric
4. If desired, go to step 2 and define different parameter values
0.0
0.1
0.2
0.3
0.4
0.5
0.0
0.5
1.0
1.5
2.0
2.5
28 30 32 34 36 38 40
Buffe
red
(MBy
tes)
Rate
(Mbp
s)
Time (s)
TCP throughput Playout rateStreaming rate MBytes buffered
1
3
4
5
2
Example: Ravg = 1.0 Mbps
1. Streaming starts at Rinit: buffer filling2. Buffer fill reaches Binit: playout starts3. Buffer fill reaches Btarget: streaming rate Ravg4. Reduced throughput, buffer < Btarget: streaming rate Rinit5. Buffer back at Btarget: streaming rate Ravg
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.0
0.5
1.0
1.5
2.0
2.5
28 30 32 34 36 38 40
Buffe
red
(MBy
tes)
Rate
(Mbp
s)
Time (s)
TCP throughput Playout rateStreaming rate MBytes buffered
1
3
2
45
Example: Ravg = 1.3 Mbps
1. Streaming starts at Rinit: buffer filling2. Buffer fill reaches Binit: playout starts3. Buffer fill reaches Btarget: streaming rate Ravg4. Reduced throughput, buffer < Btarget: streaming rate Rinit5. Buffer back at Btarget: streaming rate Ravg
0.0
0.1
0.2
0.3
0.4
0.5
0.0
0.5
1.0
1.5
2.0
2.5
28 30 32 34 36 38 40
Buffe
red
(MBy
tes)
Rate
(Mbp
s)
Time (s)
TCP throughput Playout rateStreaming rate MBytes buffered
1
2
Example: Ravg = 1.6 Mbps
1. Streaming starts at Rinit: buffer filling2. Buffer fill reaches Binit: playout starts
Buffer fill never reaches Btarget
0.0
0.1
0.2
0.3
0.4
0.5
0.0
0.5
1.0
1.5
2.0
2.5
28 30 32 34 36 38 40
Buffe
red
(MBy
tes)
Rate
(Mbp
s)
Time (s)
TCP throughput Playout rateStreaming rate MBytes buffered
1 3
42
Example: Ravg = 1.9 Mbps
1. Streaming starts at Rinit: buffer filling2. Buffer fill reaches Binit: playout starts
Buffer fill never reaches Btarget3. Buffer under-run: playout stops (video freeze)4. Buffer fill reaches Binit: playout restarts
Backup Slides
Model PseudocodeFor k = 1 to k_max // Run the simulation
for // each time interval T(k)
Switch(Model_state)
// Buffer filling, no playout
Case FILL_NOPLAY:B(k) = B(k-1) +
min(Finit, R(k))If B(k) >=
Btarget then
Model_state = MAINTAINElse If B(k) >=
Binit then
Model_state = FILL_PLAYEnd if
// Buffer filling, media playing out
Case FILL_PLAY:B(k) = B(k-1) +
min(Finit, R(k)) - PIf B(k) >=
Btarget then
Model_state = MAINTAINElse If B(k) <=
0 then
B(k) = 0
Model_state = FILL_NOPLAYEnd if
// Buffer at target, media playing out
Case MAINTAIN:B(k) = B(k-1) +
min(Fmaint, R(k)) - PIf B(k) <= 0
then
B(k) = 0
Model_state = FILL_NOPLAYElse If B(k) <
Btarget then
Model_state = FILL_PLAYEnd if
End switch
Next k
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
0 200 400 600 800 1000
Est.
enco
ded
rate
(Mbp
s)
Stre
amin
g ra
te (M
bps)
Time (s)
Streaming rate Est. encoded rate
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
0 200 400 600 800 1000
Est.
enco
ded
rate
(Mbp
s)
Stre
amin
g ra
te (M
bps)
Time (s)
Streaming rate Est. encoded rate
Bursty Delivery vs. Average Rate
• Same content – (a) received over >15 Mbps channel– (b) received over channel shaped to 1 Mbps
• Blue traces show estimated encoded rate• Rate-limited graph shows spreading of bursts
– Red circles highlight where received bursts have merged
(a) (b)
The closer the encoded rate is to the channel rate, the closer the received traffic resembles the streaming model
What About VBR and Adaptive Rate Streaming?
• VBR encoding– Encoded (and transmitted) bit rate varies around an average
value– Commonly used but not modeled here
• Ravg models CBR– Can VBR be characterized in a way that enhances the model?
• Adaptive rate streaming– Streams available at multiple encoded rates– Receiver can request different rate based on channel
performance– Better to characterize performance by testing against model
at different rates
