International Telecommunication Union
Workshop on End-to-End Quality of Service.What is it? How do we get it?Geneva, 1-3 October 2003
Are Existing Are Existing Performance Performance
Metrics Adequate?Metrics Adequate?
Alan ClarkCEO, Telchemy
21-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Outline
o IP Performance Metricso Network Characteristicso Packet Loss Characteristics and
Measuremento Jitter Characteristics and
Measuremento Recommendations
31-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Typical IP Performance Metrics
o Packet Loss• Equipment problems, misrouting,
buffer overflow, transmission errorso Jitter
• Network congestion, contention and queuing delays
o Delay• Transmission delays, quasi-stable
congestion levels
41-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Core IP Network characteristics
o Class “A” networks• High capacity optical fiber, high
throughput routers• Very low jitter and packet loss,
occasional link failureso Class “B” networks
• T1/E1 trunk connections• Significant jitter and loss due to
network congestion
51-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Corporate IP Networks
o Moving to 100BaseT Switched Ethernet but still some 10BaseT and Hubs
o Access links often T1/E1/PRI, sometimes fractional
o Mixture of digital leased line, frame relay and IP VPN
o Small offices and teleworkers may use low bandwidth links
61-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Teleworkers and Residential IP
o Low-mid bandwidth DSL and Cable Modem connections with little or no QoS control
o Often 10BaseT and IEEE802.11 LANo Can experience heavy usage of
access links leading to high levels of jitter
71-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Packet Loss
o Specifically, packets lost within the network or discarded before reaching the network API in the receiving system.
81-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Example packet loss distribution
0
20
40
60
80
100
0 20 40 60 80 100 120 140 160
Burst length
Nu
mb
er
of
pa
ck
ets
lo
st
91-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Example packet loss distribution
0
20
40
60
80
100
0 20 40 60 80 100 120 140 160
Burst length
Nu
mb
er
of
pa
ck
ets
lo
st
20% Loss Rate
ConsecutiveLoss
101-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Consecutive Loss Distribution
0
2000
4000
6000
8000
10000
12000
14000
0 1 2 3 4 5 6 7 8 9
Consecutive Lost Packets
Fre
qu
ency
111-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Lessons learned from analyzing real world traces
o Packets are usually lost singly – much less frequent to lose 2-3 consecutive packets
o Occasional very long consecutive loss periods due to link failure
o Much more common to see high loss periods – seconds in length – with loss densities of 30%
o Measurement of packet loss needs to be based around a burst model – e.g. Gilbert-Elliott, Markov Model with 3 or more states….
121-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Jitter
o Specifically – short term variations in transmission delay
o Real time systems often use a jitter buffer to remove jitter – but increases delay and packet loss
o The impact on performance is due to the combined effect of jitter and the jitter buffer
131-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Example time series plot of packet delay
0.050.060.070.080.090.10.110.120.130.140.15
0 2 4 6 8 10 12 14
Time (seconds)
Del
ay (
seco
nd
s)
141-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Jitter Characteristics
What should we measure?- Packet to packet delay?- Absolute delay?- Delay with respect to ?
151-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Jitter Characteristics
What really counts is what the end system woulddo as a result of delay variationsLesson: estimate (or count) which packets wouldbe discarded due to jitter
161-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Timing Drift
Can occur between end systems and between end andmeasuring system.Effect on VoIP is usually small however can causesignificant measurement errors for delay and jitter
171-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Estimating the effects of jitter
0.050.060.070.080.090.10.110.120.130.140.15
0 2 4 6 8 10 12 14
Time (seconds)
Del
ay (
seco
nd
s)
Jitter Buffer Emulator
Discard
181-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Measuring the combined effects of jitter and loss -
midpoint
JitterBuffer
EmulatorLoss andDiscard Events
Burst Loss Model
191-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Measuring the combined effects of jitter and loss -
endpoint
DSP – Jitter BufferVocoder, Echo Canceller
Burst Metrics API- percent lost- percent discarded- gap length/density- burst length/density
201-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Delay
o Mid-stream systems often rely on RTCP to estimate round trip delay• RTCP is often not implemented by
end systems• RTCP delay is only the VoIP packet
path delay and does not include external circuit switched path delays
211-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Sources of delay
Frame accumulation and encoding
RTP transmission delay
Receiving Jitter Buffer
Decoding, PLC, playout
External
End End System Delays can be more substantialthan transmission delay, particularly with highjitter levels and adaptive jitter buffers
221-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
RTCP XR VoIP metrics
o RTCP XR reports:-• Proportion of packets discarded• Proportion of packets lost• Length and density of bursts (periods of
high loss/discard density)• Length and density of gaps• End system delay• Packet path delay• and more…………………
231-3 October 2003 Workshop on End-to-End Quality of Service. What is it? How do we get it?
ITU-T
Recommendations
o Understand that packet path properties are time varying
o Don’t measure jitter but do:-• Measure packet discards resulting from jitter• Count discards as lost packets
o Use a “proper” model for packet loss distribution (e.g. Markov model with 3+ states, Gilbert-Elliott model…)
o Incorporate end system delays – if knowno Use RTCP XR!!!! (as this implements the above)
Top Related