TIA-921 (PN-3-0062) and ITU-T G.NIMM Network Model for Evaluating Multimedia Transmission...

TIA-921 (PN-3-0062) and ITU-T G.NIMM

Network Model for Evaluating Multimedia Transmission Performance Over Internet Protocol

Jack Douglass, SpirentChair TIA TR30.3

August 2005

TIA TR41.4-05-08-008

TIA-921 (PN-3-0062) and ITU-T G.NIMM

Jitter, Latency, Burst Loss, Gap Loss

Packet Loss, Out-of-Order Packets

Route Flapping, Link Failure

Scenario Based Time Varying

IP Network Impairments

IP Network Impairment EmulatorBased on TIA-921 and ITU-T G.NIMM

Types of DUTsIP PhonesPOTS over IPMedia Gateways MGCUser AgentsCall Agents Media ServersVideo Gatekeepers Application Servers Edge Routers GatewaysModemsFax

Source DeviceUnder Test

DestinationDevice Under

Test

IP NetworkImpairment Emulator

Ethernet Ethernet

TIA-921 (PN-3-0062) and ITU-T G.NIMM Network Model for Evaluating Multimedia Transmission Performance

TIA-921 / ITU-T G.NIMM

Network Impairments and Parameters that Affect Voice and Video Quality

• Network Architecture

• Types of Access Links

• QoS controlled Edge Routing

• MTU Size

• Packet Loss (Frame Loss)

• Out of order packets

• One Way Delay (Latency)

• Variable Delays (Jitter)

• Background Traffic (Occupancy)

• Timing Drift

• Route Flapping

• Signaling protocol mismatches

• Network faults

• Link Failures

• Voice Only Impairments– Echo

– Voice coding algorithms

– A/D and D/A Conversion

– Noise – Circuit and External

• Video Only Impairments– Video coding algorithms

– Variable and Fixed Frame Rates

– Video Complexity

TIA-921and ITU-T G.NIMM IP Network Model

Core IP NetworkLAN A

Local AccessB

1000BaseX* 100BaseT Switch100BaseT Hub10BaseT* WLAN (~4 Mbit/s)----------------------Occupancy levelPacket loss

64 kbit/s*128 kbit/s256 kbit/s*384 kbit/s512 kbit/s*768 kbit/s

*T1 (1.536 kbit/s)E1 (1.920 kbit/s)E3 (34 Mbit/s)*T3 (44 Mbit/s)

ADSL (~256 kbit/s)*Cable (~256 kbit/s)Fiber (1-10 Gbit/s)

--------------------Occupancy levelQoS edge router

LAN B

Route flappingOne-way delay

JitterPacket loss

DestinationDevice B

Local AccessA

64 kbit/s*128 kbit/s256 kbit/s*384 kbit/s512 kbit/s*768 kbit/s

*T1 (1.536 kbit/s)E1 (1.920 kbit/s)E3 (34 Mbit/s)*T3 (44 Mbit/s)

ADSL (~2 Mbit/s)*Cable (~3 Mbit/s)Fiber (1-10 Gbit/s)

--------------------Occupancy levelQoS edge router

SourceDevice A

* Case used in impairment tables

1000BaseX* 100BaseT Switch100BaseT Hub10BaseT* WLAN (~4 Mbit/s)----------------------Occupancy levelPacket loss

TIA-921and ITU-T G.NIMM IP Network Model

• Typically test are run in one direction

• Tests may be run in both directions

• Impairments occur in both directions

Core IP Netw orkLAN A

LocalAccess B

LAN data rateLAN occupancy

Access data ratesAccess occupancyMTU size

LAN B

Route f lappingLink f ailureOne-way delayJitterPacket loss

DestinationDev ice B

LocalAccess ASource

Dev ice A

Access data ratesAccess occupancyMTU size

LAN data rateLAN occupancy

Direction of Test

TIA-921 and ITU-T G.NIMMTest Profiles Based on QoS Classes

• Statistically based models for different QoS Classes

Service Test Profiles

QoS Class(Y.1541)

Applications (Examples) Node Mechanisms

Network Techniques

Well‑Managed IP Network(Profile A)

0 High quality video and VoIP, VTC (Real-time applications, loss sensitive, jitter sensitive, high interaction)

Strict QoS, guaranteed no over subscription on links

Constrained routing and distance

Best-EffortIP Network(Profile B)

1 VoIP, VTC (Real-time applications, jitter sensitive, interactive)

Separate queue with preferential servicing, traffic grooming

Less constrained routing and distances

UnmanagedIP Network,Internet(Profile C)

2 Lower quality video and VoIP, signaling, transaction data (highly interactive) Separate queue,

drop priority

Constrained routing and distance

3 Transaction data, interactive Less constrained routing and distances

4 Short transactions, bulk data (low loss)

Long queue, drop priority

Any route/path

5 Traditional Internet applications (default IP networks)

Separate queue (lowest priority) Any route/path

TIA-921 and ITU-T G.NIMMTest Profiles Based on QoS (Y.1541) Classes

Profile C Un-Managed NetworkImpairment Ranges

Table 4

Profile BBest Effort Managed Network

Impairment Ranges Table 3

Profile AWell Managed Network

Impairment Ranges Table 2

Different test profiles for different Service Level Agreements (SLAs)

Impairment Type Units Range

Jitter ms 0 to +/- 250

One Way Average Latency

ms 50 to 800

Sequential Packet Loss

# of sequential packets

2 to 500

Rate of Sequential Loss

sec-1 < 10-1

Random Packet Loss

% 0 to 20

Out of Sequence Packets

% 0 to 20




ms 50 to 200


# of sequential packets

2 to 5

Rate of Sequential Loss

sec-1 < 10-3

Random Packet Loss

% 0 to 2


% 0 to 0.1




ms 20 to 60


# of sequential packets Random

loss

onlyRate of Sequential Loss

sec-1

Random Packet Loss

% 0 to 0.05


% 0 to 0.001

TIA-921 and ITU-T G.NIMM 8 Impairment Severity Combinations

Table 10: Impairment Severity Combinations

Severity=> A B C D E F G H

Impairment Units

Profile A LOO % 50 30 15 5 0 0 0 0

Profile B LOO % 5 25 30 25 10 5 0 0

Profile C LOO % 5 5 10 15 20 25 15 5

Source Location (A) Parameters

LAN A Occupancy % 1 2 3 5 8 12 16 20

Access A Occupancy % 0 1 2 4 8 15 30 50

MTU A bytes 512 512 1508 1508 1508 1508 1508 1508

Core Network Impairments

Route flap interval seconds 0 3600 1800 900 480 240 120 60

Route flap D delay ms 0 2 4 8 16 32 64 128

Delay ms 4 8 16 32 64 128 256 512

Jitter ms 5 10 15 20 30 50 75 100

Link fail interval seconds 0 3600 1800 900 480 240 120 60

Link fail duration ms 0 64 128 256 400 800 1600 3000

Packet loss % 0 0.01 0.02 0.04 0.1 0.2 0.5 1

Destination Location (B) Parameters

Access B Occupancy % 0 1 2 4 8 15 30 50

MTU B bytes 512 512 1508 1508 1508 1508 1508 1508

LAN B Occupancy % 1 2 3 5 8 12 16 20

TIA-921 and ITU-T G.NIMM LAN and Access Rates

Table 7: Access Rates for Home Locations

Table 8: Access Rates for Business LocationsTable 6: LAN Rates for Business Locations

Table 5: LAN Rates for Home Locations

133 LAN/Access Rate Combinations

Home Rates

Business Rates

LAN Rate LOO Represents

Mbit/s %

4 75 802.11b, 10BaseT hub

20 25 802.11g, 100BaseT hub

LAN Rate LOO Represents

Mbit/s %

4 20 802.11b, 10BaseT

20 20 802.11g, 100BaseT hub

100 60 100 BaseT switched, Gbit Ethernet

Access Rate LOO Represents

Toward core From core %

kbit/s kbit/s

128 768 40 Cable, ADSL

384 1536 50 Cable, ADSL

384 3000 10 Cable, ADSL

Access Rate LOO Represents

Toward core From core %

kbit/s kbit/s

384 1536 40 ADSL entry

384 3000 15 ADSL premium

1536 1536 40 T1

43000 43000 5 T3

LAN and Access Rates (New Revision of Standard/Recomendation) will change as network conditions change

Testing with Fixed Levels of IP Impairments

• IP Impairment network emulators exist today that provide fixed or semi-fixed levels of IP impairments.

• Real IP Network Impairments Vary Over Time significantly over time and are bursty in nature

Fixed Jitter & Packet Loss

Fixed Packet Jitter

TIA-921 and ITU-T G.NIMMStatistically Based IP Network Impairment Conditions (ICs) based on SLAs

IC– 96AProbability (LOO = 0.00256% )

Call Quality Metric

Impairment Condition


Call Quality Metric


IC– 96EProbability (LOO = 0.01025% )


Link Failure

Route Flap

Call Quality Metric


IC– 96HProbability (LOO = 0.00256%)


• Each Impairment Condition has a Likely-hood of Occurrence (LOO)

– Based on real IP Network Statistics

– Based on Network Architecture

– Based on Classes of Service

• Definitive set of test conditions

– 1064 Impairment Combinations

– Tests may be automatically

– Typically 2 min per Impairment Condition

• Approximately 1.5 days to run complete set of tests

– Impairment Condition may be correlated with appropriate quality measurement

• Voice Quality Metric (MOS, PESQ, PSQM, R-Factor)

• Video Quality Metric

Mean Opinion Score - Listening Quality vs. TIA-921 Network Model Coverage(G.711 PLC, 40 ms jitter buffer)

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

% Network Model Coverage

MO

S-L

Q Test Profile A

Test Profile B

Test Profile C

TIA-921 and ITU-T G.NIMM Example of Network Model Coverage (NMC) Curve

Test Results from 1064 Statistically based IP Network Impairment Combinations

TIA-921 and ITU-T G.NIMM Example of Network Model Coverage (NMC) Curve

Test Results from 1064 Statistically based IP Network Impairment Combinations

R-Factor - Listening Quality vs. TIA-921 Network Model Coverage(G.711 PLC, 40 ms. jitter buffer)

0

10

20

30

40

50

60

70

80

90

100

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

% Network Model

R-L

Q

Test Profile A

Test Profile B

Test Profile C

History of TR30.3 Network Model Standards• TIA/EIA 496A-1989: Interface Between Data Circuit Terminating

Equipment (DCE) and the Public Switched Telephone Network

– Included Network Model for Evaluating Modem Performance

• TIA/EIA TSB 37A-1994: Telephone Network Transmission Model for Evaluating Analog Modem Performance, which became ITU-T Recommendation V.56bis-1995

• EIA/TIA TSB 38-1994: Test Procedures for Evaluation of 2-Wire 4 Kilohertz Voice Band Duplex Modems, which became ITU-T Recommendation V.56ter-1996

• ANSI/TIA/EIA 3700-1999: Telephone Network Transmission Model for Evaluating Analog Modem Performance

• ANSI/TIA/EIA 793 -2000: North American Telephone Network Transmission Model for Evaluating Analog Client and Digitally Connected Server Modems

• ANSI/TIA 876 – 2002: North American Network Access Transmission Model for Evaluating xDSL Modem Performance

TR30.3 Liaison Activities

• Liaisons activities between TR30.3 and appropriate committees to develop TIA-921 (PN-3-0062), Network Model for Evaluating Multimedia Transmission Performance Over Internet Protocol– TR41.4

– TR45.5

– ITU-T SG12

– ITU-T SG9

– ITU-T SG15

• Working with Operating Companies and Service providers to provide statistically accurate IP network impairment model

TIA-921 Target Audience for Network Model

• Network Equipment Manufacturers

• Operating Companies

• Service Providers

• Design Engineers

• Test houses

• Magazines and product reviewers

TIA-921 (PN-3-0062) and ITU-T G.NIMM Network Model for Evaluating Multimedia Transmission...

Documents

Transcript of TIA-921 (PN-3-0062) and ITU-T G.NIMM Network Model for Evaluating Multimedia Transmission...