VNPT VN2 Network Design

Microsoft Word - VNPT VN2_High_Level_Design_Release 8_Final.doc

VTN VN2High Level Design

May 2009

Final Version

Document Revision history

VersionDateAuthorComment / Changes

V0.129/12/08Ronald Lee, Leonard TanFirst draft for comments

V0.29/1/09Ronald Lee, Ma ShaowenSecond draft for comments

V0.322/1/09Ronald LeeThird draft for comments

V.0.420/1/09Ronald Lee, Leonard Tan, AnhTuan Chu, Ed SpencerFourth draft for comments

V.0.51/3/09Ronald Lee, Ed SpencerFifth draft for comments

V0.614/03/09Ed Spencer, Leonard Tan, AnhTuan ChuSixth draft version

V0.720/03/09Leonard Tan, Ma Shaowen, AnhTuan ChuSeventh draft version

V0.804/05/09Leonard Tan, Ma ShaowenIncorporated inputs from the22Apr meeting

Document Distribution

NameTitleCompany

VTN VN2 High Level Design

7 of 130

Table of Contents

Document Revision history ............................................................................................................... 1Document Revision history ............................................................................................................... 2Document Distribution....................................................................................................................... 2Table of Contents.............................................................................................................................. 31Scope ....................................................................................................................................... 52Assumptions............................................................................................................................. 53Introduction .............................................................................................................................. 54Network Design Requirements ................................................................................................. 55Physical Network Topology ...................................................................................................... 55.1Network Architecture........................................................................................................ 55.2Existing Network Architecture Caveats ............................................................................ 55.3POP Types and POP design............................................................................................ 55.3.1Type A1 ................................................................................................................... 55.3.2Type A2 ................................................................................................................... 55.3.3Type A3 ................................................................................................................... 55.3.4Type A4 ................................................................................................................... 55.3.5Type A5 ................................................................................................................... 55.3.6Type A6 ................................................................................................................... 55.3.7Type B1 ................................................................................................................... 55.3.8Type B2 ................................................................................................................... 55.3.9Type B3 ................................................................................................................... 55.3.10Type B4 ................................................................................................................... 55.3.11Type B5 ................................................................................................................... 55.3.12Type B6 ................................................................................................................... 55.3.13Type C1................................................................................................................... 55.3.14Type C2................................................................................................................... 55.3.15Type C3................................................................................................................... 55.3.16Type C4................................................................................................................... 55.3.17Type C5................................................................................................................... 55.3.18Type D..................................................................................................................... 55.3.19HNI ASBR ............................................................................................................... 55.3.20HCM ASBR ............................................................................................................. 55.3.21DNG ASBR.............................................................................................................. 55.4Aggregated Core plane facing uplinks ............................................................................. 55.5Network elements types and positioning. ......................................................................... 56Logic6.1al Network Topology ........................................................................................................5 IP Addressing and naming convention. ............................................................................ 5

6.1.1IP Addressing .......................................................................................................... 5

6.1.2Node Naming .......................................................................................................... 5

6.1.3Port Naming ............................................................................................................ 5

6.1.4Network Interface Naming ....................................................................................... 5

6.1.5Service Distribution Point Naming ........................................................................... 5

6.2IP Routing Recommendations ......................................................................................... 5

6.2.1Overview ................................................................................................................. 5

6.3Routing and Control Protocols ......................................................................................... 5

6.4End-to-End Logical Topology ........................................................................................... 56.4.1IGP .......................................................................................................................... 56.4.2IGP in Metro Networks ............................................................................................ 56.4.3In-band management to Metro Networks................................................................. 56.4.4BGP......................................................................................................................... 56.5IP Multicast Recommendations........................................................................................ 56.5.1Rendezvous Point (RP) placement.......................................................................... 56.5.2Reverse Path Forwarding (RPF) considerations...................................................... 56.6MPLS Topology and Signalling Overview. ....................................................................... 56.6.1Overview ................................................................................................................. 56.6.2Usage ...................................................................................................................... 56.6.3Backbone MPLS...................................................................................................... 56.6.4RSVP Signalled LSP Topology................................................................................ 56.6.5Traffic Engineering .................................................................................................. 56.6.6Edge MPLS ............................................................................................................. 56.7High Availability................................................................................................................ 56.7.1Border Router related Failure .................................................................................. 56.7.2PE upstream related Failure .................................................................................... 56.7.3P router RSVP Label Switch Path Recovery ........................................................... 56.8Network QoS Design ....................................................................................................... 56.8.1QoS Overview ......................................................................................................... 56.8.2Scheduling............................................................................................................... 56.8.3QoS Design ............................................................................................................. 56.8.4QoS on Juniper router ............................................................................................. 56.9Network Security Recommendation ................................................................................. 56.9.1Generic Node Access.............................................................................................. 56.9.2Authentication, Authorization, and Accounting (AAA) .............................................. 56.9.3User Accounts and Passwords ................................................................................ 56.9.4Packet Filtering Toolset ........................................................................................... 56.9.5Securing Nodes ....................................................................................................... 56.9.6Securing Client Services ......................................................................................... 56.9.7Juniper network management and security ............................................................. 57Service Network Topology........................................................................................................5 7.1Core and MANs interconnect ........................................................................................... 57.2Service Architecture ......................................................................................................... 57.2.1Ethernet Leased line service using L2 VPN............................................................. 57.2.2Wide Area Switched LAN Service using VPLS ........................................................ 57.2.3Wide area Routed LAN service using VPRN ........................................................... 57.2.4High Speed Internet Service.................................................................................... 57.2.5IPTV Video Core distribution using multicast ........................................................... 58Network Management ..............................................................................................................5 8.1Alcatel-Lucent 5620 SAM Element Manager (5620 SAM-E) ............................................ 58.2Alcatel-Lucent 5620 SAM-Assurance (SAM-A) ................................................................ 58.3Alcatel-Lucent 5620 SAM Provisioning (SAM-P).............................................................. 58.4Alcatel-Lucent 5620 SAM Open Interface (SAM-O) ......................................................... 58.5Alcatel-Lucent 5620SAM Redundancy............................................................................. 58.5.1Activity switches, failovers, and switchovers............................................................ 58.5.2Server activity switches ........................................................................................... 5

8.5.3Database switchovers ............................................................................................. 58.5.4Database failovers................................................................................................... 58.5.5Events associated with an activity switch. ............................................................... 58.6Direct 5620 SAM Client Access ....................................................................................... 58.7IP Addressing and Naming convention ............................................................................ 58.8Bandwidth Requirement................................................................................................... 58.8.1Bandwidth Requirements SAM to Network Elements .............................................. 58.8.2Details on the Bandwidth Requirements .................................................................. 58.8.3Possible consequences of insufficient bandwidth .................................................... 58.9Server Hardware .............................................................................................................. 58.10Security ............................................................................................................................ 58.10.1TACACS+ Configuration.......................................................................................... 58.10.2Communication Port Requirements for 5620-SAM Components ............................. 58.11Network Time Protocol..................................................................................................... 59NMS WANDL IP/MPLSView Solution ...................................................................................... 59.1What Can WANDL Solution Offer .................................................................................... 59.1.1Overview ................................................................................................................. 59.1.2Online Operation Management................................................................................ 59.1.3Offline Planning Simulation ..................................................................................... 59.1.4Provisioning & Service Management ....................................................................... 59.2Design of WANDL NMS Solution for VN2 ........................................................................ 59.2.1Overall Architecture ................................................................................................. 59.2.2Machines ................................................................................................................. 59.2.3User Admin.............................................................................................................. 59.2.4Backup .................................................................................................................... 510Conclusion ............................................................................................................................... 5References. ...................................................................................................................................... 5Glossary............................................................................................................................................ 5

1Scope (quy m)

The scope of this document is to provide the High-Level Design considerations to ensure a stable and scalable Network implementation using the Alcatel-Lucent7750 Service Router series. Detail is given regarding architectural high-level design principles, approaches and technology selections, however no attempt is made to detail explicit comman0064-line configuration.

2 Assumptions

This document assumes that the reader is reasonably familiar with IP/MPLS, Layer-2 and Layer-3 VPNs.


8

3Introduction

VNPT will implement a next generation network (NGN) and Alcatel-Lucent hasbeen selected to be the IPCORE Provider Edge (PE) router & Autonomous SystemBorder Router (ASBR) equipment supplier for the VN2 Network.

The VN2 network is being envisaged to house all of VNPTs existing networks, inclusive of all Metropolitan Networks, High speed Internet delivery Core and evolve into a new unified next-generation Core network.

VN2 is to be built and expanded over time to accommodate all of VNPTs existing service offerings and support the introduction of new services such as IP Television and Video-on-Demand.

This document provides high level design (HLD) & recommendations for the VN2 project specific to the PE and ASBR routers. Topics & specifications mentioned in this high level design document will provide the foundation as implementation specifications for the low level design (LLD) phase, which will translate the mentioned implementation specifications into machine specific configurations.

4Network Design Requirements

VN2 is poised to be the national NGN backbone network with the capabilities of delivering the following services as depicted by VNPT:

High Speed Internet (Broadband access)

Voice over IP (VoIP)

Multicast Video services (e.g. IPTV, Video Conferencing)

Unicast Video services (e.g. Video on Demand)

Layer 3 IP-VPN services

Layer 2 VPN services (VPLS & Pseudo-wire/VPWS)

5Physical Network Topology

This section describes the proposed physical network architecture.

5.1 Network ArchitectureThe VN2 is based on a dual Core Router plane design with 5 main core regions. These are namely: HNI, HCM, HPG, DNG & CTO. Provider edge routers within these 5 regions will be connected to the 2 Core routers located in the regional Central Office (CO).

Core Routers are T-1600 supplied by Juniper Networks. Provider Edge routers and Autonomous system border routers are 7750SR-7 supplied by Alcatel-Lucent. There are a total of 10 Core routers, 79 Provider edge routers and 5 ASBR routers in the VN2 IPCore Network. It is expected that the PE routers will have high speed connectivity to BRAS and Metro Ethernet networks to be realised in separate tenders.

Figure 1: VN2 IP Core Network

Enhancement to the requested VN2 network architecture have been proposed in this high level design document to provide increased service availability and improve the level of optimum route path selection. These enhancements are essentially inter-PE connectivity for PoP with dual PE routers as well as inter- ASBR connectivity for sites with dual ASBR PE routers.

For dual PE router POPs, Alcatel-Lucent strongly recommend that a series of Inter- PE link be added so as to provide increased service availability in the event of link failure to the P router.

5.2 Existing Network Architecture Caveats

Figure 2: Failure Scenario: Inter POP VPLS

In the above failure scenario, a VPLS service is in service between HPG POP and NDH POP. Failure of the STM-64 link of HPG PE1 to HPG P1 will render the VPLS service be split into 2 isolated VPLS islands. As the failure occurred on the MPLS network interface towards HPG P1, the connected MAN-E will not be aware of the failure until the MAC address ages out.

This failure scenario can be mitigated by introducing inter PE connection so that VPLS isolation does not occur during a link failure with the P router node. If the inter PE link is unavailable, the failure scenario can alternatively be mitigated by connecting the VPLS instance to an additional VLAN created within the MAN-E Metro Core to support the redundancy, RSTP will have to be configured on the VPLS for the purpose of breaking the layer 2 loop created. This will be the temporary solution until the inter-PE link is made available.

The recommended bandwidth for the Inter-PE node shall be at least 3.3Gbps assuming a protection ratio of 1:3. Therefore, this will translate to a 4 x 1GE LAG between the PE nodes.

5.3 POP Types and POP designThis section describes the various type of Point of Presence.

5.3.1 Type A1

Type A1 POP have 2 units of 7750 SR7 dual homed to MAN using 10GE interfaces as well as dual homed to P routers using STM-64 interfaces. Alcatel- Lucent recommends 4 x 1GE links between the PE router pair for increased availability. Alcatel-Lucent understands that this could only be added in subsequent expansion/variation order.

5.3.2 Type A2

have 20SRype nterfA2 cesOaswellasunit ualANsinho usmed ngSoTM64uteg 4 aceT P P s of 775 7 dual t M x 1GEi a d homed to P routers i - in rf s.

However, the 1GE modules are not present and it is understood that VTN will be putting up request to acquire these modules for deployment. In addition, 2 x 10GE interfaces are planned for connectivity to BRAS pair. Alcatel-Lucent recommends 4 x 1GE links between the PE router pair for increased availability. It is understood that provisioning the Inter-PE links could only be added in subsequent expansion/variation order.

5.3.3 Type A3

Type A3 POP have 2 units of 7750 SR7 dual homed to MAN using 5 x 1GE interfaces as well as dual homed to P routers using STM-64 interfaces. However, the 1GE modules are not present and it is understood that VTN will be putting up request to acquire these modules for deployment. In addition, 1 x 10GE interface is planned for connectivity to BRAS pair. Alcatel-Lucent recommends 4 x 1GE links between the PE router pair for increased availability. It is understood that provisioning the Inter-PE links could only be added in subsequent expansion/variation order.

5.3.4 Type A4

have 20SRype nterfANsinho usT A4 POP units of 775 7 dual med to M u g 5 x 1GbEi aces as well as dual homed to P routers ing STM-64 interfaces. However, the 1GE modules are not present and it is understood that VTN will be putting up request to acquire these modules for deployment. In addition, 2 x 10GbE interfaces are planned for connectivity to BRAS pair. Alcatel-Lucent recommends4 x 1GE links between the PE router pair for increased availability. It is understood that provisioning the Inter-PE links could only be added in subsequent expansion/variation order.

5.3.5 Type A5

Type A5 POP have 2 units of 7750 SR7 dual homed to MAN using 3 x 1GE interfaces as well as dual homed to P routers using STM-64 interfaces. However, the 1GE modules are not present and it is understood that VTN will be putting up request to acquire these modules for deployment. In addition, 2 x 10GE interfaces are planned for connectivity to BRAS pair. Alcatel-Lucent recommends 4 x 1GE links between the PE router pair for increased availability. It is understood that provisioning the Inter-PE links could only be added in subsequent expansion/variation order.

5.3.6 Type A6

Type A6 POP have 2 units of 7750 SR7 dual homed to MAN using 6 x 1GE interfaces as well as dual homed to P routers using STM-64 interfaces. However, the 1GE modules are not present and it is understood that VTN will be putting up request to acquire these modules for deployment. In addition, 3 x 10GE interfaces are planned for connectivity to BRAS pair. Alcatel-Lucent recommends 4 x 1GE links between the PE router pair for increased availability. It is understood that provisioning the Inter-PE links could only be added in subsequent expansion/variation order.

5.3.7 Type B1

Type B1 POP have 1 unit of 7750 SR7 dual homed to P routers using 2 x STM-16 interfaces. In addition, 1 x 1GE interface to MAN and 1 x 1GE interface to BRAS are planned.

5.3.8 Type B2

Type B2 POP have 1 unit of 7750 SR7 dual homed to P routers using 2 x STM-16 interfaces. In addition, 1 x 1GE interface to MAN and 2 x 1GE interface to BRAS are planned.

5.3.9 Type B3

Type B3 POP have 1 unit of 7750 SR7 dual homed to P routers using 2 x STM-16 interfaces. In addition, 2 x 1GE interfaces to MAN and 2 x 1GE interfaces to BRAS are planned.

5.3.10 Type B4


5.3.11 Type B5


5.3.12 Type B6


5.3.13 Type C1

Type C1 POP have 1 unit of 7750 SR7 dual homed to P routers using 3 x STM-16 interfaces. In addition, 3 x 1GE interfaces to MAN and 1 x 10GE interface to BRAS are planned. 1GE interfaces are absent from the initial order, it is understood that VTN will be putting up additional order in time for deployment.

5.3.14 Type C2

Prouun, 3Type C2 POP have 1 it of 7750 SR7 dual homed to ters using 3 x STM-16 interfaces. In addition x 1GE interfaces to MAN and 3 x 1GE interfaces to BRAS are planned.

5.3.15 Type C3

Type C3 POP have 1 unit of 7750 SR7 dual homed to P routers using 3 x STM-16 interfaces. In addition, 2 x 1GE interfaces to MAN and 4 x 1GE interfaces to BRAS are planned.

5.3.16 Type C4


5.3.17 Type C5


5.3.18 Type D

Type D POP have 1 unit of 7750 SR7 dual homed to P routers using 4 x STM-16 interfaces. Interface requirement for Mobile Access/Core is yet to be determined.

5.3.19 HNI ASBR

Hanoi ASBR POP will have 2 units of 7750 SR7 dual homed to P routers using 5 x STM-64 interfaces as well as 5 x 10GE interfaces to VDC1. Alcatel-Lucent recommends minimum of 1 x 10GE links between the ASBR router pair for increased availability and improved routing convergence.

It was understood that Inter-ASBR link could only be added in subsequent expansion/variation order. Therefore, 10GE interfaces originally planned for connectivity to VDC has been reconfigured to allow for interconnectivity between ASBR routers. Alcatel-Lucent would request for VTN to plan for the replenishment of these 10GE interfaces where possible.

5.3.20 HCM ASBR

Ho Chi Minh ASBR POP will have 2 units of 7750 SR7 dual homed to P routers using 5 x STM-64 interfaces as well as 6 x 10GE interfaces to VDC2. Alcatel- Lucent recommends minimum of 1 x 10GE links between the ASBR router pair for increased availability and improved routing convergence.

It was understood that Inter-ASBR link could only be added in subsequent expansion/variation order. Therefore, 10GE interfaces originally planned for connectivity to VDC has been reconfigured to allow for interconnectivity between

ASBR routers. Alcatel-Lucent would request VTN to plan for the replenishment of these 10GE interfaces where possible.

5.3.21 DNG ASBR

Da Nang ASBR POP will have 1 unit of 7750 SR7 dual homed to P routers using 4 x STM-64 interfaces as well as 4 x 10GE interfaces to VDC3.

5.4 Aggregated Core plane facing uplinksFor the purpose of supporting P router plane planning, the below table illustrated the aggregated bandwidth from various PE routers.

Plane 1Plane 2

STM64STM16STM64STM16

HNI424420

HPG1615

DNG313313

CTO116112

HCM51057

Bandwidth138880171120138880141360

Aggr BW310000280240

5.5 Network elements types and positioning.The offered Network Provider edge (PE) router and Autonomous System Border Router (ASBR) are both Alcatel-Lucent 7750SR-7 service routers. PE routers are equipped with 2nd generation I/O modules (IOM-2) with 40Gbps/slot capacity giving a total of 200Gbps capacity per system. PE routers can be upgraded in the future with the replacement of IOMs and MDAs to achieve greater performance of up to100Gbps/slot. ASBR routers are equipped with 3rd generation I/O modules (IOM-3XP) and Media dependent Adapters (MDA-XP) with 100Gbps/slot capacity giving a total of 500Gbps capacity per system.

The 7750 SR-7 chassis is a fully redundant system and has a total of seven front access slots. Two card slots are dedicated forredundant common equipment, each of which holds one Switch Fabric/Control Processor Module (SF/CPM). Only one SF/CPM is required for full, non- blocking operation at 500 Gbps. A second SF/CPM provides complete redundancy of the fabric and the control processors. When two SR-7 SF/CPMs are installed, the traffic is load shared across the switchfabrics. The remaining five slots are used for Input / Output Module (IOM)baseboards.

The 7750 SR-7 chassis supports redundant power and fan trays and is designed to be NEBS Level-3 compliant. Power options for the SR-7 include -48V DC or120/240V AC power, supporting 1+1 redundancy. All power connection are madein the rear via 2 separate Power Entry Modules (PEMs). In addition to the PEMs, there are also DC line conditions that must be installed in the front of the chassis. System cooling within the SR-7 uses right to back airflow. Two (2) separate fan trays provide 1+1 cooling redundancy. Both fan trays are inserted in the back of the SR-7 chassis. The dimensions for the SR-7 are 14H x 17.5W x 23.5D, meaning five SR-7s will fit in a 7 foot Telco rack with roomfor a breaker panel.

The Juniper Networks T1600 Core Router is positioned in the VN2 network in the role of the P router. The T1600 is an (8) slot chassis, supporting up to 100Gigabit/sec per slot. Line card options to be used in the VN2 network include the 100Gigabit/sec T1600 Type-4 FPC,supporting two 4-port STM-64 PIC modules, and the40Gigabit/sec Type-3 FPC, supporting up to four 1-portSTM-64 or 4-port STM-16 PIC modules.

The T1600 is a fully redundant system. Switch Interface Boards (SIB) redundancy is available as N+N, with a minimum redundancy of 4+1 available at full system capacity, great redundancy available at lower utilization,

and a capability of graceful capacity degradation in the event of multiple failures. The Routing Engine (RE) used in VN2 is the RE-A-2000, and redundancy is 1+1, with a graceful failover capability available. Power Entry Modules (PEM) are also1+1 redundant with load sharing capability and with multiple inputs per PEMproviding a greater level of protection from individual circuit failure.


6Logical Network Topology

6.1 IP Addressing and naming convention.This section addresses the recommendation for IP addressing and general naming convention to be used on the nodes.

6.1.1 IP AddressingAll nodes in VN2 (P, PE, ASBR and RR) will use public IP for loopback and interface addresses. The MAN nodes will use public IP for loopback addresses and private IP for interface addresses.

MAN should use separate contiguous IP address blocks for the loopback and interface addresses.

6.1.2 Node NamingNode names will be standardized and have a fixed length. The nodes will be named based on the function that they are performing, the node model and their physical location. The format will be as follows:

Three-letters defining a site location; for exampleHNI for Hanoi or HPG for Hai Phuong.

A three-letter code indicating the function of the node; NPE for Multi Service Network PE router, NPR for Network Core router, ASR for ASBR Router, ACC for Access Router, SAM for5620SAM Application Server & SDB for5620SAM Database Server.

A two-digit number representing the node number at this location. Where a location has multiple nodes of the same function then there will be a requirement for each node to be uniquely identified.

For example:

HNINPE01 First 7750 SR Multi Service router in Hanoi

HPGAGG01 First 7450 ESS Aggregation router in HaiPhuong

All naming should be represented in UPPER CASE.

6.1.3 Port NamingAccess/Network port names/descriptions should be standardised in terms of representation. A proposal for the format is as follows:

::

Node name representing the location of the a- end of the link.

Node name representing the location of the b- end of the link.

Port number representing the port on the b-end. For example:HNINPE01:HPGNPE01:1/1/1

A 7750 SR router in Hanoi connected to the port 1/1/1 on a 7750 SRrouter in Hai Phong

6.1.4 Network Interface NamingA network interface is a logical IP routing interface that can be associated with either a physical or logical port or an SONET/SDH channel. Note that the interface name can be up to 32 characters, but must start with a letter. For ease of re- configuration it is useful to avoid the logical port being tied in name to the physical. Therefore it is recommended that the network interface name should include the originating node name and the destination node name followed by a link iteration in the event of more that one network interface exists between the same end points.

::

Node name representing the location of the a- end of the link.

Node name representing the name of the b-end of the link.

Two-digit number representing the iteration of this a-end and b-end where multiple network interfaces exist between two sites.

For example: HNINPE01:HPGNPE01:01The first interface connection on a 7750 SR router in Hanoi defined towards another 7750 SR router in Hai Phuong

All naming should be represented in UPPER CASE.

6.1.5 Service Distribution Point NamingDistributed VPN services across PE routers use service distribution points (SDPs) to direct traffic to another PE router through a service tunnel. SDPs are created on each participating PE router, specifying the origination address (the PE router participating in the service communication) and the destination address of another PE router. SDPs are then bound to a specific customer service.

Without the binding process, far-end PE router devices are not able to participate in the service (there is no service without associating an SDP with a service).

In the creation of SDP, the SDP identifier is an integer with a value of between 1-17407. A proposal for the SDP naming convention is to use the last three digits of the destination system address in decimal. Decimal zeroes must be included. For example, a far-end destination address of 10.220.11.5 would give an SDP name of005 using the last three digits. Equally, a far-end destination address of10.220.11.26 would give an SDP name of 026 using the last three digits.

Figure 3: Alcatel-Lucent Service Routing ArchitectureFull mesh SDP will be defined to provide 100% reachability to all PE nodes andASBR nodes.

6.2 IP Routing Recommendations

6.2.1 OverviewThe logical topology of the VN2 network consists of the device and interface names, the interface, link, and virtual addresses, and the suite of routing and control protocols, including both their individual scopes and interrelations, which are used to organize and bind together the physical components into a working network.

6.3 Routing and Control ProtocolsA combination of (4) primary routing and forwarding control protocols have been selected for the VN2 network. The protocols are:

Multiprotocol BGP

IS-IS

LDP

RSVP

IS-IS is the baseline routing protocol, running on all devices, and enabled on all core facing physical interfaces, enabling reachability among all devices. IS-IS does not enable any services related traffic to traverse the networks, but each of the other protocols builds on top of it. Details of the use of IS-IS are given in the IGP section.

RSVP is the next layer, using the information in IS-IS to build MPLS paths among the P routers, across the core WAN portion of the network. LDP is then layered on top of that, by tunneling LDP through the RSVP signaled paths in the core of the network. LDP runs natively between the edge and core to build a full mesh of MPLS paths among all edge routers. This enables some services, and provides the next building block for others. Details of the use of RSVP and LDP are given in the MPLS section.

BGP is the top layer. Its operation is abstracted for the forwarding path, making use of centralized route reflector nodes, which are accessed using routing information from the underlying protocols. BGP provides routing information required to enable most services, in the form of a destination route with a next-hop which is reached through the LDP signalled MPLS path. Details of the use of BGP are given in the BGP section.

6.4 End-to-End Logical Topology

BRAS

BRAS

Level 1

Level 1

PE PE P P

ISIS Level 1

ISIS Level 2

ISIS Level 1

ISIS Area ID 3

ISIS Area ID 1 ISIS Area ID 2

MAN MPLS Domain Dot1q

Backbone MPLS Domain

Dot1q

MAN MPLS Domain

LDP (tunneled)

Single hop RSVP

RSVP full mesh

Single hop RSVP

Figure 4: End-to-End Logical Routing Architecture

An illustration of all network device types and the scope of each protocol running on each device:

6.4.1 IGPThe proposed IGP for VN2 IP/MPLS backbone and the connected MANs is Integrated Intermediate System to Intermediate System (IS-IS) and requires that participating network elements support the use of OSI IS-IS for routing in TCP/IP and dual environments (RFC1195), Dynamic Hostname Exchange Mechanism (RFC2763), Support for Intermediate System to Intermediate System Cryptographic Authentication (RFC3567), and Support for Intermediate System to Intermediate System Extensions for Traffic Engineering (RFC3784).

The following are reasons for the selection of IS-IS over OSPF as the other IGP of choice:

1. Scalability. Far better scaling for the number of router nodes in a singlelevel of more than 1,000 routers vis--vis 250 OSPF router nodes in a single area. This will primarily mean that future expansion for VNPT would be greatly simplified as there is a whole lot more headroom for expansion without having to constantly relooking at creating more areas.

2. Security. IS-IS protocol is non-IP based, it is considered more difficult to spoof or attack from the Internet and therefore is relatively more secure in an ISP environment.

3. Lesser Resource Usage. IS-IS uses single LSP per router. OSPF has different SLA types and has one destination prefix per LSA.

4. Faster Convergence. IS-IS uses less packets to propagate routing information.

On top of these, for high availability interworking, Graceful Restart will be enabled.

6.4.2 IGP in Me tro NetworksAfter the meeting held on the 22nd of April 2009, it has been decided by VNPT that the IGP domain of the MANs be part of the IGP domain of VN2. The MANs will be IS-IS level 1 areas. Please refer to figure 4.

As some of the existing MANs are already actively serving customers and are running OSPF as the de-facto IGP, we would recommend that planning should begin for those MANs to be migrated to run IS-IS. This would greatly maintain consistency throughout the network, which is very important in maintaining an operational network.

6.4.3 In-band management to Metro NetworksAs the MANs and VN2 belong to the same IGP domain, in band management of the nodes in the MANs is possible from anywhere within VN2.

6.4.3.1 IS-IS NSAP AddressingAlcatel-Lucent proposes IS-IS as the IGP of choice because of it scalability, hierarchical capabilities, good convergence rate. The IS-IS Network Entity Title (NET) will use the OSI NSAP format. The format of the NET is composed of the following:

IS-IS NSAP format has two main components:

Initial domain part (IDP)

Domain Specific Part (DSP)

Each of these are broken down as:

IDP

DSP

Authority and format identifier: AFI

Initial domain identifier: IDI

High order domain specific part (HO-DSP)

System Identifier (SysID)

NSAP Selector (NSEL)

The following parameters shall be set

AFI shall be 49 (Private Addressing).

IDI shall be set to 00

The High order DSP shall be international country STD code as definedITU E164, hence 84.

The system ID shall be encoded in hex from the routers 32 bit system interface.

The NSEL shall be 00.

The system ID is a 6-octet field and defines an intermediate system (IS) within a particular area. Within the IS-IS domain, the system ID is formed from the IPv4 system (loopback) address. For example, the system IP address 172.16.0.101 will take the form 1720.1600.0101.

The first part of the NET consists of an Area Address. A variable length field defining the area to which an Intermediate System belongs.

6.4.3.2 ISIS HierarchyIS-IS support node population in excess of more than 1000 nodes, IS-IS hierarchy is implemented as illustrated in figure 4. The MANs and BRAS will be in IS-IS level1 areas. The VN2 PE and P nodes will be in a IS-IS level 2 area. The VN2 PE nodes will be configured as IS-IS level 1/2. Each MAN area, together with the backbone area would be in different IS-IS areas.

6.4.3.3 ConvergenceIn order to utilize IS-IS as a trigger for other fast convergence/High Availability protocols (such as IBGP Multipath/BGP peer tracking) it is necessary to manipulate key timers

1. Shortest Path First (SPF) interval. Each Intermediate System creates a topology map using Dijkstras SPF algorithm. The topology is calculated as a Shortest Path Tree (SPT) with itself as root. Aside from initialisation, an SPF calculation is executed when the Shortest Path Tree (SPT) topology is changed. This may include a link failure, or a change in link metric.2. IS-IS Link State PDU (LSP) Generation Interval. Is simply the time anIntermediate System should wait before generating and transmitting anLSP.

In order to achieve fast re-convergence however, the SPF interval needs to be configurable to a sub-second value. In addition, an exponential back-off is required in order that that Intermediate Systems are not continually computing shortest paths (and consequently using valuable CPU cycles) when subsequent computations are required.

All network elements should implement sub-second SPF and exponential back-off and will be configured with an initial SPF delay of 50 milliseconds. This initial wait value is to allow the router to flood the LSP that caused the trigger before actually executing the SPF calculation, and also to ensure that only one SPF calculation is carried out at receiving intermediate systems. The incremental wait between subsequent SPFs will be configured for 100 milliseconds, whilst the maximum SPF interval should be configured for 2 seconds.

Note: In the instance of a failed network link, the Intermediate System at each end of the link will generate an LSP. If the initial wait value is too small, then the potential exists that a receiving Intermediate System may execute an SPF after receiving the first LSP, then execute a second SPF after receiving the second LSP. Increasing the initial SPF wait value gives sufficient time to receive both LSPs even if one (or both) LSPs need are routed sub-optimally (as in the case of failure).

Table 1 details the SPF iterations given the defined values.

TimeIteration

50 msecFirst

100 msec after completion of first SPF executionSecond

200 msec after completion of second SPF executionThird

400 msec after completion of third SPF executionFourth

800 msec after completion of fourth SPF executionFifth

2 seconds after completion of fifth SPF execution.Sixth

Table 1 : SPF Execution with Exponential Back-off

For LSP generation, the initial wait should be configured for 0 seconds. That is, LSPs will be generated immediately following the event. The incremental wait between subsequent LSPs being generated should be configured for 1 second, whilst the maximum LSP interval should be configured for 8 seconds.

All network elements should implement sub-second SPF and exponential back-off and will be configured with an initial SPF delay of 50 milliseconds.

Actual values used will be decided during the interop testing with Juniper.

6.4.3.4 Traffic Engineering ExtensionRSVP LSPs will be used within the VN2 network. Therefore, IS-IS TE extension will be enabled on all P and PE nodes.

6.4.3.5 AuthenticationWithin any Service Provider environment, authentication of IGP updates is a desirable attribute. The backbone should be configured to support cryptographic authentication (HMAC-MD5) of IS-IS LSPs. Next to that its worthwhile to use also MD5 authentication for Hello LSPs. The latter provides a mechanism for securely authenticating adjacencies before LSPs can even be exchanged.

Separate commands will be used to define first, the authentication type (since7750 SR also support simple password or plain text authentication) and secondly the authentication key, which will be used to verify the PDUs sent by neighbouring routers on the interface.

Configuration of MD5 cryptographic authentication of Hellos is implemented under each interface configured under the IS-IS process. The authentication-key and authentication type must be reconciled for all routers on a link/segment :

6.4.3.6 Point-to-Point Broadcast Media

On point-to-point Ethernet, the election of a Designated Intermediate System (DIS) and regular generation of CSNPs is an unnecessary function. Point-to-point Ethernet links should therefore be configured such that no DIS election takes place and reliable flooding on these links is facilitated through generation of a PSNP to explicitly acknowledge received LSPs.

6.4.3.7 BRAS connectionThe IS-IS protocol includes both the concept of areas and also makes use of a two level design. IS-IS Level1 routers learn only default routing information from Level1/2 routers in their area. Level1/2 routers pass routes from Level1 up to Level2. It is desirable to support a 2nd tier of edge router downstream of the PE, but without requiring the PE(s) to at each POP to be in separate areas, or to form Level1 adjacencies with each other. To achieve this, the PE should configure links to downstream edge routers as Level1, but in the backbone area.

The BRAS router will take advantage of the 2nd tier edge router scheme for IS-IS in order to learn the loopback address of the upstream PE routers, and to provide the upstream PE with a route to its own loopback. This will require the PE to leak its loopback into Level2. The BRAS will then establish iBGP connectivity to the PE router(s). When dual PE routers and/or dual BRAS uplinks are configured, IS-IS will provide the mechanism for rerouting around a single link or PE router failure without requiring the propagation of a change in the BGP route across the entire network.

6.4.3.8 General Parameters (Suggested)6.4.3.8.1 MaxAge and LSP Refresh Interval

IS-IS LSPs have a remaining lifetime, which starts at the value of MaxAge and decrements to zero before they are flushed from a link state database. The LSPs must clearly be refreshed before the remaining lifetime expires. In order to reduce the amount of resource a router must dedicate to generating and processing LSPs, this default configuration should be modified to reflect an LSP MaxAge of 65535 seconds (18 hours) and an LSP refresh interval of 32767 seconds (9 hours).

6.4.3.8.2 Hello Interval and Hello-Multiplier

Once an IS-IS adjacency is established, Hellos are exchanged between the Intermediate Systems that act as keepalive messages. The IS-IS Hello interval should be configured as 1 second on all interfaces, whilst the hold time or hello- multiplier should be set to a value of 3. That is, 3 x Hellos must elapse before the adjacency is declared down.

6.4.3.9 IS-IS Scalability

Table 2 details the performance and scaling characteristics for IS-IS on the 7750SR platform.

Parameter7750 SR

Number of adjacencies252

Number of adjacencies on a single interface84

Number of LSP(s)25K

Number of routes250K

Number of routers in a level25K

Time to run SPF on 10K routes< 1 second

Table 2 : IS-IS Scalability

6.4.3.10 IS-IS timing overview (Suggested)

DefaultConfigured

csnp-interval10 sec10 sec

lsp-pacing-interval100 ms100 ms

retransmit interval5 sec5 sec

lsp-lifetime12000 sec65535 sec

spf-wait10 sec / 1 sec / 1 sec2 sec / 50 ms / 100 ms

lsp-wait5 sec / 0 sec / 1 sec8 sec / 0 sec / 1 sec

IS-IS L1 hello interval9 sec9 sec

IS-IS L1 hello multiplier33

IS-IS L2 hello interval9 sec1 sec

IS-IS L2 hello multiplier33

overload-on-boot P nodedisableddisabled

overload-on-boot PE nodedisableddisabled

overload-on-boot RR nodedisableddisabled

Table 3 : IS-IS timings

6.4.4 BGPThis section will describe the use of the Multi-protocol Border Gateway Protocol(M-BGP) protocol for the distribution of routing information within the VN2 IP Core.

6.4.4.1 UsageBGP is typically used to distribute routing information pertaining to destination networks. For example: customer networks, application services, or Internet destinations. Intermediate network infrastructure and devices are typically excluded from BGP.

Specifically, the following Network Layer Reachability Information (NLRI) types will be used, and the following types of routing information will be distributed by each:

IPv4 Unicast NLRI:

Upstream and Peer Internet Routes

Customer Internet Routes

Internal Internet and Application-Service Routes

IPv4 VPN Unicast NLRI:

Customer VPN Routes

6.4.4.2 Protocol Features/ParametersExtended Communities

Graceful Restart

MD5 Authentication

Default Timers Settings

6.4.4.3 Autonomous System NumberVNPT/VTN has indicated that there would be the requirement to offer inter-service provider VPN services, therefore the Autonomous System number used to deliver BGP/MPLS IP-VPN and Internet services shall ideally be a public ASN.

VNPT has indicated that a Private ASN will be used initially within VN2.

If no BGP router ID is specified, the global router ID will be used. If no BGP router ID and no global router ID is specified, the system interface IP address will be used.

Alcatel-Lucent recommends configuring the router-id as the system ip-address.

6.4.4.4 Participants and TopologyThe network devices that are required to participate in BGP are those forming the border with the above referenced network destinations (customers, services, and

the Internet). Specifically, all BRAS, PE, ASBR, and RR routers will participate inBGP. The transport core P routers do not require BGP routing information.

6.4.4.5 Route ReflectorThe key element of the BGP route distribution logical topology is the route reflector function, provided by dedicated RR routers. Route reflectors provide a centralpoint of BGP peering in the network, and eliminate the need for a complex full mesh of BGP sessions. For scalability route reflectors will be organized into two regions, North and South, with a unique cluster-id for each region. A full mesh of normal iBGP sessions will be built among all RR routers.

PPE ASBR

RR

NorthCentral & South

iBGP (NonCluster) iBGP, North Cluster iBGP, South ClusterRR

PE

P P

Figure 5: Route Reflector Architecture in day 1

PPE ASBR

RR

NorthCentral & South

iBGP (NonCluster) iBGP, North Cluster iBGP, South ClusterRR

PE

P P

Figure 6: Route reflector Architecture in day 2

On day-one each of two RR routers will establish iBGP sessions to all PE and ASBR routers, and associate these with their regional cluster. In the future both of the two RR routers in each region will establish iBGP sessions with all PE and ASBR router in the same region, and associate them with their regional cluster.

6.4.4.6 ASBRASBR routers serve the function of learning routes associated with internet peers or transit providers, and relaying them to the RRs. The ASBRs will establish iBGP sessions with the two RR routers. Each ASBR will establish eBGP sessions with peers and transit providers, learning the full internet routing table. Default BGP behaviour is for eBGP routes to be passed on to iBGP neighbours, so these routes will be passed on to the RRs. Next-hop-self should be used to rewrite the next-hop of external routes to the loopback of the ASBR itself. The ASBRs will learn internal/customer routes from the RRs.

6.4.4.7 PEPE routers participate in BGP in order to originate customer routes into the network. All PE routers will establish iBGP sessions with the two RR routers. The PE may also establish eBGP sessions with some transit customer, learning customer routes, and relaying them to the RRs as per default BGP behaviour, but also using a next-hop-self policy. The PE will also be configured to originate customer routes learned via other protocols into BGP, including VPN routes. The PE routers will learn both internet routes and other customer/vpn routes from the RRs

Each PE router, or pair of PE routers at a single POP location, will also be configured as BGP route-reflectors, using a unique cluster-id derived from the system address of the PE. The PE routers will establish iBGP sessions with all directly connected BRAS routers, and associate these sessions with the local cluster. In this way customer routes can be learned from the BRAS routers and relayed to the global RRs. Again, a next-hop-self policy will be used toward the RR so that all customer routes including those originated by the BRAS will be reachable via the loopback of the PE itself.

6.4.4.8 BRASBRAS routers will participate in iBGP in order to originate routes for locally configured customer IP pools into BGP. BRAS routers will establish iBGP sessions with directly connected PE routers only, and will be configured to originate customer IP pool routes. They will learn default routes only from the PE routers

6.4.4.9 AuthenticationThe TCP MD5 signature option for cryptographic authentication defined in RFC2385 should be adopted for both IBGP and EBGP sessions. The TCP MD5 signature option defines a TCP option for carrying an MD5 digest in a TCP segment, acting like a signature for that segment. As BGP uses TCP as its transport, it is inherently secure if this mechanism is adopted. Keys used for Internal BGP sessions should be different to those used for external peering.

Since authentication is performed between neighboring routers, the authentication key is configured on the neighbor level.

6.4.4.10 Policies and Route SelectionThe logical topology and flow of routing information described above will result in the following route selections and traffic flow:

Traffic entering the network through a BRAS router will follow a BGP default route with a next-hop of the directly connected PE router. Normal IP routing will be used to reach that next-hop.

Traffic entering the network through a PE or ASBR router will follow a specific BGP route to its destination, with a next-hop of another PE or ASBR. That next-hop will be reachable via MPLS. Each PE/ASBR willlearn all routes from two RRs as choose the best path from the two. In most cases these two routes will have the same next-hop.

The RRs learn all routes from the PE/ASBR routers, and each RR makes a single best path decision. Only the selected best path is relayed by each RR to its cluster members, so any polices required to influence BGP route selection must be implemented on the RR.

The RRs must participate in IS-IS and LDP in order to have visibility to the routing information for the next-hops of all BGP routes, especially MPLS next-hops for VPN routes

6.4.4.11 Routing of Internet TrafficThere will be three POPs where connections are made to the VDC network, which acts as an Internet transit provider to VN2. The three POPs correspond to the main points international circuit termination points on the VDC networks. The desired strategy is for VN2 to transport traffic to the POP used as the international exit point for that destination before handing it over to VDC. And conversely for VDC to hand traffic directly back to VN2 at the same POP where it was received, which should result in a symmetrical traffic flow of 'best exit / nearest entry' from the perspective of VN2

6.4.4.11.1 OutboundFor outbound traffic to reach the best exit POP from VN2 to the VDC network each PE must receive the best exit route from the RR. Then the PE will send traffic via MPLS to the ASBR with the next-hop of that route. The BGP route selection to determine the best exit must take place on the RR.

The RR receives routes from each of the (5) ASBRs, and must have a means of differentiating them. BGP communities should be applied by the ASBR to indicate the POP in which the route was learned, and BGP communities must be sent from VDC to indicate the POP where the routes come in from the international side. Policies can be used the increase the local-preference of the routes learned at the same POP as the international link preferred of that route on the VDC side.

6.4.4.11.2 InboundFor inbound traffic to take the nearest entry POP from the VDC network toward VN2, the international facing routers in the VDC network should select routes with the next-hop of the VDC ASBR at the same POP. This behavior can be achieved in several ways, depending on the logical topology used by the VDC network.

6.4.4.11.3 IllustrationThe following is a simplified illustration of the physical links, logical BGP protocol topology, and the flow of traffic through VDC to and from the Internet that will result from the routing policies described above. The traffic flow toward ISP B traverses the VN2 network from North to South to reach the exit point closest to ISP B. The traffic flow from ISP A takes the entry point closes to ISP A and does not traverse the VDC network from North to South.

VN2

P PE

VDCASBR ISP A

NorthSouth

RRs

ASBR P

BGP

Best Exit traffic flow

ISP B

Nearest Entry traffic flow

Figure 7: Internet Traffic flow via VDC

6.4.4.12 BGP timing overview (Suggested)

DefaultConfigured

min-route-advertisement30 sec30 sec

connect-retry120 sec120 sec

hold-time90 sec90 sec

keepalive30 sec30 sec

min-as-origination15 sec15 sec

Table 4 : BGP timings

6.5 IP Multicast RecommendationsP2MP LSPs is an alternate method of distributing multicast traffic. It provides the50ms MPLS FRR capability. This feature will need to be tested and verified between the P and PE nodes to ensure a stable rollout of multicast services using this feature. Until this feature operationally tested, PIM will be used for multicast services.

Multicast traffic in VN2 is classically routed based on PIM-SM, which is best suited multicast routing protocol for sparsely dispersed sources, and more importantly,has less protocol control overhead compared with other multicast routing protocols.

It is advised that each interface has PIM-SM correctly configured, to ensure proper operation and correctly forwarding of the multicast traffic across the backbone.

PIM will be enabled on all routers in VN2 IP Core and the MANs. The MAN AggPE routers will for PIM neighbours with the IP Core PE routers.

IP Core

PIMPIM

IGMP static joins

Duplicated MC Traffic stream

MAN

PIM

Figure 8: PIM Topology

For multicast resiliency, the IP Core PE will use IGMP static joins to distribute duplicate copies of desired multicast streams to the two MAN Agg PEs. With PIM, the MAN is able to intelligently manage duplicated multicast streams effectively by ensuring that the end devices only receive a single stream.

6.5.1 Rendezvous Point (RP) placementAccording to the PIM-SM protocol, all join messages are forwarded to a specific IP address, the rendezvous point (RP). It is recommended to place the RP as close as possible to the source.

Considering IPTV as the primary use of multicast in VN2, it is understood that the VHO will be located within the vicinity of the HNI site; therefore, it is recommended that the RP be designated on HNI PE router nodes.

PPE

(Anycast RP)

IPTV Source (Primary)

Edge

PIMSM

NorthSouth

PEP

MSDP

PE

(Anycast RP)

IPTV Source (Backup)

IGMP

Figure 9: Multicast RP TopologyHowever, moving forward should the requirement for multicast VPN increases, it would be advisable to relocate the RP to a more centralise node within VN2 so that join latency is optimum throughout the entire network.

Redundant RP should be implemented using Any-Cast RP whereby both HNI PE routers will be configured with the same RP address on a loopback interface. Redundant RP in accordance to draft-ietf-pim-anycast-rp-0X should also be configured to ensure that both RP will be capable of synchronising source information. This is to ensure that any failure to either RP, the redundant RP have the most current source states and thus minimises convergence delay.

6.5.2 Reverse Path Forwarding (RPF) considerationsMulticast RPF is used in conjunction with PIM-SM to ensure loop-free forwarding of multicast packets. In multicast routing the decision to forward traffic is based upon source address and not on destination address as is the case with Unicast routing. It does this by utilizing either a dedicated multicast routing table or alternatively the router's native Unicast routing table.

When a multicast packet enters a router's interface it looks up the list of networks that are reachable via that input interface. If the router finds a matching routing entry for the source IP of the multicast packet, the RPF check passes and the packet is forwarded to all other interfaces that are participating in multicast for this multicast group. If the RPF check fails the packet will be dropped.

It is recommended that RPF-check should be enabled based on both Multicast & Unicast RIB.

6.6 MPLS Topology and Signalling Overview.

6.6.1 OverviewThis section will describe the use of Multi-Protocol Label Switching (MPLS) for forwarding traffic within the VN2 IP Core network.

P PASBR

PE Single Hop RSVP

RSVP FullMesh BetweenP routers

PEPOP A P

P POP B

RSVP Signaled LSP LDP over RSVP

Figure 10: VN2 MPLS Topology

6.6.2 UsageMPLS will be used in two fundamental ways in the VN2 network. The first is as the baseline forwarding scheme across the WAN portion of the network, providing traffic engineering and fault protection. And second tier of MPLS is used as a unified means of connectivity between edge routers, enabling end-user services like Layer3 VPNs, VPLS, and Pseudowires.

Two sets of protocols, participants, and topologies will be used for each, as follows:

Backbone MPLS:

RSVP and LDP protocols

Full mesh RSVP LSPs between P routers

LDP will be enabled for full mesh topology to be dynamically created by default

Statically configured dual mesh topology (A + B planes)

Traffic Engineering (TE) capabilities

Enhanced fault protection capabilities

Edge MPLS:

LDP and RSVP protocols

Single hop RSVP LSPs between P router and PE or ASBR

T-LDP sessions between PEs and ASBRs will be tunnelled over RSVPtunnels (LDPoverRSVP)

LDP will be enabled for full mesh topology to be dynamically created by default

Follows IGP to determine paths, dependent for re-convergence

Graceful restart and Non stop routing enabled for LDP

6.6.3 Backbone MPLSThe backbone MPLS scheme based on the Resource Reservation Protocol (RSVP) protocol provides a highly configurable framework for forwarding traffic across the core of the network. As RSVP LSPs must be explicitly configured at the origin for each source-destination pair, its use of full mesh is generally confined to the network core, in order to limit the size of the N squared full mesh of LSPs among participating PE routers. In the VN2 network, the full mesh RSVP LSPs will be limited to the P routers.

6.6.4 RSVP Signalled LSP TopologyThe following grids give a complete list of the unidirectional RSVP signaled LSPs to be configured among the P routers. Check marks indicate an LSP which corresponds to a direct physical link. Check-plus indicates an LSP with an intermediate hop.

Plane A

From:

To:P1 - HPYP1 - HNIP1 - DNGP1 - HCMP1 - CTO

P1 - HPY999+9+

P1 - HNI9999+

P1 - DNG9999

P1 - HCM9+999

P1 - CTO9+9+99

Plane B

From:

To:P2 - HPYP2 - HNIP2 - DNGP2 - HCMP2 - CTO

P2 - HPY999+9+

P2 - HNI9999+

P2 - DNG9999

P2 - HCM9+999

P2 - CTO9+9+99

Interlinks

From:To:

P1 - HPYP2 - HPY

P1 - HNIP2 - HNI

P1 - DNGP2 - DNG

P1 - HCMP2 - HCM

P1 - CTOP2 - CTO

6.6.5 Traffic EngineeringRSVP provides a number of features for fine grained control over the path selection for each LSP, collectively providing the Traffic Engineering (TE) capabilities. Those parameters include a subscribed bandwidth per LSP, admin groups (or colors) per link for inclusion or exclusion, a link metric scheme for TE, and the hop count. Also, RSVP paths can be explicitly defined on a per-hop basis, or allowed to follow the same IGP metric based path selection.

As the traffic levels are expected to be low on the VN2 network on day one, but are otherwise unknown, it is proposed to use a course bandwidth allocation scheme, well below the actual available link bandwidth. This scheme may be used forfuture true bandwidth subscription based TE, or as a means to load share paths across future parallel links.

Otherwise, the largely single-hop LSP topology, and the implied primary/backup paths given by the physical circuit topology, largely negate the need for other types of TE configuration. Only the LSP marked with a check-plus in the above gridhave any intermediate hop which might be subject to a policy based path decision, and these would be expected to be among the lowest bandwidth consumption

paths. As such, no other TE configuration is recommended, and the use of theIGP cost scheme to determine LSP paths should be sufficient.

6.6.6 Edge MPLSThe edge MPLS scheme leverages the multiprotocol capabilities of MPLS to support converged service offerings. Single hop RSVP LSPs will be used to connect the PE and P routers. The RSVP full mesh is only within the P routers. Source PE routers will use LDP over RSVP tunnels to reach the destination PE routers. With properly placed redundant links between the PE routers, thisprovides MPLS FRR capability between PE and P routers without the need of a full mesh end to end RSVP tunnels between the PE routers.

As a backup to RSVP, LDP can be used to build a full mesh of LSPs among thePE, ASBR and RR routers.

LDP must be enabled on all of the following:

Links from PE to P routers

Links from ASBR to P routers

Links from RR to P routers

Links connecting two PE routers at the same POP

Links connecting two P routers at the same POP

Via tunnelling over all RSVP LSPs connecting to P routers

LDP uses the IGP to determine the path used, and also counts on the convergence of the IGP in the event of a failure. In the event of a failure on a PE/ASBR/RR to P router link the recovery of the LSP will be dependent on the IGP convergence around the failure.

6.7 High AvailabilityThis section describes high availability features of the core VN2 topology. For VPN service high availability, please refer to Section 7.2 Service Architecture as the topic of offering VPN high availability is being discussed as an extension.

6.7.1 Border Router related FailureFor VN2 ASBR sites with redundant ASBR, single ASBR failure would not constitute to any prolonged failure. Redundant ASBR router will be capable of performing all forwarding should the primary ASBR router fails.

In the rare event of a critical failure in VDC or ASBR site, it will render traffic un- reachable to and from a particular set of ASBR routers. At this point in time, VN2BGP will converge and HSI traffic will be automatically re-routed to the next administratively nearer ASBR routers.

This behaviour covers the following scenarios:

1. Failure of VDC connect BGP routes gets withdrawn on ASBR, the update is populated to both BGP route reflectors and subsequently to other PE routers and BRAS within VN2. PE and BRAS will reconsider other available routes in the Routing Information Base and installs the new routedestination in the forwarding table for traffic to be re-routed.

2. Failure of ASBR routers it would be extremely rare for both redundant ASBR routers to fail at the same time, nevertheless, should this occurs, BGP neighbour association with BGP route reflectors would drop and all BGP routes previously learned from the affected ASBR routers will be withdrawn, this update is then populated to PE routers and BRAS within VN2. PE and BRAS will reconsider other available routes in the Routing Information Base and installs the new route destination in the forwarding table for traffic to be re-routed.

ailurFigure 11: Fe of ASBRs and/or VDC interconnectIn Figure 11, HPG PE routers re-converges upon BGP notification that the preferred traffic egress via HNI ASBR is to be changed to ASBR DNG due to a major failure on either VDC or the HNI ASBRs.

6.7.2 PE upstream related FailureIn the event of failure on the upstream of the PE router, POP with single PE router and redundant uplinks will converge and uses the alternate path towards its destination as in the example illustrated by Fig 12 for CTO POP.

re 1Figu2: PE upstream related failure

As for POPs that has dual PE routers, such as the HPG POP illustrated, uplink redundancy will have to be handled via the MAN-E. This is a temporary workaround until inter-PE links are made available.

6.7.3 P router RSVP Label Switch Path RecoveryBased on the physical topology of the VN2 network and the proposed dual mesh LSP topology, most RSVP LSPs will correspond to a direct circuit link between two routers, and all POP-to-POP circuit paths are deployed with two parallel physical circuits. As such, most recoverable LSP interruptions will be caused by link failure, and the desired response is for traffic to be shifted from the failed circuit to the parallel circuit.

Physical LinksRSVP LSP and Primary Path

POP 1LSP B, Pri. PathLSP A, FRR PathLSP A, Pri. PathRSVP LSP Recovery Path

POP 2Figure 13: RSVP LSPs and Recovery Path

This failure scenario lends itself well to link protection using Fast Reroute. TheFRR path used would be via neighbouring P routers in the same POP, and overthe parallel circuit, as in the following illustration. This would be expected to be the path of the re-established LSP as well.

6.8 Network QoS Design

6.8.1 QoS OverviewThe VN2 provides service differentiation using the Differentiated Service Model. It provides a high-level of flexibility to meet different operating environments, which can sometimes appear daunting and/or complex; indeed, there is a faint line between flexibility and complexity. Prior to detailing the recommended QoS configuration for the VN2 Network, this section provides a high-level overview of QoS and traffic management on the 7750 SR. This section is not intended to detail every aspect of traffic management within the 7750 SR, only the parts pertinent to the proposed QoS model.

6.8.1.1 Traffic ClassificationIn order to achieve both scalability and service differentiation, the 7750 SR aggregates microflows in up to eight Forwarding Classes (FC). The behaviour of a FC in terms of ingress marking interpretation, egress marking, and queuing/scheduling is configurable in order to define a different Per-Hop Behaviour (PHB) per FC.

By default the Forwarding Classes are classified into three class types; High priority, Assured, and Best Effort (BE). High-priority FCs are serviced before Assured FCs and are intended to be used for real-time delay-sensitive traffic or network control traffic. In turn, Assured FCs are serviced before Best Effort FCs and provide services with a committed rate (Committed Information Rate, or CIR) and peak rate (Peak Information Rate, or PIR). Assured FCs provide the ability to classify ingress traffic as either in-profile or out-of-profile based upon the traffic arrival rate. A queue is considered in the in-profile state if the rate at which the queue is being serviced is less than its configured CIR. A queue is considered out- of-profile if the rate at which the queue is being serviced is greater than its CIR, but less than its PIR. After the profile state of the packet is determined at service ingress, the profile state of the packet influences the packets queuing priority and drop preference at all subsequent queuing points. It is worth noting that the in- profile/out-of-profile classification based upon the rate that a queue is being serviced can only be performed on access ingress ports (not network ingress ports). When given some thought this makes sense because at network ingress ports traffic has already been policed (at access ingress).

Table 5 details the eight Forwarding Classes within the 7750 SR SR together with the type of class and its intended use.

FCFC nameClass TypeRemarks

NCNetworkControlReal TimeNetwork Control traffic

H1High-1Real TimeFor a second Network Control traffic or delay/jitter sensitive data

EFExpeditedReal TimeFor delay/jitter sensitive data

H2High-2Real TimeFor delay/jitter sensitive data

L1Low-1AssuredAssured traffic

AFAssuredAssuredAssured traffic

L2Low-2Best EffortFor BE traffic

BEBest EffortBest EffortFor BE traffic

Table 5: Forwarding Classes

In order to understand the 7750 SR QoS architecture, it is important to understand the concept of Forwarding Classes. As a standalone entity FCs do not provide any QoS capability such as classification, metering, marking or policing. FCs are a conduit and require an input (classified traffic) and an output (one or more queues).

I FC

Figure 14: Forwarding Class Concept

6.8.1.2 Components of the DiffServ ModelWithin the 7750 SR DiffServ architecture, there are four constituent components that define the PHB afforded to traffic. These are the Access Ingress, Network Egress, Network Ingress, and the Access Egress.

Ser vice Acce ssPoi nt

Ser vice Acce ssPoi nt

Ser vice (Access) Ing r ess

Networ kEg ress

Networ kIng ress

Ser vice (Access) Eg r ess

Figure 15: DiffServ PHB ComponentsAccess Ingress Associated with a Service Access Point and responsible for classification of traffic into an FC based upon Layer-2 (i.e.

MAC, 802.1p, Ethertype) Layer-3 (i.e. source/destination IP, protocol, DiffServ Codepoint/IP precedence) or Layer-4 (i.e. source/destination port) criteria. Ingress queues are subsequently associated with each FC and resource such as packet buffer; permissible traffic rates for in-profile/out-of- profile traffic, and scheduling priority (towards the fabric) are allocated to each queue.

Network Egress Associated with a network interface and responsible for mapping each FC and profile (in/out) into an associated MPLS EXP value. Egress network queues are associated with each FC and again resource such as packet buffer and scheduling priority (towards the line) are allocated to each queue.

Network Ingress Associated with a network interface and responsible for classification of traffic into FCs based on MPLS EXP or DSCP bits. Ingress network queues are associated with each FC and resource such as packet buffer and scheduling priority (towards the fabric/other network processors) are allocated to each queue.

Access Egress Associated with a Service Access Point for mapping traffic into egress queues based upon FC or DSCP. Resource such as packet buffer, permissible traffic rates, and scheduling priority (towards the line) are allocated to each queue. Remarking of Layer-2 802.1p bits can also be implemented at the Access Egress (recall that 802.1q bits are not carried over a pseudowire).

6.8.1.3 Buffer ManagementLogical default buffer pools exist at the port and media dependent adapter (MDA) levels. Each physical port has three logically associated pools; an ingress access pool, and egress access pool, and an egress network pool. If the mode of a port is configured as access, only ingress access pool and egress access pool arecreated for the port. Conversely, if the mode of a port were configured as network, only the egress network pool would be created for the port. The size of the buffer pools is automatically determined as a function of the MDA type and the port configuration (i.e. mode and speed of the port).

Each buffer pool has a reserved and a shared buffer portion. Each Forwarding Class queue with a non-zero CIR is allocated buffers drawn from the reserved portion of the buffer pool. After a queue consumes its reserved buffers, it competes with other queues in the pool for shared buffers drawn from the shared buffer portion of the pool.

CBSWREDShared BuffersRes erved buffersM BS

Res erved buffersRes erved buffersFigure 16: Shared/Reserved Buffer

Reserved buffer is allocated to queues using the Committed Burst Size (CBS) parameter and shared buffer is allocated using the Maximum Burst Size (CBS) parameter. Optionally, Weighted Random Early Detection (WRED) can operate in the shared buffer to provide differing random drop thresholds for in-profile (high- slope) and out-of-profile (low-slope) traffic.

The shared buffers are not reserved for any specific queues. Any queue within the buffer pool can use this space when its reserved buffers are full and it has not exceeded its MBS. The number of shared buffers within a buffer pool is the difference between the total number of buffers in the pool and the reserved buffers.

6.8.1.4 QueuesQueues are created within buffer pools. For example, a 10 x 1-Gigabit Ethernet MDA has 250MB of buffer, which, when spread across the ports means that each Gigabit Ethernet interface has 25MB of available buffer. This 25MB can be allocated to a single queue (unlikely), or it can be allocated to a number of queues (a more likely scenario).

Forwarding types have the intention of providing the capability to control processor intensive tasks such as packet replication at network ingress. A VPLS service supports four forwarding types; unicast, multicast, broadcast, and unknown- unicast. A VPRN will support two forwarding types; unicast and multicast. At service ingress, a queue can be allocated per-Forwarding Class, per-forwarding type. With eight possible FCs, it follows that a VPLS can support up to 32 queues, and a VPRN can support up to 16 queues. At service egress and network egress, there is no concept of forwarding type, hence each of the eight supported Forwarding Classes is allocated a single queue.

A queue has two main buffer-related parameters; CBS and MBS, and two main scheduling parameters; CIR and PIR.

CBS Specifies the number of buffers that can be drawn from the reserved buffer portion of the queues buffer pool. At service ingress and egress, the CBS is defined in Kbytes. At network ingress and egress, CBS is defined as a percentage of the buffer space of the queue buffer pool.

MBS Specifies the maximum queue size of a queue. For packets that enter a queue with a queue size between the CBS and MBS, buffers are drawn from the shared portion of the buffer pool, which may be managed by the WRED function. At service ingress and egress, the MBS for a queue is defined in Kbytes. The MBS for a network queue is defined as a percentage of buffer space of the queue buffer pool.

Note that the minimum buffer size granularity is 4 Kbytes, and as partial buffer blocks cannot be allocated, the values allocated to CBS and MBS are automatically rounded up to a number of full buffer blocks.

CIR The CIR for a queue performs two functions. The first function is Profile Marking. At service ingress it marks packets as in-profile or out-of-profile based on the CIR of the queue. For each packet that is transmitted from a service ingress queue, the CIR is checked against the current transmission rate of the queue. If the current rate is at or below the CIR threshold, the transmitted packet is internally marked as in-profile. If the current rate is above the threshold, the transmitted packet is internally marked as out-of- profile. The second function performed by the CIR value is Scheduler Queue Priority Metric. The scheduler that serves a group of ingress or egress queues prioritises individual queues based on their CIR and PIR states. Weighted queues that operate below their CIR are always serviced before queues that operate at or above their CIR. The CIR for service ingress and service egress queues is provisioned within a SAP-ingress and SAP-egress QoS policy respectively and is defined in Kb/s. The CIR for network queues is defined within the network QoS policy and is defined as a percentage of the network interface bandwidth.

PIR The PIR defines the maximum rate at which packets can be scheduled from a queue. The PIR does not specify the maximum rate at which packets can enter a queue as this is determined by the ability of the queue to absorb bursts and is defined by the MBS. The PIR for service ingress and service egress queues is provisioned within a SAP-ingress and SAP-egress QoS policy respectively and is defined in Kb/s. The PIR for network queues is defined within the network QoS policy and is defined as a percentage of the network interface bandwidth.

6.8.2 Scheduling

The hardware scheduler for a queue determines how it will be scheduled relative to the other queues at the hardware level. When a queue is defined in a SAP-ingress or SAP-egress QoS policy, it is necessary to define the hardware scheduler

(queue-type) to use for the queue when it is applied to a SAP. The definition of hardware schedulers (queue-types) can be expedited or best-effort. Scheduling ingress/egress scheduling, queues are serviced in the following order

High-priority (expedited) queues operating within their CIR

Low-priority (best effort) queues operating within their CIR

All queues that operate within their PIR but above their CIR

CI R = PI R Q s

In-Profile RR

CI R = PI R Q s

Exped ited Q ueues

Strict PriorityOut-Profile RR

CI R = PI R Q s

In-Profile RR

CI R = PI R Q s

Best Effort Q ue ues

Out-Profile RR

Biased RR

Figure 17: Scheduling

In the first pass, all the queues that are associated with the high-priority queues and operate within their CIR are serviced in a round-robin manner. The servicing of a queue is stopped after it has transmitted packets up to its operational CIR. Hence, queues get out of first pass one after another and the first pass is repeated until the last queue gets out.

In the second pass, all the queues that are associated with the low-priority queues and operate within their CIR are serviced in a round robin manner. The second pass is repeated until the last queue is serviced up to its CIR.

In the third pass, all the queues in out-of-profile state (above CIR, but within PIR) are serviced in a biased round robin manner. It is biased round robin because the queues associated with high priority queues obtain at least 50% of third pass bandwidth if there is enough traffic in those queues. Similar to the first and second pass, third pass is repeated until the last queue is serviced up to its PIR. The third pass is basically round robin, however, every time it is interrupted by the twohigher priority passes, it always resumes servicing with high priority queues; hencebiased round robin.

6.8.2.1 Virtual Hierarchical SchedulingSingle-tier scheduling provides a scalable and flexible solution to share bandwidth. However, certain scenarios require a more flexible scheduling policy at the service ingress and/or service egress. In this instance, the 7750 SR provides Hierarchical Virtual Scheduling (commonly referred to as H-QoS) to provide another level of sophisticated scheduling.

Consider the following scenario. The CIR and PIR of the three service ingress queues that correspond to Gold, Silver, and Bronze services are configured as follows

Gold CIR 10 Mb/s, PIR 10 Mb/s Silver CIR 20 Mb/s, PIR 40 Mb/s Bronze CIR 0 Mb/s, PIR 100 Mb/sUsing single-tier scheduling, each queue can burst up to its specified PIR, which basically means that up to 150 Mb/s can enter the service through the three queues.

Using virtual hierarchical scheduling, a parent scheduler can be created for the Gold, Silver, and Bronze queues, which limits the aggregate rate of all the queues to 100 Mb/s. With virtual hierarchical scheduling, the service can offer any combination of Gold, Silver, and Bronze traffic that conforms to their specified PIR values, but does not exceed 100 Mb/s in total.

The 7750 SR hierarchical scheduler structure is very similar to t-ary tree data structure. In a virtual hierarchical scheduler structure, the buffer queues form the leaves of the structure. A parent (intermediate node or root node) is always a virtual scheduler. As in the case of t-ary tree, in a hierarchical scheduler policy a parent scheduler can have any number of children. A virtual scheduler can have queues and other virtual schedulers as its child. Each child scheduler or queue can have only one parent scheduler. The number of levels in a t-ary tree is called its height. In a hierarchical scheduler structure, the number of levels is defined in the context of tiers.

The 7750 SR supports up to 3 tiers of virtual schedulers. The root schedulers are defined in Tier 1. Schedulers defined in Tier 2 can have parental associations with schedulers defined in Tier 1. Schedulers defined in Tier 3 can have parental associations with schedulers defined in Tiers 1 or 2. Queues can have parental associations with schedulers at any tier level.

In a hierarchical scheduler policy, a virtual scheduler or a queue is provisioned with a CIR and a PIR that effectively dictate how much bandwidth is allocated to that queue/scheduler and the maximum rate at which it can be serviced. In addition,the queue/scheduler is configured with a CIR Level, CIR Weight, Level, andWeight.

A parent scheduler distributes its allocated bandwidth to its children (where children can be queues or lower tier schedulers) in two passes. The first pass is

within CIR pass and the second pass is above CIR pass. The first pass of bandwidth allocation to a child queue/scheduler ends after its servicing rate reaches its configured CIR value. The second pass of bandwidth allocation to a child ends after its servicing rate reaches its configured PIR parameter value.

During the first pass, the allocation of bandwidth and the order that a child is serviced relative to other children is based on the childs CIR Level and CIR Weight parameters. During the second pass, the allocation of bandwidth and the order that a child is serviced relative to other children is based on the childs Leveland Weight parameters. The range of both CIR Level and Level is 1 to 8. A higherlevel (8 is the highest priority) is exhaustively serviced before a lower level is serviced. Effectively, CIR Level and Level are used to define the strict priority queuing order.

The range of the CIR Weight and Weight parameters is 0 to 100. In both passes, the (CIR) Weight parameter defines the weight of a child within the given (CIR) Level. When multiple children share the same (CIR) Level on a parent sc

VNPT VN2 Network Design

Documents

Transcript of VNPT VN2 Network Design