Dynamic Circuit Network
Hands-On Workshop
University of Nebraska-Lincoln Nebraska Student Union
Lincoln, NE July 19th and 20th, 2008
Welcome!
• Wireless – cannot access workshop system from
Joint Techs Wireless
• Wired connections also available
Welcome!
• This is the 7th DCN Workshop– Nysernet– MAX– NASA Ames– University of Houston– University of Hawaii (double header)– University of Nebraska - Lincoln
• Introductions
Welcome!• Key objectives of this workshop are:
– Disseminate information to the R&E community regarding the emerging class of Hybrid Network and the associated techniques for Dynamic provisioning and configuration
– Review in detail and provide instruction on how to use the control plane software currently in service on the Internet2 Dynamic Circuit Network (DCN), ESnet Science Data Network (SDN), and several regional networks.
– Obtain feedback directly from the community on how to improve the technologies…Hopefully, to help guide future development and deployment priorities and speed adoption
– Review the state of implementation and deployment of these types of dynamic networks throughout the R&E community.
Instructors
• Tom Lehman (USC/ISI)• Chris Tracy (MAX)• Andy Lake (Internet2)
• These people are involved in numerous projects related to deploying dynamic control planes:– Internet2 Dynamic Circuit Network – ESnet OSCARS Project – NSF DRAGON– Internet2 HOPI Testbed– DICE (Dante, Internet2, Canarie, Esnet) – International
development activities
Why do a workshop?• Dynamic Hybrid Networks are new…
– The service concepts are still unfamiliar to many networker experts and users… What does one gain with DCN?
– The software and hardware implementations are still evolving…
– Even the standards are still evolving…– The networks that support these capabilities are few but
growing.– The user base is small [for now]…. But will grow as the
capabilities mature and become more ubiquitous, persistent, robust, and the utility of both connection oriented services and dynamic provisioning becomes more widely recognized and accepted.
• Providing hands-on experience to design and deploy these architectures is one way to broaden and promote adoption.
Agenda• Day 1
– 9:00 am Overview of GMPLS and DRAGON– 10:00 am Exercise #1: Designing a GMPLS Control Plane
for Ethernet Data Planes– 10:15-10:45 am Break– 12noon Lunch– 1:00pm Continue working on Exercise #1– 2:00pm Overview of Web Services and OSCARS– 2:30-3:00pm Break– 3:00pm Exercise #2: IntraDomain provisioning with OSCARS– 5:00pm AdjournDay 2
– 9:00am Overview of Inter-Domain implementation in OSCARS– 10:00amExercise #3: Inter-domain Provisioning with OSCARS– 10:15-10:30am Break– 12noon Lunch– 1:00pm Continue working with Exercise #3– 2:30-3:00pm Break– 3:00pm Use of Internet2 DCN and peering dynamic networks– 4pm Adjourn
Workshop Perspective• In this workshop we focus on implementation
– We will design and build a multi-domain GMPLS controlled ethernet network– We have a mobile GMPLS test and evaluation lab consisting of 24 PCs and 12
switches
• We will be focused on the GMPLS intra-domain control plane issues
– Specifically, OSPF and RSVP protocols and Path Computation– We will do a very brief and cursory review of RSVP and OSPF.
• For detailed information on the protocols themselves see the IETF RFCs. • We will not deal with ISIS or CR/LDP or LMP
• We will focus on the “DICE” Inter-domain architecture – Web Services based topology distribution and provisioning
• We use open source software developed by the NSF DRAGON Project, the DOE OSCARS Project
– Intra-domain: Adapted versions of KOM-RSVP and Zebra OSPF plus the NARB for path computing
– This software is the only GMPLS software available to support dynamic ethernet services
– Uses OSCARS (Dept of Energy) for book-ahead scheduling and AAA– Additional software and interfaces have been developed under auspices of the
DICE effort (DANTE, Internet2, Canarie, ESnet)– The code has been adapted to support a wide variety of vendor equipment
(e.g. Force10, Extreme, Dell, Ciena, Cisco, Raptor)
OSCARSDRAGON DOE
Office of Science
Internet2 CoreDynamic Circuit Network
Green PodASN4
Red PodASN1
Yellow PodASN3
Blue PodASN2
DCN Workshop Architecture
Data Plane
Control Plane
Pod Network Elements Control and Data Planes
VLSR1-PC
VLSR1-SW
ES1
Virtual Label Switching Router- “VLSR”
VLSR1
ES2
VLSR3-PC
VLSR3-SW
VLSR3
VLSR2
NARB / IDC
Network Aware Resource Broker- “NARB”Inter-Domain Controller – “IDC”
gre2 gre4
gre3
gre6
Control Plane PC (VLSR#-PC, NARB, IDC)
Data Plane Ethernet Switch (VLSR#-SW)
End System (ES#)
D2 D4
D3D1 D5
Dynamic NetworksOverview and Status
• Objectives and of Dynamic Hybrid Networks
• Hybrid Networking and the Global R&E Community
• Standardization Efforts • Internet2 Dynamic Circuit Network
(DCN)– Control Plane Software– Network Architecture
Hybrid Networking
• There has been interest from many communities for the development of network architectures and mechanisms that utilize lower layers of the protocol stack along with IP at layer 3
• This has become known as “hybrid networking”
• It is motivated by applications from the research and education community that require greater capabilities– High bandwidth flows (for example, flows that
come close to saturating links in the shared IP backbone)
– Flows with special requirements related to quality of service, for example jitter requirements
– Network and Application Virtualization
Hybrid Networks - Motivating Factors
• Hybrid networks are intended to provide a flexible mix of IP routed service and “lower layer services” – “flexible” means the network can respond quickly to
user/application/connector requirements and requests to access both the IP Routed and/or lower layer services
– “lower layer services” means access to layer 2 and below paths which can be utilized in a multitude of ways by creative users.
• Typical user requirements for these lower layer services are based on: – critical, large bandwidth flows which may require one of
more of the following: deterministic network performance, dedicated network resources, guaranteed network capacity, freedom to use protocols other than (congestion control friendly) TCP, privacy/security requirements, scheduled services
– User/application communities which desire to build entire topologies which integrate domain specific resources along with dedicated network resources (which have one or more of the above mentioned characteristics)
Hybrid NetworksHeterogeneous By Nature
• Hybrid networks are extremely heterogeneous at several levels
• DataPlane can be constructed from– router based Multiprotocol Label Switching (MPLS)
tunnels– Ethernet VLAN based Circuits– Synchronous Optical Network / Synchronous
Digital Hierarchy (SONET/SDH) circuits– Wavelength Division Multiplexing (WDM)
connections– Combinations of the above
Hybrid NetworksHeterogeneous By Nature
• Control Planes can be based on– Multiprotocol Label Switching (MPLS)– Generalized Multiprotocol Label Switching
(GMPLS)– Web Services– Management Systems– Combinations of the above
• Client (user) services or attachment points could be– Ethernet– SONET– IP Router– InfiniBand
Multi-Domain, Multi-Layer Control Planes Key Requirements
• The “Multi-Layer” is meant to identify several items regarding how hybrid networks may be built. In this context it includes the following:– Multi-Technology - MPLS, Ethernet, Ethernet
PBB-TE, SONET, NG-SONET, T-MPLS, WDM – Multi-Level - domains or network regions may
operate in different routing areas/regions, and maybe be presented in an abstracted manner across area/region boundaries
• Multi-Domain indicates that we want to allow hybrid network service instantiation across multiple domains
• And of course all this implies that this will be a Multi-Vendor environment.
• Multi-Control – mpls, gmpls, management, vendor proprietary
Dynamic Network Services
IntraDomain•Source Address•Destination Address•Bandwidth•VLAN TAG (untagged | any | tagged | tunnel)
•User Identification (certificate)•Schedule
Client A
Client B
Circuit Request
Ethernet Mapped SONET or
SONET Circuits
Dynamically Provisioned Dedicated Resource Path (“Circuit”)
Internet2 DCN Service
Internet2 IDC
•api can run on the client, or in a separate machine, or from a web browser
XMLUSER API
Actual Network Path
DRAGON Enabled Control Plane
Dynamic Network Services
InterDomain• No difference from a client (user) perspective
for InterDomain vs IntraDomain
RON Dynamic Infrastructure Ethernet VLAN
RON Dynamic Infrastructure Ethernet VLAN
Internet2 DCNEthernet Mapped SONET
1. Client Service Request2. Resource Scheduling 5. Service Instantiation (as a result of Signaling)
A. Abstracted topology exchange
AA
22
1
USER API
XML
Multi-Domain Dynamically Provisioned Circuit
DCN Control Plane
Domain Controller
Network 1
IDC
Domain Controller
Network 2
IDC
User Request/IDC Response
IDC to IDC communication
Domain Controller
Network 3
IDC
IDC to IDC communication
DCN Control Plane Software
• OSCARS (Web Service)– Started by ESnet, merged with Internet2’s
BRUW project in 2006– Web service architecture, interfaces to lower
level network specific provisioning systems– Vendor based MPLS L2VPN (Martini Draft)
• Internet2 DCS/HOPI– DRAGON (NSF funded project in development
by USC/ISI EAST and MAX)– Uses GMPLS protocols to build layer 2
circuits
I2 DCN Software Suite
• OSCARS (IDC)– Web service layer, InterDomain messaging, AAA,
Scheduling
• DRAGON (DC)– Control of domain network elements (Core
Directors and/or Ethernet Switches)– Intra and Inter Domain Path Computation– RSVP based signaling
• Version 0.3.1 of DCNSS released April, 2008– https://wiki.internet2.edu/confluence/display/DCNSS
OSCARS-DRAGON Integration
DRAGON
• Virtual Label Switched Router(VLSR)– PC based control plane software– Manages and provisions various network
equipment such as ethernet switches, SDH/SONET
– Signaling with RSVP packets
• Network Aware Resource Broker (NARB)– Stores topology in OSPF-TE database– Performs inter/intradomain path calculation– Exchanges interdomain topology
IDC - Web Service Based Definition
• Four Primary Web Services Areas: • Topology Exchange, Resource Scheduling, Signaling, User Request
Other AAA Models Possible
• Meta-Scheduler Approach• Same set of Web Services used for linear instantiation model can be
used by a high level process to build services:• Topology Exchange, Resource Scheduling, Signaling, User Request
• A key issue is that this requires a trust relationship between the “meta-scheduler” and all the domains with which it needs to talk
Domain Routing and Path
Computation Element
WS-schedule
Client Client
User Client
Domain Routing and Path
Computation Element
Domain Routing and Path
Computation Element
Domain 2 Domain 3
User ClientProvisioning and
Edge Stitching
Provisioning and Edge Stitching
Provisioning and Edge Stitching
EthernetSONET Router
Domain 1
InternalDomainDesign
WS-top
WS-sigWS-user_request
IDC
WS-schedule
WS-sig
WS-top
WS-schedule
WS-sig
WS-top
WS-user_request
IDCIDC
MetaScheduler
Topolo
gy
Signali
ng
Sched
uling
Topology
Scheduling
Signaling
InterDomain Controller (IDC) Protocol (IDCP)
• Developed via collaboration with multiple organizations– Internet2, ESnet, GEANT2, Nortel, University of Amsterdam, others
• The following organizations have implemented/deployed systems which are compatible with this IDCP
– Internet2 Dynamic Circuit Network (DCN)– ESNet Science Data Network (SDN)– GÉANT2 AutoBahn System– Nortel (via a wrapper on top of their commercial DRAC System)– Surfnet (via use of above Nortel solution)– LHCNet (use of I2 DCN Software Suite)– Nysernet (use of I2 DCN Software Suite)– University of Amsterdam (use of I2 DCN Software Suite)– DRAGON Network
• The following "higher level service applications" have adapted their existing systems to communicate via the user request side of the IDCP:
– LambdaStation (FermiLab)– TeraPaths (Brookhaven)– Phoebus
DCN – Global NetworkInteroperation via IDCP
InterDomain Controller Protocol Standardization Activities
• Standardization process and increasing community involvement continues
• Optical Grid Forum (OGF)– Network Markup Language (NML) Working Group
• Standardizing topology schemas (perfsonar and control plane)
– Network Services Interface (NIS-WG)– Grid High Performance Networking (GHPN) Research
Group– Network Measurement (NM-WG)– Network Measurement Control (NMC-WG)– Information Services (IS-WG)
• GLIF– Control Plane Subgroup working on normalizing
between various interdomain protocols (IDCP, G-Lambda GNS-WSI, Phosphorus API)
– Also other GLIF subgroups in this and related space (global id format, PerfSonar)
Internet2 DCN Working Group• DCN WG has been formed under NTAC
– Chair: Linda Winkler (Argonne National Laboratory)
• DCN WG will drive directions and set agenda in this area
• Mailing list and Wiki available– [email protected]– https://spaces.internet2.edu/display/DCN/Home
• DCN WG BOF on Monday, July 21, 12:30 PM 1:50 PM
Internet2 DCN Infrastructure
Internet2 DCN Services
1-A-5-1-11-A-6-1-1
1-A-6-1-1
DCN Services - circuits• Physical Connection:
– 1 or 10 Gigabit Ethernet– SONET (Future)
• Circuit Service:– Point to Point Ethernet (VLAN) Framed SONET Circuit– Point to Point SONET Circuit (future)– Bandwidth provisioning in 100 Mbps increments
• How do Clients Request?– Client must specify [VLAN ID | ANY ID | Untagged |
Tunnel], SRC Address, DST Address, Bandwidth– Request mechanism options are Web Service API, Web
Page, phone call, email• What is the definition of a Client?
– Anyone who connects to an ethernet or SONET port on an Ciena Core Director; could be RON, other wide area networks, domain specific applications
DCN Services - topologies
• Individual circuits are the “atomic” service provided by the DCN and control plane
• These circuits could be intra or inter domain
• It is envisioned that higher level “services” may be developed which coordinate the instantiation of multiple individual circuits to develop entire “topologies”– co-scheduling/allocation of other resources
(compute, data storage) may also be desired– Probably a task for individual
science/application domains or someone developing middleware on their behalf
Workshop Details
Internet2 CoreDynamic Circuit Network
Green PodASN4
Red PodASN1
Yellow PodASN3
Blue PodASN2
DCN Workshop Architecture
Data Plane
Control Plane
Pod Network Elements
VLSR1-PC
VLSR1-SW
ES1
Virtual Label Switching Router- “VLSR”
VLSR1
ES2
VLSR3-PC
VLSR3-SW
VLSR3
VLSR2
NARB / IDC
Inter-Domain Controller – “IDC”Network Aware Resource Broker- “NARB”
Control Plane PC (VLSR#-PC, NARB, IDC)
Data Plane Ethernet Switch (VLSR#-SW)
End System (ES#)
Basic Pod Data Plane
ES1 ES2
End System
Ethernet Switch
Data Plane via Cat5 Patch Cable
D2D4
D3D1 D5
VLSR1-SW
VLSR2-SW
VLSR3-SW
Data Plane Ethernet Switch (VLSR#-SW)
End System (ES#)
Data Plane (D#)
VLSR1-PC
Virtual Label Switching Router- “VLSR”
VLSR3-PC
NARB / IDC
Network Aware Resource Broker- “NARB”Inter-Domain Controller – “IDC”
gre2 gre4
gre3
gre6
Control Plane PC (VLSR#-PC, NARB, IDC) End System (ES#)
Basic Pod Control Plane
Pod Network Elements Control and Data Planes
VLSR1-PC
VLSR1-SW
ES1
Virtual Label Switching Router- “VLSR”
VLSR1
ES2
VLSR3-PC
VLSR3-SW
VLSR3
VLSR2
NARB / IDC
Network Aware Resource Broker- “NARB”Inter-Domain Controller – “IDC”
gre2 gre4
gre3
gre6
Control Plane PC (VLSR#-PC, NARB, IDC)
Data Plane Ethernet Switch (VLSR#-SW)
End System (ES#)
D2 D4
D3D1 D5
Pod Management Addressing
Workshop Gateway Router
Management VLAN 192.168.<asn>.n/16
“Red” pod: ASN=1“Blue” pod: ASN=2“Yellow” pod: ASN=3“Green” pod: ASN=4 192.168.1.1
eth0 .2
.3
eth0 .4
.5
.6
.8
.10
.9 eth0
.7
VLSR1
ES1 ES2
VLSR3
NARB / IDC
VLSR2
eth1 eth1
eth0 - Management Plane Interface and Control Channel (PCs)
eth1 - Data Plane Interfaces (PCs)
Rack Layout
VLSR1-PCVLSR1-SW
GW2SW2
NARB
VLSR2-PCVLSR2-SW
VLSR3-SWVLSR3-PC
ES1ES2
VLSR1-PCVLSR1-SW
NARB
VLSR2-PCVLSR2-SW
VLSR3-SWVLSR3-PC
ES1ES2
.
VLSR1-SWVLSR1-PC
GW1SW1
NARB
VLSR2-SWVLSR2-PC
VLSR3-PCVLSR3-SW
ES1ES2
VLSR1-SWVLSR1-PC
NARB
VLSR2-SWVLSR2-PC
VLSR3-PCVLSR3-SW
ES1ES2
123456789012345678901234567890
123456789012345678901234567890
Rack 1 Rack 2
Workshop Pods
Red Pod
Green Pod
Yellow Pod
Blue Pod
Exercise #1 Intra-Domain Detail(Answer Sheet)
Workshop Gateway Router
Management VLAN 192.168.<asn>.n/16
“Red” pod: ASN=1“Blue” pod: ASN=2“Yellow” pod: ASN=3“Green” pod: ASN=4
Management VLAN 192.168.<asn>.n/16
GRE<x> = 10.<asn>.<x>.n / 30 GRE7= 10.1.7.0 / 30
192.168.1.1
eth0 .2
.3
eth0 .4
.5
.6
.8
.10
.9 eth0
.7GRE3
GRE2
GRE4
GRE6
TEaddr = 11.<asn>.<x>.n / 30
D2
D4
D3
D1D5
VLSR1-PC
VLSR1-SW
ES1 ES2
VLSR3-PC
VLSR3-SW
NARB / IDC
VLSR2-PC
VLSR2-SW
3 5
4
1
10.a.2.1
10.a.3.1
10.a.3.2
10.a.2.2
13
4
10.a.6.1
10.a.6.2
10.a.4.1
10.a.4.2
13
4
5
eth1 eth1
11.a.4.1
11.a.4.211.a.2.1
11.a.2.2
11.a.3.211.a.3.1
Dynamic Data plane port group = g3-g24Dynamic VLAN range = 100…200
Exercise #1 Data and Control links
“Red” pod: N=1“Blue” pod: ASN=2“Yellow” pod: ASN=3“Green” pod: ASN=4
GRE3
GRE2GRE4
GRE6
D2D4
D3D1
VLSR1
ES1 ES2
VLSR3
NARB / IDC
VLSR2
3 5
4
4
3
4
5
eth1 eth1
D5
3
Login information• Wireless Network:
– SSID: DCNworkshop– WPA Personal Key: Workshop!
• Login to all VLSR, ES and NARB– ssh port 22– username: user[1-16]; password: Workshop!– username: root; password: rootme
• Login to all switches– telnet port 23– username: admin; password: admin
• OSCARS configuration; login to the NARB/IDC machine– ssh port 22– username: tomcat55; password: dragon
• OSCARS axis2 login– https://idc.<color>.pod.lan:8443/axis2/axis2-admin/– username: admin; password: axis2
• OSCARS web user interface;– https://idc.<color>.pod.lan:8443/OSCARS/– username: oscars-admin; password: oscars
• Command Line Interface ports–dragond 2611–ospfd 2604 (intra-domain)–narb 2626–rce 2688
> telnet localhost 2611> password: dragon
Login information
Workshop Laboratory
• Four “Pods”: Red, Blue, Yellow, Green• Each Pod represents an independent network
domain• Each Pod has two End Systems: ES1 and ES2• Each Pod has three Virtual LSRs (VLSRs)
– Each VLSR has a PC (for ctrl plane) and a Ethernet switch (for data plane)
• Each Pod has one PC for interdomain routing support of the NARB and OSCARS
• The PCs are running Debian Linux– We have installed it and all the software required to
download, build, and run the control plane software, and to perform the workshop labs
• We installed the DRAGON software and OSCARS software– /usr/local/dragon/{bin,etc}– /usr/local/tomcat, /home/tomcat55
Workshop Exercises
• Exercise 1: Designing a GMPLS Control Plane for Ethernet Data Planes
• Exercise 2: Intra-Domain Provisioning with OSCARS
• Exercise 3: Inter-Domain Provisioning with OSCARS
Exercise #1 Designing a GMPLS Control Plane For Ethernet Data Planes• Diagram a control plane for each pod• Construct an addressing scheme for
the control plane• Configure the network elements’
data plane• Configure the control plane software• Set up an LSP
• …and if that fails…read the instructions.
GMPLS Snapshot• Generalized Multi-Protocol Label Switching – GMPLS
– Evolved from MPLS concepts, and experiences gained from deployments within the IP packet world
• GMPLS extends Traffic Engineering (TE) concepts to the multiple layers:– Packet Switching Capable (PSC) – standard MPLS LSPs– Layer2 switch capable (L2SC) – Ethernet and VLANs– TDM switch capable (TDM) – SONET/SDH – Lambda switching (LSC) – Wavelength – Fiber Switch capable (FSC) - Automated Patch Panel
• In the GMPLS, any network element that supports one of the above switching capabilities and participates in the GMPLS control plane protocols is referred to as a “Label Switching Router” or LSR.
• GMPLS Protocols: – Routing: GMPLS-OSPF-TE – Signaling: GMPLS-RSVP-TE – Link layer: LMP (not widely implemented)– ISIS and CR/LDP are also considered part of the GMPLS protocols– In this workshop we will focus only on OSPF and RSVP
What is the Control Plane?• The Control Plane is the network facilities and
associated protocols that select, allocate/deallocate, and provision network resources to fulfill a user service request.– Typically this includes routing protocols that distribute
topology and reachability information among interconnected networks and network elements
– It also includes other functions that allocate appropriate resources and put those resources into service (Path computing and signaling)
• With GMPLS, routing and signaling messages between LSRs do not travel along the same [physical] path as the circuit being established.– The set of facilities between LSRs that carry the data
circuits themselves is called the “Data Plane”– The set of facilities between LSRs that carry the routing
and signaling protocols is called the “Control Plane”
• It is good practice to design the control plane so as to be highly robust and impervious to effects of other network traffic or malicious activity
• In this workshop, our control plane and data plane will be separate as is typically the case for GMPLS networks.
Control Plane and Data Plane
CP CP CP
LabelSwitched
Paths
GMPLSProtocols
GMPLSProtocols
LabelSwitched
Paths
ControlPlane
Data Plane
A [Typical] Label Switching Router – “LSR”
• What is an “LSR”– In the MPLS world, it is any router capable of recognizing
and processing the MPLS shim header in the IP packet• In the GMPLS world, an LSR is any network element that
is able to establish “label switched paths” (LSPs) under control of the GMPLS protocol suite:– This now includes fiber switches, wave division
multiplexors, sonet (tdm) switches, ethernet switches, and traditional packet switches (MPLS routers)
Label Switching
Fabric
Data Interfaces
Switching Fabric Interface Link
Control Processor
Management Interface
Key Control Plane Features
• Routing– distribution of "data" between networks. The data that
needs to be distributed includes reachability information, resource usages, etc
• Path computation– the processing of information received via routing data
to determining how to provision an end-to-end path. This is typically a Constrained Shortest Path First (CSPF) type algorithm for the GMPLS control planes. Web services based exchanges might employ a modified version of this technique or something entirely different.
• Signaling– the exchange of messages to instantiate specific
provisioning requests based upon the above routing and path computation functions. This is typically a RVSP-TE exchange for the GMPLS control planes. Web services based exchanges might employ a modified version of this technique or something entirely different.
OSPF – “Open Shortest Path First”
• OSPF is a “Link State” Routing Protocol– OSPF routers discover each other thru a HELLO protocol
exchanged over OSPF interfaces – Routers identify themselves with a “router id” (typically the
loopback IP address or another unique IP address is used)– OSPF routers flood Link State Announcements (LSAs) to each
other that describe their connections to each other and that specify the current link state of these connections
• In the GMPLS and TE extensions to OSPF, the LSA contains information about the available bandwidth, routing metrics, switching capabilities, encoding types, etc.
• LSAs are not flooded in the direction from which they are heard
– Link State flooding does not scale well • OSPF routing is often divided into “areas” to reduce or
limit LSA flooding in large networks• Other routing protocols are used between routing
“domains” that distribute reachability information but not link state info
– Each OSPF router in an area has a full topological view of its area
– SPF identifies the next-hop for each known destination prefix
CSPF• Constrained Shortest Path First
– In OSPF TE, reachability is no longer the only criteria for deciding next-hop
• E.g. Bandwidth available on each intemediate link could be a constraint used to identify or select a path
• In GMPLS, with multiple switching capabilities, there are many constraints to be considered
– Path Computation is used differently for selecting circuit layout than for selecting the next-hop for shortest path packet forwarding
• Two identical path requests may generate two completely separate paths (unlike traditional routed IP which would select only the single “best” path for forwarding packets)
• Paths are not computed until or unless a path is needed.– Some GMPLS service models do propose precomputing paths
(or at least next-hops) based on certain apriori assumptions about the LSP – the tradeoff is generally one of scheduled “book ahead” reservations vs fast “on-demand” provisioning.
RSVP – ReSerVation Protocol• GMPLS-RSVP-TE is the signaling (provisioning)
protocol used to instantiate a Label Switched Path (LSP) thru the network
• Five basic RSVP messages we will reference:– PATH = First message issued by the source towards
the destination requesting a connection be established– RESV = Response from the destination towards the
source accepting the connection– PATH_TEAR = Message sent to tear down an LSP– PATH_ERR = Error message sent when a PATH request
is denied or encounters a problem – REFRESH = Message sent between LSRs indicating a
connection is still active (prevent timeout and deletion)
Path Computation Element• In GMPLS, the Path Computation Element (PCE) is
separated from the routing protocol. – The routing protocol distributes topology information
and builds the topology database that contains all the [visible] resources and their state – the Traffic Engineering Data Base (TEDB)
– PCE is responsible for processing the TEDB to select a path through the network that meets the constraints specified in the service request (e.g. BW, encoding, Src/Dst, Policy, etc.)
• In GMPLS, the path computed is expressed as an “Explicit Route Object” (ERO). – An ERO is simply a data structure that contains a
sequentially ordered list of routers (LSRs) that the path will travels from Source to Destination
– A “Loose Hop” ERO specifies a partial set of transit nodes – the path may contain other nodes as long as it passes through the specified nodes in the order specified.
– A “Strict Hop” ERO specifies a complete list of transit nodes – no other intervening nodes are allowed.
– RSVP includes the ERO in the PATH message to pin the path through specific nodes
DRAGON Control Plane - Key Elements
• Virtual Label Switching Router – VLSR– Open source protocols running on PC act as
GMPLS network element (OSPF-TE, RSVP-TE)– Control PCs participate in protocol exchanges and
provisions covered switch according to protocol events (PATH setup, PATH tear down, state query, etc)
• Network Aware Resource Broker – NARB– Intradomain listener, Path Computation,
Interdomain Routing and Path Computation
• More information: – dragon.east.isi.edu– dragon.maxgigapop.net
The Virtual Label Switching Router “VLSR”• The DRAGON Project developed a control
plane "proxy" element to cover non-GMPLS capable devices like standard ethernet switches.
EthernetSwitch
SNMP
Linux Control PC
Mgmt Interface
GMPLS Control Plane
Data Plane
VLSR - conceptual
CoreEthernetSwitch
OperationsAccessSwitch
Linux Control PC
Control LinksVia
GRE tunnels
VLSR – physical
Data Plane
VLSR(Virtual Label Switching Router)
• RSVP Signaling module– Originated from Martin Karsten’s C++ KOM-RSVP– Extended to support RSVP-TE (RFC 3209)– Extended to support GMPLS (RFC 3473)– Extended to support Q-Bridge MIB (RFC 2674)– For manipulation of VLANs via SNMP (cross-connect)– Extended to support VLAN control through CLI
• OSPF Routing module– Originated from GNU Zebra– Extended to support OSPF-TE (RFC 3630)– Extended to support GMPLS (RFC 4203)
• Ethernet switches tested to date– Dell PowerConnect, Extreme, Intel, Raptor, Force10
NARB(Network Aware Resource Broker)
• NARB is an agent that represents a domain• Intra-domain Listener
– Listens to OSPF-TE to acquire intra-domain topology– Builds an abstracted view of internal domain topology
• Inter-domain routing– Peers with NARBs in adjacent domains– Exchanges (abstracted) topology information– Maintains an inter-domain link state database
• Path Computation– Performs intra-domain (strict hop) TE path computation – Performs inter-domain (loose hop) TE path computation– Expands loose hop specified paths as requested by domain
boundary (V)LSRs.• Hooks for incorporation of AAA and scheduling into path
computation via a “3 Dimensional Resource Computation Engine (3D RCE)”– The Traffic Engineering DataBase (TEDB) and Constrained
Shortest Path Computation (CSPF) are extended to include dimensions of GMPLS TE parameters, AAA constraints, and Scheduling constraints.
– 3D RCE is the combination of 3D TEDB and 3D CSPF
Heterogeneous Network Environmentmulti-technology, multi-level, multi-domain,
multi-vendor, multi-provision system network environments
GMPLS MPLSManagement Plane
IDCDC
IDCDC
IDCDC
DRAGON
DRAGON GMPLS Control Plane
CoreDirector
Ciena Region
uni, tl1
CD_a CD_z
uni, tl1
CoreDirector
subnet signaling flow
IDC
• DRAGON is used as the DOMAIN Controller for I2 DCN Ciena Core Directors
GMPLS to other domains
GMPLS to other domains
to other domain IDCs
to other domain IDCs
• DRAGON allows for incorporation of non-GMPLS equipment and vendor proprietary provisioning methods into the overall GMPLS environment
Exercise #2: Intra-domain Provisioning with OSCARS• In this exercise we will bring up the OSCARS
software, configure the network topology and candidate paths, and provision LSPs across a single administrative network domain
• OSCARS:– “On-demand Secure Circuits and Advanced Reservation
System”– Provides Authentication and Authorization for LSP
requests– Provides book-ahead scheduling for network path
resources– Interim: implements the static topology distribution
function and provides precomputed static EROs for provisioning
• OSCARS is a Java based application. OSCARS runs on top of Tomcat, uses MySQL and AXIS2.
Exercise #3: Inter-domain Provisioning with OSCARS• In this exercise we will configure and use
OSCARS to accomplish InterDomain provisioning. – Design (and implement) the inter-domain Data
plane– Layout the inter-domain control plane– Configure OSCARS for inter-domain– Test
IDC - Web Service Based Definition
• Four Primary Web Services Areas: • Topology Exchange, Resource Scheduling, Signaling, User Request
DCN Web Services• Web Service Definitions• wsdl - web service definition of message
types and formats• xsd – definition of schemas used for
network topology descriptions and path definitions
• Ongoing work with OGF Working Group(s), PerfSonar, and GLIF with the goal to achieve interoperability amongst all groups.
InterDomain SpecificationWeb Services
• https://wiki.internet2.edu/confluence/display/CPD/OSCARS+Web+Service+Definition
• Specification is defined by a Web Service Desciption Language (WSDL) document and XML Schema files containing associated data types.
• OSCARS.wsdl - web service definition of OSCARS messages
• OSCARS.xsd - data types used by OSCARS.wsdl
• nmtopo-ctrlp.xsd - NMWG control plane topology schema used by OSCARS.xsd for topology-related data types
AAA and Security
• OSCARS AAA• SSL Encryption• Authentication
– X.509 Certificates• User to Domain• Domain to Domain
– Web Service Security by OASIS– SAML assertions about end-user
(future)
• Authorization– OSCARS attribute based system
DCN Control Plane uses OGF Topology Schema
Information Services Topology Service and LookUp Service
• Control Plane uses Information Services Topology Service and LookUp Service
• LookUp Service– Provides a mapping from circuit end points to user
friendly names
• Topology Service– Provides an infrastructure from which to retrieve
topologies from other domains– Will be utilized for global path computation
Information Services Topology Service and LookUp Service
DCN Information Service - Lookup Service
DCN ProvisioningWeb Page or API
Web Page Based Provisioning
Internet2 IDC
USER APIjava createReservation https://dcn.internet2.edu:axis2/services/dcn
reservation.properties
Web Service
DCN – Circuit Status Description
DCN – Circuit Status Description
Requesting a circuit - Interfaces
• Web User Interface (WBUI)– Java servlet interface used by OSCARS web
page– Not intended for use by other applications
• Web Service API– XML-based API intended for use by
applications• e.g. Phoebus, LambdaStation, TeraPaths
Requesting a circuit – WS API
• Used by applications to contact IDC• Authenticate using an X.509
certificate– Generate with command-line tools– Have CA sign (Internet2 has test CA)
• Message format defined in DICE Control Plane group
• Custom applications should use this interface
Additional Information
• DCN Software Suite– https://wiki.internet2.edu/
confluence/display/DCNSS/Home
• Java Client API– https://wiki.internet2.edu/
confluence/display/CPD/OSCARS+Client+Java+API
Workshop Details - end
DCN Control Plane Possible Future Features and Work Areas
• Improved user documentation and software installation procedures
• Improved reliability and redundancy of dynamic provisioning operations. (better automated logging and failure reporting, redundant control plane elements, automated interaction between control plane and monitoring systems and NOC operations)
• Support for VLAN Translation across a multi-domain circuits• Support for SONET Client Access ports and Interdomain Links• Design for automated multi-domain topology exchange• Enhanced user request options (additional parameters and
ability to ask questions without actually making a reservation)
• Enabling other signaling methods, e.g. RSVP (as opposed to only Web Service method)
• Continue work with international groups, standards bodies to formalize the IDC InterDomain Protocol to further increase interconnected global community for these services
Use of Internet2 DCN and peering dynamic networks
1. Physical connection2. Access to control plane software
How do I connect? – Physical Connection• Internet2 Connectors
– Connect to Internet2 DCN
• Universities and campuses– Contact Internet2 Connector
How do I connect? – Software Configuration
• Option 1: No local IDC• Option 2: Install local IDC
How do I connect? – Software Configuration
• Option 1: No local IDC– Statically configure your local network – Applications/Users can dynamically
request circuits from the nearest IDC
How do I connect? – Software Configuration
Network 1
Domain Controller
Network 2
IDCUser Request/IDC Response
Network 3
Statically Configured Dynamically Configured Statically Configured
• Option 1: No local IDC
Domain Controller
Network 1
IDC
Domain Controller
Network 2
IDC
User Request/IDC Response
IDC to IDC communication
Domain Controller
Network 3
IDC
IDC to IDC communication
How do I connect? – Software Configuration• Option 2: Install local IDC
How do I request a circuit? - Clients
• User-initiated– OSCARS Web Page– Simple command-line tools
• Program-initiated– Phoebus
• Transparently request circuit upon data transfer initiation
– Custom applications you build!
How do I request a circuit? - Interfaces• Web User Interface (WBUI)
– Java servlet interface used by OSCARS web page
– Not intended for use by other applications
• Web Service API– XML-based API intended for use by
applications• E.g. Phoebus, LambdaStation, TeraPaths
How do I write my own DCN application?• Java library for making DCN calls• Can call simple command-line client
directly from application• Google Summer of Code students
will be developing PERL, C, and Python libraries
backup
VLSR(Virtual Label Switching Router)
• GMPLS Proxy– (OSPF-TE, RSVP-TE)
• Local control channel– CLI,TL1, SNMP, others
• Used primarily for ethernet switches
Web page
XML Interface User API
CLI Interface One NARB per Domain
• Provisioning requests via CLI, XML, or ASTB
DRAGON Virtual Label Switching Router (VLSR)
– Control channels could also be provisioned out-of-band via GRE tunnels over an IP network
IPsec is one of several
mechanisms recommended for securing out-of-band
control channels provisioned
over IP networks(RFC3945)
DCN – Circuit Status Description
Laying Out the Control Plane
S
DR1
R2
R3
R4
R5
R6
D1
D2
D3
D4
D5
D6
D7
• Lay out the data plane between NEs first. – For now, we are going to ignore intervening static NEs.– Make sure all Nes and links are uniquely labeled
• Then, control links connect the dynamic network elements• If you are including end systems in the dynamic network,
you should add them where appropriate
C1
C2C3
C4
C5
C6
C7
D8
D9
C9
C8
Control Plane
R1 R2 R3
Data plane
• Often, the dynamic network elements are not directly adjacent to one another – but the control structure expects them to be (at least logically adjacent)
• We employ Generic Routing Encapsulation (GRE) tunnels for the control links in order to create logical adjacencies
– GRE Tunnels are set up between two IP hosts over the conventional internet interface. (these are the “tunnel endpoints”)
– They present a pseudo interface to the end host that appears to be directly linked to the remote endpoint, thus allowing a single common IP subnet to be allocated on this GRE (pseudo) interface.
GRE TunnelEndpoints
Control LinkEndpoints
D2
C2
D3
C1
D4
C3
D1
C4
Generic Network ElementConsider all of the components in a network element:
Case Study: Control ChannelsDRAGON Virtual Label Switching Router (VLSR)
– Linux PC implements GMPLS control plane protocols– Control channels may be provisioned in-band or
out-of-band
One goal of
DRAGON’s VLSR software is to
provide GMPLS protocol support for devices which
do not support GMPLS
Case Study: Control ChannelsDRAGON Virtual Label Switching Router (VLSR)
– Assuming underlying network uses Ethernet VLANs, control channels may be provisioned in-band with static control VLANs
In-band control channels are
considered somewhat less
vulnerable than out-of-band
(RFC3945)
Case Study: Control ChannelsDRAGON Virtual Label Switching Router (VLSR)
– Control channels could also be provisioned out-of-band via GRE tunnels over an IP network
IPsec is one of several
mechanisms recommended for securing out-of-band
control channels provisioned
over IP networks(RFC3945)
Case Study: Control Channels
Data plane
GRE TunnelEndpoints
Control LinkEndpoints
Hybrid NetworksWeb Service Control Plane Interfaces
• Web Services provides a mechanism to deal with heterogeneous control planes• inspired by the standards bodies work on control plane protocols,
but not just recreating that work at the web service level • Better described as using control plane techniques to develop a
“service plane”
Ethernet/L2SC(Dataplane)
SONET/TDM(Dataplane)
Router(MPLS)/PSC(Dataplane)
GMPLS(I-NNI)
MPLS(I-NNI)
Management System(I-NNI)
Inter-Domain Controller (IDC)
WS E-NNIWS E-NNI
WS UNI WS UNI
IDC
IDCWS I-NNI IF
WS I-NNI IF WS I-NNI IF
Hybrid NetworksControl Plane Architecture
• The benefits offered by Web Services include• standardized mechanisms for user authentication and policy
management• flexible features for interfacing with a diverse set of I-NNI
mechanisms• Allows focus on several issues that current control plane work
has not addressed in a robust manner:• scalability, stability, security, flexible application of policy, AAA,
scheduling
• Will still allow for peering domains with compatible non web service E-NNI (i.e. GMPLS based) to utilize that as desired• a domain might peer with one domain at GMPLS level, and
another at the Web Service level
Web Service based E-NNIThree Main Components
• Routing– Topology Exchange– Domain Abstraction– Varying levels of dynamic information
• Resource Scheduling– Multi-Domain path computation techniques– Resource identification, reservation, confirmation
• Signaling– path setup, service instantiation
Key Control Plane Key Capabilities
• Domain Summarization– Ability to generate abstract representations of your domain for
making available to others– The type and amount of information (constraints) needed to be
included in this abstraction requires discussion. – Ability to quickly update this representation based on
provisioning actions and other changes • Multi-layer “Techniques”
– Stitching: some network elements will need to map one layer into others, i.e., multi-layer adaptation
– In this context the layers are: PSC, L2SC, TDM, LSC, FSC– Hierarchical techniques. Provision a circuit at one layer, then
treat it as a resource at another layer. (i.e., Forward Adjacency concept)
• Multi-Layer, Multi-Domain Path Computation Algorithms– Algorithms which allow processing on network graphs with
multiple constraints– Coordination between per domain Path Computation Elements
OSCARS Architecture
End-HostApplication
User
Topology
LinkReservations
Policy
Web-UserInterface
Path Setup(MPLS)
BandwidthScheduler
AuthenticationAuthorization
Path Setup(GMPLS)
OSCARSResourceManager
ResourceManager
Customer Site External Peer
Web-Services Interface(Signed SOAP Messages)
I-NNI
Integration Core Director Domain into the End-to-End Signaling
VLSR uni-subnet
• Signaling is performed in contiguous mode.• Single RSVP signaling session (main session) for end-to-end circuit.• Subnet path is created via a separate RSVP-UNI session (subnet session),
similar to using SNMP/CLI to create VLAN on an Ethernet switch.
• The simplest case: one VLSR covers the whole UNI subnet.• VLSR is both the source and destination UNI clients.• This VLSR is control-plane ‘home VLSR’ for both CD_a and CD_z.• UNI client is implemented as embedded module using KOM-RSVP API.
CoreDirector
Ciena Region
LSRdownstream
LSRupstream
data flow signaling flow
subnet signaling flow
uni, tl1
CD_a CD_z
uni, tl1CoreDirector
DRAGON enables integration of the Core Director Domain into Multi-Domain, Multi-Layer, Multi-Service, Multi-Vendor Provisioning Environment
• Goal is to utilize Ciena Domain control plane and advanced features to maximum extent possible
• advanced provisioning, management, monitoring, restoration and protection features
• applicable to single domain, single vendor
• Integrate these capabilities into the Multi-X environment
VLSR uni-subnet1
CoreDirector
Ciena Region
LSRdownstream
LSRupstream
data flow signaling flow
subnet signaling flow
VLSR uni-subnet2
VLSR
CD_a CD_z
uni style control
CoreDirector
uni style control
Domain Boundary
Domain Boundary
Top Related