ADVANCED NETWORKING ON INTERNET2

Eric BoydSenior Director, Strategic Projects, Internet2

SDN Overlay & MPLS Support

© 2016 Internet2

Ed BalasManager, Network Analysis and Control, Indiana University

[ 2 ]

2016 Network Transformation

What does Feature Parity mean?=> Implement SDN Features

in an Overlay Network=> Preserve OESS UI Experience=> Enable MPLS Features

[ 3 ]© 2016 Internet2

Timeline: Past, Present, Future

Q4 2015 Q1 2016 Q2 2016 Q3 2016

Initiating / Planning / Logistics

Q4 2016 Q1 2017

Core Hardware Transition

MPLS + VRF Engineering

Project Management /Budgeting / Purchasing

Network Overlay 1.0Implementation

RFP

MPLS ImplementationLayer2 Services

Layer 3 Services

Management Overlay

Network Overlay 2.0Implementation

Q2 2017

[ 4 ]© 2016 Internet2

SDN Overlay

[ 5 ]

I2 SDNDevice

I2 SDNDevice

I2 SDNDevice

I2 SDNDevice

I2 SDNDevice

I2 SDNDevice

GENI Rack

GENI Rack

Internet2 Core Network

100G

100G

100G

100G

100G

100G

100G

100G

10G

10G

10G

10G

10G

10G

10G

10G

10G

10G

10G

10G

Server

Server

ServerServer

Server

ServerSDN Overlayusing

the I2 CoreNetwork

Generalized Topology

Design Ideas1) Looks like customer is

connecteddirectly to the SDN (overlay) network

2) Crashing the SDN overlay network doesnot impact underlying network

3) Only one network connectionrequired

[ 6 ]© 2016 Internet2

S

S

S

S

S

S

S

S

S Brocade SDN Overlay Node

J

JJ

J

J J

J

J

J

J

JJ

J

J

JJ

JJ

J

J

JJ

J

J

J

J

J

J

J

JJ

J

J

JJ

J

J

J

J

J

J

J

Optical Add/Drop, no L2/L3

L2 or L3 BB via 3rd Party ServiceL2 or L3 BB via Internet2 Opt

J Juniper R&E Router (w/ TR-CPS vrf)

J

NEWY1118TH

NEWY32AOA

EQNY

HART2

BOSTALBA

CLEV

PITT

ASHB

RALE

WASH

CHAR

PHILINDI

LOUI

CINC

STAR

EQCH

CHIC

KANS

SAIN

TULS

NASH

ATLA

JACK

BATO

JCSNDALLDALL3

HOUS

HOUH

MINN

MISS2

SEAT

LACY

PORT

EUGE

BOIS

SACR

SUNN

PAIX

LOSA

WILCSAND

SANL

PHOE TUCS

SALT

DENV

PUEB

ALBU

ELPA SANA

RENO

LASV

J

J

Potential 100GE L2/3 BB AugmentPlanned 100GE BB Augment

SDN Overlay

[ 7 ]

SDN Overlay Node• Brocade MLX Switch

– (N+2) x 10G– N connections to other servers (~3)– 1 connection to server– 1 customer backhaul (not shown)

• Virtualization Server– Multiple VMs– NFV + OF Controller– Use depends on experiment characteristics

• FSFW– Slicing– Removes direct management

access for external controller VMs

© 2016 Internet2

Overlay OF SwitchBrocade MLXE-4

OverlayController / NFV

Dell R630

AL2SJuniper MX960

Racklan HP 5400

MPLS Cloud

POP A POP B POP C

10gbpsData Plane

Overlay LSP

Overlay LSP

Overlay LSP

N x 10gbps Overlay to AL2S Interconnects

Researcher 1Controller

VM

ManagementVLAN

Control PlaneVLAN

Globally Routed Ext.

Access VLAN

1gbps

1gbps

1gbps

Researcher 2Controller

VM

FSFW VM

OverlayData Plane

Access

1gbps

Control Plane VLAN

[ 8 ]© 2016 Internet2

POP ZPOP A

Core Core

Interfaces dedicated to the Virt Sw A to Virt Sw Z Path

Virtualization Switch A

Virtualization Switch ZBB LSP

Virt Sw A can use all tags on its Interface connecting to the core, these tags are preserved by the core as the frames are passed to Virt Sw Z

Burn one physical 10g per degree

POP ZPOP A

Core Core

One pair of Interfaces per remote Adjacency

Virtualization Switch A

Virtualization Switch ZBB LSP A-Z

POP Y

Core Virtualization

Switch Y

BB LSP A-Y

[ 10 ]© 2016 Internet2

POP ZPOP A

Core Core

One pair of Interfaces per remote Adjacency and a shared interface pair for all customer backhaul

Virtualization Switch A

Virtualization Switch Z

Bac

khau

l

Customer

R&

E VL

AN

Burn one interface for customer backhaul

[ 11 ]

SDN Overlay Implementation• Current Status

– Brocade MLXe4 deployments (complete except for Houston)– Software deployment (complete except for Houston)– Connections between nodes (in engineering design)

• Next Steps– Complete Houston deployment– Complete interconnections– Demonstrate OESS / FSFW (“old environment”) in action (December, 2016)– Demonstrate ONOS in a slice on SDN overlay– Demonstrate GENI projects on SDN overlay

© 2016 Internet2

[ 12 ]© 2016 Internet2

OESS Support for MPLS

[ 13 ]© 2016 Internet2

OF

Switc

h

Open Exchange Software Suite (OESS)

Inter-Domain Northbound

IDCP NSIOESS UI

Intra and Inter-DomainProvisioning -O

F

Topo-OF

Meas-O

FNative Northbound Web Services

Provisioning Measurement Administration

OpenFlow NetConf SNMP

OF

Switc

h

MPL

S

MPL

S

Syslog

Topo-MPLS

Meas-M

PLS

Provisioning -MPLS

PathEvents MPLS

OESS Changes

1) Replaced DBUS with RabbitMQ (Complete)2) Added MPLS subcomponents

1) New SNMP based stats collector (Complete)

2) New subsystem for configuring MPLS using NetConf (Complete)

3) New “diffing” subsystem (Complete)4) Major rev, major time in test lab

(In progress)3) Integrate support for specific MPLS

subcomponents(Pending template completion)

4) UI adjustments based on service offering(In progress)

ADVANCED NETWORKING ON INTERNET2

Eric BoydSenior Director, Strategic Projects, Internet2

SDN Overlay & MPLS Support

© 2016 Internet2

Ed BalasManager, Network Analysis and Control, Indiana University