Architecture Optimization in IP over WDM Core NetworksPractical Considerations & Operational...

Architecture Optimization in IP over WDM Core Networks

Serge Melle, Drew Perkins, Curtis Villamizar

Infinera Confidential and Proprietary | 2

Optical Bandwidth Drivers

Access/Aggregation

IP/MPLS

Transport: Long-Haul, Metro

Cisco white paper: “The Exabyte Era”


The Result – Increasing WDM Bandwidth Needs Typical US Long-Distance Core Backbone

1

10

100

1000

10000

100000

Aver

age

WD

M L

ink

Cap

acity

(Gb/

s)

50% growth 75% growth 100% growth

IP Network Growth Expected to Scale WDM Networks >10Tb/s by 2020


Defining The Problems

Need to drive out CapEx Simplify layers? Remove equipment?

Need to drive out OpEx IP and optical have different skill sets Must reduce the operational burden as new service

demands escalate?

Improve service delivery time Why do I have to wait between 6 and 20 weeks just to

get a transponder card? Too many different transponder types!!!


IP – Optical Network Evolution Objectives

Enable IP network capacity scaling

Minimize network costs

Maximize network resiliency & reliability

Minimize latency

Unconstrained network reconfigurability

IP Network

Transport Network


Cost Issues in IPoWDM Networks

1. High cost of IP router ports


Industry Data Points

Significant difference between IP and transport equipment costs

Up to 5x differential between router & transport DWDM optics

Opportunity for greatest CapEx savings: Cap or reduce router scaling Cap or reduce number of router

ports required Maintain DWDM functionality in

transport layer Focus should be on minimizing

total network cost… …optimizing between IP and

transport layers 0%

20% 40% 60% 80%

100% 120%

Rel

ativ

e co

st p

er G

b/s

40G SR POS (router) 40G POS

DWDM (router)

10G SR POS (router) 10G DWDM

Transponder (<500km)

10G DWDM Transponder

(1500km)



1. High cost of IP router ports 2. High cost of IP router ports


The image cannot be dis

The image ca

The image can

T

The

Th

0% 2% 4% 6% 8%

10% 12% 14% 16% 18% 20%

The image cannot

The image cann

The image ca

The im

T

Th

T

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

Network Cost vs. Interface Type

High cost of Layer 3 POS router ports Historically lower cost points for Ethernet interfaces & switches Cost savings from use of converged Layer 2/3 switches (“Ethernet

router”)

IP Traffic Volume (Tb/s)

IP N

etw

ork

Cos

t/Tra

ffic

Volu

me

10GbE OC-768 POS OC-192 POS Mixed OC-192/768 POS



1. High cost of IP router ports 2. High cost of IP router ports 3. High cost of IP router ports


IP Network Architecture Present Mode of Operation

IP core links (router trunks) primarily connected between adjacent router-adjacent router

IP core links carried over dedicated wavelengths on WDM network

Little/no router bypass (and thus little pass-through traffic in WDM layer)

End-end IP demands pass through multiple router hops Consume routing capacity

for pass-through demands Increasing inefficiency &

cost as traffic volume scales


IP Network Architecture Future Mode of Operation

As IP traffic volume grows…. Minimize end-end IP demands

transiting through intermediate router hops

Increase number of direct router-to-router core links (router trunks) Router “bypass” for end-end

demands between nodes

Service demand

Traffic offloaded from three pairs of router ports


IP Network Architecture Future Mode of Operation

As IP traffic volume grows…. Minimize or reduce amount of

end-end IP demands transiting through multiple router hops

Increase number of direct router-to-router core links (router trunks) Router “bypass” for end-end

demands between nodes Optical bypass in WDM layer

Wavelength “pass-through” at intermediate WDM nodes

Leverage optical express and ROADM technologies

Minimize use of high-cost router ports and capacity Optical layer bypass for lower

total network cost Cost-efficient scaling of IP and

WDM layers


Fiber Network - Typical N. American Backbone

18,000km coast-coast network

96 cities & nodes 18 “Tier 1” cities or

junctions 13 “Tier 2” major cities 24 “Tier 3” regional cities 41 “Tier 4” regional cities

“Next-Gen” network: G.655 fiber


IP Network A-Z Demands

Develop complete city-city (A-Z) IP demand set (ie: gravity model)

Adjust model for real-world characteristics: Unequal source/sink

tendencies statistical multiplexing time-of-day demand

effects Randomization atypical cities (ie: SF

Bay area) etc.

Total traffic volume representative of real-world IP network

Sensitivity analysis: scale IP network capacity from 1x to 256x


IP Core and Regional Links

Map A-Z IP demands onto fiber network and considering IP multiplexing gain Ensure equipment and path redundancy

Select core nodes Core nodes will generally be at the largest cities which are

often major junctions. Initial design assumes common IP provider practice to

deploy core links between physically adjacent core nodes.

Map IP core demands into router-to-router core links (router trunks)

Associate regional nodes to core nodes based on proximity. Each non-core node must be associated with a region

containing one or more core node. Non-core nodes are selected based on lower demands and

proximity to core nodes. Defines total IP demands for core and regional traffic Vary degree of IP router bypass & impact on IP core link

size, router size, etc.

IP Core Links IP Regional Links


Network Cost vs. IP Bypass (10GbE)

Relative IP Network Costs (10GbE Ports)

Number of IP Bypass Links

Relative WDM Network Costs (10GbE Ports)


0.298Tb/s

0.849Tb/s

2.384Tb/s

6.75Tb/s

19.072Tb/s

38.145Tb/s

0.298Tb/s

0.849Tb/s

2.384Tb/s

6.75Tb/s

19.072Tb/s

38.145Tb/s

85%

90%

95%

100%

0 10 20 30 40 50 60 70 80 70%

75%

80%

85%

90%

95%

100%

0 10 20 30 40 50 60 70 80


Network Cost vs. IP Bypass (40G POS)

Relative IP Network Costs (40G POS Ports)


Relative WDM Network Costs (40G POS Ports)


1.192Tb/s

2.384Tb/s

4.768Tb/s 9.536Tb/s

19.072Tb/s

38.145Tb/s

1.192Tb/s

2.384Tb/s

4.768Tb/s

9.536Tb/s

19.072Tb/s

38.145Tb/s

85%

90%

95%

100%

0 10 20 30 40 50 60 70%

75%

80%

85%

90%

95%

100%

0 10 20 30 40 50 60



1. High cost of IP router ports 2. High cost of IP router ports 3. High cost of IP router ports 4. Optical network costs


Transport Layer Architecture Options

“IP over Optical” formerly known as

IPoWDM

IP over WDM a.k.a “alien waves”

IP over WDM Router client to WDM

Architecture

Router interface SR - POS LR DWDM OTN SR – POS or 10GbE

WDM Node WDM Terminal

(point-point) All-Optical ROADM

Optical ROADM or Digital ROADM

Pass through IP traffic Back-back transponders Optical Express Optical or Digital Express

Protection Router - Layer3 Router - Layer 3 Router- Layer 3

Digital ROADM: <50ms

OAMP Limited (DWDM) Medium

(WDM PHY with OTN) Medium (Optical)

High (Digital)

O-E-O O-E-O O-E-O

O-E-O O-E-O O-E-O

OE

O

OE

O

OE

O

OE

O


“Alien” Wavelengths Practical Considerations & Operational Implications

Interconnect IP routers to WDM transport via short-reach optics Clear demarcation between client signal and

transport layer Full end-end optimization, control &

management over transport network “Best-in breed” IP and transport systems

WD

M

WD

M

Optical Transport Network

IP Network

Optical Transport Network

IP Network SR

SR

SR

SR

Native Wavelengths

“Alien” Wavelengths

Integrated ITU-grid WDM optics on router input directly to WDM line system No end-end control/management plane Design optical link budgets to worse-case Loss of end-end turn-up automation Complex or no inter-network protection Loss of PM and fault sectionalization

capabilities Complex service activation CapEx savings – if any – from transponder

reduction need to be significant to outweigh design tradeoffs and operational complexity

Vend

or A

Ve

ndor

B

Vend

or A

Ve

ndor

B


Alien Wave Economics 10GbE Router-to-Router Connections

Network Traffic Volume (Tb/s) Network Traffic Volume (Tb/s)

Transponder-based WDM Alien Waves WDM


Alien Wave Economics 40G Router-to-Router Connections

Transponder-based WDM Alien Waves WDM

Network Traffic Volume (Tb/s) Network Traffic Volume (Tb/s)


Impact of “Alien Waves”

Little/no cost savings versus use of transponders Alien waves essentially results in a “cost transfer” from WDM layer to IP

layer

For 40G, regen costs contribute major portion of WDM layer costs Not impacted by alien waves

Operational impact of multi-vendor alien waves not considered Potential for significant costs and complexity for system turn-up,

provisioning, management and trouble-shooting

IPoWDM architecture with IP bypass enables significantly greater savings


Conclusions

IPoWDM network cost optimization includes both simple fixes… Minimize use of expensive router ports

…more complex optimization… Increase degree of IP mesh to maximize router bypass

on high-bandwidth links Providing cost savings in IP layer, and transport layer

…and uncertainties… Real-world cost savings of alien waves? Operational considerations

Bandwidth Virtualization Enabled by Digital Optical Networks

Serge Melle


Agile Optical Networks: Theory vs. Reality

The Theory: Agile optical networks can transparently transport and switch any service, end-to-end

2.5G 10G 40G

100G

2.5G 10G 40G 100G

The Reality: “All-optical” implementation of many optical networks impedes service and network agility


Conventional Transponder-Based WDM

New services are added optically

Trans- mission

Fiber

Dispersion comp.

(R)OADM N-stage

Filter or WSS

2.5G

10G

40G

Amp

Transponders or Muxponders


All-Optical WDM/ROADM Networks

All-optical wavelength switching using filters, ROADM, WSS, etc.

OEO only for local add/drop No sub-wavelength add/drop No inter-wavelength grooming No digital PM or OAMP

End-end services over wavelengths Wavelength contention and blocking Service limited by wavelength path

Optical reach Number of (R)OADM nodes passed Fiber characteristics

All-optical pass-through

“express” traffic

Add/Drop “local” traffic

O-E-O

O-E-O

O-E-O

O-E-O

λ1 …λN

λ1 …λN

λi, λj

O-E-O

λ1 …λN

New service blocking due to wavelength contention

End-end wavelength service


Conventional WDM/ROADMs Tie Services to Optics

• Loss = ? • Channels lit ? • Dispersion = ?


• Loss = ?


• Loss = ? • Loss = ?


• Loss = ?

• Loss = ?

Service is specifically assigned to a specific end-to-end wavelength

Impacts: Planning Service support Wavelength engineering Deployment time Cost


Why Have We Gotten Here? The Original Problem Statement

Cost of optics is the bottleneck to wide-spread use of electronics

What has this led to…? “Shoot the messenger” – Drive to reduce/eliminate the

need for OEO rather than reducing the cost of OEO…

…which then limits the use of digital electronics…

…which reduces network functionality and service flexibility (ie: less switching, less PM data, less reconfigurability, etc.)


Relative Costs – The “O” vs. the “E”

High cost of OEO conversion compared to the cost for manipulating the data

Implementing feature-rich network & service functionality incurs a cost premium

Silicon cost is not the problem

Rel

ativ

e C

osts

per

10G

Accessing the Data Manipulating the Data


100 Gb/s Transmit

100 Gb/s Receive

Changing the Paradigm #1: Lower the Cost Barrier to OEO Use

Monolithic, large-scale InP Photonic Integrated Circuits

100Gb/s “WDM system on a chip” 62 components on Tx & Rx PICs

Photonic Integration


1001 0101 0101 1010 1101 0101 0101 1010 1101 0101

Changing the Paradigm #2: Digital, instead of Optical, Add/Drop

100101011101010000101011 100101010101101011010101 110101000010101110010101 001010111011010110010101

Inte

grat

ed P

hoto

nics

Inte

grat

ed P

hoto

nics

OEO conversion enables “digital” functionality Multi-wavelength scaling using PICs Sub-λ bandwidth management (ODU1) Separate tribs (service) from line (capacity) Digital ROADM add/drop, muxing & switching

Digital ROADM

Optical (O) Electrical (E) Optical (O)

Trib



1001 0101 0101 1010 1101 0101 0101 1010 1101 0101

Changing the Paradigm #3 Increase Service Agility

100101011101010000101011 100101010101101011010101 110101000010101110010101 001010111011010110010101

Inte

grat

ed P

hoto

nics

Inte

grat

ed P

hoto

nics

Digital Optical Network

Digital ROADM


10010101110101010000 10010101010110101011

optical link management

De-couple service provisioning from optical engineering

GMPLS software intelligence Digital switching at every node Unconstrained service delivery anywhere

10010101110101010000 10010101010110101011

end-end service


Service Adapters

PIC + Digital Switch

Digital ROADM Architecture

New services are added digitally Wavelengths are virtualized into digital bandwidth

Trans- mission

Fiber

Dispersion Comp.

2.5G

10G

40G

Amp

Serv

ice

Inde

pend

ent

Electrical Interface


Digital ROADM Functionality

2.5G 10G

40G

1G

De-couple services (tribs) from waves (line) Manage 100G per line card (400G per chassis) Manage all bandwidth at 2.5G (ODU1) granularity Multiplex, switch, groom at every node

01

02

03

04

Number of λs O

DU

1s p

er λ

100Gb/s “pool of bandwidth” per line card (40 x ODU1)

λ 1

05

06

07

08 λ 2

09

10

11

12 λ 3

13

14

15

16 λ 4

17

18

19

20 λ 5

21

22

23

24 λ 6

25

26

27

28 λ 7

29

30

31

32 λ 8

33

34

35

36 λ 9

37

38

39

40 λ 10


Digital ROADM Enables “Bandwidth Virtualization”

Waves are resources, not services

Capacity and services mix are independent

Any service anywhere, without pre-planning or forklifts

20G

ava

ilabl

e

125G available

65G

ava

ilabl

e

85G available

22.5

G a

vaila

ble

105G

ava

ilabl

e

2.5G

40G

Virtu

aliz

atio

n 127.5G available 85G available


Implementing Bandwidth Virtualization

Service – Wave Decoupling

Virtualized Bandwidth Pool 100 Gb/s per DLM/XLM

Client Interfaces

100G Line-Side DWDM Capacity

100 Gb/s OTM-10.2

2.5G

10G

40G

Bandwidth Virtualization decouples service provisioning from wavelength engineering

02

01

03

04 O

DU

1s p

er λ

λ 1

05

06

07

08

λ 2

09

10

11

12

λ 3

13

14

15

16

λ 4

17

18

19

20

λ 5

21

22

23

24

λ 6

25

26

27

28

λ 7

29

30

31

32

λ 8

33

34

35

36

λ 9

37

38

39

40

λ 10


Sub-lambda and Super-lambda Services

01

02

03

04

OD

U1s

per

λ

λ 1

05

06

07

08

λ 2

09

10

11

12

λ 3

13

14

15

16

λ 4

17

18

19

20

λ 5

21

22

23

24

λ 6

25

26

27

28

λ 7

29

30

31

32

λ 8

33

34

35

36

λ 9

37

38

39

40

λ 10

02

13

14

15

16

25

26

27

28

29

31

32

33

34

35

36

37

38

39

40

30

One Digital Virtual Concatenation (DVC) Group

Service – Wave Decoupling

Super-lambda services (>10 Gb/s) • OC-768/STM-256

Sub- lambda services (<10 Gb/s) • 2 x 1 GbE • OC-48/STM-16 • OTU1*

Lambda services (10 Gb/s) • 10 GbE • OC-192/STM-64 • OTU2*

Enabled by Digital Virtual Concatenation (DVC)

1GbE

10G

40G

2.5G

08

* Future Release


Bandwidth Virtualization Digital switching – line/trib separation - VCAT

---

A Programmable Optical Network

OC-48

100 GbE

40G GbE 2G

FC OTU2 ???

PIC-based Optical Engine

Network Intelligence

10G waves

40G waves

Add/ Drop

Reach Options

CD/PMD Comp

Service Adapters


“Service ready” capacity

10 day Just-In-TAM™

GMPLS lightpaths

Plan & light each service independently

Speed as a Sales Advantage

Win deals with shorter lead times Get revenue & cash earlier

Forecast tolerance

Vs.

Vs.

Vs.

8-20 week transponder lead time

Manual processes

-based network Competitor-based network


Enabling New Business Models

Level3: 10GbE LAN PHY

10G in 10 days

Interoute 10G in 10 Days

XO: 10G in 10 Days

Cox: Owned backbone

Carphone Warehouse: Owned backbone

“We've pretty much built a national network in eight weeks since signing the contract, and we can increase capacity pretty much at the touch of a button. It has taken the guesswork out of the optical network. We often get last-minute demand from the sales teams, and you need a technology partner that can help meet that demand….” - Paul Jackson, Head of Transmission


Summary

IP networks want service transparency Provide whatever connectivity I want, where I want, when I want, at

the lowest cost

Best approach to service transparency is an optically opaque network Optical transparency:

Forces IP layer to deal with analog optics Adds complexity Adds significant operational cost

The economic driver for optical transparency (high-cost OEO) is rendered moot with PICs

Allowing Digital ROADMs to maximize service agility

Architecture Optimization in IP over WDM Core NetworksPractical Considerations & Operational...

Documents

Transcript of Architecture Optimization in IP over WDM Core NetworksPractical Considerations & Operational...