Post on 04-Jun-2018
8/13/2019 Route Revolution
1/43
8/13/2019 Route Revolution
2/43
2
The Internet is a mesh ofrouters
The Internet Core
IP Core router
IP EdgeRouter
8/13/2019 Route Revolution
3/43
3
The Internet is a mesh of IProuters, ATM switches, frame
relay, TDM,
Access
Network
AccessNetwork
AccessNetwork
AccessNetwork
Access
Network
Access
Network
AccessNetwork
8/13/2019 Route Revolution
4/43
4
The Internet is a mesh ofrouters mostly interconnected by
(ATM and) SONET
TDMTDM
TDMTDM
Circuit switchedcrossconnects, DWDM etc.
8/13/2019 Route Revolution
5/43
5
Where high performance packetswitches are used
Enterprise WAN access
& Enterprise Campus Switch
- Carrier Class Core Router
- ATM Switch
- Frame Relay Switch
The Internet Core
Edge Router
8/13/2019 Route Revolution
6/43
6
Ex: Points of Presence (POPs)
A
B
C
POP1
POP3POP2
POP4 D
E
F
POP5
POP6 POP7
POP8
8/13/2019 Route Revolution
7/43
7
What a Router Looks Like
Cisco GSR 12416 Juniper M160
6ft
19
2ft
Capacity:160Gb/s
Power:4.2kW
3ft
2.5ft
19
Capacity:80Gb/s
Power:2.6kW
8/13/2019 Route Revolution
8/43
8
Basic Architectural Components
Output
Scheduling
Control Plane
Datapathper-packet
processing
SwitchingForwarding
Table
ReservationAdmission
Control Routing Table
RoutingProtocols
Policing
& AccessControl
Packet
Classification
Ingress EgressInterconnect
1. 2. 3.
8/13/2019 Route Revolution
9/43
9
Basic Architectural ComponentsDatapath: per-packet processing
2. Interconnect 3. EgressForwarding
Table
ForwardingDecision
1. Ingress
ForwardingTable
ForwardingDecision
ForwardingTable
ForwardingDecision
8/13/2019 Route Revolution
10/43
10
Generic Router Architecture
LookupIP Address
UpdateHeader
Header Processing
AddressTable
LookupIP Address
UpdateHeader
Header Processing
AddressTable
LookupIP Address
UpdateHeader
Header Processing
AddressTable
Data Hdr
Data Hdr
Data Hdr
BufferManager
BufferMemory
Buffer
Manager
BufferMemory
BufferManager
BufferMemory
Data Hdr
Data Hdr
Data Hdr
8/13/2019 Route Revolution
11/43
11
RFC 1812: Requirements for
IPv4 Routers Must perform an IP datagram forwarding
decision (called forwarding)
Must send the datagram out theappropriate interface (called switching)
Optionally: a router MAY choose to perform specialprocessing on incoming packets
8/13/2019 Route Revolution
12/43
8/13/2019 Route Revolution
13/43
13
Special Processing Requires
Identification of Flows All packets of a flow obey a pre-defined
rule and are processed similarly by the
router E.g. a flow = (src-IP-address, dst-IP-
address), or a flow = (dst-IP-prefix,protocol) etc.
Router needs to identify the flow of everyincoming packet and then performappropriate special processing
8/13/2019 Route Revolution
14/43
14
Flow-aware vs Flow-unaware
Routers Flow-aware router: keeps track of
flows and perform similar processing
on packets in a flow Flow-unaware router (packet-by-
packet router): treats each incoming
packet individually
8/13/2019 Route Revolution
15/43
15
Why do we Need Faster Routers?
1. To prevent routers becoming thebottleneck in the Internet.
2. To increase POP capacity, and toreduce cost, size and power.
8/13/2019 Route Revolution
16/43
16
0,1
1
10
100
1000
10000
1985 1990 1995 2000
Spec95IntCPUr
esults
Why we Need Faster Routers1:To prevent routers from being the bottleneck
0,1
1
10
100
1000
10000
1985 1990 1995 2000
FiberCapacity(Gb
it/s)
TDM DWDM
Packet processing Power Link Speed
2x / 18 months 2x / 7 months
Source: SPEC95Int & David Mil ler, Stanford.
8/13/2019 Route Revolution
17/43
17
POP with smaller routers
Why we Need Faster Routers2:To reduce cost, power & complexity of POPs
POP with large routers
! Ports: Price >$100k, Power > 400W.! It is common for 50-60% of ports to be for interconnection.
8/13/2019 Route Revolution
18/43
18
Why are Fast Routers Difficult to
Make?
1. Its hard to keep up with Moores Law:
The bottleneck is memory speed.
Memory speed is not keeping up withMoores Law.
8/13/2019 Route Revolution
19/43
19
Memory BandwidthCommercial DRAM
1. Its hard to keep up with Moores Law:
The bottleneck is memory speed.
Memory speed is not keeping up withMoores Law.
0,001
0,01
0,1
1
10
100
1000
1980 1983 1986 1989 1992 1995 1998 2001
Ac
cessTime
(ns)
DRAM1.1x / 18months
Moores Law2x / 18 months
RouterCapacity2.2x / 18months
Line Capacity2x / 7 months
8/13/2019 Route Revolution
20/43
20
Why are Fast Routers Difficult to Make?
1. Its hard to keep up with Moores Law:
The bottleneck is memory speed.
Memory speed is not keeping up withMoores Law.
2. Moores Law is too slow:
Routers need to improve fasterthanMoores Law.
8/13/2019 Route Revolution
21/43
21
Router Performance Exceeds Moores Law
Growth in capacity of commercial routers:
Capacity 1992 ~ 2Gb/s
Capacity 1995 ~ 10Gb/s Capacity 1998 ~ 40Gb/s
Capacity 2001 ~ 160Gb/s
Capacity 2003 ~ 640Gb/s
Average growth rate: 2.2x / 18 months.
8/13/2019 Route Revolution
22/43
22
Things that slow routers down
250ms of buffering Requires off-chip memory, more board space, pins and power.
Multicast Affects everything!
Complicates design, slows deployment.
Latency bounds
Limits pipelining.
Packet sequence Limits parallelism.
Small internal cell size Complicates arbitration.
DiffServ, IntServ, priorities, WFQ etc.
Others: IPv6, Drop policies, VPNs, ACLs, DOS traceback,measurement, statistics,
8/13/2019 Route Revolution
23/43
23
First Generation Routers
Shared Backplane
LineInterface
CPU
Memory
RouteTable
CPU BufferMemory
LineInterface
MAC
LineInterface
MAC
LineInterfaceMAC
Fixed length DMA blocksor cells. Reassembled on egress
linecard
Fixed length cells or
variable length packets
Typically
8/13/2019 Route Revolution
24/43
24
Output 2
Output N
First Generation RoutersQueueing Structure: Shared Memory
Large, single dynamically
allocated memory buffer:N writes per cell timeN reads per cell time.
Limited by memorybandwidth.
Input 1 Output 1
Input N
Input 2
Numerous work has proven andmade possible:
Fairness
Delay Guarantees
Delay Variation Control
Loss Guarantees
Statistical Guarantees
8/13/2019 Route Revolution
25/43
25
Limitations First generation router built with 133 MHz Pentium
Mean packet size 500 bytes Interrupt takes 10 microseconds, word access take 50 ns
Per-packet processing time is 200 instructions = 1.504 !s
Copy loopregister
8/13/2019 Route Revolution
26/43
26
Second Generation Routers
RouteTable
CPU
LineCard
BufferMemory
LineCard
MAC
BufferMemory
LineCard
MAC
BufferMemory
FwdingCacheFwdingCache
FwdingCache
MAC
Slow Path
Drop PolicyDrop Policy OrBackpressure
OutputLink
Scheduling
Buffer
Memory
Typically
8/13/2019 Route Revolution
27/43
27
RouteTable
CPU
Second Generation RoutersAs caching became ineffective
LineCard
BufferMemory
LineCard
MAC
BufferMemory
LineCard
MAC
BufferMemory
FwdingTableFwdingTable
FwdingTable
MAC
ExceptionProcessor
8/13/2019 Route Revolution
28/43
28
Second Generation RoutersQueueing Structure: Combined Input and Output
Queueing
Bus
1 write per cell time 1 read per cell timeRate of writes/readsdetermined by bus speed
8/13/2019 Route Revolution
29/43
29
Third Generation Routers
LineCard
MAC
LocalBufferMemory
CPUCard
LineCard
MAC
LocalBufferMemory
Switched Backplane
LineInterface
CPU
Memory Fwding
Table
RoutingTable
FwdingTable
Typically
8/13/2019 Route Revolution
30/43
30
Arbiter
Third Generation RoutersQueueing Structure
Switch
1 write per cell time 1 read per cell timeRate of writes/readsdetermined by switch
fabric speedup
8/13/2019 Route Revolution
31/43
31
Arbiter
Third Generation RoutersQueueing Structure
Switch
1 write per cell time 1 read per cell timeRate of writes/readsdetermined by switch
fabric speedup
Per-flow/class or per-output queues (VOQs)
Per-flow/class or per-input queues
Flow-control
backpressure
8/13/2019 Route Revolution
32/43
32
Fourth Generation Routers/Switches
Switch Core Linecards
Optical links
100s
of feet
8/13/2019 Route Revolution
33/43
33
Physically Separating Switch
Core and Linecards Distributes power over multiple racks.
Multistage, clustering
Allows all buffering to be placed onthe linecard: Reduces power.
Places complex scheduling, buffer mgmt,drop policy etc. on linecard.
8/13/2019 Route Revolution
34/43
34
Do optics belong in routers?
They are already there.
Connecting linecards to switches.
Optical processing doesnt belong onthe linecard.
You cant buffer light.
Minimal processing capability.
Optical switching can reduce power.
8/13/2019 Route Revolution
35/43
35
Optics in routers
Switch Core Linecards
Optical links
8/13/2019 Route Revolution
36/43
36
Complex linecards
Physical
Layer
Framing
&Maintenance
PacketProcessing
Buffer Mgmt&
Scheduling
Buffer Mgmt&
Scheduling
Buffer& StateMemory
Buffer& StateMemory
Typical IP Router Linecard
10Gb/s linecard:!Number of gates: 30M!Amount of memory: 2Gbits!Cost: >$20k!Power: 300W
LookupTables
Switch
Fabric
Arbitration
Optics
8/13/2019 Route Revolution
37/43
37
Replacing the switch fabric with optics
SwitchFabric
Arbitration
Physical
Layer
Framing&
Maintenance
PacketProcessing
Buffer Mgmt
&Scheduling
Buffer Mgmt&
Scheduling
Buffer
& StateMemory
Buffer& State
Memory
Typical IP Router LinecardLookup
Tables
OpticsPhysical
Layer
Framing&
Maintenance
PacketProcessing
Buffer Mgmt
&Scheduling
Buffer Mgmt&
Scheduling
Buffer
& StateMemory
Buffer& State
Memory
Typical IP Router LinecardLookup
Tables
Opticselectrical
SwitchFabric
Arbitration
PhysicalLayer
Framing&
Maintenance
PacketProcessing
Buffer Mgmt
&Scheduling
Buffer Mgmt
&Scheduling
Buffer& StateMemory
Buffer& StateMemory
Typical IP Router LinecardLookupTables
OpticsPhysicalLayer
Framing&
Maintenance
PacketProcessing
Buffer Mgmt
&Scheduling
Buffer Mgmt
&Scheduling
Buffer& StateMemory
Buffer& StateMemory
Typical IP Router LinecardLookupTables
Optics
optical
Req/Grant Req/Grant
Candidate technologies1. MEMs.
2. Fast tunable lasers + passiveoptical couplers.3. Diffraction waveguides.4. Electroholographic materials.
8/13/2019 Route Revolution
38/43
38
Evolution to circuit switching
Optics enables simple, low-power,very high capacity circuit switches.
The Internet was packet switchedfor two reasons: Expensive links: statistical multiplexing.
Resilience: soft-state routing.
Neither reason holds today.
8/13/2019 Route Revolution
39/43
39
Fast Links, Slow Routers
0,1
1
10
100
1000
10000
1985 1990 1995 2000FiberCapacity
(Gb
it/s)
Fiber optics DWDM
1
10
100
1000
10000
1985 1990 1995 2000
Spec95IntCPUr
es
ults
Processing Power Link Speed (Fiber)
2x / 2 years 2x / 7 months
Source: SPEC95Int; Prof. Miller, Stanford Univ.
8/13/2019 Route Revolution
40/43
40
Fewer Instructions
1
10
100
1000
1996 1997 1998 1999 2000 2001
(log
scale)
Instructions per packet since 1996
8/13/2019 Route Revolution
41/43
41
Some Mc Keown's predictions
about core Internet routers The need for more capacity for a given power and volume
budget will mean:
Fewer functions in routers:
Little or no optimization for multicast,
Continued overprovisioning will lead to little or no support forQoS, DiffServ, ,
Fewer unnecessary requirements:
Mis-sequencing will be tolerated,
Latency requirements will be relaxed.
Less programmability in routers, and hence no networkprocessors.
Greater use of optics to reduce power in switch.
8/13/2019 Route Revolution
42/43
42
What McKeown believe is most
likelyThe need for capacity and reliability will mean:
Widespread replacement of core routerswith transport switching based on circuits: Circuit switches have proved simpler, more
reliable, lower power, higher capacity and lowercost per Gb/s. Eventually, this is going to matter.
Internet will evolve to become edge routersinterconnected by rich mesh of DWDM circuitswitches.
8/13/2019 Route Revolution
43/43
43
Click here to view forwarding
technologies