Isaac Building Blocks In v2 - NANOG...Agenda •Quick recap of EVPN fundamentals (5 slides) •EVPN...

Aldrin Isaac

Co-author RFC7432

Juniper Networks

Building Blocksin EVPN VXLAN

for Multi-Service Fabrics

Network Subsystems

Network Virtualization

Bandwidth Broker TE

WAN FabricLAN Fabric

LAN WAN

EVPN for Network Virtualization

EVPN Bandwidth Broker TE

WAN FabricLAN Fabric

LAN WAN

EVPN in the LAN Fabric

LAN Fabric

EVPN

LAN

= “The Multi-Service Fabric”

Agenda

• Quick recap of EVPN fundamentals (5 slides)

• EVPN overlay options for intra-tenant east-west traffic (16 slides)

• Examples: interesting use cases with EVPN (7 slides)

• North-south traffic through EVPN based service chains (14 slides)

• Efficient replication options in EVPN (8 slides)

Things to note about this tutorial

• Is about native EVPN building blocks that are compliant with RFCs or standards-track drafts. No proprietary technology

• Is about what has been implemented or is possible to implement on network SW/HW today

• Will not go into route and tunnel header gory details

• Is based on EVPN VLAN-Aware bridging model (vs VLAN-based)

• As we move forward, we will move faster.

7

Network Virtualization Overlay Reference Model for this Tutorial

Tenant 1

VLAN2VLAN1

E2E1E4E3

Tenant 2

VLAN4VLAN3

E5 E6E7 E8

• For this tutorial, “tenants” are groups of location-independent endpoints where:

• Groups manifest as subnets that are routed to other groups of the same tenant (i.e. east-west) via a distributed routing function

• Tenants are routed to other tenants and to external destinations (i.e. north-south) through service function chains

• Tenants and groups are implemented as IP and Ethernet overlay virtual networks

• The network virtualization edge (NVE) function may be implemented on• ToR switch: to support physical end-systems• Virtual routers: to support virtual end-points

• Note: NVE are also referred to as PE in SP networks, or VTEP in VXLAN networks.

SF

VRF2 VRF1 VRF2

E4

VRF1 VRF2

E3 E8E7E6E5

VRF1

E1 E2

OverlayEdge

VXLAN overlay data plane

BGP Route Reflectors

“NVE”“VTEP”“PE”

VLAN2 VLAN3 VLAN4VLAN1VLAN3 VLAN4 VLAN1 VLAN2

IP EVPN

Broadcast DomainEVPN Tag

VXLAN VNI


VXLAN VNI

Ethernet EVPNaka EVI zz

Virtual Switchaka MAC-VRF

Physical Switch

Physical Switch Physical Switch

RFC/Drafts: RFC7432, draft-ietf-bess-evpn-overlay, draft-ietf-bess-evpn-prefix-advertisement

EVPN Parallels with Classical Networks

VTEPNVE / PE

VLAN Table

VLAN Table

VLAN Table

VLAN Table

VLAN Table

VLAN Table

VLAN Table

VLAN Table

Virtual Routeraka VRF

Physical Router

Physical Router

Physical Router

EVPN Network Classical Network

IRB Interfaces

IP FabricMulti-Tenant Single-Tenant

MP-BGP Route Reflector

VTEP 2VTEP 3


BGP-based VPNs Overview

VTEP 1MP-BGP

EVPN

IPVPN-A


VXLAN VNI


VXLAN VNI

EVI-A

Route export with Extended Community RT 1111:1111

Route import with Extended Community RT 1111:1111

Route export with Extended Community RT 2222:2222

Route import with Extended Community RT 2222:2222

MAC-VRF-A BGP Policy

VRF-A BGP Policy

MAC-VRF-A

VRF-A

VLAN 10EVPN Tag 100

VXLAN VNI 100

VLAN 20EVPN Tag 200

VXLAN VNI 200

Tunnels

L3 Routes

L2 RoutesL1 Routes

IP Fabric

L1: Ethernet Multi-Homing • Type-4 Ethernet Segment (ES) Route

• Designated Forwarder (DF) election

• Type-1 Ethernet A-D Route• Per ES

• Split horizon, Fast convergence• Per EVI (ES:Tag)

• Aliasing

• Type-7 Multicast Join Sync Route• Selective IP multicast support

• Type-8 Multicast Leave Sync Route• Selective IP multicast support

L2: Ethernet Bridging• Type-2 MAC/IP route

• MAC-Only• MAC unicast forwarding

• MAC + IP• ARP Proxy

• Type-3 Inclusive Multicast Ethernet Tag (IMET) Route• BUM forwarding

• Type-6 Selective Multicast Ethernet Tag (SMET) Route• Selective IP multicast forwarding

EVPN Route Types – By Layer

L3: IP Routing• Type-5 IP Prefix Route

• MAC-VRF IP forwarding

• Type-5 “VRF-to-VRF” IP Prefix Route• VRF IP forwarding

“Layer 2.5”

Includes Tag onlyIncludes Tag & ESI

Includes ESI only

10

Unicast• L1: Type-1 Ethernet A-D Route per ES

• Fast convergence

• L1: Type-1 Ethernet A-D Route per EVI• Aliasing

• L2: Type-2 MAC/IP route• MAC unicast forwarding, ARP Proxy **

• L3: Type-5 Prefix Route Route• IP forwarding

EVPN Route Types – By Unicast-related Vs Replication-related

BUM and IP Multicast• L1: Type-1 Ethernet A-D Route per ES

• Split horizon

• L1: Type-4 Ethernet Segment (ES) Route• Designated Forwarder (DF) election

• L1: Type-7 Multicast Join Sync Route• Selective IP multicast support

• L1: Type-8 Multicast Leave Sync Route• Selective IP multicast support

• L2: Type-3 Inclusive Multicast Ethernet Tag (IMET) Route• BUM forwarding

• L2: Type-6 Selective Multicast Ethernet Tag (SMET) Route **• Selective IP multicast forwarding

11

Intra-Tenant (EAST-WEST) Overlay Service Models

1. Pure Bridging Overlay

14

Bridging Overlay

• Unicast MAC forwarding• EVPN Type-2 MAC-only route• Routes generated from locally learned MACs

in local VLAN table

• BUM forwarding • Type-3 Inclusive Multicast Ethernet Tag

(IMET) route• Ingress replicated by default

• Overlay transport • VXLAN tunnels are marked with the VNI of a

transported broadcast domain.• Like Ethernet trunks between physical

switches• VXLAN VNI is carried in Label and Tag field of

EVPN NLRI

• ARP suppression• Add Type-2 MAC+IP route

RFC/Drafts: RFC7432, draft-ietf-bess-evpn-overlay

ExternalGateway VRF1VRF1

VLAN1 VLAN2 VLAN2VLAN1 VLAN2VLAN1 VLAN2

VLAN1 VLAN2VLAN1 VLAN2

WAN

NVEL2VNs

NVE

Spine

BridgingOnly

BridgingOnly


MAC-VRF-TMAC-VRF-T

Bridging Overlay Detail

Leaf1 Leaf2

VLAN1 VLAN2

L2 EVPN

VLAN1VLAN2

H2 H3H1 H4

Type-2 MAC, Type-3 IMET

← MAC →

15

ARP Proxy

17

EVPN ARP Proxy -- Synchronization and Suppression

• ARP synchronization keeps the per-subnet ARP tables of tenant

VRFs synchronized

• MAC-to-IP bindings are learned by Leaf VTEP from the Sender

field of local ARP request and reply packets and advertised as

Type-2 MAC+IP routes

• MAC-to-IP bindings can be learned and advertised by Leaf VTEP

with or without local VRF

• With distributed ARP broadcast suppression, Leaf VTEP will

proxy respond to local ARP requests using the same

synchronized MAC-to-IP bindings

• Reduces the impact of ARP broadcast on routers and hosts

• MAC-to-IP bindings may be learned from DHCP messages and

coupled with sticky MAC procedures to safeguard against IP

spoofing, ARP poisoning and duplicate detection

ARP Suppression

OriginalARP response

GeneratedARP response

GeneratedARP response

Leaf3

H2 H3

ARP request ARP request

1 3

4

Leaf2

H1

Leaf1

Subnet 1 Subnet 1 Subnet 1

MAC/IP Route2

RFC/Drafts: RFC7432, draft-ietf-bess-evpn-proxy-arp-nd

Flow 2

ARP Synchronization

VRF1VRF1

ARP request

ARP response

Flow 1

15

3

MAC/IP Route

2

4

Leaf

Gateway

18

EVPN ARP suppression (cont’d) – Gratuitous ARP Proxy

• GARP proxy is a feature of EVPN ARP suppression used to avoid data-plane flooding of GARPs.

• MAC-to-IP bindings are learned from Sender field of local GARP and advertised as Type-2 MAC+IP routes

• VTEP regenerate GARP to local end systems when they receive new remote MAC-to-IP bindings via Type-2 MAC+IP routes

• Example scenarios:• VIP mobility for active-standby firewall• Mobility in bridged mode WIFI• VM mobility


GARP

RegeneratedGARP

RegeneratedGARP

Leaf3

H2 H3

1

Leaf2

H1

Leaf1


MAC/IP Route

2

3 3

2. Centrally Routed Bridging Overlay

20

Centrally Routed Bridging (CRB) Overlay• IP routing is performed with IRB at central gateway

VTEP. All default gateways for a subnet should

share same MAC and IP.

• CRB gateway role can be placed at spine, leaf or

anywhere else

• CRB access role at Leaf VTEPs only perform

bridging

• Host packets addressed to IRB MAC are forwarded

to CRB gateway for routing. Other MACs are

forwarded directly between Leaf.

• Type-2 MAC+IP route provides ARP

synchronization between central gateways

• T2 MAC+IP also supports ARP suppression at leaf

VTEP without need for local VRF

• Typical use case: where CRB gateway supports

advanced functions, such as high ACL scale,

stateful FW, NAT, etc vs CRB access


VRF1 VRF1CRB Access

CRB Border

Gateway

VLAN1 VLAN2 VLAN2 VLAN1 VLAN2 VLAN1 VLAN2VLAN1

VLAN1 VLAN2

WAN

L2VNs

Gateway1

Leaf1 Leaf2← MAC/IP → ← MAC/IP →

MAC-VRF-T

VLAN1 VLAN2

VRF-T


← MAC/IP →

MAC-VRF-TMAC-VRF-T

Centrally Routed Bridging Detail

VLAN1 VLAN2

L2 EVPN

VLAN1VLAN2

H2 H3H1 H4


Type-2 MAC, MAC+IP

Type-2 MAC, MAC+IP

21

CRB Access

CRB Access

CRB Gateway

3. Edge Routed Bridging Overlay

Edge Routed Bridging (ERB) Overlay

• Both intra and inter subnet IP forwarding are performed at

Leaf VTEP with IRB. All gateways for a subnet must share

same MAC and IP.

• Asymmetric ERB:

• Same route types as CRB

• Inter-subnet forwarding relies on ARP table

synchronization using Type-2 MAC+IP route

• Drawback: All VLANs of tenant must be provisioned at all

the VTEP where the tenant VRF is present

• Type-5 based Symmetric ERB (recommended):

• Uses Type-5 Prefix Route to exchange IP host routes for

inter-subnet forwarding – carries VRF VNI

• Locally learned ARP entries are imported into RIB and

advertised as Type-5 host routes

• Type-2 MAC+IP route is used for distributed ARP

suppression

• Advantages: L2VN/VLAN need to only be provisioned on

the VTEP that have locally attached members of that VN.

So has improved scaling over asymmetric model


ERBVRF1 VRF1VRF1 VRF1IP Border Gateway

VLAN1 VLAN2 VLAN2

WAN

L3VNsL2VNs

← MAC/IP →

MAC-VRF-TMAC-VRF-T

VLAN1 VLAN2 VLAN1VLAN2


Edge Routed Bridging Detail

Leaf1 Leaf2

← Host IP →

IP EVPNVRF-T VRF-T

L2 EVPN

H2 H3H1 H4


Local

Type-5 IP Host

, Type-2 MAC, MAC+IP

4. IP Routed Overlay

IP Routed Overlay

• IPVPN for LAN using EVPN and VXLAN. No Ethernet

Bridging.

• IP overlays are useful for

• North-south traffic flows (“service chaining”)

• Tenants that have no need for Ethernet bridging

• Uses only EVPN Type-5 Prefix route

• Requires BGP to host for IP address mobility

• May be useful for cloud fabrics as well:

• Lean core option for SaaS fabrics

• Or lightweight network-level multi-tenancy

option for SaaS operators (Ex: production and

development on same fabric)

• Additionally, overlay tunnels can enable useful

functions such as in-situ OAM and GBP

RFC/Drafts: draft-ietf-bess-evpn-prefix-advertisement section 5.4.1

IP Border Gateway

IP OnlyVRF1 VRF1 VRF1

WAN

L3VNsVRF1


Full Mesh IP EVPN

Leaf3

Leaf1 Leaf2Tk →

VRF-T VRF-T← Xj

T i→← Xj

VRF-T

Import RT-TExport RT-T



Type-5

← T k

Xi →


Hub-and-spoke IP EVPN

Border

Leaf1 Leaf2G →

VRF-X VRF-X←

G

Xi →← Xj

VRF-G

Import RT-GExport RT-X

Import RT-GExport RT-X

Import RT-XExport RT-G

Type-5


MAC-VRF-TMAC-VRF-T

Edge Routed Bridging with IP Border Gateway (N-S) Function Detail

Border

Leaf1 Leaf2

← Host IP →

Default →

IP EVPN

VLAN1

VRF-T VRF-T

VLAN2

L2 EVPN

VLAN1VLAN2

H2 H3H1 H4

← Defa

ult

Host IP →← Host IP


Local

Type-5 IP Host

Type-5 IP Prefix← Aggregates VRF-G

← MAC/IP →

IP Routed Overlay with Host Mobility• Like ERB, but with no bridging overlay.

• Mobility here means a host IP can only be at one VTEP

or another, not both. This is typical for Ethernet

bridging, but not typical for IP routing.

• ARP entries from local VLAN are imported to RIB and

exported as mobile Type-5 host routes.

• Uses Mobility Extended Community with Type-5

routes like with Type-2 routes. VTEPs with non-

highest sequence number must clear their local ARP

entry and withdraw their advertisement.

• Requires IP-move suppression like with MAC-move

suppression

• Supports subnets stretched across multiple VTEP.

• Classical proxy ARP used for non-local members

of subnet

• Broadcasts and multicast are local-only

• All gateways for a distributed subnet must share

same MAC and IP for workload mobility

• Caveat: No Ethernet multi-homing

IP Only Mobility

VRF1 VRF1VRF1 VRF1IP Border Gateway

VLAN1 VLAN2 VLAN2

WAN

L3VPN

RFC/Drafts: draft-ietf-bess-evpn-prefix-advertisement, RFC7814, draft-malhotra-bess-evpn-irb-extended-mobility-04#section-8

RFC/Drafts: draft-ietf-bess-evpn-prefix-advertisement, RFC7814, draft-malhotra-bess-evpn-irb-extended-mobility-04#section-8

IP Routed Overlay with Host Mobility

Leaf1 Leaf2

← Host IP →

IP EVPN

VLAN1

VRF-T VRF-T

VLAN2 VLAN1VLAN2

H2 H3H1 H4

Type-5 IP Host with Mobility

Classical Proxy ARP and Type-5 Host with Mobility

Local

Multi-homing

33

Ethernet Multihoming

• EVPN supports N-way Ethernet multihoming where N can

be greater than 2

• No ICL link required

• Uses EVPN Type-1 and Type-4 routes

• Adds EVPN Type-7 and Type-8 routes for selective multicast

• Multi-homed end-systems are identified in the overlay by

unique Ethernet Segment ID (ESI).

• ESI identify unique split horizon boundary.

• Only one member link of an ESI is allowed to forward

BUM packets. This member is known as the

Designated Forwarder (DF)

• ESI may be at the granularity of physical port or at the

granularity of logical interface (VLAN ID)

• EVPN Auto-ESI -- ESI generated automatically from LACP

system-id or from BPDU root bridge snooping


LAG Trunk

VLAN1 VLAN2 VLAN1 VLAN2 VLAN1 VLAN2

VLAN1 VLAN2

ESI-2ESI-1 ESI-2ESI-1 ESI-2ESI-1

LAG

VRF1 VRF1 VRF1

IP Multihoming

Routed BMS / H-visor / NF

VRF1 VRF1 VRF1

VLAN1 VLAN2 VLAN3Leaf

Ethernet-connected

eBGP

Ethernet port

IP port

• End-system IP ports connect Ethernet ports into local subnet on each leaf

• Routed via a local IRB on each local subnet• Less address management -- well suited for server

attachment• Floating IP, loopback and other routes advertised into

overlay via eBGP peering between end-system and leaf IRB interface

VRF1 VRF1 VRF1

Routed NF

Leaf

IP-connected

eBGPIP port

• Routed IP interface on either side of the link• No VLANs or IRB interfaces required at the leaf• Better for network functions, like routers• eBGP for advertising routes into overlay

Special Use Case Examples(with EVPN-native multi-homing support)

Example 1

Underlay Routed Overlay Subnets

GRT-based Edge Routed Bridging

• Single-tenant variant of symmetric ERB where IP routing is performed in the global routing table.

• No network virtualization and tunneling for IP.

• Basic use case is EVPN-based Ethernet multihoming for a GRT-routed end-system instead of MC-LAG

• Expanded use case allows a subnet to exist across any number of leaf, with routing performed in the global routing table

• Supports ARP suppression


GRT ERB

WAN

VLAN1 VLAN2 VLAN1 VLAN2

inet.0 inet.0inet.0inet.0 L2VNs

Example 2

Legacy Access Switch on EVPN

Legacy Access Switch Support

• Form of ERB where legacy Ethernet access switches (vs end-systems) are multihomed to a set of leaf VTEP

• Leaf VTEP may advertise subnet routes instead of host routes if subnet is not distributed

• EVPN multihoming down and proprietary MC-LAG up

• Great example of EVPN N-way multi-homing

• Collapsed spine pod may be part of a larger IP fabric

• Typical use case: transitional step from traditional “MC-LAG” model to a full overlay model with support for existing access switches from any vendor

VRF1 VRF1 VRF1

VLAN1 VLAN2VLAN1 VLAN2

VLAN1 VLAN2VLAN1 VLAN2VLAN1 VLAN2

Collapsed Spine

Bridged BMS / NF

Bridged H-visor / NF

VRF1 VRF1 VRF1

EVPN ESI

MC-LAG

Access Switch

VLAN1 VLAN2

VLAN1 VLAN2 VLAN1 VLAN2VLAN1 VLAN2

VLAN1 VLAN2 VLAN1 VLAN2

L3VNsL2VNsERB

Example 3

BUM-free Subnets

XBUM

41

Bum-free Subnet (Only Known MAC Unicast and IP Unicast)

Problem Statement• Some Ethernet services are unicast-only, but

unfortunately still need BUM support for ARP• Operators of these services do not want any packet

replication on their network (ex: IX, CX, Hosting, IaaS, etc)

Solution• Enable ARP suppression with GARP support• Do not import/export BUM and IP Multicast route

types 3 (IMET) and 6/7/8 (SMET).

Benefits• No BUM = no loop issues• No flood list state and related scale issues• IPVPN-like with Ethernet plug-and-play

Note• Requires GARP from host on startup ( “arping -A -c 4 -

I eth0” in dhcpcd-run-hooks ) and whenever MAC/IP binding changes or endpoint moves.


Leaf3

H2 H3

GARP

1

Leaf2

H1

Leaf1


MAC, MAC/IP Route2

RegeneratedGARP

3

RegeneratedGARP

3

Generated ARP Response

5ARP Request

4

Example 4

PVLAN Emulation

PVLAN Emulation using ERB with A/S GatewayWith support for A/A multihoming

Hub-and-SpokeIP EVPN

← D

efau

lt ← Host IP

Gateway

FW1a FW1b

ERB Tenant-GVRF-G, VNI-G

GW IP 10.2.2.1/29FW-VIP 10.2.2.2

FW1a 10.2.2.3FW1b 10.2.2.4

Routing table filter

Static route0/0 → FW-VIP

Group-BIsolated

E2 E4

ERB Tenant-BVRF-B, VNI-BIRB IP 10.1.1.1/24DHCP Relay for ERBIRB filters for PVLANPort filters for Isolated PVLAN

Same as Group-A

• Subnet 10.1.1.0/24 must be shared without overlap across two server groups, A & B

• Servers in group A and servers in group B must not be reachable to one another

• Servers within group A must be reachable to other servers within group A (“community”)

• Servers in group-B must not be reachable to other servers in Group-B (“isolated”)

• Both group A & B servers must share a common active-standby firewall gateway pair, FW1, to communicate with external endpoints

Caveats:• Need logical VRF per group• No north-south multicast yet

Problem statement:

Group-ACommunity

E3E1

DHCP

ERB Tenant-AVRF-A, VNI-A IRB IP 10.1.1.1/24DHCP Relay for ERBIRB filters for PVLAN

Located in underlaySupports option-82

Different subnet from Group A & B

PVLAN Emulation with ERB – Server Group A & B Detail

Hub-and-SpokeIP EVPN

Host

IP →

← D

efau

lt Default → ←

Host IP

Import RT-G (Default)Export RT-AB (Host)

(5) ADD Hub-Spoke IP

EVPN

IRB Input Filterdeny src 10.1.1.1deny src except 10.1.1.0/24deny dst 10.1.1.0/24 except 10.1.1.1

IRB Output Filterdeny dst except 10.1.1.0/24deny src 10.1.1.0/24 except 10.1.1.1

(3) ADD IRB filters for PVLAN

Port Input Filter: deny src Anycast-IRB-MAC

Port Output Filter: deny src except Anycast-IRB-MAC

(4) ADD port filters for Isolated PVLAN

VRF-BMAC-VRF-BVLAN-B as VNI-B

Anycast IRB IP 10.1.1.1/24Anycast IRB MAC xE:xx:xx:xx:xx:xx

(1) ERB

MAC-VRF-BMAX-VRF-A

VNI-A VNI-B

E1 E3 E2 E4

VRF-A VRF-B

DHCPRelay

DHCP Relay:remote-id = “<IRB>:10.1.1.0”source & giaddr = underlay loopback IP

(2) ADD DHCP for ERB

All server groups in a PVLAN use same subnet and same DHCP pool

DHCPLocated in underlayFor opt82 remote-id

= .*:10.1.1.0pool = 10.1.1.0/24

Example 5

VXLAN / MPLS / SRv6Coexistence

Leaf1 Leaf2

← Route Leak →← Route Leak → ← Host IP →

IP EVPNVXLAN-

VRF

VXLAN-

VRF

← Host MAC/IP →

VXLAN-MAC-VRFVXLAN-MAC-VRF

Telco Cloud EVPN-VXLAN and MPLS-IPVPN Coexistence Use Case

VLAN1 VLAN2

L2 EVPN

VLAN1VLAN2

BE1 BE2FE1 FE2

EVPN Type-2 MAC, MAC+IP

Local

EVPN Type-5 IP Host East-WestDomain

MPLS-VRFMPLS-VRF

No

rth

-So

uth

Do

mai

n

IPVPN-EVPN Local Chaining

Leaf1

Leaf1 Leaf2← Route Leak →← Route Leak → ← Host IP →

IP EVPNVXLAN-

VRFVXLAN-

VRF

← Host MAC/IP →


Telco Cloud EVPN-VXLAN and SRv6 Coexistence Use Case

VLAN1 VLAN2

L2 EVPN

VLAN1VLAN2

BE1 BE2FE1 FE2


Local

EVPN Type-5 IP Host East-WestDomain

IPv6 GRTIPv6 GRT

Nor

th-S

outh

Dom

ain

EVPN-GRT Local ChainingSR segments pushed at FEToR simply routes IPv6

Leaf1

Service-chaining N-S Traffic

Service Chaining Reference Model for “North South” Traffic

Tenant 1

BD2BD1E2E1

E4E3

Tenant 2

BD4BD3E5 E6

E7 E8

SF

WAN

SF

We have seen this before…

Hub-and-SpokeL3VN

← D

efau

lt ← Host IP

Static route0/0 → FW-VIPService

FunctionChain

VRF-A, VNI-A GW IP 10.1.1.1/24

Group-A Group-BE2 E4E3E1

VRF-B , VNI-BGW IP 10.1.2.1/24

Gateway

FW1a FW1b

VRF-G, VNI-G GW IP 10.2.2.2/29

FW-VIP 10.2.2.1

Service Function(Stateful FW)


← Host MAC/IP →

MAC VRFMAC VRF

And another SF/SFC example we have looked at…

Border

Leaf1 Leaf2

← Host IP →

IP EVPN

VLAN1

VRF-T VRF-T

VLAN2

L2 EVPN

VLAN1VLAN2

H2 H3H1 H4

Default →

← Defa

ult


← Aggregates VRF-G

Service Function(MPLS VPN Gwy)

Service Function

Chain

Leaf1 Leaf2

← Route Leak →← Route Leak → ← Host IP →

IP EVPNVXLAN-

VRFVXLAN-

VRF

← Host MAC/IP →


And another kind of SFC we have seen…

VLAN1 VLAN2

L2 EVPN

VLAN1VLAN2

BE1 BE2FE1 FE2


Local

EVPN Type-5 East-WestDomain

MPLS-VRFMPLS-VRF

Nor

th-S

outh

Dom

ain

IPVPN-EVPN Local Chaining

Leaf1

Service Function

Chain

Service Function(MPLS VPN Gwy)

Service Chaining Using Our Building Blocks

VRF-L9BD

-L2-1BD

-L2-2

Tenant-L2

VRF-L8BD

-L1-1BD

-L1-2

Tenant-L1

BD-R

9-2

Tenant-R2

VRF-

R8BD

-R8-

1BD

-R8-

2

Tenant-R1

BD-R

8-FW

1

ERB CRB

Bridged

VRF-SF1-L

SF1-L

BD-SF1-L

ERBH & S L3VN

ServiceFunction

GW1aL3

GW1bL3

Gateway

SF1aL3

SF1bL3

VRF-SF1-RBD

-SF1-R

SF1-R

VRF-SF1-L

SF2-L

IPERB H & SL3VN

ServiceFunction

SF2aL1

SF2bL1

SF2-R

VRF-SF1-L

VRF-SF1-LBD

-SF1-L

GW1-L

IP H & SL3VN

ERBERB

Service Chainswith Bi-Way Service Functions

Playing Service Chain Lego

Tail of chain

To right function

Head of chain

To left function

Tenant

Function

External

Fabric

Connector Legend

Service Function Type Examples

VRF-L VRF-

RL1

bump-in-wire

VRF-L

L1

bump-in-wire w/ external link

VRF-L L3

BD-L

VRF-

R

BD-R

ip-forwarder

VRF-L L3

BD-L

ip-forwarder w/external link

Inter VN

et

inter-tenant gateway

integrated vpn gateway

VRF-L IPVP

N

RFC/Drafts: draft-ietf-bess-service-chaining

VRF-L VRF-

T

integrated ipsec external ip links

VRF-L

BD-L

fabric

inet.0

VRF-R

L3BD

-L BD-R

l2vn-linked ip-forwarder

L3

BD-L

l2vn-linked ip-forwarderw/external link

external gateway

L3

VRF-L VRF-

RL1

At head, tail or middle of chain

VRF-L

L1

At end of chain with external link

DL DR

L1

External DeviceVRF-L

IP1

IP2

IP1 ← DL

DR → IP2

L3VN to left SF or Tenant

Service Chains -- Bump-in-Wire Service Function

IP adjacency throughbump-in-wire

L1

VRF-L VRF-RIP1 IP2

DR → IP2 IP1 ← DL

L3VN to left SF or Tenant

L3VN to right SF or TenantDL DR


Service Chains -- IP Routing Service Function

VR

F-L L3

BD

-L

VR

F-R

BD

-R

At head or middle of chain

VR

F-L L3

BD

-L

At end of chain with external link

VRF-LExternal Device

IP1

IP4

IP2 IP3

IP3 ← DL

DR → IP2

L3

DLDRL3VN to left SF or

Tenants

IP adjacency with ip-forwarder

VRF-L VRF-RIP1 IP4

IP2 IP3

DR → IP2DL

IP3 ← DL

L3

L3VN to left SF or Tenants

L3VN to right SF or Tenants

DR


Service Chains – Service Function Scaling

VRF-L

L3active VR

F-R

VRF-

L VRF-R

L3active

IP1 IP4

IP2 IP3



DL DR

VRF-L

L3standby VR

F-R

L3active

VRF-

L

IP6 IP7

IP5 IP8

VRF-R

●●

●●

●

●●

●●

●

VRF-R

Service Chains – Active/Standby Redundancy

VRF-L

L3active VR

F-R

VRF-L

L3standby VR

F-R

BD-L BD

-R

BD-L BD

-R

VRF

-L

VRF-R

L3standby

VRF

-L

L3activeIP2 IP3

IP6 IP7



BD-L

BD-R

BD-L

L2VN-RL2VN-L

VIP announcement over L2VN using GARP

VIPL VIPR

D R→

VIP

L

VIPR

← D

L

D R→

VIP

L

VIPR

← D

L

BD-R

DL DR

Service Chains – Multicast (L2 Linked Chains)

L3active

L3standby

BD-L BD-R

BD-L BD-R

L3standby

L3activeIP2 IP3

IP6 IP7



In-band PIM DR and VIP election

over L2VN

VIPL VIPR

BD-L

BD-R

BD-L BD-R

DL DRBD

-L

BD-R

Service Chains -- Multiple Chains

VRF-L2BD-L2-1

BD-L2-2

Tenant-L2

VRF-L VRF-

RDLPL1

VRF-L1BD-L1-1

BD-L1-2

Tenant-L1

VRF-L VRF-

RFWL3

BD-L BD-R

VRF-L VRF-

RDLPL1

VRF-L VRF-

RFWL3

BD-L BD-R

VRF-

R2BD

-R2-

1BD

-R2-

2

Tenant-R2

VRF-

R1BD

-R1-

1BD

-R1-

2

Tenant-R1

Service Chain 1

SC1-Instance1

SC1-Instance2

Inter VNet

Inter VNet

Service Chain 2

Inte

r VN

etIn

ter

VNet

Service Chain 3

Not all connectors in a parallel service chain

may be active


Service Chains -- Multiple Chains

VRF-L2BD

-L2-1BD

-L2-2

Tenant-L2

VRF-L VRF-

RDLPL1

VRF-L1BD

-L1-1BD

-L1-2

Tenant-L1

VRF-L VRF-

RFWL3

BD-L BD-R

VRF-L VRF-

RDLPL1

VRF-L VRF-

RFWL3

BD-L BD-R

VRF-

R2BD

-R2-

1BD

-R2-

2

Tenant-R2

VRF-

R1BD

-R1-

1BD

-R1-

2

Tenant-R1

Service Chain 1

SC1-Instance1

SC1-Instance2

Inter VNet

Inter VNet

Service Chain 2

Inte

r VN

etIn

ter

VNet

Service Chain 3

Not all connectors in a parallel service chain

may be active


Service Chains -- Branching ChainsVNet-L3

VRF-L4

BD

-L4-1B

D-L4-2

Tenant-L4

VRF-L3

BD

-L3-1B

D-L3-2

VRF-L VRF-

RLBL3

VRF-L VRF-

RLBL3

Service Chain 4

VRF-L5

BD

-L5-1B

D-L5-2

Tenant-L5

VRF-L VRF-

T

VRF-L VRF-

T

VRF-L VRF-

RFWL1

VRF-L VRF-

RFWL1

VRF-LVRF-L

Service Chain 5

Service Chain 6

ExternalNetwork

Only Service VIP is visible to external. Can be learned using BGP. Tenant address is not visible.

Tenant-L3


Service Chains -- Dependent Chains (IP/EVPN Transport)

VRF-L7

BD

-L7-1B

D-L7-2

Tenant-L7

VRF-L6

BD

-L6-1B

D-L6-2

Tenant-L6

VR

F-L VR

F-RLB

L3

VR

F-L VR

F-RLB

L3

Service Chain 7 (depends on Service Chain 8)

Fabric

VRF-L VRF-

T

VRF-L VRF-

T

Service Chain 8

ExternalIPVPN

VR

F-L VR

F-RFW

L1

VR

F-L VR

F-RFW

L1

inet.0

VR

F-R

inet.0 VR

F-R

VRF-L IPVP

N

VRF-L IPVP

N


VRF-LVRF-L

ExternalTransportNetwork

External Gateways(i.e. N-Way IP forwarders)

External Gateway

VRF-L9

BD

-L9-1B

D-L9-2

Tenant-L9

VRF-L8

BD

-L8-1B

D-L8-2

Tenant-L8


BD

-L9-FW1

BD

-L8-FW1

FW1aL3

FW1bL3

External Gateway connected to a service chain using a transit overlay

VRF-L V

RF-T

VRF-L V

RF-T

VRF-L

VRF-L

Service Chain 6

VRF-T

Transit

BD

-T-FW1

ExternalNetwork

External Gateway interfaces are members of tenant overlays

External Gateway is L3

ERB

ERB

ERB

L2 Linked Service Chain for Multicast Support

Service Chain BD extended to External Gateway

Service Chain 9

ExternalNetwork

VRF-L9BD

-L9-1BD

-L9-2

Tenant-L9

VRF-L8BD

-L8-1BD

-L8-2

Tenant-L8

BD-L8-FW

1

FW1aL3

FW1bL3

Transit

BD-L

BD-L9-FW

1

BD-L BD-RL3

BD-L BD-RL3

VRF-

R9BD

-R9-

1BD

-R9-

2

Tenant-R9

VRF-

R8BD

-R8-

1BD

- R8-

2

Tenant-R8

BD-R

8-FW

1

FW2aL3

FW2bL3

Transit

BD-R

BD-R

9-FW

1

Bridged

ERB

ERB

CRB

CRB

Bridged

Overlay Replication

Pure Overlay BUM Replication (i.e Not Underlay Assisted)

BD1

BD1

VTEP 1

VTEP 2

Source

Receivers

BD1

VTEP 3

Receivers

“Stateless” IP Core

• Overlay replication uses “over-the-top” signaling

• No hop-by-hop per-flow or per-group multicast

signaling or BUM state in underlay

• No traditional underlay multicast protocols

translates to lean core network design

• Multicast convergence “same as” unicast

convergence on transit link or node failure

Pure Overlay Efficient Replication Capabilities in EVPN

Selective Multicast Replication

VLAN1 VLAN1

IP Multicast

IP Multicast

VLAN1

IP Multicast

VTEP 1 VTEP 2 VTEP 3

Source Receivers

VLAN1

VTEP 4

MRouterNo Receivers

Selective Replication

EVPN SMET (*,G) Advertise

EVPN SMET (*,*) Advertise

SMET

Report1 PIM

Hello

3 2

VLAN1

Leave

VTEP 1

VLAN1

VTEP 2

Receiver

VLAN1

EVPN Join Sync

VTEP 1

VLAN1

VTEP 2

Receiver

DF DF

Join

Report

EVPN Leave SyncEVPN SMET Advertise

EVPN SMET Withdraw

JOIN SYNC LEAVE SYNC

Withdraw Join Sync

IGMPLMQ1 1

3

3

5

2 24

• Ensures IP multicast flow is replicated by an ingress VTEP only to egress VTEP that have at least one active receiver for that flow

• Optimizes replication load on ingress edge and also prevents consuming bandwidth at an egress edge where there is no active receivers

• Uses EVPN Type-6 SMET route

• Consumes more state – use policy to control which groups can participate in SMET

• JOIN and LEAVE SYNC ensures that multicast is only forwarded to the local receivers that requested it via IGMP

• Required to support multihomed end-systems since IGMP PDUs sent by end-system may be hashed to non-DF. Ensures DF installs appropriate forwarding state.

• Uses EVPN Type-7 Join Sync and Type-8 Leave Sync routes

RFC/Drafts: draft-sajassi-bess-evpn-igmp-mld-proxy

Optimized Overlay Replication (continued)Optimized Inter-subnet Multicast Replication (OISM)

NVE

VRF1 VRF1S-BD S-BD

Assisted Replicators VLAN1 VLAN2 VLAN1 VLAN2

Assisted BUM Replication (AR)

• OISM ensures that, for any tenant, only a single copy of an IP multicast packet is delivered to an egress VTEP, regardless of the number of subnets of the tenant at that egress VTEP with active receivers

• Works only with ERB

• Introduces distributed DR and S-BD

• New procedures, but no new route types

• Assisted replication reduces the replication load on the ingress node using designated VNI-aware replicators

• Can load-balance across replicators in a replicator set

• Significantly reduces flood-next hop state at Leaf VTEP

• New procedures, new PMSI tunnel flags, no new route types• Together with Selective Replication and OISM, Assisted Replication

brings highly efficient replication without any need for hop-by-hop replication state

RFC/Drafts: draft-lin-bess-evpn-irb-mcast, draft-ietf-bess-evpn-optimized-ir

VRF1

BD1

BD2

VRF1

BD1

BD2

S-BD

S-BD

VRF1

SRC

RCV

RCV

S-BD

BD2

RCV

RCV

VTEP1

VTEP2

VTEP3

Replicates to S-BD if Source BD is absent

IP Multicast Options in Overlay Service Models

IP Multicast Routing with External Multicast-only Routers

• Operators who do not want to support IP multicast

routing within the overlay network can delegate

multicast routing to external multicast routers

• Should use incongruent multicast with MVPN based

external multicast routers (such as MX) where

unicast and multicast would follow different paths

• Inter-subnet multicast hairpins at external multicast

routers where it is replicated into each subnet that

has receivers

• Works with both Central and Edge Routed models

• The replication heavy-lifting is performed in the overlay. Ingress leaf perform replication to egress

leaf. Egress leaf performs per-end-system replication

• Can be optimized with selective replication, and

further optimized with assisted replication when

available

RFC/Drafts: draft-sajassi-bess-evpn-igmp-mld-proxy, draft-ietf-bess-evpn-optimized-ir

BD1 BD2

NVE

VRF1 VRF1

BD2

External Multicast Routers

MR1 MR2

BD1 BD2 BD1 BD2

MRTMRT

BD1 BD2BD1 BD2

NVE

VRF1 VRF1

IP Multicast in CRB Overlay

• Classical model with PIM DR election at central gateway. Additional unique addresses are required for at gateways for PIM protocol signaling

• Inter-subnet multicast hairpins at a CRB gateway where it is replicated into each subnet that has receivers

• Can be optimized with selective replication, and further optimized with assisted replication

RFC/Drafts: draft-sajassi-bess-evpn-igmp-mld-proxy, draft-ietf-bess-evpn-optimized-ir

VRF1 VRF1BD1 BD2 BD2CRB Border

GatewayCRBAccess

BD1 BD2BD1 BD2

Multicast routing at CRB gateways with classical PIM DR election

IP Multicast in ERB Overlay (OISM)

• Introduces distributed DR and “Supplemental BD”.

• All ERB anycast gateways act as local DRs and maintain IGMP

state for local receivers across all its local subnets

• Ingress VTEP replicates to egress VTEP only over source

subnet or S-BD (if egress VTEP does not have source subnet)

• IP multicast received over the source subnet is forwarded at

each ERB gateway to local receivers across all local subnets

RFC/Drafts: draft-lin-bess-evpn-irb-mcast, draft-ietf-bess-evpn-optimized-ir

ERB w/ SBDVRF1 VRF1

VRF1 VRF1

ERBBorder

Gateway

S-BD S-BD

S-BD S-BD

Multicast with

external sources and

receivers via border

gateway

BD2BD1 BD2

• Egress ERB gateways never re/forward IP multicast across

core (i.e. into tunnels)

• A Supplemental BD is the one VLAN that must be present at

all ERB VRF for a tenant. If a source subnet is not present at

an egress VTEP, the ingress VTEP replicates to that VTEP on

the S-BD VNI.

• Optimized with selective replication, and further optimized

with assisted replication

VRF1

SRC

VRF1 VRF1

RCV RCV RCV

SBD

Leaf1 Leaf2 Leaf3

RCV RCVRCV

DR DR DR

ERB with CRB Border Gateway

• Short-term solution for lack of native

multicast support in ERB (i.e. OISM).

• Add bridging to Border Gateway

• East-west unicast is edge-routed

• North-south and east-west IP multicast

forwarded at CRB Border Gateway

• More complex options possible where CRB

gateway is not coupled with Border

Gateway.Multicast routing at central gateways with classical PIM DR election

CRB BorderGateway BD1 BD2 ERBVRF1VRF1

BD1BD2BD1BD2

BD2

VRF1VRF1

MAC-VRF-T

VLAN1 VLAN2

Border


MAC-VRF-TMAC-VRF-T

ERB with CRB Border Gateway

Leaf1 Leaf2← Host IP →

Default →IP EVPN

VLAN1

VRF-T VRF-T

VLAN2

L2 EVPN

VLAN1VLAN2

H2 H3H1 H4

← D

efau

lt



Local

Type-5 IP Host

Type-5 IP Prefix

← Aggregates VRF-G

← MAC/IP →

← SMET → ← SMET →

Type-3 IMET, Type-6 SMETMulticast

RECAP

• EVPN overlay types for intra-tenant east-west networking

• Service chain concepts for extra-tenant north-south networking using EVPN VXLAN

• Optimized replication options for different overlay service models in EVPN VXLAN

• EVPN based networks are only as complex as they need to be

• Most use cases can be satisfied with only a few key building blocks

• Complexity is proportional to the functionality required

• EVPN VXLAN is an open standard. Equivalent proprietary technology is not any

simpler.

78

The End

Isaac Building Blocks In v2 - NANOG...Agenda •Quick recap of EVPN fundamentals (5 slides) •EVPN...

Documents

Transcript of Isaac Building Blocks In v2 - NANOG...Agenda •Quick recap of EVPN fundamentals (5 slides) •EVPN...