Dell EMC Ready Solution for VMware vCloud NFV 3.0 ... · Leaf1+Leaf2 . 13 Dell EMC Ready Solution...

Dell EMC Ready Solution for VMware vCloud NFV 3.0 OpenStack Edition Platform

Deployment Manual Hardware Guide for VMware NFV 3.0 with VMware Integrated OpenStack 5.0 with Kubernetes

Dell Engineering February 2019

2 Dell EMC Ready Solution for VMware vCloud NFV 3.0 OpenStack Edition Platform | Hardware Deployment and

User Guide

Revisions

Date Description

October 2018 Initial release

February 2019 Updated

The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this

publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any software that is described in this publication requires an applicable software license.

Copyright © 2018 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other

trademarks may be the property of their respective owners. Published in the USA

Dell believes that the information in this document is accurate as of its publication date. The information is subject to change without notice.

3 Dell EMC Ready Solution for VMware vCloud NFV 3.0 OpenStack Edition Platform

Table of contents

Revisions............................................................................................................................................................................. 2

Overview ............................................................................................................................................................................. 4

1 Hardware set up for 14G .............................................................................................................................................. 5

1.1 Unpack and install equipment ............................................................................................................................ 5

1.2 Power on equipment ........................................................................................................................................... 5

1.3 Tested BIOS and firmware ................................................................................................................................. 5

2 Supported configuration ............................................................................................................................................... 7

2.1 VMware vCloud NFV inventory .......................................................................................................................... 7

3 Network topology .......................................................................................................................................................... 8

3.1 Verify network connectivity to server ports ......................................................................................................... 9

3.2 VDS DvPort group mapping with VLAN ID and related ESXi VMNIC .............................................................. 10

4 Switch configurations ................................................................................................................................................. 14

4.1 Dell EMC Networking switch default solution values ........................................................................................ 14

4.2 Network configuration overview ........................................................................................................................ 14

4.3 Switch configuration for ToR ............................................................................................................................ 15

4.4 Port channel configuration ................................................................................................................................ 20

4.5 LACP configuration on ToR .............................................................................................................................. 20

4.6 BGP configuration on ToR ................................................................................................................................ 22

4.7 Show running-config on TOR ........................................................................................................................... 23

4.8 Switch configuration on Z9100-ON leafs .......................................................................................................... 67

4.9 Show running-config on Leafs .......................................................................................................................... 67

4.10 Third-party network hardware ........................................................................................................................... 94

5 Solution archive .......................................................................................................................................................... 95

5.1 List of components ........................................................................................................................................... 97

A Reference documentation .......................................................................................................................................... 98


Overview

This guide provides information about the hardware that is recommended for the deployment of the Dell

VMware NFV platform. The Dell VMware NFV platform is designed to consolidate and deliver the networking

components that support a fully virtualized infrastructure. The components include virtual servers, storage, and

or other networks. It uses standard IT virtualization technologies that run on high-volume service, switch, and

storage hardware to virtualize network functions.

The information in this guide is intended for use by system administrators who are responsible for the

installation, configuration, and maintenance of Dell EMC 14G technology along with the suite of VMware

applications.


1 Hardware set up for 14G

1.1 Unpack and install equipment After performing all standard industry safety precautions, proceed with the following steps:

1. Unpack and install the racks.

2. Unpack and install the server hardware.

3. Unpack and install the switch hardware.

4. Unpack and install the network cabling.

5. Connect each individual machine to both power bus installations.

6. Apply power to the racks.

Note: The Dell EMC EDT team usually performs these steps.

1.2 Power on equipment

Note: The Dell EMC EDT team usually performs these steps.

To test the installation of the equipment, perform the following steps:

1. Power on each server node individually.

2. Wait for the internal system diagnostic procedures to complete.

3. Power up the network switches.

4. Wait for the internal system diagnostic procedures to complete on each of the switches.

1.3 Tested BIOS and firmware

CAUTION: Ensure that the firmware on all servers, storage devices, and switches is up-to-date as outdated

firmware may cause unexpected results to occur.

The server BIOS and firmware versions that are tested for the Dell EMC Ready Bundle for NFV platform are as

follows:

Dell EMC PowerEdge R640/R740 and R740xd tested BIOS and firmware versions

Product Firmware version

BIOS 1.4.8

iDRAC with Lifecycle Controller 3.21.23.22

rNDC - Intel(R) 4P X550-t - 10 Gbps 18.5.18

PCIe - Intel(R) 25 GbE 2P XXV710 18.5.17

(Optional) PCIe - Intel(R) 10 GbE 2P X710 18.5.17


Product Firmware version

(Optional) PCIe - QLogic 25GE 2P QL41262 14.07.07

HBA330 ADP/Mini storage Controller (Dell EMC PowerEdge R740xd/R640)

15.17.09.06

Backplane/Storage Enclosures (Dell EMC PowerEdge R640/R740)

4.26

Backplane/Storage Enclosures (Dell EMC PowerEdge R740xd) 2.25

The firmware switch versions that are tested for the Dell EMC Ready Bundle for NFV platform are as follows:

Dell Networking tested BIOS and firmware versions

Product Version

S4048T-ON firmware (1) out of band management Dell OS 9.14

Z9100-ON firmware (2) leaf-switch SWL-OS-BCF-4.7.0(0)

S6010-ON firmware (2) leaf-switch SWL-OS-BCF-4.7.0(0)


2 Supported configuration

2.1 VMware vCloud NFV inventory VMware vCloud NFV product inventory list

Product Version

ESXi 6.7.0

VMware vCenter Server 6.7.0

VMware NSX –T 2.2.0

VMware vSAN 6.7.0

VMware vRealize Log Insight 4.6.1

VMware vRealize Operations Manager 6.7.0

VIO with Kubernetes 5.0


3 Network topology Error! Reference source not found. shows the network topology that is used in this guide on Deployment

server:

ToR Switch= top of the rack (S4048T-ON)

Leaf Switch= Z9100-ON or S6010-ON

ToR switch using 10G interface for Big Cloud Fabric (p-switch management network) – vmnic0

ToR switch using 10G interface for the ESXI management network – vmnic1

ToR switch using 10G interface for the OOB management network – vmnic2

ToR switch using 10G interface for the customer management private network – vmnic3 (stamp)

In-band 1 using 10G interface – vmnic4




Leaf1 switch using 10G/25G interface for physical switch network

Leaf2 switch using 10G/25G interface for physical switch network

Deployment server network connections

The following diagram depicts the Dell EMC PowerEdge 14G nodes network topology:


Note: Big Cloud Fabric (BCF) is on top of the leaf switches running SWL-OS-BCF-4.7.0(0)

Dell EMC PowerEdge 14G nodes network topology

3.1 Verify network connectivity to server ports To ensure the network connectivity to server ports, use the information that is provided in Error! Reference

source not found.. The information that is provided ensures that the network cables are connected correctly

to the servers. The table also provides port-mapping information for the VMware VMNIC port references. The

installation process requires that the PCIe expansion card slot (riser 1) is used for network connectivity.

Note: For more information, see Appendix A to download the appropriate Dell EMC PowerEdge Owner’s

Manual and reference the Expansion card installation section.

The configuration process requires that the NIC ports that are integrated on the network adapter card, or NDC,

are connected as outlined in Error! Reference source not found.4.

Note: For more information, see Appendix A to download the appropriate Dell EMC PowerEdge Owner’s

Manual and reference the Technical specifications section.


Network connectivity configuration table of Management ESXi host

LOM/NDC ports NICs Slot 1 NICs Slot 2 NICs Slot 3

Port number 1 2 3 4 1 2 1 2 1 2

VMware VMNIC port reference

vmnic0 vmnic1 vmnic2 vmnic3 vmnic4 vmnic5 vmnic6 vmnic7 Vmnic8 Vmnic9

Servers

R640/R740 10G 10G 10G 10G 10/25G - 10/25G - - -

Network connectivity configuration table of Edge and Resource ESXi host

LOM/NDC ports NICs Slot 1 NICs Slot 3 NICs Slot 4

Port number 1 2 3 4 1 2 1 2 1 2


vmnic0 vmnic1 vmnic2 vmnic3 vmnic4 vmnic5 vmnic6 vmnic7 Vmnic8 Vmnic9

Servers

R740xd 10G 10G 10G 10G 10G/25G 10G/25G 10G/25G 10G/25G 10G/25G 10G/25G

Network connectivity configuration table of Deployment Server

LOM/NDC ports NIC Slot 1 NIC Slot 2

Port number 1 2 3 4 1 2 1 2


vmnic0 vmnic1 vmnic2 vmnic3 vmnic4 vmnic5 vmnic6 vmnic7

Servers

R640/R740 10G 10G 10G 10G 10G 10G 10G 10G

3.2 VDS DvPort group mapping with VLAN ID and related ESXi VMNIC The mapping list provides details about all VSS, VDS, VDS DvPort groups, VLAN ID and ESXi VMNICs. These

details are created and configured under management and resource pod networking.


Table 7, Table 8, and Table 9 show the VDS-DvPort group/VSS port group mappings with VLAN ID and

corresponding VMNIC present on ESXi, which are assigned as uplinks to the VDS/VSS.

For example, the VDS named ESXi Management VDS with the DvPort Group ESXi Management Network, is

configured with VLAN ID 200 and uses a pair of VMNIC which is vmnic0 and vmnic1 as uplinks for the VDS.


Management POD

VDS type VDS name Port groups VLAN ID Uplink NICs Switch

VDS (Infrastructure)

ESXi Management VDS

ESXi Management Network

200 vmnic-0 vmnic-1

ToR

Infrastructure Management VDS

vSAN Network 300 vmnic-4 vmnic-6

Leaf1+Leaf2

vMotion Network 400

Virtual Machine Network (VDS)

Management Network VDS

Management Network 176 vmnic-2 vmnic-3

ToR

API Access Network 10

vCSA HA Network 20

External Network 30

Resource POD

VDS Type VDS Name Port groups VLAN ID Uplink NICs Switch

VDS (Infrastructure)

ESXi Management VDS

ESXi Management Network

200 vmnic-0 vmnic-1

ToR

Infrastructure Management VDS

Edge Overlay Network 700 vmnic-4 vmnic-6

Leaf1+Leaf2

vSAN Network 300

vMotion Network 400

VDS (Virtual Machine Network)

Management Network VDS

Management Network 176 vmnic-2 vmnic-3

ToR


N-VDS(S) Overlay Network 700 vmnic-5 vmnic-7

Leaf1+Leaf2


NSXT External Network

Enhanced Data Path Network

VLAN N vmnic-8 vmnic-9

Leaf1+Leaf2


Deployment Server

VSS name Port groups VLAN ID Uplink NICs Switch

BCF BCF [p-switch network] 110 vmnic-0 ToR

esxi_mgmt_200 esxi_mgmt_200 200 vmnic-1 ToR

vSwitch0 VM Network 0 vmnic-2 ToR

Stamp176 Management Network 176 vmnic-3 ToR

vSwitch1 InbandPort1 4095 vmnic-4 Leaf1





4 Switch configurations

Note: The provided configurations is based on Dell EMC validated solution as an example, user can provide a

planning sheet for switch configurations and network design.

4.1 Dell EMC Networking switch default solution values Switch port defaults

CLI command Recommended default Example

IP address No IP address assigned no IP address

portmode Hybrid mode portmode Hybrid

switchport Enabled after portmode hybrid is implemented switchport

S4048T-ON MTU 9216 mtu 9216

Z9100-ON MTU 9216 BCF Controller

flowcontrol Flow control receive is On with transmit Off flowcontrol rx is On with tx Off

4.2 Network configuration overview The following are example VLAN assignments:

VLAN assignments

VLAN name Recommended VLAN

ESXi Management Network 200

vSAN 300

vMotion 400

Management Network (for VM) 176

API Access Network 10

VCSA HA Network 20

External Network connectivity 30

Overlay Network 700

Enhanced Data Path Network VLAN (N)

iDRAC (OOB) 100

On the switch configuration pages, the following listings display:

Name – Port name that is assigned to the switch

Connector – Type of connector used


Device Name – Name of the server or device that is connected to the switch

Note: When creating a device name, use a name that describes the server usage and associated bond.

Port – Network interface card, or NIC name and port number as the operating system would

generate

Untagged and Tagged - Used to indicate VLAN ports used

Port channel number – Unique number on the switch that represents the port channel

Note: Each pair in a server bond must have their own unique port channel number, which is the same on both

switches.

Mode - The mode port channel uses 802.3ad for all nodes

4.3 Switch configuration for ToR

NOTE: ToR configuration is covered for VMNIC ports 0, 1, 2, and 3 that connect to each ESXi host of the

management and resource pod.

The switch configuration includes VLAN, MTU, and IP address configuration. Definitions of VLANs are

provided in Table 12.

VLANs on ToR

VLAN name Recommended VLAN

ESXi Management Network 200

Management Network for VM’s 176

VIO API Access Network 10

vCenter Server HA Network 20

External Network connectivity 30

BCF [p-switch network] 110

iDRAC network 100

1. For VLAN configuration on the ToR switch, log in to the switch using the required credentials and enter

the following commands:

Note: Each VLAN interface that is connected is configured using MTU-9216.

ToR Switch - S4048T- ON

DellEMC>enable

DellEMC#configure


DellEMC(conf)#hostname S4048-176

S4048-176#show version

Dell EMC Real Time Operating System Software

Dell EMC Operating System Version: 2.0

Dell EMC Application Software Version: 9.14(0.1)

Copyright (c) 1999-2018 by Dell Inc. All Rights Reserved.

Build Time: Fri Aug 10 23:10:14 2018

Build Path: /build/build02/SW/SRC

Dell EMC Networking OS uptime is 4 day(s), 12 hour(s), 4 minute(s)

S4048-176#enable

S4048-176#configure

S4048-176(conf)#interface range tengigabitethernet 1/1-1/48

S4048-176(conf-if-range-te-1/1-1/48)#switchport

S4048-176(conf-if-range-te-1/1-1/48)#mtu 9216

S4048-176(conf-if-range-te-1/1-1/48)#no shutdown

S4048-176(conf-if-range-te-1/1-1/48)#exit

S4048-176(conf)#interface vlan 200

S4048-176(conf-if-vl-200)#description ESXi Management Network

S4048-176(conf-if-vl-200)#ip address 192.168.200.254/24

S4048-176(conf-if-vl-200)#mtu 9216

S4048-176(conf-if-vl-200)#no shutdown

S4048-176(conf-if-vl-200)#Aug 31 11:35:47 %STKUNIT1-M:CP %IFMGR-5-

ASTATE_UP: Changed interface Admin state to up: Vl 200

S4048-176(conf-if-vl-200)#tagged tengigabitethernet 1/11-1/26, 1/28

S4048-176(conf-if-vl-200)#show config

!

interface Vlan 200

description ESXi Management Network

ip address 192.168.200.254/24

mtu 9216

tagged TenGigabitEthernet 1/11-1/26, 1/28

no shutdown

S4048-176(conf-if-vl-200)#exit

1. Using the same procedure, configure VLAN 176 10, 20, and 30 on the ToR switch.



!

interface Vlan 176

description Management Network for VM's

ip address 192.168.176.254/24

mtu 9216

tagged TenGigabitEthernet 1/27, 1/29-1/44

no shutdown




!

interface Vlan 10

description VIO API Access Network

ip address 192.168.10.254/24

mtu 9216

tagged TenGigabitEthernet 1/27,1/29-1/44

no shutdown


!

interface Vlan 20

description vCenter Server HA Network

ip address 192.168.20.254/24

mtu 9216


no shutdown


!

interface Vlan 30

description External Network connectivity

ip address 192.168.30.254/24

mtu 9216


no shutdown


!

interface Vlan 100

description idrac network

no ip address

mtu 9216

untagged TenGigabitEthernet 1/1-1/10

no shutdown


!

interface Vlan 110

description BCF{p-switch network}

no ip address

mtu 9216

untagged TenGigabitEthernet 1/46-1/48

no shutdown


Add static route:

S4048-176# Config

S4048-176(conf)# ip route 172.16.61.1/32 192.168.30.2

S4048-176#show vlan brief

VLAN Name STG MAC Aging IP Address

---- -------------------------------- ---- --------- ------------------

1 0 1800 unassigned

10 0 1800 192.168.10.254/24

20 0 1800 192.168.20.254/24

30 0 1800 192.168.30.254/24

100 0 1800 unassigned

110 0 1800 unassigned

176 0 1800 192.168.176.254/24

200 0 1800 192.168.200.254/24

S4048-176#show lldp neighbors

Loc PortID Rem Host Name Rem Port Id Rem Chassis Id

------------------------------------------------------------------

Te 1/9 Rack176-5524 gi1/0/5 5c:26:0a:d9:b5:5a

Fo 1/49 Leaf1 ethernet30 34:17:eb:46:a8:80

Fo 1/51 Leaf2 ethernet30 34:17:eb:46:cc:80

Ma 1/1 Rack176-5524 gi1/0/10 5c:26:0a:d9:b5:5a

S4048-176#show vlan

Codes: * - Default VLAN, G - GVRP VLANs, R - Remote Port Mirroring VLANs, P

- Primary, C - Community, I - Isolated

O - Openflow, Vx - Vxlan

Q: U - Untagged, T - Tagged

x - Dot1x untagged, X - Dot1x tagged

o - OpenFlow untagged, O - OpenFlow tagged

G - GVRP tagged, M - Vlan-stack

i - Internal untagged, I - Internal tagged, v - VLT untagged, V - VLT

tagged

NUM Status Description Q Ports

* 1 Active U Te 1/45

10 Active VIO API Acess Network T Te 1/27,1/29-1/44

20 Active vCenter Server HA Network T Te 1/27,1/29-1/44

30 Active External Network connectivity T Te 1/27,1/29-1/44

100 Active idrac network U Te 1/1-1/10

110 Active BCF{p-switch network} U Te 1/46-1/48

176 Active Management Network for VM's T Te 1/27,1/29-1/44

200 Active ESXi Management Network T Te 1/18-1/34


Note: Connect NICs vmnic0 and vmnic1, from each ESXi host to ToR for ESXi management connection.

Similarly, connect NICs vmnic2 and vmnic3, from each ESXi host to ToR for management VM connection.

S4048-176#show interfaces status

Port Description Status Speed Duplex Vlan

Te 1/1 MgmtEsxi13- Up 1000 Mbit Full 100

Te 1/2 Mgmtesxi12- Up 1000 Mbit Full 100



Te 1/5 ResEsxi9-iD Up 1000 Mbit Full 100




Te 1/9 Uplink to b Up 1000 Mbit Full 100

Te 1/10 Deployment Up 10000 Mbit Full 100

Te 1/11 MgmtESXi13- Up 10000 Mbit Full 200




Te 1/15 ResESXi9-vm Up 10000 Mbit Full 200












Te 1/27 deployment- Up 10000 Mbit Full 10,20,30,176

Te 1/28 deployment- Up 10000 Mbit Full 200

Te 1/29 MgmtESXi13- Up 10000 Mbit Full 10,20,30,176




Te 1/33 resESXi9-vm Up 10000 Mbit Full 10,20,30,176





Te 1/38 MgmtEsxi12- Up 10000 Mbit Full 10,20,30,176



Te 1/41 ResEsxi9-vm Up 10000 Mbit Full 10,20,30,176


Te 1/42 ResESXi8-vm Up 10000 Mbit Full 10,20,30,176



Te 1/45 Down Auto Auto 1

Te 1/46 Up 10000 Mbit Full 110



Fo 1/49 Up 40000 Mbit Full --

Fo 1/50 Down 40000 Mbit Auto --

Fo 1/51 Up 40000 Mbit Full --




4.4 Port channel configuration

S4048-176# Configure

S4048-176(conf)#interface port-channel 10

S4048-176(conf-if-po-10)#portmode hybrid

S4048-176(conf-if-po-10)#switchport

S4048-176(conf-if-po-10)#mtu 9126

S4048-176(conf-if-po-10)#no shutdown

S4048-176(conf-if-po-10)#show config

!

interface Port-channel 10

no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

S4048-176(conf-if-po-10)#exit

4.5 LACP configuration on ToR

S4048-176(conf)#interface fortyGigE 1/49

S4048-176(conf-if-fo-1/49)#port-channel-protocol lacp

S4048-176(conf-if-fo-1/49-lacp)#port-channel 10 mode active

S4048-176(conf-if-fo-1/49-lacp)#show config

!

port-channel-protocol LACP

port-channel 10 mode active

S4048-176(conf-if-fo-1/49-lacp)#exit

S4048-176(conf-if-fo-1/49)#mtu 9216

S4048-176(conf-if-fo-1/49)#no shutdown


S4048-176(conf-if-fo-1/49)#Nov 22 20:35:43 %STKUNIT1-M:CP %IFMGR-5-

ASTATE_UP: Changed interface Admin state to up: Fo 1/49

S4048-176(conf-if-fo-1/49)#show config

!

interface fortyGigE 1/49

no ip address

mtu 9216

!



no shutdown

S4048-176(conf-if-fo-1/49)#exit

S4048-176(conf)#interface fortyGigE 1/51

S4048-176(conf-if-fo-1/51)#port-channel-protocol lacp

S4048-176(conf-if-fo-1/51-lacp)#port-channel 10 mode active

S4048-176(conf-if-fo-1/51-lacp)#show config

!



S4048-176(conf-if-fo-1/51-lacp)#exit

S4048-176(conf-if-fo-1/51)#mtu 9216

S4048-176(conf-if-fo-1/51)#no shutdown

S4048-176 (conf-if-fo-1/51)#Nov 22 20:35:43 %STKUNIT1-M:CP %IFMGR-5-

ASTATE_UP: Changed interface Admin state to up: Fo 1/51

Nov 22 20:35:43 %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state

to up: Fo 1/51

S4048-176(conf-if-fo-1/51)#show config

!

interface fortyGigE 1/51

no ip address

mtu 9216

!



no shutdown

S4048-176#configure


S4048-176(conf)# tagged port-channel 10


!

interface Vlan 30

description External Network connectivity

ip address 192.168.30.254/24

mtu 9216



tagged Port-channel 10

no shutdownn

4.6 BGP configuration on ToR Configure BGP on the ToR switch so that NSX-T Edge can communicate with the physical switch.

Note: BGP configuration is required on NSX-T Tier-0 router. See NSX-T BGP configuration for more details.

S4048-176# configure terminal

S4048-176(config)#

S4048-176(config)#route-map nsx-t permit 50

S4048-176(config-route-map)#match ip address nsx-t

S4048-176(config-route-map)#exit

S4048-176(config)#

S4048-176(conf)#ip prefix-list nsx-t

S4048-176(conf-nprefixl)#seq 30 permit 192.168.30.0/24

S4048_176(conf-nprefixl)#seq 40 permit 192.168.176.0/24

S4048_176(conf-nprefixl)#seq 60 permit 192.168.10.0/24

S4048_176(conf-nprefixl)#exit

S4048-176(config)#

S4048-176(config)# router bgp 65001

S4048-176(conf-router_bgp)# bgp router-id 192.168.30.254

S4048-176(conf-router_bgp)# network 192.168.30.0/24

S4048-176(conf-router_bgp)# redistribute connected route-map nsx-t

S4048-176(conf-router_bgp)# neighbor 172.16.61.1 remote-as 64502

S4048-176(conf-router_bgp)# neighbor 172.16.61.1 ebgp-multihop 2

S4048-176(conf-router_bgp)# neighbor 172.16.61.1 no shutdown

S4048-176(conf-router_bgp)#show config

!

router bgp 65001

bgp router-id 192.168.30.254

network 192.168.30.0/24

redistribute connected route-map nsx-t

neighbor 172.16.61.1 remote-as 64502

neighbor 172.16.61.1 ebgp-multihop 2

neighbor 172.16.61.1 no shutdown

S4048-176(config)# exit

S4048-176#show ip bgp summary

BGP router identifier 192.168.30.254, local AS number 65001

BGP local RIB : Routes to be Added 0, Replaced 0, Withdrawn 0

3 network entrie(s) using 228 bytes of memory

3 paths using 324 bytes of memory

BGP-RIB over all using 327 bytes of memory

3 BGP path attribute entrie(s) using 384 bytes of memory


1 neighbor(s) using 8192 bytes of memory

Neighbor AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down

State/Pfx

172.16.61.1 64502 7057 8080 0 0 0 1w1d:04:46:

Connect

S4048-176#show ip route

Codes: C - connected, S - static, R - RIP,

B - BGP, IN - internal BGP, EX - external BGP,LO - Locally Originated,

O - OSPF, IA - OSPF inter area, N1 - OSPF NSSA external type 1,

N2 - OSPF NSSA external type 2, E1 - OSPF external type 1,

E2 - OSPF external type 2, i - IS-IS, L1 - IS-IS level-1,

L2 - IS-IS level-2, IA - IS-IS inter area, * - candidate default,

> - non-active route, + - summary route

Gateway of last resort is not set

Destination Gateway Dist/Metric Last Change

----------- ------- ----------- -----------

S 172.16.61.1/32 via 192.168.30.2, Vl 30 1/0 1w6d

C 192.168.10.0/24 Direct, Vl 10 0/0 1w6d

C 192.168.20.0/24 Direct, Vl 20 0/0 1w6d

C 192.168.30.0/24 Direct, Vl 30 0/0 1w6d

C 192.168.176.0/24 Direct, Vl 176 0/0 1w6d

C 192.168.200.0/24 Direct, Vl 200 0/0 1w6d

4.7 Show running-config on TOR The following is an example of a complete switch configuration for the previous setup as a reference.

Run the configuration command as shown in the following example:

bcf01>

bcf01> enable

bcf01# show running-config

!

! Big Cloud Fabric 4.7.0 (bcf-4.7.0 #186)

! Current Time: 2019-02-01 02:38:38 UTC

!


version 1.0

! ntp

ntp server 192.168.176.249

! logging

logging remote vrli123.dellnfv.com


! aaa

aaa accounting exec default start-stop local

! local

local node

hostname bcf01

interface management

!

ipv4

ip 192.168.176.90/24 gateway 192.168.176.254

method manual

dns server 192.168.176.250

dns search dellnfv.com

!

ipv6

method manual


! user

user admin

full-name 'Default admin'

hashed-password method=PBKDF2WithHmacSHA512,salt=w5heJjvUBnf-

Rb10jG5ysg,rounds=25000,ph=true,mF7aAQF-so7srCuqgEy-QvpNy-OXyvx394sPbO-

J3NiW_SnAzJuY950J1Adz1jVjF7b6G9v-2tecXS0oCLuljg

! group

group admin

associate user admin

! controller

controller

cluster-name bcfcluster

virtual-ip 192.168.176.92

access-control

!

access-list api

1 permit from ::/0

2 permit from 0.0.0.0/0

!

access-list gui

1 permit from ::/0


!

access-list ns-api

1 permit from ::/0



!

access-list ssh

1 permit from ::/0


!

access-list vce-api

1 permit from ::/0


! switch

switch Leaf1

fabric-role leaf

leaf-group Rack1

mac 34:17:eb:46:a8:80

interface ethernet11:1










interface ethernet30










switch Leaf2

fabric-role leaf

leaf-group Rack1

mac 34:17:eb:46:cc:80

interface ethernet1




















! fabric

fabric

forwarding-mode high-bandwidth

!

ipam switch

dns-server 192.168.176.250

gateway 192.168.176.254

ip-range 192.168.176.93 192.168.176.95 subnet-mask-length 24

allocate

! interface-group

interface-group To-Tor

mode lacp

member switch Leaf1 interface ethernet30


backup-mode lacp

interface-group esxi10.dellnfv.com-ESXi-Management-VDS-MgmtLag1

description 'Virtual switch ESXi-Management-VDS lag MgmtLag1 in ESXi host esxi10.dellnfv.com'


mode lacp

member host esxi10.dellnfv.com interface vmnic0


origination vsphere-mgmt-VCSA

interface-group esxi10.dellnfv.com-Infra-mgmt-VDS-MgmtLag1

description 'Virtual switch Infra-mgmt-VDS lag MgmtLag1 in ESXi host esxi10.dellnfv.com'

mode lacp




interface-group esxi10.dellnfv.com-VM-mgmt-VDS-MgmtLag1

description 'Virtual switch VM-mgmt-VDS lag MgmtLag1 in ESXi host esxi10.dellnfv.com'

mode lacp






mode lacp







mode lacp






mode lacp






mode lacp






mode lacp







interface-group esxi6.dellnfv.com-ESXi-Management-VDS-ResourceLag1

description 'Virtual switch ESXi-Management-VDS lag ResourceLag1 in ESXi host esxi6.dellnfv.com'

mode lacp



origination vsphere-res-VCSA

interface-group esxi6.dellnfv.com-Infra-mgmt-VDS-vmnic4

description 'Interface group for virtual switch Infra-mgmt-VDS in ESXi host esxi6.dellnfv.com'







interface-group esxi6.dellnfv.com-NSXT-External-Network-ResourceLag1

description 'Virtual switch NSXT-External-Network lag ResourceLag1 in ESXi host esxi6.dellnfv.com'

mode lacp





interface-group esxi6.dellnfv.com-Overlay-Network-VDS-vmnic5

member switch Leaf1 interface ethernet7:1



interface-group esxi6.dellnfv.com-VM-mgmt-VDS-ResourceLag1

description 'Virtual switch VM-mgmt-VDS lag ResourceLag1 in ESXi host esxi6.dellnfv.com'

mode lacp






mode lacp















mode lacp










mode lacp







mode lacp














mode lacp











mode lacp






mode lacp















mode lacp










mode lacp




! tenant

tenant Tenant_mgmt

description auto-created

multicast-enable


origination vsphere-mgmt-vc

!

segment Tenant_mgmt-10

description ' 1 vSphere portgroups: API-Access-Network'

id Tenant_mgmt-10


!

endpoint vm-78-00-50-56-b4-14-d4

description 'VIO-LoadBalancer-0[vm-78] (Network adapter 2) on ESXi Host esxi12.dellnfv.com

[192.168.176.138, 192.168.176.132]'

mac 00:50:56:b4:14:d4


!

endpoint vm-82-00-50-56-b4-81-d0


[192.168.10.1, 192.168.10.3]'

mac 00:50:56:b4:81:d0


!

member interface-group esxi12.dellnfv.com-VM-mgmt-VDS-MgmtLag1 vlan 10

description esxi12.dellnfv.com-VM-mgmt-VDS-MgmtLag1-10


!




!



description ' 1 vSphere portgroups: VM-management-Network'

id Tenant_mgmt-176


!

endpoint vm-33-00-0c-29-26-61-e0

description 'AD-DNS-Server[vm-33] (Network adapter 1) on ESXi Host esxi10.dellnfv.com

[192.168.176.250]'

mac 00:0c:29:26:61:e0


!

endpoint vm-33-00-0c-29-3e-ea-79

description 'AD-DNS-Server[vm-33] (Network adapter 1) on ESXi Host esxi10.dellnfv.com

[192.168.176.250]'

mac 00:0c:29:3e:ea:79


!

endpoint vm-34-00-0c-29-05-4b-3a

description 'NTP-Server[vm-34] (Network adapter 1) on ESXi Host esxi10.dellnfv.com [192.168.122.1]'

mac 00:0c:29:05:4b:3a


!

endpoint vm-34-00-0c-29-10-66-d0

description 'NTP-Server[vm-34] (Network adapter 1) on ESXi Host esxi10.dellnfv.com [192.168.176.249]'

mac 00:0c:29:10:66:d0


!


endpoint vm-35-00-0c-29-16-eb-9a

description 'mgmtVcsaActive[vm-35] (Network adapter 1) on ESXi Host esxi10.dellnfv.com'

mac 00:0c:29:16:eb:9a


!

endpoint vm-35-00-0c-29-a4-83-63

description 'mgmtVcsaActive[vm-35] (Network adapter 1) on ESXi Host esxi10.dellnfv.com

[192.168.176.101]'

mac 00:0c:29:a4:83:63


!

endpoint vm-51-00-50-56-b4-51-cc

description 'resourceVcsaActive[vm-51] (Network adapter 1) on ESXi Host esxi11.dellnfv.com'

mac 00:50:56:b4:51:cc


!

endpoint vm-53-00-50-56-b4-ef-66

description 'mgmtVcsaPassive[vm-53] (Network adapter 1) on ESXi Host esxi11.dellnfv.com

[192.168.176.101]'

mac 00:50:56:b4:ef:66


!

endpoint vm-55-00-50-56-b4-e2-8d

description 'mgmtVcsaWitness[vm-55] (Network adapter 1) on ESXi Host esxi12.dellnfv.com'

mac 00:50:56:b4:e2:8d


!


endpoint vm-57-00-50-56-b4-9c-7c

description 'resourceVcsaPassive[vm-57] (Network adapter 1) on ESXi Host esxi12.dellnfv.com

[192.168.176.102]'

mac 00:50:56:b4:9c:7c


!

endpoint vm-59-00-50-56-b4-c9-a9

description 'resourceVcsaWitness[vm-59] (Network adapter 1) on ESXi Host esxi13.dellnfv.com'

mac 00:50:56:b4:c9:a9


!

endpoint vm-60-00-50-56-b4-1a-b2

description 'NsxtController1[vm-60] (Network adapter 1) on ESXi Host esxi13.dellnfv.com

[192.168.176.104]'

mac 00:50:56:b4:1a:b2


!

endpoint vm-61-00-50-56-b4-1b-f6


[192.168.176.106]'

mac 00:50:56:b4:1b:f6


!

endpoint vm-62-00-50-56-b4-0d-aa


[192.168.176.105]'

mac 00:50:56:b4:0d:aa



!

endpoint vm-63-00-50-56-b4-06-b1

description 'nsxtManager[vm-63] (Network adapter 1) on ESXi Host esxi12.dellnfv.com [192.168.176.103]'

mac 00:50:56:b4:06:b1


!

endpoint vm-64-00-50-56-b4-12-37

description 'VropsReplicaNode[vm-64] (Network adapter 1) on ESXi Host esxi12.dellnfv.com

[192.168.176.129]'

mac 00:50:56:b4:12:37


!

endpoint vm-65-00-50-56-b4-70-c9

description 'VropsMasterNode[vm-65] (Network adapter 1) on ESXi Host esxi10.dellnfv.com

[192.168.176.127]'

mac 00:50:56:b4:70:c9


!

endpoint vm-66-00-50-56-b4-a5-d7

description 'VropsDataNode[vm-66] (Network adapter 1) on ESXi Host esxi13.dellnfv.com

[192.168.176.128]'

mac 00:50:56:b4:a5:d7


!

endpoint vm-67-00-50-56-b4-38-80

description 'VrliWorker2[vm-67] (Network adapter 1) on ESXi Host esxi11.dellnfv.com [192.168.176.125]'

mac 00:50:56:b4:38:80



!

endpoint vm-68-00-50-56-b4-16-88

description 'VrliMaster[vm-68] (Network adapter 1) on ESXi Host esxi10.dellnfv.com [192.168.176.123]'

mac 00:50:56:b4:16:88


!

endpoint vm-69-00-50-56-b4-d7-a8

description 'VrliWorker1[vm-69] (Network adapter 1) on ESXi Host esxi13.dellnfv.com [192.168.176.124,

192.168.176.126]'

mac 00:50:56:b4:d7:a8


!

endpoint vm-71-00-50-56-b4-83-e0

description 'management-server[vm-71] (Network adapter 1) on ESXi Host esxi10.dellnfv.com

[192.168.176.131]'

mac 00:50:56:b4:83:e0


!

endpoint vm-77-00-50-56-b4-b9-53

description 'VIO-Controller-1[vm-77] (Network adapter 1) on ESXi Host esxi10.dellnfv.com

[192.168.176.137]'

mac 00:50:56:b4:b9:53


!

endpoint vm-78-00-50-56-b4-5c-ea


[192.168.176.138, 192.168.176.132]'

mac 00:50:56:b4:5c:ea



!

endpoint vm-79-00-50-56-b4-2a-30

description 'VIO-DB-1[vm-79] (Network adapter 1) on ESXi Host esxi12.dellnfv.com [192.168.176.134]'

mac 00:50:56:b4:2a:30


!

endpoint vm-80-00-50-56-b4-65-c1

description 'VIO-Compute-0[vm-80] (Network adapter 1) on ESXi Host esxi11.dellnfv.com

[192.168.176.140]'

mac 00:50:56:b4:65:c1


!

endpoint vm-81-00-50-56-b4-27-ec


mac 00:50:56:b4:27:ec


!

endpoint vm-82-00-50-56-b4-0e-81


[192.168.176.139]'

mac 00:50:56:b4:0e:81


!

endpoint vm-83-00-50-56-b4-39-be


mac 00:50:56:b4:39:be



!

endpoint vm-84-00-50-56-b4-13-2d

description 'VIO-Controller-0[vm-84] (Network adapter 1) on ESXi Host esxi12.dellnfv.com

[192.168.176.136]'

mac 00:50:56:b4:13:2d


!

endpoint vm-85-00-50-56-b4-8e-82

description 'VIO-Kubernetes[vm-85] (Network adapter 1) on ESXi Host esxi11.dellnfv.com

[192.168.176.152]'

mac 00:50:56:b4:8e:82


!




!




!




!





!


description ' 1 vSphere portgroups: VCSA-HA-Network'

id Tenant_mgmt-20


!

endpoint vm-35-00-50-56-b4-8c-66

description 'mgmtVcsaActive[vm-35] (Network adapter 2) on ESXi Host esxi10.dellnfv.com [192.168.20.3]'

mac 00:50:56:b4:8c:66


!

endpoint vm-51-00-50-56-b4-71-4d

description 'resourceVcsaActive[vm-51] (Network adapter 2) on ESXi Host esxi11.dellnfv.com

[192.168.20.6]'

mac 00:50:56:b4:71:4d


!

endpoint vm-53-00-50-56-b4-af-00

description 'mgmtVcsaPassive[vm-53] (Network adapter 2) on ESXi Host esxi11.dellnfv.com

[192.168.20.1]'

mac 00:50:56:b4:af:00


!

endpoint vm-55-00-50-56-b4-24-d0

description 'mgmtVcsaWitness[vm-55] (Network adapter 2) on ESXi Host esxi12.dellnfv.com

[192.168.20.2]'


mac 00:50:56:b4:24:d0


!

endpoint vm-57-00-50-56-b4-95-5c

description 'resourceVcsaPassive[vm-57] (Network adapter 2) on ESXi Host esxi12.dellnfv.com

[192.168.20.4]'

mac 00:50:56:b4:95:5c


!

endpoint vm-59-00-50-56-b4-b6-8c

description 'resourceVcsaWitness[vm-59] (Network adapter 2) on ESXi Host esxi13.dellnfv.com

[192.168.20.5]'

mac 00:50:56:b4:b6:8c


!




!




!




!





!


description ' 1 vSphere portgroups: ESXi-Management-Network'

id Tenant_mgmt-200


!

endpoint esxi10.dellnfv.com-vmk0

description 'VMKernel [192.168.200.10] with Fault Tolerance, Management traffic service'

ip 192.168.200.10

mac 24:6e:96:a4:1f:c8


!



ip 192.168.200.11

mac 24:6e:96:a4:23:90


!



ip 192.168.200.12

mac 24:6e:96:a4:23:2c



!



ip 192.168.200.13

mac 24:6e:96:a4:1b:34


!

member interface-group esxi10.dellnfv.com-ESXi-Management-VDS-MgmtLag1 vlan 200

description esxi10.dellnfv.com-ESXi-Management-VDS-MgmtLag1-200


!




!




!




!


description ' 1 vSphere portgroups: VSAN-Network'

id Tenant_mgmt-300



!


description 'VMKernel [192.200.3.10] with vSAN traffic service'

ip 192.200.3.10

mac 00:50:56:62:03:f4


!



ip 192.200.3.11

mac 00:50:56:65:4f:73


!



ip 192.200.3.12

mac 00:50:56:63:9d:03


!



ip 192.200.3.13

mac 00:50:56:60:7c:83


!


member interface-group esxi10.dellnfv.com-Infra-mgmt-VDS-MgmtLag1 vlan 300

description esxi10.dellnfv.com-Infra-mgmt-VDS-MgmtLag1-300


!




!




!




!


description ' 1 vSphere portgroups: vMotion-Network'

id Tenant_mgmt-400


!


description 'VMKernel [192.200.4.10] with vMotion traffic service'

ip 192.200.4.10

mac 00:50:56:65:04:ab



!



ip 192.200.4.11

mac 00:50:56:62:df:f1


!



ip 192.200.4.12

mac 00:50:56:62:6a:e9


!



ip 192.200.4.13

mac 00:50:56:6d:2e:e0


!




!





!




!




tenant Tenant_res


multicast-enable

origination vsphere-tenant-resvc

logical-router

!

bgp

graceful-restart

local-as 64502

log-neighbor-changes

maximum-paths 2

no logging-flag events

no logging-flag keepalives

no logging-flag updates

no redistribute-ospf

no shutdown

protocol-ip 172.16.61.1


redistribute-connected

redistribute-static

router-id 172.16.61.1

!

neighbor NSX-Tier0

connect-time 60

ebgp-ttl multihop 2

hold-time 180

neighbor-ip 172.16.60.10 remote-as 65002

soft-reconfiguration inbound

!

neighbor Tor

connect-time 60

ebgp-ttl multihop 2

hold-time 180



!

neighbor VIO-K-Tier0

connect-time 60

ebgp-ttl multihop 2

hold-time 180



interface segment Segment-To-Leaf

ip address 172.16.60.2/24


interface segment Segment-To-Tor

ip address 192.168.30.2/24

interface segment Segment-edge-overlay-700

ip address 192.200.7.254/24

interface segment Segment-vMotion-res-400

ip address 192.200.4.254/24

interface segment Segment-vSAN-res-300

ip address 192.200.3.254/24

segment Segment-To-Leaf

member interface-group esxi6.dellnfv.com-NSXT-External-Network-VDS-vmnic8-vmnic9 vlan 30




segment Segment-To-Tor

member interface-group To-Tor vlan 30

!

segment Tenant_res-176


id Tenant_res-176


!

endpoint vm-681-00-50-56-97-b1-c1

description 'NsxtEdge1[vm-681] (Network adapter 1) on ESXi Host esxi6.dellnfv.com [192.168.176.107]'

mac 00:50:56:97:b1:c1


!


endpoint vm-682-00-50-56-97-c8-7e

description 'NsxtEdge2[vm-682] (Network adapter 1) on ESXi Host esxi7.dellnfv.com [192.168.176.108]'

mac 00:50:56:97:c8:7e


!

member interface-group esxi6.dellnfv.com-NSXT-External-Network-ResourceLag1 vlan 176

description esxi6.dellnfv.com-NSXT-External-Network-ResourceLag1-176


!

member interface-group esxi6.dellnfv.com-VM-mgmt-VDS-ResourceLag1 vlan 176

description esxi6.dellnfv.com-VM-mgmt-VDS-ResourceLag1-176


!




!

member interface-group esxi7.dellnfv.com-VM-mgmt-VDS-ResourceLag1 vlan 176



!



id Tenant_res-200


!




ip 192.168.200.6

mac 24:6e:96:a4:1e:a0


!



ip 192.168.200.7

mac 24:6e:96:a4:1b:24


!



ip 192.168.200.8

mac 24:6e:96:a4:21:1c


!



ip 192.168.200.9

mac 24:6e:96:a4:1e:1c


!

member interface-group esxi6.dellnfv.com-ESXi-Management-VDS-ResourceLag1 vlan 200

description esxi6.dellnfv.com-ESXi-Management-VDS-ResourceLag1-200



!




!




!




!




!




!



id Tenant_res-300



!



ip 192.200.3.6

mac 00:50:56:68:bd:9c


!



ip 192.200.3.7

mac 00:50:56:6a:94:46


!



ip 192.200.3.8

mac 00:50:56:62:03:dd


!



ip 192.200.3.9

mac 00:50:56:64:09:63


!

member interface-group esxi6.dellnfv.com-Infra-mgmt-VDS-vmnic4 vlan 300


description esxi6.dellnfv.com-Infra-mgmt-VDS-vmnic4-300


!




!




!




!




!




!





!




!




!




!



id Tenant_res-400


!



ip 192.200.4.6

mac 00:50:56:6f:11:6c


!




ip 192.200.4.7

mac 00:50:56:6d:80:ed


!



ip 192.200.4.8

mac 00:50:56:6c:f5:e3


!



ip 192.200.4.9

mac 00:50:56:61:ac:38


!




!




!





!




!




!




!




!




!




!





!


description ' 1 vSphere portgroups: Edge-overlay-Network'

id Tenant_res-700


!

endpoint vm-681-00-50-56-97-fc-fe

description 'NsxtEdge1[vm-681] (Network adapter 2) on ESXi Host esxi6.dellnfv.com'

mac 00:50:56:97:fc:fe


!

endpoint vm-682-00-50-56-97-e4-87

description 'NsxtEdge2[vm-682] (Network adapter 2) on ESXi Host esxi7.dellnfv.com'

mac 00:50:56:97:e4:87


!




!





!




member interface-group esxi6.dellnfv.com-Overlay-Network-VDS-vmnic5 vlan 700


!




!




!















tenant test-mgmt

multicast-enable

! vcenter

vcenter mgmt-VCSA

automation-level full

exclude vlan untagged

hashed-password 2005223e1364704053

host-name vcsa101.dellnfv.com

manage-segment-for-vlan-range start-vlan untagged end-vlan 4095 tenant Tenant_mgmt

preserve-bcf-config

user-name [email protected]

vsphere-gui-plugin access-right read-write

vcenter res-VCSA



hashed-password 210a2e251573415442


manage-segment-for-vlan-range start-vlan untagged end-vlan 4095 tenant Tenant_res

preserve-bcf-config




4.8 Switch configuration on Z9100-ON leafs See section Leaf switches configuration using BCF GUI of the Dell EMC Ready Solution for VMware vCloud

NFV 3.0 OpenStack Edition Platform Deployment Manual Software Configuration Guide for switch installation

using BCF GUI.

4.9 Show running-config on Leafs

Below mentioned running config is a reference to check the configuration done in above sections on the BCF

through GUI.

Login into one of the controllers using ssh utility and run below command:

bcf01>

bcf01> enable

bcf01# show running-config

!

! Big Cloud Fabric 4.7.0 (bcf-4.7.0 #186)

! Current Time: 2019-02-01 02:38:38 UTC

!

version 1.0

! ntp

ntp server 192.168.176.249

! logging



! aaa

aaa accounting exec default start-stop local

! local

local node


hostname bcf01

interface management

!

ipv4

ip 192.168.176.90/24 gateway 192.168.176.254

method manual

dns server 192.168.176.250

dns search dellnfv.com

!

ipv6

method manual

! user

user admin

full-name 'Default admin'

hashed-password method=PBKDF2WithHmacSHA512,salt=w5heJjvUBnf-

Rb10jG5ysg,rounds=25000,ph=true,mF7aAQF-so7srCuqgEy-QvpNy-OXyvx394sPbO-

J3NiW_SnAzJuY950J1Adz1jVjF7b6G9v-2tecXS0oCLuljg

! group

group admin

associate user admin

! controller

controller

cluster-name bcfcluster

virtual-ip 192.168.176.92

access-control

!

access-list api

1 permit from ::/0


!

access-list gui

1 permit from ::/0


!

access-list ns-api

1 permit from ::/0


!

access-list ssh

1 permit from ::/0


!

access-list vce-api

1 permit from ::/0



! switch

switch Leaf1

fabric-role leaf

leaf-group Rack1

mac 34:17:eb:46:a8:80











interface ethernet30









switch Leaf2

fabric-role leaf

leaf-group Rack1

mac 34:17:eb:46:cc:80

interface ethernet1




















! fabric

fabric

forwarding-mode high-bandwidth

!

ipam switch

dns-server 192.168.176.250

gateway 192.168.176.254

ip-range 192.168.176.93 192.168.176.95 subnet-mask-length 24

allocate

! interface-group

interface-group To-Tor

mode lacp



backup-mode lacp


description 'Virtual switch ESXi-Management-VDS lag MgmtLag1 in ESXi host

esxi10.dellnfv.com'

mode lacp





description 'Virtual switch Infra-mgmt-VDS lag MgmtLag1 in ESXi host

esxi10.dellnfv.com'

mode lacp





description 'Virtual switch VM-mgmt-VDS lag MgmtLag1 in ESXi host

esxi10.dellnfv.com'

mode lacp







esxi11.dellnfv.com'

mode lacp






esxi11.dellnfv.com'

mode lacp






esxi11.dellnfv.com'

mode lacp






esxi12.dellnfv.com'

mode lacp






esxi12.dellnfv.com'

mode lacp






esxi12.dellnfv.com'

mode lacp







esxi13.dellnfv.com'

mode lacp






esxi13.dellnfv.com'

mode lacp






esxi13.dellnfv.com'

mode lacp





description 'Virtual switch ESXi-Management-VDS lag ResourceLag1 in ESXi

host esxi6.dellnfv.com'

mode lacp





description 'Interface group for virtual switch Infra-mgmt-VDS in ESXi










description 'Virtual switch NSXT-External-Network lag ResourceLag1 in

ESXi host esxi6.dellnfv.com'


mode lacp









description 'Virtual switch VM-mgmt-VDS lag ResourceLag1 in ESXi host

esxi6.dellnfv.com'

mode lacp







mode lacp

















mode lacp











esxi7.dellnfv.com'

mode lacp







mode lacp

















mode lacp











esxi8.dellnfv.com'

mode lacp







mode lacp

















mode lacp










esxi9.dellnfv.com'

mode lacp





! tenant

tenant Tenant_mgmt


multicast-enable


!


description ' 1 vSphere portgroups: API-Access-Network'

id Tenant_mgmt-10


!

endpoint vm-78-00-50-56-b4-14-d4

description 'VIO-LoadBalancer-0[vm-78] (Network adapter 2) on ESXi

Host esxi12.dellnfv.com [192.168.176.138, 192.168.176.132]'

mac 00:50:56:b4:14:d4


!

endpoint vm-82-00-50-56-b4-81-d0



mac 00:50:56:b4:81:d0


!




!




!



id Tenant_mgmt-176


!

endpoint vm-33-00-0c-29-26-61-e0

description 'AD-DNS-Server[vm-33] (Network adapter 1) on ESXi Host

esxi10.dellnfv.com [192.168.176.250]'

mac 00:0c:29:26:61:e0


!

endpoint vm-33-00-0c-29-3e-ea-79


description 'AD-DNS-Server[vm-33] (Network adapter 1) on ESXi Host

esxi10.dellnfv.com [192.168.176.250]'

mac 00:0c:29:3e:ea:79


!

endpoint vm-34-00-0c-29-05-4b-3a

description 'NTP-Server[vm-34] (Network adapter 1) on ESXi Host

esxi10.dellnfv.com [192.168.122.1]'

mac 00:0c:29:05:4b:3a


!

endpoint vm-34-00-0c-29-10-66-d0

description 'NTP-Server[vm-34] (Network adapter 1) on ESXi Host

esxi10.dellnfv.com [192.168.176.249]'

mac 00:0c:29:10:66:d0


!

endpoint vm-35-00-0c-29-16-eb-9a

description 'mgmtVcsaActive[vm-35] (Network adapter 1) on ESXi Host

esxi10.dellnfv.com'

mac 00:0c:29:16:eb:9a


!

endpoint vm-35-00-0c-29-a4-83-63


esxi10.dellnfv.com [192.168.176.101]'

mac 00:0c:29:a4:83:63


!

endpoint vm-51-00-50-56-b4-51-cc

description 'resourceVcsaActive[vm-51] (Network adapter 1) on ESXi

Host esxi11.dellnfv.com'

mac 00:50:56:b4:51:cc


!

endpoint vm-53-00-50-56-b4-ef-66

description 'mgmtVcsaPassive[vm-53] (Network adapter 1) on ESXi Host

esxi11.dellnfv.com [192.168.176.101]'

mac 00:50:56:b4:ef:66


!

endpoint vm-55-00-50-56-b4-e2-8d

description 'mgmtVcsaWitness[vm-55] (Network adapter 1) on ESXi Host

esxi12.dellnfv.com'

mac 00:50:56:b4:e2:8d


!


endpoint vm-57-00-50-56-b4-9c-7c

description 'resourceVcsaPassive[vm-57] (Network adapter 1) on ESXi

Host esxi12.dellnfv.com [192.168.176.102]'

mac 00:50:56:b4:9c:7c


!

endpoint vm-59-00-50-56-b4-c9-a9

description 'resourceVcsaWitness[vm-59] (Network adapter 1) on ESXi

Host esxi13.dellnfv.com'

mac 00:50:56:b4:c9:a9


!

endpoint vm-60-00-50-56-b4-1a-b2

description 'NsxtController1[vm-60] (Network adapter 1) on ESXi Host

esxi13.dellnfv.com [192.168.176.104]'

mac 00:50:56:b4:1a:b2


!

endpoint vm-61-00-50-56-b4-1b-f6


esxi12.dellnfv.com [192.168.176.106]'

mac 00:50:56:b4:1b:f6


!

endpoint vm-62-00-50-56-b4-0d-aa


esxi11.dellnfv.com [192.168.176.105]'

mac 00:50:56:b4:0d:aa


!

endpoint vm-63-00-50-56-b4-06-b1

description 'nsxtManager[vm-63] (Network adapter 1) on ESXi Host

esxi12.dellnfv.com [192.168.176.103]'

mac 00:50:56:b4:06:b1


!

endpoint vm-64-00-50-56-b4-12-37

description 'VropsReplicaNode[vm-64] (Network adapter 1) on ESXi Host

esxi12.dellnfv.com [192.168.176.129]'

mac 00:50:56:b4:12:37


!

endpoint vm-65-00-50-56-b4-70-c9

description 'VropsMasterNode[vm-65] (Network adapter 1) on ESXi Host

esxi10.dellnfv.com [192.168.176.127]'

mac 00:50:56:b4:70:c9



!

endpoint vm-66-00-50-56-b4-a5-d7

description 'VropsDataNode[vm-66] (Network adapter 1) on ESXi Host

esxi13.dellnfv.com [192.168.176.128]'

mac 00:50:56:b4:a5:d7


!

endpoint vm-67-00-50-56-b4-38-80

description 'VrliWorker2[vm-67] (Network adapter 1) on ESXi Host

esxi11.dellnfv.com [192.168.176.125]'

mac 00:50:56:b4:38:80


!

endpoint vm-68-00-50-56-b4-16-88

description 'VrliMaster[vm-68] (Network adapter 1) on ESXi Host

esxi10.dellnfv.com [192.168.176.123]'

mac 00:50:56:b4:16:88


!

endpoint vm-69-00-50-56-b4-d7-a8

description 'VrliWorker1[vm-69] (Network adapter 1) on ESXi Host

esxi13.dellnfv.com [192.168.176.124, 192.168.176.126]'

mac 00:50:56:b4:d7:a8


!

endpoint vm-71-00-50-56-b4-83-e0

description 'management-server[vm-71] (Network adapter 1) on ESXi


mac 00:50:56:b4:83:e0


!

endpoint vm-77-00-50-56-b4-b9-53

description 'VIO-Controller-1[vm-77] (Network adapter 1) on ESXi Host

esxi10.dellnfv.com [192.168.176.137]'

mac 00:50:56:b4:b9:53


!

endpoint vm-78-00-50-56-b4-5c-ea



mac 00:50:56:b4:5c:ea


!

endpoint vm-79-00-50-56-b4-2a-30

description 'VIO-DB-1[vm-79] (Network adapter 1) on ESXi Host

esxi12.dellnfv.com [192.168.176.134]'

mac 00:50:56:b4:2a:30



!

endpoint vm-80-00-50-56-b4-65-c1

description 'VIO-Compute-0[vm-80] (Network adapter 1) on ESXi Host

esxi11.dellnfv.com [192.168.176.140]'

mac 00:50:56:b4:65:c1


!

endpoint vm-81-00-50-56-b4-27-ec


esxi13.dellnfv.com [192.168.176.135]'

mac 00:50:56:b4:27:ec


!

endpoint vm-82-00-50-56-b4-0e-81



mac 00:50:56:b4:0e:81


!

endpoint vm-83-00-50-56-b4-39-be


esxi10.dellnfv.com [192.168.176.133]'

mac 00:50:56:b4:39:be


!

endpoint vm-84-00-50-56-b4-13-2d

description 'VIO-Controller-0[vm-84] (Network adapter 1) on ESXi Host

esxi12.dellnfv.com [192.168.176.136]'

mac 00:50:56:b4:13:2d


!

endpoint vm-85-00-50-56-b4-8e-82

description 'VIO-Kubernetes[vm-85] (Network adapter 1) on ESXi Host

esxi11.dellnfv.com [192.168.176.152]'

mac 00:50:56:b4:8e:82


!




!




!





!




!


description ' 1 vSphere portgroups: VCSA-HA-Network'

id Tenant_mgmt-20


!

endpoint vm-35-00-50-56-b4-8c-66



mac 00:50:56:b4:8c:66


!

endpoint vm-51-00-50-56-b4-71-4d

description 'resourceVcsaActive[vm-51] (Network adapter 2) on ESXi


mac 00:50:56:b4:71:4d


!

endpoint vm-53-00-50-56-b4-af-00

description 'mgmtVcsaPassive[vm-53] (Network adapter 2) on ESXi Host


mac 00:50:56:b4:af:00


!

endpoint vm-55-00-50-56-b4-24-d0

description 'mgmtVcsaWitness[vm-55] (Network adapter 2) on ESXi Host


mac 00:50:56:b4:24:d0


!

endpoint vm-57-00-50-56-b4-95-5c

description 'resourceVcsaPassive[vm-57] (Network adapter 2) on ESXi


mac 00:50:56:b4:95:5c


!

endpoint vm-59-00-50-56-b4-b6-8c

description 'resourceVcsaWitness[vm-59] (Network adapter 2) on ESXi


mac 00:50:56:b4:b6:8c



!




!




!




!




!



id Tenant_mgmt-200


!


description 'VMKernel [192.168.200.10] with Fault Tolerance,

Management traffic service'

ip 192.168.200.10

mac 24:6e:96:a4:1f:c8


!




ip 192.168.200.11

mac 24:6e:96:a4:23:90


!




ip 192.168.200.12

mac 24:6e:96:a4:23:2c


!




ip 192.168.200.13


mac 24:6e:96:a4:1b:34


!

member interface-group esxi10.dellnfv.com-ESXi-Management-VDS-MgmtLag1

vlan 200



!


vlan 200



!


vlan 200



!


vlan 200



!



id Tenant_mgmt-300


!



ip 192.200.3.10

mac 00:50:56:62:03:f4


!



ip 192.200.3.11

mac 00:50:56:65:4f:73


!



ip 192.200.3.12

mac 00:50:56:63:9d:03


!




ip 192.200.3.13

mac 00:50:56:60:7c:83


!

member interface-group esxi10.dellnfv.com-Infra-mgmt-VDS-MgmtLag1 vlan

300



!


300



!


300



!


300



!



id Tenant_mgmt-400


!



ip 192.200.4.10

mac 00:50:56:65:04:ab


!



ip 192.200.4.11

mac 00:50:56:62:df:f1


!



ip 192.200.4.12

mac 00:50:56:62:6a:e9



!



ip 192.200.4.13

mac 00:50:56:6d:2e:e0


!


400



!


400



!


400



!


400



tenant Tenant_res


multicast-enable

origination vsphere-tenant-resvc

logical-router

!

bgp

graceful-restart

local-as 64502

log-neighbor-changes

maximum-paths 2

no logging-flag events

no logging-flag keepalives

no logging-flag updates

no redistribute-ospf

no shutdown

protocol-ip 172.16.61.1

redistribute-connected

redistribute-static

router-id 172.16.61.1


!

neighbor NSX-Tier0

connect-time 60

ebgp-ttl multihop 2

hold-time 180



!

neighbor Tor

connect-time 60

ebgp-ttl multihop 2

hold-time 180



!

neighbor VIO-K-Tier0

connect-time 60

ebgp-ttl multihop 2

hold-time 180



interface segment Segment-To-Leaf

ip address 172.16.60.2/24

interface segment Segment-To-Tor

ip address 192.168.30.2/24

interface segment Segment-edge-overlay-700

ip address 192.200.7.254/24

interface segment Segment-vMotion-res-400

ip address 192.200.4.254/24

interface segment Segment-vSAN-res-300

ip address 192.200.3.254/24

segment Segment-To-Leaf

member interface-group esxi6.dellnfv.com-NSXT-External-Network-VDS-

vmnic8-vmnic9 vlan 30







segment Segment-To-Tor

member interface-group To-Tor vlan 30

!



id Tenant_res-176



!

endpoint vm-681-00-50-56-97-b1-c1

description 'NsxtEdge1[vm-681] (Network adapter 1) on ESXi Host

esxi6.dellnfv.com [192.168.176.107]'

mac 00:50:56:97:b1:c1


!

endpoint vm-682-00-50-56-97-c8-7e


esxi7.dellnfv.com [192.168.176.108]'

mac 00:50:56:97:c8:7e


!

member interface-group esxi6.dellnfv.com-NSXT-External-Network-

ResourceLag1 vlan 176



!

member interface-group esxi6.dellnfv.com-VM-mgmt-VDS-ResourceLag1 vlan

176



!





!

member interface-group esxi7.dellnfv.com-VM-mgmt-VDS-ResourceLag1 vlan

176



!



id Tenant_res-200


!




ip 192.168.200.6

mac 24:6e:96:a4:1e:a0


!





ip 192.168.200.7

mac 24:6e:96:a4:1b:24


!




ip 192.168.200.8

mac 24:6e:96:a4:21:1c


!




ip 192.168.200.9

mac 24:6e:96:a4:1e:1c


!

member interface-group esxi6.dellnfv.com-ESXi-Management-VDS-




!





!





!





!





!






!



id Tenant_res-300


!



ip 192.200.3.6

mac 00:50:56:68:bd:9c


!



ip 192.200.3.7

mac 00:50:56:6a:94:46


!



ip 192.200.3.8

mac 00:50:56:62:03:dd


!



ip 192.200.3.9

mac 00:50:56:64:09:63


!




!




!





!





!




!





!




!




!




!




!



id Tenant_res-400


!



ip 192.200.4.6

mac 00:50:56:6f:11:6c


!



ip 192.200.4.7

mac 00:50:56:6d:80:ed


!



ip 192.200.4.8


mac 00:50:56:6c:f5:e3


!



ip 192.200.4.9

mac 00:50:56:61:ac:38


!




!




!





!




!




!





!




!




!




!





!


description ' 1 vSphere portgroups: Edge-overlay-Network'

id Tenant_res-700


!

endpoint vm-681-00-50-56-97-fc-fe


esxi6.dellnfv.com'

mac 00:50:56:97:fc:fe


!

endpoint vm-682-00-50-56-97-e4-87


esxi7.dellnfv.com'

mac 00:50:56:97:e4:87


!




!




!





member interface-group esxi6.dellnfv.com-Overlay-Network-VDS-vmnic5

vlan 700


vlan 700

!




!




!







vlan 700


vlan 700




vlan 700


vlan 700




vlan 700


vlan 700

tenant test-mgmt

multicast-enable

! vcenter

vcenter mgmt-VCSA



hashed-password 2005223e1364704053


manage-segment-for-vlan-range start-vlan untagged end-vlan 4095 tenant

Tenant_mgmt

preserve-bcf-config



vcenter res-VCSA



hashed-password 210a2e251573415442


manage-segment-for-vlan-range start-vlan untagged end-vlan 4095 tenant

Tenant_res

preserve-bcf-config




4.10 Third-party network hardware If Dell EMC Networking S4048T-ON and Dell EMC Networking Z9100-ON switches are not used, program the

switches to support the cloud installation.

The BCF controller provides a “single pane of glass” for management of all leaf and spine switches. The BCF

controller supports a familiar CLI and a web-based GUI. Any custom orchestration can be run using the

industry-standard RESTful Application Programming Interface (API).

BCF supports traditional tools for debugging, including ping, traceroute, show commands, and redirecting

packets using port mirroring for fault analysis. The BCF Controller also supports unique troubleshooting tools,

such as fabric test path and fabric analytics to quickly isolate, identify, and resolve forwarding and application

faults.

IEEE 802.1Q VLAN traffic and port tagging

Use of one untagged and multiple tagged VLANs on the same port

The ability to provide a minimum of 170-Gigabit Ethernet ports, in a nonblocking configuration, within

the provisioning VLAN

Configuration of a single switch or a combination of stacked switches, to meet the additional

requirements

The ability to create link aggregation groups (LAGs) with a minimum of two physical links in each LAG

If multiple switches are stacked:

The ability to create a LAG across stacked switches

Full-bisection bandwidth

Support for available VLANs across all switches in the stack

250,000 packets-per-second capability per switch

Managed switch that supports both SSH and serial line configuration

SNMP v3 support


5 Solution archive Applying the enhanced tenant capabilities of VMware Integrated OpenStack, VMware vCloud NFV 3.0

facilitates the combination of the edge and resource functionality into a single, collapsed pod. In combination, a

smaller footprint design is possible. CSPs can use a 2-pod design to gain operational experience with vCloud

NFV. As demand grows, they scale up and out within the 2-pod construct.

Figure 33 shows a typical 2-pod design with all management functions centrally located within the

management pod. Edge and resource functions are combined into the collapsed edge/resource pod. During

initial deployment, two clusters of ESXi hosts are used - one for the management pod, and the other for the

collapsed edge/resource pod. More clusters can be added to each pod as the infrastructure is scaled up.

2-pod design overview

Management cluster inventory layout

Management cluster

ESXi1 ESXi2 ESXi3 ESXi4

AD-DNS-Server Res-VCSA-Active Mgmt-VCSA-Witness VROPS-Master

NTP-Server Mgmt-VCSA-Passive VRLI-Worker2 Res-VCSA-Witness

Mgmt-VCSA-Active VROPS-Replica-Node nsx-manager nsx-controller1

VROPS-Data-Node VRLI-Worker1 Res-VCSA-Passive VRLI-Master

Kubernetes Manager nsx-controller2 nsx-controller3 OpenStack Manager


Resource cluster

ESXi1 ESXi2 ESXi3 ESXi4


NTP-Server Mgmt-VCSA-Passive VRLI-Worker2 Res-VCSA-Witness

Mgmt-VCSA-Active VROPS-Replica-Node nsx-manager nsx-controller1


VROPS-Data-Node VRLI-Worker1 Res-VCSA-Passive VRLI-Master

Kubernetes Manager nsx-controller2 nsx-controller3 OpenStack Manager



Resource cluster inventory layout

Resource/Edge cluster

ESXi1 ESXi2 ESXi3 ESXi4 ESXi5 ESXiN

NSX-T Edge-01 NSX-T Edge-02 VNF VNF VNF VNF

5.1 List of components NFVI is the totality of all hardware and software components that build the environment where VNFs are

deployed. The following components support the creation of NFV-cloud infrastructure:

Components listing

Components

Instances (count)

VM’s = Virtual machine

ESXi 8 nodes

AD_DNS 1 VM

NTP 1 VM

vCenter Server 6 VMs

vSAN NA

NSX Manager 1 VM

NSX Controller 3 VMs

NSX Edge VM 2 VMs

vRLI 3 VMs

vROPS 3 VMs

VIO For OpenStack: 9 VMs (1 OpenStack Manager VM + 8 OpenStack VM)

For Kubernetes: 1 VM (VIO-Kubernetes 1 VM)


A Reference documentation

See the following documentation for more system information:

Dell EMC Ready Solution for VMware vCloud NFV Platform Deployment Manual Software

Configuration guide

VMware vCloud NFV Reference Architecture v3.0

Dell PowerEdge R640 Owner's Manual

Dell PowerEdge R740 Owner's Manual

Dell PowerEdge R740xd Owner's Manual

https://www.dell.com/support/home/us/en/04/product-support/product/poweredge-r640/manuals




https://www.dell.com/support/home/us/en/04/product-support/product/poweredge-r740xd/manuals

https://www.dell.com/support/home/us/en/04/product-support/product/poweredge-r740xd/manuals

Dell EMC Ready Solution for VMware vCloud NFV 3.0 ... · Leaf1+Leaf2 . 13 Dell EMC Ready Solution...

Documents

Transcript of Dell EMC Ready Solution for VMware vCloud NFV 3.0 ... · Leaf1+Leaf2 . 13 Dell EMC Ready Solution...