b Cisco NX-OS FCoE Configuration Guide

Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500First Published: July 12, 2011

Last Modified: November 01, 2012

Americas HeadquartersCisco Systems, Inc.170 West Tasman DriveSan Jose, CA 95134-1706USAhttp://www.cisco.comTel: 408 526-4000 800 553-NETS (6387)Fax: 408 527-0883

Text Part Number: OL-25809-02

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITEDWARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITHTHE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain versionof the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDINGANYOTHERWARRANTYHEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS"WITH ALL FAULTS.CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OFMERCHANTABILITY, FITNESS FORA PARTICULAR PURPOSEANDNONINFRINGEMENTORARISING FROMACOURSEOFDEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUTLIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERSHAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnershiprelationship between Cisco and any other company. (1110R)

Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shownfor illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.

© 2011-2012 Cisco Systems, Inc. All rights reserved.

http://www.cisco.com/go/trademarks

http://www.cisco.com/go/trademarks

C O N T E N T S

P r e f a c e Preface vii

Audience vii

Document Conventions vii

Related Documentation viii

Obtaining Documentation and Submitting a Service Request ix

P r e f a c e New and Changed Information xi

Cisco Nexus 7000: New and Changed Information xi

Cisco MDS 9500: New and Changed Information xii

C H A P T E R 1 Overview 1

Information About Fibre Channel Over Ethernet 1

Fibre Channel Forwarder 1

Fibre Channel Bridge 2

C H A P T E R 2 FCoE 3

Information About FCoE 3

FCoE and FIP 3

FCoE Initiation Protocol 3

FIP Virtual Link Instantiation 4

FCoE Frame Format 4

VLAN Tagging for FCoE Frames 4

FIP Ethernet Frame Format 5

DCBX 5

Data Center Bridging Exchange Protocol 5

Lossless Ethernet 6

Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 iii

Logical Link Up/Down 6

Converged Network Adapters 6

STP Lite 7

Default Settings for FCoE 7

Licensing Requirements for FCoE 7

Guidelines and Limitations 8

Cisco Nexus 7000: Guidelines and Limitations 8

Cisco MDS 9500: Guidelines and Limitations 9

Configuring FCoE 10

Enabling FCoE 10

Cisco Nexus 7000: Enabling FCoE 10

Cisco MDS 9500: Enabling FCoE 11

Disabling FCoE 13

Cisco Nexus 7000: Disabling FCoE 13

Cisco MDS 9500: Disabling FCoE 14

Configuring the FC-Map 15

Configuring the Fabric Priority 16

Setting the Advertisment Interval 17

Disabling LAN Traffic on an FCoE Link 18

Verifying the FCoE Configuration 19

Additional References for FCoE 20

C H A P T E R 3 Configuring FCoE VLANs and Virtual Interfaces 21

Information About Virtual Fibre Channel (VFC) Interfaces 21

VF Port 21

VE Port 22

Default Settings for FCoE 22





Configuring Virtual Interfaces 25

Mapping a VSAN to a VLAN 25

Creating a Virtual Fibre Channel Interface 26

Creating a Virtual Fibre Channel Port Channel Interface 28

Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500iv OL-25809-02

Contents

Associating a Virtual Fibre Channel Interface to a VSAN 29

Enabling VE Loopback Configuration 31

Verifying the Virtual Interface 32

Example:Mapping VSANs to VLANs 33

Additional References for FCoE 34

C H A P T E R 4 Storage VDC on Cisco Nexus 7000 Series Switches 37

Information About Storage VDC 37





Configuring FCoE VDCs 40

Creating a Dedicated Storage VDC 40

Cisco Nexus 7000:Licensing an FCoE Module 42

Allocating the FCoE VLAN Range 43

Allocating Shared Interfaces 44

Example: Storage VDC Configuration 46

A P P E N D I X A Feature History by Platform 49

Cisco Nexus 7000:Supported Features 49

Cisco MDS 9500:Supported Features 49

A P P E N D I X B Guidelines and Limitations 51

Verified Limits for FCoE for the Cisco Nexus 7000 Series and Cisco MDS 9500 51



Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 v

Contents

Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500vi OL-25809-02

Contents

Preface

This preface describes the audience, organization, and conventions of theCisco NX-OS FCoE ConfigurationGuide. It also provides information on how to obtain related documentation.

• Audience, page vii

• Document Conventions, page vii

• Related Documentation, page viii

• Obtaining Documentation and Submitting a Service Request, page ix

AudienceThis publication is for experienced users who configure and maintain Cisco NX-OS devices.

Document ConventionsCommand descriptions use the following conventions:

DescriptionConvention

Bold text indicates the commands and keywords that you enter literallyas shown.

bold

Italic text indicates arguments for which the user supplies the values.Italic

Square brackets enclose an optional element(keyword or argument).[x]

Square brackets enclosing keywords or arguments separated by a verticalbar indicate an optional choice.

[x | y]

Braces enclosing keywords or arguments separated by a vertical barindicate a required choice.

{x | y}

Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 vii


Nested set of square brackets or braces indicate optional or requiredchoices within optional or required elements. Braces and a vertical barwithin square brackets indicate a required choice within an optionalelement.

[x {y | z}]

Indicates a variable for which you supply values, in context where italicscannot be used.

variable

A nonquoted set of characters. Do not use quotation marks around thestring or the string will include the quotation marks.

string

Examples use the following conventions:


Terminal sessions and information the switch displays are in screen font.screen font

Information you must enter is in boldface screen font.boldface screen font

Arguments for which you supply values are in italic screen font.italic screen font

Nonprinting characters, such as passwords, are in angle brackets.< >

Default responses to system prompts are in square brackets.[ ]

An exclamation point (!) or a pound sign (#) at the beginning of a lineof code indicates a comment line.

!, #

This document uses the following conventions:

Means reader take note. Notes contain helpful suggestions or references to material not covered in themanual.

Note

Means reader be careful. In this situation, you might do something that could result in equipment damageor loss of data.

Caution

Related DocumentationThe Cisco Nexus 7000 Series documentation is available at the following URL:

http://www.cisco.com/en/us/products/ps9402/tsd_products_support_series_home.html

The Cisco MDS 9500 Series documentation is available at the following URL:

Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500viii OL-25809-02

PrefaceRelated Documentation

http://www.cisco.com/en/us/products/ps9402/tsd_products_support_series_home.html

http://www.cisco.com/en/US/products/ps5989/tsd_products_support_series_home.html

Obtaining Documentation and Submitting a Service RequestFor information on obtaining documentation, submitting a service request, and gathering additional information,see the monthlyWhat's New in Cisco Product Documentation, which also lists all new and revised Ciscotechnical documentation, at:

http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html

Subscribe to theWhat's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feedand set content to be delivered directly to your desktop using a reader application. The RSS feeds are a freeservice and Cisco currently supports RSS version 2.0.

Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 ix

PrefaceObtaining Documentation and Submitting a Service Request

http://www.cisco.com/en/US/products/ps5989/tsd_products_support_series_home.html

http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html

Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500x OL-25809-02

PrefaceObtaining Documentation and Submitting a Service Request

New and Changed Information

This chapter includes the following sections:

• Cisco Nexus 7000: New and Changed Information, page xi

• Cisco MDS 9500: New and Changed Information, page xii

Cisco Nexus 7000: New and Changed InformationThis section provides release-specific information for each new and changed features documented for theCisco Nexus 7000 Series switches.

The latest version of this document is available at the following Cisco website:

http://www.cisco.com/en/US/products/ps9670/products_installation_and_configuration_guides_list.html

To check for additional information about Cisco Nexus 7000 software, see theCisco Nexus 7000 Series NX-OSRelease Notes available at the following Cisco website:

http://www.cisco.com/en/US/products/ps9670/prod_release_notes_list.html

Table 1: New and Changed Features

Changed in ReleaseDescriptionFeature

6.1(2)FCoE support on F2e Series modulesF2e Series Module support (SPF+ only)

6.1(1)FCoE support on F2 Series modulesF2 Series Module support

6.0(1)—No changes in this release

5.2(1)Initial FCoE supportFCoE

Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 xi



Cisco MDS 9500: New and Changed InformationThis section provides release-specific information for each new and changed features documented for theCisco MDS 9500 Series switches.

The latest version of this document is available at the following Cisco website:


To check for additional information about Cisco MDS 9500 software, see the Cisco MDS 9000 Series NX-OSRelease Notes available at the following Cisco website:


Table 2: New and Changed Features

Where DocumentedChanged in ReleaseDescriptionFeature

———There are no changes sinceRelease 5.2(1)

This document5.2(1)Initial FCoE supportFCoE

Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500xii OL-25809-02

New and Changed InformationCisco MDS 9500: New and Changed Information



C H A P T E R 1Overview


• Information About Fibre Channel Over Ethernet, page 1

• Fibre Channel Forwarder, page 1

• Fibre Channel Bridge, page 2

Information About Fibre Channel Over EthernetFibre Channel over Ethernet (FCoE) allows Fibre Channel traffic to be encapsulated over a physical Ethernetlink. FCoE and FIP frames use a unique EtherType so that FCoE traffic and standard Ethernet traffic can becarried on the same link.

Classic Ethernet is a best-effort protocol, which means that in a congested network, Ethernet discards packetsand relies on higher level protocols to provide retransmission and other reliability mechanisms.

Fibre Channel traffic requires a lossless transport layer; as a data storage protocol, it is unacceptable to losea single data packet. Native Fibre Channel implements a lossless service at the transport layer using abuffer-to-buffer credit system.

Ethernet links on Cisco NX-OS switches provide twomechanisms to ensure lossless transport for FCoE traffic:link-level flow control and priority flow control.

IEEE 802.3x link-level flow control allows a congested receiver to signal the far end to pause the datatransmission for a short period of time. The pause functionality is applied to all traffic on the link.

The priority flow control (PFC) feature on Cisco NX-OS platforms applies pause functionality to specificclasses of traffic on the Ethernet link. For example, PFC can provide lossless service for the FCoE traffic andbest-effort service for the standard Ethernet traffic using IEEE 802.1p traffic classes.

Fibre Channel ForwarderFibre Channel traffic requires a lossless transport layer, which means that it requires a data storage protocolthat does not lose a single data packet. A FCoE Forwarder (FCF) communicates with FCoE end devices, suchas converged network adapters (CNAs). The FCF accepts the Fibre Channel frame that is encapsulated in anEthernet packet and forwards that packet over a VLAN across an Ethernet network to a remote FCoE end

Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 1

device. The FCF has a unique FC-MAC address in the FCoE network. An FCF can also assign FC addressIDs to the CNAs. An FCF can assign Fabric Provide MAC Addresses (FPMA) to the CNAs consisting of theFC-Map Value for the Fabric and the Fibre Channel IDs (FC IDs) assigned during Fabric Login.

Fibre Channel BridgeA Fibre Channel Bridge connects an FCoE network to a Fibre Channel network. A Fibre Channel Bridgedecapsulates an FCoE frame and sends the Fibre Channel frame to the Fibre Channel network. A Fibre ChannelBridge also encapsulates FC frames from a Fibre Channel network and forwards them to the FCoE network.

Cisco MDS 9500 switches support Fibre Channel Bridge functions.

The Cisco Nexus 7000 series switches do not offer native Fibre Channel ports and do not support FCBridge functions.

Note

Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 95002 OL-25809-02

OverviewFibre Channel Bridge

C H A P T E R 2FCoE


• Information About FCoE, page 3

• Default Settings for FCoE, page 7

• Licensing Requirements for FCoE, page 7

• Guidelines and Limitations, page 8

• Configuring FCoE, page 10

• Verifying the FCoE Configuration, page 19

• Additional References for FCoE, page 20

Information About FCoEFCoE provides a method of transporting Fibre Channel traffic over a physical Ethernet connection. FCoErequires the underlying Ethernet to be full duplex and to provide lossless behavior for Fibre Channel traffic.

Lossless behavior on Ethernet is provided by using a priority flow control (PFC) mechanism that preventspacket loss during congestion conditions.

Note

The Cisco NX-OS software supports T11-compliant FCoE on all 10-Gigabit Ethernet interfaces.

FCoE and FIP

FCoE Initiation ProtocolThe FCoE Initialization Protocol (FIP) allows the switch to discover and initialize FCoE-capable entities thatare connected to an Ethernet LAN. Cisco NX-OS switches support the Converged Enhanced Ethernet DataCenter Bridging Exchange (CEE-DCBX) protocol for T11-compliant Generation 2 CNAs.

The following switches do not support Pre-FIP on Generation 1 CNAs:


• Cisco Nexus 7000

• Cisco MDS 9500

FIP Virtual Link InstantiationFIP is used to perform device discovery, initialization, and link maintenance. FIP performs the followingtasks:

• FIP VLAN discovery— Discovers the FCoE VLAN that will be used by all other FIP protocols as wellas by the FCoE encapsulation for Fibre Channel payloads on the established virtual link. FIP VLANdiscovery occurs in the native VLAN used by the initiator or target to exchange Ethernet traffic. TheFIP VLAN discovery protocol is the only FIP protocol running on the native VLAN; all other FIPprotocols run on the discovered FCoE VLANs.

• FIP FCF discovery—When a FCoE device is connected to the fabric, it sends a Discovery Solicitationmessage. A Fibre Channel Forwarder (FCF) or a switch responds to the message with a SolicitedAdvertisement that provides an FCF MAC address to use for subsequent logins.

• FCoE virtual link instantiation— FIP defines the encapsulation of fabric login (FLOGI), fabric discovery(FDISC), logout (LOGO), and exchange link parameter (ELP) frames along with the correspondingreply frames. The FCoE devices use these messages to perform a fabric login.

• FCoE virtual link maintenance— FIP periodically sends maintenance messages between the switch andthe CNA to ensure that the connection is still valid.

FCoE Frame FormatFCoE is implemented when the switch encapsulates a Fibre Channel frame in an Ethernet packet with adedicated Ethernet type, 0x8906. The packet has a 4-bit version field. The other header fields in the frame(the source and destination MAC addresses, VLAN tags, and frame markers) are all standard Ethernet fields.Reserved bits pad the FCoE frame to the IEEE 802.3 minimum packet length of 64 bytes.

A Fibre Channel frame consists of 36 bytes of headers and up to 2112 bytes of data for a total maximum sizeof 2148 bytes. The encapsulated Fibre Channel frame has all the standard headers, which allow it to be passedto the storage network without further modification. To accommodate the maximum Fibre Channel frame inan FCoE frame, the class-fcoe is defined with a default Maximum Transmission Unit (MTU) of 2240 bytes.

VLAN Tagging for FCoE FramesThe Ethernet frames that are sent by the switch to the adapter include the IEEE 802.1Q tag. This tag includesa field for the class of service (CoS) value used by the priority flow control (PFC). The IEEE 802.1Q tag alsoincludes a VLAN field.

The switch expects frames from a FIP T11-compliant CNA to be tagged with the VLAN tag for the FCoEVLAN. Frames that are not correctly tagged are discarded.

You cannot map VLAN 1 or the native VLAN to an FCoE VSAN.Note

The following switches do not support Pre-FIP on Generation 1 CNAs:


FCoEFCoE and FIP


• Cisco MDS 9500

FIP Ethernet Frame FormatFIP is encapsulated in an Ethernet packet with a dedicated EtherType, 0x8914. The packet has a 4-bit versionfield. Along with the source and destination MAC addresses, the FIP packet also contains a FIP operationcode and a FIP operation subcode. The following table describes the FIP operation codes and subcodes.

Table 3: FIP Operation Codes

FIP OperationFIP SubcodeFIP Operation Code

Discovery Solicitation0x010x0001

Discovery Advertisement0x02

Virtual Link Instantiation Request0x010x0002

Virtual Link Instantiation Reply0x02

FIP Keepalive0x010x0003

FIP Clear Virtual Links0x02

FIP VLAN Request0x010x0004

FIP VLAN Notification0x02

DCBX

Data Center Bridging Exchange ProtocolThe Data Center Bridging Exchange (DCBX) protocol is an extension of the Link Layer Discovery Protocol(LLDP). DCBX end points exchange request and acknowledgment messages. For flexibility, parameters arecoded in a type-length-value (TLV) format. Cisco NX-OS switches support the Converged Enhanced EthernetData Center Bridging Exchange (CEE-DCBX) is supported on all T11-compliant Generation 2 CNAs.

DCBX runs on the physical Ethernet link between the switch and the CNA. By default, DCBX is enabled onEthernet interfaces. When an Ethernet interface is brought up, the switch automatically starts to communicatewith the CNA.

During the normal operation of FCoE between the switch and the CNA, DCBX provides link-error detection.

DCBX is also used to negotiate capabilities between the switch and the CNA and to send configuration valuesto the CNA.


FCoEDCBX

The CNAs that are connected to a switch are programmed to accept the configuration values that are sent bythe switch, allowing the switch to distribute configuration values to all attached CNAs, which reduces thepossibility of configuration errors and simplifies CNA administration.

Lossless EthernetStandard Ethernet is a best-effort medium which means that it lacks any form of flow control. In the event ofcongestion or collisions, Ethernet drops packets. The higher level protocols detect the missing data andretransmit the dropped packets.

To properly support Fibre Channel, Ethernet has been enhanced with a priority flow control (PFC) mechanism.

Logical Link Up/DownOn a native Fibre Channel link, some configuration actions (such as changing the VSAN) require that youreset the interface status. When you reset the interface status, the switch disables the interface and thenimmediately reenables the interface.

Caution:If an Ethernet link provides FCoE service, do not reset the physical link because this action is disruptive toall traffic on the link.

The logical link up/down feature allows the switch to reset an individual virtual link. The logical link downis signaled with a FIP Clear Virtual Link message.

Caution:If the CNA does not support the logical link level up/down feature, the CNA resets the physical link, whichmeans that all traffic on the Ethernet interface is disrupted.

Converged Network AdaptersCisco NX-OS switches support the following CNA types:

• Hardware adapter

◦Works with the existing Fibre Channel host bus adapter (HBA) driver and Ethernet NetworkInterface Card (NIC) driver in the server.

◦Server operating system view of the network is unchanged; the CNA presents a SAN interface anda LAN interface to the operating system.

• FCoE software stack

◦Runs on existing 10-Gigabit Ethernet adapters.

The following Cisco NX-OS series and platforms support Generation 2 CNAs that use the FIP to exchangeinformation about its available capabilities and to negotiate the configurable values with the switch:





FCoELossless Ethernet


• Cisco MDS 9500

To reduce configuration errors and simplify administration, the switch distributes the configuration data toall the connected adapters.

STP LiteFCoE does not require full Spanning Tree Protocol (STP) because FCoE has no bridging functionality, whichmeans that no STP loops are created in the network. STP Lite on FCoE interfaces ensures rapid convergenceacross the network by sending an agreement Bridge Protocol Data Unit (BPDU)whenever it receives a proposalBPDU. The FCoE link sends the identical agreement BPDU in response to either an Multiple Spanning Tree(MST) or a Per VLANRapid Spanning Tree Plus (PVRST+) proposal BPDU.Additionally, STP Lite suppressesthe MAC address flushing function for FCoE VLANs.

STP Lite is enabled automatically by default across the entire device for FCoE VLANs as soon as the firstFCoE VLAN comes up. At the same time, the system automatically converts all FCoE links as the STP-typenormal ports. This feature runs only in FCoE VLANs.

Default Settings for FCoEThis table lists the default settings for FCoE parameters.

Table 4: Default FCoE Parameter Settings

DefaultParameters

Not installed, disabledFCoE feature

0E.FC.00FC-Map

128Fabric priority

8 secondsAdvertisement interval

Licensing Requirements for FCoELicenseProduct

Each F Series module that runs FCoE requires anFCoE license. FCoE enabled in a nondefault VDCdoes not require the Advanced Services License. Fora complete explanation of the Cisco NX-OS licensingscheme and how to obtain and apply licenses, see theCisco NX-OS Licensing Guide.

Cisco Nexus 7000 Series


FCoESTP Lite

LicenseProduct

FCoE does not require a license. For a completeexplanation of the Cisco NX-OS licensing schemeand how to obtain and apply licenses, see the CiscoNX-OS Licensing Guide.

Cisco MDS 9500 Series

Guidelines and Limitations

Cisco Nexus 7000: Guidelines and Limitations• You cannot enable FCoE on default VLAN.

• The QoS policy must be the same on all Cisco FCoE switches in the network.

• Beginning with Cisco NX-OS Release 6.1, FCoE is supported on F2 and F2e Series modules.

◦FCoE supports only F2e (SFP+) modules.

◦FCoE does not support F2e (Copper) modules.

FCoE VDC

FCoE in a dedicated storage VDC has the following guidelines:

• Enable the FCoE feature set in only one VDC.

• Create VLANs in the FCoE allocated VLAN range.

• Do not enable any other features other than storage-related features in the dedicated FCoE VDC.

• Allocate resources for the dedicated FCoE VDC from an F Series module, such as the 32-port 1- and10-Gigabit Ethernet I/O module (PID N7K-F132XP-15) .

• Rollback is not supported in a storage VDC.

• FCoE support on F2 and F2e Series modules, SF248XP-25 requires Supervisor 2 module.

• F2 and F2e series modules cannot exist in the same VDC with any other module type. This applies toboth LAN and storage VDCs.

Shared Interfaces

Any change in protocol state that flaps the parent port of a shared interface because of any port featurealso affects the FCOE traffic on the storage vdc.

Note

The following interface config modes are not allowed while sharing an interface from Ethernet vdc to a storagevdc:

• SPAN destination


FCoEGuidelines and Limitations

• Private VLAN mode

• Port-channel interfaces

• Access mode

• mac-packet-classify

• Interfaces that are part of a VLAN that has an associated QoS policy

Shared Ethernet interfaces must be in trunk mode and only shared with one other VDC.

Storage VDC

Configuring a VDC for the OOB management interface mgmt0 is accomplished with the vrf contextmanagement command. However, a storage VDC does not support VRF, so configuring mgmt0 requires adifferent approach.

The following table shows how to configure mgmt 0 for a VDC and for a storage VDC:

Configuring mgmt 0 for storage VDCConfiguring mgmt 0 for VDC

interface mgmt 0ip address mgmt0_ip_address mgmt0_subnet_maskno shutip route 0.0.0.0/0 default_gateway

The ip route command specifies the defaultroute that points to the default gateway.

Note

vrf context managementip route 0.0.0.0/0 default_gateway

where

• mgmt0_ip_address is the mgmt0 IPv4 address.

• mgmt0_subnet_mask is the mgmt0 IPv4 netmask.

• default_gateway is the IPv4 address of the default-gateway.

Formore information about VDC, see theCisco Nexus 7000 Series NX-OSVirtual Device Context ConfigurationGuide.

Cisco MDS 9500: Guidelines and Limitations• You cannot enable FCoE on the default VLAN.

• You can only enable FCoE on an FCoE module.


• Cisco MDS 9500 does not support LAN traffic and only accepts and processes FCoE frames.

• You must install a Supervisor-2A module to configure FCoE.

• You can install an FCoE module in a fabric that uses FC-Redirect services but do not install the FCoEmodule in the same switch where FC-Redirect services application modules are enabled.

• SME does not support any FCoE connected devices including devices connected through theMDS FCoElinecard (DX-X9708-K9)


FCoECisco MDS 9500: Guidelines and Limitations

• In Cisco MDS NX-OS Release 5.2(x), you cannot install a FCoE module in a switch that is runningDMM, SME, or IOA.

Configuring FCoE

Enabling FCoE

Cisco Nexus 7000: Enabling FCoEYou must install the FCoE feature set in the default VDC and enable dependent features in order to enableFCoE in a storage VDC.

Before You Begin

• Ensure you are in the default VDC.

• Ensure you have installed the correct license for your switch.

• Do not enable FCoE on VLAN 1 or the native VLAN.

SUMMARY STEPS

1. configure terminal2. install feature-set fcoe3. feature lldp4. (Optional) feature lacp5. system qos6. service-policy type network-qos policy-name7. (Optional) show feature8. (Optional) copy running-config startup-config

DETAILED STEPS

PurposeCommand or Action

Enters configuration mode.configure terminal

Example:switch# configure terminalswitch(config)#

Step 1

Installs the FCoE feature set in the default VDC.install feature-set fcoe

Example:switch(config)# install feature-set fcoe

Step 2


FCoEConfiguring FCoE


Enables the Link Layer Discovery Protocol (LLDP) featurein the default VDC. This feature is required for FCoEoperation.

feature lldp

Example:switch(config)# feature lldp

Step 3

(Optional)Enables the Link Aggregation Control Protocol (LACP)feature in the default VDC. This feature is considered a bestpractice for FCoE operation.

feature lacp

Example:switch(config)# feature lacp

Step 4

Enters Quality of service (QoS) configuration mode.system qos

Example:switch(config)# system qos

Step 5

Enables the QoS policy that supports FCoE traffic. Thepolicy-name default is default-nq-8e-policy.

service-policy type network-qos policy-name

Example:switch(config-sys-qos)# service-policy typenetwork-qos default-nq-7e-policy

Step 6

(Optional)Displays information about the enabled features.

show feature

Example:switch(config-sys-qos)# show feature

Step 7

(Optional)Copies the running configuration to the startupconfiguration.

copy running-config startup-config

Example:switch(config-sys-qos)# copy running-configstartup-config

Step 8

What to Do Next

You must create a storage VDC and allocate resources to finish enabling FCoE.

Cisco MDS 9500: Enabling FCoEThe CiscoMDS 9500 switch automatically enables the FCoE feature when an FCoE-capable module is presentin the chassis. You must enable supporting features and the appropriate QoS policy for FCoE.

Before You Begin

• Do not enable FCoE on VLAN 1 or the native VLAN.


FCoEEnabling FCoE

SUMMARY STEPS

1. configure terminal2. (Optional) feature lacp3. system qos4. service-policy type network-qos policy-name5. (Optional) show feature6. (Optional) copy running-config startup-config

DETAILED STEPS




Step 1

(Optional)Enables the Link Aggregation Control Protocol (LACP)feature. This feature is considered a best practice for FCoEoperation.

feature lacp


Step 2

Enters Quality of Service (QoS) configuration mode.system qos

Example:switch(config)# system qos

Step 3

Enables the QoS policy that supports FCoE traffic. Thepolicy-name default is default-nq-6e-policy.

service-policy type network-qos policy-name

Example:switch(config-sys-qos)# service-policy typenetwork-qos default-nq-7e-policy

Step 4

(Optional)Displays information about the enabled features.

show feature

Example:switch(config-sys-qos)# show feature

Step 5



Example:switch(config-sys-qos)# copy running-configstartup-config

Step 6


FCoEEnabling FCoE

Disabling FCoE

Cisco Nexus 7000: Disabling FCoEYou can disable or uninstall the FCoE feature set. You can also disallow the FCoE feature set in a VDC.

Before You Begin

Ensure you are in the correct VDC.

SUMMARY STEPS

1. configure terminal2. vdc vdc_id type storage3. (Optional) no allow feature-set fcoe4. no feature-set fcoe5. exit6. no install feature-set fcoe7. (Optional) show feature-set8. (Optional) copy running-config startup-config

DETAILED STEPS




Step 1

Enters VDC configuration mode. The vdc_id can be anycase-sensitive, alphanumeric string up to 32 characters.

vdc vdc_id type storage

Example:switch(config)# vdc fcoe type storageswitch(config-vdc)#

Step 2

(Optional)Disallows any FCoE in this VDC. The default is allow.

no allow feature-set fcoe

Example:switch(config-vdc)# no allow feature-set fcoeswitch(config-vdc)#

Step 3

Disables the FCoE feature set. The no feature-set fcoecommand may take some time to complete if the size of the

no feature-set fcoe

Example:switch(config-vdc)# no feature-set fcoe

Step 4

configuration is very large. The command must clean up allof the configuration associated with the FCoE feature set.


FCoEDisabling FCoE


Exits VDC configuration mode.exit

Example:switch(config-vdc)# exitswitch(config)#

Step 5

Uninstalls the FCoE feature set. Use this command in thedefault VDC after you disable the FCoE feature set.

no install feature-set fcoe

Example:switch(config)# no install feature-set fcoe

Step 6

(Optional)Displays information about the feature sets.

show feature-set

Example:switch(config)# show feature-set

Step 7

(Optional)Copies the running configuration to the startup configuration.


Example:switch(config)# copy running-configstartup-config

Step 8

Cisco MDS 9500: Disabling FCoEAfter you disable the FCoE feature, all FCoE commands are removed from the CLI and the FCoE configurationis deleted.

SUMMARY STEPS

1. configure terminal2. no feature-set fcoe

DETAILED STEPS




Step 1

Disables the FCoE feature set. The no feature-set fcoe commandmay take some time to complete if the size of the configuration is

no feature-set fcoe

Example:switch(config)# no feature-set fcoe

Step 2

very large. The command must clean up all of the configurationassociated with the FCoE feature set.


FCoEDisabling FCoE

This example shows how to disable FCoE on the switch:switch# configure terminalswitch(config)# no feature-set fcoe

Configuring the FC-MapYou can prevent data corruption due to cross-fabric talk by configuring an FC-Map that identifies the FibreChannel fabric for this switch. When the FC-Map is configured, the switch discards the MAC addresses thatare not part of the current fabric. An FCF can assign Fabric Provide MAC Addresses (FPMA) to the CNAsconsisting of the FC-Map Value for the Fabric and the Fibre Channel ID (FCID) assigned during Fabric Login

For Cisco MDS 9500 switches, start at configure terminal below .Note

Before You Begin

For Cisco Nexus 7000 Series, you must be in the storage VDC to configure this feature.

SUMMARY STEPS

1. (Cisco Nexus 7000) switchto vdc vdc-id type storage2. configure terminal3. fcoe fcmap fabric-map

DETAILED STEPS


Switches to the storage VDC. The step is required for CiscoNexus 7000 Series only.

(Cisco Nexus 7000) switchto vdc vdc-id type storage

Example:switch# switchto vdc fcoe type storagefcoe#

Step 1


Example:fcoe# configure terminalfcoe(config)#

Step 2

Configures the global FC-Map. The default value is0x0EFC00. The range is from 0x0EFC00 to 0x0EFCFF. Usethe no fcoe map command to reset to the default value.

fcoe fcmap fabric-map

Example:fcoe(config)# fcoe fcmap 0x0efc2a

Step 3

This example shows how to configure the global FC-Map on a Cisco Nexus 7000 Series switch:switch# switchto vdc fcoe type storagefcoe# configure terminal


FCoEConfiguring the FC-Map

fcoe(config)# fcoe fcmap 0x0efc2a

This example shows how to configure the global FC-Map on a Cisco MDS 9500 switch:switch# configure terminalswitch(config)# fcoe fcmap 0x0efc2a

Configuring the Fabric PriorityThe FCoE switch advertises its priority. The priority is used by the CNAs in the fabric to determine the bestswitch to connect to.


Before You Begin


SUMMARY STEPS

1. (Cisco Nexus 7000) switchto vdc vdc-id type storage2. configure terminal3. fcoe fcf-priority fabric-priority

DETAILED STEPS


Switches to the storage VDC. The step is required for CiscoNexus 7000 Series only.



Step 1



Step 2

Configures the global fabric priority. The default value is 128.The range is from 0 (higher) to 255 (lower). Use the no fcoe

fcoe fcf-priority fabric-priority

Example:fcoe(config)# fcoe fcf-priority 42

Step 3

fcf-priority command to reset the global fabric priority to thedefault value.

This example shows how to configure the global fabric priority for a Cisco Nexus 7000 Series switch:switch# switchto vdc fcoe type storagefcoe# configure terminalfcoe(config)# fcoe fcf-priority 42


FCoEConfiguring the Fabric Priority

This example shows how to configure the global fabric priority for a Cisco MDS 9500 switch:switch# configure terminalswitch(config)# fcoe fcf-priority 42

Setting the Advertisment IntervalYou can configure the interval for Fibre Channel fabric advertisement on the switch.


Before You Begin


SUMMARY STEPS

1. (Cisco Nexus 7000) switchto vdc vdc-id type storage2. configure terminal3. fcoe fka-adv-period interval

DETAILED STEPS


Switches to the storage VDC. The step is required forCisco Nexus 7000 Series only.



Step 1



Step 2

Configures the advertisement interval for the fabric. Thedefault value is 8 seconds. The range is from 4 to 60seconds.

fcoe fka-adv-period interval

Example:fcoe(config)# fcoe fka-adv-period 8fcoe#

Step 3

This example shows how to configure the advertisement interval for the fabric on a Cisco Nexus 7000 Seriesswitch:switch# switchto vdc fcoe type storagefcoe# configure terminalfcoe(config)# fcoe fka-adv-period 42


FCoESetting the Advertisment Interval

This example shows how to configure the advertisement interval for the fabric on a Cisco MDS 9500 switch:switch# configure terminalswitch(config)# fcoe fka-adv-period 42

Disabling LAN Traffic on an FCoE LinkYou can disable LAN traffic on an FCoE link.

DCBX allows the switch to send a LAN Logical Link Status (LLS) message to a directly-connected CNA.Enter the shutdown lan command to send an LLS-Down message to the CNA. This command causes allVLANs on the interface that are not enabled for FCoE to be brought down. If a VLAN on the interface isenabled for FCoE, it continues to carry SAN traffic without any interruption.

SUMMARY STEPS

1. configure terminal2. interface ethernet slot/port3. shutdown lan4. (Optional) show interface5. (Optional) copy running-config startup-config

DETAILED STEPS




Step 1

Specifies an interface to configure, and enters interfaceconfigurationmode. Use ? to view a list of supported interfaces.

interface ethernet slot/port

Example:switch(config)# interface e 2/1switch(config-if)#

Step 2

Shuts down Ethernet traffic on the interface. If the interface ispart of an FCoE VLAN, the shutdown has no impact on the

shutdown lan

Example:switch(config-if)# shutdown lan

Step 3

FCoE traffic. Use no shutdown lan to reenable Ethernet trafficon this interface.

(Optional)Displays information about the interface.

show interface

Example:switch(config-if)# show interface

Step 4


FCoEDisabling LAN Traffic on an FCoE Link




Example:switch(config-if)# copy running-configstartup-config

Step 5

Verifying the FCoE ConfigurationTo display FCoE configuration information, perform one of these tasks:

PurposeCommand

Displays whether FCoE is enabledon the switch.

show fcoe

Displays the contents of the FCoEdatabase.

show fcoe database

Displays information about thevFC interfaces.

show interface vfc [number]

Displays the FCoE settings for aninterface or all interfaces.

show interface [interface number] fcoe

This example shows how to verify that the FCoE capability is enabled:switch# show fcoeGlobal FCF details

FCF-MAC is 00:0d:ec:6d:95:00FC-MAP is 0e:fc:00FCF Priority is 128FKA Advertisement period for FCF is 8 seconds

This example shows how to display the FCoE database:switch# show fcoe database------------------------------------------------------------------------------INTERFACE FCID PORT NAME MAC ADDRESS------------------------------------------------------------------------------vfc3 0x490100 21:00:00:1b:32:0a:e7:b8 00:c0:dd:0e:5f:76

This example shows how to display the FCoE settings for an interface.switch# show interface ethernet 1/37 fcoeEthernet1/37 is FCoE UP

vfc3 is UpFCID is 0x490100PWWN is 21:00:00:1b:32:0a:e7:b8MAC addr is 00:c0:dd:0e:5f:76


FCoEVerifying the FCoE Configuration

Additional References for FCoERelated Documents

Document TitleRelated Topic

Cisco NX-OS FCoE CommandReference for Cisco Nexus 7000and Cisco MDS 9500

Command reference

Cisco NX-OS Licensing GuideCisco NX-OS licensing

Standards and RFCs

TitleStandard/RFC

Fibre Channel Backbone 5T11 FC BB-5

MIBs

MIBs LinkMIB

To locate and downloadMIBs for selected platforms,Cisco IOS releases, and feature sets, use Cisco MIBLocator found at the following URL:

http://www.cisco.com/go/mibs

Technical Assistance

LinkDescription

http://www.cisco.com/supportThe Cisco Support website provides extensive onlineresources, including documentation and tools fortroubleshooting and resolving technical issues withCisco products and technologies.

To receive security and technical information aboutyour products, you can subscribe to various services,such as the Product Alert Tool (accessed from FieldNotices), the Cisco Technical Services Newsletter,and Really Simple Syndication (RSS) Feeds.

Access to most tools on the Cisco Support websiterequires a Cisco.com user ID and password.


FCoEAdditional References for FCoE


http://www.cisco.com/support

C H A P T E R 3Configuring FCoE VLANs and Virtual Interfaces


• Information About Virtual Fibre Channel (VFC) Interfaces, page 21

• Default Settings for FCoE, page 22



• Configuring Virtual Interfaces, page 25

• Verifying the Virtual Interface , page 32

• Example:Mapping VSANs to VLANs , page 33

• Additional References for FCoE, page 34

Information About Virtual Fibre Channel (VFC) InterfacesFibre Channel over Ethernet (FCoE) allows Fibre Channel and Ethernet traffic to be carried on the samephysical Ethernet connection between the switch and the servers.

The Fibre Channel portion of FCoE is configured as a virtual Fibre Channel interface. Logical Fibre Channelfeatures (such as interface mode) can be configured on virtual Fibre Channel interfaces.

A virtual Fibre Channel (vfc) interface must be bound to an interface before it can be used. The binding is toa physical Ethernet interface when the converged network adapter (CNA) is directly connected to the switchor port channel when the CNA connects to the Fibre Channel Forwarder (FCF) over a virtual port channel(vPC).

VF PortA virtual fabric (VF) port in an FCoE network acts as a fabric port that connects to a peripheral device (hostor disk) operating as an N port. A VF port can be attached to only one N port.


VE PortA virtual expansion (VE) port acts as an expansion port in an FCoE network. VE ports can connect multipleFCoE switches together in the network. You can bind a VE port to a physical ethernet port or a port channel.

Traffic is load balanced across equal cost E_Ports and VE_Ports based on SID, DID, and OXID. Traffic acrossmembers of a port channel that a VE_Port is bound to is load balanced based on SID, DID, and OXID.

Default Settings for FCoEThis table lists the default settings for FCoE parameters.

Table 5: Default FCoE Parameter Settings

DefaultParameters

Not installed, disabledFCoE feature

0E.FC.00FC-Map

128Fabric priority

8 secondsAdvertisement interval







Configuring FCoE VLANs and Virtual InterfacesVE Port







FCoE VDC









Shared Interfaces


Note





• Access mode




Configuring FCoE VLANs and Virtual InterfacesGuidelines and Limitations


Storage VDC






Note


where














Configuring FCoE VLANs and Virtual InterfacesCisco MDS 9500: Guidelines and Limitations

Configuring Virtual Interfaces

Mapping a VSAN to a VLANA unique, dedicated VLAN must be configured at every converged access switch to carry traffic for eachvirtual fabric (VSAN) in the SAN (for example, VLAN 1002 for VSAN 10, VLAN 1003 for VSAN 2, andso on). If you enableMST, youmust use a separateMultiple Spanning Tree (MST) instance for FCoEVLANs.

Youmust exit VLANmode to execute the configured commands on the Cisco Nexus 7000 Series Switches.Note

Before You Begin

• Ensure you have installed the correct license for FCoE.

• Ensure you have enabled FCoE.

• For Cisco Nexus 7000 Series Switches, ensure that you are in the storage VDC.

SUMMARY STEPS

1. configure terminal2. vsan database3. vsan vsan-id4. vlan vlan-id5. fcoe [vsan vsan-id]6. exit7. (Optional) show vlan fcoe8. (Optional) copy running-config startup-config

DETAILED STEPS




Step 1

Enters VSAN database configuration mode.vsan database

Example:switch(config)# vsan databaseswitch(config-vsan-db)#

Step 2


Configuring FCoE VLANs and Virtual InterfacesConfiguring Virtual Interfaces


Defines the VSAN. The VSAN number range is from 1 to4094.

vsan vsan-id

Example:switch(config-vsan-db)# vsan 200

Step 3

Enters VLAN configuration mode. The VLAN number rangeis from 1 to 4096.

vlan vlan-id

Example:switch(config-vsan-db)# vlan 200switch(config-vlan)#

Step 4

Enables FCoE for the specified VLAN and configures themapping from this VLAN to the specified VSAN. If you do

fcoe [vsan vsan-id]

Example:switch(config-vlan)# fcoe vsan 200

Step 5

not specify a VSAN number, a mapping is created from thisVLAN to the VSAN with the same number.

Exits VLAN configuration mode. You must exit this mode toexecute the configured commands on the Cisco Nexus 7000Series Switches.

exit

Example:switch(config-vlan)# exitswitch(config)#

Step 6

(Optional)Displays information about the FCoE configuration for aVLAN.

show vlan fcoe

Example:switch(config-vlan)# show vlan fcoe

Step 7



Example:switch(config-vlan)# copy running-configstartup-config

Step 8

This example shows how to map VLAN 200 to VSAN 200 on a Cisco MDS 9500 Series Switch:switch(config)# vlan 200switch(config-vlan)# fcoe vsan 200

This example shows how to map VLAN 300 to VSAN 300 on a Cisco Nexus 7000 Series Switches:switch(config)# switchto vdc fcoe_vdcswitch-fcoe_vdc# configure terminalswitch-fcoe_vdc(config)# vlan 300switch-fcoe_vd(config-vlan)# fcoe vsan 300

Creating a Virtual Fibre Channel InterfaceYou can create a virtual Fibre Channel interface. You must bind the virtual Fibre Channel interface to aphysical interface before it can be used.


Configuring FCoE VLANs and Virtual InterfacesCreating a Virtual Fibre Channel Interface

Before You Begin


• Ensure you have enabled FCoE.

SUMMARY STEPS

1. configure terminal2. interface vfc vfc-id3. switchport mode mode4. bind {interface {ethernet slot/port}}5. (Optional) show interface vfc6. (Optional) copy running-config startup-config

DETAILED STEPS




Step 1

Creates a virtual Fibre Channel interface (if it does not alreadyexist) and enters interface configuration mode. The vfc-id rangeis from 1 to 8192.

interface vfc vfc-id

Example:switch(config)# interface vfc 4switch(config-if)#

Step 2

Configures the switchport mode for a virtual Fibre Channelinterface. The mode is E or F. The default is F mode.

switchport mode mode

Example:switch(configif)# switchport mode e

Step 3

Binds the virtual Fibre Channel interface to the specifiedinterface. Use ? to see the supported interfaces and port channels.

bind {interface {ethernet slot/port}}

Example:switch(config-if)# bind interface ethernet1/4

Step 4

Use the no form of this command to unbind the virtual FibreChannel interface from the specified interface.

(Optional)Displays information about the virtual Fibre Channel interfaces.

show interface vfc

Example:switch(config-if)# show interface vfc

Step 5




Step 6


Configuring FCoE VLANs and Virtual InterfacesCreating a Virtual Fibre Channel Interface

This example shows how to bind a virtual Fibre Channel interface to an Ethernet interface:switch# configure terminalswitch(config)# interface vfc 4switch(config-if)# bind interface ethernet 1/4

This example shows how to delete a virtual Fibre Channel interface:switch# configure terminalswitch(config)# no interface vfc 4

Creating a Virtual Fibre Channel Port Channel InterfaceYou can create a virtual Fibre Channel port channel interface that automatically binds to the port channel withthe same interface number.

Before You Begin

• For the Cisco Nexus 7000 Series, ensure that you create the port channel interface before you create thevirtual Fibre Channel port channel interface.

• For the Cisco MDS 9500 switches, ensure that you create the Ethernet port channel interface before youcreate the virtual Fibre Channel port channel interface.

SUMMARY STEPS

1. configure terminal2. interface vfc-port-channel int-number3. switchport mode mode4. (Optional) show interface vfc-port-channel int-number5. (Optional) copy running-config startup-config

DETAILED STEPS




Step 1

Creates a virtual Fibre Channel interface (if it does not alreadyexist) that is bound to the port channel with the same interface

interface vfc-port-channel int-number

Example:switch(config)# interface vfc-port-channel2switch(config-if)#

Step 2

number and enters interface configuration mode. The int-numberrange is from 1 to 4096 (for Cisco Nexus 7000) or from 257 to4095 (for Cisco MDS 9500). The default switchport mode forthis interface is E.


Configuring FCoE VLANs and Virtual InterfacesCreating a Virtual Fibre Channel Port Channel Interface


Configures the switchport mode for a virtual Fibre Channelinterface. The mode is E or F. The default is F mode.

switchport mode mode

Example:switch(config-if)# switchport mode e

Step 3

(Optional)Displays information about the virtual Fibre Channel interfacesbound to port channel interfaces.

show interface vfc-port-channel int-number

Example:switch(config-if)# show interfacevfc-port-channel 2

Step 4




Step 5

Associating a Virtual Fibre Channel Interface to a VSANYoumust configure unique, dedicated VLAN at every converged access switch to carry traffic for each VirtualFabric (VSAN) in the SAN (for example, VLAN 1002 for VSAN 1, VLAN 1003 for VSAN 2, and so on). Ifyou enable MST, you must use a separate MST instance for FCoE VLANs.

Before You Begin

For Cisco Nexus 7000 Series Switches, ensure that you are in the storage VDC.

SUMMARY STEPS

1. configure terminal2. vsan database3. (Optional) vsan vsan-id4. Enter one of the following commands:

• vsan vsan-id interface vfc vfc-id

• vsan vsan-id interface vfc-port-channel vfc-id

5. (Optional) show vsan6. (Optional) copy running-config startup-config


Configuring FCoE VLANs and Virtual InterfacesAssociating a Virtual Fibre Channel Interface to a VSAN

DETAILED STEPS




Step 1

Enters VSAN configuration mode.vsan database

Example:switch(config)# vsan databaseswitch(config-vsan-db)#

Step 2

(Optional)Creates the VSAN. The vsan-id range is from 1 to 4094 and mustmap to a VLAN on the physical Ethernet interface that is bound tothe virtual Fibre Channel interface.

vsan vsan-id

Example:switch(config-vsan-db)# vsan 2

Step 3

Configures the association between the VSAN and virtual FibreChannel interface or virtual Fibre Channel port channel. The vsan-id

Enter one of the following commands:Step 4

• vsan vsan-id interface vfc vfc-id range is from 1 to 4094 and must map to a VLAN on the physicalEthernet interface or port channel that is bound to the virtual Fibre• vsan vsan-id interface vfc-port-channel

vfc-id Channel interface or virtual Fibre Channel port channel. The vfc-idrange is from 1 to 8192. Us the no form of this command todissassociate the connection between the VSAN and virtual FibreChannel interface or virtual Fibre Channel port channel.Example:

switch(config-vsan-db)# vsan 2 interfacevfc 4

(Optional)Displays information about the VSAN.

show vsan

Example:switch(config-vsan-db)# show vsan

Step 5



Example:switch(config-vsan-db)# copy running-configstartup-config

Step 6

This example shows how to associate a virtual Fibre Channel interface to a VSAN:switch# configure terminalswitch(config)# vsan databaseswitch(config-vsan-db)# vsan 2 interface vfc 4


Configuring FCoE VLANs and Virtual InterfacesAssociating a Virtual Fibre Channel Interface to a VSAN

Enabling VE Loopback ConfigurationThe VFID check verifies that the VSAN configuration is correct on both ends of a VE link. You can turn offthe VFID check for VE ports to allow VE loopback configuration between to VE ports on the same switch.

For Cisco MDS 9500 switches, start at Step 2, on page 31 .Note

Before You Begin


SUMMARY STEPS

1. (Cisco Nexus 7000) switchto vdc vdc-id type storage2. configure terminal3. fcoe veloopback

DETAILED STEPS


Switches to the storage VDC. The step is requiredfor Cisco Nexus 7000 Series only.


Example:# switchto vdc fcoe type storagefcoe#

Step 1



Step 2

Enables the VFID check for all VE ports.fcoe veloopback

Example:fcoe(config)# fcoe veloopback

Step 3

This example shows how to enable VE loopback for a Cisco Nexus 7000 Series switch:switch# switchto vdc fcoe type storagefcoe# configure terminalfcoe(config)# fcoe veloopbackThis example shows how to enable VE loopback for a Cisco MDS 9500 switch:switch# configure terminalswitch(config)# fcoe veloopback


Configuring FCoE VLANs and Virtual InterfacesEnabling VE Loopback Configuration

Verifying the Virtual InterfaceTo display configuration information about virtual interfaces, perform one of the following tasks:

PurposeCommand

Displays the detailed configuration of the specifiedFibre Channel interface.

show interface vfc vfc-id

Displays the status of all interfaces.show interface brief

Displays the mapping of FCoE VLANs to VSANs.show vlan fcoe

This example shows how to display a virtual Fibre Channel interface bound to an Ethernet interface:switch# show interface vfc 3vfc3 is up

Bound interface is Ethernet1/37Hardware is Virtual Fibre ChannelPort WWN is 20:02:00:0d:ec:6d:95:3fAdmin port mode is F, trunk mode is onsnmp link state traps are enabledPort mode is F, FCID is 0x490100Port vsan is 9311 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec0 frames input, 0 bytes0 discards, 0 errors

0 frames output, 0 bytes0 discards, 0 errors

Interface last changed at Thu May 21 04:44:42 2009

This example shows how to display the status of all the interfaces on the switch (some output has been removedfor brevity):switch# show interface brief-------------------------------------------------------------------------------Interface Vsan Admin Admin Status SFP Oper Oper Port

Mode Trunk Mode Speed ChannelMode (Gbps)

-------------------------------------------------------------------------------fc3/1 1 auto on trunking swl TE 2 --fc3/2 1 auto on sfpAbsent -- -- --...fc3/8 1 auto on sfpAbsent -- -- ---------------------------------------------------------------------------------Interface Status IP Address Speed MTU Port

Channel-------------------------------------------------------------------------------Ethernet1/1 hwFailure -- -- 1500 --Ethernet1/2 hwFailure -- -- 1500 --Ethernet1/3 up -- 10000 1500 --...Ethernet1/39 sfpIsAbsen -- -- 1500 --Ethernet1/40 sfpIsAbsen -- -- 1500 ---------------------------------------------------------------------------------Interface Status IP Address Speed MTU-------------------------------------------------------------------------------mgmt0 up 172.16.24.41 100 1500-------------------------------------------------------------------------------


Configuring FCoE VLANs and Virtual InterfacesVerifying the Virtual Interface

-------------------------------------------------------------------------------Interface Vsan Admin Admin Status SFP Oper Oper Port

Mode Trunk Mode Speed ChannelMode (Gbps)

-------------------------------------------------------------------------------vfc 1 1 F -- down -- -- --...

This example shows how to display the mapping between the VLANs and VSANs on the switch:switch# show vlan fcoeVLAN VSAN Status-------- -------- --------15 15 Operational20 20 Operational25 25 Operational30 30 Non-operational

Example:Mapping VSANs to VLANsThe following example shows how to configure the FCoE VLAN and a virtual Fibre Channel interface:

SUMMARY STEPS

1. Enable the associated VLAN and map the VLAN to a VSAN. For Cisco Nexus 7000, ensure you are inthe storage VDC.

2. Configure the VLAN on a physical Ethernet interface.3. Create a virtual Fibre Channel interface and bind it to a physical Ethernet interface. For Cisco Nexus 7000,

ensure you are in the storage VDC.4. Associate the virtual Fibre Channel interface to the VSAN. For Cisco Nexus 7000, ensure you are in the

storage VDC.5. (Optional) Display membership information for the VSAN.6. (Optional) Display the interface information for the virtual Fibre Channel interface.

DETAILED STEPS

Step 1 Enable the associated VLAN and map the VLAN to a VSAN. For Cisco Nexus 7000, ensure you are in the storage VDC.switch(config)# vlan 200switch(config-vlan)# fcoe vsan 200switch(config-vlan)# exit

Step 2 Configure the VLAN on a physical Ethernet interface.switch# configure terminalswitch(config)# interface ethernet 1/4switch(config-if)# spanning-tree port type edge trunkswitch(config-if)# switchport mode trunkswitch(config-if)# switchport trunk allowed vlan 1,200switch(config-if)# exit


Configuring FCoE VLANs and Virtual InterfacesExample:Mapping VSANs to VLANs

Step 3 Create a virtual Fibre Channel interface and bind it to a physical Ethernet interface. For Cisco Nexus 7000, ensure youare in the storage VDC.switch(config)# interface vfc 4switch(config-if)# bind interface ethernet 1/4

By default, all virtual Fibre Channel interfaces reside on VSAN 1. If the VLAN to VSANmapping is to a VSANother than VSAN 1, then proceed to Step 4.

Note

Step 4 Associate the virtual Fibre Channel interface to the VSAN. For Cisco Nexus 7000, ensure you are in the storage VDC.switch(config)# vsan databaseswitch(config-vsan)# vsan 200 interface vfc 4switch(config-vsan)# exit

Step 5 (Optional) Display membership information for the VSAN.switch# show vsan 200 membershipvsan 200 interfaces

vfc 4

Step 6 (Optional) Display the interface information for the virtual Fibre Channel interface.switch# show interface vfc 4

vfc4 is upBound interface is Ethernet1/4Hardware is Virtual Fibre ChannelPort WWN is 20:02:00:0d:ec:6d:95:3fPort WWN is 20:02:00:0d:ec:6d:95:3fsnmp link state traps are enabledPort WWN is 20:02:00:0d:ec:6d:95:3fAPort WWN is 20:02:00:0d:ec:6d:95:3fsnmp link state traps are enabledPort mode is F, FCID is 0x490100Port vsan is 2001 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec0 frames input, 0 bytes 0 discards, 0 errors0 frames output, 0 bytes 0 discards, 0 errorsInterface last changed at Thu Mar 11 04:44:42 2010

Additional References for FCoERelated Documents


Cisco NX-OS FCoE CommandReference for Cisco Nexus 7000and Cisco MDS 9500

Command reference


Configuring FCoE VLANs and Virtual InterfacesAdditional References for FCoE


Cisco NX-OS Licensing GuideCisco NX-OS licensing

Standards and RFCs

TitleStandard/RFC

Fibre Channel Backbone 5T11 FC BB-5

MIBs

MIBs LinkMIB

To locate and downloadMIBs for selected platforms,Cisco IOS releases, and feature sets, use Cisco MIBLocator found at the following URL:


Technical Assistance

LinkDescription

http://www.cisco.com/supportThe Cisco Support website provides extensive onlineresources, including documentation and tools fortroubleshooting and resolving technical issues withCisco products and technologies.

To receive security and technical information aboutyour products, you can subscribe to various services,such as the Product Alert Tool (accessed from FieldNotices), the Cisco Technical Services Newsletter,and Really Simple Syndication (RSS) Feeds.

Access to most tools on the Cisco Support websiterequires a Cisco.com user ID and password.




http://www.cisco.com/support

C H A P T E R 4Storage VDC on Cisco Nexus 7000 SeriesSwitches

This chapter contains the following sections:

• Information About Storage VDC, page 37



• Configuring FCoE VDCs, page 40

• Example: Storage VDC Configuration, page 46

Information About Storage VDCYou use a virtual device context (VDC) to separate LAN and SAN traffic on the same switch. A VDC allowsyou to maintain one physical infrastructure but separate logical data paths.

To achieve this configuration, you must perform the following tasks:

• Create a dedicated storage VDC.

• Allocate a VLAN range for FCoE.

• Optionally, share physical ports between the storage VDC and one other VDC.

For shared physical ports, the VDC where the ports are shared includes a VF Ethernet port that correspondsto the shared port. You cannot modify some details of that port because it must match the underlying sharedphysical port. If you move the source port to another VDC or delete the VDC, the shared ports are deletedand you must reconfigure them.

If the Ethernet VDC restarts or is suspended, any shared Ethernet ports are shut down in the correspondingVDC. These ports come up automatically once the Ethernet VDC is operational.

Note













FCoE VDC










Storage VDC on Cisco Nexus 7000 Series SwitchesLicensing Requirements for FCoE

Shared Interfaces


Note





• Access mode




Storage VDC






Note


where








Storage VDC on Cisco Nexus 7000 Series SwitchesCisco MDS 9500: Guidelines and Limitations







Configuring FCoE VDCs

Creating a Dedicated Storage VDC

Before You Begin


SUMMARY STEPS

1. configure terminal2. install feature-set fcoe3. system qos4. service-policy type network-qos policy-map name5. vdc vdc-name type storage6. system default switchport7. feature lldp8. (Optional) feature lacp9. (Optional) allocate interface ethernet int-numb10. (Optional) switchto vdc vdc-name11. (Optional) show feature-set12. (Optional) copy running-config startup-config

DETAILED STEPS




Step 1


Storage VDC on Cisco Nexus 7000 Series SwitchesConfiguring FCoE VDCs


Installs the FCoE feature-set.install feature-set fcoe

Example:switch(config)# install feature-set fcoe

Step 2

Enters system qos mode.system qos

Example:switch(config)# system qosswitch(config-sys-qos)#

Step 3

Enables no drop queue for FCoE class.service-policy type network-qos policy-map name

Example:switch(config-sys-qos)#service-policy typenetwork-qos default-nq-7e-policy

Step 4

Creates a dedicated storage VDC and enters VDCconfiguration mode. You can only enable storage features

vdc vdc-name type storage

Example:switch(config)# vdc fcoe-vdc type storageswitch(config-vdc)#

Step 5

in a storage VDC. You do not need to allow the feature-setor enable it in the storage VDC because this process ishandled automatically for a storage VDC.

Set ethernet ports to default access port mode.system default switchport

Example:switch(config-vdc)# system default switchport

Step 6

Enables the LLDP feature in the storage VDC.feature lldp

Example:switch(config)# feature lldp

Step 7

(Optional)Enables the LACP feature in the storage VDC.

feature lacp


Step 8

(Optional)Allocates interfaces to the storage VDC as a dedicatedFCoE port. You must allocate all interfaces in the port

allocate interface ethernet int-numb

Example:switch(config-vdc)# allocate interface ethernet2/1-2switch(config-if)#

Step 9

group. You must configure these interfaces in switchporttrunk mode as Spanning Tree Protocol (STP) edge ports.

(Optional)Switches to the storage VDC.

switchto vdc vdc-name

Example:switch(config-vdc)# switchto vdc fcoe-vdcswitch-fcoe-vdc#

Step 10

(Optional)Displays information about the feature-sets in this VDC.

show feature-set

Example:switch-fcoe-vdc# show feature-set

Step 11


Storage VDC on Cisco Nexus 7000 Series SwitchesCreating a Dedicated Storage VDC




Example:switch-fcoe-vdc# copy running-configstartup-config

Step 12

Cisco Nexus 7000:Licensing an FCoE ModuleYou must associate an FCoE license with an FCoE module to configure FCoE. You need one license for eachmodule configured for FCoE.

Before You Begin


SUMMARY STEPS

1. configure terminal2. license fcoe module module-number3. (Optional) show license usage FCOE-N7K-F132XP4. (Optional) copy running-config startup-config

DETAILED STEPS



Example:switch# configure terminalswtich(config)#

Step 1

Associates an FCoE license to a module.license fcoe module module-number

Example:swtich(config)# license fcoe module 2

Step 2

(Optional)Displays information about which FCoE modulesare using FCoE licenses.

show license usage FCOE-N7K-F132XP

Example:swtich(config)# show license usage FCOE-N7K-F132XP

Step 3



Example:swtich(config)# copy running-config startup-config

Step 4


Storage VDC on Cisco Nexus 7000 Series SwitchesCisco Nexus 7000:Licensing an FCoE Module

Allocating the FCoE VLAN Range

Before You Begin


• Ensure you are in the correct VDC.

SUMMARY STEPS

1. configure terminal2. vdc vdc-name type storage3. allocate fcoe-vlan-range vlan-range [from vdcs vdc-name]4. (Optional) show fcoe-vlan-range5. (Optional) copy running-config startup-config

DETAILED STEPS




Step 1

Enters VDC configuration mode. You can only enable storagefeature in a storage VDC. You do not need to allow the

vdc vdc-name type storage

Example:switch(config)# vdc fcoe-vdc type storageswitch(config-vdc)#

Step 2

feature-set or enable it in the storage VDC because this processis handled automatically for a storage VDC.

Allocates the VLANs that can be used for FCoE and mappedto a VSAN. You can optionally allocate the VLANs fromanother VDC.

allocate fcoe-vlan-range vlan-range [from vdcsvdc-name]

Example:switch(config-vdc)# allocate fcoe-vlan-range10-30

Step 3

(Optional)Displays information about the VLAN range allocated forFCoE.

show fcoe-vlan-range

Example:switch(config-vdc)# show fcoe-vlan-range

Step 4



Example:switch(config-vdc)# copy running-configstartup-config

Step 5


Storage VDC on Cisco Nexus 7000 Series SwitchesAllocating the FCoE VLAN Range

Related Topics

Cisco Nexus 7000: Guidelines and Limitations, on page 8

Allocating Shared InterfacesYou can share interfaces between a storage VDC and another VDC.

Before You Begin


• Ensure you are in the correct VDC.

• Ensure any shared interfaces are from an F-series module.

• Ensure you have allocated the FCoE VLAN range.

• Interfaces can only be shared between the storage VDC and one other VDC.

SUMMARY STEPS

1. configure terminal2. interface if-range3. switchport mode trunk4. spanning-tree port type edge trunk5. no shutdown6. vdc vdc-name type storage7. allocate shared interface if-range8. (Optional) show vdc shared membership9. (Optional) switchto vdc vdc-name10. configure terminal11. feature lldp12. interface if-range13. no shutdown14. (Optional) show interface if-range15. (Optional) copy running-config startup-config


Storage VDC on Cisco Nexus 7000 Series SwitchesAllocating Shared Interfaces

DETAILED STEPS




Step 1

Enters interface configuration mode for the interface inthe Ethernet VDC.

interface if-range

Example:switch(config)# interface ethernet 2/1

Step 2

Puts the Ethernet interface into trunk mode.switchport mode trunk

Example:switch(config-if)# switchport mode trunk

Step 3

Sets the interface to STP-type edge port to support STPLite for loop prevention.

spanning-tree port type edge trunk

Example:switch(config-if)# spanning-tree port type edgetrunk

Step 4

Administratively enables the Ethernet shared interface.no shutdown

Example:switch(config-if)# no shutdown

Step 5

Enters VDC configuration mode.vdc vdc-name type storage

Example:switch(config-if)# vdc fcoe-vdc type storageswitch(config-vdc)#

Step 6

Allocates interfaces that are shared with another VDCfor FCoE traffic. You must allocate the shared interfaces

allocate shared interface if-range

Example:switch(config-vdc)# allocate shared interfaceethernet 2/1

Step 7

to one of the VDC included in the FCoE VLANallocation. You can only use the shutdown or theswitchport trunk allowed vlan commands on sharedinterfaces in the storage VDC.

(Optional)Displays the interfaces that are shared for FCoE.

show vdc shared membership

Example:switch(config-vdc)# show vdc shared membership

Step 8

(Optional)Switches to the storage VDC.

switchto vdc vdc-name

Example:switch(config-vdc)# switchto vdc fcoe-vdcswitch-fcoe-vdc#

Step 9


Storage VDC on Cisco Nexus 7000 Series SwitchesAllocating Shared Interfaces



Example:switch-fcoe-vdc# configure terminalswitch-fcoe-vdc(config)#

Step 10

Enables the LLDP feature in the storage VDC.feature lldp

Example:switch-fcoe-vdc(config)# feature lldp

Step 11

Enters interface configuration mode for the sharedinterface in the storage VDC.

interface if-range

Example:switch-fcoe-vdc(config)# interface ethernet 2/1

Step 12

Administratively enables the FCoE shared interface.no shutdown

Example:switch-fcoe-vdc(config-if)# no shutdown

Step 13

(Optional)Displays information about the shared interface.

show interface if-range

Example:switch-fcoe-vdc(config-if)# show interfaceethernet 2/1

Step 14



Example:switch-fcoe-vdc(config-if)# copy running-configstartup-config

Step 15

Example: Storage VDC ConfigurationShared Interfaces in the Storage VDC!Enable associated features and interface to share:switch# configure terminalswitch(config)# feature lldpswitch(config)# feature lacpswitch(config)# system qosswitch(config-sys-qos)# service-policy type network-qos default-nq-7e-policy

switch(config-sys-qos)# interface ethernet 2/1switch(config-if)# switchport mode trunkswitch(config-if)# spanning-tree port type edge trunkswitch(config-if)# no shutdown

!Create Storage VDC and allocate resources:switch(config-if)# install feature-set fcoeswitch(config)# vdc fcoe_vdc type storage


Storage VDC on Cisco Nexus 7000 Series SwitchesExample: Storage VDC Configuration

switch(config-if)# allocate fcoe-vlan-range 10-20 from vdc switchswitch(config-vdc)# allocate shared interface ethernet 2/1

!Switch to storage VDC and bring up the shared interface:switch(config-vdc)# switchto vdc fcoe_vdcswitch-fcoe_vdc# configure terminalswitch-fcoe_vdc(config)# feature lldpswitch-fcoe_vdc(config)# interface ethernet 2/1switch-fcoe_vdc(config-if)# no shutdown

Ethernet 2/1 must be from an F-series module.Note



A P P E N D I X AFeature History by Platform

This chapter contains the following sections:

• Cisco Nexus 7000:Supported Features, page 49

• Cisco MDS 9500:Supported Features, page 49

Cisco Nexus 7000:Supported FeaturesFeature InformationReleaseFeature

Supports the T11-compliant FIP.5.2(1)FIP

This feature was introduced.5.2(1)FCoE

Supports F2 Series modules.6.1(1)FCoE

Supports F2e Series modules (SFP+ only).6.1(1)FCoE

Cisco MDS 9500:Supported FeaturesFeature InformationReleaseFeature

Supports the T11-compliant FIP.5.2(1)FIP

This feature was introduced.5.2(1)FCoE



Feature History by PlatformCisco MDS 9500:Supported Features

A P P E N D I X BGuidelines and Limitations


• Verified Limits for FCoE for the Cisco Nexus 7000 Series and Cisco MDS 9500 , page 51

• Cisco Nexus 7000: Guidelines and Limitations, page 51

• Cisco MDS 9500: Guidelines and Limitations, page 53

Verified Limits for FCoE for the Cisco Nexus 7000 Series andCisco MDS 9500

The configuration limits are documented in the Cisco Nexus 7000 Series NX-OS Verified Scalability Guide.






FCoE VDC










Shared Interfaces


Note





• Access mode




Storage VDC






Note


where





Guidelines and LimitationsCisco Nexus 7000: Guidelines and Limitations











Guidelines and LimitationsCisco MDS 9500: Guidelines and Limitations

b Cisco NX-OS FCoE Configuration Guide

Documents

Transcript of b Cisco NX-OS FCoE Configuration Guide