Issue

EMC Symmetrix disk array configuration in Common Serial Number Mode [C-bit setting]

Solution

Dynamic Multi-Pathing (DMP) works with the EMC Symmetrix disk array only if the disk array is configured in the Common Serial Number Mode [C-bit set]. Without this mode, DMP cannot provide multipathing functionality.

Until EMC Symmetrix started supporting devices above FFF, if digits 6 to 8 of the EMC Symmetrix disk serial number are zeroed out then the C-bit is set and the array is configured in the Common Serial Number Mode as required by DMP. But after EMC Symmetrix started supporting devices above FFF, they used the last bit to set it to 8 to represent a device above 0xFFF. This "8" is added with a logical "OR" operation so that you device would end with 378 (370 or 008) on port 7dA and 3A8 (3A0 or 008) on port 10dA.

With the "C" bit, the serial number must be identical (common) on all the ports. The "8" remains to represent a device above 0xFFF, the the rest is set to 0. 000 or 008 = 008.

The disk serial number can be obtained with various commands including the VERITAS vxdmpinq, vxdmpdebug command or the vxexplorer output.

/etc/vx/diag.d/vxdmpinq /dev/rdsk/c3t12d15s2Vendor id : EMCProduct id : SYMMETRIXRevision : 5266Serial Number : 1004E000a

or

/etc/vx/diag.d/vxdmpinq /dev/rdsk/c3t5006048C52A820D6d766s2Inquiry for /dev/rdsk/c3t5006048C52A820D6d766s2, evpd 0x0, page code 0x0 Vendor id : EMC Product id : SYMMETRIX Revision : 5772 Serial Number : 273E2008a

The serial number above shows that digits 6-8 are zeroed out. The array is setup in Common Serial Number Mode as required by DMP.

DMP needs to see identical device serial numbers from each path.If the C-bit is not set then DMP sees multiple disks of the same device through different paths.If the C-bit is set then DMP sees one unique disk device through multiple paths.

The layout for the EMC Symmetrix disk serial number is:

1004E000a (or) 273E2008aSSVVVDDPx

SS = Last 2 digits of the Symmetrix serial numberVVV = Symmetrix volume number in hex. 000-FFFDD = Director number [host adapter] 1-32P = Port number on director cardx = lower case letter which does not belong to serial number according to EMC.

Note: If the C-bit is not set, data on an existing disk can easily be wiped out by disk operations to the other path which appears like a different disk all together. Setting the C-bit nulls out digits 6 through 8 of the serial number.

Consult the Storage Foundation Hardware Notes for a complete list of DMP pre-requisites regarding EMC Symmetrix arrays.

Legacy ID

325308

Article URL http://www.symantec.com/docs/TECH70715

Common Serial Number(C) Host SCSI Compliance 2007(OS2007) SCSI-3 SPC-2 Compliance(SPC2) SCSI-3 compliance(SC3) Common serial number (C) Auto negotiation (EAN) enabled Fibrepath enabled on this port (ACLX) SCSI 3 (SC3) (Optional) Unique Worldwide Name (UWN) SPC 2 (Decal) (SPC2) SPC 2 flag set Common serial number (C) Auto negotiation (EAN) enabled Fibrepath enabled on this port (VCM) SCSI 3 (SC3) (Optional) PTOP for FC-SWORHDAD for FC-AL VCM if using Volume Logix (optional) UWN for Unique WWN SC3 for heterogeneous port sharing (optional) C or common serial number bit Common Serial Number(C) Host SCSI Compliance 2007(OS2007) SCSI-3 SPC-2 Compliance(SPC2) SCSI-3 compliance(SC3) For FC Switch Base Topology (FCSW), Enable Auto Negotiation(EAN), PointtoPoint(PP), Unique WWN(UWN). For FC Loop Base Topology (FCAL), Enable Auto Negotiation(EAN),Unique World Wide Name(UWN)

1. 1. What are the "port flags" settings for 5875 Enginuity and what do they mean?

Port flags in Enginuity are used to set behavior that enables a connected server/operating system combination to work with the Symmetrix port (seen as a disk drive to the server) and to optimize the operation of that server and disk combination. Symmetrix VMAX with 5875 is delivered with the correct default port flag settings for most users so that few or no changes will be needed during installation of the new system.EMCstrives to make setup and operation of our products as simple and easy as possible. We believe that Enginuity 5875 for VMAX is delivered to you with correct default port flag settings for most users and that few or no changes will be needed during installation of your new system. Here is a summary of the default flag settings for the most popular operating systems. and changes required by some of those operating systems.2. What Servers can connect to 5875 Enginuity without changing Port Flags?Operating EnvironmentPort Settings

EMCCelerraEnable 'ARB' and 'D' flags

IBMAIX5875 Defaults, Unchanged

INM iEnable 'AS4' flag

LINUXEnable 'D' flag

Hewlett-Packard OpenVMSEnable 'OVMS' flag

Hewlett-Packard True64Enable 'OVMS' flag, Disable 'SC3 flag

Hewlett-Packard HP-UX 11i V1 &V2Enable 'V' flag, Disable 'SC3' and 'OS07' flag

Hewlett-Packard HP-UX 11i V3Enable 'V' flag, Disable 'SC3'

Microsoft Windows Server 20085875 Defaults, Unchanged

Microsoft Windows Server 20035875 Defaults, Unchanged

Oracle Solaris5875 Defaults, Unchanged

VMwareESX5875 Defaults, Unchanged

3. What are some additional considerations? Directly connecting servers to the array, not using switches or otherSANcomponents, you are using Fibre-Channel Arbitrated Loop. This requires that the PP flag to be DISABLED and a Loop ID assigned to a particular number ranging from 0-126. FC-AL is used in certain markets and with some specialty servers.

Using Fujitsu (formerly Siemens), Novell Netware or Teradata systems, please look at the support matrix atEMC.com for specific recommendations.

Use of Clusters can impact port flag settings. Please look at ELab's Host Connectivity Guide of the particular operating system for information for details of a cluster's required port flag settings.

4. Port Flag usage notes: These new default values are in some cases different from the 5874 settings. Connection of a server to both 5874 and 5875 Enginuity may require changes to the 5875 defaults or the user to set the flags at the Initiator to match the previously installed array with its existing connection to the server. Note in general that changes of the flags will not become effective until the host is rebooted or FC connectivity is bounced.

Script behavior can change based upon these new 5875 default Port Flag settings. Existing scripts should be tested to ensure port flag settings and scripts are compatible.

Your VMAX is set to operate with the ACLX flag ENABLED, which enables masking of specific devices to specific ports on the array. Note that user must still provision ACLX volumes to enable hosts management via Solution Enabler.

The OS07 flag, ENABLED by default, can generate warnings (I/O retry) on Solaris, OpenVMS, and some other hosts. Some high performance or sensitive applications may be degraded by the additional retry.EMCbelieves we support the SCSI standard correctly, and retries should not occur, but that said, it can happen on some systems. Such environments should DISABLE this flag. Although the OS07 flag is on by default, specifically for VMWAREESXthis setting may need to be disabled.

Note:that if you DISABLE the OS07 flag and wish to re-ENABLE it, you must be sure that theSPC2 flag is ENABLED first. Also note that theSPC2 flag value should not be changed from DISABLED to ENABLED on installed servers as it changes the disk name utilized by the server and the disks will no longer be visible to that server.

If SCSI-3 persistent device reservation bit (PER) is used (note that this flag is assigned to devices and not to ports and so is not covered by this document), then the SC3 bit should be set. Please refer to the Elab Host Connectivity Guide for documentation on the platforms that can utilize the SCSI-3 reservation bit.For additional information regarding to port flags, specific flags for a given server, impacts of changing flags and any other known issues, please go toEMC.com and search PRIMUS for additional port flag information.

5. How did 5875EnginuityPortFlag Defaults Change from earlier releases?

The following is the table of default settings for the commonly set flags for 5875 Enginuity versus other recent Enginuity releases:FlagEnginuity 5773 Enginuity 5874 Enginuity 5875

PPEnabledEnabledEnabled

UWNEnabledEnabledEnabled

EANEnabledEnabledEnabled

CEnabledEnabledEnabled

SPC2EnabledEnabledEnabled

OS07DisabledDisabledEnabled

ACLXDisabledDisabledEnabled

SC3DisabledDisabledEnabled

6.What are the port flag descriptions?

Flags come in two types. The first type are flags set only on the arrays storage port. These flags affect the connectivity behavior of all servers connected to that port. The second type of flags can be set per storage port or initiator group, a group defined on the array as one or more initiators (HBAs) for a given server. With this type, the flags settings for the initiator group override, for the group, the settings on the port itself. In this way, a port can be shared by servers with different flag requirements.

The following table list where the flags should be set:FlagFlag DescriptionFlags set only attheStoragePortFlags that can be set attheStoragePortorInitiator Groups

PPWhen enabled, specifies a point-to-point (direct or switched)topology in the initialization sequence. When disabled (default),it is initialized as an arbitrated loop.YESN/A

EANWhen enabled, allows two fibre ports to handshakeand settle on an optimal speed for data transfer.YESN/A

ACLXWhen this flag is set, it enables the InitiatorGroups/Access Logix1 function on the port.YESN/A

UWNWhen enabled, ensures unique World Wide Names (WWNs)within the fibre environment by incorporating Symmetrix serialnumbers and port numbers for each WWN. Enabled by defaultfor all environment configuration changes and new environments.When disabled, WWNs do not have to be changed.YESN/A

NPWhen enabled along with the Hard Addressing flag, theFibre Channel director only uses hard-assigned addressingwhen it initializes the loop, otherwise, soft-assignedaddressing is used during loop initialization (the default).YESN/A

CWhen enabled, this flag causes the serial numberof a specific volume to be the same regardless ofthe port on which it is visible.N/AYES

VWhen enabled along with the Disk_Array flagfor HP-UX hosts, the volume setaddressing modeis selected. VSA mode allows octal addressing.N/AYES

ARBTypical behavior on a SCSI bus reset is to propagate the reset toall channels that share devices mapped to the port that receivedthe reset. When this bit is set, the reset only occurs to the port onwhich it is received, and is not broadcasted.N/AYES

DWhen this flag is enabled, a Unit Attention (UA) that is propagatedfrom another director does not flush the queue for this device on thisdirector. Used for hosts that do not expect the queue to be flushed ona SCSI 0629 sense (only on a Hard Reset).N/AYES

SC3When this flag is set, it alters the Inquiry data returned by any deviceon the port to report that Symmetrix supports SCSI -3. If this flag is notset, it reports that SCSI-2 is the supported protocol. This flag also enablethe support for Auto Contingence Allegiance which is required byIBMAIX.N/AYES

OS07The effect of the OS07 setting can be summarized as follows:1. Automated device discovery of newly configured devices and/or device paths.2. Automated creation of DSFs (device special files) for newly configured devices.3. Automated detection and adaptation of device attribute changes including LUN4. Capacity change event.N/AYES

SPC2This flag enables the support forSPC-2 SCSI standard thatare recommended for FC, FCoE, and iSCSI connectivityN/AYES

AS4This flag should be enabled for any AS/400 (IBMi)hosts connecting to the port.N/AYES

OVMSEnabled for an OpenVMS Fibre Channel connection.N/AYES

EMC VMAX Notes On Setting FA Port Flags ForVMwareJune 4, 2014David RingVMAX,VMwareEAN,EMC,esx,FA Port,FLAGS,OS2007,SCSI-3,SPC-2,VMAX,VMWAREWhen configuring a VMAX for VMware ESX connectivity, I will always firstly consult theEMC Simple Support Matrix EMC Symmetrix VMAX Series Director Bit Settingsfoundherefor the latest listing of port settings required. Below I will list the relevant VMware flags to be enabled on the Director FA ports which have ESX Hosts zoned:ACLX Allows Initiator Group/Access Logix function on the port.Common Serial Number (C) When enabled, this flag causes the serial number of a specific volume to be the same regardless of the port on which it is visible and also allows multi-path configurations.Unique WWN (UWN) When enabled, ensures unique World Wide Names (WWNs) within the fibre environment by incorporating Symmetrix serial numbers and port numbers for each WWN. Enabled by default for all environment configuration changes and new environments.Init Point to Point (PP) Specifies a point-to-point (direct or switched) topology.Enable Auto Negotiate (EAN) With EAN enabled this allows the FA port to Auto Negotiate the port speed with the switch port it is connected to.SCSI-3 Interface (SC3) SC3 provides SCSI-3 (Full range of SCSI-3 commands) support for the ESX hosts. Also note that SC3 is required for Linux VMs with RDMs attached.SCSI Support1 (OS2007) OS2007 enables automatic discovery of newly created LUNs/Paths. When Enabled, this flag provides a stricter compliance with SCSI standards for managing device identifiers, multi-port targets, unit attention reports, and the absence of a device at LUN 0. If using VPLEX 5.2 and higher, the OS2007 flag MUST be enabled.SCSI Primary Commands (SPC2) This Flag is required in order for the FA port to conform to industry standards resulting in an NAA identifier being used for the LUNs presented to the ESX hosts. To change the status the Port must be offline.Note: These Flags may already be enabled by default for the associated Director Ports and no additional configuration change is required. Consulting the ESM for default settings will verify this.FA Port Settings from Factory Bin File:Green = EnabledYellow = Disabled

Setting director port attributes via Unisphere:1. Select the Symmetrix system.2. Select System > Dashboard.3. From the Hardware panel, click Front End Directors to open the Front End Directors list view.4. Select the director, and click Set Port Attributes to open the Set Port Attributes dialog box.

5. Optional:Select a port whose flag settings you want to copy.

VIA SYMCLIto enable flags on FA Port 1F:1, create a text file as per the following example:symconfigure -sid xxxx -file c:\file.txtwhere file.txt has the following contents:set port 1F:1 Common_Serial_Number=enable, SCSI_3=enable, SPC2_Protocol_Version=enable, SCSI_Support1=enable;View 1F:1 Port configuration include -v in order to view the Flag settings:symcfg -sid xxxx list -fa 1F -p 1 -vList all Ports:symcfg -sid xxxx list -v -dir allWhile all the port flags described above can be set at a Director Port level, in some cases you may not want the enabled flags to affect different host types connected to the Port such as Linux or Windows, in this case it may be a better option to set the Flags at an Initiator Group level. By enabling the relevant host type Flag settings at the Initiator Group level you are only affecting the ESX Hosts included in the Initiator Group.The Initiator Group VMware Specific flags are :C, SC3, OS2007, SPC2In this way the flag settings for the initiator group override flag settings on the port itself, thus allowing for the port to be shared by servers with differing flag requirements.To set the required flags at an IG level use the following symaccess cmd:symaccess -sid xxxx -type initiator -name IGName set ig_flags on C,OS2007,SC3,SPC2There is also an option to set the Flags at the host initiator (WWN) level. For example setting the flags on a host initiator connected to port 1F:1 then use the following symmask commands:symmask -sid xxxx -wwn xxxxxxxxxxxxx -dir 1F -p 1 set hba_flags on C,OS2007,SC3,SPC2 -enablesymmask refreshTo display what director flags have been configured per host initiator:symmaskdb -sid xxxx list database -vNote:If there is a requirement to enable SPC-2 in a live VMware environment then please reference EMC Support material; as doing so may result in a LUN UUID change, depending on the version of VMware. One such solution to overcome this problem was to create a new path and datastore then perform an svmotion. Also changing the OS2007, SPC2, SC3 FA flags then the hosts affected by these changes will require a reboot as the SCSI protocols in use will have changed at the FA level and the HBA needs to reinitialize communication with the FA.

How to check bits / flags$ symmaskdb -sid list database -v -wwn (e.g 10000000c951c57c)$ symcfg list -sid 0426 -v -dir all | grep -E "[a-zA-Z1_ ]+\(.*\)" | grep abled | sort | uniqAuto_Negotiate(EAN) : DisabledAuto_Negotiate(EAN) : EnabledAvoid_Reset_Broadcast(ARB) : DisabledCommon_Serial_Number(C) : EnabledDisable_Q_Reset_on_UA(D) : EnabledEnviron_Set(E) : DisabledHP3000_Mode(B) : DisabledInit_Point_to_Point(PP) : EnabledNegotiate_Reset(N) : DisabledNon_Participating(NP) : DisabledOpenVMS(OVMS) : DisabledSCSI_Support1(OS2007) : DisabledSPC2_Protocol_Version(SPC2) : DisabledSequent(SEQ) : DisabledSiemens(S) : DisabledSoft_Reset(S) : DisabledSunapee(SCL) : DisabledUnique_WWN(UWN) : EnabledVCM_State(VCM) : EnabledVolume_Set_Addressing(V) : DisabledVolume_Set_Addressing(V) : EnabledDevice Masking (VCM) Config State : Enable