Post on 14-Apr-2018
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
1/123
Netwell
.
, ,
.
Netwell NetApp.
NETAPP TECHNICAL REPORT
NetApp
VMware vSphere
Vaughn Stewart, Larry Touchette, Mike Slisinger, Peter Learmonth,
Sitakanta Chaudhury, Julian Cates, NetApp
2010 | TR-3749 | Version 3.0
:
NetApp
NetApp
VMware vSphere 4.x 5.0.
Ethernet (iSCSI, NFS, FCoE),
, ,
NetApp vCenter NetApp Virtual
Storage Console 2.x.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
2/123
2
1 .................................................................................................................................................. 7
1.1 ............................................................................................................................ 7
1.2 .................................................................................................................................. 7
1.3 ........................................................................................................................... 8
1.4 ........................................................................................................................ 8
2 VMware ................................................................................................... 8
2.1 .......................................................... 8
2.2 ................................................................. 9
2.3 '80/20' ................................................................................................................................. 9
......................................................................................... 9
( ) ............................................... 9
2.4 VMFS .............................................................................................................................. 10
Spanned VMFS ............................................................................................................ 11
VMFS LUN NetApp....................................................................................................... 11
2.5 NFS ................................................................................................................................. 12
NFS NetApp ................................................................................................................. 12
2.6 Raw Device Mapping ........................................................................................................................ 13
RDM LUN NetApp ........................................................................................................................... 13
2.7 ...................................................................... 14
2.8 VMware ........................................................................................ 15
Thick........................................................................................................... 15
Thin ............................................................................................................ 16
Eager Zeroed Thick.................................................................................... 16
2.9 .......................................................................... 17
2.10 Thin Provisioning ................................................................................................. 18
Thin Provisioning NetApp ..................................................................................................... 18
2.11 ............................................................................. 18
VMFS RDM LUN ........................................................................ 20
NFS ................................................................. 20
VMware ......................................................................................... 21
2.12 vStorage Array Integration VMware vSphere ............................................................................. 21
vStorage APIs for Array Integration (VAAI) ......................................................................................... 21
Storage I/O Control (SIOC) .................................................................................................................. 22
.................................................................... 23
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
3/123
3
3 ................................................................................................. 25
...................................................... 25
3.1 SAN NAS ................................................................................................. 25
3.2 Fibre Channel ............................................................................................ 26
........................................................................................................ 26
...................................................................................... 26
3.3 Ethernet..................................................................... 27
VLAN Tagging 802.1Q ................................................................................................................... 27
Flow Control ........................................................................................................................................ 27
Spanning Tree .................................................................................................................... 29
IP- ................................................................................................. 30
Jumbo Frames ................................................................................................................. 30
Ethernet ................................................. 32
NetApp Virtual Interfaces (EtherChannel) ........................................................................................... 32
3.4 Cisco Nexus 1000V VNetwork Distributed Switch............................................................................ 33
Cisco Nexus 1000V System VLAN ........................................................................................................ 33
Jumbo Frames ................................................................................................................. 33
3.5 IP-, .............................. 34
3.6 Multiswitch Link Aggregation ................................................. 36
MSLA ......................................................................................................................... 36
..................................................................................................................... 36
IP- NetApp ................................................................ 37
VMkernel ESX/ESXi ............................................ 41
3.7 Ethernet-............................... 43
Single Mode ......................................................................................................................................... 43
Single Mode EtherChannel ............................................................. 43
Single Mode EtherChannel ................................................................... 43
multimode ......................................................... 44
..................................................................................... 44
VMkernel ............................................................................................................. 44
ESX iSCSI ...................................... 45
ESX NFS........................................ 46
............................................................................................................................ 46
IP- NetApp................................................................ 46 VMkernel ESX/ESXi ............................................ 50
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
4/123
4
3.8 Multiple TCP Sessions iSCSI ............................................................... 51
iSCSI VMKernels Multiple TCP Sessions iSCSI................... 52
4 .................................................................................. 55
4.1 : ............................................... 55
NetApp VCenter ............................................................................................................ 57
4.2 ................................................................................................ 57
4.3 NetApp ................................................................................... 57
RAID-DP ................................................................................................................. 58
Aggregates ........................................................................................................................................... 58
Flexible .............................................................................................................................. 59
LUN ...................................................................................................................................................... 59
........................................................................................................................ 59
4.4 NetApp........................................................................... 59
NetApp System Manager .................................................................................................................... 59
Data ONTAP ......................................................... 60
............................................................................ 60
NetApp System Manager ...................................................... 61
NetApp Operations Manager ................................................ 62
TCP/IP, VSC ............................................................ 62
4.5 VSC 2.0 ................................................................... 63
.......................................................................................... 63
4.6 Provisioning and Cloning VSC 2.0 .................................................... 65
................................................................................................................................. 66
5 vSphere .................................................................... 68
5.1 vCenter ............................................................... 68
NetApp Virtual Storage Console 2.0 vCenter plug-in ...................................................... 68
5.2 Virtual Storage Console 2.0 ............................................................................................ 70
VSC 2.0............................................................................................................................... 70
5.3 Virtual Storage Console ..................................... 72
5.4 ESX/ESXi .............................................................................. 73
5.5 Provisioning and Cloning ................................... 74
5.6 Provisioning and Cloning ........................................................ 76
5.7 vCenter .................................................................................. 77
5.8 ......................................................... 80 ..................................................................................... 80
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
5/123
5
VM NetApp Snapshots ....................................... 81
: VMkernel swap ............ 81
: VM swap/Pagefile ............................... 83
5.9 vCenter ................................................................................... 84
5.10 vCenter ............................................................. 85
NFS ............................................................................................................ 86
Capacity ................................................................................................................................. 86
Volume .................................................................................................................................. 87
Deduplication ........................................................................................................................ 87
6 ..................................................... 88
6.1 VM OS Windows....................................................... 88
Windows ..................... 88
...................................................................................................... 88
6.2 ............................................. 89
VM SCSI BUS ................................................................................................................ 89
6.3 ..................................................... 89
VM VMFS .......................................................... 89
.................................................................................................................. 90
............................................................................. 90
6.4 VM ........................................................... 90
...................................................................................... 90
6.5 ............................................................................................... 91
OS Windows ................................................................................ 91
NetApp MBRtools: ................................................. 91
6.6 VM ................................................................. 92
VM ........................................................................................... 92
NetApp MBRtools .............................. 92
............................... 92
6.7 VM ...................................................... 93
VMDK VM DISKPART .......................... 93
6.8 VM ............................................................................... 93
(VMDK) ........................................................................................ 93
OS (NTFS EXT3) ................................................. 95
OS ................................................................................... 957 ............................................................................................. 95
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
6/123
6
7.1 , ...................................................................... 95
7.2 NetApp ........................................................... 96
8 vSphere 5 ............................................................................................................................ 99
8.1 VMware ESXi .................................................................................................................................... 99
8.2 VMware vCenter Server ................................................................................................................... 99
8.3 .............................................................................................................. 100
8.4 Host Profiles ................................................................................................................................... 101
9 vSphere 5 ............................................................................................................................ 101
9.1 .................................................................................................. 101
9.2 VMFS 5 ............................................................................................................................................ 103
9.3 NFS vSphere 5 ....................................................................................................... 104
NFS Heap Memory .............................................. 104
.................................................................................. 104
IP- NFS ............................................ 104
9.4 Storage Distributed Resource Scheduler (Storage DRS) ................................................................ 105
Datastore Cluster (PODs) .................................................................................................................. 105
Storage DRS .............................................. 107
..................................................................................................... 108
(Affiity Rules) Maintenance Mode ...................................... 109
SDRS Data ONTAP................ 109
NetApp Snapshot SDRS .................................................................................................................. 110
NetApp SDRS ........................................................................................................ 110
NetApp Datastore Thin Provisioning SDRS ..................................................................................... 111
9.5 vSphere Storage I/O Control .......................................................................................................... 111
10 vSphere 5 .................................................................................................................... 112
10.1 vSphere Standard vSwitches ........................................................................................................ 112
MTU/Jumbo Frames .......................................................................................................................... 112
Flow Control ...................................................................................................................................... 113
10.2 Multiple TCP Sessions iSCSI ........................................................... 114
11 ............................................................................................................................................... 120
12 ............................................................................................. 121
................................................................................................................................................... 121
NetApp .................................................................................................................................................. 121
VMware ................................................................................................................................................. 12213 ............................................................................................................................ 123
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
7/123
7
1
Virtual Infrastructure
1.1
NetApp
, .
, ,
, ,
,
.
NetApp
VMware vSphere. NetApp VMware
2001 . NetApp
Data ONTAP ESX/ESXi Server.
.
.
1.2 , , ,
. NetApp VMware ,
, , , ,
,
,
, . NetApp ,
( )
.
NetApp VMware vSphere. ,
,
NetApp, NetApp c vSphere.
: NetApp vSphere Data ONTAP
7.3.1.1 . Data ONTAP,
.
, NetApp professional services
, .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
8/123
8
, .
1.3
, NetApp
Technical Library,
, VMware Virtual Infrastructure.
, , VMware ESX/ESXi Server 4.0,
vCenter Server 4.0, NetApp Data ONTAP 7G.
, ,
.
1.4
, TR-3749, ,
NetApp, NetApp , VMware.
,
-, .
,TR-3880: CLI Configuration Processes
for NetApp and VMware vSphere Storage Arrays Running Data ONTAP and ESX/ESXi Server.
.
2 VMware
Virtual Infrastructure
2.1
VMware ESX
: VMware Virtual Machine File System (VMFS),
Network-attached Storage (NAS), , raw
device mappings (RDM). ,
VMware ,
VMware, High-Availability (HA), Distribution Resource
Scheduler (DRS), VMotion, Fault Tolerance (FT).
.
VMware
, .
.
http://media.netapp.com/documents/tr-3880.pdfhttp://media.netapp.com/documents/tr-3880.pdfhttp://media.netapp.com/documents/tr-3880.pdfhttp://media.netapp.com/documents/tr-3880.pdfhttp://media.netapp.com/documents/tr-3880.pdfhttp://media.netapp.com/documents/tr-3880.pdf7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
9/123
9
2.2
, ,
.
, ,
(datastore). - ,
,
.
, , ,
.
VMware NetApp .
,
,
. SAN NAS,
, ,
.
, Fibre Channel (FC) Ethernet (NFS, iSCSI, FCoE),
NetApp ,
,
, .
, , .
2.3 '80/20'
80/20, 80%
. 20%
business-critical, , , ,
, .
.
.
VM -, VM
.
, ().
( )
.
-.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
10/123
10
, .
, , .
NFS,
, , SAN,
VM.
; ,
NFS/ VMFS.
, ,
, ,
80/20, NetApp VMware
, ,
.
2.4 VMFS
VMware Virtual Machine File System (VMFS)
,
. VMFS LUN-, Fibre Channel,
iSCSI, Fibre Channel over Ethernet (FCoE). VMFS LUN-
ESX .
1) ESX VMFS FC, FCoE, iSCSI.
VMFS VMware
. (shared) ,
VMware ,
.
VMware vCenter Server.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
11/123
11
, ,
, VMFS.
NetApp ,
,
VM.
, . ,
, -
,
- .
Spanned VMFS
VMware VMFS, LUN-
, spanned datastore. spanned datastore
2TB LUN,
- (I/O queues)
LUN. ,
LUN ( -
), - .
NetApp, OS Data ONTAP
, NetApp
spanned VMFS datastores.
2) ESX, spanned VMFS.
VMFS LUN NetApp
NetApp VMFS
, thin provisioning;
; , ;
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
12/123
12
, Site Recovery Manager (SRM), SnapManager for
Virtual Infrastructure, Rapid Cloning Utility (RCU), Virtual Storage Console (VSC),
SANscreen VM Insight. LUN, ,
VMFS ,
.
2.5 NFSvSphere NFS
ESX.
VMFS. NFS, NetApp,
, ,
.
3 . , ,
VMFS,
- (I/O queue), NetApp FAS.
3) ESX, NFS.
NFS NetApp
VMware NFS NetApp
, ,
,
, , , Fibre Channel.
VM
. NFS,
,
, ,
, . NFS,
VMware WAFL,
NetApp.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
13/123
13
VM, , VM
, thin provisioning ,
,
, .
NetApp , Site Recovery Manager (SRM),
SnapManager for Virtual Infrastructure (SMVI), Rapid Cloning Utility (RCU), Virtual Storage Console
(VSC).
2.6 Raw Device Mapping
ESX LUN-
, , , P2V clustering
. raw device
mapping (RDM)
Fibre Channel, iSCSI, Fibre Channel over Ethernet (FCoE). , ESX
VM . VMFS NFS,
RDM VM. .4
.
RDM , physical-to-
virtual, , s, Microsoft Cluster Server (MSCS). RDM
LUN ,
-,
.
4) ESX, VM RDM LUN FC iSCSI.
RDM LUN NetApp
RDM : (physical)
(virtual). VMware, VMotion
HA DRS.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
14/123
14
SCSI, VM.
MSCS VMware.
NetApp RDM thin provisioning
, , ,
SnapDrive,
SnapManager, RDM LUN FlexClone.
: , MSCS,
Most Recently Used(MRU). Round Robin (RR)
VMware RDM LUN MSCS.
2.7
.
. VMware
VMware ESX and ESXi Server Configuration Guide.
1) .
FC/FCoE iSCSI NFS
VMFS RDM VMFS RDM NetApp WAFL
LUN-
256 256 64
64TB 64TB 16TB 100TB*
LUN/ NAS
2TB 512b 2TB 512b 16TB 100TB*
queue
depth LUN
64 64 N/A
4 8Gb FC 10Gb
Ethernet
1 10Gb Ethernet 1 10Gb Ethernet
* 100TB 64-bit aggregates
NetApp.
2) VMware.
FC/FCoE iSCSI NFS
VMotion Storage VMotion
VMware HA
DRS
VCB
MSCS VM , RDM
Shared LUN
GOS
Fault Tolerance , eager-zeroed
thick VMDK
Virtual Mode RDM**
, eager-zeroed
thick VMDK
Virtual Mode RDM**
, eager-zeroed
thick VMDK
Site Recovery Manager
Thin Provisioned VM *VMware Native Multipathing N/A
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
15/123
15
Boot from SAN HBA
* VM NetApp NFS
** NetApp SnapDrive for Windows virtual-mode RDM devices
3) NetApp.
FC/FCoE iSCSI NFS
Thin Provisioning RDM RDM
SANscreen VM Insight
SnapDrive OS *
ESX Host Utilities/VSC 2.0
Backup Recovery VSC 2.0
Provisioning Cloning VSC 2.0 * NFS- VMDK SnapDrive
SnapDrive.
4) .
FC/FCoE iSCSI NFS
Replicated backup SRM
SnapMirror RDM RDM VM
SnapVault RDM RDM VM
VMDK
VCB VCB VCB, VIC FileExplorer
VMDK
VCB,
Windows
VCB,
Windows
VCB,
NDMP VM
2.8 VMware
,
vSphere. , , , ,
.
Thick
,
VM. ,
. VMDK
. , , , ,
, , .
,
, .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
16/123
16
5) thick VMDK.
Thin
thick, ,
. ,
VMDK (chunks),
. VMFS 1MB 8MB (
), NFS 4KB.
VMFS , SCSI locks
, .
, , ,
.
6) thin VMDK.
thick, thin VMDK . ,
.
, ,
.
thickthin- , ,
- ,
,
, , thin,
.
Eager Zeroed Thick
thick,
; , thick thin, eager-zeroed thick
,
.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
17/123
17
7) eager-zeroed thick VMDK.
2.9
NetApp ,
, VMware thin-provisioning.
FAS data deduplication thin provisioning VMFS RDM LUN
FAS. NetApp, -
, ESX.
: , NetApp NFS, thin-provisioned
VMDK VMware.
8) , VM NetApp NFS thin provisioned
.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
18/123
18
2.10 Thin Provisioning
, ,
, VMware .
, ,
,
.
100% ,
, oversubscribe
, , (
CPU, , , ).
thin provisioning.
; thin provisioning
. ,
thin-provision , ,
, VM
.
, ,
,
.
thin provisioning oversubscribing , VM
, .
Thin Provisioning NetApp
thin provisioning NetApp thin
provisioning VMDK VMware, LUN-, VMFS,
, VMDK
( thick-, thin-). , LUN-,
RDM, s thin provisioned.
NetApp thin-provisioned LUN-,
FlexVol, thin provision , 2x LUN.
LUN , FlexVol (quota).
, LUN FlexVol aggregate.
2.11
VMware
(VM templates). VM template
VM (.vmx) (.vmdk),
, -,
.
,
, ,
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
19/123
19
.
VM. 9
vSphere.
9) .
NetApp FAS data deduplication.
NetApp FAS data deduplication,
VMware , . , ,
NetApp FAS , ,
.
,
VMware.
NetApp FAS , , CPU ESX.
. 10
vSphere.
,
,
. , NetApp
OS ,
.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
20/123
20
10) .
VMFS RDM LUN
LUN- .
LUN ,
LUN- . ,
,
LUN-, ,
LUN-.
, LUN-, NetApp LUN
thin provisioning. ,
, VMware
, LUN, LUN-
, , (thick-) thin-
provisioned. NetApp Virtual Storage Console (VSC) VI
.
NetApp , thin-provisioned LUN-, FlexVol, thin provision , 2x
LUN. LUN-, FlexVol
(quota). , LUN FlexVol
aggregate
NFS
LUN-, NFS,
VMware.
VI.
- .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
21/123
21
VMware
NetApp vCenter plug-ins, VMware ,
, .
5.
2.12 vStorage Array Integration VMware vSphere
vSphere 4.1 VMware
vSphere. : vStorage APIs for array
integration (VAAI), storage I/O control(SIOC),
.
vStorage APIs for Array Integration (VAAI)
VAAI , ,
. VAAI
, (CPU, , -,
) , , , .
VAAI VMFS.
NFS NetApp, ,
, ,
Virtual Storage Console FlexClone. VAAI
VMFS. VAAI :
Full copy: ESX/ESXi , VAAI ,
.
vMotion.
Block zeroing:, eager-zeroedthick VMDK, .
VAAI
,
.
Hardware-assisted locking: VMFS , ,
,
. VAAI ,
SCSI-2 reservations, .
,
,
VMFS, .
VAAI .
VAAI NetApp, Data ONTAP
8.0.1 ( . VAAI Data ONTAP ESX/ESXi. 2.0 Virtual Storage
Console , VAAI .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
22/123
22
Storage I/O Control (SIOC)
Storage I/O Control, vSphere 4.1, quality of service
, ,
, CPU . SIOC ,
VM ,
VM, ,
IOPS, , . SIOC
VMFS, FC iSCSI.
SIOC ,
VM . VM Resources VM Edit
Settings. VMs
IOPS.
11) SIOC VM vSphere.
, SIOC ,
.,
, VM
. , (latency). 30ms,
. low, normal, high; 500, 1000,
2000 resource shares, , *.
resource shares , ,
SIOC
. , SIOC , VM 1000 shares
, VM 500 shares.
, VM,
.
* resource shares , .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
23/123
23
SIOC vSphere, , virtual
machine.
VM IOPS.
maximum IOPS VM vSphere
VM IOPS, latency, ,
. , VM MB/,
IOPS, -
: , VM -
8k 10MB/, maximum IOPS
VM 1280. : MB/sec I/O size = IOPS.
: 10240000 8000 = 1280.
, vSphere 4.1
, vCenter,
esxtop. VI3
vCenter
.
NetApp, , FC
NetApp, LUN ,
vCenter performance monitor .
LUN- ,
;
vSphere 4.0. ,
NFS ,
VI3 vSphere 4.0. , VM vSphere 4.0
. 12.
12) , vSphere 4.0.
, vSphere 4.1,
- , FC
FibreChannel. : FC, FCoE,
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
24/123
24
iSCSI, NFS. VM
VM . . 13
, vSphere 4.1.
13) , vSphere 4.1.
NAS ,
NFS , VM
. . 14 , vSphere 4.1.
14) , vSphere 4.1.
5) ,
, vSphere.
vCenter ESXTOP*
ESX/ESXi
Datastore (throughput)
(latency)
FC, iSCSI, NFS 4.1 4.1
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
25/123
25
Storage Adapter (throughput)
(latency)
FC 4.1 4.0+
Storage Path (throughput)
(latency)
FC 4.1 4.0+
LUN (throughput)
(latency)
FC, iSCSI 4.0+ 4.0+
VM Datastore (throughput) (latency)
FC, iSCSI, NFS 4.1 4.1
Virtual Disk (throughput)
(latency)
FC, iSCSI, NFS 4.1 4.1
* esxtop ESXi vSphere CLI resxtop.
3
Virtual Infrastructure
. ,
Ethernet, .
Ethernet . -, Fibre Channel , Fibre
Channel Protocol (FCP). ,
. -,
Ethernet (
converged networks) Ethernet
, , .
:
,
3.1 SAN NAS
vSphere SAN NAS multipathing. ESX/ESXi, NFS
, , iSCSI, FC, FCoE ,
multipathing (, iSCSI ).
Multipathing NFS
LUN masking( iSCSI FC, FCoE) ,
VMware .
, RCU,
.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
26/123
26
3.2 Fibre Channel
Fibre Channel
VMware. FC ,
, ESX 2.0. , FC
,
VMware Fibre Channel NetApp.
ESX, NetApp SAN
host bus adapter (HBA). FCoE
converged network adapter(CNA). HBA/CNA initiator(ESX/ESXi)
target (NetApp). ,
World Wide Name (WWN).
WWN, , LUN NetApp.
NetApp, VMware
ESX/ESXi .
VMware FC VMware Fibre Channel SAN Configuration Guide.
SAN,
-
(zoning). SAN zoning Fibre Channel
Fibre Channel.
(hard zoning) (soft zoning).
port zoning, . LUN ,
. port zoning, ,
. , port zoning
.
WWN zoning, World
Wide Port Names (WWPNs) , .
WWPN zoning ,
.
NetApp VMware single initiator, multiple
storage target.
FC FCoE. , WWN IQN
ESX, Storage Adapters Configuration ESX/ESXi
vCenter Server WWN.
http://www.vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_san_cfg.pdfhttp://www.vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_san_cfg.pdfhttp://www.vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_san_cfg.pdfhttp://www.vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_san_cfg.pdf7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
27/123
27
15) WWPN IQN vCenter.
3.3 Ethernet
10GBE Data Center Ethernet (DCE)
NetApp Data ONTAP VMware ESX/ESXi 4, 10GbE. 10GbE
, ,
, blade-.
VMware compatibility guide.
VLAN Tagging 802.1Q
VLAN,
VLAN VLAN- VLAN
tagging.
, , , ,
blade-, VLAN- . channel
ESX . VLAN-, IP- VLAN-
. NetApp ,
- VLAN, VMkernel IP-
VMotion VLAN.
VLAN- VLAN tagging , IP-
. NFS IP-, IP
storage VLAN. NetApp
iSCSI / VLAN.
, IP-. VLAN-
, ,
.
Flow Control
Flow control
, ,
. Flow control
ESX/ESXi, FAS, .
, 10GbE, NetApp flow control (send
off receive off), .
http://www.vmware.com/resources/compatibility/search.php?action=base&deviceCategory=iohttp://www.vmware.com/resources/compatibility/search.php?action=base&deviceCategory=iohttp://www.vmware.com/resources/compatibility/search.php?action=base&deviceCategory=io7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
28/123
28
, Gigabit Ethernet,
, NetApp flow control
, ESX NetApp,
send.
Flow control FAS
FilerView or NetApp System Manager ( . 16). , X1139A/X1140A Unified Target
Adapter flow control full.
16) NetApp System Manager > Edit Network Interface >Advanced settings > Flow Control.
flow control ESX ESXi ethtool
esxcfg-module, NIC ESX.
ethtool :
[root@esx8 ~]# ethtool -A vmnic2 autoneg off rx off tx off
:
[root@esx8 ~]# ethtool -a vmnic2
Pause parameters for vmnic2:
Autonegotiate: offRX: off
TX: off
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
29/123
29
autonegotiate RX TX.
, autonegotiate (flow
control), , .
/etc/rc.local .
flow control vSphere, esxcfg-module
Intel e1000, VMware KB 1013413.
NetApp flow control
send off receive off.
( 10G Ethernet)
, ESX FAS Desired ,
, send off receive on.
17) flow control.
Spanning Tree
Spanning Tree Protocol (STP) ,
. OSI STP OSI Layer-2. STP
, , ,
bridge loops, /
. bridge loops,
(flood).
ESX NetApp Ethernet, NetApp
, Ethernet , ,
RSTP edge ports Cisco - portfast.
Cisco portfast
802.1q VLAN trunking ESX NetApp,
spanning-tree portfast trunk.
edge port RSTP , edge
port .
Cisco portfast. ,
edge port
portfast.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1013413http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1013413http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1013413http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=10134137/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
30/123
30
IP-
, , NetApp IP-
, .
.
Jumbo Frames
Ethernet, maximum transmission unit (MTU) . MTU 1500 .
,
. jumbo frame, 9000 .
jumbo frames ,
ESX ,
.
: FCoE , baby jumbo frame
(MTU size 2112 ), , FCoE.
jumbo frames
, ,
. ( ) jumbo frames (, ,
NetApp ESX), jumbo frames ( Ethernet).
, jumbo frames,
VLAN trunking, - . Ethernet
, , Ethernet header(18 ),
VLAN tag (4 , VLAN). , jumbo frames ( )
Ethernet ( 9000 ). ,
, VLAN, 9018
9022 ( VLAN tagging). - ,
( Ethernet)
.
, ,
9216 . ,
, Ethernet , (18 ), VLAN tag (4 ),
.
jumbo frame
, ,
, .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
31/123
31
18) MTU jumbo frames.
ESX/ESXi vSwitch VMkernel,
jumbo frames.
jumbo frames vSwitch, MTU size,
:
1. ESX/ESXi ( vSphere CLI).2. MTU size:
vicfg-vswitch -m
MTU vSwitch.
MTU MTU,
, vSwitch.
3. :vicfg-vswitch -l
jumbo frames VMkernel, .
1. ESX/ESXi ( vSphere CLI).2. MTU size :
esxcfg-vmknic a I -n -m
VMkernel jumbo frame.
3. :esxcfg-vmknic l
, esxcfg- vicfg- . , , ,
, ESX SSH ( )
vSphere CLI ( Windows Linux, appliance
VM: vSphere Management Assistant, vMA) ESX ESXi. vMA
esxcfg-* vicfg-* command. vMA
vSphere CLI :
vMA:www.vmware.com/appliances/directory/178973 vSphere CLI:www.vmware.com/support/developer/vcli/
MTU size FAS, (
16 Flow Control):
http://www.vmware.com/appliances/directory/178973http://www.vmware.com/appliances/directory/178973http://www.vmware.com/appliances/directory/178973http://www.vmware.com/support/developer/vcli/http://www.vmware.com/support/developer/vcli/http://www.vmware.com/support/developer/vcli/http://www.vmware.com/appliances/directory/1789737/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
32/123
32
1. NetApp System Manager.2. .3. Edit.4. , ,
jumbo frames.
Ethernet NetApp IP-
IP- ,
VLAN. ESX vSwitch
NIC. ESX VMkernel vSwitch.
VMware HA ESX
(service console port).
NetApp
VMkernel . , default gateway VMkernel storage network. NFS service console port
VMkernel ESX.
IP-c , VMkernel, vmkping.
NFS, :
vmkping .
NetApp Virtual Interfaces (EtherChannel)
(VIF), NetApp EtherChannel,
.
, VIF . VIF-
, ,
, .
NetApp VIF :
LACP (IEEE 802.3ad). NetApp active-passive
VIF, single-mode VIF, . NetApp VIF
VIF-
VIF .
Multimode VIF EtherChannel. multimode VIF, VIF, .
, ,
. , ,
MAC-
. , VIF
.
LACP VIF (IEEE 802.3ad) EtherChannel. LACP VIF
, multimode
VIF, .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
33/123
33
LACP VIF
.
, , ,
, ,
EtherChannel. LACP VIF ,
static multimode VIF,
. standard multimode VIF
(link loss).
Multimode LACP EtherChannels, IP-
. IP- , MAC,
TCP/UDP*.
NetApp IP-
, , , ,
IP-.
single-mode VIF .
,
, standby. single-mode VIF
, , VIF,
. , single-mode VIF.
3.4 Cisco Nexus 1000V VNetwork Distributed Switch
Cisco Nexus 1000V System VLAN
1000V vNetwork Distributed Switch (vDS), ,
, Virtual Supervisor Module (VSM) Virtual Ethernet Module (VEM).
VSM , 1000V. VSM ,
- , vDS. VSM-
active-passive. appliance
ESX.
DRS.
VEM ESX/ESXi, VSM.
, Cisco Nexus 1000V vDS.
, VSM , VSM
VM, VSM . ,
NetApp Cisco , service console VMkernel (vswif
vmknic) VLAN (System VLAN). VLAN
, .
VM VLAN
Jumbo Frames
Cisco Nexus 1000V vDS jumbo frames,
jumbo frames VMkernel. jumbo frames
* VIF TR-3802
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
34/123
34
VMkernel, vSwitch VMkernel,
jumbo frames, 1000V vDS.
jumbo frames vSwitch, :
1. vSwitch VMkernel jumbo frame ( 3.3).2. ESX/ESXi vCenter client.3. VMkernel vSwitch.4. vSwitch 1000V vDS (. . 19).
19) vSwitch 1000V.
3.5 IP-,
, ,
IP-.
Ethernet multiswitch link aggregation (MSLA).
, Cisco Nexus (virtual port channels),
Cisco Catalyst 3750 series (cross-stack EtherChannel), Cisco Catalyst 6500 series VSS
1440 (multichassis EtherChannel), .
3.6 .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
35/123
35
20) multiswitch link aggregation.
, multiswitch
link aggregation,
ESX/ESXi, Data ONTAP, .
21) IP-.
3.7 .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
36/123
36
3.6 Multiswitch Link Aggregation
IP- multiswitch link aggregation.
; , multimode
LACP VIF IP.
,
,
,
.
MSLA
. NIC . . . EtherChannel. VMkernel IP ,
.
22) Multimode VIF multiswitch EtherChannel.
IP- , EtherChannel IP-
.
.
Rapid Cloning Utility(RCU).
23
ESX .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
37/123
37
23) multiswitch EtherChannel.
IP- NetApp
, IP, NFS,
iSCSI, Ethernet
EtherChannel. EtherChannel
.
, -
ESX/ESXI.
NetApp EtherChannel, 802.3ad LACP, /static
EtherChannel. Data ONTAP EtherChannel NetApp virtual interfaces
(VIF). NetApp EtherChannel VIF LACP VIF,
.
multiswitch link aggregation, multilink
EtherChannelVIF, IP-. IP-
vmnics, - ESX/ESXi. NSM.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
38/123
38
24) LACP EtherChannel NIC .
: EtherChannel
;
NIC. ,
EtherChannel. ,
EtherChannel,
,
EtherChannel, .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
39/123
39
25) LACP EtherChannel NIC
.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
40/123
40
26) LACP EtherChannel
. , EtherChannel
.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
41/123
41
27) LACP EtherChannel. ,
EtherChannel,
. IP- EtherChannel.
VMkernel ESX/ESXi
, IP- , multiswitch
EtherChannel trunking virtual port channeling, ESX
, . VMkernel IP-.
, ,
,
IP- (target IP).
ESX, . 28, vSwitch (
vSwitch1) IP- .
vSwitch ( vmnic1 vmnic2).
,
cross-stack EtherChannel trunk.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
42/123
42
: VMware LACP, IEEE 802.3ad,
Ethernet.
28) NIC ESX multiswitch EtherChannel.
vSwitch1 VMkernel (VMkernel 1) IP-,
(teaming) NIC VMkernel :
VMkernel 1: IP- 192.168.1.101. VMkernel 1 : Load-balancing Route based on IP hash.
29) VMkernel ESX multiswitch EtherChannel.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
43/123
43
3.7 Ethernet-
IP multiswitch link aggregation,
.
, .
IP- ,
,
.
Single Mode
, single-mode ,
single-mode (active-passive) EtherChannel VIF. VIF
IP-, ,
IP- . vif favor , VIF
. ,
- .
Single Mode EtherChannel
( ).
.
. non-EtherChannel. ESX/ESXi multiswitch hop.
Single Mode EtherChannel
VMkernel IP storage ports. - IP .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
44/123
44
30) Single-mode VIF .
multimode
, ,
TR-3880: CLI Configuration Processes
for NetApp and VMware vSphere Storage Arrays Running Data ONTAP and ESX/ESXi Server.
IP-
, EtherChannel IP-
, VMkernel -
ESX/ESXi. ,
. Rapid
Cloning Utility(RCU), .
VMkernel
VMkernel NetApp
,
, . NetApp
VMkernel .
iSCSI NFS. VMkernel
IP .
iSCSI NFS ,
VMkernel .
31. VMkernel vSwitch,
vmnics vSwitch.
NFS, VMkernel vmnic,
vmnic standby. , vmnic
VMkernel.
http://media.netapp.com/documents/tr-3880.pdfhttp://media.netapp.com/documents/tr-3880.pdfhttp://media.netapp.com/documents/tr-3880.pdfhttp://media.netapp.com/documents/tr-3880.pdfhttp://media.netapp.com/documents/tr-3880.pdfhttp://media.netapp.com/documents/tr-3880.pdf7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
45/123
45
31) VMkernel iSCSI NFS.
32
ESX .
32) EtherChannel.
ESX iSCSI
ESX ( NIC),
, , .
multipathing VMware,
VMkernel. , .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
46/123
46
ESX NFS
ESX (- NIC),
, , ,
, .
. ,
. EtherChannel .
, ,
, ,
, NetApp. ,
.
33) ESX vSwitch1.
34) ESX vSwitch .
IP- NetApp
, IP, NFS,
iSCSI, Ethernet
EtherChannel. EtherChannel
.
, -
ESX/ESXI.
NetApp EtherChannel, 802.3ad LACP, /static
EtherChannel. Data ONTAP EtherChannel NetApp virtual interfaces
(VIF). NetApp EtherChannel VIFs single mode
,
.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
47/123
47
, single-mode
EtherChannel VIF, IP-.
NSM.
35) LACP EtherChannel NIC .
: EtherChannel
;
NIC. ,
EtherChannel. ,
EtherChannel,
,
EtherChannel, .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
48/123
48
36) single-mode EtherChannel
NIC .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
49/123
49
37) single-mode EtherChannel
. , EtherChannel
.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
50/123
50
38) single-mode EtherChannel.
EtherChannel,
.
VMkernel ESX/ESXi
, ESX/ESXi
, , VMkernel,
. ,
IP-. VMkernel, (VMkernel)
,
vmnic vSwitch. ,
VMkernel .
IP- , ,
vmnic.
ESX/ESXi . 39, vSwitch ( vSwitch1)
IP- .
vSwitch ( vmnic1 vmnic2).
.
39) NIC ESX Ethernet.
vSwitch1, VMkernel (VMkernel 1 VMkernel 2). VMkernel IP- , NIC teaming
VMkernel , :
VMkernel 1: IP- 192.168.1.101. VMkernel 1:
Enable the override vSwitch failover order option. NFS iSCSI, vmnic1. NFS, standby-vmnic2. iSCSI, unused.
VMkernel 2: IP- 192.168.2.101. VMkernel2:
Enable the override vSwitch failover order option.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
51/123
51
NFS iSCSI, vmnic2. NFS, standby-vmnic1. iSCSI, unused.
40) VMkernel ESX Ethernet.
3.8 Multiple TCP Sessions iSCSI
vSphere multiple TCP sessions iSCSI.
( round robin)
VMware native multipathing VMkernel vmnic
iSCSI.
iSCSI , VMkernel
vmnic. standby vmnics VMkernel .
VMkernel iSCSI, , ,
VMkernel NIC teaming, , ,
. iSCSI
native multipathing ESX/ESXi.
iSCSI , FC. multiple
TCP session support iSCSI ESX/ESXi, NFS, ,
, , NFS
iSCSI VMkernel, .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
52/123
52
NetApp , iSCSI, NFS,
TCP NIC
teaming, .
41) VMkernel iSCSI multiple TCP
sessions.
: ESX/ESXi 4 iSCSI .
iSCSI VMKernels Multiple TCP Sessions iSCSI
1. vCenter Server.2. ESX.3. Configuration.4. Hardware, Networking.5. , Add Networking Add Network.6. VMkernel Next.
: VMkernel Ethernet, iSCSI. VMkernels .
7. VMkernel, . default gateway VMkernel IP storage network.
8. VMkernel , iSCSI VMkernel. VMkernel
standby-. . . 42 43.
iSCSI daemon VMkernel.
CLI.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
53/123
53
9. ESX ESXi :esxcli swiscsi nic add n -d
:
esxcli swiscsi nic add -n vmk0 -d vmhba33
esxcli swiscsi nic add -n vmk1 -d vmhba33
10.iSCSI-to-VMkernel. ESX ESXi :esxcli swiscsi nic list -d
:
esxcli swiscsi nic list -d vmhba33
. 44.
42) iSCSI VMkernel 0: vmnic0.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
54/123
54
43) iSCSI VMkernel 1: vmnic1.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
55/123
55
44) (binding) iSCSI VMkernel multiple TCP sessions.
4
Virtual Infrastructure
4.1 :
NetApp, ,
,
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
56/123
56
45) NetApp vCenter plug-in.
,
, .
,
(aggregates, FlexVol )
.
,
, VMware
, , LUN masking,
, ,
.
, ,
,
, VMware
.
, ,
VMware, NetApp, ,
.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
57/123
57
NetApp VCenter
NetApp vCenter, Virtual Storage Console (VSC) Rapid Cloning Utility (RCU),
VSC 2.0.
, VSC 2.0 NetApp SnapManager for Virtual
Infrastructure (SMVI), vSphere.
VSC ESX/ESXi .
, VSC 2.0
, VMware.
NetApp
, , ,
.
4.2
,
, , :
- . , Ethernet- (NFS,
iSCSI, FCoE). () (VLAN),
.
, IP (NFS iSCSI), IP .
.
IP (NFS iSCSI) (EtherChannel), Ethernet . NetApp
VIF ( ). NetApp ,
, LACP VIF- multimode.
4.3 NetApp
.
,
. , VMware
, .
VM, VMware HA
DRS, VMware Fault
Tolerance, VM, ,
ESX VMotionstorage VMotion, .
,
,
HBA,
, .
, ,
.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
58/123
58
RAID-DP
,
, , .
,
. NetApp RAID-DP RAID,
NetApp FAS. RAID-DP
RAID-. RAID-DP
; RAID- 12.5%.
RAID-DP , RAID 5 ,
RAID 10. NetApp RAID-DP RAID,
VMware.
46) NetApp RAID-DP.
Aggregates
aggregate NetApp,
flexible volume . Aggregate
,
IOPS, aggregate, IOPS
.
.
NetApp
root aggregate. , NetApp
aggregate. root aggregate,
RAID-DP. ,
RAID type RAID-DP RAID-4. , ,
.
root volume root aggregate,
root volume aggregate.
aggregates. - VMware
(random) ,
,
-. FAS,
aggregate,
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
59/123
59
.
aggregate.
Flexible
flexible LUN- ,
VMware ESX.
NetApp
VMware flexible- NetApp.
NetApp VMware desktop pools flexible-
NetApp.
VMware,
.
SnapMirror , NetApp
flexible volume.
LUN
LUN- SCSI- (FC, iSCSI, FCoE),
ESX/ESXi, VMFS raw
device mapping (RDM). iSCSI/Fibre Channel SAN configuration guide.
NetApp .
,
,
. ,
disaster recovery
.
NetApp :
aggregate: . FlexVol: . LUN VMFS: . LUN RDM: , (volume
label) .
4.4 NetApp
,
NetApp.
NetApp System Manager
NetApp NetApp Systems Manager (NSM)
, NetApp
. OS Data ONTAP,
(CLI) - FilerView, , ,
NetApp Operations Manager.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
60/123
60
NSM
. , ,
TR-3880: CLI Configuration Processes for NetApp and VMware vSphere
Storage Arrays Running Data ONTAP and ESX/ESXi Server.
Data ONTAP
NSM NOW Windows.
NetApp.
Ethernet ,
IP- DHCP.
,
setup.
NSM autodiscover
, IP- , .
NSM, NetApp Systems
Manager Quick Start Guide.
47) NetApp Systems Manager.
FC FCoE,
. Rapid
Cloning Utility (RCU) LUN,
, .
http://media.netapp.com/documents/tr-3880.pdfhttp://media.netapp.com/documents/tr-3880.pdfhttp://media.netapp.com/documents/tr-3880.pdfhttp://now.netapp.com/NOW/knowledge/docs/netapp_sys_mgr/rel10/html/software/qkstart/index.htmhttp://now.netapp.com/NOW/knowledge/docs/netapp_sys_mgr/rel10/html/software/qkstart/index.htmhttp://now.netapp.com/NOW/knowledge/docs/netapp_sys_mgr/rel10/html/software/qkstart/index.htmhttp://now.netapp.com/NOW/knowledge/docs/netapp_sys_mgr/rel10/html/software/qkstart/index.htmhttp://now.netapp.com/NOW/knowledge/docs/netapp_sys_mgr/rel10/html/software/qkstart/index.htmhttp://now.netapp.com/NOW/knowledge/docs/netapp_sys_mgr/rel10/html/software/qkstart/index.htmhttp://media.netapp.com/documents/tr-3880.pdfhttp://media.netapp.com/documents/tr-3880.pdf7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
61/123
61
48) NetApp Systems Manager.
IP-, NFS,
iSCSI, Ethernet
EtherChannel. , EtherChannel,
. ,
, 3.
NetApp System Manager
NetApp Systems Manager ,
NetApp FAS,
.
, thin provisioning
, aggregate.
,
, aggregate
.
NetApp e-mail
, SNMP. NSM.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
62/123
62
49) NetApp Systems Manager.
NetApp Operations Manager
NetApp Operations Manager, , NetApp FAS
, . NetApp thin provisioning,
NetApp Operations Manager e-mail
. thin-provisioned,
aggregates.
,
, aggregate .
DataFabric Manager Server: Operations Manager
Administration Guide.
TCP/IP, VSC
Virtual Storage Console (VSC) vSphere client,
vCenter Server, VSC server, NetApp. ,
. VSC
6 , vCenter Server, VSC.
6) TCP/IP, VSC.
80 VMware vCenter Server NetApp
HTTP.443 VMware vCenter Server NetApp
secure HTTP (SSL).
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
63/123
63
8043 Backup and recovery https-.
backup and recovery, CLI
.
8143 VSC https-.
4.5 VSC 2.0 root NetApp,
VSC, . ,
Data ONTAP service
account VSC. ,
VSC 2.0,
role-based access controls (RBAC) .
, RBAC VSC 2.0.
RBAC (Role-Based Access Control)
, , ,
.
Data ONTAP ( SSH, telnet).
: copy-paste
,
, .
.
NetApp CLI.
1: , , VSC Core Features
1. .2. .useradmin role add "role_name" -a login-http-admin,api-aggr-list-info,api-cf-get-
partner,api-cf-status,api-disk-list-info,api-ems-autosupport-log,api-fcp-adapter-
list-info,api-fcp-get-cfmode,api-license-list-info,api-lun-get-vdisk-attributes,api-
lun-list-info,api-lun-map-list-info,api-nfs-exportfs-list-rules,api-qtree-list,api-
snmp-get,api-snmp-get-next,api-system-get-info,api-system-get-version,api-volume-
autosize-get,api-volume-list-info,api-volume-options-list-info
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
64/123
64
50) VSC core feature service account.
2:
1. , .2. .useradmin group add "group_name" -r "role_name"
51) VSC core feature service account.
3: , VSC1. , ,
.
2. .useradmin user add "user_name" -g "group_name"
: .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
65/123
65
52) VSC core feature service account.
4:
1. , .2. .useradmin user list "user_name"
,
.
53) VSC core feature service account.
service account VSC.
4.6 Provisioning and Cloning VSC 2.0
, .
Rapid Cloning Utility .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
66/123
66
,
vCenter.
vCenter.
Provisioning and Cloning VSC 2.0
, 54.
. RBAC
, VSC.
Destroy Datastores:
Modify Datastores:
Create Datastores: SAN NAS
Create Clones: VM
54) VSC.
,
Provisioning and Cloning.
,
vCenter. root
, VMware
.
, VSC, 5.3.
:
1:
.
, VSC Provisioning and Cloning.
VSC.
.
4.5.
useradmin role add create_clones -a login-http-admin,api-system-get-version,api-
system-get-info,api-system-cli,api-license-list-info,cli-ifconfig,api-aggr-list-
info,api-volume-list-info,api-lun-list-info,api-lun-map-list-info,api-igroup-list-
info,api-ems-autosupport-log,api-file-get-file-info,api-clone-*,api-file-create-
directory,api-file-read-file,api-file-delete-file,api-file-write-file,cli-mv,api-
file-delete-directory,cli-ndmpd,cli-ndmpcopy,api-useradmin-user-list,api-cf-
status,api-snapshot-list-info,api-volume-autosize-get,api-iscsi-session-list-
info,api-iscsi-portal-list-info,api-fcp-service-status,api-iscsi-service-status,cli-
df,api-snapmirror-get-volume-status,api-quota-report,api-qtree-list,api-system-api-
list,api-vfiler-list-info
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
67/123
67
2:
,
(provisioning) (cloning),
VSC .
4.5.
useradmin role add create_datastores -a api-volume-create,api-volume-set-option,api-
volume-autosize-set,api-sis-enable,api-sis-start,api-snapshot-create,api-snapshot-
set-reserve,api-volume-clone-create,api-nfs-exportfs-list-rules-2,api-nfs-exportfs-
modify-rule-2,api-nfs-exportfs-load-exports,api-igroup-create,api-lun-create-by-
size,api-lun-map,api-lun-set-comment,api-igroup-add,cli-qtree,cli-iscsi
useradmin role add modify_datastores -a api-volume-create,api-volume-set-option,api-
volume-autosize-set,api-sis-enable,api-sis-start,api-snapshot-create,api-snapshot-
set-reserve,api-volume-clone-create,api-nfs-exportfs-list-rules-2,api-nfs-exportfs-modify-rule-2,api-nfs-exportfs-load-exports,api-igroup-create,api-lun-create-by-
size,api-lun-map,api-lun-set-comment,api-igroup-add,cli-qtree,cli-iscsi ,api-volume-
size,api-sis-disable,api-sis-stop,api-lun-resize
useradmin role add destroy_datastores -a api-volume-create,api-volume-set-option,api-
volume-autosize-set,api-sis-enable,api-sis-start,api-snapshot-create,api-snapshot-
set-reserve,api-volume-clone-create,api-nfs-exportfs-list-rules-2,api-nfs-exportfs-
modify-rule-2,api-nfs-exportfs-load-exports,api-igroup-create,api-lun-create-by-
size,api-lun-map,api-lun-set-comment,api-igroup-add,cli-qtree,cli-iscsi,api-volume-
size,api-sis-disable,api-sis-stop,api-lun-resize,api-volume-offline,api-volume-destroy,api-lun-offline,api-lun-destroy
3: Provisioning Cloning
1. , .2. .useradmin group add "group_name" -r "role1_name" "role2_name"
create_clones .
4.5.
4:
1. , , .
2. . 4.5.
useradmin user add "user_name" -g "group_name"
: .
5: 1. , .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
68/123
68
2. .useradmin user list "user_name"
,
.
,
Provisioning and Cloning VSC 2.0.
5 vSphere
Virtual Infrastructure
5.1 vCenter
(. . 45), vSphere, NetApp
,
, NetApp.
Provisioning and Cloning, Virtual
Storage Console 2.0 vSphere plug-in.
NetApp Virtual Storage Console 2.0 vCenter plug-in
Virtual Storage Console (VSC) 2.0 , NetApp.
, VSC NetApp vSphere.
, VSC plug-in, :
Virtual Storage Console: VSC 2.0. NetApp.
Provisioning and Cloning: , NetApp Rapid Cloning Utility(RCU). ,
, ,
, .
, Provisioning and Cloning, NetApp
FlexClone , .
Backup and Recovery: NetApp SnapManagerfor Virtual Infrastructure, VSC 2.0 plug-in framework.
VMware , SMVI, SnapRestore. SnapMirror
,
FlexClone , NFS.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
69/123
69
55) Virtual Storage Console 2.0.
VSC
ESX/ESXi. VSC.
- FC, FCoE, iSCSI, NFS ESX/ESXi
FC, FCoE, iSCSI
VAAI , ESX/ESXi
,
, , ESX/ESXi,
-.
Provisioning and Cloning VSC
SAN NAS NetApp. Provisioning and Cloning
VSC 2.0:
Provisioning and Cloning.
FC, FCoE, iSCSI, NFS. multipathing ,
VMware pluggable storage architecture LUN-,
ALUA, NFS round robin.
, . LUN maskingNFS exports.
FC, FCoE, iSCSI, NFS .
, , ESX/ESXi
, -.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
70/123
70
5.2 Virtual Storage Console 2.0
VSC ESX/ESXi 4.0 ,
ESX/ESXi 3.5. VSC,
, .
vCenter Server version 4.0 . VSC vCenter Server .
VM, 32- 64-bit Windows Server 2008, 2003 SP1 , 32-bit XP
Professional SP2 .
Data ONTAP 7.3.1.1 . firmware, NetApp Interoperability Matrix.
VSC 2.0
1. Windows2. 3. .
VSC 2.0,
(. . 56). , ,
Virtual Storage Console ( ).
Provisioning and Cloning Backup and Recovery Rapid Cloning Utility
SnapManager for Virtual Infrastructure,
, . , NetApp
Provisioning and Cloning, , ,
.
4. VSC vCenter Server , , .
vCenter
.
VSC
plug-in vCenter Server. vCenter administrator
. ,
VSC 2.0, .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
71/123
71
56) VSC 2.0.
57) vCenter.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
72/123
72
58) VSC vCenter Server.
5.3 Virtual Storage Console
VSC ,
.
1. vSphere client, vCenter.2. , NetApp.3. Virtual Storage Console.
, VSC
, Data ONTAP, ESX/ESXi
. ,
ESX/ESXi vSphere client NetApp
. VSC ,
.
Controller Credentials,
, VSC .
root, VSC,
.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
73/123
73
59) VSC.
5.4 ESX/ESXiVSC
ESX/ESXi 4.x, NetApp. VMware
ESX/ESXi,
.
HBA CNA, ,
- (NFS iSCSI).
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
74/123
74
60) ESX/ESXi 4.x,
multipathing LUN.
5.5 Provisioning and Cloning
Provisioning and Cloning VSC 2.0 -
, , .
vSphere client:
1. vCenter2. NetApp 3. Provisioning and Cloning.4. Add .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
75/123
75
61) Provisioning and Cloning.
5. Storage Controllers6. Add.
,
vCenter.
7. , . root
Provisioning and Cloning.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
76/123
76
62) RCU.
5.6 Provisioning and Cloning
, RCU ,
, .
, FlexVol( SAN), aggregates (
NAS). :
1. Provisioning and Cloning VSC2. 3. Resources4. .
prevent further changes (. 63),
,
. ,
,
VSC.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
77/123
77
63) VSC
.
5.7 vCenter
, Provisioning and Cloning VSC,
VMware vCenter.
1. , ESX/ESXi2. NetApp;3. Provisioning and Cloning,4. Provision datastore.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
78/123
78
64) .
NetApp Datastore Provisioning wizard, :
; (VMFS NFS); , , , block size ( VMFS); LUN thin provisioned.
.
, iSCSI, FC, FCoE, VSC , initiator groups,
ALUA, LUN masking, path selection policies, LUN
VMFS. , NFS, VSC
exports, .
, , thin-
provisioned LUN-,
.
65 Fibre Channel TR3749 LUN
thin-provisioned, FlexVol.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
79/123
79
65) VMFS NetApp Datastore Provisioning wizard.
NFS VMFS.
NFS
autogrow. 66
thin provisionedautogrow.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
80/123
80
66) NFS NetApp Datastore Provisioning wizard.
5.8
, ,
.
.
, , ,
WAN, .
, VMware ,
, VM. VM
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
81/123
81
home directory. , VM home directory.
VM, , VMkernel
swapfile, VMware, NVRAM VMware, .
, VM home directory
. . 67 .
67) vswap VMware .
VM NetApp Snapshots
, NetApp
VMware NetApp,
SnapManager, SnapMirror / SnapVault.
NetApp
(-, ) , - ,
.
, - NetApp,
VMware ,
. ,
, , ,
,
, ,
.
: VMkernel swap
ESX VMkernel swap vswap VM.
; vswap ,
VM. ,
VM , Site Recovery Manager,
NetApp VMkernel swap VM
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
82/123
82
home directory, NetApp,
VMkernel swap. 68 .
68) : vswap
.
vswap.
VMware swap file , NetApp
thin-provisioned LUN FlexVol autogrow. thin-provisioned LUN Autogrow FlexVol
swap-.
swap,
.
, vswap, . NetApp
ESX/ESXi vswap,
VMotion.
69 VMkernel
swap vCenter Server.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
83/123
83
69) VMkernel swap.
: VM swap/Pagefile
. ,
.
, , .
.
VM swap, pagefile ( ),
1,5 2 VM .
,
( ),
, business-critical .
swap pagefile OS
, , NetApp volume.
70 .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
84/123
84
70) :
VM pagefile pagefile datastore
vswap datastore .
, .
, ,
, -,
.
VMware
vCenter Site Recovery Manager. , -
DR-.
, SRM VM,
pagefile VMDK DR- SRM recovery
plan. SRM,
TR-3671: VMware vCenter Site Recovery Manager in a NetApp Environment.
5.9 vCenter
VSC 2.0, VMware
, . VSC
VMFS NFS,
NFS. ,
datastore vCenter, NetApp, Provisioning and Cloning,
Resize.
http://media.netapp.com/documents/tr-3671.pdfhttp://media.netapp.com/documents/tr-3671.pdfhttp://media.netapp.com/documents/tr-3671.pdf7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
85/123
85
71) .
72) VMFS.
5.10 vCenter
VSC VMware
, , , .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
86/123
86
SAN NAS ,
, .
NFS
NetApp vCenter, Storage Details - NASStorage Details - SAN,
. ,
.
NFS.
73) Storage Details VSC.
Capacity
Capacity Storage Details , FlexVol,
aggregate . NFS, FlexVol
, .
aggregate.
VMFS, LUN ,
.
FlexVol aggregate. FlexVol LUN, LUN thin
provisioned, FlexVol autogrow ,
LUN. LUN
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
87/123
87
FlexVol, , LUN offline,
, , .
85% aggregate.
aggregate
VM aggregate .
NetApp NetApp Data Motion,
, .
Volume
Volume FlexVol.
Provisioning and Cloning. VSC
NFS , VSC,
, NetApp.
Deduplication
, ,
Deduplication. NFS,
,
VM VM.
, NetApp,
Deduplication management.
74) NetApp .
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
88/123
88
75) .
6
Virtual Infrastructure
6.1 VM OS Windows
Windows
-,
,
NTFS.
-,
. :
1. Windows VM.2. Start > Run CMD.3. : fsutil behavior set disablelastaccess 1.
, NetApp,
,
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
89/123
89
WAFL OS.
OS
VM, , the NetApp Global Support Center.
6.2
VM SCSI BUS VSC, GOS
timeout scripts, ISO,
, SCSI
.
VMware (GOS timeout scripts),
:
1. ISO, VSC.2. vCenter Server, VM .3. edit settings.4. CDROM ISO.5. ISO, OS .6. OK.7. VM.8. OS VM.9. ISO.10. VM.
6.3
VM VMFS
.
, ,
OS VM.
- ,
VMFS
.
- ,
, .
NetApp, VMware, ,
VMware
VMFS .
VMFS OS
:
Citrix, Microsoft, NetApp, VMware. TR-3747: Best Practices for File System Alignment inVirtual Environments
Dell.Designing and Optimizing SAN Configurations EMC.Celerra IP Storage with VMware Virtual Infrastructure EMC.CLARiiON Integration with VMware ESX Server
http://media.netapp.com/documents/tr-3747.pdfhttp://media.netapp.com/documents/tr-3747.pdfhttp://media.netapp.com/documents/tr-3747.pdfhttp://www.dell.com/downloads/global/power/ps4q04-20040149-Mehis.pdfhttp://www.dell.com/downloads/global/power/ps4q04-20040149-Mehis.pdfhttp://www.dell.com/downloads/global/power/ps4q04-20040149-Mehis.pdfhttp://www.vmware.com/files/pdf/VMware_VI3_and_EMC_Celerra_IP.pdfhttp://www.vmware.com/files/pdf/VMware_VI3_and_EMC_Celerra_IP.pdfhttp://www.vmware.com/files/pdf/VMware_VI3_and_EMC_Celerra_IP.pdfhttp://www.vmware.com/pdf/clariion_wp_eng.pdfhttp://www.vmware.com/pdf/clariion_wp_eng.pdfhttp://www.vmware.com/pdf/clariion_wp_eng.pdfhttp://www.vmware.com/pdf/clariion_wp_eng.pdfhttp://www.vmware.com/files/pdf/VMware_VI3_and_EMC_Celerra_IP.pdfhttp://www.dell.com/downloads/global/power/ps4q04-20040149-Mehis.pdfhttp://media.netapp.com/documents/tr-3747.pdfhttp://media.netapp.com/documents/tr-3747.pdf7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
90/123
90
IBM.Storage Block Alignment with VMware Virtual Infrastructure Vizioncore.vOptimizer Pro FAQ VMware.Recommendations for Aligning VMFS Partitions
NetApp VMFS
LUN- NetApp iSCSI, FC, FCoE. , LUN
, LUN VMware. ,
VMware NFS, .
, VMFS NFS, , OS,
.
NetApp FAS, 4096.
, Microsoft Windows 2000, 2003, XP 32256. 4096 .
, Microsoft Windows Server 2008
Windows 7, Windows Vista
1048576. .
: Windows 2008 Windows Vista
Windows, ,
, ,
.
6.4 VM
.
.
.
-
, - .
, .
, -,
,
- . -
, .
ftp://service.boulder.ibm.com/storage/isv/NS3593-0.pdfftp://service.boulder.ibm.com/storage/isv/NS3593-0.pdfftp://service.boulder.ibm.com/storage/isv/NS3593-0.pdfhttp://www.vizioncore.com/products/vOptimizerPro/documents/vOptimizerProFAQ.pdfhttp://www.vizioncore.com/products/vOptimizerPro/documents/vOptimizerProFAQ.pdfhttp://www.vizioncore.com/products/vOptimizerPro/documents/vOptimizerProFAQ.pdfhttp://www.vmware.com/pdf/esx3_partition_align.pdfhttp://www.vmware.com/pdf/esx3_partition_align.pdfhttp://www.vmware.com/pdf/esx3_partition_align.pdfhttp://www.vmware.com/pdf/esx3_partition_align.pdfhttp://www.vizioncore.com/products/vOptimizerPro/documents/vOptimizerProFAQ.pdfftp://service.boulder.ibm.com/storage/isv/NS3593-0.pdf7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
91/123
91
6.5
OS Windows
,
OS Windows, VM msinfo32.
. msinfo32, Start > All Programs > Accessories > System Tools >
System Information (. 76).
76) msinfo32
.
NetApp MBRtools:
NetApp MBRScan, ESX
Windows Linux,
VMFS NFS.
MBRScan ,
. VM (
), .
MBRScan
VM, msinfo32,
VM. MBRScan VSC.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
92/123
92
6.6 VM
VM
,
,
(template).
.
NetApp MBRtools
VSC, NetApp , MBRAlign,
ESX MBR-
primary secondary. MBRAlign ,
(powered off) .
MBRAlign . ,
, thinthick
vmdk. MBRAlign NetApp. VM, , ,
.
MBRAlign , VSC. NetApp
NetApp Global Support Center
.
: Linux, GRUB boot loader,
MBRAlign.
1. Linux VM CD Linux CDROM ISO.2. VM.3. boot from the CD.4. CD GRUB .
,
VMDK. NetApp
, ,
VM.
,
, .
:
VMware vCenter Converter 5 Quest ( Vizioncore) vOptimizer Pro Novell PlateSpin Migrate Paragon Alignment Tool 3.0
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
93/123
93
6.7 VM
VMDK VM DISKPART
, OS,
. OS Windows,
Windows Preinstall Environment boot CD, "live CD/DVD".
. 77.
1. VM Microsoft WinPE CD.2. Start > Run DISKPART.3. Select disk0.4. Create Partition Primary Align=32.5. .6. OS .
77) diskpart .
VMDK fdisk
ESX.
6.8 VM
(VMDK) ESX/ESXi 4 , VM .
; VM.
, C:\ Windows '/' Linux
, .
OS .
OS .
:
1. vCenter Server.2. VM.3. VM Properties.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
94/123
94
4. ( . 78).5. OK.6. VM.
78) .
VMware ESX and ESXi Server
Configuration Guide.
http://www.vmware.com/pdf/vi3_server_config.pdfhttp://www.vmware.com/pdf/vi3_server_config.pdfhttp://www.vmware.com/pdf/vi3_server_config.pdfhttp://www.vmware.com/pdf/vi3_server_config.pdfhttp://www.vmware.com/pdf/vi3_server_config.pdf7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
95/123
95
79) VMDK VM.
OS (NTFS EXT3)
RDM ,
. ,
OS, .
1. VM.2. . Windows VM, diskpart
. :http://support.microsoft.com/default.aspx?scid=kb;en-us;300415.
Linux VM, ext2resize
. :http://sourceforge.net/projects/ext2resize
VM Microsoft Windows, NetApp SnapDrive,
VM RDM.
OS
OS, OS,
.
Windows 2008. OS
, C:\ Windows VM '/' Linux VM
OS .
,
-. VMDK LUN, ,
OS, ,
6.8 . ,
. VM ,
, , VM.
.
7
Virtual Infrastructure
7.1 ,
VMware vSphere
. ,
, ,
,
. NetApp
1992 , NetApp,
http://support.microsoft.com/default.aspx?scid=kb;en-us;300415http://support.microsoft.com/default.aspx?scid=kb;en-us;300415http://support.microsoft.com/default.aspx?scid=kb;en-us;300415http://sourceforge.net/projects/ext2resizehttp://sourceforge.net/projects/ext2resizehttp://sourceforge.net/projects/ext2resizehttp://sourceforge.net/projects/ext2resizehttp://support.microsoft.com/default.aspx?scid=kb;en-us;3004157/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
96/123
96
VMware, , ,
, .
VMware Snapshot (point-in-time) VM,
. VMware Snapshot
, ,
vCenter Server. VMware Snapshot
ESX vSphere.
VMware Snapshot, ,
vSphere Basic System Administration Guide.
NetApp Snapshot VMware,
crash-consistent
VM, VM,
. ,
.
VMware ,
, (hardware-based) Snapshot
, software-based . ,
vCenter Server,
. vSphere Basic System
Administration GuideVMware ESX and ESXi Server Configuration Guide.
7.2 NetApp
,
, VMware . NetApp Backup and Recovery
Virtual Storage Console 2.0.
, SnapManager for Virtual Infrastructure (SMVI).
NetApp SnapManager,
,
VM,
, ,
, SMVI ,
.
http://www.vmware.com/pdf/vsphere4/r40/vsp_40_admin_guide.pdfhttp://www.vmware.com/pdf/vsphere4/r40/vsp_40_admin_guide.pdfhttp://www.vmware.com/pdf/vsphere4/r40/vsp_40_admin_guide.pdfhttp://www.vmware.com/pdf/vsphere4/r40/vsp_40_admin_guide.pdfhttp://www.vmware.com/pdf/vsphere4/r40/vsp_40_admin_guide.pdfhttp://www.vmware.com/pdf/vi3_server_config.pdfhttp://www.vmware.com/pdf/vi3_server_config.pdfhttp://www.vmware.com/pdf/vi3_server_config.pdfhttp://www.vmware.com/pdf/vi3_server_config.pdfhttp://www.vmware.com/pdf/vsphere4/r40/vsp_40_admin_guide.pdfhttp://www.vmware.com/pdf/vsphere4/r40/vsp_40_admin_guide.pdfhttp://www.vmware.com/pdf/vsphere4/r40/vsp_40_admin_guide.pdf7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
97/123
97
80) Backup and
Recovery VSC.
81) Backup and Recovery VSC.
7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
98/123
98
vSphere
NetApp Snapshot, TR-3737: SnapManager 2.0 for Virtual Infrastructure Best
Practices.
http://media.netapp.com/documents/tr-3737.pdfhttp://media.netapp.com/documents/tr-3737.pdfhttp://media.netapp.com/documents/tr-3737.pdfhttp://media.netapp.com/documents/tr-3737.pdfhttp://media.netapp.com/documents/tr-3737.pdfhttp://media.netapp.com/documents/tr-3737.pdf7/30/2019 RUS TR-3749 v3.0 NetApp and VMware vSphere BP
99/123
99
8 vSphere 5 vSphere 5.0, VMware ,
vSphere.
. vSphere,
.
8.1 VMware ESXi
vSphere 5 ,
ESX, Linux,
ESXi. :
, Busybox. SSH . . .
VMkernel. stateless server,
, RAID, SAN.
, VMware vSphere Hypervisor(ESXi), ,
, ESXi, vSphere,
, vCenter,
,
.
NetApp, , NetApp Virtual Storage Console,
, vCenter Server.
VMware ESXi
, -
.
vSphere installation bundle (VIB),
VMware ESXi
. NetApp VIB,
.
8.2 VMware vCenter Server
VMware vCenter Server ,
vSphere. vSphere 5 , vCenter