Usage Example of VMware vSphere Space Reclamation with IBM ...

© IBM Copyright, 2017 Version 1.0, 08/04/2017

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/TD106413 Usage Example of VMware vSphere Space Reclamation with IBM System Storage DS8880 /13

Usage Example of VMware vSphere Space

Reclamation with IBM® System Storage® DS8880

®

This document can be found on the IBM Intranet, http://w3.ibm.com/support/techdocs

on IBM PartnerWorld®, http://partners.boulder.ibm.com

and on the Internet, http://www.ibm.com/support/techdocs

Search for document number TD106413 under the category of “White papers”.

Version 1.0

4 August 2017

IBM ATS System Storage Europe

Ingo Dimmer, Peter Kimmel

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/TD106413

http://w3.ibm.com/support/techdocs

http://partners.boulder.ibm.com/

http://www.ibm.com/support/techdocs



Disclaimer

This document reflects the authors’ experience from testing the described configurations.

It is presented “As-Is” and IBM® does not assume responsibility for the statements expressed herein.

Notices and Trademarks

Copyright © 2017 by International Business Machines Corporation.

No part of this document may be reproduced or transmitted in any form without written permission from IBM

Corporation. Product data has been reviewed for accuracy as of the date of initial publication. Product data is

subject to change without notice. This information may include technical inaccuracies or typographical errors. IBM

may make improvements and/or changes in the product(s) and/or programs(s) at any time without notice.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such

products, programs or services available in all countries in which IBM operates or does business. THE

INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS

OR IMPLIED. IBM EXPRESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR

PURPOSE OR NON-INFRINGEMENT.

IBM shall have no responsibility to update this information. IBM products are warranted according to the terms

and conditions of the agreements (e.g., IBM Customer Agreement, Statement of Limited Warranty, International

Program License Agreement, etc.) Under which they are provided. IBM is not responsible for the performance or

interoperability of any non-IBM products discussed herein. The performance data contained herein was obtained

in a controlled, isolated environment. Actual results that may be obtained in other operating environments may

vary significantly. While IBM has reviewed each item for accuracy in a specific situation, there is no guarantee that

the same or similar results will be obtained elsewhere. Statements regarding IBM’s future direction and intent are

subject to change or withdraw without notice, and represent goals and objectives only. The provision of the

information contained herein is not intended to, and does not, grant any right or license under any IBM patents or

copyrights. Inquiries regarding patent or copyright licenses should be made, in writing, to:

IBM Director of Licensing

IBM Corporation

North Castle Drive

Armonk, NY 10504-1785

U.S.A.

IBM, the IBM logo, DS8000, PartnerWorld, System Storage and System Storage DS are trademarks of International

Business Machines Corporation in the United States, other countries, or both.

VMware, the VMware logo, VMware vCenter Server, VMware ESXi and VMware vSphere are registered trademarks

or trademarks of VMware, Inc. or its subsidiaries in the United States and/or other jurisdictions.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United

States, other countries, or both.

Other company, products or service names may be trademarks or service marks of others.




Table of Contents

Disclaimer...................................................................................................................................................... 2

Notices and Trademarks ............................................................................................................................... 2

1. Usage Example of VMware vSphere Space Reclamation with IBM System Storage DS8880 ............... 4

1.1 DS8880 Configuration ................................................................................................................... 4

1.2 ESXi Host Configuration ................................................................................................................ 6

1.3 Virtual Machine Cloning ................................................................................................................ 7

1.4 Virtual Machine Deletion ............................................................................................................ 11

1.5 Space Reclamation for IBM System Storage DS8880 .................................................................. 11

1.6 Summary ..................................................................................................................................... 13

References .................................................................................................................................................. 13




1. Usage Example of VMware vSphere Space Reclamation with IBM

System Storage DS8880

This technical document shows how allocated storage capacity on IBM System Storage DS8880

thinly provisioned Extent Space Efficient (ESE) volume which has already been logically deleted

from the corresponding VMware datastore can be reclaimed again from the VMware ESXi host

by invoking a SCSI UNMAP command as a VMware vSphere Storage APIs for Array Integration

(VAAI) primitive for thin provisioning which is newly supported by IBM System Storage DS8880

R8.2.3 and later.

Our usage example shows a manual space reclamation by a VMware administrator for thinly

provisioned storage which is supported with ESXi 5.5 and 6.0. With VMware ESXi 6.5 and

VMFS6 datastores support for automatic unmap was re-introduced. This automatic unmap runs

asynchronously on a scheduled basis with priority levels that can be set per VMFS datastore.

To illustrate the manual space reclamation function from an ESXi 6.0 host via the SCSI UNMAP

command we cloned an existing virtual machine into a VMware datastore created on a thinly

provisioned ESE volume, deleted the cloned VM again, reclaimed the space from the ESXi host

and checked the corresponding decrease in the allocated storage capacity for the ESE volume.

The following sections show the detailled configuration and steps we performed to illustrate

the new DS8880 space reclamation support for VMware.

1.1 DS8880 Configuration

The following DS8880 DS Command Line Interface (CLI) session outputs show our used DS8880

logical storage configuration.

Verifying that the DS8880 machine is on R8.2.3 or later code level to support the SCSI UNMAP

command:

dscli> lsserver -l

Date/Time: 01 August 2017 15:30:29 CEST IBM DSCLI Version: 7.8.20.280 DS: -

ID Image ID Image Name Power Control SFI State LIC Version OS Version Bundle Version

=========================================================================================

00 1 SF75GNC80ESS01 0 online 7.8.23.91 7.1.4.204 88.23.20.5120

01 1 SF75GNC80ESS11 0 online 7.8.23.91 7.1.4.204 88.23.20.5120

Note: Support of the SCSI UNMAP command for thinly provisioned DS8880 volumes requires

the volumes to be configured in a 16 MiB small-extent pool.




Listing the standard (thick) volume 0x1000 and thinly provisioned (ESE) volume 0x1100 we used for our VMware datastores: dscli> lsfbvol -l 1000 1100

Date/Time: 01 August 2017 15:27:48 CEST IBM DSCLI Version: 7.8.20.280 DS: IBM.2107-75GNC81

Name ID accstate datastate configstate deviceMTM datatype extpool sam captype cap (2^30B) cap (10^9B) cap (blocks) volgrp

reqcap (blocks) eam perfgrp resgrp GUID

======================================================================================================================================

=======================================================================

ESXi_1000 1000 Online Normal Normal 2107-900 FB 512 P2 Standard DS 200.0 - 419430400 V5

419430400 managed PG0 RG0 6005076307FFD7690000000000001000

ESXi_1100 1100 Online Normal Normal 2107-900 FB 512 P3 ESE DS 400.0 - 838860800 V5

838860800 managed PG0 RG0 6005076307FFD7690000000000001100

Verifying that the ESE volume 0x1100 was created within a 16 MiB “small-extent” pool: dscli> lsextpool -l P2 P3


Name ID stgtype rankgrp status availstor (2^30B) %allocated available reserved numvols numranks encryptgrp numtiers etmanaged

extsize

======================================================================================================================================

===

FBStress P2 fb 0 below 10575 22 676799 1216 7 2 - 2 yes

16MiB

FBStress P3 fb 1 below 11378 17 728185 1216 6 2 - 2 yes

16MiB

Looking at the ESE volume account allocation after VMware VFMS datastore creation which shows 39 extents (624 MiB) allocated: dscli> showfbvol 1100


Name ESXi_1100

ID 1100

accstate Online

datastate Normal

configstate Normal

deviceMTM 2107-900

datatype FB 512

addrgrp 1

extpool P3

exts 25600

cap (MiB) 409600

captype DS

cap (2^30B) 400.0

cap (10^9B) -

cap (blocks) 838860800

volgrp V5

ranks 2

dbexts 0




sam ESE

repcapalloc -

eam managed

reqcap (blocks) 838860800

realextents 39

virtualextents 41

realcap (MiB) 624

migrating 0

migratingcap (MiB) 0

perfgrp PG0

migratingfrom -

resgrp RG0

tierassignstatus -

tierassignerror -

tierassignorder -

tierassigntarget -

%tierassigned 0

etmonpauseremain -

etmonitorreset unknown

GUID 6005076307FFD7690000000000001100

1.2 ESXi Host Configuration

The following ESXi host command line interface session outputs show our ESXi host storage

configuration.

Displaying the ESXi version we used for testing: [root@localhost:~] esxcli system version get

Product: VMware ESXi

Version: 6.0.0

Build: Releasebuild-3620759

Update: 2

Patch: 34

Displaying the VAAI support status as reported from DS8000 for thick standard (0x1000) and

ESE volume (0x1100) on our ESXi host: [root@localhost:~] esxcli storage core device vaai status get -d

naa.6005076307ffd7690000000000001000

naa.6005076307ffd7690000000000001000

VAAI Plugin Name:

ATS Status: supported

Clone Status: supported

Zero Status: supported

Delete Status: unsupported

[root@localhost:~] esxcli storage core device vaai status get -d

naa.6005076307ffd7690000000000001100

naa.6005076307ffd7690000000000001100

VAAI Plugin Name:

ATS Status: supported

Clone Status: unsupported

Zero Status: supported

Delete Status: supported i.e. SCSI UNMAP is supported for thinly provisioned volume space reclamation




1.3 Virtual Machine Cloning

Our source virtual machine “Win2016_DS8k” which we are using for cloning was created with

the default VM disk format of “thick provision lazy zeroed” as shown in Figure 1:

Figure 1: vCenter Server VM Settings for source VM virtual disk type




Within the vSphere Web Client connected to our vCenter Server we cloned the existing VM

“Win2016_DS8k” with its datastore “DS8k_STD_1000” located on DS8880 thick volume 0x1000

to another datastore “DS8k_ESE_1100” on DS8880 thin volume 0x1100 like shown in Figure 2

and Figure 3.

Figure 2: Using the vSphere Web Client for cloning a VM




Figure 3: vSphere Web Client selection of the thin target datastore on the DS8880 for cloning the VM

Looking at the disk usage within our Microsoft Windows Server 2016 virtual machine we can

see that the VM has ~11 GB of its 60 GB virtual machine disk (VMDK) allocated as shown in

Figure 4.

Figure 4: Virtual machine disk usage




The DS8880 thinly-provisioned ESE volume 0x1100 for the cloned VM also shows about 11 GB

allocated (“realcap”) as the DS8880 recognizes the trailing zeroes of the unallocated space from

the 60 GB volume: dscli> showfbvol 1100


Name ESXi_1100

ID 1100

accstate Online

datastate Normal

configstate Normal

deviceMTM 2107-900

datatype FB 512

addrgrp 1

extpool P3

exts 25600

cap (MiB) 409600

captype DS

cap (2^30B) 400.0

cap (10^9B) -


volgrp V5

ranks 2

dbexts 0

sam ESE

repcapalloc -

eam managed


realextents 724

virtualextents 41

realcap (MiB) 11584

migrating 0


perfgrp PG0

migratingfrom -

resgrp RG0

tierassignstatus -

tierassignerror -

tierassignorder -

tierassigntarget -

%tierassigned 0

etmonpauseremain -


GUID 6005076307FFD7690000000000001100




1.4 Virtual Machine Deletion

We delete our previously created VM clone “Win2016_DS8k_Clone” again like shown in Figure

5 which logically deletes its associated VM disk files from the datastore and allows us to be able

to demonstrate reclaiming the corresponding DS8880 thinly-provisioned storage space.

Figure 5: Using the vSphere Web Client for virtual machine deletion

1.5 Space Reclamation for IBM System Storage DS8880

From the VMware ESXi host point of view the VM files for the deleted VM

“Win2016_DS8k_Clone” have already logically been deleted from the datastore

“DS8k_ESE_1100”: [root@localhost:~] ls -l /vmfs/volumes/DS8k_ESE_1100

lrwxr-xr-x 1 root root 35 Aug 1 18:02 /vmfs/volumes/DS8k_ESE_1100 -> 59809138-

c6cdc670-bc07-0090fa1e0264

[root@localhost:~] ls -l /vmfs/volumes/59809138-c6cdc670-bc07-0090fa1e0264

total 0




However the previously used VM storage capacity is still allocated within the thinly provisioned

volume on our DS8880 storage system but can be reclaimed by manually invoking the VAAI thin

provisioning primitive for SCSI UNMAP from our ESXi host on the corresponding datastore

“DS8k_ESE_1100” as follows:

[root@localhost:~] esxcli storage filesystem list

Mount Point Volume Name UUID

Mounted Type Size Free

------------------------------------------------- ------------- -------------------------------

---- ------- ------ ------------ ------------

/vmfs/volumes/59773c4e-ee1a0569-c0b4-0090fa1e0264 Win2016_local 59773c4e-ee1a0569-c0b4-

0090fa1e0264 true VMFS-5 146565758976 65933410304

/vmfs/volumes/59809040-d34eb027-5a53-0090fa1e0264 DS8k_STD_1000 59809040-d34eb027-5a53-

0090fa1e0264 true VMFS-5 214479929344 144543055872

/vmfs/volumes/59809138-c6cdc670-bc07-0090fa1e0264 DS8k_ESE_1100 59809138-c6cdc670-bc07-

0090fa1e0264 true VMFS-5 429228294144 428205932544

/vmfs/volumes/5978895d-f53d103d-e316-0090fa1e0264 5978895d-f53d103d-e316-

0090fa1e0264 true vfat 299712512 87293952

/vmfs/volumes/920be7e7-00854172-72ff-b82b4169e297 920be7e7-00854172-72ff-

b82b4169e297 true vfat 261853184 85819392

/vmfs/volumes/f5dc20ed-c37892c2-b075-3582fcb460b7 f5dc20ed-c37892c2-b075-

3582fcb460b7 true vfat 261853184 84566016

[root@localhost:~] esxcli storage vmfs unmap -l DS8k_ESE_1100

Looking at our DS8880 thinly provisioned ESE volume 0x1100 after the synchronous unmap

operations has finished we can see that the allocated real capacity dropped back from around

11.5 to 1.5 GiB:

dscli> showfbvol 1100


Name ESXi_1100

ID 1100

accstate Online

datastate Normal

configstate Normal

deviceMTM 2107-900

datatype FB 512

addrgrp 1

extpool P3

exts 25600

cap (MiB) 409600

captype DS

cap (2^30B) 400.0

cap (10^9B) -


volgrp V5

ranks 2

dbexts 0

sam ESE

repcapalloc -

eam managed


realextents 93

virtualextents 41

realcap (MiB) 1488

migrating 0


perfgrp PG0

migratingfrom -

resgrp RG0

tierassignstatus -




tierassignerror -

tierassignorder -

tierassigntarget -

%tierassigned 0

etmonpauseremain -


GUID 6005076307FFD7690000000000001100

1.6 Summary

Our usage example showed that with the new DS8880 R8.2.3 support for VMware VAAI thin

provisioning space reclamation we were able to reclaim about 92% of the previously allocated

capacity within our thinly-provisioned VMware datastore volumes. The cirumstance that we

weren’t able to reclaim 100% is mainly related to the 16 MiB extent boundary where storage

space can only be reclaimed if it spans across a whole extent.

References

IBM DS8880 Thin Provisioning (REDP-5343) http://www.redbooks.ibm.com/redpapers/pdfs/redp5343.pdf

IBM DS8000 Series Version 8 Release 2.3 Command-Line Interface User’s Guide (SC27-8526-05) http://www-01.ibm.com/support/docview.wss?uid=ssg1S7005236&aid=1

ESXCLI Commands for VAAI operations (2131056) https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2131056

Using the esxcli storage vmfs unmap command to reclaim VMFS deleted blocks on thin-

provisioned LUNs (2057513) https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2057513


http://www.redbooks.ibm.com/redpapers/pdfs/redp5343.pdf

http://www-01.ibm.com/support/docview.wss?uid=ssg1S7005236&aid=1

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2131056

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2057513

Usage Example of VMware vSphere Space Reclamation with IBM ...

Documents

Transcript of Usage Example of VMware vSphere Space Reclamation with IBM ...