1

Hitachi Dynamic Provisioning Lessons Learned Best Practices Whitepaper

August 14, 2009 Version 1.1 Authors: Steve Burr and John Harker HDS Confidential – for HDS and Partner Use only Copyright © 2009, Hitachi Data Systems

2

Executive Summary: This whitepaper is a collection of best practices from experience learned by Hitachi technical and services teams over the last several years. It is designed to help you understand, configure and operate Hitachi Dynamic Provisioning on both Adaptable Modular Storage 2000 and Universal Storage Platform V and VM storage systems. The materials here cover basics common to all application usages. Further information on the use of HDP with specific applications is available in individual whitepapers available on the Dynamic Provisioning product page on hds.com. Getting Started .................................................................................................................... 3

Overview......................................................................................................................... 3 Important considerations................................................................................................. 4

Configuration Planning and Design.................................................................................... 5 Configuration planning ................................................................................................... 5 Capacity Monitoring ....................................................................................................... 5 Pool design:..................................................................................................................... 6 Virtual volume design..................................................................................................... 7 External storage .............................................................................................................. 8 Pool design algorithms.................................................................................................... 9 Allocating pool LDEVS (USP V/VM) ......................................................................... 10 Disk alignment .............................................................................................................. 10 Data availability considerations of wide striping.......................................................... 11

Performance ...................................................................................................................... 11 General performance..................................................................................................... 11 Common pool or separate pool ..................................................................................... 12 Performance of mixed workloads: ................................................................................ 12 Performance and Capacity Planning Tools................................................................... 13

Server Side ........................................................................................................................ 14 Server side considerations............................................................................................. 14 Managing space and server side expansion .................................................................. 14 Filesystems.................................................................................................................... 15 Dynamic Provisioning and VMware ............................................................................ 16

Operations ......................................................................................................................... 18 Growing pools............................................................................................................... 18 Deleting devices............................................................................................................ 18 Migrations ..................................................................................................................... 19 Zero Page Reclaim........................................................................................................ 19

3

Getting Started

Overview

Hitachi Dynamic Provisioning technology allows storage administrators to maintain a pool of free space to service the data growth requirements of all their applications without pre-committing the storage to them. This alleviates the poor utilization rates common on traditional storage arrays where large amounts of capacity are allocated to individual applications, but remain unused.

Dynamic Provisioning also simplifies and reduces costs of application storage provisioning because it decouples provisioning to an application from the physical addition of resources to the storage system and automates performance optimization. Customers who have experienced these benefits on servers with products such a VMware, where virtual systems can be created on ‘on-the-fly’ now can on storage systems as well.

And on top of this, Dynamic Provisioning adds the additional advantage in many cases of improving storage performance.

Benefits Easier management

Simpler planning Avoid or defer tough decisions about LDEV size Control, change and optimize: capacity, performance, reliability, cost tier

Naturally balances performance

HDP Virtual Volumes

Servers

4

With large pools there are more spindles than static provisioning Can scale performance by growing pool

Over provisioning capacity Save on storage expenses Simplify capacity planning Automatic capacity leveling Rapid provisioning Reduce need for “urgent capacity change” Reduce occasions where need for –“disk capacity on server Y has run out.

We’ll have to re-allocate the whole estate.”

Important considerations When implementing Dynamic Provisioning it is important that realistic utilization objectives are set. “Hope for the best, plan for the worst” is an old saying still true. You should never use Thin Provisioning unless you understand your plan to rebuild if disks fail, or how to manage the risk of exhausting your capacity.

What level of overprovisioning you use is extremely dependent on your individual applications and usage patterns, and on your risk management preferences and strategies. If you wish to overprovision it is critical to understand application behavior and patterns in terms of file allocation and file growth (or shrinkage). Long term ‘thinness’ is dependent on application behavior. This falls under the discipline of capacity planning. Lacking that, you must institute artificial controls when putting unknown applications under Dynamic Provisioning

Generally, target no higher than 70-80 percent capacity utilization per pool. A buffer should be provided for unexpected growth or a “runaway” application that consumes more physical capacity than was originally planned for. There should at least be sufficient free space in the storage pool equal to the capacity that might be needed for the largest as yet unallocated thin device. Automating the monitoring of alert notifications is critical to maintaining an effective Dynamic Provisioning operation, as well as adopting the operational procedures to take immediate action when a pool threshold trigger is encountered. The user selectable level should be set at where the pool can not run out of free capacity before additional pool capacity is added. Aside from the default and user-specified pool thresholds available in Dynamic Provisioning via Storage Navigator and Device Manager, a customer can implement additional user-specified thresholds through monitoring capability in Tuning Manager

5

Configuration Planning and Design

Configuration planning When planning a configuration using thin devices the first step involves determining how many separate thin pools are needed and the required composition (disk and RAID types) of each thin data pool. Typically, this will involve conceptually organizing disk storage into separate classes, with further subdivision as needed to allow pool isolation. Depending on the mix of applications to be placed on thin devices, it will often be necessary to create multiple thin pools. But generally, the most efficient use of resources will be achieved by using a minimal number of pools Typically, a thin pool should be designed for use by a given application, or set of related applications, aligned with a given business group. The applications sharing a thin pool will compete for back-end resources, including thin pool storage capacity, so applications should not share the same pool if this is not acceptable. The devices comprising a thin pool will have the same performance and protection properties, so the applications sharing a thin pool should have compatible performance and protection requirements.

Capacity Monitoring Automating the monitoring of alert notifications is critical to maintaining an effective Dynamic Provisioning operation, as well as adopting the operational procedures to take immediate action when a pool threshold trigger is encountered. The user selectable levels should be set at where the pool can not run out of free capacity before additional pool capacity is added. On the AMS two customer definable pool utilization levels may be set per pool to generate alerts. You also have two over provisioning ratio thresholds that are set to monitor over-provisioning ratio (Total virtual volume capacity/Total pool capacity). These will tell you if too large a volume is allocated, or you are beyond your growth plan. Alert options are: Blinking LED on system; Email message; SIM; and SNMP trap issued. Note that Device Manager is required for the SNMP trap support. The system stores the DP Pool Trend information for the past year and the information is available for export as a CSV file On the USP V and VM there are two thresholds for Dynamic Provisioning Pool capacity shortage management and one for each virtual volume. For the pool, there is a fixed threshold at 80% and a variable threshold to be set between 5% and 95%. If either threshold exceeded an alert (SIM/SNMP Trap) is reported. For each Dynamic Provisioning volume a single variable threshold is provided with a variable threshold to be set between 5% and 300% for each DP V-Volume, where the percent represent x% times the current unallocated capacity If free capacity in the DP Pool for this DP V-Volume falls below this threshold (in

6

capacity) then it means there isn’t enough free space in the pool to cover and an alert (SIM/SNMP Trap) reported Aside from pool thresholds available in Dynamic Provisioning via Storage Navigator and Device Manager, a customer can implement additional user-specified thresholds through monitoring capability in Tuning Manager

Pool design: On the USP V and VM, all LDEVs used to create a Pool must be OPEN-V and between 8GB and 4TB in size. On the AMS entire RAID groups are directly put into pools. In either case, for a normal pool the recommended pool size, where optimum performance is a requirement, is minimum of 4 array groups dedicated to single pool. Rather than trying to create 1 or 2 really big pools, be prepared to design 4 or more smaller pools for some isolation between candidate applications, based on workload profile, production vs. test/dev, and/or thin-friendly vs. not (including unknown application behaviors).

Decide on RAID-10, RAID-5, or RAID-6 array groups for a given Pool depending on normal application design rules and analysis of application workload that is to be supported. Do not intermix different RAID configurations in the same Pool. Note RAID-6 provides some extra insulation from a failed array group from destroying a pool.

Dedicate whole array group(s) to a Pool. Don’t configure too many drives in the raid group. Each parity group should be used 100% for Dynamic Provisioning. On the USP define one large LDEV per array group as a pool volume

On the USP with a minimum of 4 array groups per pool, each assigned AG should be behind a different BED pair (if less than 4 BED pairs, spread across all BED pairs installed).

Normal design principals apply:

RAID level has the same design effect as normal and back-end performance design is the same as normal performance design:

o The more spindles the better the performance. o As a consequence the smaller the disks the faster the pool. o 15K is faster than 10K is faster than 7K2. o FC is faster than SATA. o External storage works but has similar slowdown characteristics. Wide-

striping can ameliorate this. o A RAID-1 pool has generally RAID-1 characteristics.

7

o A RAID-5 pool has generally RAID-5 characteristics (q.v. sequential randomization).

o A RAID-6 pool has generally RAID-6 characteristics.

So, if your design analysis says "four parity groups of 146GB 15K Raid 2+2 will meet a specific application performance requirement" then the pool you want should have at least the same design.

In addition there is an important new added design option available with Dynamic Provisioning. This is that the performance design requirement may be best met through pooling together multiple application requirements (see ‘performance of mixed workloads’ in the Performance section).

o You have to aggregate the sum of the applications in the pool - each workload needs to be modeled and then added together. The total pool design is the sum of the individual requirements.

o However, on the USP you might see a small amount of "randomization"; sequential workloads at the front end generating more random workloads than expected at the back end. This will mainly happen if you have multiple workloads which will tend to fragment the sequential data. This effect is reduced by: write buffering and wide-striping.

o Don't design for more performance than you need.

Follow the usual rules about distribution over multiple parity groups, BEDs etc. as many resources as you have. And as always keep ShadowImage P-VOLs and S-VOLs on different parity groups (different pools).

Virtual volume design

Because Dynamic Provisioning can be sparse if used right then there can be less reason to be restrictive about device sizes. If you want, every device can be right sized? Management may be simpler if you KISS however.

You no longer have the problem of fitting objects into the restriction of the parity group, devices can be any size. They can be bigger than any normal LDEV

DP-VOLs can't be LUSEd so you can make LUSEd normal devices bigger than the largest DP-VOL.

Put one DP-VOL in each V-VOL group. This seems silly and a pain. But If you don't do this you can't resize them later.

On the AMS, set the Virtual-LUs on 1GB size boundaries to avoid wasting pool space.

DP-VOL/V-VOL Group considerations

8

Put only a single DP-VOL in each V-VOL Group. If you don't you cannot resize the DP-VOL later. This is going to be enforced automatically soon.

External storage In cases where external storage is being used behind a USP V or VM you can extend Dynamic Provisioning benefits to it as well. Connect the external storage as usual, taking care to balance the I/O across the available ports on the storage being virtualized. Use existing RAID groups or create them as desired. One or many LUNs may be created and used, but best practice is to create a single LDEV on each RAID Group and then present it to the USP V for use as Dynamic Provisioning pool volumes. Reliability and integrity is critical as the whole pool relies on all components functioning continuously. Pools do not like disruption from other sources; they require predictable access to all the bandwidth. Dynamic Provisioning is not tolerant of imbalance in the pool vols. If one pool-vol is able to deliver only half the IO of the others, the whole pool will be half speed. So make sure that you continue the relevant internal rules: Same raid level, same disk size and type, same pool-vol size. Dedicate your array groups to pool use for a single pool. Keep the connection between USP V and external storage as simple and reliable as possible. Consider direct attach for this simplicity, once installed you are not going to be changing this. You might want to consider separating the volumes used for Dynamic Provisioning onto their own ports on the external storage thus logically separating the virtualized storage used onto its own path groups. Pay particular attention to designing the correct queue depth and having adequate I/O bandwidth, and of course follow all design guidelines associated with the external storage itself. Running HDP on externally attached AMS2000 storage systems With the availability of Dynamic Provisioning on the AMS2000 series it is possible to create Dynamic Provisioning pools on the AMS and then present "thin" volumes to the USP-V or VM. The USP-V would then present these external V-Vols to servers. You would be running Dynamic Provisioning on the AMS instead of the USP. There are significant tradeoffs to be considered in making this decision; depending on your priorities you may choose one or the other:

Positives Negatives Dynamic Provisioning run on the USP V/VM

Much cleaner management, fewer points of administration

More fine-grain wide-striping, able to spread out smaller volumes better

Substantially easier to expand virtual volumes and pools

You cannot de-virtualize the AMS

9

drawn from virtualized storage Delivers direct replication

savings (storage, bandwidth, license fees)

Current support for auto rebalance upon pool expansion and Zero Page Reclaim

Future support for SCSI Write-Same reclamation

Dynamic Provisioning run on the AMS2000 behind a USP V/VM

Saves money on licensing Dynamic Provisioning

Dedicated sequential loads can be faster than with USP-based HDP

Permits de-virtualization of the AMS from the USPV without data loss (assuming virtualization best practices are followed - note if you CVS or LUSE virtualized volumes then you cannot de-virtualize).

Future support for auto rebalance upon pool expansion, and Zero Page Reclaim

It is substantially more complicated to create and expand virtual volumes

SI, TC, UR will appear as thick volumes (loss of space savings). HDP on AMS2000 will deliver no replication space, bandwidth or license fee savings

You lose integration with Tiered Storage Manager to auto create destination volumes during migrations.

If you do chose use Dynamic Provisioning on an externally attached AMS, do not in turn use these thin volumes for a Dynamic Provisioning Pool on the USP. Two levels of thin provisioning are not productive and because of the increased overhead will be counterproductive. And with external storage in general do not intermix internal and external LDEVs in the same Pool, even if in the same CLPR (this is enforced by the system).

Pool design algorithms This simple method can be followed:

1. Determine storage capacity required. Include planned growth and risks associated with any planned over-provisioning. a. One approach with over-provisioning is to assume that one application

will "go rogue" and expand to its full capacity but to happen with two would be rare.

b. Therefore include capacity in the pool to accommodate all of the most over-provisioned application.

c. Monitor all applications against their plan. (DP-VOL monitoring may help here).

10

If one does go rogue, then you need to determine what went wrong and either:

i. Accept it and expand the pool i. Migrate the rogue out of the pool to ensure it doesn’t

block and then fix the cause of the over-run. 2. Determine total pool IOPS required. Include planned growth. 3. Choose Raid Level, Disk Type, Size etc. All must be the same. 4. Decide number of array groups and types needed to support required

capacity. 5. Decide number of array groups and types needed to support IOPS

requirement. 6. Take max of 4 and 5 and round up to four AG minimum. This may result in more capacity or IOPS than you initially need:

a. Spare IOPS and spare capacity is OK (unless you’ve priced it too high). b. Spare IOPS but no spare capacity is OK (unless you’ve over-priced it). c. No spare IOPS and Spare capacity is bad unless you have a “no IOPS

workload” like a rarely touched archive you can use to fill the space. Consider: smaller devices, RAID-1, 15K as appropriate.

d. No spare IOPS and No Spare capacity is just right. 7. Repeat 3-6 until cost is optimized and there isn’t too much waste 8. Create Raid Groups, On the USP V and VM, don’t use concatenation. 9. Create one LDEV on each AG. 10. Create pool.

Allocating pool LDEVS (USP V/VM) The following applies to USP V and VM systems with pre-V5 microcode. With V5 and above, just allocate LDEVs as you would normally. On the USP V/VM pool LDEVs should be formatted using SVP Install or Storage Navigator VLL function with all LDEVs for a given Pool created/formatted simultaneously, using same CU and LDEV ID dispersal. All LDEVs assigned to a Pool should be added to the pool at the same time and in LDEV ID order. Finish the operation by pressing the Optimize button on the Pool window, which will optimize the free page list. Note, if volumes have not been added to the Pool, the Optimize button will do nothing. When adding a group of Pool volumes, do not add one Pool volume at a time followed by pressing Optimize button for each Pool volume. Add all volumes in one operation then press the Optimize button once. Also on the USP V/VM, only OPEN-V volumes can be created and all Array Groups used to build a Pool must be mapped to the same CLPR.

Disk alignment If it is recommended to align a workload for best performance for normal volumes then the same alignment is required for Dynamic Provisioning. For example for

11

performance it is recommend you align an Exchange workload to 64K. (note the default for windows is 63 blocks - this is guaranteed to be misaligned on all systems!)

Data availability considerations of wide striping Data availability considerations that apply to a thin device configuration are the same as those that apply to a configuration in which device-level wide striping is achieved using RAID. In addition wide striping may increase the number of LUNs affected by a data loss event. When designing a configuration involving thin devices, consider the following availability aspects of the drives underlying the thin pool:

Mean time between drive failure (MTBF) for drives underlying the set of data devices

The type of RAID protection used

The number of RAID groups over which the set of server volume is spread

Mean time to repair (MTTR) including drive rebuild time

So availability is dependent upon on the number of RAID groups underlying the thin pool. Because of the use of wide striping, the availability of any thin device using a pool may be impacted by the failure of any RAID group used by the thin pool. The dependency on the MTTR should also be noted.

It is recommended that the drives underlying a thin pool be configured with available permanent spares. RAID-6 provides some extra insulation from a failed array group from destroying a pool.

When designing a configuration involving thin devices (or any other approach that results in devices that are widely striped), device level availability implications should be carefully considered.

Performance

General performance The basic rule of thumb is that designing and configuring for dynamic configuration performance is the same as designing and configuring for static provisioning performance except it is a lot easier. Using Dynamic Provisioning automatic performance optimization the news is all good! So, if you start a design with a workload requirement for X IOPS and Y GB you would do analysis to determine a specific RAID level and drive type. With static provisioning this might imply a configuration of A array groups, each of RAID-R. If you wanted to put the same workload onto HDP you would want the same (or

12

larger) pool design - A array groups, each of RAID-R. The workloads aggregate together for all the applications you will be putting on the pool. In terms of existing applications, Dynamic Provisioning is best able to deliver a performance improvement if the original workload had cool-spots and hot-spots across the storage. If this same workload is aggregated using HDP then the hot-spots lower their heat level and the low-spots move up in heat level and overall the system performs better. If the system was already optimally balanced less gain may be seen. Performance design requirements

A pool should be constructed from four or more array groups. If you have multiple BEDs then spread the load evenly across them. The array groups used in HDP pools should not be used for any other

purpose. The array groups used in HDP pools should be used for only one pool. Pools must be homogenous: All internal or all external. All same disk type All same disk rotational speed All same Raid Level With identical Pool-VOL sizes Pools volumes should occupy the whole array group.

There are few hard limits on any of the above. You can create a pool which breaks all the above rules but we don't recommend it for production use.

Common pool or separate pool The advantage of larger pools is that there is more opportunity for smoothing out workload differences. In general the bigger the better. Tests have been done on an exchange database with separate pools for log/data and a common pool. An overall reduction in access time was observed for the common pool. You should however put the database and log files on different V-VOLS so that the cache algorithms can schedule for the different random/sequential characteristics. The possible disadvantage of large pools with multiple workloads is that you cannot prevent one workload from "stealing" all the performance. For replication, we recommend that you put P-VOLs and S-VOLs in separate pools, just as with static provisioning.

Performance of mixed workloads:

We have found very good results combining logs and data (WP hoped for).

13

There is an argument that the larger you make a pool the more likely you will be to benefit from natural load balancing and reduction in hot spots.

But there is the counter argument that some workloads must be isolated from one another.

o These are often operational and management decisions outside of a technical driven storage design

o So, you might pool together several workloads that had similar, level, characteristics. E.G. several OLTP systems.

o But avoid mixing together a bursty workload (BI or DW) in with a response sensitive one like OLTP.

You should however get some interesting benefits from mixing different workloads. There are four rough classes. See the matrix below:

Requirement Few GB Many GB

Many IOPS HL HH

Few IOPS LL LH

If the IOPS requirements are all low then any pool design will work. You might consider the lowest cost approach with Raid-5, SATA or External Storage.

The more GB you have in the pool the more spindles it needs to implement it. As a consequence the Many IOPS + Many GB requirements are relatively easy to do - but you must do the analysis to determine you have adequate support.

The problem one is Many IOPS with few GB. This cannot work, either there are too few spindles for the IOPS or too many spindles for the capacity! This is true on Dynamic Provisioning or normal provisioning.

But you can leverage the perfromance levelling effect of large pools with wide-striping...

A low IOPS+ many GB application has IOPS to spare! If you combine this workload in the same pool with a Many IOPS with few GB appliction then both requirements are served. You might even be able to

Performance and Capacity Planning Tools Performance Monitor has limited reporting capability, but it can show real-time DP-VOL performance statistics. Tuning Manager has the best capability for reporting and trending of Pool and DP-VOL utilization, as well as I/O statistics. It will not report exactly which Pool volumes or array groups that a DP-VOL is “striped” across, but most DP-VOLs should be spread across all Pool volumes assigned to the Pool. Tuning Manager also provides implementation of additional thresholds besides the 3 standard thresholds provided by Dynamic Provisioning.

14

Storage Navigator and Device Manager are used for provisioning and utilization information. Device Manager groups Pool volumes under the Open-Reserved category, and reports these LDEVs with Volume Attr of DP-Pool. With 6.2 Device Manager has a new Pools tab and shows and can report on the total Virtual Capacity provisioned from a Dynamic Provisioning Pool. This is the total capacity that can be demanded of the Pool. It also shows the amount consumed from the Pool and the amount of capacity provisioned and consumed for a DP-VOL. The latter represents aggregate total of all Pool pages allocated to the DP-VOL. This is also reported in Tiered Storage Manager and Tuning Manager. You can use Device Manager CLI to automate reporting and optionally provide summaries.

Server Side

Server side considerations With USP V and VM systems it is recommended that Windows servers be placed in Host Storage Domains with a mode setting of 2C (Windows Extension). This is more future proof than 0C and will be a pre-requisite for future enhancements to HDP and other HDS software. No additional overhead is incurred by using mode 2C NTFS partitions should be formatted with the Windows Quick Format option. A Windows Full Format on Windows Server 2008 beta version has been seen to consume all possible pages for a DP-VOL. Avoid tools which write all over the disk:

Defragmentation low level UNIX media format or check volume level copy tools such as dd – use file level copies instead

And for the same reason, don’t use software RAID-1 mirror or RAID-5.

Managing space and server side expansion

From a server admin point of view, every system should:

Have only moderate over-provisioning (the amount would depend on plan and platform flexibility).

Leverage volume managers where feasible. Volume managers are more likely to have thin friendly options.

Size partitions/file objects/volume objects for future growth Have monitored use of both virtual volumes and the Pool:

15

o monitor usage file side vs. pool side - alert on discrepancy - if present, confirm whether expected, if not change data class to: "leaky", "bad managed" etc.

o monitor rate of usage vs. planned rate of usage - alert on discrepancy - if present, do management review for impact analysis

o monitor usage vs. planned usage - alert on discrepancy - if present, do management review for impact analysis

Have expansion plan: when and how often to increase

Where applications will over time fill in thin volumes, such as with some file systems and databases use a server partition expansion strategy:

1. Create and map over-provisioned volume 2. But don’t allocate all to partition 3. When top of partition is reached OS is forced to reclaim 4. If total space gets low 5. Use OS to expand partition 6. Can be scripted 7. Generally easier than hardware add, lunresize etc.

An example: Databases are recommended to use auto-grow feature, with growth increment set at whole number multiple of 32/42MB page size and ideally with size <= 1GB

Filesystems Long term behavior of each filesystem will depend upon application/user interaction and in most cases over time the day-to-day file creations and deletions will cause the capacity allocated to be consumed. At this point the benefits of improved provisioning and performance optimization will continue to apply, but continued over provisioning would require additional physical storage. The following table shows the initial Dynamic Provisioning pool consumption by Operating System and filesystem and gives an indication of which have tested to be thin friendly.

OS File System Meta-Data Writing

(The case of Default parameter)

Initial DP Pool Capacity Consumed

Windows Server2003 NTFS Writes metadata to first block.

Small (one page)

Windows Server2008 NTFS Under investigation Under investigation Linux XFS Writes metadata in Depends upon allocation group size.

Thin Friendly Low advantage No advantage

16

Allocation Group Size intervals.

The amount of pool space consumed will be approximately USP V/VM - [Virtual LU Size]*[64MB/Allocation Group Size] AMS2000 - [Virtual LU Size]*[32MB/Allocation Group Size]

Ext2 / Ext3 Writes the meta-data in 128MB intervals.

About 30% of the size of the Virtual-LU.

UFS Writes the metadata in 52MB increments.

Size of Virtual-LU.

VxFS Writes metadata to first block.

Small (one page)

Solaris

ZFS Writes metadata in Virtual Storage Pool Size intervals.

Small (when the Virtual-LU is more than 1GB)

JFS Writes metadata in 8MB intervals.

Size of Virtual-LU

JFS2 Writes metadata to first block.

Small (one page)

AIX

VxFS Writes metadata to first block.

Small (one page)

JFS(VxFS) Writes metadata to first

block Small (one page) HP-UX

HFS Writes metadata in 10MB intervals

Size of Virtual-LU.

Windows Server2003

NTFS Writes metadata to first block

Small (one page)

Windows Server2008

NTFS Under investigation Under investigation

VMware

Linux Ext2 / Ext3 - About 30% of the size of the Virtual-LU.

Dynamic Provisioning and VMware Dynamic Provisioning is a popular choice with VMware. Many Dynamic Provisioning customers are using VMware for servers. VMware works as you would hope, just use the rules appropriate for the client OS (windows NTFS, Linux, etc.). VMFS itself is thin friendly:

It generates little file system metadata o there is only 1 VMFS formatting option that is not thin-friendly

(eagerzeroedthick) VMFS reclaims space efficiently Most Recently Used Space Allocation Can leverage over-provisioning both at the VMWare level and at the client

OS leve). Is sparse (if you use sparse and MRU flags) - even when copying? The most important thing is to avoid putting too many guests on the same

LUN (where 1 DP-VOL = 1 LUN = 1 VMFS in VMware) to limit issues with SCSI reserve contention; recommendation is 5 per, and no more than

17

10. The cases implemented have been using a mixture of Windows, Unix & Linux guests.

Regarding thin-friendliness, when you add a LUN (DP-VOL) to VMware control, you either create a VMFS on it or you define it as an RDM (raw device, pass-through to the guest OS). VMFS is thin-friendly, it only writes metadata at the top (may use additional space based on snapshots or clones). RDM writes nothing on the DP-VOL, so thin-friendliness is entirely dependent on the guest OS’ file system (i.e. Windows NTFS vs. Linux EXT3).

After creating the VMFS, you create a virtual disk for each guest OS (or possibly multiple virtual disks per guest); this is analogous to creating an LV in Veritas VxVM. There is a parameter that controls whether VMware will perform a hard format or write zeros on the virtual disk. All values for this parameter except one, including the default value, will not cause any page allocation. Bottom line, as long as you stay away from option “eagerzeroedthick”, then at the point that you have deployed the virtual disk (also known as a VMDK) to the guest, that VMDK will still be “thin”. After this, whether it remains thin depends on: (a) the file system used by the guest (i.e. NTFS vs. EXT3 vs. Solaris UFS), (b) how the Application adds/deletes files and (c) the reclaim policy of the file system (LRU vs. MRU vs. something in-between like VxFS’ allocation extents).

Just like with normal open systems hosts, if a server/guest will not be thin-friendly due to its file allocation pattern, in order to constrain its consumption of Pool space and keep it from eating up too much of the Pool, you must either not over-provision or limit the amount of over-provisioning.

This is very easy with VMware, because the VMware SysAdmin can limit the size of a particular VMDK, constraining that guest to the limited size, but have a much larger VMFS that is itself over-provisioned. In this way, when the guest really runs out of space on his VMDK, the VMware SysAdmin can expand the VMDK giving it more of the leftover or virtual space in the VMFS.

In the case where you are cloning VMware virtual disk onto thin provisioned Virtual Volumes, note vmkfstools writes, byte for byte, a copy of the importfile or source virtual disk file to the new cloned virtual disk. vmkfstools offers disk format options during cloning that are similar to those offered by the creation command, but the list is limited to rdm, rdmp, raw, and thin formats.

In a typical cloning operation, an administrator might use a small “golden image” of a bootable operating system disk, use vmkfstools to clone it, and boot another machine. Ideally the golden image is small, utilizing just enough space to load the operating system, while the cloned machine may require significantly more storage. The challenge comes in trying to meet the needs for a larger volume without over-allocating storage.

18

There are two ways to create clones that continue to preserve space. You can create a small boot operating system image, clone it with vmkfstools, then use the vmkfstools -x option to extend the size of the virtual disk to the desired, larger virtual disk capacity. When vmkfstools extends virtual disks, it does so without zeroing out the underlying volume, resulting in a virtual disk that is thin. The alternate is to use the vmkfstools -d thin option when cloning the image. This option creates a VMware virtual volume of the same size as the original, which could also be using the VMware thin volume disk format.

Operations

Growing pools Expansion of a pool must maintain the original design rules. The whole pool must be:

All internal or all external. All same disk type All same disk rotational speed All same Raid Level With identical Pool-VOL sizes

When expanding a Pool: For Pools created on an AMS2000 system or on the USP V/VM before v5

of HDP: Always perform expansion with addition of the same number of dedicated AGs as the Pool was originally created with.

o For example, if Pool is initially created with 4 AGs, expand with 4 more; if initially created with 8 AGs, expand with 8 more.

o Assuming the initially assigned AGs are virtually exhausted of free pages, adding in equal increments of #AGs is critical to maintain the applications’ existing performance characteristics.

For Pools created on the USP V or VM with v5 or later: Expand the pool using any number of dedicated array groups. HDP v5 performance an automatic rebalance of page assignments across the pool so adding even one array group will retain a balanced pool result.

STRONG RECOMMENDATION: Always expand HDP pool so that it has at least 20% free space remaining, to maintain page allocation dispersion

Deleting devices

Thin devices can be deleted once they are unbound from the thin storage pool. When thin devices are unbound, the extents that have been allocated to them from the thin pool are freed, causing all data from the thin device to be discarded.

19

Data devices may also be disabled and removed from a storage pool. Prior to disabling a data device, all allocated tracks on the data device must be released by unbinding the thin devices that have allocated extents on the data device. Because data on the thin device will be striped across all enabled devices in the pool, in practice all thin devices in a pool will need to be unbound before data devices can be disabled and removed from the pool.

Migrations

On the USP V and VM pretty much all the standard migration techniques available with non-HDP volumes are now available including pool to pool migration.

This includes migrating dynamic-provisioned volumes to static-provisioned and migrating static-provisioned volumes to dynamic-provisioned but of course the second would fully allocate the target DP-VOL.

As always with migration, the volumes must be the same capacity at the time of migration (and not changing in size).

Any other normal limitations also apply. You cannot migrate back to the same pool. The target pool cannot be full or blocked (obviously) and you’ll be warned

if you would cause it to go over a threshold.

On the AMS, on the storage system only Shadowimage is supported for migration first release. On both the AMS and USP, the server-based Symantec (Veritas) Volume Manager and SmartMove are useful when migrating old systems from traditional LDEVs to DP-VOLs as this is friendly to the oversubscription feature of HDP. Migration Restrictions You can’t migrate LUSE to non-LUSE (whether it’s Dynamic Provisioning or not). This isn’t a Dynamic Provisioning issue. There is a method to do this but its complex and not an end-user option. HDS GSS does have a service for this, as part of a GSS migration engagement we can:

1. virtualize the USP V behind itself 2. the LUSE is now presented as a non-LUSE 3. we then migrate that.

You cannot migrate POOL-vols. Therefore you cannot change a pool from Raid5 to Raid10; you have to move the DP-VOLs not the pool.

Zero Page Reclaim For existing traditional volumes that have migrated to Dynamic Provisioning virtual volumes, on the USP V and VM Hitachi Dynamic Provisioning supports the ‘Zero Page Reclaim’ operation. See the figure below. Once (1) a traditional

20

volume has been migrated via Tiered Storage Manager, or some other full volume copy technique, then (2) the target Dynamic Provisioning volume’s physical pool capacity can be examined and where the firmware determines that no data other than binary zeros exist on a HDP pool page (modulo 42M on a page boundary) then the physical storage is unmapped from the HDP volume and ‘returned’ to the pool’s free capacity.

For volumes that have been ‘thin friendly’ but were initially set up on traditional ‘thick’ physical volumes, Zero Page Reclaim can result in a substantial initial storage reclamation after migrating to a dynamic provisioned volume. A 42MB page of all zeros will likely be ones that have never been written to. This is because on the HDS storage system, pages are set to zeros when they are formatted. For less common cases where pages have subsequently been meaningfully written with zero data, it again works because it looks no different than newly formatted capacity. To initiate a Zero Page Reclaim operation after a volume migration the user must use Storage Navigator and select the dynamic provisioned volumes to scan. The firmware then scans each page mapped to the HDP volume and when it finds a page of all zeros, the page is unmapped from the volume, the physical page pointer is replaced with the default zero pointer, and the unmapped page is then available for future physical capacity requirements. Note however that Zero Page Reclaim will not be useful to reclaim space from deleted files or be helpful in an ongoing effort to maintain thinness on a volume that due to application or filesystem behavior will not stay thin. It is also not generally useful to reclaim space a filesystem has freed up. Zero Page Reclaim is only beneficial after initial migration/restore actions where the original volume was heavy with untouched space. You must be at microcode level 60-04-04 or higher to use this feature.

21

Hitachi Data Systems Corporation Corporate Headquarters 750 Central Expressway, Santa Clara, California 95050-2627 USA Contact Information: + 1 408 970 1000 www.hds.com / info@hds.com Asia Pacific and Americas 750 Central Expressway, Santa Clara, California 95050-2627 USA Contact Information: + 1 408 970 1000 www.hds.com / info@hds.com Europe Headquarters Sefton Park, Stoke Poges, Buckinghamshire SL2 4HD United Kingdom Contact Information: + 44 (0) 1753 618000 www.hds.com / info.uk@hds.com Hitachi is a registered trademark of Hitachi, Ltd., and/or its affiliates in the United States and other countries. Hitachi Data Systems is a registered trademark and service mark of Hitachi, Ltd., in the United States and other countries. All other trademarks, service marks, and company names in this document or on this website are properties of their respective owners. Notice: This document is for informational purposes only, and does not set forth any warranty, expresed or implied, concerning any equipment or service offered or to be offered by Hitachi Data Systems. This document describes some capabilities that are conditioned on a maintenance contract with Hitachi Data Systems being in effect and that may be configuration dependent, and features that may not be currently available. Contact your local Hitachi Data Systems sales office for information on feature and product availability. © Hitachi Data Systems Corporation 2009. All Rights Reserved