Considerations for using tsm in a SAN.pdf.pdf

8/10/2019 Considerations for using tsm in a SAN.pdf.pdf

1/23

1

IBM Software Group

August 2003 2003 IBM Corporation

Considerations for using TSM in a SAN

Steve Strutt,Tivoli Software, IBM [email protected]

Sept 2002

Revision 5

IBM Software Group | Tivoli software

Considerations for using TSM in a SANl 2003 IBM Corporation2

Agenda SAN Exploitation - LAN-Free backup Performance characteristics

RequirementsLAN, Hardware, Software, Device, dependencies

SAN considerationsDevice fail-over

HBA considerationsSAN design considerations

Device addressing considerations

Going Live

Testing, Diagnosing Problems

Hints and Tips

Question and Answer


2/23

2



LAN-free Backup

Advantages:- client data can be local or SAN-attached- transparent to application/database- takes backup traffic off the LAN- reduces CP cycles on backup server (no I/O)- faster speed (usually)- only one backup server needs administration

Disadvantages:- still requires CP cycles on client for backup I/O

- careful scheduling to avoid tape drive contention(or exploit disk pooling)

TAPE

FC device

DISK

LAN

DAT A

DAT A

DATA

Direct to tape

Disk pool staging



Performance characteristics

LAN-Free is not necessarily faster

Only network eliminated as bottleneck

Could be other bottlenecks

Tape drives, Disk subsystem

Data types

good performance for large files and databases

small files, performance limited by file system and TSMarchitecture

LAN-Free to tape

Potentially better performance for large files, as bottleneck becomes filesystem or tape device

Small files cause tape drives to stop-start more and drives drop out ofstreaming mode.

LAN-Free to disk

Ideal for small files, no stop-start overhead


3/23

3



Customer Performance Figures

Large UK High Street Retailer

TDP for SQL Server on ESS to 3584 LTO

Backup 61.4GB hour (17MB/s) to single drive

Restore 44.2GB hour (12.3MB/s) from single drive

NT Filesystem on ESS disk to 3584 LTO - small files

Backup 10GB hour to LTO

Restore 6.2GB hour from LTO

Could be slower than LAN if tape drives do more stop/start operations

Large UK Bank

TDP for Exchange to 3583 LTO

Backup 52GB/h (14.4M B/s) to single driveRestore 51GB/h (14.2MB/s) from single drive



Agenda - Requirements

Hardware

LAN

Library support for LAN-Free

SAN device support

Software Evolving TSM support for LAN-Free

TSM code dependencies


4/23

4



Requirements - LAN

LAN-Free still requires LAN for meta data For Large files and databases

Minimal usage

Small files

Maybe the same if not more meta data on LAN than data on SAN if filesare very small.

LAN performance and loading still important



SAN Device support

Initially hardware configurations were certified by Tivoli

many different combinations, not all could be tested

Common configurations now tested The Tivoli view is that hardware is transparent to TSM

If the hardware vendors have validated thedevices use in a SAN and TSM supports

the device, then it is supported in a SAN

configuration with

Tivoli Storage Manager


5/23

5



Library support for LAN-Free

Native TSM LAN-Free support for:

3494 Libraries

SCSI Libraries

Controlled via SCSI control path

SCSI connect

Fibre Channel Connect

NO Native TSM LAN-Free support for

STK and ADIC AML libraries

Require Gresham EDT on every TSM server and Storage Agent.

STK - ACSLS, Library Station (s390)

ADIC DAS for AML/2, AML/J



TSM evolving support for LAN-Free

LAN-Free supports

BA Client, file level

BA Client, Volume level/Image Backup

Prior to 5.1.5, the tape was rewound between each volume

No support for Backup Set restore

NT/W2K

Supported from 4.1.0, NT/W2K server and NT/W2K TDPs

4.2.1 for Backup Archive Client

Sun Solaris and AIX

Supported from 4.2.0, TDPs and Backup Archive Client

HP-UX

Supported from 5.1.0, Full TSM device driver support

Linux (x86, zLinux and zOS)

Supported from 5.2.0


6/23


7/23


8/23

8



Fail-over and Tape

SCSI Reserve/Release used to serialize access to drives

Maintains integrity if device definitions are wrong

Outstanding SCSI Reserves remain if server using drives go down

Drives only released if:

Failed server restarted

Drive powered off and on.

SCSI Reset issued

Potential Impact on other users

SCSI Attached Tape drives can be reset using SCSI Reset

Fibre attached tape drives do not honor SCSI Resets



Supported SCSI Fail-over configurations

TSM 4.2+ with MSCS on W2K

Devices registered in wizard, SCSI reset on fail-over

TSM 5.1.5 with HACMP on AIX

Devices defined in startserver script, issues verifydevice to reset device

Shared

Disk

Shared Tape

Shared

SCSI

Bus

TSM Server

in Cluster

TSM Clients

IP network


9/23

9



Supported Fibre Failover configurations

TSM Server

in HACMP cluster

TSM 5.2.0 with HACMP on AIX

Devices defined in startserver script, issues verifyfcdevice to reset device



Fail-over and LAN-Free and Tape Library Sharing

Outstanding SCSI Reserves remain if storage agents using drives go down Drives become unavailable to other LAN-Free/Tape Sharing users

Only released if:

Failed server restarted

Drive powered off and on.

Potential drive mapping issues

FC Reset issued

Potential Impact on other users

Potentially need more tape drives

Some horrible messages if TSM server tries to dismount tapes belonging to failedstorage agents.


10/23

10



Failure scenario

ANR8925W Drive DRIVE0 in library ATLP1000 has not been confirmed for use by server UKSAN1_SA

for over 1200 seconds.

Drive will be reclaimed for use by others.

ANR8336I Verifying label of DLT volume 00157D in drive DRIVE0 (MT6.1.0.1).

ANR8311E An I/O error occurred while accessing drive DRIVE0 (MT6.1.0.1) for SETMODE operation,

errno = 1.

ANR8355E I/O error reading label for volume 00157D in drive DRIVE0 (MT6.1.0.1).

ANR8311E An I/O error occurred while accessing drive DRIVE0 (MT6.1.0.1) for OFFL operation, errno =

1.

ANR8469E Dismount of DLT volume 00157D from drive DRIVE0 (MT6.1.0.1) in library ATLP1000 failed.

ANR9999D mmsscsi.c(1647): ThreadId Volume may still be in the drive DRIVE0 (MT6.1.0.1).ANR8446I Manual intervention required for library ATLP1000.

Server running Storage Agent goes down (hardware failure

Fibre loss), when using a tape device

TSM makes drive unavailable



Supporting LAN-Free fail-over No TSM support for fail-over of Storage Agent

Supporting Storage Agent Fail-over

Configure two storage agents, one on each server

Different Storage Agent Names

No dependency between TSM clients and Storage Agent

BA client and TDP failover, Storage Agent does not

BA client and TDP automatically use new SA on fail-over server

No device issues if LAN-Free client machine fails-over when not

using tape devices

Devices lost if LAN-Free client machine fails-over when using tape

drives

Require more tape drives

Procedures to recover lost drives


11/23

11



HBA and tape Setup considerations

Check configuration recommendations of tape hardware vendor

IBM recommendations

NT/W2K

Max Scatter-Gather must be set to 65 or greater

Unable to write to new tapes on Storage Agent

W2K creates tapes which cannot be read

TSM will check for this in 4.2.1.11 and higher levels.

Issues message, unable to use drive.

W2K and LTO use 5.0.2.4 or higher level of Ultrium LTO driver.



HBA Sharing

Sharing of disk and tape on same HBA not always supported byhardware vendors

Disk OK

Access to tape drives lost under high workload conditions

drives timeout, go offline

IBM supports disk and 3590 on AIX with 6227/8 adapter under moderate

workloads

IBM-SSG do not recommend sharing disk and tape in other configurations.

i.e. LTO

some evidence that it is OK in low workload environments, such asprevious NT/SQL server LAN-Free environment.

Possible solution, with new generation of adapters being certified tosupport dual use without restrictions


12/23

12



SAN design

Most SANs designed for disk access

data flow is optimized for hosts disks

stovepipe design, separate SAN islands

Tape backup requires flow across SAN

May need additional Inter-Switch Links

More ports because of HBA Sharing issues

DiskArray

TAPE



Attaching tape drives to SANs

Most tape drives are Arbitrated Loop devices Note some directors do not support direct attachment of FC-AL

devices

Brocade switches support Public and Private Loop devices in fabric

InRange OK

McData directors do not support Arbitrated Loop Devicesrequire use of Sphereon 4300 or similar departmental switch.

Or SCSI attached via Fibre-to-SCSI Gateway/Router

TAPETAPE

ED5000

4300 etc

TAPE


13/23

13



SAN Operational considerations

Using fibre channel arbitrated loops (Hubs)

During boot up sequence LIP will interrupt tape operations for shareddevices.

Rebooting servers can cause tape failures

When devices and servers are on the same loop, devices should not berebooted while tape is in use by a TSM server or Storage Agent

Power up sequence

SAN, tape devices, then TSM Server and Storage Agents

Scheduling to avoid drive contention

Storage Agents must wait if no drives available.

Use MountWait parameter to avoid backup failures Managing device access

Device Class, Mount Limit

Device PATHs



Device Addressing Considerations All device addresses are defined centrally on TSM server for LAN-Free

Library, Device and Path statements.

Each host knows devices by different device names

Changes in device names or SCSI addresses can cause failures

Requirement to manage device addressing

All definitions must point to same physical device

Question: How to map all device names to the same physical device?

Device Name

lb1.0.1.3

//./tape0

//./tape1

/dev/rmt0

/dev/rmt1

/dev/rmt/0st

/dev/rmt/1st

TSM Definition

Library Lib1 lb1.0.1.3

Drive Drive0 //./tape0


Path Drive0 /dev/rmt0

Path Drive1 /dev/rmt1

Path Drive0 /dev/rmt/0st


TAPE

TAPE

W2K

TSM Server

AIX

Storage Agent

Solaris

Storage Agent

F C d e v i c e

WWN1

WWN2


14/23

14



Tape

DrivesSAN

Gateway/Router

OS Device

Name to

TSM Device

Name

SCSI ID

to

OS Device

Name

Device

WWN

to

SCSI ID

Device WWN

SCSI ID to LUN

ID1

ID2

ID3

SCSI Bus

WWN

WWN

Host

TSMOS

DeviceDriver

HBA

Managing Device Addressing in SANs - 1 Host Device Names and SCSI addresses can change

devices added or removed

devices failing

A number of address mappings occur.WWN to SCSI address

SCSI address to device address

Gateway/Router - device SCSI address to LUN

Described in Redpiece: Managing device addressing of SAN attached tape foruse with Tivoli Storage Manager, REDP-0150-00



Managing Device Addressing in SANs - 2

Changes are more likely to happen if there are large numbers of devices andhosts in SAN.

New devices being added, or removed

Solution

Use HBA Persistent Naming

Fixes SCSI address to device WWN

Static device name mapping

Device Names remain unchanged

Fixed device name to SCSI address mapping

TSM 5.2

Automatic device tracking


15/23

15



HBA persistent naming support

HBA maintains a fixed WWN to SCSI address relationship Support matrix

Platform Emulex Qlogic

AIX Not Applicable(use 6227/8 adapter)

Not Applicable(use 6227/8 adapter)

WindowsNT/W2K

YES Yes(from 8.1.3 with

SANblade Manager)

Solaris YES YES



Persistent Naming with Emulex on Windows


16/23

16



Qlogic HBA on Windows

Qlogic from 8.1.3 onwards with SANBlade Manager/SANSurfer

Download management utility from Qlogic Website



Static device naming with Microsoft Windows TSM device driver uses a naming convention which does not change as

devices are added or removedmtx.y.z.n

LTO device driver uses default Windows device naming\\.\tape0

Can change if new devices added or removed

With W2K can change in-flight if devices added or removed from SAN

Recommendation to use mtx.y.z.n,

Use information from TSM Device Information screen to determinemtx.y.z.n device addresses

In TSM 5.1, the mtx.y.z.n name is shown.


17/23

17



Static device naming on AIX and Solaris

AIX

Device drivers automatically track devices using serial numbers

This mapping is static

Solaris

Static device naming convention

Uses symbolic link to map device name to SCSI address

Requires HBA Persistent Naming to be configured

HBA configuration file

l s l / dev/ r mt / *l r wxrwxrwx 1 r oot ot her 45 J an 3 14: 22 / dev/ r mt / 0mt - >. . / . . / devi ces/ pci @1f, 0/ pci @1/ scsi @2/ mt @5, 1:mt



TSM 5.2 Automatic device tracking

TSM tracks devices by serial number Specify S/N when defining devices

Automatic discovery of S/N when device defined

At start of each operation TSM server and SA will check the deviceis the one it expects it to be:

Windows Initiates a search for the device and changes mapping to point to

new device and then continues operation.

UNIX

Issues message and fails operation on that device

Avoids need for persistent binding on Windows Qlogic complex to set up.

Extra mgmt utilities.

Persistent binding options not tested by hardware vendors!


18/23

18



Mapping device names to devices -1

In a SAN all definitions for a shared device must point to the samephysical device.

Each host usually has a different device name for the devicedepending on the hardware and configuration.

Only common information available on each platform is the deviceWWN and serial number.

Device Name

lb1.0.1.3

//./tape0

//./tape1

/dev/rmt0/dev/rmt1

/dev/rmt/0st

/dev/rmt/1st

TSM Definition

Library Lib1 lb1.0.1.3



Path Drive0 /dev/rmt0Path Drive1 /dev/rmt1



TAPE

TAPE

W2K

TSM Server

AIX

Storage Agent

Solaris

Storage Agent

F C d e v i c e

WWN1

WWN2




1. Define drives on TSM server and determine relationship betweenTSM device name, host device name and device WWN/Serial No.

Also determine the Element Number of the device in the library

2. For each (Storage Agent) host determine WWN/Serial # and

device name for each tape device.

3. Use WWNs/Serial # to relate device names on each host to TSMdevice name.

4. Define drives on TSM server using PATH statements


19/23

19



Mapping device names to devices-3

TSM Device Name Drive0 Drive1 Drive

WWN/Serial No.

Library ElementNumber

TSMServer

StorageAgent1

Storage

Agent2StorageAgent3

HostDevice

Names



Mapping device names to devices Windows 1

TSM V5.1 TSM Utilities, Device Information

Shows a devices WWN and Serial Number as well as device address(mtx.y.z.n)

Also allows central discovery of Storage Agent device mappings

Requires SNIA HBA SAN Mgmt API to be installed


20/23

20



Mapping device names to devices Windows 2 TSM Server, Device Information display with Qlogic SNIA SAN

Mgmt API Installed Device information can also be displayed from this screen for

Storage Agents in the same Windows Domain




AIX

lsattr El mtx/rmtx shows devices WWN

lscfg vl mtx/rmtx shows devices Serial Number

Solaris

Relate device name to WWN using SCSI and LUN addresses

ls -l shows device name and SCSI/LUN mapping

dmesg output shows SCSI Target address to WWN mapping

l s l / dev/ r mt / *l r wxrwxrwx 1 r oot ot her 45 J an 3 14: 22 / dev/ r mt / 0mt - >. . / . . / devi ces/ pci @1f, 0/ pci @1/ scsi @2/ mt @5, 1:mt

dmesg ( / var / adm/ messages). . . . . .ql a2200-hba0- SCSI - t arget - i d-5- f i bre- channel - name="100000e00201d0d7";


21/23

21



Agenda - Going Live

Testing

testing

Diagnosing Problems

Storage agent messages

TSM device utilities



Testing

Test after every hardware change

Changes can introduce errors which can cause failure to recover data.

Proving data integrity

TSM 5.1 includes end to end CRC checking

use only during testing


22/23

22



Testing

Check tape hardware works reliably with TSM server in LANconfiguration first

Check TDPs and B/A Client work on LAN first Test each drive with each Storage Agent to check they are properly

defined and accessible.

Use BA Client

ANR8779E ( Sessi on: 7, Or i gi n: UKSAN4_SA) Unabl e toopen dr i ve / dev/ mt 1, er r or number =2.

I nval i d devi ce speci f i ed

err or number =16, Devi ce Busy ( SCSI Reserved t o another

sys t em)



Diagnosing problems

Storage Agent can be run in foreground session, to see allmessages.

All Storage Agent messages should be logged centrally in the serverActivity Log

Can issue commands from TSM server console

storage_agent1: QUERY SESSION

mttest and lbtest utilities

Provided in utilities or devices directory

test operation of tape devices or library operations.

Shows device serial numbers


23/23



Question and Answer

Considerations for using tsm in a SAN.pdf.pdf

Documents

Transcript of Considerations for using tsm in a SAN.pdf.pdf