Enterprise Removable Media Manager - University of...

IBM TotalStorage

Enterprise Removable Media Manager

A Service Offering forOpen System Tape Management

Ulf Troppens, eRMM Development, IBM Mainztroppens@de.ibm.com

IBM TotalStorage

Disclaimer

IBM TotalStorage

Agenda

eRMM OvervieweRMM DetailseRMM Offline Demo (Screenshots)eRMM & TSMeRMM RoadmapConclusion

IBM TotalStorage

Agenda

IBM TotalStorage

Customer Problems and Requirements today

Libraries with SCSI Media Changer(e.g. IBM 3584)

Removable Media Hardware

Media Changer Devices

Streaming Devices (Drives)

SCSI/FC SCSI/FC

Removable Media Hardware

Media Changer Devices

Streaming Devices (Drives)

SCSI/FC prop. InterfaceLibrary Mgmt.

Libraries with integratedManagement (e.g. IBM 3494)

Problems:• No centralized access control, administration and reporting

• Media management has to be done by every application

• Difficult (or nearly impossible) to share libraries, drives and scratch pools, especially in heterogeneous environments

• Multitude of Media Changer andLibrary Management Interfaces(SCSI, IBM 3494, STK ACSLS/LS, …)

• Adding storage or upgrading storage technology requires changes to every application

Requirements:• Improved manageability to cope with rapidly growing storage• Reduced TCO• Efficient usage of tape storage capacity• Increased utilization of tape drives• Integrated support for vaulting

IBM TotalStorage

Customer Scenarios

Statically partitioned IBM 3584

Legato Networker and HiBack Server

TSM Server

…EDT

STK SN 6000

STK Silos

STK ACSLSServer

IBM Tape Hardware

TSM on z/OSTSM LAN-free clients for open systems

Customer wants:• consolidated media

management for Open Systems first, later with z/OS too

• centralized administration and control

• to get rid of Gresham EDT

Customer wants:• enhanced utilization of drives• to create drive groups and

cartridge pools that span multiple libraries

• centralized media management administration and control for Networker and HiBackapplications

• enhanced monitoring and reporting

Customer wants:• consolidated media

management for heterogenous environment

• Integrated support for vaulting• policy-based cartridge

lifecycle management

Scenario IScenario II

Scenario III

IBM TotalStorage

Case Study: Backup Consolidation on zLinux (1/2)

Business needThe customer wants to consolidate its backup infrastructure with TSM on zLinux.

Business valueThe customer builds a backup infrastructure which is very flexible and which scales very well. The advanced management functions of zLinux/VM allow to provision new TSM Servers on demand.

Use case (planned new service)At the customer a business unit can request to establish a new security zone including a dedicated VPN to protect certain applications and its confidential data. The infrastructure team automatically creates a new zLinux Instance with a new TSM server which is only visible in the new VPN. The concept of having a dedicated TSM server for each security zone simplifies the mechanisms to secure confidential backup data.

IBM TotalStorage

Case Study: Backup Consolidation on zLinux (2/2)

Similar setup available in collocated data centerBoth data centers connected via DWDM (Gigabit Ethernet + Fibre Channel)Fail-over of zLinux instances between both data centers planned for laterNew TSM servers added on demand

eRMM TSM1

… TSMn

TSMn+1

TSMn+2

TSMn+3

… TSMn+m

z/VM 1 z/VM 2

40 zFCP Ports for Tape, 8 zFCP Ports for disk

Fibre Channel SAN

IBM 3584 Tape Library65 x IBM 3592 Tape Drive2320 x 3592 JA Cartridge

IBM DS 830014,7 TB RAID 5 netDisk cache for TSM

eRMM’s advanced tape management capabilitieshelp to master the complexity of this frequently changing setup.

IBM TotalStorage

Value of the Enterprise Removable Media Manager

Improve the flexibility of the removable media infrastructureResults

Improved Application Availability– Create cross library drive and scratch pools

Optimized Storage Resource Utilization– Share drives between heterogeneous applications– Provide a cross application scratch pool– Define policies that priories concurrent mount requests– Define policies to move cartridges to vaults

Enhanced Storage Personnel Productivity– Create a single point of control, administration, reporting, security and auditing

for removable media resources – Move, add or change removable media resources (drives, cartridges, libraries)

without requiring modifications to applications like TSM

IBM TotalStorage

Tape Configuration without eRMM

IBM 3584 IBM 3494 StorageTek

IBM TotalStorage

IBM 3584

IBM 3494 StorageTek

TSMCartridges

Networker Cartridges

STK DrivesLTO Drives

Common Scratch Pool

Drive Pool• Pools of capacity• Segmented basedon business need

Scratch Pool• For all applications• Segmented basedon business needs

Cartridge Pool• Access for owningapplication only

IBM TotalStorage

IBM 3584

IBM 3494 StorageTek

TSMCartridges

Networker Cartridges

STK DrivesLTO Drives

Common Scratch Pool

Policies• Drive and cartridgeallocation based onpriorities

• Cartridge movement• Cartridge life-cyclemanagement

New Efficiencies• Cross applicationdrive sharing

• Cross applicationscratch pool

• Mount request queuing• Centralized reporting• Audit trails• Integrated vaulting

IBM TotalStorage

Dynamic grouping and pooling of drives and cartridges

DriveGroup 1 DriveGroup 2

CartridgeGroup 1

Cartridge Group 2

Library 1 Library 2

DriveGroups:• Used to aggregate drives• Drives can be added and removed dynamically

• Used to implement access permission model and preferential usage policy

• May span multiple libraries

CartridgesGroups:• Used to aggregate cartridges• Cartridges can be added and removed dynamically

• Used to implement access permission model and preferential usage policy

• May span multiple libraries

IBM TotalStorage

Access Roles and Preferential Usage Policy for DriveGroups and CartridgeGroups

DriveGroup 1 DriveGroup 2

CartridgeGroup 1

CartridgeGroup 2

Library

App 1 App 2

Preferential Usage Policy:• DriveGroup-Application relations have priority attribute

• Drives have priority attribute too• Priority is used by eRMM to find drive for a mount operation

Access Roles:• Applications are subject to access control• Two – level access permission model• Administrative applications may access every Drive• Standard applications may only access Drives within DriveGroups which have been assigned to them by an administrative application

IBM TotalStorage

eRMM Features

Dynamic library and drive sharing across heterogeneous application boundaries and heterogeneous operating systemsDynamic drive and cartridge poolingMount request queuingPolicy-based drive and cartridge allocationPolicy-based media life cycle management (*)Integrated off-site media management and tracking (Vaulting) (*)Centralized access control, administration and reportingAdvanced reporting and auditing (*)

(*) Planned for future releases

IBM TotalStorage

App 3App 2App 1

Media Management and Sharing

Eliminate changes when storage is added and minimize impact of technology upgrades.

Adding storage or upgrading storage technology requires changes to every application

Provide integrated support for manually operated libraries and vaults together with policy-based cartridge life cycle management (planned for future release).

No integrated support for Vaulting

Provide centralized access control, administration and reporting accessible through open standards based API (IEEE1244) and SSG Console WUI

No centralized access control, administration and reporting

Provide centralized mainframe-class media management accessible through open standards based API (IEEE1244)

Media management has to be done by every application

Provide dynamic sharing of tape drives including mount request queuing + additional utilization enhancements through dynamic grouping and pooling of drives

Inefficient drive utilization because its difficult to share drives (physical mapping App->Lib. Partition)

Provide access to and sharing of a common scratch pool and dynamically resizable cartridge groups with access control.

Inefficient capacity usage because tape storage is statically partitioned by application

ERMM SolutionTop Customer Problems/Requirements

IBM TotalStorage

Agenda

IBM TotalStorage

eRMM Architecture

IBM 3584 IBM 3494 STK ACSLS

Media Manager

LibraryManager

Admin Console

DM DM DM DM

IBM TotalStorage

eRMM Architecture

Media Manager

LibraryManager

Admin Console

•Coordinates accessto drives and cartridges

• Central repository• Log of all activities

• Controls libraries onbehalf of the mediamanager

• Encapsulateslibrary hardware

• Command lineinterface and GUI

DM DM DM DM

• Controls driveson behalf of themedia manager

IBM TotalStorage

eRMM Control Flow

Media Manager

LibraryManager

Admin Console

(1.) TSM Server wants to mount a scratch tape

(2.) TSM ELM sends request to Media Manager via TCP

DM DM DM DM

IBM TotalStorage

eRMM Control Flow

Media Manager

LibraryManager

Admin Console

(3.) Media Manager selects scratch cartridge and idle drive

(4.) Media Manager forwards request via TCP

(6.) Drive Manager loads cartridge into drive

(5.) Library Manager moves cartridge to drive

DM DM DM DM

IBM TotalStorage

eRMM Control Flow

Media Manager

LibraryManager

Admin Console

(8.) Response to TSM Server via TCP and ELM

(7.) Response to Media Manager via TCP

DM DM DM DM

IBM TotalStorage

eRMM Control Flow

Media Manager

LibraryManager

Admin Console

(9.) Direct access to drive

DM DM DM DM

IBM TotalStorage

Agenda

eRMM OvervieweRMM DetailseRMM Offline Demo (Screenshots)eRMM & TSM eRMM RoadmapConclusion

IBM TotalStorage

eRMM Offline Demo (Screenshots)

I. SetupII. Drive and cartridge sharing

(between EMC/Legato Networker and TSM)III. ReportsIV. Advanced Reporting

IBM TotalStorage

Setup (1) – managed libraries

IBM TotalStorage

Setup (2) – managed drives

IBM TotalStorage

Setup (3) – managed cartridges

IBM TotalStorage

Setup (4) – authorized applications

IBM TotalStorage

Setup (5) – drive groups

IBM TotalStorage

Setup (6) – drive groups (authorized applications)

IBM TotalStorage

Setup (7) – cartridge group

IBM TotalStorage

Setup (8) – cartridge group (authorized application)

IBM TotalStorage

Sharing (1) – common scratch pool

IBM TotalStorage

Sharing (2) – Networker begins using one drive …

1. Allocate scratch volume

3. Start backup

2. Label volume

IBM TotalStorage

Sharing (2) – TSM wants two drives …

4. TSM is waiting for second drive

3. MOVE DATA needs two drives

IBM TotalStorage

Sharing (3) – eRMM synchronizes drive access …

5. TSM is blocked

IBM TotalStorage

Sharing (4) – Networker is done and TSM continues …

6. Networker is ready and unmounts cartridge

7. TSM gets second drive

IBM TotalStorage

Reports (1) – drive “LTO263” was shared

Drive Sharingbetween TSMand Networker

IBM TotalStorage

Reports (2) – cartridge groups have changed

IBM TotalStorage

Reports (3) – drive usage statistics

IBM TotalStorage

Advanced Reporting

The following screen shotsshow prototyped features,

which may become part of eRMMin a future release.

IBM TotalStorage

Advanced Reporting (1a) – Audit trailsStep 1: Select filter in GUI

Which Cartridge was mounted in

“DRIVE1” in January 2004?

IBM TotalStorage

Advanced Reporting (1b) – Audit trailsStep 2: Download report as CSV file

IBM TotalStorage

Advanced Reporting (1c) – Audit trailsStep 3: Import CSV file into Excel

IBM TotalStorage

Advanced Reporting (2a) – Error logsCentral error log file for all drive errors

IBM TotalStorage

Advanced Reporting (2b) – Error logsDetails for a specific error

IBM TotalStorage

Advanced Reporting (2c) – Error logsError statistics by day

IBM TotalStorage

Advanced Reporting (3a) – Drive UtilizationTotal number of drives in use

IBM TotalStorage

Advanced Reporting (3b) – Drive UtilizationUtilization of a single drive

IBM TotalStorage

Advanced Reporting (4) – PerformanceTotal throughput by day

IBM TotalStorage

Agenda

IBM TotalStorage

Value add in a homogeneous TSM environment (1/6)

TSM provides its own tape management capabilities(e.g. TSM Library Sharing)eRMM is designed to support heterogeneous applications

Why do I need eRMM in a homogeneous TSM environment?

IBM TotalStorage

Flexible provisioning of drives and cartridges– sharing of TSM drives with other applications like EMC Legato NetWorker

and OS tools like mksysb and tar– simplified setup of TSM LAN-free environments

no need to define tape paths inside TSMall tape paths are automatically generated inside eRMMcentral check whether drive is used by TSM Server or Storage Agent

– flexible adding and removing of drives to TSM storage poolsall changes are handled inside eRMMupdates of the TSM servers or other applications are not required

– enabling of cross library scratch pools– enabling of cross library drive pools– automatic labeling of scratch cartridges (can be disabled in config file)

IBM TotalStorage

Advanced reporting and auditing– eRMM provides statistics on how many drives were used when by

which application (future).– eRMM provides statistics on how many data was written during a

mount. This data can be aggregated to provide throughput reportsfor drives, drives groups, applications and a whole library.

– eRMM provides statistics on how often a cartridge was mounted and how many data was written/read for the whole cartridge life cycle (future). TSM looses its data, when a cartridge is returned to the TSM scratch pool.

– The TSM server stores tape alerts in the TSM activity log. In addition to that, eRMM stores all log sense pages in the eRMM database. This additional data can be very useful for error analysis.

IBM TotalStorage

Enhanced access control to library resources– In an outsourcing environment eRMM can share the drives across several

TSM servers whilst it strictly separates the cartridges of each TSM server.– eRMM can also provide a separate scratch pool for each TSM server thus

cartridges are not exchanged between the TSM servers.

Additional operational benefits– dismount of cartridge when TSM server goes down– physical erase of cartridges before return to scratch pool (future)– improved cartridge life-cycle management

(e.g. threshold for wornout tapes) (future)

IBM TotalStorage

Value add in a homogeneous TSM environment (5/6)Customer EnvironmentThe customer backs-up a large application server using TDP for Oracle. This backup job starts four sessions and thus uses four drives.

Customer IssueWhen TSM starts a backup of the TSM database and all drives are in use, then TSM dismounts one volume and uses it for the TSM database backup. As a result the backup of the large application server fails, even when it was running for several hours and even when this backup job was nearly complete.

eRMM SolutionConfigure TSM with two external libraries which use eRMM managedresources. Use one library for the backup of the application server and the other for the backup of the TSM database. Inside eRMM map these two libraries to the same resources (DriveGroup, CartridgeGroup).

As a result eRMM will queue the backup of the TSM database.Now the TSM administrator can decide which job to cancel.

IBM TotalStorage

TSM provides its own tape management capabilities(e.g. TSM Library Sharing)eRMM is designed to support heterogeneous applications

eRMM’s advanced tape management capabilitiesprovides value add even in a homogeneous TSM environment!

IBM TotalStorage

Agenda

IBM TotalStorage

Supported platforms (eRMM 1.2)eRMM Media Manager

– SLES 8 for zSeries GA– AIX 5.2/5.3 GA– SLES 8 for Intel RPQ

Applications– Tivoli Storage Manager GA– Native OS commands (e.g. tar, mksysb) GA– EMC Legato Networker RPQ– Veritas NetBackup future– iSeries BRMS future– zSeries DFSMSrmm future– Other backup applications future

Tape Libraries– IBM 3584 Open Systems GA– IBM 3494 Open Systems GA– StorageTek ACSLS GA– StorageTek HSC-LS RPQ– Other tape libraries future

IBM TotalStorage

Implemented features (eRMM 1.2)

Dynamic/heterogeneous drive sharing GADynamic/heterogeneous drive and cartridge pooling GAMount request queuing GATape library virtualization GAPolicy-based drive and cartridge allocation GACentralized access control, administration and reporting GAPolicy-based cartridge lifecycle management futureOffsite-media management and tracking (vaulting) futureAdvanced reporting and auditing future

IBM TotalStorage

Agenda

IBM TotalStorage

Summary

eRMM is an enterprise tape library virtualization solution whichscales very well and provides very high availability and reliability.eRMM provides mainframe class, centralized management functions like tape cartridge pooling, tape drive pooling, tape drive sharing, mount request queuing, policy based cartridge lifecycle management, offsite media management, centralized access controland auditing, and advanced reporting functions.eRMM’s intelligent sharing capabilities increase tape drive, tape cartridge and therefore tape library utilization whilst it reduces the time spent for tape library administration.

The goal of eRMM is to dramatically reduce the Total Cost of Ownership for data stored on tape by improving its access and its management across heterogeneous application, server, and tape library boundaries.

IBM TotalStorage

eRMM Resources

Internethttp://www.ibm.com/Search/?q=eRMM&v=14&lang=en&cc=us

eRMM Business DevelopmentDetlef KrauseMail: dkrause@de.ibm.comMobile: +49 171 3 33 87 41

Enterprise Removable Media Manager - University of...

Documents

Transcript of Enterprise Removable Media Manager - University of...

Using IBM® TotalStorage® Enterprise Storage ... - VMware€¦ · VMware ESX Server. With the launch of VMware VirtualCenter in 2003, and the groundbreaking VMotion™ technology,

IBM TotalStorage DS4000 Storage Manager Installation and ...ps-2.kev009.com/pccbbs/pc_servers_pdf/gc26-7705-00.pdf · IBM TotalStorage DS4000 Storage Manager 9 Installation and Support

SAP liveCache 7.5 and MaxDB 7.5 on IBM TotalStorage

IBM TotalStorage FAStT Storage Manager Version 9: Installation …ps-2.kev009.com/pccbbs/pc_servers_pdf/gc26-7648-00.pdf · 2004-08-31 · IBM TotalStorage FAStT Storage Manager Version

Using IBM® TotalStorage® Enterprise Storage Server ... · IBM® TotalStorage® Enterprise Storage Server® FlashCopy® Function with the VMware ESX 2.5 Server ... Figure 1 Disk

Performance - Oxford University TSM Symposiatsm-symposium.oucs.ox.ac.uk/2003/papers/PeterPijpelink.pdf · • Make sure you monitor actlog for hardware errors. Network performances

Veritas Volume Replicator and IBM-Enterprise …eval.veritas.com/.../High_Availability/ibm_veritas_vvr_wp.pdfStorage Solutions with the IBM TotalStorage ... needed to backup the data.

IBM Storage Virtualisation Technologies - University of …tsm-symposium.oucs.ox.ac.uk/2005/papers/Storage Virtualization... · IBM Storage Virtualisation Technologies ... 6 IBM TotalStorage

Planning and implementing IBM Tivoli Storage Managertsm-symposium.oucs.ox.ac.uk/2003/papers/StevenPemberton.pdf · 1 Planning and implementing IBM Tivoli Storage Manager Steven Pemberton

IBM TotalStorage LTO Ultrium Tape Drive: SCSI Reference · IBM TotalStorage LTO Ultrium Tape Drive SCSI Reference GA32-0450-00

IBM TotalStorage FAStT Hardware Maintenance Manual and Problem Determination Guide

IBM TotalStorage Peer-to-Peer Virtual Tape Server Planning and Implementation Guide€¦ · · 2006-06-292.5.5 Selective Dual Copy ... vi IBM TotalStorage Peer-to-Peer Virtual Tape

Front cover IBM TotalStorage Enterprise Storage Serverps-2.kev009.com/basil.holloway/ALL PDF/sg245420.pdfIBM TotalStorage Enterprise Storage Server models F10 and F20 New Fibre Channel/FICON

IBM TotalStorage DS4000 EXP700 and EXP710 Storage Expansion

NAS Attached SAN Attached Storage IBM TotalStorage

IBM TotalStorage ® IBM logo must not be moved, added to, or altered in any way. © 2005 IBM Corporation IBM TotalStorage ® Virtualise for Tiered Storage.

IBM TotalStorage DS4500 Fibre Channel Storage Server: User ...ps-2.kev009.com/pccbbs/pc_servers_pdf/gc26-7726-00.pdf · Storage Server User’s Guide GC26-7726-00. IBM TotalStorage

TotalStorage FAStT Storage Manager Version 8 - …ps-2.kev009.com/pccbbs/pc_servers_pdf/gc26-7519.pdf · TotalStorage FAStT Storage Manager Version 8.3 Installation and Support Guide

IBM TotalStorage New IBM TotalStorage Enterprise Tape Drive 3592 · 2010-08-11 · ® TotalStorage ® Enterprise Tape Drive 3592 Product Directions: Statements in this document represent

TotalStorage DS300 M DS400 - ps-2.kev009.comps-2.kev009.com/pccbbs/pc_servers_pdf/40k4630.pdfWNGzm,kjITB=h: