Optimizing IT Data Services - Gateway/400 · PDF fileOptimizing IT Data Services ... Storage...
Transcript of Optimizing IT Data Services - Gateway/400 · PDF fileOptimizing IT Data Services ... Storage...
© 2013 IBM Corporation
Optimizing IT Data Services
Christopher L Poelker
© 2013 IBM Corporation
US Commission on Cloud Computing
Commission on the Leadership Opportunity in U.S. Deployment of the Cloud
(CLOUD2)
Commission Full Report Text Available online at: www.techamericafoundation.org/cloud2
http://www.computerworld.com/author/chris-poelker/ComputerWorld Blog
Storage Area Networks for Dummies:
http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470385138.html
3
Tech over Time: Creation vs. Evolution
One Shot and
Hope you Hit it
Random Evolution
Intelligent Design
Omnipotent and
Omnipresent
APPROACHES TO IT MATURITY
4
Analysts take on IT Maturity
MATURITY REQUIRES CHANGE
Today
5
The IT Life Cycle Runs in Waves
5
http://www.computerworld.com/blog/intelligent-storage-networking
Mainframe Computer
Converged
1960 1970 1980 1990 2000
Mini-Computer
PC Computer
Distributed
Utility Computing
Dot Com Bust
Converged
Laptop
Tablet
Smartphone
Distributed
2003 2009 2012 2014
Cloud Computing
Hyper-Converged ?
Virtualization
2017
Similar to Gartner Tech Hype-Cycle
IT CHANGES RAPIDLY
The State of the Art of Storage: Data Moving Towards the App.
Movement of Data
The Cloud
7
A SINGLE APPROACH MAY NOT BE FOR EVERYONE
© 2013 IBM Corporation
NIST: Cloud Reference Architecture
© 2013 IBM Corporation
Infrastructure Working Group: Choosing Applications for the Cloud
© 2013 IBM Corporation
Cloud Formation Chart
IBM iSeries
© 2013 IBM Corporation
12 Steps to Become Cloud Ready
1- Enable application and data mobility by virtualizing servers and storage
2- Audit applications to assess areas where cloud would be beneficial
3- Embrace encryption at rest and robust key management guidelines
4- Assess utilization / costs of existing infrastructure and operations
5- Determine data growth trends and dedupe or delete where required
6- Audit data assets by capacity and access metrics and assign classes
7- Create data storage tiers for structured and unstructured data classes
8- Consolidate infrastructure and minimize complexity (Policies / Automate)
9- Perform detailed analysis of application interdependencies
10- Outsource where appropriate
11- Head to the nearest Bar
12- Don’t Worry, Be Happy!
Challenges in IT
12
CURRENT REALITIES
Application Security, Regulations, Records Management, etc..
Applications & Platforms (Physical & Virtual)
Storage Platforms (Physical, virtual, SSD, Spindles, Tape)
Typical Data Center
13
CURRENT REALITIES
IT Operations Stack
14
CURRENT REALITIES
Topology of an Optimized Data Center
Production Disk
Pools
Abstraction/
Data Services
WAN
Physical Physical Virtual
Compute Resources
Sector-level
duplicate
data
elimination
Virtual
Disk to Disk
Backup Pool
FC
SAS/SATA
DR Disk Pools
Physical Virtual Virtual
Abstraction/
Data Services
Physical
and
Virtual
Tape Pool
Compute Resources
MM
Thin
T1 Pool
Thin
Backup Pool
Backup
Node
Thin
T2 Pool
Thin
Tiered DR Pool
Backup
Node
> 40% Reduction in Complexity, 100% Data Agility
Current With Intelligent Abstraction
The IT Stack
16
SIMPLIFY AND AUTOMATE
Intelligent Abstraction (IA) combines the power of virtualization and artificial intelligence with policy based unified data services
to enable freedom and automation of IT
+
The Advent of Intelligent Abstraction
17
THE NEXT EVOLUTION
-
18
IT MATURITY PYRAMID
Distributed Hyper-Converged
Policy Based Distributed Converged = 2 x IBM i
Virtual Abstraction + Unified Data Services
19
SOFTWARE DEFINED REFERENCE ARCHITECTURE
Rest API&
Policy Manager
Intelligent Abstraction Services Stack
Application Integration Services
Converged Network Services
Data Mobility Services
Automated Protection Services
Storage Virtualization Services
Distributed Hyper-Converged Building Blocks
20
INTELLIGENT HYPER-CONVERGED
Compute Resources Compute Resources
Physical VirtualPhysicalVirtual
M
Cloud
= management point
M M
Intelligent Abstraction Services Stack
Application Integration Services
Converged Network Services
Data Mobility Services
Automated Protection Services
Storage Virtualization Services
Intelligent Abstraction Services Stack
Application integration Services
Converged Network Services
Data Mobility Services
Automated Protection Services
Storage Virtualization Services
Securely Manage, Protect, and Share Information
© 2013 IBM Corporation
Five Steps to Optimize IT Data Services
1.First focus on low hanging fruit: Backup and Continuity
2.Implement snapshots and continuous data protection
3.Leverage protection storage for test and development
4.Virtualize servers and storage to consolidate and commoditize
5.Centralize and automate operations
Before moving to the cloud, first optimize current IT infrastructure and operations to
reduce costs and then assess whether moving to a public cloud makes sense.
© 2013 IBM Corporation
What are Data Services?
Focus on the 4 Main Aspects of IT Data Services
Provisioning
Protection
Replication
Recovery
© 2013 IBM Corporation
Changing the Data Services Paradigm through Innovation
1. Server, Storage and Tape Virtualization
2. Continuous Data Protection and Snapshots
3. Global Deduplication and WAN Optimization
Technical Innovations to Optimize Data Services
© 2013 IBM Corporation
Virtualization: Complete Data Mobility
Move or copy data between tiers or arrays without application downtime
Migrate data to different storage tiers while apps are up
Replicate data between unlike storage arrays
Implement thin provisioning on existing storage for better utilization
Increase overall performance
Synch
copy Asynch
copy
Copy
within
array
Abstraction / Data Services
SSD
IP/iSCSI
FC
Direct
Attached
Disk
© 2013 IBM Corporation
Continuous Protection :Change the Physics of Backup
Maintain a lower cost tier of protection storage for critical
system recovery and dev / test
Move data only
as it changes, not
in bulk, to minimize
impact
Move only unique data
to conserve space
Direct
Attached
Disk
Production
Storage Protection Pool
Data Services
Data Services
CDP
VTL
© 2013 IBM Corporation
Use Dedupe to Optimize Data Storage, WAN, and Archives
Optimize backup and archives with host-free, SAN-based
data movement from protection pool
Direct
Attached
Disk
Production PoolProtection Pool
CDP, VTL and Snapshots
Data Services
Backup
Server
Pyhsical Tape
Library
Data Services
CDP
VTLServerless
Backup
Virtual to Physical
© 2013 IBM Corporation
Understanding The Problem of Data Growth and Costs
Primary Data:
• 20 Terabytes of data
• 2% change in data, 3% growth of data
• Five week retention
• Weekly backup of all data
• Daily incremental backup of new data
• Total = Operations managing 110 Terabytes of data on tape
100TB Example:
• 100TB means operations manages 550 terabytes of data on
tape! (5 weeks x 110TB)
• Tape restoration will require weekly full and daily tape restores
until day of failure
• Data created post backup will have to be recreated
© 2013 IBM Corporation
Week 1
Sample parameters:Data volume = 20TB; 2% growth, 3% change weekly
Onsite Retention = 5 weeks
Week 5Week 4Week 3Week 2
20TB
41TB
63TB
86TB
110+TB
Why dedupe is Important: Without Dedupe
© 2013 IBM Corporation
Week 1 Week 5Week 4Week 3Week 2
20TB10TB 12TB 14TB 15.2TB
Sample parameters:Data volume = 20TB; 2% growth, 3% change weekly
Onsite Retention = 5 weeks
Total data stored = 15.2TB
Redundant data NOT stored: 94.8TB
6.6TB
…With Dedupe
Perspective:
2:1 =50%
5:1 =80%
10:1 =90%
20:1 =95%
30:1 =97%
© 2013 IBM Corporation
Current Costs
Assuming LTO3 drives and 20 terabytes of production
data:
• 20TB x 5 week retention = 110TB/400GB (capacity of LTO3) = 282
tapes
• 282 x $70 per tape media = $19,740
• 80MB/s (speed of LTO3) = 6.91TB per day
• To backup 20TB in a 12-hour window, you need 6 drives
• 6 drives = $3,214 x 6 = $22,500
• $22,500 + $19,740 = $42,240
Total = $42,240 for each 20TB of primary data.
Calculating the Benefits
© 2013 IBM Corporation
Calculating the Benefits of Dedupe
• Implement 2 x 80TB dedupe appliances at an approx. cost of $35,400
• $42,240 – $35,400 = $6,840 savings for every 20TB
Savings:
• Cost of offsite tape storage contract
• Cost of any array licenses and storage for replication
• No more media costs
• Minimal WAN costs (97% less WAN to replicate deduped data)
• Faster recovery and DR
• No more shipping, storing or recalling tape
Optimize by Adding Dedupe
Dedupe
Appliance
Dedupe
Appliance
© 2013 IBM Corporation
Low End Tape Versus Disk Based Backup
Cost are relative to the amount of data needing protection
© 2013 IBM Corporation
High End Tape versus Disk Based Backup
© 2013 IBM Corporation
DR: Fighting the “Insurance Policy” Mindset of Data Protection
Here is an example of lost-revenue per
hour per industry section based on
publicly available data. This is useful for
determining baseline outage costs.
More accurate data can be obtained
from an internal analysis.
Remember these are HOURLY costs.
Time to recover from tape?
Industry
Lost Revenue
per Hour (U.S.
Dollars)
Energy $2,800,000
Telecom $2,000,000
Manufacturing $1,600,000
Finance $1,500,000
Information
Technology $1,350,000
Insurance $1,200,000
Retail $1,100,000
Pharmaceutical $1,100,000
Chemical $700,000
Transport $670,000
Utilities $640,000
Healthcare $640,000
Media $340,000
© 2013 IBM Corporation
Recovery: Tape versus Continuous Protection
1. Locate & mount the Tape 2. Restore (transfer) data to the host 3. Process the data
Virtual LUN
Instant data recovery
over SAN without tape
or “restore”
Time-Consuming Restore
process via LAN (1Gb/s)
LAN
SAN (FC, iSCSI) No need to transfer data
Any server
Any serverCDP Target
3rd party tape
backup software
Instant Recovery Replaces Data Restore
© 2013 IBM Corporation
Cross Platform Consistent Recovery
Virtual LUN
Instant data recovery
over SAN without tape
or “restore”
SAN (FC, iSCSI) No need to transfer data
CDP TargetCDP
Instant Recovery Replaces Data Restore
VTL
IPL
Consistency Group
IP
© 2013 IBM Corporation
Dedupe together with CDP based recovery
Calculate the benefits of implementing CDP and IPL from VTL. Assuming the cost of downtime is
calculated using the numbers provided in my chart for a media company (340K per hour) and an
average current recovery time of only four hours, the calculations are as follows:
• $340,000 x 4 hours = $1,360,000 (current outage costs)
• Cost of new solution (2 sites at $100,000 per location = $200,000)
• Recovery time for NEW solution = 30 minutes ($340,000/2 = $170,000)
• $1,360,000 - $200,000 -$170,000 = $990,000
Total $990,000 savings on first outage!
Calculating the Benefits
© 2013 IBM Corporation
Leverage Protection Storage for Production
Mount protection storage for recent snapshot copies for test/dev.
Use snapshots as source for moving data to archives
Data Warehouse
Test Dev
Hadoop
Big Data
Production Pool
Protection Pool
CDP, VTL and Snapshots
Data Services
Data Services
© 2013 IBM Corporation
Leverage Protection Pool for Test and Development
Production Storage Costs: $2000 TB
Protection storage costs: $600 TB
Prospective Savings: $1400 per TB for Test and Development
Calculating the Benefits
© 2013 IBM Corporation
Virtualize servers and storage
Implementing virtualization can have a huge impact in multiple areas:
• Server virtualization commoditizes servers and enables
server consolidation and mobility
• Storage virtualization commoditizes storage and enables
complete data mobility and site resiliency
• Virtual tape for iSeries backup and DR enables rapid cost
efficient protection for mission critical apps.
Virtualize Everything
© 2013 IBM Corporation
Summary: Calculating the Benefits
• The ability to move data more efficiently via virtualization can
reduce storage costs by 50 percent or more.
• Consolidating applications onto virtual servers can reduce
infrastructure costs between 30-60 percent.
• Dense virtual storage and servers reduces data center
power, cooling, and floor space requirements. (For example,
a single blade server may be able to run 50 applications
versus 50 physical servers). Similar to an LPAR!
• Virtual disk and tape storage with dedupe lowers costs,
simplifies provisioning, and speeds recovery.
© 2013 IBM Corporation
Primary
e-hub
DR
e-hub
r-hub r-hub
r-hub r-hub
Small
Office
Small
Office
Small
Office
Small
Office
Small
Office
Small
Office
Small
Office
Small
Office
Small
Office
Small
Office
T1 Bi-Directional Data Link
OC-3 Bi-Directional
Active/Active Data link
512k Small Office Data Link
Legend
Enterprise Wide Data Management
r-hub
Centralize and Automate
Consolidated
Management
Console
Optimized Continuous Deduped Replication
© 2013 IBM Corporation
Summary: Optimized Data Services with Public / Private Cloud
Optimized
Single
Instance of
Data
Edge
Core
Optimized Data Services
Simplify Operations
Automate Processes
Virtualize Infrastructure
Enterprise Wide
DR
© 2013 IBM Corporation
Optimized Private / Hybrid Cloud Example
Production Pool
Data Services
WAN
Physical Physical Virtual
Compute Resources
97% Less
WAN
VirtualPhysical Virtual Virtual
Data Services
Compute Resources
MM
Protection Pool
Public or Private
Cloud
© 2013 IBM Corporation
Fast System Restore via IPL
Virtual tapes function as a data source for system IPL (boot)
Fast recovery of systems either locally or at remote site (use for DR, test, lab, etc.)
IBM i
IPL IPL
Local Recovery Remote Recovery
Virtual Tape
image
© 2013 IBM Corporation
iSeries Deployment Example
Insurance company, Western Europe
LPAR
LPAR
LPAR
LPAR
LPAR
LPAR
LPAR
LPAR
LPAR
LPAR
LPAR
VLIB
VLIB
VLIB
VLIB
VLIB
VLIB
VLIB
VLIB
VLIB
VLIB
VLIB
VLIB
VLIB
TSM Version
6.3
Windows, Unix
and Linux
Servers
Each
LPAR has
a
dedicated
virtual
library
VLIB
iSeries with V7R1,
V6, and V5
Replication Site
160 Km
STK 9310
Tape Library
Non-
replicated
data copied to
tape
Specific libraries
set for replication
over 1 Gb/s link
TSM has
two
dedicated
virtual
libraries
• Combines TSM and IBM i backup to one VTL (one solution)
• Specific libraries set to replicate tapes to second site to
protect critical data (flexibility)
• Data maintained both on disk with deduplication and on physical tape
(choice)
Virtual LTO-x
© 2013 IBM Corporation
Automatic Tape Caching Simplifies Implementation
Moves backups to physical tape based on
policies
– Age, time, space, etc.
– Virtual tape can be kept for a period
after copy to tape
Backup
Restore
Backup
Restore
VTL to Tape
Direct restore from
tape
Policy drivenBackup driven
What the IBM i server
sees
What is actually
happening
Tape
Library
IBM i
server
• Barcodes
maintained
• Invisible to backup
software
• One backup job for
disk and tape
IBM i
server
VTL
© 2013 IBM Corporation
Benefits of Deduped VTL vs. Internal i5/OS Virtual Tapes
Limited operational
integration with tape
Consumes costly
iSeries specific
storage
Need to manage virtual
tapes in relation to
system disk
consumption and
integrated file system
No impact on system
disk consumption, no
management overhead
Storage
independent of
iSeries storage
Direct operational
integration with tape
Virtual Tape LibraryI5/OS Virtual Tapes
© 2013 IBM Corporation
Simple Implementation
The FalconStor VTL exactly emulates existing physical tape infrastructure
for IBM i
–Virtual IBM 3580, 3590, 3592, TS1120 tape drives
–Virtual LTO-1, LTO-2, LTO-3, LTO-4, and LTO-5 tape drives
–Virtual IBM 3583, 3584, and 3590 libraries
All existing backup tools and methods can continue to be used in the
same manner
–BRMS, LXI MMS, SAVLIB, Attempo, media policies, etc.
Daily operational impact is near zero
© 2013 IBM Corporation
Summary: Private/Hybrid Cloud Value Proposition
Simplify datacenter infrastructure via reference building blocks
Provision storage anywhere in a few minutes
95% better recovery time (RTO)
99.999% improvement in recovery point (RPO)
Save 80% or more on disk space
80% savings on bandwidth required for DR
Use modular storage tiers to reduce CapEx by over 50%
Eliminate backup windows
Open systems protection is server-less, and LAN free
Eliminate most array-based licenses (except for RAID)
Everything is stored and moved as efficiently as possible
Simplify operations
© 2013 IBM Corporation
Technical References
http://www.falconstor.com
FalconStor VTL
Cloud Commission Full Report Text Available online at:
www.techamericafoundation.org/cloud2
http://www.computerworld.com/author/chris-poelker/
ComputerWorld Blog
Storage Area Networks for Dummies:
http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470385138.html
AS/400 Professionals
Linkedin Groups: