Hitachi data ingestor
description
Transcript of Hitachi data ingestor
1 © Hitachi Data Systems Corporation 2012. All Rights Reserved.1
HITACHI DATA INGESTOR
BOTTOMLESS, BACKUP-FREE CLOUD ON-RAMP DELIVERS IT SERVICES TO THE EDGE
PRESENTER NAMEDATE
2
Market drivers and customer challenges
Hitachi Data Ingestor solution overview
Use cases
Summary and benefits
AGENDA
3
Management Lower costs
Simplified infrastructure
Faster time to market
Minimized risk
Simplified IT delivery models
Users Access all the time
On-demand capacity
Ability to restore and recover
Fast response
INFORMATION TECHNOLOGY DEMANDS
4
Disparate islands of information and infrastructure Unruly OPEX and CAPEX costs for management and backup Lack of content control
CUSTOMER CHALLENGES IN A DISTRIBUTED WORLD
5
Eliminate backups at remote locations Reduce capital costs and management overhead Optimize utilization, scale and operations
SIMPLIFY, CONSOLIDATE AND REDUCE COSTS
6
AGENDA
Market drivers and customer challenges
Hitachi Data Ingestor solution overview
Use cases
Summary and benefits
7
Integrated edge-to-core solution: Greatly reduces the cost and complexity of providing IT services to distributed consumers
- Remote or branch offices
- Cloud storage consumers Hitachi Data Ingestor:
Minimal- footprint or virtual appliance that sends data from the edge to a core infrastructure
- Bottomless and backup-free
- Caching filer Hitachi Content Platform:
Massively scalable, multitenant distributed object store
- Delivers advanced storage and data management capabilities
EDGE-TO-CORE: HITACHI DATA INGESTOR AND HITACHI CONTENT PLATFORM
Branch 1
Branch 2Data center
Edge
Core
Data
Data
Data
Branch 3
Hitachi Data Ingestor
Hitachi Content Platform
8
A bottomless and backup-free filer- Native read and write file system access (CIFS and NFS) to content in Hitachi Content Platform - Migrates data to Hitachi Content Platform and maintains a local link to the content- Fully integrated with AD and LDAP - Tightly integrated with Hitachi Content Platform to provide seamless access and a wide range of advanced
storage features- Provides a management API for tight integration with Hitachi Content Platform and 3rd party management UIs- Support for leading WAN acceleration solutions
Hitachi Data Ingestor
Hitachi Content Platform
FROM THE EDGE: HITACHI DATA INGESTOR
Features All content migrated and backed up in
Hitachi Content Platform Advanced cache management supports
400 million files Supports hundreds of users per node Transparent NAS migration for existing
filers and servers File restore for user self-service Content sharing in a distributed
environment
9
A number of discrete use cases from a single platform with rich functionality, robust data
protection and a single point of management
Features Up to 40PB of usable capacity in
a single physical cluster- Multitenant- Intelligent objects- Chargeback- Compression and
deduplication Security and compliance
- Encryption of data at rest- Authentication- WORM- Compliance and retention
Built-in protection, preservation and replication- High-availability architecture- Continuous data integrity
checking- Advanced replication and
disaster recovery
TO THE CORE: HITACHI CONTENT PLATFORM
Hitachi Content Platform
Ultimate investment protection
Active archivecontent depot
Data protectionbackup reduction
Private cloud Public cloud
10
Hitachi Data Ingestor (HDI) offers 3 configuration options HDI cluster HDI single node HDI VMware appliance
HITACHI DATA INGESTOR CHOICES
Highly available cluster pair SAN-attached to Hitachi storage
• Supports AMS, VSP, USP V, USP VM
Deploy on VMware vSphere Hypervisor
Customer-defined hardware and storage configuration
Available in HA architecture
1. HDI cluster 2. HDI single node 3. HDI VMware appliance
Non-redundant configuration Internal direct attached
storage Deploy in rack or by itself
11
HDI HCP
Application
REST/HTTP(S)
WRITE REPLICATE
READ RECALL
CIFS/NFS
Application writes a file to Hitachi Data Ingestor (HDI) Hitachi Data Ingestor replicates the file to Hitachi Content Platform (HCP) Users can “pin” files of their choices to make them stay on HDI all the time When system capacity reaches 90%, HDI deletes “non-pinned” files in excess of the
threshold and creates 4KB link stubs to replace them- Users access the files as they always have, since links are transparent to clients
Reading a link recalls the file back into HDI- Recalled files are deleted from HDI later and replaced by another link, based on HDI
system capacity and least-recently-used criteria
HOW HITACHI DATA INGESTOR MIGRATES FILES
12
Features Intelligent cache management
- Identifies and retains newer and more frequently accessed files
- Cache tuned to edge deployments with WAN connectivity
Advanced capacity management and high performance
Benefits Faster access to frequently used files
– active files stay on HDI all the time Enhanced replication performance Maximum capacity utilization Free up bandwidth over the WAN
HITACHI DATA INGESTOR: INTELLIGENT CACHING
CAPABILITY METHOD
Stubbing frequency
Files are stubbed every hour
LRU algorithm Oldest files are purged first, based on atime and mtime attributes; newer files stay in cache longer
Read cache Recalled files stay in cache until the next capacity- management process
13
Benefits Satisfy multiple applications, varying SLAs and workload types or organizations Determine utilization and chargeback per customer Edge dispersion: each HDI can access another when set up that way Enable advanced features at one branch or at more granular level
• Examples: replication, encryption, DPL levels (how many copies to keep), compliance and retention, compression and versioning
Tenant CTenant ANamespace 1
Hitachi Content Platform
HDIFS 2FS 1
Branch A
Client Client
Namespace 2
Tenant BNamespace 1
HDIFS 2FS 1
Branch B
Client Client
Namespace 2 Namespace 1
HDIFS 2FS 1
Branch C
Client Client
Namespace 2
Clients write to assigned file systems Each namespace can be shared by multiple HDIs
HOW HITACHI DATA INGESTOR MAPS TO THE CORE
Read-only
Content sharing enables edge dispersion Each file system is mapped to its designated namespace
14
COMPLIANCE AND GOVERNANCE CAPABILITIES
Flexible retention support and WORM functionality using an industry-standard atime-based interface and automated settings
Retention is an optional setting Allows you to specify a file’s retention date; the file is then protected
by strict WORM restrictions- While under retention, WORM files cannot be deleted- WORM files are immutable, even after retention expires- WORM files cannot be renamed- Directories containing WORM files cannot be renamed
Retention settings are transferred to Hitachi Content Platform as files are migrated
Retention can be tunable per file system Support for backup and restore of retention properties
15
CONTENT SHARING
A Hitachi Data Ingestor system can map to a Hitachi Content Platform (HCP) tenant to write content and potentially read content from other HCP tenants
Sharing supports a single writer with many readers
Read
SITE A SITE B
file system A file system A file system B
tenant A tenant B
namespace A namespace B
WANread/write access
read-only access
SITE C
file system A file system C
tenant C
Namespace C
16
CONTENT SHARING USE CASE: MEDICAL IMAGE FILE SHARING
Hospital A stores medical files to file system A of Hitachi Data Ingestor A
The files are migrated to namespace A on Hitachi Content Platform (HCP)
Hospital B can read the files through file system A, which is mapped to namespace A on Hitachi Content Platform
Read
HOSPITAL A HOSPITAL B
File system A File system A File system B
Tenant A Tenant B
Namespace A Namespace B
WANRead and write access
Read-only access
17
CONTENT SHARING USE CASE: RETAIL CATALOG UPDATES
Headquarters stores updated pricing catalog files to file system A of Hitachi Data Ingestor (HDI) A
The files are migrated to namespace A on Hitachi Content Platform Stores HDI B-n read the updated catalog and implement new pricing
policies
Headquarters
STORE B
File system A
File system A
Tenant A
Namespace A
WAN
Read and write access
Read-only access
STORE C
File system A
STORE n
File system A
18
FILE RESTORE
READ ONLY
…
Hitachi Data Ingestor
Hitachi Content Platform
Dir0current: 20110317v1: 20110215
File1current: 20110317v2: 20110301v1: 20110215
・・・
Hitachi Data Ingestor (HDI) creates a snapshot directory for each migration Snapshot are created under a history directory User can access snapshot files for specified migrations under “.history/<date+time>/<original path>”
and can restore a snapshot file to the current working directory
FS0 (share)
Dir0 .history
20110317 20110301
DIr0 at 20110301
Dir0 at 20110317
File1at
20110301
File1at
20110317
file1
file2
file3
19
NAS MIGRATION
HDI
CIFS Client
Migration source NAS
HCP
Migration source NAS
Migration source NAS
Hitachi Data Ingestor (HDI) migrates data from a customer’s production NAS to HDI file system and then to Hitachi Content Platform (HCP)
NAS migration is an automated background task, designed to be transparent to users of NAS system being migrated
Supported platforms are NetApp and Windows storage servers
CIFS only
Both execute at the same timeBoth execute at the same time
20
Active
DATA PROTECTION AND DISASTER RECOVERY
TenantNamespace 1 Namespace 2 HCP 1
HDI Cluster
Passive
TenantNamespace 1 Namespace 2 HCP 2
Features Both Hitachi Data Ingestor
(HDI) and Hitachi Content Platform (HCP) have high-availability architectures
- No single point of failure- Active data protection, data-
integrity monitoring and self healing
Disaster Recovery- Both have replication and
failover capabilities- HDI can rebuild its entire view
from a local or remote HCP
Benefits Backups no longer required Data protected on multiple
levels
Active
Replication
Failover
Failo
ver
Aut
hent
icat
e
21
AGENDA
Market drivers and customer challenges
Hitachi Data Ingestor solution overview
Use cases
Summary and benefits
22
BOTTOMLESS AND BACKUP-FREE EDGE-TO-CORE SOLUTION
For customers who Run distributed IT environments and
are looking for help in managing them Have remote office, branch office
storage with low-end Windows/NFS servers
Have legacy file systems that could benefit from cloud storage
Are looking to reduce their storage footprint and administrative overhead at the edge
Features Bottomless and backup-free storage Local cache for fast retrieval of
frequently accessed content Standard on-ramp connecting into a
Hitachi Content Platform-based cloud Supports multiple tenants and
namespaces AD and LDAP support
Branch 1 Branch 3
Branch 2
Tenant 1 Tenant 2 Tenant 3
Hitachi Content Platform
Data center
Hitachi Data Ingestor Hitachi Data Ingestor
Hitachi Data Ingestor
23
CLOUD ON-RAMP
For customers who are Cloud service providers or telcos
looking to connect customers to their cloud via a standards-based on-ramp
New customers looking to deliver more efficient and cost-effective storage solutions to branch or distributed users or as a private cloud
Customers looking to burst to the cloud for on-demand storage consumption
Features Seamless and integrated connection
from users to cloud Fast-time-to-value, low-touch on-ramp
into the cloud Chargeback customers based on
consumption and utilization Secure partitioning of customer data
Hitachi Data Ingestor
Acme Co.
Widget Inc.ABC Ltd.
ABC Ltd. Acme Co. Widget Inc.Hitachi Content Platform
Cloud service provider
Hitachi Data Ingestor Hitachi Data Ingestor
24
FILE SYSTEM GATEWAY TO HITACHI CONTENT PLATFORM
For customers who Require faster performance from Hitachi Content
Platform (HCP) than native CIFS and NFS gateways deliver
Require HCP namespace support Want to support AD for end users to access HCP Have applications that need to modify data after
it is written Are not planning to modify in-house applications
to integrate to HCP REST APIs immediately
Features Provides read and write access to HCP
for applications using CIFS/NFS Supports full HCP features set
- Multitenancy and namespace
- Compliance and retention
- Data protection and disaster recovery
Provides native CIFS/NFS access speed
Data
Hitachi Data Ingestor
Hitachi Content Platform
Data Center
25
AGENDA
Market drivers and customer challenges
Hitachi Data Ingestor solution overview
Use cases
Summary and benefits
26
Reduces costs Eliminates backups with a bottomless edge device Improves efficiency and utilization
Simplifies IT Reduces islands of storage and infrastructure Standard connection into Hitachi Content Platform (HCP)
Reduces risk Supports compliance and governance capabilities Security with AD and LDAP
Streamlines cloud Multitenant, multiple namespaces Seamless connection into a central-core HCP
HITACHI DATA INGESTOR SOLUTION BENEFITS
2727
QUESTIONS AND DISCUSSION
2828
THANK YOU