AWS Services for Data Migration · 2018-05-25 · Glacier and AWS Tape Gateway • Introduced...

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Luke Anderson

Head of Storage, AWS APAC

AWS Services for Data

Migration


Data movement

On

lin

eO

ffli

ne

Data security

and management

Complete Set Of Data Building Blocks

Amazon

EFS

Amazon

EBS

AWS Snow family

AWS Storage Gateway

Family

AWS Direct Connect

Amazon EFS File Sync

Amazon S3

Transfer Acceleration

Storage Partners

Amazon Kinesis

Data Streams

Amazon Kinesis

Video Streams

Amazon

S3

Amazon

Glacier

AWS KMS

AWS IAM

AWS CloudWatch

AWS CloudTrail

AWS Cloud Formation

AWS Lambda

Amazon Macie

AWS QuickSight


Moving To The Cloud

Common drivers of moving data to the cloud:

Pain points of data migration:

While every cloud adoption journey is unique, all require a data migration step, which

can be hindered by blockers such as:

Cost

effectiveness

Workforce

productivity

Business

agility

Operational

resilience

Access to

innovative

services

Disruption Speed Costs Channels Cloud

compatibility


S3EC2

EFS EBS Aurora EMR

Amazon Glacier RDSIOT Redshift

Migrating Data: 5 Key Questions

1) What kind of data is it, and where is it

going?

4) How much data & time do you

have?

2) 1-time or continuous

movement?3) 1-way or bi-directional?

5) How about the WAN?

Files Block

VolumesDatabase

sIOT Streams


Direct Connect


AWS Direct Connect

Reduce

Bandwidth

Costs

Consistent

Network

Performance

Compatible with

all AWS Services

Private

Connectivity to

VPC

Elastic Simple

• Establish private connectivity between AWS and your data center

• Dedicated connection can be partitioned into multiple virtual interfaces

• Maintain network separation between public and private environments

BENEFITS


Snow*


AWS Snow* Family

AWS Snowball AWS Snowball Edge AWS Snowmobile

• Petabyte-scale data migration

• Rugged 8.5 G impact case

• Rain and dust resistant

• Data encryption end-to-end

• 80 TB capacity/10 Gb network

• Move 250 TB in 1 Week!

• Compute and storage for hybrid/edge

workloads

• 100 TB local storage

• Local compute equivalent to an Amazon

EC2 m4.4xlarge instance

• 10GBase-T, 10/25 Gb SFP28, and 40 Gb

QSFP+ copper, and optical networking

• Ruggedized and rack-mountable

• Exabyte-scale data migration

• Up to 200 PB capacity

• Data encryption end-to-end

• Dedicated security personnel

• GPS tracking, alarm monitoring, 24/7

surveillance, and optional additional

security


AWS Snow* Family

How Snowball moves data into and out of AWS

Create

a job

Connect the

Snowball

Copy data to

the Snowball

Your data move

to Amazon S3

In transit to you Delivered to you Delivered to AWS At AWS

Job created Job completed


EFS File Sync


Amazon EFS File SyncSync data from existing file systems into EFS file systems

FastUp to 5x faster than

standard Linux copy tools

SecureEncrypted parallel data

transfer to AWS

SimpleSet up and manage easily

from the AWS Console

File systems from on-premises to EFS

DIY in-cloud file systems to EFS

EFS file systems between AWS Regions

Use EFS File Sync to copy…


EFS File SyncFast and simple way to copy file systems to Amazon EFS

File Server

/NAS

NFS

Sync Agent

Customer Premises Region

file systemEFS file

system

Amazon

EFSHTTPS

• Sync Agent deployed as a VM

• No data is stored on the Sync Agent

• Optimized TCP stack for high-throughput

• Data is compressed and encrypted across the network


S3 Transfer Acceleration


S3 Bucket AWS Edge

Location

Uploader

Optimized

Throughput!

Change your endpoint, not your code

Leverages 59 global edge locations

Optimized protocols

No firewall exceptions

No client software required

Amazon S3 Transfer Acceleration


How fast is Amazon S3 Transfer Acceleration?

Rio De Janeiro Warsaw New York Atlanta Madrid Virginia Melbourne Paris Los Angeles Seattle Tokyo Singapore

Tim

e [h

rs]

500 GB upload from these edge locations to a bucket in Singapore

Public InternetAmazon S3 Transfer

Acceleration

On average, we have seen 171% improvement over

regular Amazon S3 when uploading over long distances




Case Study

Typical Friday during football season: More than 35 hours

of video every minute is uploaded

S3-TA = >20% increase in upload and encoding speeds

”“ Amazon S3 transfer acceleration reduces the average time it

takes for us to ingest videos from our global user base by

almost half. This gives our customers the ability to edit and

share videos sooner where speed is a critical factor.

- Brian Kaiser, CTO


AWS Storage Gateway Family


Storage Gateway Hybrid Storage SolutionsUse Standard Storage Protocols To Access AWS Storage Services

AWS Storage

Gateway

Amazon EBS

snapshots

Amazon S3

Amazon Glacier

AWS Identity and Access

Management (IAM)AWS Key Management

Service (KMS)

AWS

CloudTrail

Amazon

CloudWatch

Files

Volumes

TapesO

n-p

rem

ises A

WS

Clo

ud


File GatewayOn-premises file storage maintained as objects in Amazon S3

Customer Premises

• Data stored and retrieved from your Amazon S3 buckets

• One-to-one mapping from files to objects

• File metadata stored in object metadata

• Bucket access managed by IAM role you own and manage

• Use Amazon S3 lifecycle policies, versioning, or CRR to manage data

Amazon

Glacier

S3 Standard S3 Standard-

Infrequent

Access

HTTPSNFS

v3 / v4.1

Application

Server

Storage

GatewayStorage

Gateway VM


GlacierS3 Standard S3-

Infrequent

Access

File GatewayPetroBank

Application

ServersLTO

NAS

Active Archive Migration From LTO Cost-effective storage in AWS with local data access

AWS Direct

Connect

Self service loading of data

Reduced time-to-data by days or weeks

Storage archive costs reduced by 90%

Halliburton Data Center

AWS Lambda


Tape GatewayVirtual tape storage in Amazon S3 and Amazon Glacier with VTL management

• Virtual tape storage in S3 and Glacier accessed via tape gateway

• Data compressed in-transit and at-rest

• Up to 1 PB total tape storage per gateway, unlimited archive capacity

• Supports leading backup applications

• **3-5 hour tape retrieval from Amazon Glacier

Media ChangerVirtual Tapes

stored in

Amazon S3

Archived Tapes stored in

Amazon Glacier

Backup

Server

iSCSI

Storage

Gateway

Service

Tape Gateway VM

Tape Drive

HTTPS



Move Your Tape Archive

Backup

Server

Customer Premises

Tape

Library

Offsite Vault

Backup

Disk

Appliance


Move Your Tape Archive

Media ChangerVirtual Tapes

stored in

Amazon S3

Archived Tapes

stored in

Amazon Glacier

iSCSI

Storage

Gateway

Service

Tape Gateway VM

Tape Drive

HTTPS

RegionBackup

Server

Customer Premises

Tape

Library

Offsite Vault

• Deploy Storage Gateway VTL as an archive target

• Old archives can expire, or be recalled and written to SGW

• Decommission Tape Library when all tapes are expired

Backup

Disk

Appliance


Tape Gateway: Case Study

• 16,000 users and 3.5 staff in the Cascadia earthquake zone

• ~500 man hours/year handlingtape and repairing drives

• Seamlessly replaced 2 libraries, hundreds of tapes, and offsite fees with Amazon Glacier and AWS Tape Gateway

• Introduced disaster recovery from AWS region to AWS region or to local datacenter

• Zero impact on existing data protection processes, backup jobs, or catalog

“Using AWS Storage Gateway,

Southern Oregon University switched from

physical to virtual tape backup simply by

dropping the gateway’s virtual appliance into

our existing Veeam workflow. Setting it all up

took three hours, at most.

We can now provision virtual tapes on AWS

with the click of a button.”

- Jesse Martinich

Network service manager


Volume Gateway

• Presents block storage on top of Amazon S3 accessed via the volume gateway

• Compression of data in transit and at rest

• Back up on-premises volumes to EBS snapshots

• Create on-premises volumes from EBS snapshots

• Up to 1 PB of total volume storage per gateway

Application

ServerAmazon EBS

snapshotsStorage Gateway

ServiceVolume Gateway

VM

Volume Stored in

Amazon S3

HTTPSiSCSI


On-premises volume storage backed by Amazon S3 with EBS snapshots


Volume Stored

volume

Amazon EBS

snapshots

Customer Premises

iSCSI

Storage Gateway VM

Region

stored

volumeApp

Server

complete volume

stored in the cloud

and on-premises

Amazon

CloudWatch

Storage Gateway

Service

Supporting

storage can be

SAN, VMDK, VHD

cache

upload

buffer


Migration with Storage Gateway Volumes


App

Server

Existing

Volume

Existing Volume

SAN, VMDK, VHD based volume



volume


App

Server

Storage

Gateway

Service

stored

volume

iSCSI

Initial Sync

HTTPS

Stored Volume Mode

Storage Gateway VM

Attach volume to GW

Disconnect volume from app

server and connect it to GW

(data in place)

Accessible

over iSCSI



volume Amazon EBS

snapshots


App

Server

Storage

Gateway

Service

stored

volume

iSCSI HTTPS

Volume Stored Mode

Storage Gateway VM

Create EBS snapshot



volume Amazon EBS

snapshots


App

Server

stored

volume

iSCSI HTTPS

Volume Stored Mode

Storage Gateway VM

EC2 instance

Windows

Server

DATAStorage

Gateway

Service

Attach EBS

Snapshot to EC2

instance


Migration With Storage Gateway Volumes

Region

EC2 instance

Windows

Server

DATA

Customer Premises


Canada’s largest biotech firm

Data sovereignty required local hot files

and tape archives in each of 10 global offices

• AWS Volume Gateway eliminated 50-hour

backup windows and tape archive systems

• Cut on-premises storage CAPEX 40%;

dropped RTO from 48 hours to 10 minutes

• Meets cloud strategy while retaining local

ownership and data sovereignty

• Enabled datacenter exit in next 12 months

“It made no sense to keep buying

big disk silos, especially as we opened

up new global offices, and now we can

recover in the cloud from a snapshot if

we ever had to.”

- Adam Leggett

IT manager

Backup And Restore


AWS Partner Network – Migration & Storage

The AWS Partner Network (APN) helps you seamlessly integrate AWS services into your existing storage environment


Storage Training

For Enterprise Storage Engineers

• Learn how to architect and manage

highly available solutions on AWS

storage services

• Advance toward AWS certifications

• Help your organization migrate to

the cloud faster

Online at www.aws.training

• Access 100+ new digital training

courses including advanced

training on storage

• Deep Dives on S3, EFS, and EBS

• Migrating and Tiering Storage to

AWS (Hybrid Solutions)


Want to Learn More?

Learn more with Classroom Based Training

Take the two-day, classroom-based training

course covering the entire migration process:

• Application portfolio discovery

• Migration planning and design

• Application migration

• Post-migration validation

and application optimization

IT Project Managers Solutions Architects

Technical Essentials Architecting on AWS

Migrating to AWS

Free AWS digital training

Visit https://www.aws.training/

Get Started with Free Digital Training

Access free digital training to learn about AWS

services and solutions for migrating to AWS.

https://www.aws.training/


We hope you found it interesting! A kind reminder to complete the survey.

Let us know what you thought of today’s event and how we can improve the event

experience for you in the future.

Thank You For Attending

AWS Migrating to the Cloud Webinar Series.

[email protected]

twitter.com/AWSCloud

facebook.com/AmazonWebServices

youtube.com/user/AmazonWebServices

slideshare.net/AmazonWebServices

twitch.tv/aws

AWS Services for Data Migration · 2018-05-25 · Glacier and AWS Tape Gateway • Introduced...

Documents

Transcript of AWS Services for Data Migration · 2018-05-25 · Glacier and AWS Tape Gateway • Introduced...