(BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS...
-
Upload
amazon-web-services -
Category
Technology
-
view
259 -
download
2
Transcript of (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS...
![Page 1: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/1.jpg)
![Page 2: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/2.jpg)
![Page 3: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/3.jpg)
![Page 4: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/4.jpg)
Archive:
Data retained for the
long term – for
compliance or potential
future reference
Backup:
Data retained to
support near-term
business continuity
![Page 5: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/5.jpg)
Archive:
Data retained for the
long term – for
compliance or potential
future reference
![Page 6: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/6.jpg)
Archive:
Data retained for the
long term – for
compliance or potential
future reference
![Page 7: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/7.jpg)
What is archival?: Traditional archival approaches
![Page 8: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/8.jpg)
![Page 9: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/9.jpg)
Metered usage:
Pay as you go
No capital investment
No commitment
No risky capacity planning
Avoid opex and risks of
physical media
handling
Control your
geographic locality for
performance and
compliance
![Page 10: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/10.jpg)
![Page 11: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/11.jpg)
Amazon
Glacier
Application
HTTP(S) /
REST APIs
Data collection Local index
Compression
Raw data
![Page 12: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/12.jpg)
Amazon
Glacier
Lifecycle policies
Amazon
S3
HTTP(S) /
REST APIs
Raw data
![Page 13: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/13.jpg)
![Page 14: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/14.jpg)
![Page 15: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/15.jpg)
![Page 16: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/16.jpg)
Create vault (films)
1
Configure access policies
2
ArchiveApp user policy
Effect:Allow
Resource:
arn:aws:glacier:<accountId>:vaults/Films
Action: glacier:UploadArchive
3
Upload archivesUploadArchive(data) ->
Archive ID
![Page 17: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/17.jpg)
Initiate job
ArchiveId: AE99F…
Vault: Films -> Job ID
1
3-5 hours for job completion
2
3
Job completion
notification
4
Download output
![Page 18: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/18.jpg)
![Page 19: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/19.jpg)
1. Primary storage hardware (primary / remote site)
2. DR / remote site storage hardware
3. Raw to utilized storage (both primary and DR)
4. Storage growth (cost of upgrades)
5. Storage management software and 3rd party tools
6. Professional services
7. Hardware maintenance
8. Software maintenance
9. Backup software
10.Backup hardware (primary / remote site)
11.Offsite tape storage / vault
12.Archive software
13.Archive hardware
14.Power
15.Cooling
16.Space
17.Labor
18.Cost of capital
19.Training
20.Asset depreciation
21.Migration
22.Decommission / remove
23.Recycle
![Page 20: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/20.jpg)
![Page 21: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/21.jpg)
![Page 22: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/22.jpg)
Checksum 2
Checksum 1
File 2
Checksum 3
. . .
Local index
File 1 offset
File 1
File 2 offset
File 3 offset
Index/directory
…
Checksum & metadata
Checksum & metadata
Checksum & metadata
Archive
![Page 23: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/23.jpg)
AWS Direct
ConnectDedicated bandwidth between
your site and AWS
InternetTransfer data in a secure SSL tunnel over
the public Internet
AWS Import/ExportPhysical transfer of media into and
out of AWS
![Page 24: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/24.jpg)
Arc
hiv
e
Parallel Uploads
Parts
![Page 25: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/25.jpg)
![Page 26: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/26.jpg)
Media Cloud ServicesOn-demand cloud-based solutions designed to capture,
collect, create, manage, share, produce, and archive high-
value, high-resolution content.
What is Sony MCS?
![Page 27: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/27.jpg)
Your professional, all-in-one media production
workspace.Instantly connect your team to your media files, hardware,
and applications for streamlined communication, faster
turnaround times, and reduced production costs.
Ci
![Page 28: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/28.jpg)
TELEVISION EDITORIAL & LEGAL REVIEW
MAJOR MOTION PICTURE DAILIES PREVIEWING
SMALL BUDGET PRODUCTIONS & ORIGINAL CONTENT
SHARE CONTENT BETWEEN STATIONS
ARCHIVECONTENT
STOCK FOOTAGE STORAGE & OPERATIONS
THOUSANDS OF ASSETS
DISTRIBUTED USERS
TRACKING/ MONITORING
![Page 29: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/29.jpg)
![Page 30: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/30.jpg)
Same challengeCREATIVE DRIVERS BUSINESS DRIVERS
Better Image QualityHigh dynamic range
High resolution
More Raw FootageHigher shooting ratios
High frame rate
3D / 4K
More ContentGreater diversity
More collateral marketing
Greater Distribution NeedsMulti-screen expectation
Physical, satellite, Internet
![Page 31: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/31.jpg)
![Page 32: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/32.jpg)
![Page 33: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/33.jpg)
0
2
4
6
8
10
12
14
FY13 FY14 FY15 FY16 FY17 FY18
Storage (PB)
Storage (PB)
Growth projections
![Page 34: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/34.jpg)
Cost comparison
$-
$1,000,000.00
$2,000,000.00
$3,000,000.00
$4,000,000.00
$5,000,000.00
$6,000,000.00
$7,000,000.00
FY13 FY14 FY15 FY16 FY17 FY18
On Prem
Ci/AWS
![Page 35: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/35.jpg)
Transfer to
S3
Copy to
archive
bucket
Ingest
pipeline
Register in
Ci
Worker
updates Ci
status
![Page 36: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/36.jpg)
Custom
app calls
Ci API
Ci calls S3
API
Worker
monitors
Ci status
updated
Update Ci
status
![Page 37: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/37.jpg)
![Page 38: (BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Amazon Glacier | AWS re:Invent 2014](https://reader035.fdocuments.net/reader035/viewer/2022081404/55a0f7271a28abda4f8b4864/html5/thumbnails/38.jpg)
Please give us your feedback on this session.
Complete session evaluations and earn re:Invent swag.
http://bit.ly/awsevals