Windows Azure Blob Storage
-
Upload
ylew15 -
Category
Technology
-
view
2.686 -
download
3
description
Transcript of Windows Azure Blob Storage
WINDOWS AZURE BLOB STORAGE DEEP DIVE
Wely Lau ([email protected]) Microsoft MVP, Windows AzureSolutions Architect, NCS Pte LtdBlog : http://wely-lau.net
AGENDA
• Windows Azure Storage Overview (10 mins)
• Blob Storage Basic (15 mins)• Understanding container and blob• Main web service operations • Metadata• Snapshot
• Deep Dive (20 mins)• Block blob &page blob• Shared Access Signature• Content Delivery Network
• Tip of the day (10 mins)
• Q & A (5 mins)
WINDOWS AZURE STORAGE OVERVIEW
WINDOWS AZURE STORAGE
• Storage in the Cloud• Highly available with 99.9% monthly SLA• Anywhere at anytime access• Data Redundancy
• Data is replicated for resilience and protection with 3 copies within the same data center.
• Another 3 copies geo-replicated in another data center• Cost effective
• Storage capacity: $ 0.14 per GB per month• Storage transaction: $ 0.01 per 10,000 transaction
• User specified globally unique account name• Can choose geo-location to host storage account
• US – “North Central” and “South Central”• Europe – “North” and “West”• Asia – “East” and “Southeast”
• Can co-locate storage account with compute account• Explicitly or using affinity groups
• Some Facts• 70 Petabytes raw storage today• Grows to >200 Petabytes by start of 2012
WINDOWS AZURE STORAGE ABSTRACTION
Tables
Structured storage. A table is a set of entities; an entity is a set of properties.
Queues
Reliable storage and delivery of messages for an application.
Blobs
Simple named files along with metadata for the file.
Drives
Durable NTFS volumes for Windows Azure applications to use. Based on Blobs.
STORAGE EMULATOR
• Provides a local “Mock” storage• Emulates storage in cloud• Allows offline development• Requires SQL Express 2005/2008 or above
There are some differences between Cloud and Dev Storage.http://msdn.microsoft.com/dd320275
ACCESSING WINDOWS AZURE STORAGE VIA API
• REST-based Web Service• Platform independent• Use when a library isn’t available• Lacks Intellisense• Results in opaque code
• Storage Client Libraries• .NET, Java, Python, Ruby, PHP,
Perl, JavaScript (Node), Objective-C, etc.
• More at http://tinyurl.com/was-library
• These provide an abstraction layer on top of the REST API and is the most common way to access storage.
Windows Azure Storage
REST-based API
.NET Java PHPetc..
.
Developers…
ACCESSING WINDOWS AZURE STORAGE VIA TOOLS• Cloud Storage Studio by Cerebrata• Azure Storage Explorer by Neudistic• CloudXplorer by ClumsyLeaf• CloudBerry Explorer by CloudBerry Lab• More at http://tinyurl.com/was-explorers
WINDOWS AZURE STORAGE
Creating Windows Azure Storage AccountUsing tools to access Windows Azure Storage
demo
BLOB STORAGE BASIC
WINDOWS AZURE BLOB STORAGE
• You can have as many containers and BLOBs as you want.
• Containers can be created/destroyed on the fly
Account: MovieConversion
- Job1.mpg- Job2.mpg- Header.png
Container: Originals
- Job 9.mpg- Index.docx- Job8.pdf
Container: Completed
http://<account>.blob.core.windows.net/<container>/<blobname>http://cohowinery.blob.core.windows.net/images/pic01.jpg
blobs
CONTAINERS
• Similar to a top level folder• Special $root container
• Has an unlimited capacity• Can only contain BLOBs• Associate Metadata with Container
Each container has an access level:- Private
- Default, will require the account key to access
- Public read access for blobs only- Blob data within this container can be read via anonymous request, but container data
is not available
- Full public read- Container and blob data can be read via anonymous request
http://cohowinery.blob.core.windows.net/pic01.jpghttp://cohowinery.blob.core.windows.net/$root/pic01.jpg
BLOB
• Main Web Service Operations• PutBlob• GetBlob• DeleteBlob• CopyBlob• SnapshotBlob • LeaseBlob
• Associate Metadata with Blob• Metadata is <name, value> pairs, up to 8KB per blob
• Blob always accessed by name• Can include ‘/‘ or other delimeter in name • e.g. /<container>/myblobs/blob.jpg
BLOB STORAGE BASIC
• Main Web Service Operations • Metadata• Snapshot• Upload / Download Blob
demo
BLOB STORAGE DEEP DIVE
TWO TYPES OF BLOBS UNDER THE HOOD
Block Blob• Targeted at streaming
workloads
• Each blob consists of a sequence of blocks• Each block is identified by a Block ID
• Size limit 200GB per blob
• Optimistic Concurrency via Etags
Page Blob• Targeted at random read/write
workloads
• Each blob consists of an array of pages • Each page is identified by its offset
from the start of the blob
• Size limit 1TB per blob
• Optimistic or Pessimistic (locking) concurrency via leases
TheBlob.wmv
UPLOADING A BLOCK BLOB
• Uploading a large blob with Put Block List
10 GB Movie
Blo
ck I
d 1
Blo
ck I
d 2
Blo
ck I
d 3
Blo
ck I
d N
blobName = “TheBlob.wmv”;PutBlock(blobName, blockId1, block1Bits);PutBlock(blobName, blockId2, block2Bits);…………PutBlock(blobName, blockIdN, blockNBits);PutBlockList(blobName,
blockId1,…,blockIdN);
TheBlob.wmv
BenefitEfficient continuation and retryParallel and out of order upload of blocks
THE BLOB
Windows AzureStorage
PAGE BLOB – RANDOM READ/WRITE
Create MyBlobSpecify Blob Size = 10 GbytesSparse storage - Only charged for pages with data stored in them
Fixed Page Size = 512 bytesRandom Access Operations
PutPage[512, 2048)PutPage[0, 1024)ClearPage[512, 1536)PutPage[2048,2560)
GetPageRange[0, 4096) returns valid data ranges:
[0,512) , [1536,2560)GetBlob[1000, 2048) returns
All 0 for first 536 bytesNext 512 bytes are data stored in [1536,2048)
0
10 GB
512
1024
1536
2048
2560
10 G
B A
dd
ress S
pace
BLOCK BLOB / PAGE BLOB
demo
SHARED ACCESS SIGNATURE
• Fine grain access rights to blobs and containers
• Sign URL with storage key – permit elevated rights
• Revocation• Use short time periods and re-issue• Use container level policy that can be deleted
• Two broad approaches• Ad-hoc• Policy based
AD HOC SIGNATURES
• Create Short Dated Shared Access Signature• Signedresource Blob or Container• AccessPolicy Start, Expiry and Permissions• Signature HMAC-SHA256 of above fields
• Use case• Single use URLs• E.g. Provide URL to Silverlight client to upload to container
http://...blob.../pics/image.jpg?sr=c&st=2009-02-09T08:20Z&se=2009-02-10T08:30Z&sp=w
&sig= dD80ihBh5jfNpymO5Hg1IdiJIEvHcJpCMiCMnN%2fRnbI%3d
POLICY BASED SIGNATURES
• Create Container Level Policy• Specify StartTime, ExpiryTime, Permissions
• Create Shared Access Signature URL• Signedresource Blob or Container• Signedidentifier Optional pointer to container policy• Signature HMAC-SHA256 of above fields
• Use case• Providing revocable permissions to certain users/groups• To revoke: Delete or update container policy
http://...blob.../pics/image.jpg?sr=c&si=MyUploadPolicyForUserID12345
&sig=dD80ihBh5jfNpymO5Hg1IdiJIEvHcJpCMiCMnN%2fRnbI%3d
SHARED ACCESS SIGNATURE
demo
CONTENT DELIVERY NETWORK
• High-bandwidth global blob content delivery• 24 locations globally (US, Europe, Asia, Australia and South
America), and growing• Same experience for users no matter how far they are from
the geo-location where the storage account is hosted
• Blob service URL vs. CDN URL:• Windows Azure Blob URL:
http://images.blob.core.windows.net/
• Windows Azure CDN URL: http://<id>.vo.msecnd.net/
• Custom Domain Name for CDN: http://cdn.contoso.com/
CONTENT DELIVERY NETWORK
North Central US
South Central US
North Europe
West Europe
East Asia
South East Asia
CONTENT DELIVERY NETWORK
To Enable CDN:• Register for CDN
via Dev Portal• Set container
images to publicpic1.jp
gpic1.jpg
GEThttp://guid01.vo.msecnd.net/images/pic.1jpg
http://sally.blob.core.windows.net/images/pic1.jpg
http://sally.blob.core.windows.net/ http://guid01.vo.msecnd.net/
pic1.jpg
404
TTL Content Delivery Network
Windows Azure Blob Service
EdgeLocation
EdgeLocation
EdgeLocation
CONTENT DELIVERY NETWORK
demo
URL Time No CDN AVG
http://welyncus.blob.core.windows.net/mseinstall.exe
11:23 58 seconds50 seconds11:24 49 seconds
11:27 43 seconds
http://az145199.vo.msecnd.net/mseinstall.exe
11:28 40 seconds39 seconds11:29 37 seconds
11:30 40 seconds
http://welydemostorage.blob.core.windows.net/mseinstall.exe
11:32 38 seconds37.67 seconds11:32 34 seconds
11:33 41 seconds
http://az28746.vo.msecnd.net/mseinstall.exe
11:35 36 seconds31 seconds11:36 29 seconds
11:37 28 seconds
TIP OF THE DAY
LIVE PERFORMANCE COUNTER
• Checkout: http://tinyurl.com/wely-live-perf
• Hosting web content in Blob Storage
• Query Blob Storage from the browser
Idea courtesy to Steve Marx
THANK YOU!
Wely Lau ([email protected]) Microsoft MVP, Windows AzureSolutions Architect, NCS Pte LtdBlog : http://wely-lau.net