Windows Azure Storage Anton Boyko.NET developer.
-
Upload
bertina-richards -
Category
Documents
-
view
233 -
download
0
Transcript of Windows Azure Storage Anton Boyko.NET developer.
![Page 1: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/1.jpg)
Windows Azure Storage
Anton Boyko.NET developer
![Page 2: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/2.jpg)
Agenda
Windows Azure Storage
Blob Storage
Drives
Tables
![Page 3: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/3.jpg)
Windows Azure Storage
Storage in the CloudScalable, durable, and availableAnywhere at anytime accessOnly pay for what the service uses
Exposed via RESTful Web ServicesUse from Windows Azure ComputeUse from anywhere on the internet
![Page 4: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/4.jpg)
Windows Azure Storage AccountUser specified globally unique account name
North Central USNorthern Europe
Western Europe East Asia
South East Asia
US Europe Asia
Can choose geo-location to host storage account:
South Central US
West US East US
![Page 5: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/5.jpg)
Windows Azure Storage AccountCan CDN Enable AccountBlobs delivered via 24 global CDN nodes
Can co-locate storage account with compute accountExplicitly or using affinity groups
Accounts have two independent 512 bit shared secret keys
100 TBs per account
![Page 6: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/6.jpg)
New Features
Geo-ReplicationStorage AnalyticsLogs: Provide trace of executed requests for your storage accountsMetrics: Provide summary of key capacity and request statistics for Blobs, Tables, and Queues
![Page 7: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/7.jpg)
Storage in the Development FabricProvides a local “Mock” storageEmulates storage in cloudAllows offline developmentRequires SQL Express 2005/2008 or above
http://msdn.microsoft.com/en-us/gg433135
![Page 8: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/8.jpg)
The Storage Client API
In this presentation we’ll cover the underlying
RESTful APICan call these from any HTTP cliente.g. Flash, Silverlight, etc…
Client API from SDK Microsoft.WindowsAzure.StorageClientProvides a strongly typed wrapper around REST services
![Page 9: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/9.jpg)
Storage Security
Windows Azure Storage provides simple security for calls to storage serviceHTTPS endpointDigitally sign requests for privileged operations
Two 512bit symmetric keys per storage accountCan be regenerated independently
More granular security via Shared Access Signatures
![Page 10: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/10.jpg)
Windows Azure Storage Abstractions
TablesStructured storage. A table is a set of entities; an entity is
a set of properties.
QueuesReliable storage and delivery of messages for an application.
BlobsSimple named files along with metadata for the file.
DrivesDurable NTFS volumes for Windows Azure applications to use. Based on Blobs.
![Page 11: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/11.jpg)
Blob Storage
![Page 12: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/12.jpg)
Blob Storage Concepts
BlobContainerAccount
http://<account>.blob.core.windows.net/<container>/<blobname>
Pages/ Blocks
contoso
PIC01.JPG
Block/Page
Block/Page
PIC02.JPG
images
VID1.AVIvideos
![Page 13: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/13.jpg)
Blob Details
Main Web Service
Operations
PutBlobGetBlobDeleteBlobCopyBlobSnapshotBlob LeaseBlob
![Page 14: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/14.jpg)
Blob Details
Associate Metadata with Blob
Standard HTTP metadata/headers (Cache-Control, Content-Encoding, Content-Type, etc)
Metadata is <name, value> pairs, up to 8KB per blob
Either as part of PutBlob or independently
![Page 15: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/15.jpg)
Blob Details
Blob always accessed by
name
Can include ‘/‘ or other delimeter in name e.g. /<container>/myblobs/blob.jpg
![Page 16: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/16.jpg)
Blob ContainersMultiple Containers per AccountSpecial $root container
Blob ContainerA container holds a set of blobsSet access policies at the container level Associate Metadata with ContainerList the blobs in a containerIncluding Blob Metadata and MD5 NO search/query. i.e. no WHERE MetadataValue = ?
Blobs ThroughputEffectively in Partition of 1Target of 60MB/s per Blob
![Page 17: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/17.jpg)
GET http://.../products?comp=list&prefix=Tents&delimiter=/
<Blob>Tents/PalaceTent.wmv</Blob><Blob>Tents/ShedTent.wmv</Blob>
Enumerating Blobs
GET Blob operation takes parametersPrefixDelimiterInclude= (snapshots, metadata etc…)
http://adventureworks.blob.core.windows.net/ Products/Bikes/SuperDuperCycle.jpg Products/Bikes/FastBike.jpg Products/Canoes/Whitewater.jpg Products/Canoes/Flatwater.jpg Products/Canoes/Hybrid.jpg Products/Tents/PalaceTent.jpg Products/Tents/ShedTent.jpg
![Page 18: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/18.jpg)
http://.../products?comp=list&prefix=Canoes&maxresults=2
&marker=MarkerValue
<Blob>Canoes/Hybrid.jpg</Blob>
Pagination
Large lists of Blobs can
be paginatedEither set maxresults or;Exceed default value for maxresults (5000)
http://.../products?comp=list&prefix=Canoes&maxresults=2
<Blob>Canoes/Whitewater.jpg</Blob><Blob>Canoes/Flatwater.jpg</Blob><NextMarker>MarkerValue</NextMarker>
![Page 19: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/19.jpg)
Two Types of Blobs Under the Hood
Block BlobTargeted at streaming workloads
Each blob consists of a sequence of blocksEach block is identified by a Block ID
Size limit 200GB per blob
Optimistic Concurrency via Etags
Page BlobTargeted at random read/write workloads
Each blob consists of an array of pages Each page is identified by its offset from the start of the blob
Size limit 1TB per blob
Optimistic or Pessimistic (locking) concurrency via leases
![Page 20: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/20.jpg)
TheBlob.wmv
Uploading a Block Blob
Uploading a large blob
10 GB Movie
Blo
ck I
d 1
Blo
ck I
d 2
Blo
ck I
d 3
Blo
ck I
d N
blobName = “TheBlob.wmv”;PutBlock(blobName, blockId1, block1Bits);PutBlock(blobName, blockId2, block2Bits);…………PutBlock(blobName, blockIdN, blockNBits);PutBlockList(blobName,
blockId1,…,blockIdN);
TheBlob.wmv
BenefitEfficient continuation and retryParallel and out of order upload of blocks
THE BLOB
Windows AzureStorage
![Page 21: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/21.jpg)
Page Blob – Random Read/Write
Create MyBlobSpecify Blob Size = 10 GbytesSparse storage - Only charged for pages with data stored in them
Fixed Page Size = 512 bytesRandom Access Operations
PutPage[512, 2048)PutPage[0, 1024)ClearPage[512, 1536)PutPage[2048,2560)
GetPageRange[0, 4096) returns valid data ranges:
[0,512) , [1536,2560)GetBlob[1000, 2048) returns
All 0 for first 536 bytesNext 512 bytes are data stored in [1536,2048)
0
10 GB
512
1024
1536
2048
2560
10 G
B A
dd
ress S
pace
![Page 22: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/22.jpg)
Shared Access Signatures
Fine grain access rights to blobs and containersSign URL with storage key – permit elevated rightsRevocationUse short time periods and re-issueUse container level policy that can be deleted
Two broad approachesAd-hocPolicy based
![Page 23: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/23.jpg)
Content Delivery Network (CDN)
High-bandwidth global blob content delivery24 locations globally (US, Europe, Asia, Australia and South America), and growing
Same experience for users no matter how far they are from the geo-location where the storage account is hosted
Blob service URL vs. CDN URL:Windows Azure Blob URL: http://images.blob.core.windows.net/
Windows Azure CDN URL: http://<id>.vo.msecnd.net/
Custom Domain Name for CDN: http://cdn.contoso.com/
![Page 24: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/24.jpg)
pic1.jpg
Windows Azure CDN
To Enable CDN:Register for CDN via Dev PortalSet container images to public
pic1.jpg
GEThttp://guid01.vo.msecnd.net/images/pic.1jpg
http://sally.blob.core.windows.net/images/pic1.jpg
http://sally.blob.core.windows.net/ http://guid01.vo.msecnd.net/
pic1.jpg
404
TTL Content Delivery Network
Windows Azure Blob Service
EdgeLocation
EdgeLocation
EdgeLocation
![Page 25: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/25.jpg)
Drives
![Page 26: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/26.jpg)
Windows Azure DrivesDurable NTFS volume for Windows Azure InstancesUse existing NTFS APIs to access a network attached durable driveUse System.IO from .NET
BenefitsMove existing apps using NTFS more easily to the cloudDurability and survival of data on instance recycle Drives can be up to 1TB
A Windows Azure Drive is an NTFS VHD Page BlobMounts Page Blob over the network as an NTFS driveLocal cache on instance for read operationsAll flushed and unbuffered writes to drive are made durable to the Page Blob
![Page 27: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/27.jpg)
Drive Details
Operations performed via Drive API not REST CallsOperations on DrivesCreateDriveCreates a new NTFS formatted VHD in Blob storage
MountDrive/UnmountDriveMounts a drive into Instance at new drive letter
Unmounts a drive freeing drive letter
Get Mounted DrivesList mounted drives; underlying blob and drive letter
Snapshot DriveCreate snapshot copy of the drive
![Page 28: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/28.jpg)
VM
How Windows Azure Drives Works Drive is a formatted page blob stored in blob
service
Mount obtains a blob lease
Mount specifies amount of local storage for cache
NTFS flushed/unbuffered writes commit to blob store before returning to app
NTFS reads can be served from local cache or from blob store (cache miss)
DemoBlob
OS
Application
Drive X:
Windows Azure Blob Service
Local Cache
![Page 29: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/29.jpg)
Failover with Drives
Must issue NTFS Flush command
to persist dataUse System.IO.Stream.Flush()
Read/Write Drives protected with leases1 Minute lease expiryMaintained by Windows Azure OS DriverUnmount on RoleEntryPoint.OnStop
On failureLease will timeout after 1 minuteRe-mount drive on new instance
![Page 30: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/30.jpg)
Tables
![Page 31: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/31.jpg)
Table Storage Concepts
EntityTableAccount
contoso
Name =…Email = …
Name =…EMailAdd=
customers
Photo ID =…Date =…
photos
Photo ID =…Date =…
![Page 32: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/32.jpg)
Table Details
InsertUpdate Merge – Partial update
Replace – Update entire entity
UpsertDeleteQueryEntity Group TransactionsMultiple CUD Operations in a single atomic transaction
Create, Query, DeleteTables can have metadata
Not an RDBMS! Table
Entities
![Page 33: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/33.jpg)
Entity PropertiesEntity can have up to 255 propertiesUp to 1MB per entity
Mandatory Properties for every entityPartitionKey & RowKey (only indexed properties)Uniquely identifies an entityDefines the sort order
Timestamp Optimistic ConcurrencyExposed as an HTTP Etag
No fixed schema for other propertiesEach property is stored as a <name, typed value> pairNo schema stored for a tableProperties can be the standard .NET types String, binary, bool, DateTime, GUID, int, int64, and double
![Page 34: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/34.jpg)
No Fixed Schema
FIRST LAST BIRTHDATE
Wade Wegner 2/2/1981
Nathan Totten 3/15/1965
Nick Harris May 1, 1976
FAV SPORT
Canoeing
![Page 35: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/35.jpg)
Querying
FIRST LAST BIRTHDATE
Wade Wegner 2/2/1981
Nathan Totten 3/15/1965
Nick Harris May 1, 1976
?$filter=Last eq ‘Wegner’
![Page 36: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/36.jpg)
Purpose of the PartitionKeyEntity LocalityEntities in the same partition will be stored togetherEfficient querying and cache localityEndeavour to include partition key in all queries
Entity Group TransactionsAtomic multiple Insert/Update/Delete in same partition in a single transaction
Table ScalabilityTarget throughput – 500 tps/partition, several thousand tps/accountWindows Azure monitors the usage patterns of partitionsAutomatically load balance partitionsEach partition can be served by a different storage nodeScale to meet the traffic needs of your table
![Page 37: Windows Azure Storage Anton Boyko.NET developer.](https://reader035.fdocuments.net/reader035/viewer/2022062315/5697bf771a28abf838c815e7/html5/thumbnails/37.jpg)
PARTITIONKEY(CATEGORY)
ROWKEY(TITLE)
TIMESTAMP MODELYEAR
Bikes Super Duper Cycle … 2009
BikesQuick Cycle 200 Deluxe
… 2007
… … … …
Canoes Whitewater … 2009
Canoes Flatwater … 2006
PARTITIONKEY(CATEGORY)
ROWKEY(TITLE)
TIMESTAMP MODELYEAR
Rafts 14ft Super Tourer … 1999
… … … …
SkisFabrikam Back Trackers
… 2009
… … … …
Tents Super Palace … 2008
PARTITIONKEY(CATEGORY)
ROWKEY(TITLE)
TIMESTAMP MODELYEAR
Bikes Super Duper Cycle … 2009
BikesQuick Cycle 200 Deluxe
… 2007
… … … …
Canoes Whitewater … 2009
Canoes Flatwater … 2006
Rafts 14ft Super Tourer … 1999
… … … …
SkisFabrikam Back Trackers
… 2009
… … … …
Tents Super Palace … 2008
Partitions and Partition Ranges
Server ATable = Products
Server BTable = Products
[Canoes - MaxKey)
Server ATable = Products
[MinKey - Canoes)