Hands On Windows Azure Building A Twitter Clone
-
Upload
nik-kalyani -
Category
Technology
-
view
2.049 -
download
2
description
Transcript of Hands On Windows Azure Building A Twitter Clone
Session Code: NE.09
Windows Azure: Building a Twitter Clone
Nik Kalyani
DotNetNuke Corporation
About
Nik Kalyani• Co-founder, DotNetNuke Corporation• Technology Entrepreneur, 15 years• Microsoft MVP (2005-2009)
Online • Email: [email protected]• Twitter: @techbubble• Blog: http://www.techbubble.net
Session Objective
Provide insights on how to architect a large-scale, asynchronous application with the same capabilities as Twitter on Azure
while leveraging its non-RDMS capabilities
Agenda
• Azure Architecture
• Storage Concepts
• Micro-blogging Application – Requirements– Architecture– Processing
ServiceBus
AccessControl
Workflow
…
Database
Reporting
Analytics
…
Compute Storage Manage
Identity
Devices
Contacts
…
…
…
Your Applications
Azure Architecture
Azure Application Architecture
Cloud Storage (blob, table, queue)
Web RoleLBn
Worker Role
m
Azure Durable Storage
• Durable, scalable, available store• Simple interface
– REST– ADO.NET Data Services
• Simple abstractions
Blobs Tables
…
Queues
Azure Data Storage Concepts
Account
Container Blobs
Table Entities
Queue Messages
http://<account>.blob.core.windows.net/<container>
http://<account>.table.core.windows.net/<table>
http://<account>.queue.core.windows.net/<queue>
Blob Storage Concepts
BlockBlobContainerAccount
erica
pictures
IMG001.JPG
IMG002.JPG
movies MOV1.AVI
Block 1
Block 2
Block 3
Blobs: Features and Functions
• Store large objects
• Associate metadata with blob– Metadata is <name, value> pairs– Set/Get with or separate from blob data bits
• Standard REST Interface
• Limitations– Block Id - 64 Bytes Per Block– PutBlob - 64 MB Total– Blob MetaData - 8 KB Per Blob– PutBlock / PutBlockList - 50 GB Each– Blocks - 4 MB Each
Table Storage Concepts
EntityTableAccount
erica
users
Name =…
Email = …
Name =…
Email = …
photo index
Photo ID =…
Date =…
Photo ID =…
Date =…
Tables: Features and Functions• Massively scalable, highly available and
durable structured storage– Not RDMS -- no joins, no maintenance of foreign keys
• LINQ and REST interfaces via ADO.Net Data Services– Partition key is exposed to the application
• Limitations– Up to 255 Entity Properties (including PartitionKey,
RowKey, and Timestamp)– 1 MB Max for Combined Size of All Data in an Entity's
Properties
Queue Storage Concepts
MessageQueueAccount
erica
thumbnail jobs
128x128, http://…
256x256, http://…
photo processing
jobs
http://…
http://…
Queues: Features and Functions• An account can create many queues..
– Queue Name is scoped by the account
• A Queue contains messages..– No limit on number of messages stored in a queue– A message is stored for at most a week in a queue
• http://<Account>.queue.core.windows.net/<QueueName>
• Messages..– Message size <= 8 KB– To store larger data, store data in blob/entity storage,
and the blob/entity name in the message
• Relationships
– Any user can “follow” any other user
– A user can “unfollow” another user he/she is “following”
Micro-blogging ApplicationRequirements
• Micro-blogging
– Any user can post a message of up to 140 characters in length
– All message posts are public
Micro-blogging ApplicationRequirements
• Timeline
– A public timeline of all posted messages is available to anybody for up to 24 hours from the time the message was first posted
– A user timeline of all messages posted by a user is available to anybody forever
– A “following” timeline of all messages posted by all users that a user is following in the prior 24 hours is available to each user
Micro-blogging ApplicationRequirements
• Micro-blogging
– The number of users will determine volume of messages
– However, common situation is that a small number of users will post majority of the messages
Micro-blogging ApplicationArchitectural Considerations
• Timelines
– The public timeline is transient and will have a high frequency of reads and writes
– A user timeline is permanent and will have a small number of writes and a large number of reads
– A “following” timeline is transient and will have a large number of irregularly timed writes and a small number of reads
Micro-blogging ApplicationArchitectural Considerations
Micro-blogging ApplicationProfile, Messages and Relationships Storage
Table = UserPartitionKey = UsernameRowKey=string.Format("{0:D19}", DateTime.MaxValue.Ticks – DateTime.UtcNow.Ticks);
• Use Table storage sharded by user
• Entities with the same PartitionKey are stored on the same storage node
• Obtaining recent messages, followers or following list require only a single query for each
• Adding or Deleting Follower or Following requires two writes (i.e. one for each user)
Properties
EntityType = “Message”Source = UsernameMessage Text
EntityType=“Follower”Username
EntityType=“Following”Username
EntityType=“Profile”<Any profile property>
Micro-blogging ApplicationFollow
User A Partition
Follower[User C]
User B Partition
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
User A follows User B
Micro-blogging ApplicationFollow
User A Partition
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Micro-blogging ApplicationFollow
User A Partition
Follower[User C]
User B Partition
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
User B follows User A
Follower[User A]
Following[User B]
Micro-blogging ApplicationFollow
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Micro-blogging ApplicationMessage Posting
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
User A posts a message
Micro-blogging ApplicationMessage Posting
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Message added to Message Queue
User AMessage
Micro-blogging ApplicationMessage Posting
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
After Message Worker Role processes…
Msg (A)
Msg (A)
Msg (A)
User B User C
Micro-blogging ApplicationMessage Posting
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Msg (A)
Msg (A)
Msg (A)
After Maintenance Worker Role processes…
Micro-blogging ApplicationMessage Maintenance
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Msg (A)
Msg (A)
Msg (A)
Msg (D)Msg (D)25 hours
Msg (X)
User A posts a message
Micro-blogging ApplicationMessage Maintenance
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Msg (A)
Msg (A)
Msg (A)
Msg (D)Msg (D)25 hours
Msg (X)
Message added to Message Queue
User AMessage
Micro-blogging ApplicationMessage Maintenance
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Msg (A)
Msg (A)
Msg (A)
Msg (D)Msg (D)25 hours
Msg (X)
After Message Worker Role processes…
User B User C
Msg (A)
Msg (A)
Msg (A)
Micro-blogging ApplicationMessage Maintenance
User A Partition
Follower[User B]
Follower[User C]
Following[User B]
User B Partition
Follower[User A]
Following[User A]
Following[User D]
User C Partition
Follower[User Y]
Follower[User Z]
Following[User A]
Following[User X]
Message Queue
Maintenance Queue
Msg (A)
Msg (A)
Msg (A)
Msg (D)
Msg (X)
Msg (A)
Msg (A)
Msg (A)
After Maintenance Worker Role processes…
• Cloud app storage significantly different from RDMS…requires shift in thinking
• Queues make apps efficient … processing is done asynchronously
• Duplication of data is often required and this is OK
• Sharding is an important consideration for the scalability of your cloud application
• Absence of join, sort make application design challenging but with planning this is manageable
Micro-blogging ApplicationTakeaways
Evaluation form
Vul je evaluatieformulier in en maak kans op een van de prachtige prijzen!!
Fill out your evaluation form and win one of the great prizes!!
Session Code: NE.09