Marco Parenzan – Microsoft Azure MVP

Data Opportunities with Azure

You have: An ASP.NET (MVC), three tier, app A stateful app A relational database

Publish ASP.NET (MVC) application in a Azure Web Site

The PaaS solution for HTTP(s) based applications Automatic (and pre-configured) scalability

Prelude

From a developer point of view A developer (you!) can have fear about

#cloud Talk about

Architecture… Functionalities… …content…

But… …talk with sysadmin about “quality”,

“performances”, “security”, … Give them infrastructure responsibility (security,

identity, backup and so on…)

How do you approach #cloud migration? Many different scenarios

Data Opportunities with Azure

You can handle #data and #cloud at different levels

Use these scenarios as a reference

Scenarios are not exaustive Scenario #1: reuse your experience Scenario #2: adapt your experience Scenario #3: rewrite your experience Scenario #4: enhance your experience

AgendaScenarios

Scenario #1: reuse your experience

You don’t want to change the bits

Scenario #1: reuse your experience

You can create a VM with Linux or Windows Server

You can install any software you want if you have your licence If you need a licence If it’s Open Source

In case you don’t have a licence, you have VM images ready with software and pay as you go (hour/minute basis)

Azure is establishing partnership with major players In this case, Oracle, for example

You can scale in “minutes” …or in “real” hours… …but not weeks (or at least days!)

Handle relational data with IaaS

• Pro

• Full power of a Relational DB

• Simple Scaling Up• No licencing• Pay Per Use (no CAPEX)• Full power control• Ready images (no

installation)– With specific configurations– Just Click, Name, choose,

deploy

• Cons

• OPEX: highest– Relational (non open)

DBMS are expensive

• Bits+Management– You have to be a real

Administrator

• Scalability: you can ONLY scale up– Current sharding needs

application to be updated

– Current sharding is in experimentation stage

Handle relational data with IaaS

Backup works if move data in a far and safe place You do it on a daily basis It can be tested

SQL Server 2012/2014 can backup over Azure Storage Set azure credentials in SQL Server to

authenticate on-premise SQL over Azure (share a certificate between Azure and on-premise host)

Indicate the container and the blob you configure for this activity

Do the backup Restore backup to the new host

Make a SQL backup over Azure

Some screenhots From SQL Backup on Azure

demo

You don’t want to handle management issues

You don’t want vertical scalability-related issues

You don’t need special infrastructural architecture

Handle relational data with SaaS

• Pro

• OPEX: lower than IaaS• Bits+Management• Full power of a

Relational DB– Editions– Clustering– CPU+Memory

• No licencing (No CAPEX)

• Pay Per Use (OPEX)• No need for images

• Cons

• OPEX: lower, than IaaS, but more than other

• Partial power of a Relational DB– …

• Configure what is just available

• Scalability: can ONLY scale up

Handle relational data with SaaS

Not all features of On-Premise SQL are availavable in SQL Azure A “backup” is not enough to bring DB to

the cloud Exporting a script

Some tools are available, updated Developer Way: Database Project,

targeting SQL Azure Operations Way:

https://sqlazuremw.codeplex.com/ Fri Oct 10, 2014 at 9:00 AM

Migrating your on-premise SQL to SQL Azure

Microsoft is making huge investments

Compatibility with full t-sql will improve

DB per customer Hundreds/thousands of databases

SQL Azure performance considerations

demo

IaaS with VMs and share file up to now

Ready in Azure File in Azure Sorage without IaaS No more sharing issues with VMs

Automatic geo-replicazione Orly access From the same region

Local Storage vs. File System vs. Azure Files

demo

Scenario #2: adapt your experience

You are avaible to change some bits But don’t change SQL!

Scenario #2: adapt your experience

None of the following is true: “Network is reliable” “Latency is near to zero or irrelevant” “Bandwidth is unlimited” “Network is secure” “Topology doesn’t change” “Transport cost is irrelevant” “Network is homogeneous”

New development challenges

From Mauro Servienti presentationhttp://www.slideshare.net/mauroservienti/in-the-brain-designing-distributed-scalable-and-reliable-systems

SQL Server Long latencies and timeouts Transient errors Retry strategy

Latency and Retries

Opportunity Lower accesses to SQL Azure and cost less

Cache the page? Cache the content of a Page

Azure Caching

• Session

• Statefulness • Essential to avoid

storing partial running data

• Minimize data

• Cache

• Statelessness (difficult to invalidate data)

• Minimize roundtrips to the database and incur on latency and timeout

Difference between session and cache

Declarative approach Configure ASP.NET MVC to use Redis Session State or Output cache Look At Andrea Dottor presentazione about it

Redis as an efficient in memory key/value store

Find many .NET clients in nuget Example Stack Exchange implementation

Implements a cache-aside pattern Query cache first

If not found, Query datasource and storie info incache

Cache invalidaton rules have to be handled Explicit Timeout

Handle your “repeatable data” with Redis

demo

Cache blobs and static contents Readonly files

Help users far from data Minimize latency Higher responsiveness

Enable this from Azure Storage

Handle your read-only data with CDN

demo

Scenario #3: Change your experience

You are avaible to change the bits and the relational database

Scenario #3: change your experience

Costs Relational DB is me most expensive data

storage

Opportunities Geo Caching Disaster recovery & Business continuity

Consider Stale Data Stale DataEventual consistency

CQRS and DDD No more DB centric

Why change the relational database?

Normalize To avoid data consistency issues Leave consistency assurance to RDBMS

Query Take the responsibility to obtain any

answer also not documented Document, create a PBI in SCRUM for

every query you need to answer

Why query? Why normalize?

New development architectures

Domain model persistence with repository pattern and Persist json data in NoSql documenti database

Publish domain events in Publisher/subscribe event container

Implement inexpensive inde readmodel to outperform readonly operations

An aggregate root persist in a generic repository

DDD point of view Complex aggregate roots that fits

costly into a relational db mapping cost is higher than access cost

(crud)

Eventual consistency is a must Acid costs

DocumentDB is a new service, not in GA

Elastic with sharding

NoSql domain model persistence

demo

Advantages

CRUD scenarios Don’t need normalization Simplify persistence Paging can be not an issue Search is always an issue

Aggregate don’t have to know who needs to known that the entity has changed Lower coupling Turn responsibility

Handlers declare that are interested to an event To Update a read model To Update another aggregate/domain To inform another app To coordinate a workflow

Decoupling with “queue” or “bus”

demo

Entities that changes and that persist in a non relational db imply that some integrity activity have to be done Probably some data duplication

Many handlers can be interested For example read models handlers One handler responsible for one read model

What is a read model? A data aggregation for data information for one

use case Not data join Just filter by values in read model Probably paged

Build the read model

Cheap flat tables Performing tables

Scaling Geo-replication

Great for read models

Azure Table Storage

demo

Scenario #4: “Serve” your experience

Hybrid Cloud is one of the strategies

for Global Cloud Adoption

Office 365 is in the center

of the modern, agile, hybrid Company

Office 365 is a rich platform for

extending and integrating with

Office 365 APIs

SharePoint

Sites

Exchange & Outlook.com

Calendar

Contacts

Mail

Active Directory

Users

Groups

OneDrive

OneDrive for Business

OneDrive

Client API

Search

Sites, Lists and Libs

Taxonomy

BCSWorkflow

…

Your identity goes with you

PCs and devices

3rd party clouds/hosting

Azure AD

Login username/ password Roles Profile Claims User Management – Create, edit

and delete Users Role Management – Create, edit,

delete Roles and manage Users. External Logins – Social providers +

Azure AD Identity Storage Extensibility

ASP.NET Identity 2.1 Features

demo

Office 365 APIs Mail Message API Calendar Events API Contacts API

Office 365 APIs accessible through REST https://outlook.office365.com/ews/odata/Me/Inbox/

Messages https://outlook.office365.com/ews/odata/Me/Events https://outlook.office365.com/ews/odata/Me/Contacts

Office 365 APIs accessible through ExchangeClient library A library which abstracts away sending and receiving

REST request

Office 365 APIs for Calendar, Mail and Contacts

Office 365 APIs for Calendar, Mail and Contacts

Office 365 APIs Mail Message API Calendar Events API Contacts API

Office 365 APIs accessible through REST https://outlook.office365.com/ews/odata/Me/Inbox/Messages https://outlook.office365.com/ews/odata/Me/Events https://outlook.office365.com/ews/odata/Me/Contacts

Office 365 APIs accessible through

ExchangeClient library A library which abstracts away sending and

receiving REST request

Send GET request to Contacts endpoint Office 365 Exchange API only returns JSON responses Paging is accomplished using $skip and $top

Reading Contacts using REST

POST to Add endpoint Provide minimum of @data.type

and GivenName property

Adding a new Contact

demo

Conclusions

PricingTransacti

onSupport

Secondary

Indexes

Comparing Azure Managed Data Services

SQL Database

Tables

StorageAbstracti

onsCatego

ry

DocumentDB

Relational

Key/value store

Document store

Tables,rows,

columns

Collections,

documents

All rows and tables

in a database

Yes

No

Yes

Units of throughp

ut

GBs of storage

Units of throughp

ut

Tables,partitions,

entities

All entities in the same

partition

All documents

in the same collection

Stored Procedures/Trigge

rs

Written in T-SQL

None

Written in JavaScript

QueryLangua

ge

SQL

Subset of OData queries

Extended subset of SQL

Maximum

Database

Size500 GB

100s of TBs

100s of TBs

Choosing a Data TechnologyOptions in the SQL+ world

Use SQL Database when:

You want relational data

You want to get your application up as fast as possible

You want your application to require minimal management

Use DocumentDB, Tables, when:

You need more scale than relational allows

You want a non-relational data model

You don’t want to be locked into a schema

Use Azure Search when:

You want to provide a search interface to your users

http://azure.microsoft.com/en-us/pricing/calculator/

Please distinguish between Fixed Costs and Current Costs!

How much does it costs?

feedback

10

• Email: marco.parenzan@live.it • Slideshare:

http://www.slideshare.net/marcoparenzan

• GitHub: https://www.gihtub.cob/marcoparenzan

• Twitter: http://twitter.com/marco_parenzan

Contatti