Scale Your Data Tier with Windows Server AppFabric

Chris Dufour, ASP MVPSoftware Architect, Compuware

chris.dufour@wigets.net

Agenda

The Problem

A Possible Solution

New Possibilities

What to do next

Simple

Do the simplest thing that will possibly work

Scalability

Able to support the required quality of service as the system load increases

-Wiktionary

Typical Web Architecture

•Need to get routed to same machine (i.e. sticky sessions)

Users•Each

machine round trips for data

•Data that is cached is stored in the memory of one server

Web Tier

•CPU and disk can get saturated due to traffic

•Service access is slow

Data Tier

Web Explosion

Database

IIS/ASP.NET Application

Web Site’s too

slow!!

Database is hot!!

Where did my

shopping cart go?

IIS/ASP.NET Application

IIS/ASP.NET Application

Servers are crashing

Services are slow

Agenda

The Problem

A Possible Solution

New Possibilities

What to do next

Data Near Processing

ASP.NET

Web ServiceDatabase

Smart Client Browser

Cache

CacheCache

CacheCache

Good but…

Cache is scoped to machine / process◦Machines die◦Processes recycle

Cache memory is limited

What if?

You could have as much cache as you wanted?

You could share a giant cache across servers, services and even clients?

What if this was something you could simply add to the platform for 1free?

1Some features may require certain editions of Windows Server

Windows Server AppFabric

AppFabricCACHING

MONITORING

WORKFLOW HOSTING

SERVICE HOSTING

SCALE OUTHIGH AVAILABILITY MANAGEMENT

Unified Cache View

What is AppFabric Caching?An explicit, distributed, in-memory

application cache for all kinds of data Caching clients can be across machines or

processes

Clients Access the Cache as if it

was a large single cache

Cache Layer distributes data

across the various cache

nodes

AppFabric Cache

•No need for sticky sessionsUsers

•Cached data available to all servers

Web Tier

•Multiple machines means scale and high-availability•Lots of cache memory

Caching Tier

•Reduces load on database•Duplicate Service Calls eliminated

Data Tier

Data Distribution - Partitioned Cache

… Web Tier

Cache TierC

ach

e Serv

ice

Cach

e Serv

ice

Cach

e

Serv

ice

A C EDB G IHF

ASP.Net App

Caching Client

ASP.Net App

Caching Client

ASP.Net App

Caching Client

AC ED BGI HF

Scale on Data Size◦ More machines => More memory to cache

Scale on Cache Throughput◦ More machines => keys distributed across more machines => better throughput

Scale Test OutputLoad

Throughput

Latency

1 Cache ServerAs load increases, throughput fails

to scale latency increases

Caching Tier

Add a Second Cache Server

Load

Throughput

Latency

Load MaxThroughput increases

Latency decreases

Caching Tier

Add a Second Cache Server

Load

Throughput

Latency

Caching Tier

Associated Press

Caches metadata and news

Serves 16 million hits per day

Increased the amount of cached data 6 times.

System.Web.Cache

AppFabric DataCache

Cach

ing

Serv

ice

Cach

ing

Serv

ice

Cach

ing

Serv

ice

Configuration Store

Install AppFabric

Copy Client DLLs

… Application Application Application<hosts>

<host name="BL1CDB8083714“ cachePort="22233" cacheHostName="DistributedCacheService"/> ….. </hosts> <localCache isEnabled=“true" ../><security … />

Caching Access Layer

Caching Access Layer

Caching Access Layer

.NET 3.5 SP1 OR .NET 4

Configure AppFabric

.NET 4

Update Web.config

Deployment

Usage Pattern – Cache Aside (Explicit Caching)

// Read from CacheToy toyObj = (Toy) catalog.Get("toy-101");

Application

Cach

ing

Serv

ice

Database

// If Not present in the cacheif (toyObj == null){ // Read from backend.. toyObj = ReadFromDatabase(); // Populate Cache catalog.Put("toy-101", toyObj);

return toyObj;}

Caching Access Layer

Administration

PowerShell cmdlets are used to administer the cache cluster

Rich set of cmdlets for ◦Cache cluster management◦Cache creation and monitoring

Demo

Hello AppFabric Cache

Using PowerShell

Remember – PowerShell can also be called from .NET Code!

Security

Domain Based Security Option◦Domain Account / Local Account based

Authentication◦Only authorized servers can join the cluster◦Only authorized clients can connect to the

cluster

Transport Level Security◦Turn on/off Signing or Encryption

Can turn off Cache Security◦Use Firewalls, IPSec, VLANs to protect cache

grant-cacheallowedclientaccount  RedDomain\Machine1$ grant-cacheallowedclientaccount  RedDomain\John

Logical Hierarchy

Machine Host

◦ Physical processes hosting AppFabric Caching instance.

Named Caches◦ Can span across machines◦ Defined in the configuration file

Regions◦ Physically co-located Container of

Cache Items◦ May be implicit or explicitly

created Cache Item

◦ Key, Payload (Object ), Tags, TTL, Timestamps, Version

Regions Region A

Key Payload Tags Key Payload Tags 121 xxxx “Toy” “Child”

123 yyyy “Toy” “Chair”..

AppFabric Caching Service

Named Cache : Product Catalog

Named Cache : Electronics Inventory

AppFabric Caching Service

AppFabric Caching Service

AppFabric Caching Service

AppFabric Caching API// Create instance of cachefactory (reads appconfig)DataCacheFactory fac = new DataCacheFactory();

// Get a named cache from the factoryDataCache catalog = fac.GetCache("catalogcache");

// Simple Get/Putcatalog.Put("toy-101", new Toy("Puzzle", .,.));

// From the same or a different clientToy toyObj = (Toy)catalog.Get("toy-101");

// Region based Get/Putcatalog.CreateRegion("toyRegion");

// Both toy and toyparts are put in the same region catalog.Put("toy-101", new Toy( .,.), “toyRegion”);Catalog.Put("toypart-100", new ToyParts(…), “toyRegion”);

Toy toyObj = (Toy)catalog.Get("toy-101“,"toyRegion");

Access APIs – Tagging Items

Tag hotItem = new Tag("hotItem");

catalog.Put("toy-101", new Toy("Puzzle"), new Tag[]{hotItem}, “toyRegion”);

catalog.Put("toy-102", new Toy("Bridge"), “toyRegion”);

// From the same or a different clientList<KeyValuePair<string, object>> toys = catalog.GetAnyMatchingTag("toyRegion", hotItem);

Types of Data

Reference Activity Resource

Primary Read Only Read-Write Not shared

Read-Write, Shared

Catalog Data Shopping Cart Auction Data/Seat Assignment

Web Tier

Distributed Cache

Shopping Cart

Grocery Catalog

Grocery Inventory

Grocery Shop

ApplicationApplication

AppFabric Caching Client

Reference Data – Performance

Catalog data doesn’t change often Unnecessary network cost to access from different machines Solution – Local Cache

Put(K2, v3)

Routing Table

Cache2Cache1

Primary for K2,V2

K2, V2

Primary for K1,V1

K1, V1

Cache3

Primary for K3,V3

K3, V3

AppFabric Caching Client

Local CacheRouting Table

K2, V2

Get(K2)Get(K2)

K2, V3

Reference Data – Bulk Get

Bulk Fetch from region◦200-300k ops per second◦Fewer network calls

Catalog.BulkGet( new List<string>(){“toy-101”, “toy-102”} ,

“toyRegion”);

Activity Data – Session Integration

…Session State

stored in AppFabric Caching

Cach

e

Serv

ice

Cach

ing

Serv

ice

Load Balance RequestsNo more sticky routing

Cach

ing

Serv

ice

Scale your Session StoreDynamically

Highly Available

Drop in AppFabric Caching

SessionStoreProvider

Allows session state to be shared amongst multiple applications

Application

Caching Access Layer

Application ApplicationCaching Access

LayerCaching Access

Layer

<sessionState mode="Custom“ customProvider="SessionStoreProvider"><providers> <add name="SessionStoreProvider" type=“Microsoft.Data.Caching.DataCacheSessionStoreProvider, ClientLibrary“ cacheName="<YourNamedCache>"/></providers></sessionState>

ApplicationApplication

(K2, V2)

Cache2Cache1 Cache3

Primary for

Activity Data - Availability

Get(K2)

Primary for Primary for

K3, V3

AppFabric Caching Client

Routing Table

K2

, V2

PUT

Secondary for

K2, V2

K1, V1

Secondary for

K3, V3

Secondary for

K1, V1

AppFabric Caching Client

Routing Table

K2, V2

Replication AgentK2, V2

Resource Data - Optimistic Locking GetCacheItem returns a version object Every update to an object internally increments it's version Supply the version obtained along with the Put/Remove Put/Remove will succeed only if the passed in version matches the

version in the cache

Version Based Update

Time Client1 Client2 (Different Thread or process)

T0 CacheItem item = catalog.GetCacheItem(“PlayerRegion”, ”Zune”);

CacheItem item = catalog.GetCacheItem(“PlayerRegion”, ”Zune”);

T1 ((ZuneObject)item.Object).inventory --;

((ZuneObject)item.Object).inventory--;

T2 catalog.Put(“PlayerRegion”, “Zune”, item.Object, item.Version);

T3 catalog.Put(“PlayerRegion”, “Zune”, item.Object, item.Version);// Version mismatch// Client must retry again

Two clients access the same item

Both update the item

Second Client gets in first; put succeeds because item version matches; atomically

increments the version

First client tries put;Fails because the versions

don’t match

K1

Resource Data - Pessimistic Locking

Take locks on non-existent keys◦Allows you to co-ordinate calls for data

Client1: GetAndLock ("k1")

Client2: GetAndLock ("k1")

Client3: Get ("k1")

Regular Get succeeds

GetAndLock gets lock handle

Other GetAndLock on same item fails

Data Race

GET GETGET

Lock Non-Existent Key

GET/LOCK GET/LOCKGET/LOCK

Cach

e

Serv

ice

Cach

ing

Serv

ice

Cach

e

Serv

ice

Composite Race

CALL WAITWAIT

Cach

e

Serv

ice

Cach

ing

Serv

ice

Cach

e

Serv

ice

Composite Race

PUTUNLOCK

GETGET

Cach

e

Serv

ice

Cach

ing

Serv

ice

Cach

e

Serv

ice

Resource/Activity Data – Tracking ChangesCache Event notificationsRegister on any client to notify changesBatched Notifications

DataCache.RegisterCacheLevelCallback( int filter, DataCacheChangeCallback delegate);

DataCache.RegisterRegionLevelCallback( String region, int filter, DataCacheChangeCallback delegate);

DataCache.RegisterKeyLevelCallback( String region, String key, int filter, DataCacheChangeCallback delegate);

Application

Cache2Cache1

Primary for

K2, V2

Primary for

K1, V1

Cache3

Primary for

K3, V3

Scalable Notifications

AppFabric Caching Client

Routing Table

Register Notification for Key “K3"

Map Keys to Partition

Poll Required Nodes

Nodes Return List of Changes LSN Order

Partition: P2

Last LSN: 19

Call DelegateStore Last LSN

Change LogPartition P11 Add K22 Del

K32

Change Log

(Partition

P2)

18 Del K32

19 Del K43

Change Log33 Add K134 Del

K22

Agenda

The Problem

A Possible Solution

New Possibilities

What to do next

Data Center

Pre-Fetch

Hospital

Data Center

Pre-Fetch

HospitalRemote Clinic

WAN

Slow!!

Data Center

Pre-Fetch

HospitalRemote Clinic

Cach

e

Serv

ice

WAN

Agenda

The Problem

A Possible Solution

New Possibilities

What to do next

Web Platform Installer

Select Enterprise

Install AppFabric

Install AppFabric

AppFabric on MSDN

http://msdn.microsoft.com/AppFabric

Thanks to our contributors