Caching:How It Works and When You Should Use It Shy CohenPrincipal, Shy Cohen Consulting

AAP314

2

What Gurus Need to Know

What’s a cache and what it doesCore and advanced conceptsCaching on premises and in Windows Azure

Goal: Grok CachingReally, really understand it!Become caching gurus

Non-goal: DemosDownloadSamplesVideos

Concepts Guidance

3

What is Caching?

A. A mechanism for storing data close to where it’s used

B. A mechanism used to speed up a systemC. A mechanism for decreasing load on a system

componentD. All of the above

Not a new concept

4

The Cardinal Rule of Caching

In order to use caching, your app needs to be OK working with possibly stale data

Possibly different than what’s stored in the System of Record

Cached data should be considered “historical”Captured at some point in the pastMay not represent current state

There are ways to overcome thisUsing specific patterns, which we’ll cover in this talk

5

Example: Caching in a Web Application

CacheCache

Speeds up the

application

Reduces database load

Shared

CacheV1 V2

Update

Same dataacross all servers

Load Balancer

High load on the

database

Long load time & delays

Users see different data, depending on the server

they get routed to!

6

Windows Server AppFabric Caching

A free, distributed, in-memory shared cacheCombines the memory capacity of multiple cache servers to create a large cache

A place for applications to temporarily store information they use often

The cache holds key-value pairsCan hold any (serializable) CLR Object

Broader scoped than on-machine cachingMore memory, more users, more possibilitiesBetter overall performance & system utilization

Basic Concepts

CacheConcepts

8

Unified View

Nodes are coordinated to create a unified view of the cache for the cache clients

Cluster is abstracted by the APINo need to know on which node the data is or where it should be stored

Client’s View

Cache Segment

#3

Cache Segment

#2

Cache Segment

#1

CacheConcepts

9

Data Categorization

Reference Data“Read-Heavy”, Shared between users (concurrent access)Examples: Product description, customer profile data

Resource DataRead-Write, Shared between users (concurrent access)Examples: Number of units in stock, online survey results

Activity DataRead-Write, User specific (no concurrent access)Examples: Shopping cart content, survey response

10

Caching Guru Pop-Quiz

NameData

Affinity

Access Patter

n

Shared“Read-Heavy”

SharedRead-Write

User Specific

Read-Write

Reference

Resource

Activity

CacheGuidance

11

Cache

Determine What Data to Cache

Categorize your data and cache appropriately

Affects where to store, how long to store for, whether you want explicit deletion, etc.

Cache the right dataOutput is not always the most efficient thing to storeInstead, consider storing independently-meaningful sub-sets that can be used independently

User Settings Service

Company Settings

User Customizatio

n

C1 U1

Caller

C1+U1

U2

C1

C1+U2

Get Settings for User 1 of Company 1

Get Settings for User 2 of Company 1

C1+U1

C1+U2

CacheGuidance

12

Assume “Best Effort” Caching

Application must withstand cache being unavailableHardware failure, configuration problems, upgrades

Application must withstand cached data unavailability

Do not trust information to stay in the cacheCache doesn’t guarantee data retention

Do not store “important” information only in the cache

Costly to reproduceCrucial for correct operation of your application

CacheGuidance

13

Use an Effective Caching Strategy

Carefully balance cached objects’ size vs. number of objects

Performance degrades when objects are largeCache the smallest objects possiblePerformance also affected by number of the round-trips you need to make to the cache clusterMeasure performance & optimize size/number for your scenario

Keep data coherentDo not break down a large object to multiple small ones since you may not be able to reconstruct it

CacheGuidance

14

Craft Your Keys Wisely

Use the minimal but compete set of unique identifiers for the data you’re caching

Keys consume space in the cache

CacheConcepts

15

Write-Aside Cache

Provides a place for the application tostore data “nearby” for quick accessClient is responsible for managing the content

Storing, fetching, updating, deleting*Client can optionally specify cached objects’ lifetime

Optionally changing the cluster’s default expiration policyOptionally set per-item expiration when adding dataOptionally update the expiration time of a cached objectThe cluster will auto-remove objects based on these settings

Cache

Service / Website

Data Sourc

e

Caller

CacheGuidance

16

Use Write-Aside Cache Appropriately

Use it for Reference DataPractically no data access contentionBut there’s a more efficient solutions (details soon)

Use it for Resource DataOnly with the appropriate concurrency control mechanism in place (details soon)

Use it for Activity DataPerfect fit, since there’s no data access contention

CacheConcepts

17

Regions

“Name areas” within the cacheThere’s always a default one (called “default”)You can programmatically create and delete regions

Useful when multiple apps share a cache clusterAllows you to avoid key name-collisionsCan also be used for different areas within a single app

Cache Cluster “default” “Permissions” “Last Viewed Items”“Friends’

Purchases”

CacheConcepts

18

Tags

Metadata to describe an object in the cacheA string, encapsulated in a DataCacheTag instanceItems can have zero, one, or more tags

Tags are specified when the object is createdCan’t update or remove tags for existing objectsTIP: Replace the entire object to update its tags

You can search the cache based on these tagsAn API lets you find objects based on one tag or multiple tags (“any” or “all”)

Basic Concepts - Review

Unified ViewData CategorizationWrite-Aside CacheTags & Regions

Auto Object Removal

CacheConcepts

21

Object Removal

Explicit DeletionUser explicitly removes an item from the cache

ExpirationWhen an object is proactively removed from the cache because of staleness

EvictionWhen an object is reactively removed from the cache due to low memory conditions

22

Specifying Expiration

Expiration is a hint to the system on how long to keep an object in the cache

“I won’t need this object after X minutes”Cache will not return expired objects, but might keep them in memory

ConfigurableFor the cache cluster, orAt each Add/Put call

Relative to when the object was addedA special technique allows exact expiration as well (details soon)

Can extend the expiration period for existing cached objects

23

Empty Space

Expiration and Eviction Thresholds

Cached Data

Cached Data

Empty Space

Empty Space

90%

70%

Non ExpiredObjects Evicted

Expired Objects Evicted

Cached Data

High Watermark

Low Watermark

Eviction can be turned off.Use with great

care!

CacheGuidance

24

TIP: Local-Time Based “Expiration”

Enable expiry at a certain (predetermined) timeRelative to the client’s time-zone

Create a function to map “Now” to a time segmentThis hour, Today, This week

Add the function’s result to the key you’re generatingKey = original key + suffixE.g. Product Price Cache Key = SKU + Today’s Date

Use the key to store the data in the cacheWhen “Now” maps to a different time segment

The generate key will no longer match the key used for storingTherefore, you will no longer “find” the object in the cache

Local Cache

CacheConcepts

26

Client-Side Caching

Essentially, another layer of cachingGreat for Reference DataMay not be suitable for other types of data

When enabled, client stores a localreference to a deserialized version offetched cache-objects

Stored in the memory of the client applicationSaves network access time and deserialization

On subsequent requests for a previously fetched object, it returns a reference to the locally cached one

Without contacting the server

CacheService

/ Website

Data Source

Caller

27

Object Removal from the Local Cache

Local cache is more constrained then the cache cluster

Configurable limit on the number of objects stored locallyDefault is 10,000 objects

Local cache removal policy is based onMax objects: when reached, the local client will attempt to remove ~20% of the LRU locally cached objectsLocal lifetime restrictions: configured per clientPull-based notifications: optional pull for cache changes

With a per-client configurable pull interval

CacheGuidance

28

Use Local Cache With Care

Local caching increases memory pressure on the client

This can actually make the application run slower

Pull-notifications are not cost-freeLocal cache doesn’t make copies of the objects

I.e. all local callers get same instance of the locally cached objectExercise care when changing the object from multiple places

The 2nd layer of caching increases stalenessUse it for client-side immutable objects that change infrequently on the server

Concurrency

CacheConcepts

30

Data Item Versions

Every Cached items has a versionVersion is auto-assigned by the cache

Version can be used forSaving bandwidth by only fetching changed items

GetIfNewer takes the current version you have

Optimistic concurrencyVersion-dependent overloads for Put and Remove

CacheGuidance

31

Fetch Only When Cache Data Changed

Specify last fetched version when asking for an object from the cache

Remember the objects you fetched, and their version numbersRe-fetch objects only if they changed

Guidance only applies for non-trivial objectsMight not be worth the effort for very small objectsWill depend on your specific configurationMeasure to find out your specific size threshold

CacheConcepts

32

Optimistic and Pessimistic Concurrency

Optimistic concurrencyManaged through versionsMost apps don’t have conflict scenariosBut resolving conflicts may not be trivial

Pessimistic concurrencyUse when concerned about the app making concurrent updatesLock based. Only one client can own the lock at a timeGetAndLock returns a DataCacheLockHandle PutAndUnlock requires a DataCacheLockHandle

CacheGuidance

33

Use Locking API Carefully

Locking is not enforcedI’s an opt-in modelAll users must “play nice” in order for this to work

Use regions and a naming convention to help developers know they need to use lockingLocked objects won’t expire, but they may expire immediately if unlocked after their expiration time

TIP: PutAndUnlock has overloads to extend the expiration period

Advanced Concepts

35

The Case for Cache Data-Protection

Easy to recreate data In-memory cache only“Important” data In-memory cache, backed by persistent storage (e.g. SQL)Something in the middle ?

Data that you’d prefer not to loseData that’s really expensive to produceExamples: User session state, Mainframe data

CacheConcepts

36

High Availability

Provides cache data-protectionImplemented by saving 2 copies of the cached data on different nodes in the cluster

Safeguards against cache node failuresUpon node failure, single-copy data is replicatedDoesn’t protect against more than one node failing

CacheGuidance

37

Verify Your Need for HA

By default, your app should withstand cache data lossHigh-availability costs

Performance is lowered due to replicationHA halves your available memory

Consider setting up separate cache clusters for high-availability scenarios

CacheConcepts

38

Graceful Shutdown

Added in AppFabric Caching v1.1Moves node data to other cluster nodes before shutting a node downUseful for planned upgrades, patches, etc.

Relevant for both HA and non-HA scenarios

39

The Case for Cache-Source Coherency

To maintain cache-source data coherency, every the cache needs updating every time the source is updated

Can we enforce that?Read involves 2 or 6 network “legs”

Can we make this more efficient?We have 2 data access concepts

Name-value pairs for the cacheSomething else for the data sourceCan we unify this?

Cache

CacheConcepts

40

Read Through & Write Behind

Read-ThroughUser asks for data from the cacheIf the data is found in the cache, it’s returnedIf data is not found, the cache fetches the data using a specialized providerData is cached and returned to the user

Write-BehindUser asks to write/update data in the cacheThe cache stores the data in memoryUsing a specialized provider, the cache stores the data in permanent storage

Writing a provider is not hard, but not always trivialYou need to be able to map from the key to a “query” (database, web service, mainframe, etc.)

CacheProvide

r

CacheGuidance

41

When to use RT and WB

When it’s important to lower data round-tripsWhen clients always need non-stale data

Example: Inventory levelsOnly achievable if the data being manipulated is access exclusively through the cache

When wanting to save time and/or moneyExamples: Remote data source, complex queries, pay-per-query, data obtained from a MainframeAchievable by exposing the data source only through the cache

In most cases the RT/WB data will be dataResource

Implementing Caching

43

Windows Server AppFabric Cache

Cache runs on a cluster of serversCalles a “Cache Cluster”

All the servers in a cluster are part of a “fabric”Managed using PowerShell and behave as a single entityCommunicate, CooperateShare configuration via SQL or XML in a file share

Cache Cluster typically separate from the client (web) tierEach node runs the Cache as a Windows ServiceMore memory is better

Max recommended is 16GB per node

44

Windows Azure Shared Cache

On demand provisioning of cache resources128MB to 4GB per named cache

You don’t have to manage the cache clusterIncrease & decrease capacity using portal without losing data*

Compatible API with some differences: No H/A, notifications, disabling eviction, regions and tagsDefault expiration set at 48 hours (can change per-object)Connection is to a named cache, not cache cluster

45

Windows Azure Caching (Preview)

Full client API parity with Windows Server AFC

Option #1: Added to an existing cloud roleUses the instance’s memory (control over percentage used)Scales with roleNo additional cost

Option #2: Used a dedicated Worker RoleUses as much memory as you wantScales separately from existing rolesCosts like any other compute node

CacheGuidance

46

Choose the Right Cloud Caching Model

In all cases, it’ll be the new model No predefined cache sizes and much larger cachesFlexibility, control, and isolation of cache from other usersLower latencyNo extra cost on top of compute

47

What In-Role Dedicated SharedFeatures Full Full Limited

Cost No additional Pay for role Pay by memory

Scalability With app / tier Separate Separate

Competition for local resources

Yes No No

Memcached API compatible

Yes Yes No

Provisioning and Administration

Configuration Configuration Provision through portal

ASP.NET Caching Yes Yes Yes

Expiration Policy None, Absolute, Sliding Window (from last access)

None, Absolute, Sliding Window (from last access)

Absolute (48hr)

High Availability Yes Yes No

Notifications Yes Yes No

Disabling eviction Yes Yes No

Regions and Tags Yes Yes No

48

Caching Does Little to Reduce Cost

Accessing cloud data is not free, but it’s very cheapAzure storage access ($0.01 for 10K storage transactions)Cross data-center traffic ($0.12 per 1GB)

Cache access in same geo-region is freeConsider cost of cache

Windows Azure Shared CacheDedicated cache instances(No additional cost for the on-instance cache)

CacheGuidance

49

Use Cloud Caching for the Right Reasons

Speed of access to dataAzure SQL databaseAzure storageRemote storage and services

Avoiding hitting rate limitsUp to 5,000 entities/messages/blobs per second per accountUp to 500 messages per second from a single queueUp to 500 entities per second from a single table partition

CacheGuidance

50

Understand the Implications of Cloud Caching

Your cloud role resources are restricted

Co-located cache deploymentCompetes for local memoryCompetes for local CPUCompetes for the network

Virtual Machine

Size CPU Cores Memory

Allocated Bandwidth

(MBPS)

Temporary Local

Storage

Maximum Data Disks (1 TB each)

ExtraSmall Shared 768 MB 5 20 GB 1

Small 1 1.75 GB 100 50 GB 2

Medium 2 3.5 GB 200 100 GB 4

Large 4 7 GB 400 200 GB 8

Extra Large 8 14 GB 800 400 GB 16

CacheConcepts

51

ASP.NET Session State Provider

Stores session state in shared memoryRequired for Web FarmsFaster than SQL

Simple configuration change to your web app Enables session state to be cached across a clusterUses cache-aside, not RT+WB (session is Activity Data)TIP: Use HA to reduce the chances of sessions state loss

<sessionState mode="Custom" customProvider="SessionStoreProvider"> <providers> <add name="SessionStoreProvider“ type="Microsoft.ApplicationServer.Caching.DataCacheSessionStoreProvider, Microsoft.ApplicationServer.Caching.Client, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35“ cacheName=“MyCache"/> </providers></sessionState>

CacheConcepts

52

ASP.NET Page Output Caching

Cache full HTTP responses (page output caching)Plugs into the ASP.NET 4 output cache provider extensibility pointSimple configuration + page updates

<caching> <outputCache defaultProvider="DistributedCache"> <providers> <add name="DistributedCache" type="Microsoft.Web.DistributedCache.DistributedCacheOutputCacheProvider, Microsoft.Web.DistributedCache" cacheName="default" dataCacheClientName="default" /> </providers> </outputCache></caching>

<%@ OutputCache Duration="60" VaryByParam="*" %>

Summary

54

Summary

Caching is important to application performance and scalability, but it needs to be centralizedWindows Server AppFabric Cache is a free, scalable, distributed, in-memory cacheWindows Azure Caching (Preview) doesn’t cost extra, and gives all the benefits of the on-premises cacheUnderstanding the type of data you cache is key to your caching strategy and implementation choice

What: Reference Data, Resource Data, Activity DataHow: Cache Aside, RT/WB, Client Side Caching, Versions, Concurrency, High Availability, …

55

What Next?

Analyze & categorize your dataImplement caching correctly & efficiently

ASP.NET Session Provider is an easy startMeasure & Optimize usageMake things run FAST!

Related Content

AZR309 - Windows Azure Caching: New Capabilities in the Next Release

Resources

Connect. Share. Discuss.

http://northamerica.msteched.com

Learning

Microsoft Certification & Training Resources

www.microsoft.com/learning

TechNet

Resources for IT Professionals

http://microsoft.com/technet

Resources for Developers

http://microsoft.com/msdn

Complete an evaluation on CommNet and enter to win!

Please Complete an Evaluation Your feedback is important!

Multipleways to Evaluate Sessions

Scan the Tagto evaluate thissession now on myTechEd Mobile

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS

PRESENTATION.