Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013
-
Upload
amazon-web-services -
Category
Technology
-
view
3.742 -
download
3
description
Transcript of Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013
![Page 1: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/1.jpg)
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
DAT307 - Deep Dive into Amazon ElastiCache
Architecture and Design Patterns
Nate Wiger, Principal Solutions Architect
November 14, 2013
![Page 2: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/2.jpg)
Contents
• Caching: What’s all this then?
• Amazon ElastiCache
• Laziness, impatience, and hubris
• From one to a dozen nodes
• Memcached vs. Redis showdown
![Page 3: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/3.jpg)
![Page 4: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/4.jpg)
Device Fragmentation
• Phones, tablets, PCs, toasters
• HTML, apps, JSON APIs
• Presentation differs
• Data is the same
• CDN for static images, videos
• Doesn’t help “Welcome Back, Kotter!”
![Page 5: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/5.jpg)
Death By 1000 Queries
• Login, session
• New messages, recent posts
• Calls to Facebook, Twitter APIs
• Your friends love the new Coldplay album!!!
• Sudden viral traffic spikes
![Page 6: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/6.jpg)
![Page 7: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/7.jpg)
cache (noun)
a group of things that have been stored in a secret
place because they are illegal or have been stolen
![Page 8: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/8.jpg)
Typical Web 2.0 App
ELB App
External APIs
![Page 9: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/9.jpg)
Amazon ElastiCache
• Managed cache service
• Memcached or Redis
• Launch cluster of nodes
• Scale up / down
• Monitoring + alerts
![Page 10: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/10.jpg)
Memcached
• In-memory
• Slab allocator
• Multithreaded
• No persistence
• Gold standard
![Page 11: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/11.jpg)
Fire It Up
![Page 12: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/12.jpg)
Fire It Up
![Page 13: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/13.jpg)
Fire It Up
![Page 14: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/14.jpg)
Wire It Up
![Page 15: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/15.jpg)
Wire It Up
![Page 16: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/16.jpg)
Wire It Up # Ruby
require ‘dalli’
cache = Dalli::Client([
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’
])
cache.set("some_key", "Some value")
value = cache.get("some_key")
cache.set("another_key", 3)
cache.delete("another_key”)
![Page 17: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/17.jpg)
Multiple Cache Nodes
ELB App
External APIs
![Page 18: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/18.jpg)
Sharding Across Nodes server_list = [
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’
]
server_index = hash(key) % server_list.length
server = server_list[server_index]
![Page 19: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/19.jpg)
Sharding Across Nodes server_list = [
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’
]
server_index = hash(key) % server_list.length
server = server_list[server_index]
BAD
![Page 20: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/20.jpg)
Consistent Hashing
![Page 21: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/21.jpg)
It’s All Been Done Before
• Ruby – Dalli
• Python – HashRing / MemcacheRing
• Node.js – node-memcached
• PHP – libketama or ElastiCache Client
• Java – SpyMemcached or ElastiCache Client
![Page 22: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/22.jpg)
So Far
• Launched a cache cluster
• Got the node names
• Connected our client
• Figured out sharding
![Page 23: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/23.jpg)
![Page 24: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/24.jpg)
What To Cache?
• Everything!
• Database records
• Full HTML pages
• Page fragments
• Remote API calls
![Page 25: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/25.jpg)
How To Cache It?
• Lazy population
• Write-through
• Timed refresh
![Page 26: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/26.jpg)
Laziness is a Virtue # Python
def get_user(user_id):
record = cache.get(user_id)
if record is None:
# Run a DB query
record = db.query("select * from users where id = ?", user_id)
cache.set(user_id, record)
return record
# App code
user = get_user(17)
![Page 27: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/27.jpg)
Ship It
• Most data is never accessed
• Ensures cache is filled
• Caches fail and scale
• But cache miss penalty
• Best approach for most data
![Page 28: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/28.jpg)
Foresight is 20-20 # Python
def save_user(user_id, values):
record = db.query("update users ... where id = ?", user_id, values)
cache.set(user_id, record)
return record
# App code
user = save_user(17, {"name": "Nate Dogg"})
![Page 29: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/29.jpg)
Laziness vs. Impatience
• Ensures cache is always current
• Write penalty vs. read penalty
• But missing data on scale up
• Plus excess data / cache churn
• Still need lazy fetch too
![Page 30: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/30.jpg)
Combo Move! def save_user(user_id, values):
record = db.query("update users ... where id = ?", user_id, values)
cache.set(user_id, record, 300) # ttl
return record
def get_user(user_id):
record = cache.get(user_id)
if record is None:
record = db.query("select * from users where id = ?", user_id)
cache.set(user_id, record, 300) # ttl
return record
# App code
save_user(17, {"name": "Nate Diddy"})
user = get_user(17)
![Page 31: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/31.jpg)
Timed Refresh
• Run job to periodically update cache
• Good for Top-N lists
• Time-intensive rankings
• Trending items
![Page 32: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/32.jpg)
![Page 33: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/33.jpg)
Monitoring
+ Alerts
![Page 34: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/34.jpg)
Monitoring
• Integration with CloudWatch metrics
• Setup alarms to send via email
• Memory usage
• Evictions
• Which ElastiCache metrics should I monitor?
![Page 35: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/35.jpg)
![Page 36: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/36.jpg)
![Page 37: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/37.jpg)
![Page 38: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/38.jpg)
![Page 39: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/39.jpg)
![Page 40: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/40.jpg)
Node Discovery
• Setup an Amazon SNS topic for ElastiCache
• Have app listen for events – ElastiCache:AddCacheNodeComplete
– ElastiCache:RemoveCacheNodeComplete
• Reconfigure connections
• See Event Notifications and Amazon SNS
![Page 41: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/41.jpg)
Programmable Scaling
ELB App
External APIs SNS
Add Node
![Page 42: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/42.jpg)
Node Auto-Discovery # PHP
$server_endpoint = "mycache.z2vq55.cfg.usw2.cache.amazonaws.com";
$server_port = 11211;
$cache = new Memcached();
$cache->setOption(
Memcached::OPT_CLIENT_MODE, Memcached::DYNAMIC_CLIENT_MODE);
# Set config endpoint as only server
$cache->addServer($server_endpoint, $server_port);
# Lib auto-locates nodes
$cache->set("key", "value");
![Page 43: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/43.jpg)
Redis
• Also in-memory
• Advanced data types
• Atomic operations
• Single-threaded
• Persistence
• Read replicas
![Page 44: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/44.jpg)
Leaderboard with Sorted Sets ZADD leaderboard 556 "Andy"
ZADD leaderboard 819 "Barry"
ZADD leaderboard 105 "Carl"
ZADD leaderboard 1312 "Derek"
ZREVRANGE leaderboard 0 -1
1) "Derek"
2) "Barry"
3) "Andy"
4) "Carl"
ZREVRANK "Barry"
2
![Page 45: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/45.jpg)
Follow the Leader def save_score(user, score):
record = db.query("update users ... where id = ?", user_id, score)
redis.zadd("leaderboard", score, user)
def get_rank(user)
return redis.zrevrank(user) + 1
# App code
save_score("Andy", 556)
save_score("Barry", 819)
save_score("Carl", 105)
save_score("Derek", 1312)
get_rank("Barry") # 2
![Page 46: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/46.jpg)
Redis Replicas
ELB App
External APIs
Replication Group
Reads Writes
![Page 47: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/47.jpg)
Redis Sharding
• Same concept as Memcached
• BUT
• Can't shard – Lists
– Sets / sorted sets
– Hashes
• Require single in-memory structure
![Page 48: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/48.jpg)
Anti-Pattern: Dedicated Nodes
• Spawn multiple nodes
• Use for different features – Leaderboard
– Counters
• Can still shard key-value ops
![Page 49: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/49.jpg)
Dedicated Redis Nodes
ELB App
External APIs
Counters Leaderboard
![Page 50: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/50.jpg)
Summary
• Caching is good
• Good caching is hard
• ElastiCache eases deployment
• Memcached or Redis
• More to come
![Page 51: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.fdocuments.net/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/51.jpg)
Please give us your feedback on this
presentation
As a thank you, we will select prize
winners daily for completed surveys!
DAT307 - Nate Wiger