Edge performance with in memory nosql
-
Upload
liviu-costea -
Category
Software
-
view
127 -
download
2
Transcript of Edge performance with in memory nosql
![Page 1: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/1.jpg)
Edge Performance - NoSqlIn-Memory NoSQL: Memcache, Redis, Aerospike
Liviu CosteaSoftware DeveloperPlugAndTravel.com
![Page 2: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/2.jpg)
About me
Technical guy
Email: [email protected]
Twitter: clm160
Stack Overflow: user:4138058
Doing software for the travel industry
CTO @ PlugAndTravel.com
![Page 3: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/3.jpg)
Do you need an In-Memory Server?
![Page 4: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/4.jpg)
Scalability at PlugAndTravel?
Moving from one to more servers has challenges:
❏ Session❏ Cached items❏ Static items
Stop using Sticky sessions!
The destination is autoscaling
![Page 5: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/5.jpg)
Autoscaling solutions
❏ Database - could really work, but..
❏ In memory key value stores:❏ Memcached❏ Redis❏ Aerospike
![Page 6: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/6.jpg)
Questions to have answers for
❏ So now you believe an In-Memory NoSql server might be a solution to performance / scalability
![Page 7: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/7.jpg)
First Question
❏ Is it expensive?❏ Linode: 1 cent per GB RAM/h❏ AWS: 1.1 cents per GB RAM/h (~0.7)❏ Azure: 1.2 cents per GB RAM/h
![Page 8: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/8.jpg)
Second Question
❏ RAM is volatile, so I can lose everything?!❏ You can have:
❏ Backups❏ Disk persistence❏ Replica-sets / replication factor
![Page 9: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/9.jpg)
Third Question
❏ Can it scale?❏ Yes, server side clusters will automatically rebalance
themselves after adding more machines
![Page 10: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/10.jpg)
Meet the main characters
❏ Each of these products were created to solve a problem
![Page 11: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/11.jpg)
Memcached - the veteran
❏ Simple KV store to use as a cache❏ Easy ‘immutable’ API❏ Limits: key max 250 bytes, value max of 1MB❏ No clustering, no high availability❏ Not built to add/remove servers live
![Page 12: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/12.jpg)
Redis - the mature
“Redis is an open source (BSD licensed), in-memory data structure store, used as database, cache and message broker…” (redis.io)
On AWS & Azure you have it as a platform
Very popular, big community: Twitter, Airbnb, Flickr, StackOverflow, GitHub, PlugAndTravel and many others
![Page 13: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/13.jpg)
Going with Redis
❏ single threaded server❏ data structures with many usages❏ it has scripting in Lua (stored procedures like)❏ a high-availability solution (replica set & monitoring)❏ server side cluster (newest addition)❏ persistence and backups
![Page 14: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/14.jpg)
Redis - Data structures
❏ Strings (everything is a string)
❏ Lists (double linked lists)
❏ Sets (unique lists)
❏ Hashes (like an object)
❏ Sorted sets (by a value at creation)
Efficiently modeling of data is your main concern!
![Page 15: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/15.jpg)
Redis Demohttps://github.com/lcostea/Redis-node-tutorial
![Page 16: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/16.jpg)
Redis - pay attention to
❏ Beware of the long running processes
❏ Keys or Flush commands can bring your master down
❏ Nice to have: memory inspection command(s)
❏ Timeouts will occur, better be ready
![Page 17: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/17.jpg)
Aerospike - the new guy
“Whether you are building new category defining apps or reimagining legacy apps to take advantage of more data for better outcomes, Aerospike is the only database that delivers both speed and scale” (aerospike.com)
Community and Enterprise editions
AWS and Azure Markets, vagrant box
Gaining popularity: Kayak, AppNexus, AdForm, PlugAndTravel and others
![Page 18: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/18.jpg)
Going with Aerospike
❏ Less data structures than Redis
❏ Scripting support with Lua
❏ Storage models: RAM, RAM + Disk, SSD optimized ->
❏ Synchronous replication and async cross datacenter replication❏ Replication factor in cluster
❏ Namespaces, sets, bins and indexes
![Page 19: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/19.jpg)
Aerospike - Storage models
A namespace can have the storage:❏ In memory (without any persistence)❏ In memory, with persistence; data is backed up on disk❏ SSD storage, with indexes in memoryEach namespace is configured according to its storage requirements.
![Page 20: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/20.jpg)
Aerospike Demohttps://github.com/lcostea/Aerospike-Console-App
![Page 21: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/21.jpg)
Aerospike - pay attention to
❏ Replication is synchronously, so pay attention to cross data center (AZs on AWS not recommended)
❏ Cold / fast restart❏ Row size is max 1M (while on disk)❏ Bin (column names) can not be > 14 bytes (not for storing
values)❏ Your indexes should fit in RAM
![Page 22: Edge performance with in memory nosql](https://reader033.fdocuments.net/reader033/viewer/2022051709/58720e991a28ab176b8b80ff/html5/thumbnails/22.jpg)
Small review
❏ In-memory will mean High Performance and Scalability❏ It is the path to AutoScale❏ Memcached: Immutable cache❏ Redis: Remote dictionary server❏ Aerospike: providing hybrid approaches