Use case for using the ElastiCache for Redis

in Production.

Table of contents

• Self-introduction• The minimum knowledge needed Redis

operation• gumi’s Redis Operation• Warning for operating ElastiCache for Redis• Summary

Table of contents

• Self-introduction• The minimum knowledge of Redis operation• gumi’s Redis Operation• Warning for operating ElastiCache for Redis• Summary

Self-introduction

• Tomonori Honma • gumi Inc.

• System Operation Engineer• All Game Server’s operarion in Japan• I don’t develop Game Events

• My favorite AWS Service:Amazon S3 @CkReal

Table of contents

• Self-introduction• The minimum knowledge needed Redis

operation• gumi’s Redis Operation• Warning for operating ElastiCache for Redis• Summary

The minimum knowledge of Redis operation

• Key Value Store• In-memory is very fast

• When is storing data over memory in Server, is controlled by maxmemory-policy(Parameters Group) settings. (default:volatile-lru)

• Two methods for Data Persistence• Append-only file• RDB file

• Can use multiple databases

Table of contents

• Self-introduction• The minimum knowledge needed Redis

operation• gumi’s Redis Operation• Warning for operating ElastiCache for Redis• Summary

gumi's Previous Generation Architecture

memcached

RDS(MySQL5.5)

app

player-shard

others

staticjobqredis

M

S

M

S

masterSQS

gumi's RecentGeneration Architecture

redis

RDS(MySQL5.6)

app

jobq

S

SQSstatic(S3)

The Use caseof Application

• User’s actions in game events can be immediately reflected in Redis.

• example• Realtime Ranking• Matching process of each player• Card Deck Information

The Use caseof Operation

• Error Tracking Tool(Sentry) use as a Buffer• Recent Operation tools sometimes uses Redis

• For quick checks, EC2 is very troublesome.

gumi’s Redis Operation

• Using almost master-slave• A few games use master only

• master : AOF and no RDB, slave : AOF and RDB• One GET operation takes sometimes 2.47sec???• Fork time in different systems

• Multiple backups of RDB are saved• This is to avoid problem where master DB is

flushed and that wipes out slave backup too.• The use multiple databases in one Redis node• Instead of memcached, use redis

• Developer is difficult to understand the timing of memcached's eviction.

Table of contents

• Self-introduction• The minimum knowledge needed Redis

operation• gumi’s Redis Operation• Warning for operating ElastiCache for Redis• Summary

Possible and Not (yet) possible

Possible Not (yet) possible

No EC2 instance!!Import data file from S3Configure AOF settingUse awscliPromotion of replica node

Auto failover like RDSBackup RDB file to S3Change instance typePoint-In-Time-Recovery

Possible and Not (yet) possible

Possible Not (yet) possible

No EC2 instance!!Import data file from S3Configure AOF settingUse awscliPromotion of replica node

Auto failover like RDSBackup RDB file to S3Change instance typePoint-In-Time-Recovery

2014/04/24 ReleaseBackup time includes time

taken to upload RDB file to S3?EC2 : few min

ElastiCache : Tens of minutes

Possible and Not (yet) possible

Possible Not (yet) possible

No EC2 instance!!Import data file from S3Configure AOF settingUse awscliPromotion of replica node

Auto failover like RDSBackup RDB file to S3Change instance typePoint-In-Time-Recovery

Possible and Not (yet) possible

Possible Not (yet) possible

No EC2 instance!!Import data file from S3Configure AOF settingUse awscliPromotion of replica node

Auto failover like RDSBackup RDB file to S3Change instance typePoint-In-Time-Recovery

ElastiCache for Redis is still nothigh-performance like RDS

ElastiCache for Redisimpression

• There is nothing wrong in performance• When you create a snapshot, it is recommended to run

from slave• Backup and Restore ElastiCache Redis Clusters

• appendonly settings is not available in cache.t1.micro• We have not experienced a big problem with

ElastiCache for Redis• When AOF file is corrupted, I have restored from

RDB file of slave server.

Table of contents

• Self-introduction• The minimum knowledge needed Redis

operation• gumi’s Redis Operation• Warning for operating ElastiCache for Redis• Summary

Summary

• Can prepare redis nodes in a short period of time• Since we have very few operation enginners in

our company, this is very helpful :)• Initially, functions we assumed to exist actually

were not yet implemented• But feature set has grown

• Currently, there are no big problems• we are increasing the number of production nodes

Thank you!!