Post on 07-Jul-2015
description
Reactor patternEvent handling pattern
Dejan PekterSoftware Development Engineer II
Nordeus and me
● Over 150 employees● Top Eleven
○ ~ 12 000 000 MAU○ ~ 5 000 000 DAU○ Cross-platform pioneer ○ Most popular online sports game
● Backend Software Development Engineer○ 250+ servers○ 10+ types○ 70.000 requests / minute○ Every fail is expected
Goal
● Share knowledge on Reactor design pattern used on our servers
● In example of server that is ○ Highly concurrent○ Highly responsive○ Well designed
● Consider different solutions○ Incrementally find solution○ Pros○ Cons
Prerequisites
● Design pattern ● Thread● Context switch● Request● Response
Easy solution
● Wait for clients to connect● For every accepted client
○ Create thread which will take responsibility over that client
Easy solution - conclusion
● Good:○ Intuitive○ Readable code○ Easy to understand
● Bad:○ 1000 users ~ 1000 threads○ Memory overhead○ Context switch overhead○ Create/kill overhead
Improved solution
● Create thread pool with N threads● Accept client
○ Pass client to thread pool ○ Thread pool assigns thread to that client
Improved solution - conclusion
● Good:○ Constrained number of threads○ Threads are reused
● Bad:○ Harder to implement○ Input/output is mixed with business logic○ Possibility of starvation
Basic nuclear reactor theory
NO! Re-act!
How would you react on good joke (event) that I can’t really tell?
FunnyThingHandler
How would you react onLOGIN_OR_REGISTER
You would not!Server would.
Event granulation
CLIENT MESSAGE
Reactor pattern
Reactor
Events
Callbacks
Event handler
EventHandler
+ handleEvent()+ getEventId()
LoginHandler
+ handleEvent()+ getEventId()
RegisterHandler
+ handleEvent()+ getEventId()
Reactor detailed design
EventHandler
+ handleEvent()+ getEventId()
LoginHandler
+ handleEvent()+ getEventId()
RegisterHandler
+ handleEvent()+ getEventId()
Server
Reactor
+ handleEvents()+ registerHandler()+ removeHandler()
dispatches 1 *
creates
Communicator
+ getEvents()
1
1
Reactor phases
● Initialize phase○ Registration of event handlers
● Event handling phase○ Gather all ready messages○ Process all messages
■ Get requests■ Identify event handler■ Execute callback
Reactor - Sequence Diagram
Main Program Event Handler Reactor
EventsregisterHandler()
instantiateHandler()
handleEvents()
handleEvent()
1) Initialize phase
2) Event handling phase
concreteHandler
event
Reactor solution - conclusion
● Good○ Logic is separated○ Easy to add new event handler○ Granulation to message level
● Bad○ Harder to debug○ Not intuitive○ Single threaded
Our improvements
● One reactor is not enough N reactors
● Separate logic even more○ Server - single thread
■ Accept clients ■ Dispatch to reactor by round robin
○ Reactor■ IO responsibility■ Build application level messages■ Find event handler■ send to dispatcher
○ Dispatcher - thread pool■ Execute callback
Our improvements - conclusion
● Good○ Responsibilities are fully separated○ 70.000 requests per minute in production○ Tested even for order of magnitude more○ Good design
● Bad○ Even harder debugging
Conclusion
● Reactor pattern○ Great maintainability○ Forget about how messages arrive, they just do○ Single threaded by definition○ Multithreaded in production
● Business○ Invest time and money in good solution○ You get new stuff almost for free
Remember...
● Don’t reinvent the wheel● Got the wheel, but you need a plane
○ think○ improve○ build it yourself
Don’t forget to...WORK AND PLAY
Thanks for your attention!Questions?