Using Coroutines to Create Efficient, High-Concurrency Web Applications
-
Upload
matt-spitz -
Category
Documents
-
view
7.365 -
download
3
Transcript of Using Coroutines to Create Efficient, High-Concurrency Web Applications
![Page 1: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/1.jpg)
Using Coroutines to Create Efficient, High-Concurrency Web Applications
Matt Spitzmeebo, inc.
![Page 2: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/2.jpg)
2
What’s a Web Application, Anyway?
Database
Application
Application
![Page 3: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/3.jpg)
3
High-Concurrency Web Applications
Database
Application
Application
![Page 4: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/4.jpg)
4
High-Concurrency Web Applications
• Many requests per second• Optimization opportunities– Hardware cost– Response time– Concurrency– Database impact
![Page 5: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/5.jpg)
5
Meebo Bar
![Page 6: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/6.jpg)
6
Meebo Bar
• 1000+ sites• Quantcast: 197 MM monthly uniques*• LOTS of pageviews• LOTS of ad requests
* http://bit.ly/xAPXx
![Page 7: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/7.jpg)
7
Meebo’s Ad Server
• Given– User features– Available ads
• Objective– Maximize revenue
• P(click)• Price
– Satisfy advertisers• Respect targeting• Smooth campaign delivery
• Complex application• Lots of concurrent requests
![Page 8: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/8.jpg)
8
Sample App: FortuneTeller
![Page 9: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/9.jpg)
9
Sample App: FortuneTeller
• Given– Username– Available fortunes
• Objective– Select fortune for user• JaccardSimilarity(username, fortune)
• username=PyConIsForLovers– “Generosity and perfection are your everlasting
goals.”
![Page 10: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/10.jpg)
10
Hosting FortuneTeller
• Apache CGI• Apache mod_wsgi• Twisted• gevent + gunicorn
![Page 11: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/11.jpg)
11
Hosting FortuneTeller
• Evaluation metrics– Code complexity– Library support– Memory efficiency– Multi-core support
![Page 12: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/12.jpg)
12
Take One: Apache CGI
• One process per request• O/S schedules CPU
![Page 13: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/13.jpg)
13
Take One: Apache CGI
• Advantages– Straightforward, synchronous code– Isolated requests
• Disadvantages– Process overhead– Cold cache
![Page 14: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/14.jpg)
14
EvaluationCode
ComplexityLibrarySupport
MemoryEfficiency
MulticoreSupport
Apache CGI Apache mod_wsgi
Twisted
gevent + gunicorn
![Page 15: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/15.jpg)
15
Performance
• Environment– 4-core VM, 1 GB RAM– Ubuntu Server 10.10– MySQL on host machine– 25ms interface delay
• 1024 requests, X concurrent
![Page 16: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/16.jpg)
16
Performance
1 2 4 8 16 32 64 128
apache_cgi
Concurrent Requests
Med
ian
Resp
onse
Tim
e
![Page 17: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/17.jpg)
17
Take Two: Apache mod_wsgi
• Using mpm_prefork– Worker processes handle requests– One concurrent request per process– Memory cached between requests
• O/S schedules CPU
![Page 18: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/18.jpg)
18
Take Two: Apache mod_wsgi
• Advantages– Straightforward, synchronous code– Cached memory
• Disadvantages– Resource inefficient
• Need working set in each process• Cold cache on restart
– Managing worker count• Too few: 502• Too many: OOM? Database DoS?
![Page 19: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/19.jpg)
19
EvaluationCode
ComplexityLibrarySupport
MemoryEfficiency
MulticoreSupport
Apache CGI Apache mod_wsgi Twisted
gevent + gunicorn
![Page 20: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/20.jpg)
20
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgi
Concurrent Requests
Med
ian
Resp
onse
Tim
e
![Page 21: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/21.jpg)
21
Take Three: Twisted
• Asynchronous framework– Events and callbacks– Twisted orchestrates context switches
• Twisted server– Single event loop– Concurrent requests
![Page 22: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/22.jpg)
22
Quick Break: Event Loopss = socket.socket(…)s.setblocking(ISBLOCKING)s.connect((HOST, PORT))greeting = s.recv(1024)s.close()
• Blocking– Wait for data
• Nonblocking– Initiate, return immediately
• Data (if available)• Exception: “I’m not done yet”
– Requires more plumbing
![Page 23: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/23.jpg)
23
Process events that are ready (select/poll).
Quick Break: Event Loops
• Nonblocking sockets in an event loop
Events
f(x):s = NonBlockingSocket(…)greeting = s.recv(1024)print x, “|”, greeting
fd=18, fp=f, {x: 80}
Call recv().
1.
2.Create context, add to the event loop.3.
4.
Return to context when data is ready.5.
“80 | Hello from socket s!”6.
fd=5, fp=g, {s: ‘hi’, a: 5}
fd=2, fp=f, {x: 8080}
fd=3, fp=myfunc, {}
fd=18, fp=f, {x: 80}
fd=3, fp=myfunc, {}
fd=5, fp=g, {s: ‘hi’, a: 5}
fd=18, fp=f, {x: 80}
![Page 24: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/24.jpg)
24
Take Three: Twisted
• Asynchronous framework– Events and callbacks– Twisted orchestrates context switches
• Twisted server– Single event loop– Concurrent requests
![Page 25: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/25.jpg)
25
Take Three: Twisted
• Advantages– Shared memory– User space context switches
• Disadvantages– Develop asynchronously
• Stuck in the framework
– Asynchronous libraries• No I/O in C
– Unfair scheduling– Using multiple cores
![Page 26: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/26.jpg)
26
EvaluationCode
ComplexityLibrarySupport
MemoryEfficiency
MulticoreSupport
Apache CGI Apache mod_wsgi Twisted gevent + gunicorn
![Page 27: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/27.jpg)
27
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgitwisted
Concurrent Requests
Med
ian
Resp
onse
Tim
e
![Page 28: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/28.jpg)
28
Take Four: gevent + gunicorn
• gevent– Networking library• Uses event loop• Synchronous API
– Synchronous code running asynchronously• Monkey patching
– Rewrites standard modules
• Coroutines for function context– Lightweight threads, no stack– greenlet implementation
![Page 29: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/29.jpg)
29
Take Four: gevent + gunicorn
• gunicorn (“Green Unicorn”)– Lightweight WSGI server– Multiple worker processes• Share queued requests
– gevent support
![Page 30: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/30.jpg)
30
Take Four: gevent + gunicorn
• Advantages– Best of both worlds!
• mod_wsgi– Straightforward, synchronous code– No framework, just python– Multicore support
• Twisted– Shared memory– User space context switches
• Disadvantages– Pure-python libraries– Unfair scheduling
![Page 31: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/31.jpg)
31
EvaluationCode
ComplexityLibrarySupport
MemoryEfficiency
MulticoreSupport
Apache CGI Apache mod_wsgi Twisted gevent + gunicorn
![Page 32: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/32.jpg)
32
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgitwistedgunicorn_1
Concurrent Requests
Med
ian
Resp
onse
Tim
e
![Page 33: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/33.jpg)
33
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgitwistedgunicorn_1gunicorn_2
Concurrent Requests
Med
ian
Resp
onse
Tim
e
![Page 34: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/34.jpg)
34
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgitwistedgunicorn_1gunicorn_2gunicorn_4
Concurrent Requests
Med
ian
Resp
onse
Tim
e
![Page 35: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/35.jpg)
35
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgitwistedgunicorn_1gunicorn_2gunicorn_4gunicorn_6
Concurrent Requests
Med
ian
Resp
onse
Tim
e
![Page 36: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/36.jpg)
36
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgitwistedgunicorn_1gunicorn_2gunicorn_4gunicorn_6gunicorn_8
Concurrent Requests
Med
ian
Resp
onse
Tim
e
![Page 37: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/37.jpg)
37
“Evented” Development
• Synchronous code still runs asynchronously– Requests aren’t independent
• Things to keep in mind– Duplicate work– Socket caching– CPU hogging
![Page 38: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/38.jpg)
38
gunicorn + gevent in Production• Managing gunicorn
– greins• Randall Leeds (tilgovi): github/meebo/greins
– Multiple apps• URL routing
– Server hooks• Worker launch• Pre/post requests
– Daemon interface• Debugging gevent
– gevent-profiler• Shaun Lindsay (srlindsay): github/meebo/gevent-profiler
– Execution trace– Time spent
![Page 39: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/39.jpg)
39
Load-tested, unicorn-approved!
• Blocking code is simple• Nonblocking code is efficient• gevent + gunicorn– Simple – Efficient– Reliable
![Page 40: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/40.jpg)
40
Load-tested, unicorn-approved!
![Page 41: Using Coroutines to Create Efficient, High-Concurrency Web Applications](https://reader036.fdocuments.net/reader036/viewer/2022062513/554bd274b4c905706a8b4d8a/html5/thumbnails/41.jpg)
41
Thanks!