The future of async i/o in Python
-
Upload
saul-ibarra-corretge -
Category
Technology
-
view
3.664 -
download
1
description
Transcript of The future of async i/o in Python
![Page 1: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/1.jpg)
Saúl Ibarra Corretgé
The future of async i/o in Python
Amsterdam Python Meetup Group, 2nd October 2013
Thursday, October 3, 2013
![Page 2: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/2.jpg)
__repr__(self)
@saghul
I work on VoIP and Real Time Communications (RTC) stuff
Python is my (main) hammer
Thursday, October 3, 2013
![Page 3: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/3.jpg)
while Truelibuv, pyuv
tulip, rose
tornado-pyuv
twisted-pyuv
uvent
evergreen
Thursday, October 3, 2013
![Page 4: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/4.jpg)
help(async_io)
Blocking Is Bad (TM)
Threads → overhead
i/o multiplexors: “Iʼll let you know when you can write”
Windows: “Iʼll let you know when Iʼm done writing”
Thursday, October 3, 2013
![Page 5: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/5.jpg)
import platform
Thursday, October 3, 2013
![Page 6: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/6.jpg)
raise NotImplementedError“Included batteries donʼt fit”
asyncore, asyncchat
Many frameworks
Twisted
Tornado
Eventlet / Gevent
...
Thursday, October 3, 2013
![Page 7: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/7.jpg)
Thursday, October 3, 2013
![Page 8: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/8.jpg)
class MyLoop
Almost no reusable code between frameworks
Multiple equivalent event loop implementations
Multiple protocol implementations
HTTP, anyone?
Thursday, October 3, 2013
![Page 9: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/9.jpg)
import tulipThursday, October 3, 2013
![Page 10: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/10.jpg)
import tulip
PEP-3156: Async I/O support rebooted
A set of components / primitives for async i/o
Python >= 3.3
First version to ship with Python 3.4
(Boring) name for stdlib: asyncio
Thursday, October 3, 2013
![Page 11: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/11.jpg)
dir(tulip)Pluggable event loop
Easy to build adapters
Transport / protocol abstractions
Inspired by Twisted
High-level scheduler based on PEP-380 (yield from), Futures, Tasks
Coroutine friendly Queue, Lock, Event, ...
Thursday, October 3, 2013
![Page 12: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/12.jpg)
print(“hello world”)
from tulip import events
def test(): print("Hello Tulip!") loop.stop()
loop = events.new_event_loop()
loop.call_soon(test)loop.run_forever()
Thursday, October 3, 2013
![Page 13: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/13.jpg)
from tulip import tasks
from tulip import events, tasks
@tasks.coroutinedef sleeper(name, n): while True: yield from tasks.sleep(n, loop=loop) print('Hello from', name)
loop = events.new_event_loop()t1 = tasks.async(sleeper('Task1', 1), loop=loop)t2 = tasks.async(sleeper('Task2', 2), loop=loop)loop.run_forever()
Thursday, October 3, 2013
![Page 14: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/14.jpg)
echo echo echoimport osfrom tulip import events, protocols
class EchoProtocol(protocols.Protocol): def connection_made(self, transport): print('Client connected') self.transport = transport def data_received(self, data): print('Received data:',data) self.transport.write(data) def connection_lost(self, exc): print('Connection closed', exc)
def Factory(): return EchoProtocol()
loop = events.new_event_loop()
f = loop.start_serving(Factory, '127.0.0.1', int(os.getenv('PORT') or 1234))sockets = loop.run_until_complete(f)print('Server started')
loop.run_forever()
Thursday, October 3, 2013
![Page 15: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/15.jpg)
print(sys.modules)People already started implementing stuff with Tulip
Redis client, websockets, Gunicorn worker, ...
https://code.google.com/p/tulip/wiki/ThirdParty
Make your next library Tulip compatible!
(bonus points if it has a nice flower name)Thursday, October 3, 2013
![Page 16: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/16.jpg)
import __future__
All current frameworks would integrate with Tulip, directly or using an adapter
Reusable protocols
Integration with Twistedʼs Deferred
...
Thursday, October 3, 2013
![Page 17: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/17.jpg)
help()
Checkout the code
http://code.google.com/p/tulip/
Join the mailing list
https://groups.google.com/forum/?fromgroups#!forum/python-tulip
Thursday, October 3, 2013
![Page 18: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/18.jpg)
input(‘Questions?’)
bettercallsaghul.comThursday, October 3, 2013
![Page 19: The future of async i/o in Python](https://reader031.fdocuments.net/reader031/viewer/2022020207/554bd172b4c9058f6c8b4c64/html5/thumbnails/19.jpg)
We are hiring!
VoIP, Instant Messaging, Presence
Python, C
PyQt, PyObjC
Windows, Linux, OSX
...
http://ag-projects.com
Thursday, October 3, 2013