Introduction to asyncio
-
Upload
saul-ibarra-corretge -
Category
Technology
-
view
2.556 -
download
1
description
Transcript of Introduction to asyncio
@saghul
Introduction to asyncioSaúl Ibarra Corretgé
PyLadies Amsterdam - 20th March 2014
github.com/saghul
Sockets 101
import socket
server = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM)server.bind(('127.0.0.1', 1234))server.listen(128)print("Server listening on: {}".format(server.getsockname()))
client, addr = server.accept()print("Client connected: {}".format(addr))
while True: data = client.recv(4096) if not data: print("Client has disconnected") break client.send(data)
server.close()
Scaling Up!
import socketimport _thread
def handle_client(client, addr): print("Client connected: {}".format(addr)) while True: data = client.recv(4096) if not data: print("Client has disconnected") break client.send(data.upper())
server = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM)server.bind(('127.0.0.1', 1234))server.listen(128)print("Server listening on: {}".format(server.getsockname()))
while True: client, addr = server.accept() _thread.start_new_thread(handle_client, (client, addr))
Scaling Up! (really?)
Threads are too expensive
Also, context switching
Use an event loop instead!
The Async Way (TM)
Single thread
Block waiting for sockets to be ready to read or write
Perform i/o operations and call the callbacks!
Repeat
(Windows is not like this)
Why asyncio?
asyncore and asynchat are not enough
Fresh new implementation of Asynchronous I/O
Python >= 3.3
Trollius: backport for Python >= 2.6
Use new language features: yield from
Designed to interoperate with other frameworks
Components
Event loop, policy
Coroutines, Futures, Tasks
Transports, Protocols
asyncio 101
import asyncio
loop = asyncio.get_event_loop()
@asyncio.coroutinedef infinity(): while True: print("hello!") yield from asyncio.sleep(1)
loop.run_until_complete(infinity())
Coroutines, futures & tasks
Coroutines, futures & tasksCoroutine
generator function, can receive values
decorated with @coroutine
Future
promise of a result or an error
Task
Future which runs a coroutine
FuturesSimilar to Futures from PEP-3148
concurrent.futures.Future
API (almost) identical:
f.set_result(); r = f.result()
f.set_exception(e); e = f.exception()
f.done(); f.cancel(); f.cancelled()
f.add_done_callback(x); f.remove_done_callback(x)
Futures + Coroutines
yield from works with Futures!
f = Future()
Someone will set the result or exception
r = yield from f
Waits until done and returns f.result()
Usually returned by functions
Undoing callbacks
@asyncio.coroutinedef sync_looking_function(*args): fut = asyncio.Future() def cb(result, error): if error is not None: fut.set_result(result) else: fut.set_exception(Exception(error)) async_function(cb, *args) return (yield from fut)
Tasks
Unicorns covered in fairy dust
It’s a coroutine wrapped in a Future
WAT
Inherits from Future
Works with yield from!
r = yield from Task(coro(...))
Tasks vs coroutines
A coroutine doesn’t “advance” without a scheduling mechanism
Tasks can advance on their own
The event loop is the scheduler!
Magic!
Echo Serverimport asyncio
loop = asyncio.get_event_loop()
class EchoProtocol(asyncio.Protocol): def connection_made(self, transport): print('Client connected') self.transport = transport def data_received(self, data): print('Received data:',data) self.transport.write(data) def connection_lost(self, exc): print('Connection closed', exc)
f = loop.create_server(lambda: EchoProtocol(), 'localhost', 1234)server = loop.run_until_complete(f)print('Server started')
loop.run_forever()
Echo Server Reloadedimport asyncio
loop = asyncio.get_event_loop()clients = {} # task -> (reader, writer)
def accept_client(client_reader, client_writer): task = asyncio.Task(handle_client(client_reader, client_writer)) clients[task] = (client_reader, client_writer) def client_done(task): del clients[task] task.add_done_callback(client_done)
@asyncio.coroutinedef handle_client(client_reader, client_writer): while True: data = (yield from client_reader.readline()) client_writer.write(data)
f = asyncio.start_server(accept_client, '127.0.0.1', 1234)server = loop.run_until_complete(f)loop.run_forever()
HTTP Serverimport asyncioimport aiohttpimport aiohttp.server
class HttpServer(aiohttp.server.ServerHttpProtocol):
@asyncio.coroutine def handle_request(self, message, payload): print('method = {!r}; path = {!r}; version = {!r}'.format( message.method, message.path, message.version)) response = aiohttp.Response(self.transport, 200, close=True) response.add_header('Content-type', 'text/plain') response.send_headers() response.write(b'Hello world!\r\n') response.write_eof()
loop = asyncio.get_event_loop()f = loop.create_server(lambda: HttpServer(), 'localhost', 1234)server = loop.run_until_complete(f)loop.run_forever()
Redis
import asyncioimport asyncio_redis
@asyncio.coroutinedef subscriber(channels): # Create connection connection = yield from asyncio_redis.Connection.create(host='localhost', port=6379) # Create subscriber. subscriber = yield from connection.start_subscribe() # Subscribe to channel. yield from subscriber.subscribe(channels) # Wait for incoming events. while True: reply = yield from subscriber.next_published() print('Received: ', repr(reply.value), 'on channel', reply.channel)
loop = asyncio.get_event_loop()loop.run_until_complete(subscriber(['my-channel']))
More?
We just scratched the surface!
Read PEP-3156 (it’s an easy read, I promise!)
Checkout the documentation
Checkout the third-party libraries
Go implement something cool!
“I hear and I forget. I see and I remember.
I do and I understand.” - Confucius
Questions?
@saghulbettercallsaghul.com