asyncio/aiohttpAnton Kasyanov

Some slides adopted from A. Svetlov

Who?

• Python enthusiast

• RWTH Aachen Master student

• Contributed into aiohttp, aiozmq, aioes, aiohttp-session

• Using asyncio in production for >1 year

Asynchronous programming

Sync.Images from http://krondo.com/

Asynchronous programming

Sync. ThreadedImages from http://krondo.com/

Asynchronous programming

Sync. Threaded Real worldImages from http://krondo.com/

Asynchronous programming

AsyncImages from http://krondo.com/

Asynchronous programming

AsyncImages from http://krondo.com/

I/O only!

Asynchronous programming• Non-blocking IO

• Superior (comparing to threads) speed of web servers

Asynchronous programming• Non-blocking IO

• Superior (comparing to threads) speed of web servers

• But: not elegant code, callbacksfs.readdir(source, function(err, files) { if (err) { console.log('Error finding files: ' + err) } else { files.forEach(function(filename, fileIndex) { console.log(filename) gm(source + filename).size(function(err, values) { if (err) { console.log('Error identifying file size: ' + err) } else { console.log(filename + ' : ' + values) aspect = (values.width / values.height) widths.forEach(function(width, widthIndex) { height = Math.round(width / aspect) console.log('resizing ' + filename + 'to ' + height + 'x' + height)

Two years ago

• twisted

• tornado

• gevent

• etc.

One year ago

• asyncio came out

• no http

• no database drivers

Hello world

import asyncioimport time

@asyncio.coroutinedef sleepy(): print("before sleep", time.time()) yield from asyncio.sleep(5) print("after sleep", time.time())

asyncio.get_event_loop().run_until_complete(sleepy())

Today• aiohttp

• aiopg

• aiomysql

• aioredis

• aiozmq

• aioes

• aiomemcache

• aiohttp_jinja2

• aiohttp_session

• aiocouchdb

• aiogreen

• aiokafka

aiohttp: client API

aiohttp: clientimport asyncioimport aiohttp

@asyncio.coroutinedef fetch_page(url): response = yield from aiohttp.request('GET', url) body = yield from response.read() return body

content = asyncio.get_event_loop().run_until_complete( fetch_page('http://python.org'))print(content)

aiohttp: client

>>> session = aiohttp.ClientSession()>>> yield from session.get(... 'http://httpbin.org/cookies/set/my_cookie/my_value')>>> r = yield from session.get('http://httpbin.org/cookies')>>> json = yield from r.json()>>> json['cookies']['my_cookie']'my_value'

aiohttp: server API

aiohttp: simple server

import asynciofrom aiohttp import web

@asyncio.coroutinedef handle(request):

text = “Hello, world!” return web.Response(body=text.encode(‘utf-8'))

app = web.Application()app.router.add_route('GET', '/', handle)

aiohttp: simple server

loop = asyncio.get_event_loop()f = loop.create_server(app.make_handler(), '127.0.0.1', 8080)srv = loop.run_until_completed(f)print(‘Server started on’,srv.sockets[0].getsockname())try: loop.run_forever()except KeyboardInterrupt: pass

aiohttp: upload files

<form action="/store_mp3" method="post" accept-charset="utf-8" enctype="multipart/form-data">

<label for="mp3">Mp3</label> <input id="mp3" name="mp3" type="file" value="" /><input type="submit" value="submit" />

</form>

aiohttp: upload files

@asyncio.coroutinedef store_mp3_view(request): data = yield from request.post() filename = data['mp3'].filename input_file = data['mp3'].file content = input_file.read() return web.Response(body=content, headers=MultiDict( {'CONTENT-DISPOSITION': input_file})

aiohttp: websockets@asyncio.coroutinedef websocket_handler(request): ws = web.WebSocketResponse() ws.start(request) while True: data = yield from ws.receive_str() if data == 'close': yield from ws.close() else: ws.send_str(data + '/answer') return ws

aiohttp: more

• middleware

• aiohttp_sessions

• aiohttp_jinja2

benchmarks

• around the same as tornado

• twisted.web is a bit faster

• WSGI is slower

• WIP

What’s next?

aiopgimport sqlalchemy as safrom aiopg.sa import create_enginemetadata = sa.MetaData()

tbl = sa.Table('tbl', metadata, sa.Column('id', sa.Integer, primary_key=True), sa.Column('val', sa.String(255)))

app = web.Application()app[‘db’] = yield from create_engine(…)

aiopg@asyncio.coroutinedef handler(request): with (yield from request.app['db']) as conn: yield from conn.execute( tbl.insert().values(val=‘abc’))

res = yield from conn.execute(tbl.select()) for row in res: print(row.id, row.val)

aiozmqimport asynciofrom aiozmq import rpc

class Handler(rpc.AttrHandler):

@rpc.method def remote(self, arg1, arg2): return arg1 + arg2

aiozmq@asyncio.coroutinedef go(): server = yield from rpc.serve_rpc( Handler(), bind='tcp://127.0.0.1:5555')

client = yield from rpc.connect_rpc( connect='tcp://127.0.0.1:5555')

ret = yield from client.call.remote(1, 2) assert ret == 3

PEP 492async def read_data(db): data = await db.fetch('SELECT ...') ...

async for row in Cursor(): print(row)

async with session.transaction(): ... await session.update(data) ...

Links

• https://docs.python.org/3/library/asyncio.html

• http://aiohttp.readthedocs.org/

• http://aiozmq.readthedocs.org/

• http://aiopg.readthedocs.org

Questions?

antony.kasyanov@gmail.com http://antonkasyanov.com