The State of WebSockets in Django Rami Sayar - @ramisayar

Technical Evangelist Microsoft Canada

Agenda • Where is WebSockets right now? • Where is Socket.IO? • Where does Django fit in? • Using Django SwampDragon. • Using Gevent.

CONFOO - @RAMISAYAR

Assumptions • Working knowledge of Django, JavaScript & Python.

Note: Slides will be made available.

CONFOO - @RAMISAYAR

What is a production web app?

• Django is surrounded by tons of applications, services and code in production.

• Advanced Django architecture is needed to support Django in production without failing hard.

• WebSockets falls apart from the rest.

CONFOO - @RAMISAYAR

WebSockets •  “WebSocket is a protocol for

full-duplex communication channels over a single TCP connection.” Wikipedia

•  Image Reference

CONFOO - @RAMISAYAR

WebSockets

CONFOO - @RAMISAYAR

Socket.IO •  “Socket.IO enables real-time

bidirectional event-based communication. It works on every platform, browser or device, focusing equally on reliability and speed.” Reference

• Use by everyone: Microsoft Office, Yammer, Zendesk, Trello, etc…

CONFOO - @RAMISAYAR

Socket.IO Version 1.0 is here! Technically, now at v1.3.4

CONFOO - @RAMISAYAR

Python, Django & WebSockets.

CONFOO - @RAMISAYAR

Python and WebSockets • Python and WebSockets has notoriously been very difficult.

• Two reasons: •  WSGI protocol does not support WebSockets. Most popular form of

web request handling in the Python web apps. Most frameworks are designed for use with FastCGI or mod_wsgi, etc.

•  Default Python networked apps not exactly designed for event-based communication, you have to use things like gevent and twisted/tornado.

CONFOO - @RAMISAYAR

Django & WebSockets • Django applications complete HttpResponses. • Django not well suited for WebSocket event handling structure

nor the channel approach in Socket.IO.

CONFOO - @RAMISAYAR

Presenting 3 Methods for WebSockets in Django

CONFOO - @RAMISAYAR

gevent •  “gevent is a coroutine-based Python networking library that

uses greenlets to provide a synchronous API on top of the libev event loop.”

• Event loop based on libev (same as Node.js) • Lightweight execution units, a.k.a. “lightweight threads”. • Cooperative sockets with SSL support.

• Similar programming model to Node.js.

CONFOO - @RAMISAYAR

gevent-socketio • gevent-socketio is a pure Python implementation of the

Socket.IO protocol. Basically, it is a port of Socket.IO and typically behind in major releases.

• Designed to work with multiple different WSGI-based web frameworks.

• Handles the “WSGI” problem by providing a custom gevent WSGI server for use with gevent-socketio.

• Only dependencies are gevent. •  Imperfect solution: Django is still blocking (monkey

patching is not perfect).

CONFOO - @RAMISAYAR

Demo 1: Django chat engine with gevent-socketio https://github.com/abourget/gevent-socketio/tree/master/examples/django_chat

CONFOO - @RAMISAYAR

“Native” Django Implementation • Previous approach requires significant modifications to your

current Django app serving stack. Native as in you don’t need to know anything besides Django.

• Building an implementation of WebSockets on top of Django. •  Identified two problems for WebSockets in Django:

•  Problems with WSGI still present EXCEPT in uWSGI. •  Networked event handling model is problematic.

• Solution: •  Use Nginx which follows – event-driven async architecture. •  uWSGI & Gevent for event handling.

CONFOO - @RAMISAYAR

Django-WebSocket-Redis • Module implements websockets on top of Django. • Use Redis as a data store for messaging behind uWSGI and Nginx. • Runs a separate Django main loop in a cooperative concurrency

model using gevent -> 1 thread for all open websockets. • Django can be started as usual with ‘./manage.py runserver’. • Normal requests communication with separate main (websocket)

loop through Redis. • Read:

Offloading Websockets and Server-Sent Events AKA “Combine them with Django safely”

CONFOO - @RAMISAYAR

Demo 2: Django-WebSockets-Redis https://github.com/jrief/django-websocket-redis

CONFOO - @RAMISAYAR

SwampDragon • New project that adds websocket support • Self-publishing models • Django ORM Style Data Filter for Subscription Channels –

Similar to WebSockets + Data Filter • Relies on Redis for messaging + Tornado • Angular JS Support

CONFOO - @RAMISAYAR

Demo 3: Django Swampdragon https://github.com/jonashagstedt/swampdragon

CONFOO - @RAMISAYAR

On Redis • Redis is extremely popular as an easy and simple publish-

subscribe. • Redis is very particular about memory. Running out of memory

in Redis -> CRASH AND BURN! •  If you don’t have enough memory in your VM to run Redis

Cache -> Change your approach, use Apache Kafka.

CONFOO - @RAMISAYAR

What did we learn? • The State of WebSockets in Django. • Django and WebSockets is not impossible.

CONFOO - @RAMISAYAR

Thank You! Questions? Follow @ramisayar

CONFOO - @RAMISAYAR

Resources • http://swampdragon.net/ • http://django-websocket-redis.readthedocs.org/en/latest/ • https://github.com/jrief/django-websocket-redis • https://github.com/abourget/gevent-socketio/tree/master/

examples/django_chat • http://jpadilla.com/post/74391616727/the-easiest-way-to-add-

websockets-to-django • http://ferretfarmer.net/2013/09/05/tutorial-real-time-chat-with-

django-twisted-and-websockets-part-1/

CONFOO - @RAMISAYAR

©2013  Microso-  Corpora1on.  All  rights  reserved.  Microso-,  Windows,  Office,  Azure,  System  Center,  Dynamics  and  other  product  names  are  or  may  be  registered  trademarks  and/or  trademarks  in  the  U.S.  and/or  other  countries.  The  informa1on  herein  is  for  informa1onal  purposes  only  and  represents  the  current  view  of  Microso-  Corpora1on  as  of  the  date  of  this  presenta1on.  Because  Microso-  must  respond  to  changing  market  condi1ons,  it  should  not  be  interpreted  to  be  a  commitment  on  the  part  of  Microso-,  and  Microso-  cannot  guarantee  the  accuracy  of  any  informa1on  provided  a-er  the  date  of  this  presenta1on.  MICROSOFT  MAKES  NO  WARRANTIES,  EXPRESS,  IMPLIED  OR  STATUTORY,  AS  TO  THE  INFORMATION  IN  THIS  PRESENTATION.