The State of WebSockets in Django Rami Sayar - @ramisayar
Technical Evangelist Microsoft Canada
Agenda • Where is WebSockets right now? • Where is Socket.IO? • Where does Django fit in? • Using Django SwampDragon. • Using Gevent.
CONFOO - @RAMISAYAR
Assumptions • Working knowledge of Django, JavaScript & Python.
Note: Slides will be made available.
CONFOO - @RAMISAYAR
What is a production web app?
• Django is surrounded by tons of applications, services and code in production.
• Advanced Django architecture is needed to support Django in production without failing hard.
• WebSockets falls apart from the rest.
CONFOO - @RAMISAYAR
WebSockets • “WebSocket is a protocol for
full-duplex communication channels over a single TCP connection.” Wikipedia
• Image Reference
CONFOO - @RAMISAYAR
WebSockets
CONFOO - @RAMISAYAR
Socket.IO • “Socket.IO enables real-time
bidirectional event-based communication. It works on every platform, browser or device, focusing equally on reliability and speed.” Reference
• Use by everyone: Microsoft Office, Yammer, Zendesk, Trello, etc…
CONFOO - @RAMISAYAR
Socket.IO Version 1.0 is here! Technically, now at v1.3.4
CONFOO - @RAMISAYAR
Python, Django & WebSockets.
CONFOO - @RAMISAYAR
Python and WebSockets • Python and WebSockets has notoriously been very difficult.
• Two reasons: • WSGI protocol does not support WebSockets. Most popular form of
web request handling in the Python web apps. Most frameworks are designed for use with FastCGI or mod_wsgi, etc.
• Default Python networked apps not exactly designed for event-based communication, you have to use things like gevent and twisted/tornado.
CONFOO - @RAMISAYAR
Django & WebSockets • Django applications complete HttpResponses. • Django not well suited for WebSocket event handling structure
nor the channel approach in Socket.IO.
CONFOO - @RAMISAYAR
Presenting 3 Methods for WebSockets in Django
CONFOO - @RAMISAYAR
gevent • “gevent is a coroutine-based Python networking library that
uses greenlets to provide a synchronous API on top of the libev event loop.”
• Event loop based on libev (same as Node.js) • Lightweight execution units, a.k.a. “lightweight threads”. • Cooperative sockets with SSL support.
• Similar programming model to Node.js.
CONFOO - @RAMISAYAR
gevent-socketio • gevent-socketio is a pure Python implementation of the
Socket.IO protocol. Basically, it is a port of Socket.IO and typically behind in major releases.
• Designed to work with multiple different WSGI-based web frameworks.
• Handles the “WSGI” problem by providing a custom gevent WSGI server for use with gevent-socketio.
• Only dependencies are gevent. • Imperfect solution: Django is still blocking (monkey
patching is not perfect).
CONFOO - @RAMISAYAR
Demo 1: Django chat engine with gevent-socketio https://github.com/abourget/gevent-socketio/tree/master/examples/django_chat
CONFOO - @RAMISAYAR
“Native” Django Implementation • Previous approach requires significant modifications to your
current Django app serving stack. Native as in you don’t need to know anything besides Django.
• Building an implementation of WebSockets on top of Django. • Identified two problems for WebSockets in Django:
• Problems with WSGI still present EXCEPT in uWSGI. • Networked event handling model is problematic.
• Solution: • Use Nginx which follows – event-driven async architecture. • uWSGI & Gevent for event handling.
CONFOO - @RAMISAYAR
Django-WebSocket-Redis • Module implements websockets on top of Django. • Use Redis as a data store for messaging behind uWSGI and Nginx. • Runs a separate Django main loop in a cooperative concurrency
model using gevent -> 1 thread for all open websockets. • Django can be started as usual with ‘./manage.py runserver’. • Normal requests communication with separate main (websocket)
loop through Redis. • Read:
Offloading Websockets and Server-Sent Events AKA “Combine them with Django safely”
CONFOO - @RAMISAYAR
Demo 2: Django-WebSockets-Redis https://github.com/jrief/django-websocket-redis
CONFOO - @RAMISAYAR
SwampDragon • New project that adds websocket support • Self-publishing models • Django ORM Style Data Filter for Subscription Channels –
Similar to WebSockets + Data Filter • Relies on Redis for messaging + Tornado • Angular JS Support
CONFOO - @RAMISAYAR
Demo 3: Django Swampdragon https://github.com/jonashagstedt/swampdragon
CONFOO - @RAMISAYAR
On Redis • Redis is extremely popular as an easy and simple publish-
subscribe. • Redis is very particular about memory. Running out of memory
in Redis -> CRASH AND BURN! • If you don’t have enough memory in your VM to run Redis
Cache -> Change your approach, use Apache Kafka.
CONFOO - @RAMISAYAR
What did we learn? • The State of WebSockets in Django. • Django and WebSockets is not impossible.
CONFOO - @RAMISAYAR
Thank You! Questions? Follow @ramisayar
CONFOO - @RAMISAYAR
Resources • http://swampdragon.net/ • http://django-websocket-redis.readthedocs.org/en/latest/ • https://github.com/jrief/django-websocket-redis • https://github.com/abourget/gevent-socketio/tree/master/
examples/django_chat • http://jpadilla.com/post/74391616727/the-easiest-way-to-add-
websockets-to-django • http://ferretfarmer.net/2013/09/05/tutorial-real-time-chat-with-
django-twisted-and-websockets-part-1/
CONFOO - @RAMISAYAR
©2013 Microso- Corpora1on. All rights reserved. Microso-, Windows, Office, Azure, System Center, Dynamics and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The informa1on herein is for informa1onal purposes only and represents the current view of Microso- Corpora1on as of the date of this presenta1on. Because Microso- must respond to changing market condi1ons, it should not be interpreted to be a commitment on the part of Microso-, and Microso- cannot guarantee the accuracy of any informa1on provided a-er the date of this presenta1on. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.