Rapid Web Development with Tornado Web and MongoDB
Ikai LanTwitter: @ikai
Friday, June 10, 2011
About the speaker
• Developer Relations at Google
• Software Engineering background
• Lives in San Francisco, CA
• Writes Java, Python on a day to day basis
• Twitter: @ikai
Friday, June 10, 2011
This talk
• Why Tornado Web and MongoDB?
• Whirlwind tour of Tornado (get it?!)
• Intro to MongoDB
• Brief look at a demo app
Friday, June 10, 2011
What I won’t be talking about
• Scalability - loaded topic
• Reliability - another loaded topic
Friday, June 10, 2011
So why tornado/mongo?
Friday, June 10, 2011
Joy.Friday, June 10, 2011
Where does this joy come from?
Friday, June 10, 2011
Freedom.Friday, June 10, 2011
So really, my talk should have been called “The
Joy of Tornado Web and Mongo DB”
Friday, June 10, 2011
Intangibles
• Tornado - fast development, very few rules
• MongoDB - freedom from a predefined schema
• Python, of course. Dynamically typed variables, duck typing plus everything else that is cool about the language
Friday, June 10, 2011
Tornado Web history
• Open Sourced by Facebook
• ... formerly of FriendFeed
• ... former Googlers (that’s why the framework looks like App Engine’s webapp - they wrote it)
• Tornado Web powers FriendFeed
Friday, June 10, 2011
A whirlwind tourFriday, June 10, 2011
Tornado features
• Out of the box auth with Twitter, Facebook, Google and FriendFeed
• Out of box support for localized strings
• Simple templating that allows arbitrary Python code
• Asynchronous requests
• Small codebase
Friday, June 10, 2011
Simple handlersimport tornado.ioloopimport tornado.web
class HelloHandler(tornado.web.RequestHandler): def get(self): self.write("Hello, world :)")
class GoodbyeHandler(tornado.web.RequestHandler): def get(self): self.render("goodbye.html", message=”Goodbye, world”)
application = tornado.web.Application([ (r"/hello", HelloHandler), (r"/goodbye", GoodbyeHandler), ])
if __name__ == "__main__": application.listen(8888) tornado.ioloop.IOLoop.instance().start()
Friday, June 10, 2011
Asynchronous natureclass MainHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous def get(self): http = tornado.httpclient.AsyncHTTPClient() http.fetch("http://friendfeed-api.com/v2/feed/bret", callback=self.on_response)
def on_response(self, response): if response.error: raise tornado.web.HTTPError(500) json = tornado.escape.json_decode(response.body) self.write("Fetched " + str(len(json["entries"])) + " entries " "from the FriendFeed API") self.finish()
Friday, June 10, 2011
Unrestrictive templating {% for student in [p for p in people if p.student and p.age > 23] %} <li>{{ escape(student.name) }}</li> {% end %}
# Sample of arbitrary functions in templates def add(x, y): return x + y template.execute(add=add)
### The template {{ add(1, 2) }}
Friday, June 10, 2011
Lies, damn lies and benchmarks
Friday, June 10, 2011
Saving dataFriday, June 10, 2011
MongoDB is an object database
• No schema - anything that can be a JSON object can be stored
• You can define new collections on the fly
Friday, June 10, 2011
Getting started with Mongo in 4 steps
• Download a binary for your system
• Create a data directory (mkdir -p /data/db)
• Run mongod
• Install pymongo
• ... start writing code!
Friday, June 10, 2011
Pymongo code sampleconnection = pymongo.Connection()database = connection["your_database"]
def get_or_create_location_by_id(location_id): """ Attempts to fetch location data from database. If it doesn't exist, create it. Note that this is NOT concurrency safe. """ location_data = database["locations"].find_one({ "_id" : location_id }) if location_data is None: location_data = { "_id" : location_id, "guards": [], "owner" : None, "history" : [], "last_extort_time" : None } database.location.save(location_data, safe=True) return location_data
Friday, June 10, 2011
Pymongo queries# Add Tyson as a guard to every location owned by “Joe Smith”locations = database["locations"].find({ "owner" : "Joe Smith" })for location in locations: location["guards"].append("Tyson") database["locations"].save(location)
# Find everyone who has Tyson as a guardlocations = database["locations"].find({"guards" : "Tyson"})
# Find everyone who has *ONLY* Tyson as a guardlocations = database["locations"].find({"guards" : ["Tyson"]}) # Note []s
# Find everyone who has Tyson as a guard whose owner is “Ikai Lan”locations = database["locations"].find({"guards" : "Tyson", "owner" : "Ikai Lan" })
Friday, June 10, 2011
Mold as you go alongFriday, June 10, 2011
Freedom from
• ... waiting to get started
• ... predefining a schema
• ... worrying about complex data structures that don’t fit well into the SQL box. Anything that is JSON can be stored!
Friday, June 10, 2011
More MongoDB features
• Simple Geospatial indexing
• GridFS - distributed filesystem running on MongoDB
• Out of the box mapreduce
• Out of the box replication, data partitioning
Friday, June 10, 2011
Putting it all together
• MobSquare, a location based game that uses the Facebook API.
• Mashup: Mafia Wars + FourSquare
• https://github.com/ikai/mobsquare-demo
• “Check in” to locations, take them over, extort money and fight other gangs
Friday, June 10, 2011
Why are Tornado/Mongo a fit?
• If I haven’t said it enough yet, they’re fun
• Facebook API performance varies - asynchronous so we can serve requests while waiting
• Highly structured player data
Friday, June 10, 2011
OAuth 2 upgrade flowclass OnLoginHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous def get(self): # Store this somewhere code = self.get_argument("code") access_token_url = ACCESS_TOKEN_URL_TPL + code client = httpclient.AsyncHTTPClient() client.fetch(access_token_url, self.on_fetched_token) def on_fetched_token(self, response): """ Callback inokved when the auth_token is fetched """ matches = ACCESS_TOKEN_REGEX.search(response.body) if matches: access_token = matches.group(1) client = httpclient.AsyncHTTPClient() # lambda is effectively a function factory for us client.fetch(API["profile"] % access_token, lambda response: self.on_profile_fetch(response, access_token)) def on_profile_fetch(self, response, access_token): """ Callback invoked when we have fetched the user's profile """ profile = json.loads(response.body) profile["access_token"] = access_token profile_id = db.save_profile(profile) self.set_secure_cookie("user_id", str(profile_id)) self.redirect("/") # implictly calls self.finish()
Friday, June 10, 2011
Known gotchas
• Immaturity of frameworks, tools
• Tornado templating errors result in somewhat useless stack traces
• MongoDB nuances
• Tornado community fairly small
Friday, June 10, 2011
Questions?
• Ikai Lan - @ikai on Twitter
• Github: https://github.com/ikai
• Google Profile: https://profiles.google.com/ikai.lan/about
• Thank you for having me at Pycon!
Friday, June 10, 2011
Attributions
• Slide 6: Kirstin Jennings - “big smile!” http://www.flickr.com/photos/methyl_lives/2973265796/
• Slide 8: Tony Blay - “I am a bird now!” http://www.flickr.com/photos/toniblay/59415205/
• Slide 12: Antonio Sofi - “whirlwind” http://www.flickr.com/photos/webgol/36546734
• Slide 17: Tornado Web documentation - http://www.tornadoweb.org/documentation
• Slide 18: Christine of Robot Brainz - “Cassette” http://www.flickr.com/photos/robotbrainz/2786347158/
• Slide 23: Samuel Stroube - “Play-Doh on a Picnic Table” http://www.flickr.com/photos/samoube/203586664
Friday, June 10, 2011
Top Related