Developing RESTful Web APIs with Python, Flask and MongoDB

194
RESTful Web API With Python, Flask and Mongo Nicola Iarocci mercoledì 4 luglio 2012

description

Presented at EuroPython 2012. The abstract: "In the last year we have been working on a full featured, Python powered, RESTful Web API. We learned quite a few things on REST best patterns, and we got a chance to put Python’s renowned web capabilities under review, even releasing a couple Open Source projects in the process. In my talk I will share what we learned. We will consider ‘pure’ REST API design and its many hurdles. We will look at what Python as to offer in this field and finally, we will dig further down by looking at some of the code we developed. Some of the technologies/stacks I’ll cover are (in no particular order): Flask, PyMongo, MongoDB, REST, JSON, XML, Heroku. Did you know? Like it or not, there is going to be a REST API in your future."

Transcript of Developing RESTful Web APIs with Python, Flask and MongoDB

Page 1: Developing RESTful Web APIs with Python, Flask and MongoDB

RESTful Web APIWith Python, Flask and Mongo

Nicola Iaroccimercoledì 4 luglio 2012

Page 2: Developing RESTful Web APIs with Python, Flask and MongoDB

Good Morning.

mercoledì 4 luglio 2012

Page 4: Developing RESTful Web APIs with Python, Flask and MongoDB

Full Disclosure

mercoledì 4 luglio 2012

Page 5: Developing RESTful Web APIs with Python, Flask and MongoDB

I’m a .NET guy20 years in the Microsoft ecosystem

Scary, Yes.

mercoledì 4 luglio 2012

Page 6: Developing RESTful Web APIs with Python, Flask and MongoDB

mercoledì 4 luglio 2012

Page 7: Developing RESTful Web APIs with Python, Flask and MongoDB

Still with me?Great.

mercoledì 4 luglio 2012

Page 9: Developing RESTful Web APIs with Python, Flask and MongoDB

Your Typical Old School Desktop App...

... now going web & mobilemercoledì 4 luglio 2012

Page 10: Developing RESTful Web APIs with Python, Flask and MongoDB

Enter PythonFlask and Mongo

mercoledì 4 luglio 2012

Page 11: Developing RESTful Web APIs with Python, Flask and MongoDB

RESTSo What Is REST All About?

mercoledì 4 luglio 2012

Page 12: Developing RESTful Web APIs with Python, Flask and MongoDB

REST is not a standard

mercoledì 4 luglio 2012

Page 13: Developing RESTful Web APIs with Python, Flask and MongoDB

REST is not a protocol

mercoledì 4 luglio 2012

Page 14: Developing RESTful Web APIs with Python, Flask and MongoDB

REST is an architectural style

for networked applications

mercoledì 4 luglio 2012

Page 15: Developing RESTful Web APIs with Python, Flask and MongoDB

REST defines a set of

simple principlesloosely followed by most API implementations

mercoledì 4 luglio 2012

Page 16: Developing RESTful Web APIs with Python, Flask and MongoDB

#1resource

the source of a specific information

mercoledì 4 luglio 2012

Page 17: Developing RESTful Web APIs with Python, Flask and MongoDB

A web page is not a resource

rather, the representation of a resource

mercoledì 4 luglio 2012

Page 18: Developing RESTful Web APIs with Python, Flask and MongoDB

#2global

permanent identifierevery resource is uniquely identified

(think a HTTP URI)

mercoledì 4 luglio 2012

Page 19: Developing RESTful Web APIs with Python, Flask and MongoDB

#3standard interface

used to exchange representations of resources(think the HTTP protocol)

mercoledì 4 luglio 2012

Page 20: Developing RESTful Web APIs with Python, Flask and MongoDB

#4set of constraints

separation of concerns, stateless, cacheability, layered system, uniform interface...

we’ll get to

these later

mercoledì 4 luglio 2012

Page 21: Developing RESTful Web APIs with Python, Flask and MongoDB

The World Wide Web is built on REST

and it is meant to be consumed by humans

mercoledì 4 luglio 2012

Page 22: Developing RESTful Web APIs with Python, Flask and MongoDB

RESTful Web APIs are built on REST

and are meant to be consumed by machines

mercoledì 4 luglio 2012

Page 23: Developing RESTful Web APIs with Python, Flask and MongoDB

How I Explained REST to My Wife

by Ryan Tomaykohttp://tomayko.com/writings/rest-to-my-wife

Beginners Reading

mercoledì 4 luglio 2012

Page 24: Developing RESTful Web APIs with Python, Flask and MongoDB

Representational State Transfer (REST)

by Roy Thomas Fieldinghttp://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm

The Real Stuff

mercoledì 4 luglio 2012

Page 25: Developing RESTful Web APIs with Python, Flask and MongoDB

RESTful Web APINuts & Bolts

mercoledì 4 luglio 2012

Page 26: Developing RESTful Web APIs with Python, Flask and MongoDB

The Toolsor why I picked Flask and Mongo

mercoledì 4 luglio 2012

Page 27: Developing RESTful Web APIs with Python, Flask and MongoDB

Flaskweb development, one drop at a time

mercoledì 4 luglio 2012

Page 28: Developing RESTful Web APIs with Python, Flask and MongoDB

Simple & Elegant

from flask import Flaskapp = Flask(__name__)

@app.route("/")def hello(): return "Hello World!"

if __name__ == "__main__": app.run(debug=True)

mercoledì 4 luglio 2012

Page 29: Developing RESTful Web APIs with Python, Flask and MongoDB

RESTful request [email protected]('/user/<username>')def show_user_profile(username): return 'User %s' % username

@app.route('/post/<int:post_id>')def show_post(post_id): return 'Post %d' % post_id

mercoledì 4 luglio 2012

Page 30: Developing RESTful Web APIs with Python, Flask and MongoDB

from flask import Flaskapp = Flask(__name__)

@app.route("/")def hello(): return "Hello World!"

if __name__ == "__main__": app.run(debug=True)

Built-in development server & debugger

mercoledì 4 luglio 2012

Page 31: Developing RESTful Web APIs with Python, Flask and MongoDB

from flask import Flaskapp = Flask(__name__)

@app.route("/")def hello(): return "Hello World!"

if __name__ == "__main__": app.run(debug=True)

Explicit & passable application objects

mercoledì 4 luglio 2012

Page 32: Developing RESTful Web APIs with Python, Flask and MongoDB

from flask import Flaskapp = Flask(__name__)

@app.route("/")def hello(): return "Hello World!"

if __name__ == "__main__": app.run(debug=True)

100% WSGI Compliante.g., response objects are WSGI applications themselves

mercoledì 4 luglio 2012

Page 33: Developing RESTful Web APIs with Python, Flask and MongoDB

Only 800 lines of source code

Minimal Footprint

mercoledì 4 luglio 2012

Page 34: Developing RESTful Web APIs with Python, Flask and MongoDB

1500 lines of tests

Heavily Tested

mercoledì 4 luglio 2012

Page 35: Developing RESTful Web APIs with Python, Flask and MongoDB

one day I will make good use of it

Unittesting Support

mercoledì 4 luglio 2012

Page 36: Developing RESTful Web APIs with Python, Flask and MongoDB

we aim for flexibility

Bring Your Own Batteries

mercoledì 4 luglio 2012

Page 37: Developing RESTful Web APIs with Python, Flask and MongoDB

No built-in ORMwe want to be as close to the bare metal as possible

mercoledì 4 luglio 2012

Page 38: Developing RESTful Web APIs with Python, Flask and MongoDB

No form validationwe don’t need no freaking form validation

mercoledì 4 luglio 2012

Page 39: Developing RESTful Web APIs with Python, Flask and MongoDB

Python offers great tools to manipulate JSON,we can tinker something ourselves

No data validation

mercoledì 4 luglio 2012

Page 40: Developing RESTful Web APIs with Python, Flask and MongoDB

built on Werkzeug, Jinja2, WSGI

Layered API

mercoledì 4 luglio 2012

Page 41: Developing RESTful Web APIs with Python, Flask and MongoDB

The Pocoo Team did Werkzeug, Jinja2, Sphinx, Pygments, and much more

Built by the Pros

mercoledì 4 luglio 2012

Page 42: Developing RESTful Web APIs with Python, Flask and MongoDB

Over 200 pages, lots of examples and howtos

Excellent Documentation

mercoledì 4 luglio 2012

Page 43: Developing RESTful Web APIs with Python, Flask and MongoDB

Widely adopted, extensions for everything

Active Community

mercoledì 4 luglio 2012

Page 44: Developing RESTful Web APIs with Python, Flask and MongoDB

Kenneth Reitz, DjangoCon 2012

“Flask is a sharp tool for building sharp

services”

mercoledì 4 luglio 2012

Page 46: Developing RESTful Web APIs with Python, Flask and MongoDB

made NoSQL easy to grasp (even for a dumbhead like me)

Similarity with RDBMS

mercoledì 4 luglio 2012

Page 47: Developing RESTful Web APIs with Python, Flask and MongoDB

Terminology

RDBMS Mongo

Database Database

Table Collection

Rows(s) JSON Document

Index Index

Join Embedding & Linking

mercoledì 4 luglio 2012

Page 48: Developing RESTful Web APIs with Python, Flask and MongoDB

true selling point for me

JSON-style data store

mercoledì 4 luglio 2012

Page 49: Developing RESTful Web APIs with Python, Flask and MongoDB

JSON & RESTful API

JSONaccepted media type

Client

JSON(BSON)

Mongo

GET

maybe we can push directly to client?

mercoledì 4 luglio 2012

Page 50: Developing RESTful Web APIs with Python, Flask and MongoDB

JSON & RESTful API

JSONaccepted media type

Client

JSON(BSON)

Mongo

JSON/dictmaps to python dict

API

GET

almost.

mercoledì 4 luglio 2012

Page 51: Developing RESTful Web APIs with Python, Flask and MongoDB

JSON & RESTful API

JSONobjects

Client

JSON(BSON)

Mongo

JSON/dictmaps to python dict

(validation layer)

API

POST

also works when posting (adding) items to the database

mercoledì 4 luglio 2012

Page 52: Developing RESTful Web APIs with Python, Flask and MongoDB

Queries in MongoDB are represented as JSON-style objects

What about Queries?

// select * from things where x=3 and y="foo"db.things.find({x: 3, y: "foo”});

mercoledì 4 luglio 2012

Page 53: Developing RESTful Web APIs with Python, Flask and MongoDB

JSON & RESTful API

nativeMongoquery syntax

Client

JSON(BSON)

Mongo

(very) thin parsing

& validation layer

API

FILTERING & SORTING

?where={x: 3, y: "foo”}

mercoledì 4 luglio 2012

Page 54: Developing RESTful Web APIs with Python, Flask and MongoDB

mapping to and from the database feels more natural

JSON all along the pipeline

mercoledì 4 luglio 2012

Page 55: Developing RESTful Web APIs with Python, Flask and MongoDB

dynamic objects allow for a painless evolution of our schema (because yes, a schema exists at any point in time)

Schema-less

mercoledì 4 luglio 2012

Page 56: Developing RESTful Web APIs with Python, Flask and MongoDB

Where we’re going we don’t need ORMs.

ORM

mercoledì 4 luglio 2012

Page 57: Developing RESTful Web APIs with Python, Flask and MongoDB

official Python driver all we need to interact with the database

PyMongo

mercoledì 4 luglio 2012

Page 58: Developing RESTful Web APIs with Python, Flask and MongoDB

Also in MongoDB

• setup is a breeze

• lightweight

• fast inserts, updates and queries

• excellent documentation

• great support by 10gen

• great community

mercoledì 4 luglio 2012

Page 59: Developing RESTful Web APIs with Python, Flask and MongoDB

The Little MongoDB Book

by Karl Seguinhttp://openmymind.net/2011/3/28/The-Little-MongoDB-Book/

A Great Introduction To MongoDB

mercoledì 4 luglio 2012

Page 60: Developing RESTful Web APIs with Python, Flask and MongoDB

Il Piccolo Libro di MongoDB

by Karl Seguin, traduzione di

Nicola Iaroccihttp://nicolaiarocci.com/il-piccolo-libro-di-mongodb-edizione-italiana/

Shameless Plug

mercoledì 4 luglio 2012

Page 61: Developing RESTful Web APIs with Python, Flask and MongoDB

MongoDB Interactive Tutorial

http://tutorial.mongly.com/tutorial/index

mercoledì 4 luglio 2012

Page 62: Developing RESTful Web APIs with Python, Flask and MongoDB

RESTful Web APIs are really just

collection of resourcesaccesible through to a uniform interface

mercoledì 4 luglio 2012

Page 63: Developing RESTful Web APIs with Python, Flask and MongoDB

#1each resource is identified by a

persistent identifierWe need to properly implement Request Dispatching

mercoledì 4 luglio 2012

Page 64: Developing RESTful Web APIs with Python, Flask and MongoDB

CollectionsAPI’s entry point + plural nouns

http://api.example.com/v1/contacts

mercoledì 4 luglio 2012

Page 65: Developing RESTful Web APIs with Python, Flask and MongoDB

CollectionsFlask URL dispatcher allows for variables

@app.route('/<collection>')def collection(collection): if collection in DOMAIN.keys(): (...) abort(404)

api.example.com/contactsapi.example.com/invoices

etc.

mercoledì 4 luglio 2012

Page 66: Developing RESTful Web APIs with Python, Flask and MongoDB

validation dictonary

CollectionsFlask URL dispatcher allows for variables

@app.route('/<collection>')def collection(collection): if collection in DOMAIN.keys(): (...) abort(404)

mercoledì 4 luglio 2012

Page 67: Developing RESTful Web APIs with Python, Flask and MongoDB

we don’t know this collection,

return a 404

CollectionsFlask URL dispatcher allows for variables

@app.route('/<collection>')def collection(collection): if collection in DOMAIN.keys(): (...) abort(404)

mercoledì 4 luglio 2012

Page 68: Developing RESTful Web APIs with Python, Flask and MongoDB

@app.route('/<regex("[\w]*[Ss]"):collection>')def collection(collection): if collection in DOMAIN.keys(): (...) abort(404)

regular expressions can be used to better narrow a

variable part URL.However...

RegExby design, collection URLs are plural nouns

mercoledì 4 luglio 2012

Page 69: Developing RESTful Web APIs with Python, Flask and MongoDB

RegExWe need to build our own Custom Converter

class RegexConverter(BaseConverter): def __init__(self, url_map, *items): super(RegexConverter, self).__init__(url_map) self.regex = items[0]

app.url_map.converters['regex'] = RegexConverter

subclass BaseConverter and pass the new converter to

the url_map

mercoledì 4 luglio 2012

Page 70: Developing RESTful Web APIs with Python, Flask and MongoDB

And eventually by an alternative lookup value

Document

http://api.example.com/v1/contacts/CUST12345

Documents are identified by ObjectIDhttp://api.example.com/v1/contacts/4f46445fc88e201858000000

mercoledì 4 luglio 2012

Page 71: Developing RESTful Web APIs with Python, Flask and MongoDB

@app.route('/<regex("[\w]*[Ss]"):collection>/<lookup>')@app.route('/<regex("[\w]*[Ss]"):collection>' '/<regex("[a-f0-9]{24}"):object_id>')def document(collection, lookup=None, object_id=None): (...)

URL dispatcher handles multiple variables

http://api.example.com/v1/contacts/CUST12345

Document

mercoledì 4 luglio 2012

Page 72: Developing RESTful Web APIs with Python, Flask and MongoDB

Document

and of course it also handles multiple RegEx variables

http://api.example.com/v1/contacts/4f46445fc88e201858000000

@app.route('/<regex("[\w]*[Ss]"):collection>/<lookup>')@app.route('/<regex("[\w]*[Ss]"):collection>' '/<regex("[a-f0-9]{24}"):object_id>')def document(collection, lookup=None, object_id=None): (...)

mercoledì 4 luglio 2012

Page 73: Developing RESTful Web APIs with Python, Flask and MongoDB

Document

Different URLs can be dispatched to the same function just by piling up

@app.route decorators.

@app.route('/<regex("[\w]*[Ss]"):collection>/<lookup>')@app.route('/<regex("[\w]*[Ss]"):collection>' '/<regex("[a-f0-9]{24}"):object_id>')def document(collection, lookup=None, object_id=None): (...)

mercoledì 4 luglio 2012

Page 74: Developing RESTful Web APIs with Python, Flask and MongoDB

#2representation of

resources via media typesJSON, XML or any other valid internet media type

depends on the

request and not

the identifier

mercoledì 4 luglio 2012

Page 75: Developing RESTful Web APIs with Python, Flask and MongoDB

Accepted Media Typesmapping supported media types tocorresponding renderer functions

mime_types = {'json_renderer': ('application/json',), 'xml_renderer': ('application/xml', 'text/xml', 'application/x-xml',)}

JSON rendering function

mercoledì 4 luglio 2012

Page 76: Developing RESTful Web APIs with Python, Flask and MongoDB

Accepted Media Types

mime_types = {'json_renderer': ('application/json',), 'xml_renderer': ('application/xml', 'text/xml', 'application/x-xml',)}

corresponding JSON internet media type

mapping supported media types tocorresponding renderer functions

mercoledì 4 luglio 2012

Page 77: Developing RESTful Web APIs with Python, Flask and MongoDB

Accepted Media Types

mime_types = {'json_renderer': ('application/json',), 'xml_renderer': ('application/xml', 'text/xml', 'application/x-xml',)}

XML rendering function

mapping supported media types tocorresponding renderer functions

mercoledì 4 luglio 2012

Page 78: Developing RESTful Web APIs with Python, Flask and MongoDB

Accepted Media Types

mime_types = {'json_renderer': ('application/json',), 'xml_renderer': ('application/xml', 'text/xml', 'application/x-xml',)}

corresponding XML internet media types

mapping supported media types tocorresponding renderer functions

mercoledì 4 luglio 2012

Page 79: Developing RESTful Web APIs with Python, Flask and MongoDB

JSON Renderdatetimes and ObjectIDs call for further tinkering

renderer function mapped to the appication/json

media type

class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj)

def json_renderer(**data): return json.dumps(data, cls=APIEncoder)

mercoledì 4 luglio 2012

Page 80: Developing RESTful Web APIs with Python, Flask and MongoDB

JSON Renderdatetimes and ObjectIDs call for further tinkering

standard json encoding is not enough, we need a

specialized JSONEncoder

class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj)

def json_renderer(**data): return json.dumps(data, cls=APIEncoder)

mercoledì 4 luglio 2012

Page 81: Developing RESTful Web APIs with Python, Flask and MongoDB

class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj)

def json_renderer(**data): return json.dumps(data, cls=APIEncoder)

JSON Render

Python datetimes are encoded as RFC 1123 strings: “Wed, 06 Jun 2012 14:19:53 UTC”

datetimes and ObjectIDs call for further tinkering

mercoledì 4 luglio 2012

Page 82: Developing RESTful Web APIs with Python, Flask and MongoDB

JSON Render

class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj)

def json_renderer(**data): return json.dumps(data, cls=APIEncoder)

Mongo ObjectId data types are encoded as strings: “4f46445fc88e201858000000”

datetimes and ObjectIDs call for further tinkering

mercoledì 4 luglio 2012

Page 83: Developing RESTful Web APIs with Python, Flask and MongoDB

JSON Render

we let json/simplejson handle the other data types

class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj)

def json_renderer(**data): return json.dumps(data, cls=APIEncoder)

datetimes and ObjectIDs call for further tinkering

mercoledì 4 luglio 2012

Page 84: Developing RESTful Web APIs with Python, Flask and MongoDB

RenderingRender to JSON or XML and get WSGI response object

best match between request Accept header

and media types supported by the

service

def prep_response(dct, status=200): mime, render = get_best_mime() rendered = globals()[render](**dct) resp = make_response(rendered, status) resp.mimetype = mime return resp

mercoledì 4 luglio 2012

Page 85: Developing RESTful Web APIs with Python, Flask and MongoDB

RenderingRender to JSON or XML and get WSGI response object

call the appropriate render function and retrieve the encoded

JSON or XML

def prep_response(dct, status=200): mime, render = get_best_mime() rendered = globals()[render](**dct) resp = make_response(rendered, status) resp.mimetype = mime return resp

mercoledì 4 luglio 2012

Page 86: Developing RESTful Web APIs with Python, Flask and MongoDB

RenderingRender to JSON or XML and get WSGI response object

flask’s make_response() returns a WSGI response object wich we can use

to attach headers

def prep_response(dct, status=200): mime, render = get_best_mime() rendered = globals()[render](**dct) resp = make_response(rendered, status) resp.mimetype = mime return resp

mercoledì 4 luglio 2012

Page 87: Developing RESTful Web APIs with Python, Flask and MongoDB

RenderingRender to JSON or XML and get WSGI response object

and finally, we set the appropriate mime type in the response header

def prep_response(dct, status=200): mime, render = get_best_mime() rendered = globals()[render](**dct) resp = make_response(rendered, status) resp.mimetype = mime return resp

mercoledì 4 luglio 2012

Page 88: Developing RESTful Web APIs with Python, Flask and MongoDB

Flask-MimeRender“Python module for RESTful resource representation using

MIME Media-Types and the Flask Microframework”

!"!#"$%&'((#)('%*+,",-.-$/-.

mercoledì 4 luglio 2012

Page 89: Developing RESTful Web APIs with Python, Flask and MongoDB

Flask-MimeRenderRender Functions

render_json = jsonifyrender_xml = lambda message: '<message>%s</message>' % messagerender_txt = lambda message: messagerender_html = lambda message: '<html><body>%s</body></html>' % \

message

mercoledì 4 luglio 2012

Page 90: Developing RESTful Web APIs with Python, Flask and MongoDB

Flask-MimeRenderthen you just decorate your end-point function

@app.route('/')@mimerender( default = 'html', html = render_html, xml = render_xml, json = render_json, txt = render_txt)def index(): if request.method == 'GET': return {'message': 'Hello, World!'}

mercoledì 4 luglio 2012

Page 91: Developing RESTful Web APIs with Python, Flask and MongoDB

Flask-MimeRenderRequests

$ curl -H "Accept: application/html" example.com/<html><body>Hello, World!</body></html>

$ curl -H "Accept: application/xml" example.com/<message>Hello, World!</message>

$ curl -H "Accept: application/json" example.com/{'message':'Hello, World!'}

$ curl -H "Accept: text/plain" example.com/Hello, World!

mercoledì 4 luglio 2012

Page 92: Developing RESTful Web APIs with Python, Flask and MongoDB

“GET, POST, PUT, DELETE and all that mess”

#3resource manipulation through HTTP verbs

mercoledì 4 luglio 2012

Page 93: Developing RESTful Web APIs with Python, Flask and MongoDB

HTTP MethodsVerbs are handled along with URL routing

@app.route('/<collection>', methods=['GET', 'POST'])def collection(collection): if collection in DOMAIN.keys(): if request.method == 'GET': return get_collection(collection) elif request.method == 'POST': return post(collection) abort(404)

accepted HTTP verbsa PUT will throw a

405 Command Not Allowed

mercoledì 4 luglio 2012

Page 94: Developing RESTful Web APIs with Python, Flask and MongoDB

@app.route('/<collection>', methods=['GET', 'POST'])def collection(collection): if collection in DOMAIN.keys(): if request.method == 'GET': return get_collection(collection) elif request.method == 'POST': return post(collection) abort(404)

HTTP MethodsVerbs are handled along with URL routing

the global request object provides access to clients’

request headers

mercoledì 4 luglio 2012

Page 95: Developing RESTful Web APIs with Python, Flask and MongoDB

@app.route('/<collection>', methods=['GET', 'POST'])def collection(collection): if collection in DOMAIN.keys(): if request.method == 'GET': return get_collection(collection) elif request.method == 'POST': return post(collection) abort(404)

HTTP MethodsVerbs are handled along with URL routing

we respond to a GET request for a ‘collection’ resource

mercoledì 4 luglio 2012

Page 96: Developing RESTful Web APIs with Python, Flask and MongoDB

@app.route('/<collection>', methods=['GET', 'POST'])def collection(collection): if collection in DOMAIN.keys(): if request.method == 'GET': return get_collection(collection) elif request.method == 'POST': return post(collection) abort(404)

HTTP MethodsVerbs are handled along with URL routing

and here we respond to a POST request. Handling HTTP

methods is easy!

mercoledì 4 luglio 2012

Page 97: Developing RESTful Web APIs with Python, Flask and MongoDB

CRUD via REST

Acttion HTTP Verb Context

Get GET Collection/Document

Create POST Collection

Update PATCH* Document

Delete DELETE Document

* WTF?

mercoledì 4 luglio 2012

Page 98: Developing RESTful Web APIs with Python, Flask and MongoDB

Retrieve Multiple Documents (accepting Queries)http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}}

GET

mercoledì 4 luglio 2012

Page 99: Developing RESTful Web APIs with Python, Flask and MongoDB

Collection GET

def get_collection(collection): where = request.args.get('where') if where: args['spec'] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = []

cursor = db(collection).find(**args) for document in cursor: documents.append(document)

response[collection] = documents return prep_response(response)

request.args returns the original URI’s query

definition, in our example:

where = {“age”: {“$gt”: 20}}

http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}}

mercoledì 4 luglio 2012

Page 100: Developing RESTful Web APIs with Python, Flask and MongoDB

Collection GET

def get_collection(collection): where = request.args.get('where') if where: args['spec'] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = []

cursor = db(collection).find(**args) for document in cursor: documents.append(document)

response[collection] = documents return prep_response(response)

as the query already comes in as a Mongo expression:

{“age”: {“$gt”: 20}}

we simply convert it to JSON.

http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}}

mercoledì 4 luglio 2012

Page 101: Developing RESTful Web APIs with Python, Flask and MongoDB

Collection GET

def get_collection(collection): where = request.args.get('where') if where: args['spec'] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = []

cursor = db(collection).find(**args) for document in cursor: documents.append(document)

response[collection] = documents return prep_response(response) String-to-datetime

conversion is obtained via the object_hook mechanism

http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}}

mercoledì 4 luglio 2012

Page 102: Developing RESTful Web APIs with Python, Flask and MongoDB

Collection GET

def get_collection(collection): where = request.args.get('where') if where: args['spec'] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = []

cursor = db(collection).find(**args) for document in cursor: documents.append(document)

response[collection] = documents return prep_response(response)

find() accepts a python dict as query expression, and returns a cursor we can

iterate

http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}}

mercoledì 4 luglio 2012

Page 103: Developing RESTful Web APIs with Python, Flask and MongoDB

Collection GET

def get_collection(collection): where = request.args.get('where') if where: args['spec'] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = []

cursor = db(collection).find(**args) for document in cursor: documents.append(document)

response[collection] = documents return prep_response(response) finally, we encode the

response dict with the requested MIME media-type

http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}}

mercoledì 4 luglio 2012

Page 104: Developing RESTful Web APIs with Python, Flask and MongoDB

On encoding JSON dates

Interlude

mercoledì 4 luglio 2012

Page 105: Developing RESTful Web APIs with Python, Flask and MongoDB

On encoding JSON dates

• We don’t want to force metadata into JSON representation:(“updated”: “$date: Thu 1, ..”)

• Likewise, epochs are not an option

• We are aiming for a broad solution not relying on the knoweldge of the current domain

mercoledì 4 luglio 2012

Page 106: Developing RESTful Web APIs with Python, Flask and MongoDB

Because, you knowthe guy

behind

Redis

mercoledì 4 luglio 2012

Page 107: Developing RESTful Web APIs with Python, Flask and MongoDB

Parsing JSON dates

object_hook is usually used to deserialize JSON to

classes (rings a ORM bell?)

>>> source = '{"updated": "Thu, 1 Mar 2012 10:00:49 UTC"}'>>> dct = json.loads(source, object_hook=datetime_parser)>>> dct{u'updated': datetime.datetime(2012, 3, 1, 10, 0, 49)}

def datetime_parser(dct):    for k, v in dct.items():        if isinstance(v, basestring) and re.search("\ UTC", v):            try:                dct[k] = datetime.datetime.strptime(v, DATE_FORMAT)            except:                pass    return dct

this is what I came out with

mercoledì 4 luglio 2012

Page 108: Developing RESTful Web APIs with Python, Flask and MongoDB

Parsing JSON dates

the resulting dct now has datetime values instead of string representations of

dates

>>> source = '{"updated": "Thu, 1 Mar 2012 10:00:49 UTC"}'>>> dct = json.loads(source, object_hook=datetime_parser)>>> dct{u'updated': datetime.datetime(2012, 3, 1, 10, 0, 49)}

def datetime_parser(dct):    for k, v in dct.items():        if isinstance(v, basestring) and re.search("\ UTC", v):            try:                dct[k] = datetime.datetime.strptime(v, DATE_FORMAT)            except:                pass    return dct

this is what I came out with

mercoledì 4 luglio 2012

Page 109: Developing RESTful Web APIs with Python, Flask and MongoDB

Parsing JSON dates

the function receives a dict representing the decoded

JSON

>>> source = '{"updated": "Thu, 1 Mar 2012 10:00:49 UTC"}'>>> dct = json.loads(source, object_hook=datetime_parser)>>> dct{u'updated': datetime.datetime(2012, 3, 1, 10, 0, 49)}

def datetime_parser(dct):    for k, v in dct.items():        if isinstance(v, basestring) and re.search("\ UTC", v):            try:                dct[k] = datetime.datetime.strptime(v, DATE_FORMAT)            except:                pass    return dct

this is what I came out with

mercoledì 4 luglio 2012

Page 110: Developing RESTful Web APIs with Python, Flask and MongoDB

Parsing JSON dates

strings matching the RegEx (which probably should be

better defined)...

>>> source = '{"updated": "Thu, 1 Mar 2012 10:00:49 UTC"}'>>> dct = json.loads(source, object_hook=datetime_parser)>>> dct{u'updated': datetime.datetime(2012, 3, 1, 10, 0, 49)}

def datetime_parser(dct):    for k, v in dct.items():        if isinstance(v, basestring) and re.search("\ UTC", v):            try:                dct[k] = datetime.datetime.strptime(v, DATE_FORMAT)            except:                pass    return dct

this is what I came out with

mercoledì 4 luglio 2012

Page 111: Developing RESTful Web APIs with Python, Flask and MongoDB

Parsing JSON dates

...are converted to datetime values

>>> source = '{"updated": "Thu, 1 Mar 2012 10:00:49 UTC"}'>>> dct = json.loads(source, object_hook=datetime_parser)>>> dct{u'updated': datetime.datetime(2012, 3, 1, 10, 0, 49)}

def datetime_parser(dct):    for k, v in dct.items():        if isinstance(v, basestring) and re.search("\ UTC", v):            try:                dct[k] = datetime.datetime.strptime(v, DATE_FORMAT)            except:                pass    return dct

this is what I came out with

mercoledì 4 luglio 2012

Page 112: Developing RESTful Web APIs with Python, Flask and MongoDB

Parsing JSON dates

if conversion fails we assume that we are dealing a

normal, legit string

>>> source = '{"updated": "Thu, 1 Mar 2012 10:00:49 UTC"}'>>> dct = json.loads(source, object_hook=datetime_parser)>>> dct{u'updated': datetime.datetime(2012, 3, 1, 10, 0, 49)}

def datetime_parser(dct):    for k, v in dct.items():        if isinstance(v, basestring) and re.search("\ UTC", v):            try:                dct[k] = datetime.datetime.strptime(v, DATE_FORMAT)            except:                pass    return dct

this is what I came out with

mercoledì 4 luglio 2012

Page 113: Developing RESTful Web APIs with Python, Flask and MongoDB

Editing a Resource

PATCH

mercoledì 4 luglio 2012

Page 114: Developing RESTful Web APIs with Python, Flask and MongoDB

Why not PUT?

• PUT means resource creation or replacement at a given URL

• PUT does not allow for partial updates of a resource

• 99% of the time we are updating just one or two fields

• We don’t want to send complete representations of the document we are updating

• Mongo allows for atomic updates and we want to take advantage of that

mercoledì 4 luglio 2012

Page 115: Developing RESTful Web APIs with Python, Flask and MongoDB

‘atomic’ PUT updates are ok when each field

is itself a resourcehttp://api.example.com/v1/contacts/<id>/address

mercoledì 4 luglio 2012

Page 116: Developing RESTful Web APIs with Python, Flask and MongoDB

Enter PATCH“This specification defines the new method, PATCH, which

is used to apply partial modifications to a resource.”

RFC5789

mercoledì 4 luglio 2012

Page 117: Developing RESTful Web APIs with Python, Flask and MongoDB

PATCH

• send a “patch document” with just the changes to be applied to the document

• saves bandwidth and reduces traffic

• it’s been around since 1995

• it is a RFC Proposed Standard

• Widely adopted (will replace PUT in Rails 4.0)

• clients not supporting it can fallback to POST with ‘X-HTTP-Method-Override: PATCH’ header tag

mercoledì 4 luglio 2012

Page 118: Developing RESTful Web APIs with Python, Flask and MongoDB

PATCHingdef patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400)

key, value = docs.popitem()

response_item = {} object_id = original[ID_FIELD]

# Validation validate(value, collection, object_id) response_item['validation'] = value['validation']

if value['validation']['response'] != VALIDATION_ERROR: # Perform the update updates = {"$set": value['doc']} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)

request.form returns a dict with request form data.

mercoledì 4 luglio 2012

Page 119: Developing RESTful Web APIs with Python, Flask and MongoDB

PATCHingdef patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400)

key, value = docs.popitem()

response_item = {} object_id = original[ID_FIELD]

# Validation validate(value, collection, object_id) response_item['validation'] = value['validation']

if value['validation']['response'] != VALIDATION_ERROR: # Perform the update updates = {"$set": value['doc']} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)

we aren’t going to accept more than one document here

mercoledì 4 luglio 2012

Page 120: Developing RESTful Web APIs with Python, Flask and MongoDB

PATCHingdef patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400)

key, value = docs.popitem()

response_item = {} object_id = original[ID_FIELD]

# Validation validate(value, collection, object_id) response_item['validation'] = value['validation']

if value['validation']['response'] != VALIDATION_ERROR: # Perform the update updates = {"$set": value['doc']} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)

retrieve the original document ID, will be used by

the update command

mercoledì 4 luglio 2012

Page 121: Developing RESTful Web APIs with Python, Flask and MongoDB

PATCHingdef patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400)

key, value = docs.popitem()

response_item = {} object_id = original[ID_FIELD]

# Validation validate(value, collection, object_id) response_item['validation'] = value['validation']

if value['validation']['response'] != VALIDATION_ERROR: # Perform the update updates = {"$set": value['doc']} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)

validate the updates

mercoledì 4 luglio 2012

Page 122: Developing RESTful Web APIs with Python, Flask and MongoDB

PATCHingdef patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400)

key, value = docs.popitem()

response_item = {} object_id = original[ID_FIELD]

# Validation validate(value, collection, object_id) response_item['validation'] = value['validation']

if value['validation']['response'] != VALIDATION_ERROR: # Perform the update updates = {"$set": value['doc']} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)

add validation results to the response dictionary

mercoledì 4 luglio 2012

Page 123: Developing RESTful Web APIs with Python, Flask and MongoDB

PATCHingdef patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400)

key, value = docs.popitem()

response_item = {} object_id = original[ID_FIELD]

# Validation validate(value, collection, object_id) response_item['validation'] = value['validation']

if value['validation']['response'] != VALIDATION_ERROR: # Perform the update updates = {"$set": value['doc']} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)

$set accepts a dictwith the updates for the db

eg: {“active”: False}.

mercoledì 4 luglio 2012

Page 124: Developing RESTful Web APIs with Python, Flask and MongoDB

PATCHingdef patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400)

key, value = docs.popitem()

response_item = {} object_id = original[ID_FIELD]

# Validation validate(value, collection, object_id) response_item['validation'] = value['validation']

if value['validation']['response'] != VALIDATION_ERROR: # Perform the update updates = {"$set": value['doc']} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)

mongo update() method commits updates to the database. Updates are

atomic.

mercoledì 4 luglio 2012

Page 125: Developing RESTful Web APIs with Python, Flask and MongoDB

PATCHingdef patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400)

key, value = docs.popitem()

response_item = {} object_id = original[ID_FIELD]

# Validation validate(value, collection, object_id) response_item['validation'] = value['validation']

if value['validation']['response'] != VALIDATION_ERROR: # Perform the update updates = {"$set": value['doc']} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)

udpate() takes the unique Id of the document andthe

update expression ($set)

mercoledì 4 luglio 2012

Page 126: Developing RESTful Web APIs with Python, Flask and MongoDB

PATCHingdef patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400)

key, value = docs.popitem()

response_item = {} object_id = original[ID_FIELD]

# Validation validate(value, collection, object_id) response_item['validation'] = value['validation']

if value['validation']['response'] != VALIDATION_ERROR: # Perform the update updates = {"$set": value['doc']} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)

as always, our response dictionary is returned with

proper encding

mercoledì 4 luglio 2012

Page 127: Developing RESTful Web APIs with Python, Flask and MongoDB

Creating Resources

POST

mercoledì 4 luglio 2012

Page 128: Developing RESTful Web APIs with Python, Flask and MongoDB

POSTing

we accept multiple documents (remember, we are at

collection level here)

def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item['validation']['response'] != VALIDATION_ERROR: document = item['doc'] response_item[ID_FIELD] = db(collection).insert(document) response_item['link'] = get_document_link(collection, response_item[ID_FIELD]) response_item['validation'] = item['validation'] response[key] = response_item return {'response': response}

mercoledì 4 luglio 2012

Page 129: Developing RESTful Web APIs with Python, Flask and MongoDB

def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item['validation']['response'] != VALIDATION_ERROR: document = item['doc'] response_item[ID_FIELD] = db(collection).insert(document) response_item['link'] = get_document_link(collection, response_item[ID_FIELD]) response_item['validation'] = item['validation'] response[key] = response_item return {'response': response}

POSTing

we loop through the documents to be inserted

mercoledì 4 luglio 2012

Page 130: Developing RESTful Web APIs with Python, Flask and MongoDB

POSTing

perform validation on the document

def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item['validation']['response'] != VALIDATION_ERROR: document = item['doc'] response_item[ID_FIELD] = db(collection).insert(document) response_item['link'] = get_document_link(collection, response_item[ID_FIELD]) response_item['validation'] = item['validation'] response[key] = response_item return {'response': response}

mercoledì 4 luglio 2012

Page 131: Developing RESTful Web APIs with Python, Flask and MongoDB

POSTing

push document and get its ObjectId back from Mongo.

like other CRUD operations, inserting is trivial in

mongo.

def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item['validation']['response'] != VALIDATION_ERROR: document = item['doc'] response_item[ID_FIELD] = db(collection).insert(document) response_item['link'] = get_document_link(collection, response_item[ID_FIELD]) response_item['validation'] = item['validation'] response[key] = response_item return {'response': response}

mercoledì 4 luglio 2012

Page 132: Developing RESTful Web APIs with Python, Flask and MongoDB

POSTing

a direct link to the resource we just created is

added to the response

def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item['validation']['response'] != VALIDATION_ERROR: document = item['doc'] response_item[ID_FIELD] = db(collection).insert(document) response_item['link'] = get_document_link(collection, response_item[ID_FIELD]) response_item['validation'] = item['validation'] response[key] = response_item return {'response': response}

mercoledì 4 luglio 2012

Page 133: Developing RESTful Web APIs with Python, Flask and MongoDB

POSTing

validation result is always returned to the client, even

if the doc has not been inserted

def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item['validation']['response'] != VALIDATION_ERROR: document = item['doc'] response_item[ID_FIELD] = db(collection).insert(document) response_item['link'] = get_document_link(collection, response_item[ID_FIELD]) response_item['validation'] = item['validation'] response[key] = response_item return {'response': response}

mercoledì 4 luglio 2012

Page 134: Developing RESTful Web APIs with Python, Flask and MongoDB

POSTing

standard response enconding applied

def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item['validation']['response'] != VALIDATION_ERROR: document = item['doc'] response_item[ID_FIELD] = db(collection).insert(document) response_item['link'] = get_document_link(collection, response_item[ID_FIELD]) response_item['validation'] = item['validation'] response[key] = response_item return {'response': response}

mercoledì 4 luglio 2012

Page 135: Developing RESTful Web APIs with Python, Flask and MongoDB

Data ValidationWe still need to validate incoming data

mercoledì 4 luglio 2012

Page 136: Developing RESTful Web APIs with Python, Flask and MongoDB

Data ValidationDOMAIN = {}DOMAIN['contacts'] = { 'secondary_id': 'name', 'fields': { 'name': { 'data_type': 'string', 'required': True, 'unique': True, 'max_length': 120, 'min_length': 1 },

DOMAIN is a Python dict containing our validation rules and schema structure

mercoledì 4 luglio 2012

Page 137: Developing RESTful Web APIs with Python, Flask and MongoDB

Data Validation

every resource (collection) maintained by the API has a

key in DOMAIN

DOMAIN = {}DOMAIN['contacts'] = { 'secondary_id': 'name', 'fields': { 'name': { 'data_type': 'string', 'required': True, 'unique': True, 'max_length': 120, 'min_length': 1 },

mercoledì 4 luglio 2012

Page 138: Developing RESTful Web APIs with Python, Flask and MongoDB

Data Validation

if the resource allows for a secondary lookup field, we

define it here

DOMAIN = {}DOMAIN['contacts'] = { 'secondary_id': 'name', 'fields': { 'name': { 'data_type': 'string', 'required': True, 'unique': True, 'max_length': 120, 'min_length': 1 },

mercoledì 4 luglio 2012

Page 139: Developing RESTful Web APIs with Python, Flask and MongoDB

Data Validation

known fields go in the fields dict

DOMAIN = {}DOMAIN['contacts'] = { 'secondary_id': 'name', 'fields': { 'name': { 'data_type': 'string', 'required': True, 'unique': True, 'max_length': 120, 'min_length': 1 },

mercoledì 4 luglio 2012

Page 140: Developing RESTful Web APIs with Python, Flask and MongoDB

Data Validation

validation rules for ‘name’ field. data_type is mostly needed to process datetimes

and currency values

DOMAIN = {}DOMAIN['contacts'] = { 'secondary_id': 'name', 'fields': { 'name': { 'data_type': 'string', 'required': True, 'unique': True, 'max_length': 120, 'min_length': 1 },

mercoledì 4 luglio 2012

Page 141: Developing RESTful Web APIs with Python, Flask and MongoDB

Data Validation

we can define custom validation functions when

the need arises

(...)'iban': { 'data_type': 'string', 'custom_validation': { 'module': 'customvalidation', 'function': 'validate_iban' }}(...)

mercoledì 4 luglio 2012

Page 142: Developing RESTful Web APIs with Python, Flask and MongoDB

Data Validation

or we can define our own custom data types...

(...)'contact_type': { 'data_type': 'array', 'allowed_values': [ 'client', 'agent', 'supplier', 'area manager', 'vector' ]}(...)

mercoledì 4 luglio 2012

Page 143: Developing RESTful Web APIs with Python, Flask and MongoDB

Data Validation

... like the array, which allows us to define a list of accepted values for the

field

(...)'contact_type': { 'data_type': 'array', 'allowed_values': [ 'client', 'agent', 'supplier', 'area manager', 'vector' ]}(...)

mercoledì 4 luglio 2012

Page 144: Developing RESTful Web APIs with Python, Flask and MongoDB

I will spare you the validation function

It’s pretty simple really

mercoledì 4 luglio 2012

Page 145: Developing RESTful Web APIs with Python, Flask and MongoDB

Hey but!You’re building your own ORM!

Just a thin validation layer on which I have total control

AKASo What?

mercoledì 4 luglio 2012

Page 146: Developing RESTful Web APIs with Python, Flask and MongoDB

#4Caching and

concurrency controlresource representation describes how

when and if it can be used, discarded or re-fetched

mercoledì 4 luglio 2012

Page 147: Developing RESTful Web APIs with Python, Flask and MongoDB

Driving conditional requests

Servers use Last-Modified and ETag response headers to drive conditional requests

mercoledì 4 luglio 2012

Page 148: Developing RESTful Web APIs with Python, Flask and MongoDB

Last-ModifiedGenerally considered a weak validator since it has a

one-second resolution

“Wed, 06 Jun 2012 14:19:53 UTC”

mercoledì 4 luglio 2012

Page 149: Developing RESTful Web APIs with Python, Flask and MongoDB

ETagEntity Tag is a strong validator since its value can be

changed every time the server modifies the representation

7a9f477cde424cf93a7db20b69e05f7b680b7f08

mercoledì 4 luglio 2012

Page 150: Developing RESTful Web APIs with Python, Flask and MongoDB

On ETags

• Clients should be able to use ETag to compare representations of a resouce

• An ETag is supposed to be like an object’s hash code.

• Actually, some web frameworks and a lot of implementations do just that

• ETag computed on an entire representation of the resource may become a performance bottleneck

mercoledì 4 luglio 2012

Page 151: Developing RESTful Web APIs with Python, Flask and MongoDB

Last-Modified or ETag?

You can use either or both. Consider the types of client consuming your service. Hint: use both.

mercoledì 4 luglio 2012

Page 152: Developing RESTful Web APIs with Python, Flask and MongoDB

Validating cached representations

Clients use If-Modified-Since and If-None-Match in request headers for validating cached representations

mercoledì 4 luglio 2012

Page 153: Developing RESTful Web APIs with Python, Flask and MongoDB

If-Mod-Since & ETagdef get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get('If-None-Match') if header_etag and header_etag == etag: return prep_response(dict(), status=304)

if_modified_since = request.headers.get('If-Modified-Since') if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304)

response[collection.rstrip('s')] = document return prep_response(response, last_modified, etag) abort(404)

retrieve the document from the database

mercoledì 4 luglio 2012

Page 154: Developing RESTful Web APIs with Python, Flask and MongoDB

If-Mod-Since & ETagdef get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get('If-None-Match') if header_etag and header_etag == etag: return prep_response(dict(), status=304)

if_modified_since = request.headers.get('If-Modified-Since') if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304)

response[collection.rstrip('s')] = document return prep_response(response, last_modified, etag) abort(404)

compute ETag for the current representation. We test ETag first, as it is a stronger

validator

mercoledì 4 luglio 2012

Page 155: Developing RESTful Web APIs with Python, Flask and MongoDB

If-Mod-Since & ETagdef get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get('If-None-Match') if header_etag and header_etag == etag: return prep_response(dict(), status=304)

if_modified_since = request.headers.get('If-Modified-Since') if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304)

response[collection.rstrip('s')] = document return prep_response(response, last_modified, etag) abort(404)

retrieve If-None-Match ETag from request header

mercoledì 4 luglio 2012

Page 156: Developing RESTful Web APIs with Python, Flask and MongoDB

If-Mod-Since & ETagdef get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get('If-None-Match') if header_etag and header_etag == etag: return prep_response(dict(), status=304)

if_modified_since = request.headers.get('If-Modified-Since') if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304)

response[collection.rstrip('s')] = document return prep_response(response, last_modified, etag) abort(404) if client and server

representations match, return a 304 Not Modified

mercoledì 4 luglio 2012

Page 157: Developing RESTful Web APIs with Python, Flask and MongoDB

If-Mod-Since & ETagdef get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get('If-None-Match') if header_etag and header_etag == etag: return prep_response(dict(), status=304)

if_modified_since = request.headers.get('If-Modified-Since') if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304)

response[collection.rstrip('s')] = document return prep_response(response, last_modified, etag) abort(404)

likewise, if the resource has not been modified since

If-Modifed-Since, return 304 Not Modified

mercoledì 4 luglio 2012

Page 158: Developing RESTful Web APIs with Python, Flask and MongoDB

Concurrency controlClients use If-Unmodified-Since and If-Match in request headers as preconditions for concurrency control

mercoledì 4 luglio 2012

Page 159: Developing RESTful Web APIs with Python, Flask and MongoDB

def edit_document(collection, object_id, method): document = find_document(collection, object_id) if document: header_etag = request.headers.get('If-Match') if header_etag is None: return prep_response('If-Match missing from request header', status=403) if header_etag != get_etag(document[LAST_UPDATED]): # Precondition failed abort(412) else: if method in ('PATCH', 'POST'): return patch_document(collection, document) elif method == 'DELETE': return delete_document(collection, object_id) else: abort(404)

Concurrency controlCreate/Update/Delete are controlled by ETag

retrieve client’s If-Match ETag from the request header

mercoledì 4 luglio 2012

Page 160: Developing RESTful Web APIs with Python, Flask and MongoDB

Concurrency controlCreate/Update/Delete are controlled by ETag

editing is forbidden if ETag is not provided

def edit_document(collection, object_id, method): document = find_document(collection, object_id) if document: header_etag = request.headers.get('If-Match') if header_etag is None: return prep_response('If-Match missing from request header', status=403) if header_etag != get_etag(document[LAST_UPDATED]): # Precondition failed abort(412) else: if method in ('PATCH', 'POST'): return patch_document(collection, document) elif method == 'DELETE': return delete_document(collection, object_id) else: abort(404)

mercoledì 4 luglio 2012

Page 161: Developing RESTful Web APIs with Python, Flask and MongoDB

Concurrency controlCreate/Update/Delete are controlled by ETag

def edit_document(collection, object_id, method): document = find_document(collection, object_id) if document: header_etag = request.headers.get('If-Match') if header_etag is None: return prep_response('If-Match missing from request header', status=403) if header_etag != get_etag(document[LAST_UPDATED]): # Precondition failed abort(412) else: if method in ('PATCH', 'POST'): return patch_document(collection, document) elif method == 'DELETE': return delete_document(collection, object_id) else: abort(404)

client and server representations don’t match.

Precondition failed.

mercoledì 4 luglio 2012

Page 162: Developing RESTful Web APIs with Python, Flask and MongoDB

Concurrency controlCreate/Update/Delete are controlled by ETag

def edit_document(collection, object_id, method): document = find_document(collection, object_id) if document: header_etag = request.headers.get('If-Match') if header_etag is None: return prep_response('If-Match missing from request header', status=403) if header_etag != get_etag(document[LAST_UPDATED]): # Precondition failed abort(412) else: if method in ('PATCH', 'POST'): return patch_document(collection, document) elif method == 'DELETE': return delete_document(collection, object_id) else: abort(404)

client and server representation match, go ahead with the edit

mercoledì 4 luglio 2012

Page 163: Developing RESTful Web APIs with Python, Flask and MongoDB

Sending cache & concurrency directives

back to clients

mercoledì 4 luglio 2012

Page 164: Developing RESTful Web APIs with Python, Flask and MongoDB

Cache & Concurrencydef prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add('Cache-Control',

'max-age=%s,must-revalidate' & 30) resp.expires = time.time() + 30 if etag: resp.headers.add('ETag', etag) if last_modified: resp.headers.add('Last-Modified', date_to_str(last_modified)) return resp

encodes ‘dct’ according to client’s accepted

MIME Data-Type

(click here see that slide)

mercoledì 4 luglio 2012

Page 165: Developing RESTful Web APIs with Python, Flask and MongoDB

Cache & Concurrency

Cache-Control, a directive for HTTP/1.1 clients (and

later) -RFC2616

def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add('Cache-Control', 'max-age=%s,must-revalidate' & 30) resp.expires = time.time() + 30 if etag: resp.headers.add('ETag', etag) if last_modified: resp.headers.add('Last-Modified', date_to_str(last_modified)) return resp

mercoledì 4 luglio 2012

Page 166: Developing RESTful Web APIs with Python, Flask and MongoDB

Cache & Concurrency

Expires, a directive for HTTP/1.0 clients

def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add('Cache-Control', 'max-age=%s,must-revalidate' & 30) resp.expires = time.time() + 30 if etag: resp.headers.add('ETag', etag) if last_modified: resp.headers.add('Last-Modified', date_to_str(last_modified)) return resp

mercoledì 4 luglio 2012

Page 167: Developing RESTful Web APIs with Python, Flask and MongoDB

Cache & Concurrency

ETag. Notice that we don’t compute it on the rendered representation, this is by

design.

def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add('Cache-Control', 'max-age=%s,must-revalidate' & 30) resp.expires = time.time() + 30 if etag: resp.headers.add('ETag', etag) if last_modified: resp.headers.add('Last-Modified', date_to_str(last_modified)) return resp

mercoledì 4 luglio 2012

Page 168: Developing RESTful Web APIs with Python, Flask and MongoDB

Cache & Concurrency

And finally, we add the Last-Modified header tag.

def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add('Cache-Control', 'max-age=%s,must-revalidate' & 30) resp.expires = time.time() + 30 if etag: resp.headers.add('ETag', etag) if last_modified: resp.headers.add('Last-Modified', date_to_str(last_modified)) return resp

mercoledì 4 luglio 2012

Page 169: Developing RESTful Web APIs with Python, Flask and MongoDB

Cache & Concurrency

the response object is now complete and ready to be returned to the client

def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add('Cache-Control', 'max-age=%s,must-revalidate' & 30) resp.expires = time.time() + 30 if etag: resp.headers.add('ETag', etag) if last_modified: resp.headers.add('Last-Modified', date_to_str(last_modified)) return resp

mercoledì 4 luglio 2012

Page 170: Developing RESTful Web APIs with Python, Flask and MongoDB

#5HATEOAS

“Hypertext As The Engine Of Application State”

that’s one long ass acronym

mercoledì 4 luglio 2012

Page 171: Developing RESTful Web APIs with Python, Flask and MongoDB

HATEOAS in a Nutshell

• clients interact entirely through hypermedia provided dynamically by the server

• clients need no prior knowledge about how to interact with the server

• clients access an application through a single well known URL (the entry point)

• All future actions the clients may take are discovered within resource representations returned from the server

mercoledì 4 luglio 2012

Page 172: Developing RESTful Web APIs with Python, Flask and MongoDB

It’s all about Linksresource representation includes links to related resources

mercoledì 4 luglio 2012

Page 173: Developing RESTful Web APIs with Python, Flask and MongoDB

{ "links":[    "<link rel='parent' title='home' href='http://api.example.com/' />",    "<link rel='collection' title='contacts'

href='http://api.example.com/Contacts' />",    "<link rel='next' title='next page' 

href='http://api.example.com/Contacts?page=2' />"   ],   "contacts":[      {         "updated":"Wed, 06 Jun 2012 14:19:53 UTC",         "name":"Jon Doe",         "age": 27,         "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08",         "link":"<link rel='self' title='Contact'

href='http://api.example.com/Contacts/4f46445fc88e201858000000' />",

         "_id":"4f46445fc88e201858000000",      }, ]}

Collection Representation

every resource representation provides a

links section with navigational info for

clients

mercoledì 4 luglio 2012

Page 174: Developing RESTful Web APIs with Python, Flask and MongoDB

{ "links":[    "<link rel='parent' title='home' href='http://api.example.com/' />",    "<link rel='collection' title='contacts'

href='http://api.example.com/Contacts' />",    "<link rel='next' title='next page' 

href='http://api.example.com/Contacts?page=2' />"   ],   "contacts":[      {         "updated":"Wed, 06 Jun 2012 14:19:53 UTC",         "name":"Jon Doe",         "age": 27,         "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08",         "link":"<link rel='self' title='Contact'

href='http://api.example.com/Contacts/4f46445fc88e201858000000' />",

         "_id":"4f46445fc88e201858000000",      }, ]}

Collection Representation

the rel attribute provides the relationship between the linked resource and the one

currently represented

mercoledì 4 luglio 2012

Page 175: Developing RESTful Web APIs with Python, Flask and MongoDB

{ "links":[    "<link rel='parent' title='home' href='http://api.example.com/' />",    "<link rel='collection' title='contacts'

href='http://api.example.com/Contacts' />",    "<link rel='next' title='next page' 

href='http://api.example.com/Contacts?page=2' />"   ],   "contacts":[      {         "updated":"Wed, 06 Jun 2012 14:19:53 UTC",         "name":"Jon Doe",         "age": 27,         "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08",         "link":"<link rel='self' title='Contact'

href='http://api.example.com/Contacts/4f46445fc88e201858000000' />",

         "_id":"4f46445fc88e201858000000",      }, ]}

Collection Representation

the title attribute provides a tag (or description) for the linked resource. Could be used as a caption for a

client button.

mercoledì 4 luglio 2012

Page 176: Developing RESTful Web APIs with Python, Flask and MongoDB

{ "links":[    "<link rel='parent' title='home' href='http://api.example.com/' />",    "<link rel='collection' title='contacts'

href='http://api.example.com/Contacts' />",    "<link rel='next' title='next page' 

href='http://api.example.com/Contacts?page=2' />"   ],   "contacts":[      {         "updated":"Wed, 06 Jun 2012 14:19:53 UTC",         "name":"Jon Doe",         "age": 27,         "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08",         "link":"<link rel='self' title='Contact'

href='http://api.example.com/Contacts/4f46445fc88e201858000000' />",

         "_id":"4f46445fc88e201858000000",      }, ]}

Collection Representation

the href attribute provides and absolute path to the resource (the “permanent

identifier” per REST def.)

mercoledì 4 luglio 2012

Page 177: Developing RESTful Web APIs with Python, Flask and MongoDB

{ "links":[    "<link rel='parent' title='home' href='http://api.example.com/' />",    "<link rel='collection' title='contacts'

href='http://api.example.com/Contacts' />",    "<link rel='next' title='next page' 

href='http://api.example.com/Contacts?page=2' />"   ],   "contacts":[      {         "updated":"Wed, 06 Jun 2012 14:19:53 UTC",         "name":"Jon Doe",         "age": 27,         "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08",         "link":"<link rel='self' title='Contact'

href='http://api.example.com/Contacts/4f46445fc88e201858000000' />",

         "_id":"4f46445fc88e201858000000",      }, ]}

Collection Representation

every resource listed exposes its own link, which will allow the client to

perform PATCH, DELETE etc. on the resource

mercoledì 4 luglio 2012

Page 178: Developing RESTful Web APIs with Python, Flask and MongoDB

{ "links":[    "<link rel='parent' title='home' href='http://api.example.com/' />",    "<link rel='collection' title='contacts'

href='http://api.example.com/Contacts' />",    "<link rel='next' title='next page' 

href='http://api.example.com/Contacts?page=2' />"   ],   "contacts":[      {         "updated":"Wed, 06 Jun 2012 14:19:53 UTC",         "name":"Jon Doe",         "age": 27,         "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08",         "link":"<link rel='self' title='Contact'

href='http://api.example.com/Contacts/4f46445fc88e201858000000' />",

         "_id":"4f46445fc88e201858000000",      }, ]}

Collection Representation

while we are here,notice how every resource also exposes its own etag,

last-modified date.

mercoledì 4 luglio 2012

Page 179: Developing RESTful Web APIs with Python, Flask and MongoDB

HATEOASThe API entry point (the homepage)

the API homepage responds to GET requests and provides links to its top level

resources to the clients

@app.route('/', methods=['GET'])def home(): response = {} links = [] for collection in DOMAIN.keys(): links.append("<link rel='child' title='%(name)s'" "href='%(collectionURI)s' />" % {'name': collection, 'collectionURI': collection_URI(collection)}) response['links'] = links return response

mercoledì 4 luglio 2012

Page 180: Developing RESTful Web APIs with Python, Flask and MongoDB

HATEOASThe API entry point (the homepage)

for every collection of resources...

@app.route('/', methods=['GET'])def home(): response = {} links = [] for collection in DOMAIN.keys(): links.append("<link rel='child' title='%(name)s'" "href='%(collectionURI)s' />" % {'name': collection, 'collectionURI': collection_URI(collection)}) response['links'] = links return response

mercoledì 4 luglio 2012

Page 181: Developing RESTful Web APIs with Python, Flask and MongoDB

HATEOASThe API entry point (the homepage)

... provide relation, title and link, or the persistent

identifier

@app.route('/', methods=['GET'])def home(): response = {} links = [] for collection in DOMAIN.keys(): links.append("<link rel='child' title='%(name)s'" "href='%(collectionURI)s' />" % {'name': collection, 'collectionURI': collection_URI(collection)}) response['links'] = links return response

mercoledì 4 luglio 2012

Page 182: Developing RESTful Web APIs with Python, Flask and MongoDB

Wanna see it running?Hopefully it won’t explode right into my face

mercoledì 4 luglio 2012

Page 183: Developing RESTful Web APIs with Python, Flask and MongoDB

Only complaint I have with Flask so far...Most recent HTTP methods not supported

mercoledì 4 luglio 2012

Page 184: Developing RESTful Web APIs with Python, Flask and MongoDB

508 NOT MY FAULTNot supported yet

mercoledì 4 luglio 2012

Page 185: Developing RESTful Web APIs with Python, Flask and MongoDB

208 WORKS FOR MENot supported yet

mercoledì 4 luglio 2012

Page 186: Developing RESTful Web APIs with Python, Flask and MongoDB

Just kidding!it isn’t e

ven

my joke!

mercoledì 4 luglio 2012

Page 187: Developing RESTful Web APIs with Python, Flask and MongoDB

IntroducingMy next open source project

Page 188: Developing RESTful Web APIs with Python, Flask and MongoDB

EveEffortlessly build and deploy

a fully featured proprietary API

Page 189: Developing RESTful Web APIs with Python, Flask and MongoDB

Eve is Open Sourceand brings at your fingertips all the features

mentioned in this talk

Page 194: Developing RESTful Web APIs with Python, Flask and MongoDB

Thank you.@nicolaiarocci

mercoledì 4 luglio 2012