Meet Couch DB

Esti Álvarez @esti

Conferencia Rails 2010viernes 5 de noviembre de 2010

Relational DatabasesDocumental Databases

Schema-free Document Oriented Database

viernes 5 de noviembre de 2010

Key Features

• Completely dynamic schema

• Key-value store in JSON format with B-Tree indexing

• Append-only (all or nothing)

• Concurrent reads and writes

• Very reliable (Erlang)


Replication

• Bidirectional replication with built in conflict resolution

• Offline by default

• Incremental replication


Database interaction

• Data is stored and returned in JSON format

• Queried via HTTP RESTful API

• Index building language: Javascript


Database interaction

• Data is stored and returned in JSON format

• Queried via HTTP RESTful API

• Index building language: Javascript“Django may be built for the Web, but CouchDB is built of the Web. I’ve never

seen software that so completely embraces the philosophies behind HTTP. “ Creator of Django


When should I use CouchDB?

• Concurrent Distributed systems

• Systems where data consistency is important

• Offline by default


Futon. http://localhost:5984


Learn by example


JSON data storage

{ "name": "Homer", "last_names": ["Simpson", "Duff"], "phone_numbers": { "mobile": "555-666-777", "work": "666-777-555" }, "interests": ["beer", "donuts", "couches"]}


Schema-less

{ "_id": "debd7e7385464f4874dd2a38043f7825", "_rev": "3-839e43865b653a1ed73c3d21cc17c5dd", "name": "Homer", "last_names": ["Simpson", "Duff"], "phone_numbers": { "mobile": "555-666-777", "work": "666-777-555", "bar": "777-666-555" }, "interests": ["beer", "donuts", "couches"]}


JSON data storage

{ "_id": "debd7e7385464f4874dd2a38043f7825", "_rev": "3-839e43865b653a1ed73c3d21cc17c5dd", "name": "Homer", "last_names": ["Simpson", "Duff"], "phone_numbers": { "mobile": "555-666-777", "work": "666-777-555" }, "interests": ["beer", "donuts", "couches"]}


HTTP RESTful API

$ curl http://localhost:5984/ypages

HTTP/1.1 404 Object Not Found

{"error":"not_found","reason":"no_db_file"}


Creating the Database

$ curl -X PUT http://localhost:5984/ypages

HTTP/1.1 201 Created

{"ok":true}


Creating a Document$curl -v -X PUT http://localhost:5984/ypages/homer -d

'{"name": "Homer",

"last_names": ["Simpson", "Duff"],

"phone_numbers": { "mobile": "555-666-777",

"work": "666-777-555"},

"interests": ["beer", "donuts", "couches"]}'


Etag: "1-4a9c28dc2e4227cac13872c89a7875d8"

{"ok":true,"id":"homer","rev":"1-4a9c..."}


Getting a Document

$ curl -X GET http://localhost:5984/ypages/homer

HTTP/1.1 200 OK

Etag: "1-4a9c28dc2e4227cac13872c89a7875d8"

{"_id":"homer","_rev":"1-4a9c...","name":"Homer"...}


Updating a Document

$ curl -v -X PUT http://localhost:5984/ypages/homer -d

'{"_rev": "1-4a9c...",

"name": "Marge",

...}'


Etag: "2-ca3a39967947f974ac021d9f1bdafd1a"

{"ok":true,"id":"homer","rev":"2-ca3a39..."}


Querying the database

• Views: the way to arrange data to answer our questions

• Method to build views:

Incremental MapReduce using Javascript


Find by name

function(doc) { if (doc.name) { emit(doc.name, null); }}


Find by name

$ curl '/ypages/_design/app/_view/by_name'

{"total_rows":4,"offset":0, "rows":[ {"key":"Bart","value":null}, {"key":"Homer","value":null}, {"key":"Lisa","value":null}, {"key":"Marge","value":null} ]}


Find by name

$curl '.../by_name?key="Homer"&include_docs=true'

{"total_rows":4,"offset":2, "rows":[ {"id":"homer", "key":"Homer", "value":1, "doc": {"_id":"debd", "_rev":"3-839", "name":"Homer", "last_names":["Simpson","Duff"], "phone_numbers":{"mobile":"555-666-777", "work":"666-777-555"}, "interests":["beer","donuts","couches"] } } ]}


Find by phone type

Map:

function (doc) { var type; if (doc.name && doc.phone_numbers) { for (type in doc.phone_numbers) { emit(type, 1); } }}


Count by phone type

Reduce:

function (keys, values, rereduce) { return sum(values);}


Count by phone type

$ curl '/ypages/_design/app/_view/by_phone'

{"rows":[{"key":null,"value":9}]}

(reduced by default)


Find by phone type

$ curl '.../by_phone?group=true'

{"rows":[

{"key":"home","value":1},

{"key":"mobile","value":4},

{"key":"work","value":4}

]}


Find by phone type

$ curl '.../by_type?key="work"&reduce=false'

{"total_rows":9,"offset":5,"rows":[

{"key":"work","value":1,"id":"..."},

{"key":"work","value":1,"id":"..."},

{"key":"work","value":1,"id":"..."},

{"key":"work","value":1,"id":"..."}

]}


Many more options• startkey, endkey, order, limit...

• View’s keys can be anything you like.

Ej. birthdate:{"total_rows":4,"offset":0, "rows":[ {"key":[2000, 10, 10], "value":"Bart"}, {"key":[1998, 07, 13], "value":"Lisa"}, {"key":[1950, 05, 10], "value":"Homer"}, {"key":[1950, 02, 12], "value":"Marge"} ]}


CouchRest

• A RESTful CouchDB client based on Heroku's RestClient and Couch.js

• http://github.com/couchrest/couchrest


http://github.com/couchrest/couchrest

http://github.com/couchrest/couchrest

CouchRest

• dbr=CouchRest.database("http://localhost:5984/ypages")

• record = dbr.get("homer")

=> {"name"=>"Homer", "_rev"=>"3-b87d", "_id"=>"homer"}

• record['name'] = "Moo"

=> "Moo"

• record.save

=> true


http://localhost:5984/ypages

http://localhost:5984/ypages

Couch Potato

• http://github.com/langalex/couch_potato

• ActiveRecord-like syntax


http://github.com/langalex/couch_potato

http://github.com/langalex/couch_potato

Couch Potatoclass Contact include CouchPotato::Persistence

property :name property :last_names property :phone_numbers validates_presence_of :name view :all, :key => :name view :by_phone, :map => "function(doc) { var phone_type; if (doc.name && doc.phone_numbers) { for (phone_type in doc.phone_numbers) { emit(phone_type, 1); } } }"


Couch Potatocontacts = CouchPotato.database.view Contact.by_phone

:key => “work”, :include_docs => true

=> [#<Contact _id: "8f4aa”...>, ...]

contacts.each do |contact|

contact.name = “Dr. #{contact.name}”

CouchPotato.database.save_document contact

end


Couch Potato

• Validations

• Callbacks

• Test suite

• ...


Simply Stored

• http://github.com/peritor/simply_stored

• Built on top of Couch Potato

• Associations: has_many, belongs_to

• Sintactic sugar: find_by_name


CouchRest Model

• http://github.com/couchrest/couchrest_model

• For Rails 3


Where?

• Ubuntu CouchDesktop (Ubuntu 9.10)

• BBC: fault tolerant, scalable, multi-data center key-value store.

• 32 nodes in two datacenters.

• A terabyte of data/day

• ~ 5 billion of reqs/year. 150 million reqs/day


Our use case

Tracking the activity on a website during a live-broadcasted twitter-interview.


Video streaming server logs (Flash media server)

{ "_id": "ffb297816c756289188ca61dc66d61c7", "_rev": "1-b78bc6ddfbb1f27259c333694d16ba6a", "type": "streaming", "datetime": "2010-03-26 18:15:56", "x-event": "pause", "c-client-id": "872845283", ...}

http://www.irekia.euskadi.net


Web server logs (Apache)

{ "_id": "fde06b5ecd70736e0deb6a22effff25a", "_rev": "1-f496524ccba837b529426e0ad2177524", "type": "web", "datetime": "2010-10-29 08:32:01", "path": "es/web_tv/agentes-sociales-economicos-suscriben-pacto-social-vivienda", "user-agent": "Mozilla/4.0 ...", ...}


Questions made by citizens via Twitter

{ "_id": "219d1d871ad0b9a31b5f10302c8ac03e", "_rev": "1-a4466ffd843164f0f538532aca7d10c6", "datetime": "2010-10-26 18:24:37", "text": "#patxilopez Yo queria preguntar sobre un tema de actualidad q se esta hablando mucho en la calle: Cuando tendremos Bob Esponja en Euskera?", "type": "tweet" ...}


I want more!

• Get it! http://couchdb.apache.org/

• The Definitive Guide:

http://guide.couchdb.org/

• Case studies:

http://www.couchone.com/case-studies


Questions?Sparktography - http://flickr.com/photos/sparktography/65005867/


Meet Couch DB

Technology

Transcript of Meet Couch DB