A Morning with MongoDB Barcelona: MongoDB Basic Concepts

MongoDB Basic Concepts

Norberto Leite

Senior Solutions Architect, [email protected]

@nleite

Sunday, 21 October 12

mailto:[email protected]


Agenda

•Overview•Replication•Scalability•Consistency & Durability•Flexibility, Developer Experienc


http://bit.ly/OT71M4

Your data needs started here...


http://jungleg.com/wp-content/uploads/2008/10/scalability.jpg

http://jungleg.com/wp-content/uploads/2008/10/scalability.jpg

http://bit.ly/Oxcsis

...but soon you had to be here


http://sysperf.files.wordpress.com/2008/09/scalability.jpg

http://sysperf.files.wordpress.com/2008/09/scalability.jpg

Basic Concepts

Horizontally Scalable

{ author : “steve”, date : new Date(), text : “About MongoDB...”, tags : [“tech”, “database”]}

Document Oriented

Application

Fully Consistent

High Performance


depth of functionality

scal

abili

ty &

per

form

ance •memcached

•key/value

• RDBMS

Tradeoff: Scale vs Functionality


Replication


Why do we need replication

•Failover •Backups•Secondary batch jobs •High availability


Replica SetsData Availability across nodes• Data Protection

• Multiple copies of the data• Spread across Data Centers, AZs

• High Availability• Automated Failover• Automated Recovery


Replica Sets

Primary

Secondary

Secondary

Read

Write

Read

Read

App

Asynchronous Replication


Replica Sets

Primary

Secondary

Secondary

Read

Write

Read

Read

App


Replica Sets

Primary

Primary

Secondary

Read

Write

Read

Automatic Election of new Primary

App


Replica Sets

Recovering

Primary

Secondary

Read

Write

Read

New primary serves data

App


Replica Sets

Secondary

Primary

Secondary

Read

Write

Read

Read

App


Scalability


Horizontal Scalability


ShardingData Distribution across nodes• Data location transparent to your code• Data distribution is automatic• Data re-distribution is automatic• Aggregate system resources horizontally• No code changes


Sharding - Range distribution

shard01 shard02 shard03

sh.shardCollection("test.tweets", {_id: 1} , false)


Sharding - Range distribution


a-i j-r s-z


Sharding - Splits


a-i ja-jz s-z

k-r


Sharding - Splits


a-i ja-ji s-z

ji-js

js-jw

jz-r


Sharding - Auto Balancing


a-i ja-ji s-z

ji-js

js-jw

jz-r

js-jw

jz-r


Sharding - Auto Balancing


a-i ja-ji n-z

ji-js

js-jw

jz-r


Sharding - Routed Query


a-i ja-ji n-z

ji-js

js-jw

jz-r

find({_id: "norberto"})


Sharding - Routed Query


a-i ja-ji n-z

ji-js

find({_id: "norberto"})

js-jw

jz-r


Sharding - Scatter Gather


a-i ja-ji n-z

ji-js

js-jw

jz-r

find({email: "[email protected]"})




Sharding - Caching

shard01

a-i

j-r

n-z

300

GB

Dat

a

300 GB

96 GB Mem3:1 Data/Mem


Aggregate Horizontal Resources


a-i j-r n-z


100 GB 100 GB 100 GB

300

GB

Dat

a




Consistency & Durability


Two choices for consistency

•Eventual consistency•Allow updates when a system has been partitioned•Resolve conflicts later•Example: CouchDB, Cassandra

•Immediate consistency•Limit the application of updates to a single master node for a given slice of data

•Another node can take over after a failure is detected•Avoids the possibility of conflicts•Example: MongoDB


Durability

•For how long is my data available?•When do I now that my data is safe?•Where?

•Mongodb style•Fire and Forget•Get Last Error•Journal Sync•Replica Safe


Fire and Forget

Driver Primary


Fire and Forget

Driver Primary

write


Fire and Forget

Driver Primary

write

apply in memory


Get Last Error

Driver Primary


Get Last Error

Driver Primary

getLastError

write


Get Last Error

Driver Primary

getLastErrorapply in memory

write


Journal Sync

Driver Primary


Journal Sync

Driver Primary


write

j:true


Journal Sync

Driver Primary


write

j:true

write to journal


Replicas Safe

Driver Primary Secondary


Replicas Safe

Driver Primary


write

w:2

Secondary


Replicas Safe

Driver Primary


write

w:2

replicate

Secondary


Flexibility


• Cost effective operationalize abundant data (clickstreams, logs, tweets, ...)

• Relaxed transactional semantics enable easy scale out

• Auto Sharding for scale down and scale up

• Applications store complex data that is easier to model as documents

• Schemaless DB enables faster development cycles

What MongoDB solves

Agility

Flexibility

Cost


Challenges for Databases

✓ Build a database for scaleout• Run on clusters of 100s of commodity machines

•… that enables agile development

•… and is usable for a broad variety of applications


Data Model

• Why JSON?• Provides a simple, well understood encapsulation of data• Maps simply to the object in your OO language• Linking & Embedding to describe relationships


Json

place1 = { name : "10gen HQ", address : "578 Broadway 7th Floor", city : "New York", zip : "10011", tags : [ "business", "tech" ]}


Schema DesignRelational Database


Schema DesignMongoDB embedding

linkingSunday, 21 October 12

Schemas in MongoDB

Design documents that simply map to your application

post = {author: "Hergé", date: new Date(), text: "Destination Moon", tags: ["comic", "adventure"]}

> db.posts.save(post)


> db.blogs.find( { author: "Hergé"} )

{ _id : ObjectId("4c4ba5c0672c685e5e8aabf3"), author : "Hergé", date : ISODate("2011-09-18T09:56:06.298Z"), text : "Destination Moon", tags : [ "comic", "adventure" ], comments : [! {! ! author : "Kyle",! ! date : ISODate("2011-09-19T09:56:06.298Z"),! ! text : "great book"! } ] }

Embedding


JSON & Scaleout

• Embedding removes need for• Distributed Joins• Two Phase commit

• Enables data to be distributed across many nodes without penalty


http://bit.ly/UmUnsUSunday, 21 October 12

http://images.mirror.co.uk/upl/m4/mar2009/0/2/satnav-cliff-hanger-pic-ross-parry-image-3-213726463.jpg

http://images.mirror.co.uk/upl/m4/mar2009/0/2/satnav-cliff-hanger-pic-ross-parry-image-3-213726463.jpg

http://bit.ly/cnP77LSunday, 21 October 12

http://schoolloans.org/blog/wp-content/uploads/2009/06/money_tree.jpg

http://schoolloans.org/blog/wp-content/uploads/2009/06/money_tree.jpg

http://bit.ly/ODoMhhSunday, 21 October 12

http://www.simpleadhdexpert.com/wp-content/uploads/2009/08/frustration.jpg

http://www.simpleadhdexpert.com/wp-content/uploads/2009/08/frustration.jpg

http://bit.ly/uW2nk


http://www.travelblog.org/Photos/3871362

http://www.travelblog.org/Photos/3871362

A Morning with MongoDB Barcelona: MongoDB Basic Concepts

Documents

Transcript of A Morning with MongoDB Barcelona: MongoDB Basic Concepts