MongoDB @ Frankfurt NoSql User Group

45
Chris Harris Email : [email protected] Twitter : cj_harris5 Frankfurt NoSQL User Group Wednesday, 22 February 12

description

 

Transcript of MongoDB @ Frankfurt NoSql User Group

Chris Harris Email : [email protected]

Twitter : cj_harris5

Frankfurt NoSQL User Group

Wednesday, 22 February 12

As simple as possible, but no simpler

Depth of functionality

Sca

labi

lity

& P

erfo

rman

ce

MemcachedKey / Value

RDBMS

Wednesday, 22 February 12

MongoDBKey/Value Store

Relational Database

Wednesday, 22 February 12

Terminology

RDBMS MongoDBTable CollectionRow(s) JSON DocumentIndex IndexJoin Embedding & LinkingPartition ShardPartition Key Shard Key

Wednesday, 22 February 12

Here is a “simple” SQL Modelmysql> select * from book;+----+----------------------------------------------------------+| id | title |+----+----------------------------------------------------------+| 1 | The Demon-Haunted World: Science as a Candle in the Dark || 2 | Cosmos || 3 | Programming in Scala |+----+----------------------------------------------------------+3 rows in set (0.00 sec)

mysql> select * from bookauthor;+---------+-----------+| book_id | author_id |+---------+-----------+| 1 | 1 || 2 | 1 || 3 | 2 || 3 | 3 || 3 | 4 |+---------+-----------+5 rows in set (0.00 sec)

mysql> select * from author;+----+-----------+------------+-------------+-------------+---------------+| id | last_name | first_name | middle_name | nationality | year_of_birth |+----+-----------+------------+-------------+-------------+---------------+| 1 | Sagan | Carl | Edward | NULL | 1934 || 2 | Odersky | Martin | NULL | DE | 1958 || 3 | Spoon | Lex | NULL | NULL | NULL || 4 | Venners | Bill | NULL | NULL | NULL |+----+-----------+------------+-------------+-------------+---------------+4 rows in set (0.00 sec)

Wednesday, 22 February 12

Flexible “Schemas”

{ “author”: “brendan”, “text”: “...” }

{ “author”: “brendan”, “text”: “...”, “tags”: [“mongodb”, “nosql”] }

Wednesday, 22 February 12

The Same Data in MongoDB

{ "_id" : ObjectId("4dfa6baa9c65dae09a4bbda5"), "title" : "Programming in Scala", "author" : [ { "first_name" : "Martin", "last_name" : "Odersky", "nationality" : "DE", "year_of_birth" : 1958 }, { "first_name" : "Lex", "last_name" : "Spoon" }, { "first_name" : "Bill", "last_name" : "Venners" } ]}

Wednesday, 22 February 12

CRUD Operations

Wednesday, 22 February 12

db.test.insert({fn: “Chris”, ln: “Harris”})

Wednesday, 22 February 12

db.test.find({fn: “Chris”})

Wednesday, 22 February 12

Cursors

> var c = db.test.find({x: 20}).skip(20).limit(10)> c.next()> c.next()...

$gt, $lt, $gte, $lte, $ne, $all, $in, $nin, $or,$not, $mod, $size, $exists, $type, $elemMatch

query

first N results + cursor id

getMore w/ cursor idnext N results + cursor id or 0

...

Wednesday, 22 February 12

db.blogs.ensureIndex({author: 1})

1 = ascending-1 = descending

An index on _id is automatic.

For more use ensureIndex:

Creating Indexes

Wednesday, 22 February 12

Compound Indexes

db.blogs.save({ author: "James", ts: new Date() ...});

db.blogs.ensureIndex({author: 1, ts: -1})

Wednesday, 22 February 12

db.blogs.save({ author: "James", title: "My first blog" ...});

db.blogs.ensureIndex({title: 1}, {unique: true})

Unique Indexes

Wednesday, 22 February 12

db.blogs.save({ title: "My First blog", stats : { views: 0, followers: 0 }});

db.blogs.ensureIndex({"stats.followers": -1})

db.blogs.find({"stats.followers": {$gt: 500}})

Indexing Embedded Documents

Wednesday, 22 February 12

db.blogs.save({ title: "My First blog", comments: [ {author: "James", ts : new Date()} ]});

db.blogs.ensureIndex({"comments.author": 1})

db.blogs.find({"comments.author": "James"})

Indexing Embedded Arrays

Wednesday, 22 February 12

db.blogs.save({ title: "My Second blog", tags: ["mongodb", "NoSQL"]});

db.blogs.ensureIndex({tags: 1})

db.blogs.find({tags: "NoSQL"})

Multikeys

Wednesday, 22 February 12

• From 1.8.0• Query resolved in index only• Need to exclude _id from items projected

db.blogs.save({ author: "James", title: "My first blog"});

db.blogs.ensureIndex({author: 1})

db.blogs.find({author: "James"}, {author: 1, _id:0}))

Covered Indexes

Wednesday, 22 February 12

• From 1.8.0• Key value included if and only if the value is present• Reduces size of index• Limited to a single field

db.blogs.ensureIndex({loc: 1}, {sparse: true})

// loc key stored for Ben & Sarah onlydb.blogs.save({author: "Jim"})db.blogs.save({author: "Ben", loc: null})db.blogs.save({author: "Sarah", loc: "CA"})

Sparse Indexes

Wednesday, 22 February 12

db.blogs.save({ loc: { long: 40.739037, lat: 40.739037 }});

db.blogs.save({ loc: [40.739037, 40.739037]});

db.blogs.ensureIndex({"loc": "2d"})

Geospatial• Geo hash stored in B-Tree• First two values indexed

Wednesday, 22 February 12

Replication

Wednesday, 22 February 12

Types of outage

• Planned• Hardware upgrade• O/S or file-system tuning• Relocation of data to new file-system / storage• Software upgrade

• Unplanned• Hardware failure• Data center failure• Region outage• Human error• Application corruption

Wednesday, 22 February 12

• A cluster of N servers• Any (one) node can be primary• Consensus election of primary• Automatic failover• Automatic recovery• All writes to primary• Reads can be to primary (default) or a secondary

Replica Set features

Wednesday, 22 February 12

How MongoDB Replication works

Member 1

Member 2

Member 3

•Set is made up of 2 or more nodes

Wednesday, 22 February 12

How MongoDB Replication works

Member 1

Member 2

PRIMARY

Member 3

•Election establishes the PRIMARY•Data replication from PRIMARY to SECONDARY

Wednesday, 22 February 12

How MongoDB Replication works

Member 1

Member 2

DOWN

Member 3

negotiate new master

•PRIMARY may fail•Automatic election of new PRIMARY if majority exists

Wednesday, 22 February 12

How MongoDB Replication works

Member 1

Member 2

DOWN

Member 3

PRIMARY

•New PRIMARY elected•Replication Set re-established

Wednesday, 22 February 12

How MongoDB Replication works

Member 1

Member 2RECOVERING

Member 3

PRIMARY

•Automatic recovery

Wednesday, 22 February 12

How MongoDB Replication works

Member 1

Member 2

Member 3

PRIMARY

•Replication Set re-established

Wednesday, 22 February 12

Sharding

Wednesday, 22 February 12

http://community.qlikview.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/theqlikviewblog/Cutting-Grass-with-Scissors-_2D00_-2.jpg

Wednesday, 22 February 12

http://www.bitquill.net/blog/wp-content/uploads/2008/07/pack_of_harvesters.jpg

Wednesday, 22 February 12

MongoDB Scaling - Single Node

write

read

node_a1

Wednesday, 22 February 12

Write scaling - add Shards

write

read

shard1

node_c1

node_b1

node_a1

shard2

node_c2

node_b2

node_a2

Wednesday, 22 February 12

Write scaling - add Shards

write

read

shard1

node_c1

node_b1

node_a1

shard2

node_c2

node_b2

node_a2

shard3

node_c3

node_b3

node_a3

Wednesday, 22 February 12

MongoDB Sharding

• Automatic partitioning and management

• Range based

• Convert to sharded system with no downtime

• Fully consistent

Wednesday, 22 February 12

How MongoDB Sharding works

> db.posts.save( {age:40} )

-∞   +∞  

-∞   40 41 +∞  

•Data in inserted•Ranges are split into more “chunks”

Wednesday, 22 February 12

How MongoDB Sharding works

> db.posts.save( {age:40} )> db.posts.save( {age:50} )

-∞   +∞  

-∞   40 41 +∞  

41 50 51 +∞  

•More Data in inserted•Ranges are split into more“chunks”

Wednesday, 22 February 12

How MongoDB Sharding works

> db.posts.save( {age:40} )> db.posts.save( {age:50} )> db.posts.save( {age:60} )

-∞   +∞  

-∞   40 41 +∞  

41 50 51 +∞  

61 +∞  51 60

Wednesday, 22 February 12

Balancing

Shard 1 Shard 2 Shard 3 Shard 4

5

9

1

6

10

2

7

11

3

8

12

4

17

21

13

18

22

14

19

23

15

20

24

16

29

33

25

30

34

26

31

35

27

32

36

28

41

45

37

42

46

38

43

47

39

44

48

40

mongos

balancerconfig

config

config

Chunks!

Wednesday, 22 February 12

Balancingmongos

balancerconfig

config

config

Shard 1 Shard 2 Shard 3 Shard 4

5

9

1

6

10

2

7

11

3

8

12

4

21 22 23 24 33 34 35 36 45 46 47 48

ImbalanceImbalance

Wednesday, 22 February 12

Balancingmongos

balancer

Move chunk 1 to Shard 2

config

config

config

Shard 1 Shard 2 Shard 3 Shard 4

5

9

1

6

10

2

7

11

3

8

12

4

21 22 23 24 33 34 35 36 45 46 47 48

Wednesday, 22 February 12

Balancingmongos

balancerconfig

config

config

Shard 1 Shard 2 Shard 3 Shard 4

5

9

6

10

2

7

11

3

8

12

4

21 22 23 24 33 34 35 36 45 46 47 48

1

Wednesday, 22 February 12

Balancingmongos

balancerconfig

config

config

Shard 1 Shard 2 Shard 3 Shard 4

5

9

6

10

2

7

11

3

8

12

4

21 22 23 24 33 34 35 36 45 46 47 48

1

Wednesday, 22 February 12

Balancingmongos

balancer

Chunk 1 now lives on Shard 2

config

config

config

Shard 1 Shard 2 Shard 3 Shard 4

5

9

16

10

2

7

11

3

8

12

4

21 22 23 24 33 34 35 36 45 46 47 48

Wednesday, 22 February 12