Module 3: Creating and Tuning Indexes. Planning Indexes Creating Indexes Optimizing Indexes.
Indexes, what Indexes?
42
-
Upload
alvin-john-richards -
Category
Documents
-
view
268 -
download
0
description
B-Trees, Sparse and Covered indexes and how MongoDB uses them still confusing you? In this talk we will dive into the different indexes that are available, how query plans are executed and the internals of index evaluation including understanding of the explain output.
Transcript of Indexes, what Indexes?
What’s Easy About MongoDB Indexing?
It’s almost the same as in your RDBMS
2Wednesday, May 25, 2011
What’s Hard About MongoDB Indexing?
It’s almost the same as in your RDBMS
3Wednesday, May 25, 2011
Indexes Maintain Order
Index on a: ascending
{a: 0, b: 9}
{a: 2, b: 0}
{a: 7, b: 1}
{a: 3, b: 2}
{a: 3, b: 5}
{a: 3, b: 7}
{a: 9, b: 1}
4Wednesday, May 25, 2011
Indexes Maintain Order
Index on a: ascending, b: descending
{a: 0, b: 9}
{a: 2, b: 0}
{a: 7, b: 1}
{a: 3, b: 7}
{a: 3, b: 2}
{a: 3, b: 5}
{a: 9, b: 1}
5Wednesday, May 25, 2011
Query for {a: 7}
{...} {...} {...} {...} {...} {...} {...} {...} {...} {...} {...}
[-‐∞, 5)[5, 10)
[10, ∞)
[5, 7) [7, 9) [9, 10)[10, ∞) buckets[-‐∞, 5) buckets
With Index
Without index -‐ Scan
Index on a: ascending
6Wednesday, May 25, 2011
db.blogs.ensureIndex({author: 1})
1 = ascending-‐1 = descending
An index on _id is automatic.
For more use ensureIndex:
Creating Indexes
7Wednesday, May 25, 2011
Compound Indexes
db.blogs.save({ author: "James", ts: new Date()});
db.blogs.ensureIndex({author: 1, ts: -‐1})
8Wednesday, May 25, 2011
db.blogs.save({ ... title: "My first blog"});
db.blogs.ensureIndex({title: 1}, {unique: true})
Unique Indexes
9Wednesday, May 25, 2011
db.blogs.save({ ... stats : { views: 0, followers: 0 }});
db.blogs.ensureIndex({"stats.followers": -‐1})
db.blogs.find({"stats.followers": {$gt: 500}})
Indexing Embedded Documents
10Wednesday, May 25, 2011
db.blogs.save({ ... comments: [ {author: "James", ts : new Date()} ]});
db.blogs.ensureIndex({"comments.author": 1})
db.blogs.find({"comments.author": "James"})
Indexing Embedded Arrays
11Wednesday, May 25, 2011
db.blogs.save({ ... tags: ["mongodb", "NoSQL"]});
db.blogs.ensureIndex({tags: 1})
db.blogs.find({tags: "NoSQL"})
Multikeys
12Wednesday, May 25, 2011
• New in 1.8• Query resolved in index only• Need to exclude _id from items projected
db.blogs.save({ ... author: "James",});
db.blogs.ensureIndex({author: 1})
db.blogs.find({author: "James"}, {author: 1, _id:0}))
Covered Indexes
13Wednesday, May 25, 2011
• New in 1.8• Key value included if and only if the value is present• Reduces size of index• Limited to a single field
db.blogs.ensureIndex({loc: 1}, {sparse: true})
// loc key stored for Ben & Sarah onlydb.blogs.save({author: "Jim"})db.blogs.save({author: "Ben", loc: null})db.blogs.save({author: "Sarah", loc: "CA"})
Sparse Indexes
14Wednesday, May 25, 2011
• Null and not-present are differentdb.blogs.ensureIndex({url:1}, {sparse:true,unique:true});
db.blogs.save({url:"www.10gen.com"})
// Can only have a single null valuedb.blogs.save({url:null}) db.blogs.save({url:null})E11000 duplicate key error index: test.blogs.$url_1 dup key: { : null }
// Can have multiple missing valuesdb.blogs.save({author:"James"})db.blogs.save({author:"Bob"})
Unique Sparse Indexes
15Wednesday, May 25, 2011
db.blogs.save({ loc: { long: 40.739037, lat: 40.739037 }});
db.blogs.save({ loc: [40.739037, 40.739037]});
db.blogs.ensureIndex({"loc": "2d"})
Geospatial• Geo hash stored in B-Tree• First two values indexed
16Wednesday, May 25, 2011
db.blogs.getIndexes()
db.blogs.totalIndexSize()
db.blogs.dropIndex({"author": 1})
Other Index Operations
17Wednesday, May 25, 2011
When is an Index Used?Index on {a: 1}
db.c.find({a: 0})db.c.find({a: {$in: [0, 2]}})db.c.find({a: {$gt: 5}})db.c.find().sort({a: -‐1})db.c.find({a: 0}, {a:1, _id:0})db.c.count({a: 0})db.c.find({loc: {$near: [50,50] }})
db.c.find({b: 0}).sort({a: -‐1})
Partially:
18Wednesday, May 25, 2011
When isn’t an Index Used?
Index on {a: 1, b: -‐1}
db.c.find({b: 0})db.c.find({a: {$ne: 1} })db.c.find({a: {$nin: [1, 3, 5, 7]} })
19Wednesday, May 25, 2011
db.c.ensureIndex({x:1})db.c.ensureIndex({y:-‐1})
db.c.find({x: 10, y: “foo”})
scan
index on x
index on y remember
terminated
Picking an index
20Wednesday, May 25, 2011
•Frequently used queries•Low response time•Avoid full collection scans
When are Indexes Needed?
21Wednesday, May 25, 2011
Indexes Slow Down Writes
•Indexes take up space•Index maintenance will cause writes
22Wednesday, May 25, 2011
Does my query use an Index?
db.blogs.find({title:"My blog"}).explain();
23Wednesday, May 25, 2011
db.blogs.find({title:"My blog"}).explain();
{ "cursor" : "BasicCursor", "indexBounds" : {} "nscanned" : 57594, "nscannedObjects" : 57594, "n" : 3, "millis" : 108}
Explain - Scan all documents
24Wednesday, May 25, 2011
{ "cursor" : "BtreeCursor x_1", "indexBounds" : {...}, "nscanned" : 123, "nscannedObjects" : 123, "n" : 10, "millis" : 4}
db.blogs.ensureIndex({title: 1});db.blogs.find({title: "My blog"}).explain();
Explain - Index used
25Wednesday, May 25, 2011
{ "cursor" : "BtreeCursor x_1", "indexBounds" : {...}, "nscanned" : 123, "nscannedObjects" : 123, "n" : 10, "millis" : 4, "indexOnly" : true}
db.blogs.ensureIndex({title: 1});db.blogs.find({title: "My blog"}, {title: 1, _id: 0}).explain();
Explain - Covered Index used
26Wednesday, May 25, 2011
96
{_id:4,x:6}
1 2 3 4 5 6 6
{_id:5,x:6}
{_id:1,x:6}
db.c.find({x: 6})
Index Internals - Equality match
27Wednesday, May 25, 2011
db.c.find({x:6}).explain(){ "cursor" : "BtreeCursor x_1", "nscanned" : 3, "nscannedObjects" : 3, "n" : 3, "millis" : 1, "nYields" : 0, "nChunkSkips" : 0, "isMultiKey" : false, "indexOnly" : false, "indexBounds" : { "x" : [ [ 6, 6 ] ] }}
Equality match
28Wednesday, May 25, 2011
"indexBounds" : {
"x" : [
[
6,
6
]
]
}
Equality match
Exact match will have an inclusive
range
29Wednesday, May 25, 2011
"nscanned" : 3,
"nscannedObjects" : 3,
"n" : 3,
Equality match
Number of matching documents is 3
30Wednesday, May 25, 2011
1
2
3 4
5
6
6
6 9
Equality matchdb.c.find({x: 6})
31Wednesday, May 25, 2011
91 2 3 4 5 6 6 6
{y:4,x:6} {y:5,x:6}
{y:1,x:6}
Full document matchdb.c.ensureIndex({x: 1})
db.c.find({x: 6, y: 1})
32Wednesday, May 25, 2011
db.c.find({x: 6, y: 1}).explain(){ "cursor" : "BtreeCursor x_1", "nscanned" : 3, "nscannedObjects" : 3, "n" : 1, "millis" : 1, "nYields" : 0, "nChunkSkips" : 0, "isMultiKey" : false, "indexOnly" : false, "indexBounds" : { "x" : [ [ 6, 6 ] ] }}
Full document match
33Wednesday, May 25, 2011
"indexBounds" : { "x" : [ [ 6, 6 ] ] }
Full document match
Exact match will have an inclusive
range
34Wednesday, May 25, 2011
"nscanned" : 3,
"nscannedObjects" : 3,
"n" : 1,
Full document match
Documents for all matching keys
scanned, but only one document
matched on non-index keys
35Wednesday, May 25, 2011
81 2 3 4 5 6 7 9
Range matchdb.c.ensureIndex({x: 1})
db.c.find({x:{$gte:4, $lte:7}})
36Wednesday, May 25, 2011
db.c.find( {x:{$gte:4,$lte:7}} ).explain(){ "cursor" : "BtreeCursor x_1", "nscanned" : 4, "nscannedObjects" : 4, "n" : 4, "millis" : 1, "nYields" : 0, "nChunkSkips" : 0, "isMultiKey" : false, "indexOnly" : false, "indexBounds" : { "x" : [ [ 4, 7 ] ] }}
Range match
37Wednesday, May 25, 2011
"indexBounds" : {
"x" : [
[
4,
7
]
]
Range match
Range match will have an inclusive
range
38Wednesday, May 25, 2011
"nscanned" : 4,
"nscannedObjects" : 4,
"n" : 4,
Range match
Number of matching documents is 4
39Wednesday, May 25, 2011
1
2
3 4
5
6
7
8 9
Range match
40Wednesday, May 25, 2011
insert query update delete getmore command flushes mapped vsize res locked % idx miss % 5345 0 0 0 0 1 0 8.75g 11.1g 132m 1 2 230 0 0 0 0 1 0 8.75g 11.1g 132m 1 7 1740 20 0 0 0 2 0 8.75g 11.1g 132m 3 19 0 120 0 0 0 1 0 8.75g 11.1g 132m 0 17 0 117 0 0 0 1 0 8.75g 11.1g 132m 0 16 0 119 0 0 0 1 0 8.75g 11.1g 132m 0 17
Mongostat
41Wednesday, May 25, 2011
@mongodb
conferences, appearances, and meetupshttp://www.10gen.com/events
http://bit.ly/mongoF Facebook | Twitter | LinkedIn
http://linkd.in/joinmongo
download at mongodb.org
We’re Hiring [email protected]
42Wednesday, May 25, 2011
