© 2017 Percona1

MongoDB Index TypesHow, when and where should they be used?

Percona Webinar - Wed July 12th 11:00AM PSTAdamo ToneteMongoDB Senior Service Technical Service Engineer

© 2017 Percona2

{ me : '@adamotonete' }

Adamo Tonete,

I've been working for Percona since late 2015 as a Senior Technical Services Engineer.

I live in São Paulo/Brazil

© 2017 Percona3

Agenda

● What is an index?● How do indexes work?● Costs of maintaining an index● Types of Indexes in MongoDB and differences among

them.● Q&A session.

© 2017 Percona4

What is an index?

● Here the library act as a database,each bookshelf is a collectionand each book is a document.

© 2017 Percona5

What is an index?

The execution plan:

● Could you please find all the books published in 1977 and

1978?

Image from http://feepik.com

© 2017 Percona6

What is an index?The execution plan:

The only option is to read the entire bookshelf andcheck the book publication date.

© 2017 Percona7

How do indexes work?

The Index

This is an illustration, in the real world, of what an index looks like.

If you were the guy in the last picture, what would you do? Would you read all the books or simply check the indexes?

Library catalog

© 2017 Percona8

Indexes in MongoDBbigcollection (~ 100000 documents) - No Indexdb.bigcollection.find({name : 'PERCONA'}).explain(true)

"executionStats" : {"executionSuccess" : true,"nReturned" : 0,"executionTimeMillis" : 350,"totalKeysExamined" : 0,"totalDocsExamined" : 1000000,"executionStages" : {

"stage" : "COLLSCAN","filter" : {

"name" : {"$eq" : "PERCONA"

}},"nReturned" : 0,

© 2017 Percona9

smallcollection (100 documents) - No Indexdb.smallcollection.find({name : 'PERCONA' }).explain(true)

"executionStats" : {"executionSuccess" : true,"nReturned" : 0,"executionTimeMillis" : 0,"totalKeysExamined" : 0,"totalDocsExamined" : 100,"executionStages" : {

"stage" : "COLLSCAN","filter" : {

"name" : {"$eq" : "PERCONA"

}},"nReturned" : 0,

Indexes in MongoDB

© 2017 Percona10

Indexes in MongoDBbigcollection (~ 100000 documents) With Indexesdb.bigcollection.find({name : 'PERCONA' }).explain(true)

"executionStats" : {"executionSuccess" : true,"nReturned" : 0,"executionTimeMillis" : 0,"totalKeysExamined" : 0,"totalDocsExamined" : 0,"executionStages" : {

"stage" : "FETCH","nReturned" : 0,"executionTimeMillisEstimate" : 0,

"inputStage" : {"stage" : "IXSCAN","nReturned" : 0,"executionTimeMillisEstimate" : 0,

© 2017 Percona11

Indexes in MongoDBsmallcollection (100 documents) - With Indexesdb.smallcollection.find({name : 'PERCONA' }).explain(true)

"executionStats" : {"executionSuccess" : true,"nReturned" : 0,"executionTimeMillis" : 0,"totalKeysExamined" : 0,"totalDocsExamined" : 0,"executionStages" : {

"stage" : "FETCH","nReturned" : 0,"executionTimeMillisEstimate" : 0,

"inputStage" : {"stage" : "IXSCAN","nReturned" : 0,"executionTimeMillisEstimate" : 0,"works" : 1,

© 2017 Percona12

How do indexes work?

A-Z

A-L

A-G H-L M-S T-Z

M-Z

PERCONA

PERCONA

Q-SM - (percona) P

© 2017 Percona13

Costs of maintaining indexes:

● Each Index adds cost to the writing process.● Writes will occur in real time –– there is no "delay" when updating indexes.● A lot of indexes will probably slow down the write performance.

© 2017 Percona14

Costs of maintaining an index

A-Z

A-L

A-G H-L M-S T-Z

M-Z

PERCONA

PERCONA

Q-SM-P

PERCONA

© 2017 Percona15

Indexes in MongoDB

The index types are:

_id GeoIndexes (geoHaystack, 2d spherical, flat 2d indexes)

Single Field MultiKey indexes

Compound Indexes Hash indexes

Text Sparse

TTL indexes Unique

Partial (filtered)

© 2017 Percona16

Types of Indexes

● _id is the default primary key in MongoDB.

● _id is not a clustered index and the database must perform another operation so as to read all the values of the document.

© 2017 Percona17

Types of Indexes

● Single field is also called simple index and it indexes only one field at a time.

● Creating a single field index will help the query optimizer find the desired document quickly.

● A high cardinality field is a good candidate to be indexed.

© 2017 Percona18

Types of Indexes

● When single field is not enough by itself, the compound indexes help the execution plan to filter as many documents as possible.

● If we filter by name and lastname and only the name field is indexed, we may need to open a lot of documents to read the lastname. On the other hand, if lastname is already indexed, there is no need to do so.

● Use the higher cardinality field as first field in the btree.

© 2017 Percona19

Types of Indexes

● Multikey Indexes are used when the field value is an array;

● For each key in the array there is an entry with its values.

● For the document { likes : ['mongodb', 'mysql'] }, the multikey will generate 2 different index keys pointing to the same document.

m

mongodb mysql

© 2017 Percona20

Types of Indexes

● Unique keys are unique! ● There isn't much to say here... except that unique keys don’t work in a

sharded environment.

© 2017 Percona21

Types of Indexes

● There are plenty of GeoIndexes. MongoDB offers rich geo algorithms;

● For flat calculations, we can use 2d indexes;

● It supports 2d sphere;

● We can also use geoHaystack in order to get responses - This will compare small squares instead of reading whole collection: It is very similar to geohashes.

© 2017 Percona22

Types of Indexes

● Text indexes behave similarly to the single field index, but there are some extended properties such as collations and case insensitive. We can also give weight to words when running a search.

© 2017 Percona23

● Sparse and Partial indexes.

● Although they sound very similar, sparse indexes only create index keys in documents that do have the field.The filtered index will create an index only for a field value that matches the argument. E.g { 'age' : {$gte : 10 }}

Types of Indexes

© 2017 Percona24

Types of Indexes

● Hash indexes are commonly used in shards to create random keys for the writes and increase the write performance.

© 2017 Percona25

Types of Indexes

● TTL indexes are very useful.Every minute mongodb will run a "TTL round", where all the documents matching time to live will be removed from the database.

● It is very useful, but when you are creating a TTL in a big collection, it is better to first delete the expired documents, otherwise the database load will increase significantly.

© 2017 Percona26

Quick Review

● Indexes are good for you if well used.

● Avoid creating more indexes than required.

● Use the right index for your application. Indexes use space in the disk and memory. Indexing all the fields in one collection is useless in 90% of the cases.

● There is no magic, fast processors and a good amount of memory will decrease the response time.

© 2017 Percona27

Questions?

DATABASE PERFORMANCEMATTERS

Database Performance MattersDatabase Performance MattersDatabase Performance MattersDatabase Performance MattersDatabase Performance Matters

© 2017 Percona29

Percona Live Europe Call for Papers & Registration are Open!

Championing Open Source Databases▪ MySQL, MongoDB, Open Source Databases

▪ Time Series Databases, PostgreSQL, RocksDB

▪ Developers, Business/Case Studies, Operations

▪ September 25-27th, 2017

▪ Radisson Blu Royal Hotel, Dublin, Ireland

Submit Your Proposal by July 17th!www.percona.com/live/e17