Schema Design
Transcript of Schema Design
![Page 1: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/1.jpg)
Software Engineer, 10gen
Jeremy Mikola
MongoDB Miami February 7, 2013
Schema Design
![Page 2: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/2.jpg)
Single Table En
Agenda
• Working with documents
• Evolving a Schema
• Queries and Indexes
• Common Patterns
![Page 3: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/3.jpg)
Terminology
RDBMS MongoDB Database ➜ Database Table ➜ Collection Row ➜ Document Index ➜ Index Join ➜ Embedded Document Foreign Key ➜ Reference
![Page 4: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/4.jpg)
Working with Documents
![Page 5: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/5.jpg)
M O NXRays
Checkups
Allergies
Modeling Data
![Page 6: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/6.jpg)
Documents Provide flexibility and performance
![Page 7: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/7.jpg)
Normalized Data
User·Name·Email address
Category·Name·URL
Comment·Comment·Date·Author
Article·Name·Slug·Publish date·Text
Tag·Name·URL
![Page 8: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/8.jpg)
De-Normalized (embedded) Data
User·Name·Email address
Article·Name·Slug·Publish date·Text·Author
Comment[]·Comment·Date·Author
Tag[]·Value
Category[]·Value
![Page 9: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/9.jpg)
Relational Schema Design Focus on data storage
![Page 10: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/10.jpg)
Document Schema Design Focus on data use
![Page 11: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/11.jpg)
Schema Design Considerations
• How do we manipulate the data? – Dynamic Ad-Hoc Queries – Atomic Updates – Map Reduce
• What are the access patterns of the application? – Read/Write Ratio – Types of Queries / Updates – Data life-cycle and growth rate
![Page 12: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/12.jpg)
Data Manipulation
• Query Selectors – Scalar: $ne, $mod, $exists, $type, $lt, $lte, $gt, $gte – Vector: $in, $nin, $all, $size
• Atomic Update Operators – Scalar: $inc, $set, $unset – Vector: $push, $pop, $pull, $pushAll, $pullAll, $addToSet
![Page 13: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/13.jpg)
Data Access
• Flexible Schemas
• Ability to embed complex data structures
• Secondary Indexes
• Multi-Key Indexes
• Aggregation Framework – $project, $match, $limit, $skip, $sort, $group, $unwind
• No Joins
![Page 14: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/14.jpg)
Getting Started
![Page 15: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/15.jpg)
Library Management Application
• Patrons
• Books
• Authors
• Publishers
![Page 16: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/16.jpg)
An Example One to One Relations
![Page 17: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/17.jpg)
patron = { _id: "joe", name: "Joe Bookreader” }
address = { patron_id = "joe", street: "123 Fake St. ", city: "Faketon", state: "MA", zip: 12345 }
Modeling Patrons
patron = { _id: "joe", name: "Joe Bookreader", address: { street: "123 Fake St. ", city: "Faketon", state: "MA", zip: 12345 } }
![Page 18: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/18.jpg)
One to One Relations
• Mostly the same as the relational approach
• Generally good idea to embed “contains” relationships
• Document model provides a holistic representation of objects
![Page 19: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/19.jpg)
An Example One To Many Relations
![Page 20: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/20.jpg)
patron = { _id: "joe", name: "Joe Bookreader", join_date: ISODate("2011-10-15"), addresses: [ {street: "1 Vernon St.", city: "Newton", state: "MA", …}, {street: "52 Main St.", city: "Boston", state: "MA", …} ] }
Modeling Patrons
![Page 21: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/21.jpg)
Publishers and Books
• Publishers put out many books
• Books have one publisher
![Page 22: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/22.jpg)
MongoDB: The Definitive Guide, By Kristina Chodorow and Mike Dirolf Published: 9/24/2010 Pages: 216 Language: English
Publisher: O’Reilly Media, CA
Book
![Page 23: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/23.jpg)
book = { title: "MongoDB: The Definitive Guide", authors: [ "Kristina Chodorow", "Mike Dirolf" ], published_date: ISODate("2010-09-24"), pages: 216, language: "English", publisher: { name: "O’Reilly Media", founded: "1980", location: "CA" } }
Modeling Books – Embedded Publisher
![Page 24: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/24.jpg)
publisher = { name: "O’Reilly Media", founded: "1980", location: "CA" }
book = { title: "MongoDB: The Definitive Guide", authors: [ "Kristina Chodorow", "Mike Dirolf" ], published_date: ISODate("2010-09-24"), pages: 216, language: "English" }
Modeling Books & Publisher Relationship
![Page 25: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/25.jpg)
publisher = { _id: "oreilly", name: "O’Reilly Media", founded: "1980", location: "CA" }
book = { title: "MongoDB: The Definitive Guide", authors: [ "Kristina Chodorow", "Mike Dirolf" ], published_date: ISODate("2010-09-24"), pages: 216, language: "English", publisher_id: "oreilly" }
Publisher _id as a Foreign Key
![Page 26: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/26.jpg)
publisher = { name: "O’Reilly Media", founded: "1980", location: "CA" books: [ "123456789", ... ] }
book = { _id: "123456789", title: "MongoDB: The Definitive Guide", authors: [ "Kristina Chodorow", "Mike Dirolf" ], published_date: ISODate("2010-09-24"), pages: 216, language: "English" }
Book _id as a Foreign Key
![Page 27: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/27.jpg)
Where Do You Put the Foreign Key?
• Array of books inside of publisher – Makes sense when many means a handful of items – Useful when items have bound on potential growth
• Reference to single publisher on books – Useful when items have unbounded growth (unlimited #
of books)
• SQL doesn’t give you a choice, no arrays
![Page 28: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/28.jpg)
Another Example One to Many Relations
![Page 29: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/29.jpg)
Books and Patrons
• Book can be checked out by one Patron at a time
• Patrons can check out many books (but not 1000’s)
![Page 30: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/30.jpg)
patron = { _id: "joe", name: "Joe Bookreader", join_date: ISODate("2011-10-15"), address: { ... } }
book = { _id: "123456789", title: "MongoDB: The Definitive Guide", authors: [ "Kristina Chodorow", "Mike Dirolf" ], ... }
Modeling Checkouts
![Page 31: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/31.jpg)
patron = { _id: "joe", name: "Joe Bookreader", join_date: ISODate("2011-10-15"), address: { ... }, checked_out: [ { _id: "123456789", checked_out: "2012-10-15" }, { _id: "987654321", checked_out: "2012-09-12" }, ... ] }
Modeling Checkouts
![Page 32: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/32.jpg)
De-normalize for speed
Denormalization Provides data locality
![Page 33: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/33.jpg)
patron = { _id: "joe", name: "Joe Bookreader", join_date: ISODate("2011-10-15"), address: { ... }, checked_out: [ { _id: "123456789", title: "MongoDB: The Definitive Guide", authors: [ "Kristina Chodorow", "Mike Dirolf" ], checked_out: ISODate("2012-10-15") }, { _id: "987654321" title: "MongoDB: The Scaling Adventure",
... }, ... ] }
Modeling Checkouts: Denormalized
![Page 34: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/34.jpg)
Referencing vs. Embedding
• Embedding is a bit like pre-joined data
• Document-level ops are easy for server to handle
• Embed when the 'many' objects always appear with (i.e. viewed in the context of) their parent
• Reference when you need more flexibility
![Page 35: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/35.jpg)
An Example Single Table Inheritance
![Page 36: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/36.jpg)
book = { title: "MongoDB: The Definitive Guide", authors: [ "Kristina Chodorow", "Mike Dirolf" ], published_date: ISODate("2010-09-24"), kind: "loanable", locations: [ ... ], pages: 216, language: "English", publisher: { name: "O’Reilly Media", founded: "1980", location: "CA" } }
Single Table Inheritance
![Page 37: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/37.jpg)
An Example Many to Many Relations
![Page 38: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/38.jpg)
Relational Approach
User·Name·Email address
Category·Name·URL
Comment·Comment·Date·Author
Article·Name·Slug·Publish date·Text
Tag·Name·URL
![Page 39: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/39.jpg)
book = { title: "MongoDB: The Definitive Guide", authors = [ { _id: "kchodorow", name: "K-Awesome" }, { _id: "mdirolf", name: "Batman Mike" }, ] published_date: ISODate("2010-09-24"), pages: 216, language: "English" }
author = { _id: "kchodorow", name: "Kristina Chodorow", hometown: "New York" }
Books and Authors
![Page 40: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/40.jpg)
book = { _id: 123456789, title: "MongoDB: The Definitive Guide", authors = [ "kchodorow", "mdirolf" ], published_date: ISODate("2010-09-24"), pages: 216, language: "English" }
author = { _id: "kchodorow", name: "Kristina Chodorow", hometown: "Cincinnati", books: [ 123456789, ... ] }
Relation stored on both sides
![Page 41: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/41.jpg)
An Example Trees
![Page 42: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/42.jpg)
book = { title: "MongoDB: The Definitive Guide", authors: [ "Kristina Chodorow", "Mike Dirolf" ], published_date: ISODate("2010-09-24"), pages: 216, language: "English", category: "MongoDB" }
category = { _id: MongoDB, parent: "Databases" } category = { _id: Databases, parent: "Programming" }
Parent Links
![Page 43: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/43.jpg)
book = { _id: 123456789, title: "MongoDB: The Definitive Guide", authors: [ "Kristina Chodorow", "Mike Dirolf" ], published_date: ISODate("2010-09-24"), pages: 216, language: "English" }
category = { _id: MongoDB, children: [ 123456789, … ] } category = { _id: Databases, children: ["MongoDB", "Postgres"} category = { _id: Programming, children: ["DB", "Languages"] }
Child Links
![Page 44: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/44.jpg)
Modeling Trees
• Parent Links
- Each node is stored as a document
- Contains the id of the parent
• Child Links
- Each node contains the id’s of the children
- Can support graphs (multiple parents / child)
![Page 45: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/45.jpg)
book = { title: "MongoDB: The Definitive Guide", authors: [ "Kristina Chodorow", "Mike Dirolf" ], published_date: ISODate("2010-09-24"), pages: 216, language: "English", categories: ["Programming", "Databases", "MongoDB” ] }
book = { title: "MySQL: The Definitive Guide", authors: [ "Michael Kofler" ], published_date: ISODate("2010-09-24"), pages: 216, language: "English", parent: "MongoDB", ancestors: [ "Programming", "Databases", "MongoDB"] }
Array of Ancestors
![Page 46: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/46.jpg)
An Example Queues
![Page 47: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/47.jpg)
book = { _id: 123456789, title: "MongoDB: The Definitive Guide", authors: [ "Kristina Chodorow", "Mike Dirolf" ], published_date: ISODate("2010-09-24"), pages: 216, language: "English", available: 3 }
db.books.findAndModify({ query: { _id: 123456789, available: { "$gt": 0 } }, update: { $inc: { available: -1 } } })
Book Document
![Page 48: Schema Design](https://reader033.fdocuments.net/reader033/viewer/2022052900/5562530fd8b42a1b4b8b4f10/html5/thumbnails/48.jpg)
Software Engineer, 10gen
Jeremy Mikola
MongoDB Miami, February 7, 2013
Thank You