MongoDB & NoSQL 101

49
MongoDB & NoSQL 101 2014.9.24 Jollen, Founder <[email protected]> www.mokoversity.com Mokoversity

description

Introducing MongoDB - NoSQL, Document-oriented, JSON Document, Schema-less, MongoDB Shell & Script, CRUD, Schema Design, Query Criteria, Query Operator, Aggregation Pipeline and its Operator and Node.js Driver. All at level 101. 本課程邀請到 Jollen 為大家介紹 MongoDB 與 NoSQL 的基本觀念。NoSQL(Not Only SQL)的簡寫。NoSQL 和傳統的 RDBMS(關聯式資料庫)是很不一樣的技術。MongoDB 搭配 MapReduce 技術,除了是 Big Data 的技術核心,也是資料科學家愛好的技術。

Transcript of MongoDB & NoSQL 101

Page 1: MongoDB & NoSQL 101

MongoDB & NoSQL 101 2014.9.24

Jollen, Founder <[email protected]>

www.mokoversity.com

Mokoversity

Page 2: MongoDB & NoSQL 101

認識 NoSQL 資料庫

Mokoversity

Page 3: MongoDB & NoSQL 101

NoSQL• Not Only SQL

• Document-Oriented

• Not a replacement for SQL database

• Schema-less

• Full Text Search, Data-processing, read faster, write faster

• etc

Page 4: MongoDB & NoSQL 101

Document-Oriented 與 JSON

Mokoversity

Page 5: MongoDB & NoSQL 101

Step Zero$ mongo!MongoDB shell version: 2.4.2!...!> !> db!test!> show dbs!local! 0.078125GB!vcard! 0.203125GB!> use vcard!switched to db vcard!>

Cards

mongo

db

showdbs

use

Page 6: MongoDB & NoSQL 101

認識 Collections

> use vcard!switched to db vcard!> show collections!messages!system.indexes!users!> exit!bye

Cards

show collections

exit

Page 7: MongoDB & NoSQL 101

什麼是 Collections• Document Store (aka folder)

• In JSON and BSON format

• aka XML、MS Word、MS Excel

• Grouping documents

• collections

• Directory hierarchies

• Collections could be considered analogous to tables

Page 8: MongoDB & NoSQL 101

Key-Value Paris• Key–Value stores use the associative

array

• The fundamental data model of document store

• In-memory document store

• Memory Cache

• Redis and etc.

Page 9: MongoDB & NoSQL 101

Create a new Database

$ mongo!!> show dbs!local! 0.078125GB!messages! 0.203125GB!test! 0.203125GB!> use mytest!switched to db mytest!> db.dropDatabase()!{ "dropped" : "mytest", "ok" : 1 } Cards

use <new-db-name>

Page 10: MongoDB & NoSQL 101

Save a new Document

> use mytest!switched to db mytest!> db.mytest.save({name: 'jollen'})!> db.mytest.find()!{ "_id" : ObjectId("53bf7682af97b69f131d970a"), "name" : "jollen" }!>

Cards

save()

find()⾃自動產⽣生

Page 11: MongoDB & NoSQL 101

認識 Schema• NoSQL 資料庫不需要事先定義欄位

• Table Definition

• Document 的欄位定義稱為 Schema

• Has Schema 或 No Schema

Page 12: MongoDB & NoSQL 101

Query a Collection• find()

• find({name: ‘Jollen’})

• findOne()

Page 13: MongoDB & NoSQL 101

Remove a Document• remove()

• remove({name: 'jollen'})

Page 14: MongoDB & NoSQL 101

Update a Document• update({name: 'jollen2'}, {name:

'jollen3'})

• update({name:'jollen'}, {$set: {name: 'jollen2'}}, {multi: true})

http://docs.mongodb.org/manual/reference/method/db.collection.update/

Query criteria

fields

New document !content

Page 15: MongoDB & NoSQL 101

基本的 CRUD 操作• find()-Read

• Query Criteria

• Projection

• Aggregation

• update()-Update

• Query Criteria

Page 16: MongoDB & NoSQL 101

學習要點• NoSQL 初學者應先學習 Collection 的操作

• CRUD

• 類似於使⽤用 SQL 語法做 Table 的查詢、新增、更新與刪除

• 學習 Schema Design 觀念與實作

• 學習 Query Criteria

• 學習 Aggregation

• MapReduce 的基礎

Page 17: MongoDB & NoSQL 101

Query Criteria• 各式「條件查詢」的做法 • {age: {$gt: 18} }

• WHERE

http://docs.mongodb.org/manual/core/read-operations/

Page 18: MongoDB & NoSQL 101

MongoDB CRUD

Mokoversity

Page 19: MongoDB & NoSQL 101

Implement CRUD• MongoDB Shell

• MongoDB Script

• MongoDB Drivers

• Node.js (JavaScript)

• Python / Ruby

• C / C++

• etc - http://docs.mongodb.org/ecosystem/drivers/

Page 20: MongoDB & NoSQL 101

MongoDB Script• 使⽤用 JavaScript 語法

• 進⾏行 CRUD 操作

• 撰寫管理功能 • 簡單的 Data Processing (MapReduce)

Page 21: MongoDB & NoSQL 101

Create One Document// 0001!!{! var db = connect('localhost/test');!! db.vipData.save({name: 'jollen', tel: '09384567182'});!! print('0001-create-one-document finished.')!}

Page 22: MongoDB & NoSQL 101

List Collection// 0002!!{! var db = connect('localhost/test');!! db.vipData.find().forEach(function(user) {! print("User: " + user.name + ", tel: " + user.tel);! });!! print("Info: 0002-list-collection finished.");!}!

Page 23: MongoDB & NoSQL 101

Update Document

// 0003!!{! var db = connect('localhost/test');!! db.vipData.update({name: 'jollen'}, { $set: {name: 'jollenchen'} });!! print('0004-update-document finished');!}

Page 24: MongoDB & NoSQL 101

Add News Field// 0004!!{! var db = connect('localhost/test');!! db.vipData.find().forEach(function(data) {! data.isActive = true;! db.vipData.save(data);! });!! print('0004-add-new-field finished');!}

Page 25: MongoDB & NoSQL 101

Remove Document

// 0005!!{! var db = connect('localhost/test');!! db.vipData.remove({name: 'jollen'});!! print('0005-remove-document finished.');!}

Page 26: MongoDB & NoSQL 101

Schema Design

Mokoversity

Page 27: MongoDB & NoSQL 101

基本觀念• 類似關鍵式資料庫的 Table 設計

• Flexible Schema

Page 28: MongoDB & NoSQL 101

Schema 設計原則• When designing data models, always consider

the application usage of the data (i.e. queries, updates, and processing of the data) as well as the inherent structure of the data itself.

• There are two tools that allow applications to represent these relationships: references and embedded documents.

• References

• Embedded Datahttp://docs.mongodb.org/manual/core/data-modeling-introduction/

Page 29: MongoDB & NoSQL 101

Embedded DocumentPost Collection

_id

title

content

tags

Nickname

Fullname

Mobile

Age

Page 30: MongoDB & NoSQL 101

Reference Model 使⽤用原則• 亦稱為 Normalized Data Model

• 主要⽤用途 • One-to-Many Relations*

• Many-to-Many Relations

• 主要缺點 • ⽂文件變⼤大時造成 write performance 變差

Page 31: MongoDB & NoSQL 101

One-To-Many 的情況 #1

{! _id: "joe",! name: "Joe Bookreader”,! address:! {! patron_id: "joe",! street: "123 Fake Street",! city: "Faketon",! state: "MA",! zip: "12345"! }!}!!

{! _id: "joe",! name: "Joe Bookreader"! address:! {! patron_id: "joe",! street: "1 Some Other Street",! city: "Boston",! state: "MA",! zip: “12345"! }!}

Page 32: MongoDB & NoSQL 101

Reference Model

http://docs.mongodb.org/manual/core/data-modeling-introduction/

Page 33: MongoDB & NoSQL 101

關於 Reference Model• 建議使⽤用的 Data Model

• Using references between documents

• Avoid duplication of data

• Provide sufficient read performance advantage

Page 34: MongoDB & NoSQL 101

Schema DesignUser Collection

_id

name

email

address

age Message Collection

_id

message

uid

Page 35: MongoDB & NoSQL 101

專案描述• 將 Microsoft Excel 資料庫,轉換⾄至 NoSQL 的開發任務

• 客⼾戶資料是以 Excel 維護,現在想將客⼾戶資料「上雲端」,並製作 App 以隨時追蹤或查詢

• 將客⼾戶資料轉換為 NoSQL 資料庫。以便進⾏行 RESTful Web Service 的開發,以及 App 的製作

Page 36: MongoDB & NoSQL 101

CRUD: Task 1• 建⽴立 vcard 資料庫

• 新增資料⾄至 vcard.users collections

• 資料欄位 • name

• phone

• email

• address

• age

Page 37: MongoDB & NoSQL 101

提⽰示• 將 users.xls 匯⼊入 vcard.users collection

• 註:users.xls 是程式產⽣生的假資料,⾮非真實資料

Page 38: MongoDB & NoSQL 101

CRUD: Task 2• 新增資料⾄至 vcard.messages collections

• 資料欄位 • uid(紀錄 ObjectId)

• message(⽂文章內⽂文)

Page 39: MongoDB & NoSQL 101

Aggregation (MapReduce)

Mokoversity

Page 40: MongoDB & NoSQL 101

使⽤用 Query Criteria

http://docs.mongodb.org/manual/core/crud-introduction/

Page 41: MongoDB & NoSQL 101

CRUD: Task 3• 公司想針對 45 歲以下的客⼾戶進⾏行⾏行銷推廣

• 找出 18 歲以上的客⼾戶

Page 42: MongoDB & NoSQL 101

使⽤用 Aggregationexports.query = function(req, res) {!! var model = req.app.db.models.User;!!! model!! .aggregate([! {! ! $project: { _id: 1, name: 1, age: 1 }! },! {! ! $match: {age: {$gt: 45} }! },! {! ! $match: {age: {$lt: 60} }! }])!! .exec(function(err, users) {!! ! res.send(users);!! ! res.end();!! });!}

Page 43: MongoDB & NoSQL 101

REST API- Using Node.js and mongoose

Mokoversity

Page 44: MongoDB & NoSQL 101

REST API 規劃

Page 45: MongoDB & NoSQL 101

認識 CRUD

Page 46: MongoDB & NoSQL 101

User Collection 的 Schema// Model name: ‘User’!var userSchema = new mongoose.Schema({! name: { type: String, default: ''},! email: String,! address: String,! age: { type: Number, default: 0 }!});

User Collection

name

email

address

age

Page 47: MongoDB & NoSQL 101

Message Collection 的 Schema

// Model name: ‘Message’!var messageSchema = new mongoose.Schema({! uid: { type: mongoose.Schema.Types.ObjectId, ref: 'User' },! message: String!});

Message Collection

_id

message

uid

Page 48: MongoDB & NoSQL 101

使⽤用 $in Operator• 興趣有 ‘sport’ 的會員

{ interests: { $in: [ 'sport' ] } }

Page 49: MongoDB & NoSQL 101

Populate

model! .find()! .populate('uid')! .exec(function(err, posts) {! res.send(posts);! res.end();! });