MongolDB Commands Tutorials

Joshi

Levels of Data1. Database2. Collection3. Document

Common Mistakes• Don’t put the key in quotation:

ObjectId("53baf7f6cbda7100842d810d")

• Use the $ Sign before the aggregate commands but don’t use quotation if you are at RoboMongo – Quotation are used when you do the same in python (or else python thinks it as a variable)• If there is just on element don’t use curly brackets

• { '$project': { "_id": 1, "Address.City":1 } } ,

Date Format• ISODate("2002-01-12T20:15:31Z")

Basic Command with 2 condition • db.getCollection('Violations').find({"Type":'Elevator',"PropertyId":Obje

ctId("53bb0687cbda7100842f78c7")})

And Command – Sample Code – Find the errordb.getCollection('Jobs').find({ $and ( "EntityOwner.UserId": ObjectId("563f474ffbffa908f42e899c"), "StatusDetailsDate":{$gt: ISODate("2002-01-12T20:15:31Z")} )})

Correct – Placement of Brackets [{ from helpdb.getCollection('Jobs').find({ $and : [ {"PropertyId": ObjectId("53bb0677cbda7100842f775f")} , { "StatusDetailDate":{$gt: ISODate("2002-01-12T20:15:31Z")} } ] })

Greater than command• There are only two curly brackets allowed so any comparison for area

would come in the first curly brackets {}• Don’t add extra brackets after the ones you select• This has nothing to do with [] – these are only used in aggreagate

commands

db.getCollection('PropertyDetail').find({'AreaTotalsTotal': {$gt: 10000} }).count()

And with Countdb.getCollection('Jobs').find({ $and: [ { PropertyId: ObjectId("53baf812cbda7100842d885c") } , {'StatusDetailDate' : { $gt : ISODate('1950-01-01 00:00:00.000Z')} } ] } ).count()

It is equal to using and where in SQL

Selecting only the columns wanted• db.getCollection('LoanDetails').find({'Lender.Name':'METROPOLITAN

NATIONAL BANK'},{'Lender.Name':1})• db.test.find({"shapes.color": "red"}, {"shapes.color": 1})

• The 2nd curly brackets help you to select the columns that you want to display

Aggregate – Group match and Sortdb.LoanDetails.aggregate([ { $match: { 'Risk.Score': {$gt:547} } }, { $group: { _id: "$Lenders.Name", total: { $avg: "$Term.Amount" } } }, { $sort: { total: -1 } } ])

Example Aggregation (Groping and Sorting)db.getCollection('PropertyDetail').aggregate([ { $project: { 'Address.City': 1} } , { $group : { _id : '$Address.City' , count: {"$sum": 1} }} , { $sort : { 'count' : -1 }} ] )• The brackets {} comes in only when you give more than one entry• _id should not be in brackets• The aggregate bracket starts with ( but when you have group, sort and

project you use []

Not working - Where CommandWhen they are used?

db.getCollection('Jobs').find( { active: true, $where :"StartDate==ExpirationDate"})

Not working – Map Reduce var mapFunction1 = function() { emit( db.getCollection('PropertyDetail')._id, db.getCollection('PropertyDetail').AreaTotalsTotal ); }; var reduceFunction1 = function(keyCustId, valuesPrices) { return Array.sum(valuesPrices); }; db.orders.mapReduce( mapFunction1, reduceFunction1, { out: "map_reduce_example" } )

Average• db.getCollection('PropertyDetail').aggregate([{$group: {_id:null, pop:

{$avg:"$BuildingCount"} } }])

Something Happened – still to figure out the outputdb.getCollection('PropertyDetail').mapReduce(function() { emit ( '_id', 'BuildingCount' ); } , function(key_Id, valuesPrices) { return Array.avg(valuesPrices); } , { out : "resultName" } )

String search• db.getCollection('Jobs').find({"Description" : /building/}).count() =11k• db.getCollection('Jobs').find({"Description" :

{$regex : /.*violation.*/}}).count()• db.getCollection('Jobs').find({"Description" : {$regex :

".*violation.*"}}).count()

• Regax is not case sensitive

Groupby Max• db.getCollection('RiskScores').aggregate(• [ { $project: { 'EntityId': 1} } , • { $group : { _id : '$Date' , max: {"$max": 1} }• } , { $sort : { 'Date' : -1 }} ] )

For applying max• RiskScoresU = RiskScores.groupby('EntityId').apply(lambda t:

t[t.DateFractional==t.DateFractional.max()])

_x and _y trouble while merging• Deleting them as you get them

Loans for Zip code • Loans for Zip code • Pulling the first array component

• db.getCollection('LoanDetails').find({'Properties.0._id':ObjectId("54634a82cbda712858a78f75"), 'Properties.0.PropertyType':'Retail'})

Searching violation score • db.getCollection('RiskScores').find({'ScoreType':'Property.Partial.Operatio

nal'})

• db.getCollection('RiskScores').find({'EntityID': ObjectId("54bd12a8cbda71674c024b9f"),'ScoreType':'Property.Partial.Operational'})

• db.getCollection('PropertyDetail').find({'_id': ObjectId("54bd12a8cbda71674c024b9f")})

Inside a field – in its first element • db.getCollection('Violations').find({'IssuanceAuthorities.0.DateIssued':

{'$gte':'2011-03-26 00:00:00.000Z'}})

db.getCollection('Violations').aggregate([ { $project: { 'Status': 1} } , { $group : { _id : '$Status' , count: {"$sum": 1} }} , { $sort : { 'count' : -1 }} ] )

• Output for count 0 is 2,244,808 and 1 is 1,008,467• 3253275 = 3253275

Type in MongoDB• Refering agency and so on 1279029.000000

db.getCollection('Violations').aggregate([ { $project: { 'Type': 1} } , { $group : { _id : '$Type' , count: {"$sum": 1} }} , { $sort : { 'count' : -1 }} ] )

Using PipeLine db.getCollection('Violations').aggregate([ { $match: { 'IssuanceAuthorities.0.DateIssued': {'$gte':ISODate("2009-02-08 00:00:00.000Z") } } } ,{ $project: { 'Type': 1} } , { $group : { _id : '$Type' , count: {"$sum": 1} }} , { $sort : { 'count' : -1 }} ] )

Working code violationdb.getCollection('Violations').aggregate([ { $match: { $and: [{ 'IssuanceAuthorities.0.DateIssued': {'$gte':ISODate("2009-02-08 00:00:00.000Z")}} , { 'IssuanceAuthorities.0.Source': 'ECB'} ] } } ,{ $project: { 'Type': 1} } , { $group : { _id : '$Type' , count: {"$sum": 1} }} , { $sort : { 'count' : -1 }} ] )

• db.getCollection('Violations').find({'IssuanceAuthorities.0.DateIssued': {'$gte':("2011-03-26 00:00:00.000Z")}})

Playing Joinsdb.getCollection('Violations').aggregate([ {$match: { 'IssuanceAuthorities.0.DateIssued': {$gte:ISODate("2009-02-08 00:00:00.000Z")}} }, {$lookup: { from: 'Properties', localField: 'PropertyId', foreignField: '_id', as: "NameSJ"} }])

db.getCollection('Violations').aggregate([ {$match: { 'IssuanceAuthorities.0.DateIssued': {$gte:ISODate("2015-08-08 00:00:00.000Z")}}}, {$lookup: { from: 'Properties', localField: 'PropertyId', foreignField: '_id', as: "NameSJ"} } , {$project: { _id: 0, name: "$BuildingName" }} ])

db.getCollection('Violations').aggregate([ {$match: { 'IssuanceAuthorities.0.DateIssued': {$gte:ISODate("2015-08-08 00:00:00.000Z")}}}, {$lookup: { from: 'Properties', localField: 'PropertyId', foreignField: '_id', as: "NameSJ"} } , {$project: { _id: 0, name: "$BuildingName" }} ])

db.getCollection('Violations').aggregate([ {$match: { 'IssuanceAuthorities.0.DateIssued': {$gte:ISODate("2015-08-08 00:00:00.000Z")}}}, { '$group' : { "_id" : '$PropertyId' , 'countPid': {"$sum": 1} }} , {$lookup: { from: 'Properties', localField: '_id', foreignField: '_id', as: "NameSJ"} } , {$project: { _id:1 , namept: "$NameSJ.PropertyType.Main" , name: "$NameSJ.Name" }} , { $sort: {namept: 1} }, ])

• db.getCollection('Violations').aggregate([• {$match: {• 'IssuanceAuthorities.0.DateIssued': {$gte:ISODate("2015-09-09 00:00:00.000Z")}}}, • • { $group :• { '_id' : "$PropertyId" , • 'Lastdate': { $max: "$IssuanceAuthorities.DateIssued" }• }• }

• ])

• db.getCollection('Violations').aggregate([• {$match: {• 'IssuanceAuthorities.0.DateIssued': {$gte:ISODate("2015-08-08 00:00:00.000Z")}}}, • { $project : {• 'PropertyId' : 1 ,• '_id' : 1,• 'IssuanceAuthorities.0.DateIssued' : 1 • }},• { '$group' : { "_id" :{ "date": '$IssuanceAuthorities.0.DateIssued' ,"propid":'PropertyId' } • , 'maxQuantity': { $max: "$IssuanceAuthorities.0.DateIssued" }• ,'count': {"$sum": 1} }} ])

• db.getCollection('Violations').find({'Status':'Pending','DateClosed' :{$exists:false}}).count()

• db.getCollection('Violations').find({'DateClosed' :null}).count()

PropertyID using Jobs – said no space (and worked later)db.getCollection('Jobs').aggregate([ { '$group' : { "_id" : '$PropertyId' , 'countPid': {"$sum": 1} }} , {$lookup: { from: 'Properties', localField: '_id', foreignField: '_id', as: "NameSJ"} } , {$project: { _id:1 , namept: "$NameSJ.PropertyType.Main" , name: "$NameSJ.Name" }} , { $sort: {namept: 1} }, ])

Worked as less than 16gb• db.getCollection('Jobs').aggregate([• • { '$group' : { "_id" : '$PropertyId' , 'countPid': {"$sum": 1} }} ,• {$lookup: {• from: 'Properties', localField: '_id', foreignField: '_id',• as: "NameSJ"}• } , {$project: { _id:1 }} ,• { $sort: {_id: 1} }, ])

Joining using MongoDB – worked!db.getCollection('Jobs').aggregate([{ '$group' : { "_id" : '$PropertyId' , 'countPid': {"$sum": 1} }} ,{$lookup: { from: 'Properties', localField: '_id', foreignField: '_id', as: "NameSJ"} } , {$project: {_id:1 , namept: "$NameSJ.Lot.City" , name: "$NameSJ.Name" }} ,{$sort: {namept: 1} },{$match: { 'namept': 'Chicago'}} ])

• db.getCollection('Jobs').aggregate([• • { '$group' : { "_id" : '$PropertyId' , 'countPid': {"$sum": 1} }} ,• {$lookup: {• from: 'Properties', localField: '_id', foreignField: '_id',• as: "NameSJ"}• } , {$project: { _id:1 , namept: "$NameSJ.PropertyType.Main" ,• name: "$NameSJ.Name" }} ,• { $sort: {namept: 1} },

• {$match: {• 'NameSJ.PropertyType.Main': 'Retail'}}

• ])

MongoDB Query Structure • You can write several lines one after each other till the time that

name doesn’t changes• You can use match before and after the and based on match you can

change things • You can select by project anytime and when not needed you can leave

the thing in between

Trying to group by Weighted average • db.collection.aggregate({• $group : {• _id : 'PropertyId', // build any group key ypo need• numerator: { $sum: { $multiply: [ "$price", "$quantity" ] } },• denominator: { $sum: "$quantity" }• }• }, {• $project: {• average: { $divide: [ "$numerator", "$denominator" ] }• }• })

• var date_test = ISODate ("2013-07-26T22:35:40.373Z")• date_test.getTime()

• db.getCollection('Violations').find({'DateClosed':1})

Groupby Play • db.getCollection('Violations').aggregate([• { '$group' : { "_id" : '$PropertyId' , 'countPid': {"$sum": 1} }} ,• {$project: {_id:1 , 'countPid':1, }} ,• ])

Trying to convert dates to year faileddb.getCollection('Violations').aggregate([{ '$group' : { "_id" : 'PropertyId': '$PropertyId' , '$IssuanceAuthorities.0.DateIssued' : '$IssuanceAuthorities.0.DateIssued' } , 'countPid': {"$sum": "$IssuanceAuthorities.0.DateIssued"} }} ,{$project: {_id:1 , 'countPid':1, }} ,])

Failed – tried to convert dates and groupby dates• db.getCollection('Violations').aggregate([• {• '$project': {"_id" :1, 'PropertyId':1, 'year': { '$cond': [{ '$ifNull': ['$timestamp',

0] },• { '$year': '$IssuanceAuthorities.0.DateIssued' }, -1] }• }}• , { '$group' : { "_id" : '$PropertyId' • , 'countPid': {"$avg":"$year"} • }} , {'$project': {'_id':1 , 'countPid':1 }} ,• ])

Links:• http://usuaris.tinet.cat/bertolin/pdfs/mongodb_%20the%20definitive

%20guide%20-%20kristina%20chodorow_1401.pdf• https://docs.mongodb.org/v3.0/reference/method/js-collection/• https://docs.mongodb.org/manual/tutorial/model-tree-structures-with-

nested-sets/• https://docs.mongodb.org/v3.0/reference/method/

db.collection.aggregate/• http://www.tutorialspoint.com/mongodb/mongodb_map_reduce.htm• https://docs.mongodb.org/manual/tutorial/query-documents/