NoSQL(Orientation)Niklas Fors,EDAF20,2019-02-21

NoSQL• NoSQL– term(re-)introducedin2009• RetroactivelyreinterpretedasNotOnlySQL• Easiertoscalehorizontally (morecomputers)• Usuallyfavorsavailabilityoverconsistency

• E.g.,whathappensifanetworkpartitionoccur?• NormallyuseseventualconsistencyinsteadofACID

• Oftenschema-less(orschema-on-read)• Drivenby

• Aneedforgreaterscalability• Specializedqueriesnotwellsupportedinrelationaldatabases• Toorestrictiveschemasinrelationaldatabases

• Mostdatabasesareactivelybeingdeveloped(thus,achangingarea)

RelationalDatabases

• Requiresyoutospecifyschemasbeforeaddingdata(schema-on-write)• Goodsupportforjoinsandtransactions• Maturequerylanguage(SQL)andtools• Favorsconsistencyoveravailability• GoodACIDsupport

• Hardtoscalehorizontally (whenaddingmorecomputers)• Scalesvertically (whengettingabettercomputer)

NoSQLDatabases

• Graph. Nodesandedges,bothattachedwithproperties.Usefulforgraph-likestructuressuchassocialnetworks.• Examples:Neo4j,…

• Key-value.(Distributed)dictionary:mappingkeystovalues.• Examples:Redis,...

• Document. Askey-value,but thevalue isadocument (think JSON).• Examples:MongoDB,...

• Columns.Storesvalues column-based rather than row-based.Usefulforanalyses overasmallsubset of alarge setof columns (e.g.,200).• Examples:Cassandra,…

https://db-engines.com

Neo4j

• (Property)Graphs• Usefulformodellingcertainproblemslikesocialnetworks,etc.,

• Nodes• Zerotomanylabels (e.g.,Person)

• Relationships- directededgesbetweennodes• Onerelationshiptype(e.g.,BOSS_OF)

• Properties• Canbeattachedtonodesandrelationships• Key-valuepairs(e.g.,name:'Anna')

• Cypher• QuerylanguagesimilartoSQL

CreateNodesandRelationships

CREATE (Anna:Person {name:'Anna', salary:40000})CREATE (Peter:Person {name:'Peter', salary:30000})CREATE (Eva:Person {name:'Eva', salary:30000})CREATE (Andrzej:Person {name:'Andrzej', salary:25000})CREATE (Ali:Person {name:'Ali', salary:25000})CREATE (Jan:Person {name:'Jan', salary:25000})CREATE (Erik:Person {name:'Erik', salary:20000})CREATE (Anna)-[:BOSS_OF]->(Peter),

(Anna)-[:BOSS_OF]->(Eva),(Eva)-[:BOSS_OF]->(Andrzej),(Eva)-[:BOSS_OF]->(Ali),(Eva)-[:BOSS_OF]->(Jan),(Jan)-[:BOSS_OF]->(Erik)

...

CreatesnodeswithlabelPersonandwithtwoproperties:nameandsalary.

Saveareferencetothenodecreatedtobeusedwhencreatingrelationships.

CreatesrelationshipswiththetypeBOSS_OF,e.g.,representing that'Anna'isthebossof'Peter'(notethedirection).

SimpleMatching

// Match all persons and return them$ MATCH (p:Person)

RETURN pResult shown to the right ----------->

// A table of names$ MATCH (p:Person)

RETURN p.name...

// A table of names and salaries$ MATCH (p:Person)

RETURN p.name, p.salary...

Notethedirection(arrow)oftheedge!

Filtering// Filter persons by name$ MATCH (p:Person {name: "Anna"})

RETURN p

Neo4jhasaweb-baseduserinterfacewhereyoucansubmitqueriesandviewtheresultin.

Youcanviewtheresultas:- Graph- Table(likeSQLresult)- Text- Code(JSON)

RelationshipFiltering

// Match all persons <p>// that have Anna as a direct boss.// Thus, there should be one BOSS_OF edge// from Anna to the person p.$ MATCH (boss:Person {name:"Anna"})

- [:BOSS_OF] -> (p)RETURN p.name

"Eva""Peter"

One orMore Steps

// Match all persons <p>// that have Anna as a boss.// Note the use of * here, which means// one or more BOSS_OF edges from Anna.$ MATCH (boss:Person {name:"Anna"})

- [:BOSS_OF*] -> (p)RETURN p.name

"Peter""Eva""Andrzej""Ali""Jan""Erik"

// Reverse the edge pattern// to get all bosses of Erik$ MATCH (p:Person {name:"Erik"})

<- [:BOSS_OF*] - (boss)RETURN boss.name

"Jan""Eva""Anna"

ReversingtheDirection

MoreFiltering

// Get Erik's top-level boss, by filtering// over bosses of Erik that has no boss$ MATCH (p:Person {name:"Erik"})

<- [:BOSS_OF*] - (boss)WHERE NOT () - [:BOSS_OF] -> (boss)RETURN boss.name

"Anna"

Aggregation

// Compute the salary for Anna and// her employees using the SUM function$ MATCH (boss:Person {name:"Anna"})

- [:BOSS_OF*] -> (p)RETURN SUM(p.salary) + boss.salary

195000

Aggregation(2)

// Compute the number of employees// (directly and indirectly) for all bosses$ MATCH (boss:Person)

- [:BOSS_OF*] -> (p)return boss.name, COUNT(p)

"Chris" 6"Josefin" 1"Per" 2"Anna" 6"Eva" 4"Jan" 1

AnotherExample – Cities andRoads

CREATE (a:City {name: 'A'})CREATE (b:City {name: 'B'})CREATE (c:City {name: 'C'})CREATE (d:City {name: 'D'})CREATE (e:City {name: 'E'})CREATE (f:City {name: 'F'})CREATE

(a)-[:ROAD {time: 10}]->(b), (b)-[:ROAD {time: 5}]->(c), (a)-[:ROAD {time: 30}]->(c), (c)-[:ROAD {time: 20}]->(d), (c)-[:ROAD {time: 15}]->(e), (d)-[:ROAD {time: 10}]->(f), (e)-[:ROAD {time: 10}]->(f)

Thepersonscouldbeconnectedtocities(describing inwhichcitytheylive).Thus,it'spossibletohavegraphswithnodesofdifferent labelsandedgesofdifferenttypes.

Shortest Path (Ignoring Times)

// Compute shortest path between A and F// (in terms of the number of edges)$ MATCH

(start:City { name: 'A' }),(end:City { name: 'F' }),p = shortestPath((start)-[*..15]-(end))

RETURN pA -> C -> E -> F

Shortest Path (using Dijsktra's algorithm)

// Compute shortest path between A and F// and consider the time it takes using// Dijkstra's algorithm (defined in an // external library)MATCH

(start:City {name:"A"}), (end:City {name:"F"})call apoc.algo.dijkstra(start, end,

'ROAD', 'time')YIELD path, weight

return path, weight;A -> B -> C -> E -> F(weight: takes 40 in time)

Redis

• Redis isakey-valuestorethatmapskeystovalues• OneusageofRedis iscachingcomputedvalues• Exampleofvaluetypes• Simpletypessuchasintegers,strings,etc• Lists– orderedcollectionofelements• Sets– unorderedcollectionofuniqueelements• Sortedsets(z-set)• Hashes– objectswithproperties

• Supportsclustering/sharding (horizontalscaling)• Tutorial:http://try.redis.io

Redis – SetandGet

> SET name NiklasOK> GET name "Niklas"> SET users 5OK> INCR users -- atomic update(integer) 6> INCR users(integer) 7> GET users"7"> DEL users(integer) 1> GET users(nil)

Redis – TimeToLive(TTL)

> SET first-page "hello there"OK> EXPIRE first-page 15 -- make first-page expire after 15 seconds(integer) 1> GET first-page"hello there"...> TTL first-page -- first-page will be valid for 5 more seconds(integer) 5...> GET first-page -- the value has been expired(nil)

Redis – Lists

Lists– orderedcollectionofelements

> RPUSH students "Alice"> RPUSH students "Bob"> RPUSH students "Charlie"> RPUSH students "Alice"

> LRANGE students 0 -1 -- return the whole list1) "Alice"2) "Bob"3) "Charlie"4) "Alice"

> LPOP students -- remove the first student> LRANGE students 0 -11) "Bob"2) "Charlie"3) "Alice"

Redis – Sets

Sets– unordered collectionofuniqueelements

> SADD fruits apple> SADD fruits banana> SADD fruits pear> SMEMBERS fruits -- all members of the set1) "apple"2) "banana"3) "pear"

> SISMEMBER fruits apple -- is apple a member of the set?(integer) 1> SISMEMBER fruits android(integer) 0

MongoDB

• Document-based• Database(databaseinSQL)• Asetof collections

• Collections(tablesinSQL)• Documents(rowsinSQL)• StoredasBSON(BinaryJSON)

• Noschemafordocuments• Twodocumentsinthesamecollectioncanhavedifferentcontent

MongoDB (using mongoshell)

// Select database> use test

// Add documents to the collection persons> db.persons.insert( {name: 'Niklas'} )> db.persons.insert({

name: "Andreas", email: "andreas@gmail.com"

})> db.persons.insert({

name: "Anna", email: "anna@gmail.com", phoneNumbers: [

{ type: "home", number: "212 555-1234"}, { type: "office", number: "646 555-4567"} ]})

Finding Documents – Examples

// Find all documents> db.persons.find(){ "_id" : ObjectId("5c6da87aabdf9f47a7974162"), "name" : "Niklas" }{ "_id" : ObjectId("5c6da96aabdf9f47a7974163"), "name" : "Andreas", "email" : "andreas@gmail.com" }{ "_id" : ObjectId("5c6da9caabdf9f47a7974164"), "name" : "Anna", "email" : "anna@gmail.com", "phoneNumbers" : [ { "type" : "home", "number" : "212 555-1234" }, { "type" : "office", "number" : "646 555-4567" } ] }(An ID has been added to all documents)

// Specify filter> db.persons.find({ name: "Niklas"}){ "_id" : ObjectId("5c6da87aabdf9f47a7974162"), "name" : "Niklas" }

// Find all persons who have a home numberdb.persons.find({"phoneNumbers.type":"home"}){ "name" : "Anna", ... }

MongoDBGoodforone-to-manyrelationships• Ifallinformationisinonedocument– goodlocalityandnojoinsneeded

Alsosupports:• Sorting• Aggregations(counting,mapreduce,etc)• Joins(some)• Transactions(some)• Indexes• Sharding• Horizontalscaling- distributingdataacrossmultiplemachines• Documentsinacollectioncanbepartitionedovermultiplemachines

• …