Post on 28-Nov-2014
description
@doanduyhai
NodeJS Cassandra driver DuyHai DOAN, Technical Advocate
@doanduyhai
Shameless self-promotion!
2
Duy Hai DOAN Cassandra technical advocate • talks, meetups, confs • open-source devs (Achilles, …) • Cassandra technical point of contact
duy_hai.doan@datastax.com
@doanduyhai
Agenda!
3
C* quick intro • Cluster, Replication, Consistency, Data Model NodeJS Driver • Architecture, Streaming, API
Live DEMO!
@doanduyhai
Cassandra history!
4
NoSQL database • created at Facebook • open-sourced since 2008 • current version = 2.1 • column-oriented ☞ distributed table
@doanduyhai
Cassandra in 5 points!
5
• Linear scalability • Continuous availability (≈100% up-time) • Multi-data centers • Consistent performance (99% percentile) • Operational simplicity
Cassandra quick intro!
Cluster Replication
Consistency
@doanduyhai
Cluster!
7
Random: hash of #partition → token = hash(#p) Hash: ]0, 2127-1] Each node: 1/8 of ]0, 2127-1]
n1
n2
n3
n4
n5
n6
n7
n8
@doanduyhai
Linear scalability!
8
n1
n2
n3
n4
n5
n6
n7
n8
n1
n2
n3 n4
n5
n6
n7
n8 n9
n10
8 nodes 10 nodes
@doanduyhai
Failure tolerance!
9
Replication Factor (RF) = 3
n1
n2
n3
n4
n5
n6
n7
n8
1
2
3
@doanduyhai
Coordinator node!Incoming requests (read/write) Coordinator node handles the request
Every node can be coordinator àmasterless
n1
n2
n3
n4
n5
n6
n7
n8
1
2
3
coordinator request
@doanduyhai
Consistency!
11
Tunable at runtime • ONE • QUORUM (strict majority w.r.t. RF) • ALL Apply both to read & write
@doanduyhai
Write consistency!Write ONE • write request to all replicas in // • wait for ONE ack before returning to
client • other acks later, asynchronously
n1
n2
n3
n4
n5
n6
n7
n8
1
2
3
coordinator
@doanduyhai
Write consistency!Write QUORUM • write request to all replicas in // • wait for QUORUM acks before
returning to client • other acks later, asynchronously
n1
n2
n3
n4
n5
n6
n7
n8
1
2
3
coordinator
@doanduyhai
Read consistency!Read ONE • read from one node among all replicas • contact the least-loaded node (stats)
n1
n2
n3
n4
n5
n6
n7
n8
1
2
3
coordinator
@doanduyhai
Read consistency!Read QUORUM • read from one least-loaded node • AND request digest from other
replicas to reach QUORUM • return most up-to-date data to client • repair if digest mismatch n1
n2
n3
n4
n5
n6
n7
n8
1
2
3
coordinator
@doanduyhai
Consistency trade-off!
16
@doanduyhai
CRUD Operations!
17
INSERT INTO users(login, name, age) VALUES(‘jdoe’, ‘John DOE’, 33);
UPDATE users SET age = 34 WHERE login = jdoe;
DELETE age FROM users WHERE login = jdoe;
SELECT age FROM users WHERE login = jdoe;
@doanduyhai
DDL!
18
CREATE TABLE users ( login text, name text, age int, … PRIMARY KEY(login));
partition key (#partition)
Cassandra NodeJS Driver!
Architecture!Streaming!
API!
@doanduyhai
Connection pooling!
20
n2
n3
n4
Client
Pool1
Pool2
Pool3
CallBack1
CallBack2
CallBack3
@doanduyhai
Request Pipelining!
21
Client Cassandra
@doanduyhai
Request Pipelining!
22
Client Cassandra
StreamID
StreamID
@doanduyhai
Nodes Discovery!
23
n2
n3
n4
n5
n6
n7
n8
n1 Client
Control Connection
@doanduyhai
Round Robin Load Balancing!
24
n2
n3
n4
n5
n6
n7
n8
n1 Client 1
2 3
4
@doanduyhai
DC Aware Load Balancing!
25
Client1 DC1
DC2 Client2
⤫
@doanduyhai
DC Aware Load Balancing!
26
Client1 DC1
DC2 Client2
⤫
@doanduyhai
Token Aware Load Balancing!
27
n2
n3
n4
n5
n6
n7
n8
n1 Client
1
2
3
⤫
@doanduyhai
Combining Load Balancing Policies!
28
Load Balancing Policy
Round Robin DC Aware Round Robin
Token Aware
extends
wraps
Default config
@doanduyhai
Automatic Failover!
29
n2
n3
n4
Client
Pool1
Pool2
Pool3
Request
⤫
@doanduyhai
Other policies!
30
Retry policy • write/read timeout • node unavailable Reconnection policy • constant schedule • exponential schedule
@doanduyhai
Where do I get the driver ?!
31
$ npm install cassandra-driver
@doanduyhai
Common Usage!
32
var cassandra = require('cassandra-driver');
var client = new cassandra.Client({contactPoints: [’192.168.0.12', ’node2'], keyspace: ’my_keyspace'});
var query = 'SELECT email, last_name FROM users WHERE login=?';
client.execute(query, [’jdoe'], function(err, result) { console.log('got user with email ' + result.rows[0].email);
});
@doanduyhai
API!
33
Insert/Update
var query = ‘’INSERT INTO sensor_data(id,date,value) VALUES(?, ?, ?)’’; client.query(query,
[‘mySensor’, ‘2014-10-15 12:00:00’, 34.5], {prepare: true, consistency: cassandra.types.consistencies.one}, function(err) { //error handling });
@doanduyhai
API!
34
Select
var query = ‘’SELECT * sensor_data WHERE id = ?’’; client.query(query,
[‘mySensor’], {prepare: true, consistency: cassandra.types.consistencies.one}, function(err, result) { result.rows.forEach(function(row) { console.log(‘value = ‘ + row.value); }); });
@doanduyhai
API!
35
Client creation
new cassandra.Client( { policies: {}, // load balancing, retry, reconnection queryOptions: {}, // default consistency, fetchSize protocolOptions: {}, // port pooling: {}, // #connection/host socketOptions: {}, //timeout authProvider: {}, //plain text login/password, SSL maxPrepared: 500});
@doanduyhai
Streaming!
36
var query = ‘’SELECT * FROM sensor_data‘’; var stream = client.stream(query, [ ], {autoPage: true});
Default fetchSize = 5000
@doanduyhai
Streaming!
37
stream .on(‘end’, function() { //no more data }) .on(‘readable’, function() { while (row = this.read()) { // do something with the row } }) .on(‘error’, function(err) { // process error });
@doanduyhai
Types!
38
Cassandra Javascript bigint Long (dCodeIO)
blob Buffer
boolean Boolean
counter Long (dCodeIO)
Decimal Buffer
Double Number
Float Number
int Number
list Array
map Object
set Array
Cassandra Javascript text String
timestamp Date
timeuuid String
uuid String
varchar* String
varint* Buffer
ascii* String
@doanduyhai
Special types!
39
var cassandra = require('cassandra-driver'); //generate a new v4 uuid var id1 = cassandra.types.uuid(); //generate a new v1 uuid var id2 = cassandra.types.timeuuid(); //big int value var bigIntValue = new cassandra.types.Long.fromString("5764607523034211");
Code demo
Q & R
! " !
Thank You @doanduyhai
duy_hai.doan@datastax.com