Cassandra Summit 2014: Understanding CQL3 Inside and Out

The CQL3/Cassandra Mapping

John Berryman

OpenSource Connections


Outline

•  What Problem does CQL Solve? •  The Cassandra Data Model •  Pain Points of “Old” Cassandra •  Introducing CQL •  Understanding the CQL/Cassandra Mapping •  CQL for Sets, Lists, and Maps •  Putting it All Together


What Problem does CQL Solve?

•  The Awesomeness that is Cassandra: o  Distributed columnar data store o  No single point of failure o  Optimized for availability (though “Tunably” consistent) o  Optimized for writes o  Easily maintainable o  Almost infinitely scalable

.




•  Cassandra’s usability challenges o  NoSQL – “Where are my JOINS? No Schema? De-normalize!?” o  BigTable – “Tables with millions of columns!?”

.




•  Cassandra’s usability challenges o  NoSQL – “Where are my JOINS? No Schema? De-normalize!?” o  BigTable – “Tables with millions of columns!?”

•  CQL saves the day! o  A best-practices interface to Cassandra o  Uses familiar SQL-like language


C* Data Model


Keyspace

C* Data Model


Keyspace

Column Family Column Family

C* Data Model


Row Key

C* Data Model


Row Key

Column Name

Column Value (or Tombstone)

Timestamp

Time-to-live

Column

C* Data Model


●  Row Key, Column Name, Column Value have types

●  Column Name has comparator ●  RowKey has partitioner ●  Rows can have any number of

columns - even in same column family ●  Rows can have many columns ●  Column Values can be omitted ●  Time-to-live is useful! ●  Tombstones

Row Key

Column Name

Column Value (or Tombstone)

Timestamp

Time-to-live

Column

C* Data Model: Writes


Mem Table

CommitLog

Row Cache

●  Insert into MemTable

●  Dump to CommitLog

●  No read ●  Very Fast! ●  Blocks on CPU

before O/I!

Key Cache

SSTable

SSTable

SSTable

SSTable Key

Cache Key

Cache Key

Cache

Bloom Filter


Mem Table

CommitLog

Row Cache

Key Cache

SSTable

SSTable

SSTable

SSTable Key

Cache Key

Cache Key

Cache

Bloom Filter

●  Get values from Memtable ●  Get values from row

cache if present ●  Otherwise check bloom

filter to find appropriate SSTables

●  Check Key Cache for fast SSTable Search

●  Get values from SSTables ●  Repopulate Row Cache ●  Super Fast Col. retrieval ●  Fast row slicing

C* Data Model: Reads

Cassandra Pain Points

•  Twitter Example •  My tweets

o  SET tweets[JnBrymn][2013-07-19 T 09:20] = “Wonderful morning. This coffee is great.”!

o  SET tweets[JnBrymn][2013-07-19 T 09:21] = “Oops, smoke is coming out of the SQL server!”!

o  SET tweets[JnBrymn][2013-07-19 T 09:51] = “Now my coffee is cold :-(”

•  Get John’s tweets o  GET tweets[JnBrymn] (output is as expected)



•  Twitter Example •  My tweets

o  SET tweets[JnBrymn][2013-07-19 T 09:20] = “Wonderful morning. This coffee is great.”!

o  SET tweets[JnBrymn][2013-07-19 T 09:21] = “Oops, smoke is coming out of the SQL server!”!

o  SET tweets[JnBrymn][2013-07-19 T 09:51] = “Now my coffee is cold :-(”

•  Get John’s tweets o  GET tweets[JnBrymn] (output is as expected)

•  Pain-point: schema-less means that you have to read code to understand data model



•  My timeline (other’s tweets) •  More complicated – must store corresponding user

names •  Bad Option 1: keep multiple column families

o  SET timeline_from[JnBrymn][2013-07-19 T 09:20] = “softwaredoug”!

o  SET timeline_text[JnBrymn][2013-07-19 T 09:20] = “Hey John I posted on reddit, upvote me!”

•  Get John’s timeline o  GET timeline_from[JnBrymn]!o  GET timeline_text[JnBrymn]!



•  My timeline (other’s tweets) •  More complicated – must store corresponding user

names •  Bad Option 1: keep multiple column families

o  SET timeline_from[JnBrymn][2013-07-19 T 09:20] = “softwaredoug”!

o  SET timeline_text[JnBrymn][2013-07-19 T 09:20] = “Hey John I posted on reddit, upvote me!”

•  Get John’s timeline o  GET timeline_from[JnBrymn]!o  GET timeline_text[JnBrymn]!

•  Pain-point: Multiple queries required.



•  My timeline •  Bad Option 2: shove into single column value

o  SET timeline[JnBrymn][2013-07-19 T 09:20] = {from:”softwaredoug”, text: “Hey John I posted on reddit, upvote me!”

•  Get John’s timeline o  GET timeline[JnBrymn] (…not too bad.)!



•  My timeline •  Bad Option 2: shove into single column value

o  SET timeline[JnBrymn][2013-07-19 T 09:20] = {from:”softwaredoug”, text: “Hey John I posted on reddit, upvote me!”

•  Get John’s timeline o  GET timeline[JnBrymn] (…not too bad.)!

•  Pain-point: Updates require a read-then-modify



•  My timeline •  Best Option: composite column names

o  SET timeline[JnBrymn][2013-07-19 T 09:20|from] = ”softwaredoug”!

o  SET timeline[JnBrymn][2013-07-19 T 09:20|text] = “Hey John, I posted on reddit, upvote me!”

•  Get John’s timeline o  GET timeline[JnBrymn] (extract from and text in client)

•  Resolves prior pain points! Scales well!!



•  My timeline •  Best Option: composite column names

o  SET timeline[JnBrymn][2013-07-19 T 09:20|from] = ”softwaredoug”!

o  SET timeline[JnBrymn][2013-07-19 T 09:20|text] = “Hey John, I posted on reddit, upvote me!”

•  Get John’s timeline o  GET timeline[JnBrymn] (extract from and text in client)

•  Resolves prior pain points! Scales well!!•  Pain-point: Even more code reading to understand

data model!



•  Justin Bieber’s timeline (e.g. many tweets) •  Previous solution fails if number of columns > 2Billion •  Best Option: composite row names

o  SET timeline[bieber|2013-07][19 T 09:20|from] = ”softwaredoug”!

o  SET timeline[bieber|2013-07][19 T 09:20|text] = “Justin Bieber, you complete me.”

•  Get Justin’s timeline o  GET timeline[bieber|2013-07] (get other months too)



•  Justin Bieber’s timeline (e.g. many tweets) •  Previous solution fails if number of columns > 2Billion •  Best Option: composite row names

o  SET timeline[bieber|2013-07][19 T 09:20|from] = ”softwaredoug”!

o  SET timeline[bieber|2013-07][19 T 09:20|text] = “Justin Bieber, you complete me.”

•  Get Justin’s timeline o  GET timeline[bieber|2013-07] (get other months too)

•  Pain-point: Even more code reading to understand data model!


Introducing CQL

•  CQL is a reintroduction of schema so that you don’t have to read code to understand the data model.

•  CQL creates a common language so that details of the data model can be easily communicated.

•  CQL is a best-practices Cassandra interface and hides the messy details.


Introducing CQL

•  CQL is a reintroduction of schema so that you don’t have to read code to understand the data model.

•  CQL creates a common language so that details of the data model can be easily communicated.

•  CQL is a best-practices Cassandra interface and hides the messy details.


Let’s see it!

Introducing CQL

CREATE TABLE users ( id timeuuid PRIMARY KEY,! lastname varchar,! firstname varchar,! dateOfBirth timestamp );!!

!

!


Introducing CQL


INSERT INTO users (id,lastname, firstname, dateofbirth) VALUES (now(),'Berryman',’John','1975-09-15');!!!


Introducing CQL


INSERT INTO users (id,lastname, firstname, dateofbirth) VALUES (now(),’Berryman’,’John’,’1975-09-15’);!!

UPDATE users SET firstname = ’John’ WHERE id = f74c0b20-0862-11e3-8cf6-b74c10b01fc6;!!


Introducing CQL


INSERT INTO users (id,lastname, firstname, dateofbirth) VALUES (now(),'Berryman',’John','1975-09-15');!!

UPDATE users SET firstname = 'John’ WHERE id = f74c0b20-0862-11e3-8cf6-b74c10b01fc6;!!

SELECT dateofbirth,firstname,lastname FROM users ;!!

dateofbirth | firstname | lastname!--------------------------+-----------+----------! 1975-09-15 00:00:00-0400 | John | Berryman!


Introducing CQL

“Hey sweet! It’s exactly the same as MySQL!”


Introducing CQL

“Hey sweet! It’s exactly the same as MySQL!” Hold your horses. There are some

important differences.


Introducing CQL



“Wait? What happened to the Cassandra’s wide rows?”


Introducing CQL




There’s still there. Understanding the mapping is crucial!


Introducing CQL




There’s still there. Understanding the mapping is crucial!


Remember this: • Cassandra finds rows fast

• Cassandra scans columns fast • Cassandra does not scan rows

The CQL/Cassandra Mapping

CREATE TABLE employees (! name text PRIMARY KEY,! age int,! role text!);!!





name | age | role!-----+-----+-----!john | 37 | dev!eric | 38 | ceo!!




name | age | role!-----+-----+-----!john | 37 | dev!eric | 38 | ceo!!

age! role!

john! 37! dev!

age! role!

eric! 38! ceo!


CREATE TABLE employees (! company text,! name text,! age int,! role text,! PRIMARY KEY (company,name)!);!!





company | name | age | role!--------+------+-----+-----!OSC | eric | 38 | ceo!OSC | john | 37 | dev!RKG | anya | 29 | lead!RKG | ben | 27 | dev!RKG | chad | 35 | ops!

The CQL/Cassandra Mapping CREATE TABLE example (! A text,! B text,! C text,! D text,! E text,! F text,! PRIMARY KEY ((A,B),C,D)!);!




A | B | C | D | E | F !--+---+---+---+---+---!a | b | c | d | e | f!a | b | c | g | h | i!a | b | j | k | l | m!a | n | o | p | q | r!s | t | u | v | w | x!



A | B | C | D | E | F !--+---+---+---+---+---!a | b | c | d | e | f!a | b | c | g | h | i!a | b | j | k | l | m!a | n | o | p | q | r!s | t | u | v | w | x!

! c:d:E! c:d:F! c:g:E! c:g:F! j:k:E! j:k:F!

a:b! e! f! h! i! l! m!

! o:p:E! o:p:F!

a:n! q! r!

! u:v:E! u:v:F!

s:t! w! x!

CQL for Sets, Lists, and Maps

•  Collection Semantics o  Sets hold list of unique elements o  Lists hold ordered, possibly repeating elements o  Maps hold a list of key-value pairs

•  Uses same old Cassandra data structure




•  Uses same old Cassandra data structure •  Declaring


CREATE TABLE mytable(! X text,! Y text,! myset set<text>,! mylist list<int>,! mymap map<text, text>,! PRIMARY KEY (X,Y)!);!



•  Uses same old Cassandra data structure •  Declaring


CREATE TABLE mytable(! X text,! Y text,! myset set<text>,! mylist list<int>,! mymap map<text, text>,! PRIMARY KEY (X,Y)!);!

Collection fields can not be used in primary keys


•  Inserting !

INSERT INTO mytable (row, myset) VALUES (123, { ‘apple’, ‘banana’});



•  Inserting !



INSERT INTO mytable (row, mylist) VALUES (123, [‘apple’,’banana’,’apple’]);


•  Inserting !



INSERT INTO mytable (row, mylist) VALUES (123, [‘apple’,’banana’,’apple’]); INSERT INTO mytable (row, mymap) VALUES (123, {1:’apple’,2:’banana’})

CQL for Sets, Lists, and Maps •  Updating UPDATE mytable SET myset = myset + {‘apple’,‘banana’} WHERE row = 123; UPDATE mytable SET myset = myset - { ‘apple’ } WHERE row = 123;!




UPDATE mytable SET mylist = mylist + [‘apple’,‘banana’] WHERE row = 123; UPDATE mytable SET mylist = [‘banana’] + mylist WHERE row = 123;!



UPDATE mytable SET mylist = mylist + [‘apple’,‘banana’] WHERE row = 123; UPDATE mytable SET mylist = [‘banana’] + mylist WHERE row = 123;!

UPDATE mytable SET mymap[‘fruit’] = ‘apple’ WHERE row = 123 UPDATE mytable SET mymap = mymap + { ‘fruit’:‘apple’} WHERE row = 123!

CQL for Sets, Lists, and Maps SETS !CREATE TABLE mytable(! X text,! Y text,! myset set<int>,! PRIMARY KEY (X,Y)!);!!




X | Y | myset !---+---+------------! a | b | {1,2}! a | c | {3,4,5}!



X | Y | myset !---+---+------------! a | b | {1,2}! a | c | {3,4,5}!

! b:myset:1! b:myset:2! c:myset:3! c:myset:4! c:myset:5!

a!



LISTS !CREATE TABLE mytable(! X text,! Y text,! mylist list<int>,! PRIMARY KEY (X,Y)!);!!



X | Y | mylist !---+---+------------! a | b | [1,2]!




X | Y | mylist !---+---+------------! a | b | [1,2]!

! b:mylist:f7e5450039..8d! b:mylist:f7e5450139..8d!

a! 1! 2!


CQL for Sets, Lists, and Maps MAPS !CREATE TABLE mytable(! X text,! Y text,! mymap map<text,int>,! PRIMARY KEY (X,Y)!);!!




X | Y | mymap !---+---+------------! a | b | {m:1,n:2}! a | c |{n:3,p:4,q:5}!



X | Y | mymap !---+---+------------! a | b | {m:1,n:2}! a | c |{n:3,p:4,q:5}!

! b:mymap:m! b:mymap:n! c:mymap:n! c:mymap:p! c:mymap:q!

a! 1! 2! 3! 4! 5!

Peek Behind the Scenes! Do it! (in cqlsh) CREATE KEYSPACE test WITH replication = ! {'class': 'SimpleStrategy', 'replication_factor': 1};!USE test;!CREATE TABLE stuff ( a int, b int, myset set<int>,! mylist list<int>, mymap map<int,int>, PRIMARY KEY (a,b));!UPDATE stuff SET myset = {1,2}, mylist = [3,4,5], mymap = {6:7,8:9} WHERE a = 0 AND b = 1;!SELECT * FROM stuff;!!

(in cassandra-cli) use test;!list stuff ;!!

(in cqlsh) SELECT key_aliases,column_aliases from system.schema_columnfamilies WHERE keyspace_name = 'test' AND columnfamily_name = 'stuff';!


Putting it All Together …you already know •  CQL is a reintroduction of schema •  CQL creates a common data modeling language •  CQL is a best-practices Cassandra interface

.



…now you know •  CQL let’s you take advantage of the C* Data structure

.



…now you know •  CQL let’s you take advantage of the C* Data structure

…but also •  CQL protocol is binary and therefore interoperable with

any language •  CQL is asynchronous and fast (Thrift transport layer is

synchronous) •  CQL allows the possibility for prepared statements


Thanks! Follow me on Twitter @JnBrymn

Check out the OpenSource Connection Blog

http://www.opensourceconnections.com/blog/


Cassandra Summit 2014: Understanding CQL3 Inside and Out

Technology

Transcript of Cassandra Summit 2014: Understanding CQL3 Inside and Out