Cassandra Community Webinar - Introduction To Apache Cassandra 1.2

download Cassandra Community Webinar  - Introduction To Apache Cassandra 1.2

of 93

  • date post

    27-Jun-2015
  • Category

    Technology

  • view

    553
  • download

    2

Embed Size (px)

description

Slides from the April 25th presentation http://www.youtube.com/watch?v=zFCjekgK7ZY

Transcript of Cassandra Community Webinar - Introduction To Apache Cassandra 1.2

  • 1. CASSANDRA COMMUNITY WEBINARS APRIL 2013INTRODUCTION TOAPACHE CASSANDRA 1.2Aaron MortonApache Cassandra Committer, Data Stax MVP for Apache Cassandra@aaronmortonwww.thelastpickle.comLicensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License

2. Cassandra Summit 2013June 11 & 12San FranciscoUse SFSummit25 for 25% off 3. Cassandra Summit 2013DataStax Ac*ademyFree certication during thesummit. 4. OverviewThe ClusterThe NodeThe Data Model 5. Cassandra- Started at Facebook- Open sourced in 2008- Top Level Apache projectsince 2010. 6. Used by...Netix,Twitter, Reddit,Rackspace... 7. Inspiration- Google Big Table (2006)- Amazon Dynamo (2007) 8. Why Cassandra?- Scale- Operations- Data Model 9. OverviewThe ClusterThe NodeThe Data Model 10. Store foo key with Replication Factor 3.Node 1 - fooNode 2 - fooNode 4Node 3 - foo 11. Consistent Hashing.- Evenly map keys to nodes- Minimise key movementswhen nodes join or leave 12. Partitioner.RandomPartitionertransforms Keys to Tokensusing MD5.(Default pre version 1.2.) 13. Partitioner.Murmur3Partitionertransforms Keys to Tokensusing Murmur3.(Default in version 1.2.) 14. Keys andTokens?token 0 99key fop foo10 90 15. Token Ring.footoken: 90foptoken: 1099 0 16. Token Ranges pre v1.2.Node 1token: 0Node 2token: 25Node 4token: 75Node 3token: 501-2576-0 17. Token Ranges withVirtual Nodes in v1.2.Node 1Node 2Node 3Node 4 18. LocateToken Range.Node 1Node 2Node 3Node 4footoken: 90 19. Replication Strategy selectsReplication Factor number ofnodes for a row. 20. SimpleStrategy with RF 3.Node 1Node 2Node 3Node 4footoken: 90 21. NetworkTopologyStrategy usesa Replication Factor per DataCentre.(Default.) 22. Multi DC Replication with RF 3 and RF 2.Node 1Node 2Node 3Node 4footoken: 90Node 1Node 2Node 3Node 4West DC East DC 23. The Snitch knows which DataCentre and Rack the Node isin. 24. SimpleSnitch.Places all nodes in the sameDC and Rack.(Default, there are others.) 25. EC2Snitch.DC is set to AWS Region anda Rack to Availability Zone. 26. The Client and the Coordinator.Node 1Node 2Node 3Node 4footoken: 90Client 27. Multi DC Client and the Coordinator.Node 1Node 2Node 3Node 4footoken: 90ClientNode 10Node 20Node 30Node 40 28. Gossip.Nodes share information witha small number of neighbours.Who share information with asmall number of neigh.. 29. Consistency Level (CL).- Specied for each request- Number of nodes to waitfor. 30. Consistency Level (CL)- Any*- One,Two Three- QUORUM- LOCAL_QUORUM, EACH_QUOURM* 31. QUOURM at Replication Factor...ReplicationFactorQUOURM2 or 3 4 or 5 6 or 72 3 4 32. Write foo at QUOURM with Hinted Handoff.Node 1Node 2Node 3Node 4foo for #3footoken: 90Client 33. Read foo at QUOURM.Node 1Node 2Node 3Node 4footoken: 90Client 34. Column Timestampsused to resolvedifferences. 35. Resolving differences.Column Node 1 Node 2 Node 3purplecromulent(timestamp 10)cromulent(timestamp 10)monkeyembiggens(timestamp 10)embiggens(timestamp 10)debigulator(timestamp 5)dishwashertomato(timestamp 10)tomato(timestamp 10)tomacco(timestamp 15) 36. Consistent read for foo at QUOURM.Node 1Node 2Node 3Node 4ClientcromulentcromulentNode 1Node 2Node 3Node 4Clientcromulent cromulent 37. Strong ConsistencyW + R > N(#Write Nodes + #Read Nodes> Replication Factor) 38. Achieving Strong Consistency.- QUOURM Read + QUORUM Write- ALL Read + ONE Write- ONE Read + ALL Write 39. Achieving Consistency- Consistency Level- Hinted Handoff- Read Repair- Anti Entropy 40. OverviewThe ClusterThe NodeThe Data Model 41. Optimised forWrites. 42. Write pathAppend to WriteAhead Log.(fsync every 10s by default, other options available) 43. Write path...Merge Columnsinto Memtable.(Lock free, always in memory.) 44. (Later.)Asynchronously ushMemtable to new les.(May be 10s or 100s of MB in size.) 45. Data is stored inimmutable SSTables.(Sorted String table.) 46. SSTable les.*-Data.db*-Index.db*-Filter.db(And others) 47. SSTables.SSTable 1foo:dishwasher (ts 10):tomatopurple (ts 10):cromulentSSTable 2foo:frink (ts 20):ayvenmonkey (ts 10):embigginsSSTable 3 SSTable 4foo:dishwasher (ts 15):tomaccoSSTable 5 48. Read purple, monkey, dishwasher.SSTable 1-Data.dbfoo:dishwasher (ts 10):tomatopurple (ts 10):cromulentSSTable 2-Data.dbfoo:frink (ts 20):ayvenmonkey (ts 10):embigginsSSTable 3-Data.db SSTable 4-Data.dbfoo:dishwasher (ts 15):tomaccoSSTable 5-Data.dbBloom FilterIndex SampleSSTable 1-Index.dbBloom FilterIndex SampleSSTable 2-Index.dbBloom FilterIndex SampleSSTable 3-Index.dbBloom FilterIndex SampleSSTable 4-Index.dbBloom FilterIndex SampleSSTable 5-Index.dbMemoryDisk 49. Key Cache caches row keyposition in *-Data.db le.(Removes up to1disk seek per SSTable.) 50. Read with Key Cache.SSTable 1-Data.dbfoo:dishwasher (ts 10):tomatopurple (ts 10):cromulentSSTable 2-Data.dbfoo:frink (ts 20):ayvenmonkey (ts 10):embigginsSSTable 3-Data.db SSTable 4-Data.dbfoo:dishwasher (ts 15):tomaccoSSTable 5-Data.dbKey CacheIndex SampleSSTable 1-Index.dbKey CacheIndex SampleSSTable 2-Index.dbKey CacheIndex SampleSSTable 3-Index.dbKey CacheIndex SampleSSTable 4-Index.dbKey CacheIndex SampleSSTable 5-Index.dbMemoryDiskBloom Filter Bloom Filter Bloom Filter Bloom Filter Bloom Filter 51. Row Cache caches entire row.(Removes all disk IO.) 52. Read with Row Cache.Row CacheSSTable 1-Data.dbfoo:dishwasher (ts 10):tomatopurple (ts 10):cromulentSSTable 2-Data.dbfoo:frink (ts 20):ayvenmonkey (ts 10):embigginsSSTable 3-Data.db SSTable 4-Data.dbfoo:dishwasher (ts 15):tomaccoSSTable 5-Data.dbKey CacheIndex SampleSSTable 1-Index.dbKey CacheIndex SampleSSTable 2-Index.dbKey CacheIndex SampleSSTable 3-Index.dbKey CacheIndex SampleSSTable 4-Index.dbKey CacheIndex SampleSSTable 5-Index.dbMemoryDiskBloom Filter Bloom Filter Bloom Filter Bloom Filter Bloom Filter 53. Compaction merges truth frommultiple SSTables into oneSSTable with the same truth.(Manual and continuous background process.) 54. Compaction.Column SSTable 1 SSTable 2 SSTable 4 Newpurplecromulent(timestamp 10)(timestamp 15)(timestamp 15)monkeyembiggens(timestamp 10)embiggens(timestamp 10)dishwashertomato(timestamp 10)tomacco(timestamp 15)tomacco(timestamp 15) 55. OverviewThe ClusterThe NodeThe Data Model 56. Cassandra is good atreading data from a row in theorder it is stored. 57. Typically an efcient data model willdenormalize data and use thestorage engine order. 58. To create a good data modelunderstand the queries yourapplication requires. 59. API ChoiceThriftOriginal and still fullysupported API. 60. API ChoiceCQL3New and fully supported API. 61. CQL 3A Table Orientated, SchemaDriven, Data Model andQuery language similar toSQL. 62. CQL 3A Table Orientated, SchemaDriven, Data Model andQuery language similar toSQL. 63. Twitter cloneusing CQL 3 via the cqlshtool.bin/cqlsh 64. Queries?- Post Tweet to Followers- Get Tweet by ID- List Tweets by User- List Tweets in User Timeline- List Followers 65. KeyspaceA Namespace container. 66. Our KeyspaceCREATE KEYSPACEcass_communityWITH replication ={class:NetworkTopologyStrategy,datacenter1:1}; 67. TableA sparse collection of wellknown, ordered columns. 68. FirstTableCREATE TABLE User(user_name text,password text,real_name text,PRIMARY KEY (user_name)); 69. Some userscqlsh:cass_community> INSERT INTO User... (user_name, password, real_name)... VALUES... (fred, sekr8t, Mr Foo);cqlsh:cass_community> select * from User;user_name | password | real_name-----------+----------+-----------fred | sekr8t | Mr Foo 70. Some userscqlsh:cass_community> INSERT INTO User... (user_name, password)... VALUES... (bob, pwd);cqlsh:cass_community> select * from User where user_name =bob;user_name | password | real_name-----------+----------+-----------bob | pwd | null 71. Data Model (so far)Table /ValueUseruser_name Primary Key 72. TweetTableCREATE TABLE Tweet(tweet_id bigint,body text,user_name text,timestamp timestamp,PRIMARY KEY (tweet_id)); 73. TweetTable...cqlsh:cass_community> INSERT INTO Tweet... (tweet_id, body, user_name, timestamp)... VALUES... (1, The Tweet,fred,1352150816917);cqlsh:cass_community> select * from Tweet where tweet_id = 1;tweet_id | body | timestamp | user_name----------+-----------+--------------------------+-----------1 | The Tweet | 2012-11-06 10:26:56+1300 | fred 74. Data Model (so far)Table /ValueUser Tweetuser_name Primary Key Fieldtweet_id Primary Key 75. UserTweetsTableCREATE TABLE UserTweets(tweet_id bigint,user_name text,body text,timestamp timestamp,PRIMARY KEY (user_name, tweet_id)); 76. UserTweetsTable...cqlsh:cass_community> INSERT INTO UserTweets... (tweet_id, body, user_name, timestamp)... VALUES... (1, The Tweet,fred,1352150816917);cqlsh:cass_community> select * from UserTweets whereuser_name=fred;user_name | tweet_id | body | timestamp-----------+----------+-----------+--------------------------fred | 1 | The Tweet | 2012-11-06 10:26:56+1300 77. UserTweetsTable...cqlsh:cass_community> select * from UserTweets whereuser_name=fred and tweet_id=1;user_name | tweet_id | body | timestamp-----------+----------+-----------+--------------------------fred | 1 | The Tweet | 2012-11-06 10:26:56+1300 78. UserTweetsTable...cqlsh:cass_community> INSERT INTO UserTweets... (tweet_id, body, user_name, timestamp)... VALUES... (2, Second Tweet, fred, 1352150816918);cqlsh:cass_community> select * from UserTweets where user_name = fred;user_name | tweet_id | body | timestamp-----------+----------+--------------+--------------------------fred | 1 | The Tweet | 2012-11-06 10:26:56+1300fred | 2 | Second Tweet | 2012-11-06 10:26:56+1300 79. UserTweetsTable...cqlsh:cass_community> select * from UserTweets where user_name = fred order bytweet_id desc;user_name | tweet_id | body | timestamp-----------+----------+--------------+--------------------------fred | 2 | Second Tweet | 2012-11-06 10:26:56+1300fred | 1 | The Tweet | 2012-11-06 10:26:56+1300 80. UserTimelineCREATE TABLE UserTimeline(user_name text