C* Summit EU 2013: Cassandra Internals
-
Upload
planet-cassandra -
Category
Technology
-
view
1.004 -
download
3
description
Transcript of C* Summit EU 2013: Cassandra Internals
![Page 1: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/1.jpg)
CASSANDRA EU 2013
CASSANDRA INTERNALS
Aaron Morton @aaronmorton
!
Co-Founder & Principal Consultant www.thelastpickle.com
Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License #CassandraEU
![Page 2: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/2.jpg)
About The Last Pickle. Work with clients to deliver and improve
Apache Cassandra based solutions.
Apache Cassandra Committer, DataStax MVP, Hector Maintainer, Apache Usergrid Committer.
Based in New Zealand & Austin, TX.
#CassandraEUwww.thelastpickle.com
![Page 3: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/3.jpg)
Architecture Code
#CassandraEUwww.thelastpickle.com
![Page 4: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/4.jpg)
Cassandra Architecture.
API's
Cluster Aware
Cluster Unaware
Clients
Disk
#CassandraEUwww.thelastpickle.com
![Page 5: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/5.jpg)
Cassandra Cluster Architecture.
API's
Cluster Aware
Cluster Unaware
Clients
Disk
API's
Cluster Aware
Cluster Unaware
Disk
Node 1 Node 2
#CassandraEUwww.thelastpickle.com
![Page 6: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/6.jpg)
Dynamo Cluster Architecture.
API's
Dynamo
Database
Clients
Disk
API's
Dynamo
Database
Disk
Node 1 Node 2
www.thelastpickle.com #CassandraEU
![Page 7: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/7.jpg)
Architecture API
Dynamo Database
#CassandraEUwww.thelastpickle.com
![Page 8: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/8.jpg)
API Transports. !
Thrift Native Binary
!
#CassandraEUwww.thelastpickle.com
![Page 9: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/9.jpg)
Thrift Transport. !
//Custom TServer implementations o.a.c.thrift.CustomTThreadPoolServer o.a.c.thrift.CustomTHsHaServer
#CassandraEUwww.thelastpickle.com
![Page 10: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/10.jpg)
API Transports.
Thrift Native Binary
#CassandraEUwww.thelastpickle.com
![Page 11: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/11.jpg)
Native Binary Transport. !
Beta in Cassandra 1.2, now GA. Uses Netty. CQL 3 only.
#CassandraEUwww.thelastpickle.com
![Page 12: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/12.jpg)
o.a.c.transport.Server.run() !
//Setup the Netty server new ExecutionHandler() new NioServerSocketChannelFactory() ServerBootstrap.setPipelineFactory()
#CassandraEUwww.thelastpickle.com
![Page 13: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/13.jpg)
o.a.c.transport.Message.Dispatcher.messageReceived() !
//Process message from client ServerConnection.validateNewMessage() Request.execute() ServerConnection.applyStateTransition() Channel.write()
#CassandraEUwww.thelastpickle.com
![Page 14: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/14.jpg)
Messages. !
Defined in the Native Binary Protocol
$SRC/doc/native_protocol.spec
#CassandraEUwww.thelastpickle.com
![Page 15: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/15.jpg)
API Services. !
JMX Thrift
CQL 3 !
#CassandraEUwww.thelastpickle.com
![Page 16: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/16.jpg)
JMX Management Beans. !
Spread around the code base.
Interfaces named *MBean
#CassandraEUwww.thelastpickle.com
![Page 17: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/17.jpg)
JMX Management Beans. !
Registered with names such as org.apache.cassandra.db:
type=StorageProxy
#CassandraEUwww.thelastpickle.com
![Page 18: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/18.jpg)
API Services. !
JMX Thrift CQL 3
!
#CassandraEUwww.thelastpickle.com
![Page 19: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/19.jpg)
o.a.c.thrift.CassandraServer !
// Implements Thrift Interface // Access control // Input validation // Mapping to/from Thrift and internal types
#CassandraEUwww.thelastpickle.com
![Page 20: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/20.jpg)
Thrift Interface. !
Thrift IDL $SRC/interface/cassandra.thrift
#CassandraEUwww.thelastpickle.com
![Page 21: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/21.jpg)
o.a.c.thrift.CassandraServer.get_slice() !
// get columns for one row Tracing.begin() ClientState cState = state() cState.hasColumnFamilyAccess() multigetSliceInternal() !
#CassandraEUwww.thelastpickle.com
![Page 22: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/22.jpg)
CassandraServer.multigetSliceInternal() !
// get columns for may rows ThriftValidation.validate*() // Create ReadCommands getSlice() !
#CassandraEUwww.thelastpickle.com
![Page 23: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/23.jpg)
CassandraServer.getSlice() !
// Process ReadCommands // return Thrift types !
readColumnFamily() thriftifyColumnFamily() !
#CassandraEUwww.thelastpickle.com
![Page 24: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/24.jpg)
CassandraServer.readColumnFamily() !
// Process ReadCommands // Return ColumnFamilies !
StorageProxy.read() !
#CassandraEUwww.thelastpickle.com
![Page 25: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/25.jpg)
API Services. !
JMX Thrift
CQL 3 !
#CassandraEUwww.thelastpickle.com
![Page 26: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/26.jpg)
o.a.c.cql3.QueryProcessor !
// Prepares and executes CQL3 statements // Used by Thrift & Native transports // Access control // Input validation // Returns transport.ResultMessage
!
!
#CassandraEUwww.thelastpickle.com
![Page 27: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/27.jpg)
CQL3 Grammar. !
ANTLR Grammar $SRC/o.a.c.cql3/Cql.g
#CassandraEUwww.thelastpickle.com
![Page 28: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/28.jpg)
o.a.c.cql3.statements.ParsedStatement !
// Subclasses generated by ANTLR // Tracks bound term count // Prepare CQLStatement prepare()
#CassandraEUwww.thelastpickle.com
![Page 29: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/29.jpg)
o.a.c.cql3.statements.CQLStatement !
checkAccess(ClientState state) validate(ClientState state) execute(ConsistencyLevel cl, QueryState state, List<ByteBuffer> variables)
#CassandraEUwww.thelastpickle.com
![Page 30: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/30.jpg)
statements.SelectStatement.RawStatement !
// Implements ParsedStatement // Input validation prepare()
#CassandraEUwww.thelastpickle.com
![Page 31: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/31.jpg)
statements.SelectStatement.execute() !
// Create ReadCommands StorageProxy.read()
www.thelastpickle.com #CassandraEU
![Page 32: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/32.jpg)
Architecture API
Dynamo Database
#CassandraEUwww.thelastpickle.com
![Page 33: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/33.jpg)
Dynamo Layer. o.a.c.service
o.a.c.net !
o.a.c.dht o.a.c.gms
o.a.c.locator o.a.c.stream
#CassandraEUwww.thelastpickle.com
![Page 34: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/34.jpg)
o.a.c.service.StorageProxy !
// Cluster wide storage operations // Select endpoints & check CL available // Send messages to Stages // Wait for response // Store Hints
#CassandraEUwww.thelastpickle.com
![Page 35: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/35.jpg)
o.a.c.service.StorageService !
// Ring operations // Track ring state // Start & stop ring membership // Node & token queries
#CassandraEUwww.thelastpickle.com
![Page 36: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/36.jpg)
o.a.c.service.IResponseResolver !
preprocess(MessageIn<T> message) resolve() throws DigestMismatchException !
RowDigestResolver RowDataResolver RangeSliceResponseResolver
#CassandraEUwww.thelastpickle.com
![Page 37: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/37.jpg)
Response Handlers / Callback.
implements IAsyncCallback<T> !
response(MessageIn<T> msg) !
#CassandraEUwww.thelastpickle.com
![Page 38: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/38.jpg)
o.a.c.service.ReadCallback.get()
//Wait for blockfor & data response condition.await(timeout, TimeUnit.MILLISECONDS) !
throw ReadTimeoutException() !
resolver.resolve()
#CassandraEUwww.thelastpickle.com
![Page 39: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/39.jpg)
o.a.c.service.StorageProxy.fetchRows() !
getLiveSortedEndpoints() new RowDigestResolver() new ReadCallback() MessagingService.sendRR() --------------------------------------- ReadCallback.get() # blocking catch (DigestMismatchException ex) catch (ReadTimeoutException ex)
#CassandraEUwww.thelastpickle.com
![Page 40: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/40.jpg)
Dynamo Layer !
o.a.c.service o.a.c.net
!
o.a.c.dht o.a.c.gms
o.a.c.locator o.a.c.stream
#CassandraEUwww.thelastpickle.com
![Page 41: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/41.jpg)
o.a.c.net.MessagingService.verb<<enum>> !
MUTATION READ REQUEST_RESPONSE TREE_REQUEST TREE_RESPONSE
(And more...)
#CassandraEUwww.thelastpickle.com
![Page 42: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/42.jpg)
o.a.c.net.MessagingService.verbHandlers !
new EnumMap<Verb, IVerbHandler>(Verb.class)
#CassandraEUwww.thelastpickle.com
![Page 43: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/43.jpg)
o.a.c.net.IVerbHandler<T> !
doVerb(MessageIn<T> message, String id);
!
#CassandraEUwww.thelastpickle.com
![Page 44: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/44.jpg)
o.a.c.net.MessagingService.verbStages !
new EnumMap<MessagingService.Verb, Stage>(MessagingService.Verb.class)
#CassandraEUwww.thelastpickle.com
![Page 45: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/45.jpg)
o.a.c.net.MessagingService.receive() !
runnable = new MessageDeliveryTask( message, id, timestamp); !
StageManager.getStage( message.getMessageType()); !
stage.execute(runnable);
#CassandraEUwww.thelastpickle.com
![Page 46: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/46.jpg)
o.a.c.net.MessageDeliveryTask.run() !
// If dropable and rpc_timeout MessagingService.incrementDroppedMessages(v
erb); return; !
MessagingService.getVerbHandler(verb) verbHandler.doVerb(message, id)
#CassandraEUwww.thelastpickle.com
![Page 47: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/47.jpg)
Architecture API Layer
Dynamo Layer Database Layer
#CassandraEUwww.thelastpickle.com
![Page 48: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/48.jpg)
Database Layer !
o.a.c.concurrent o.a.c.db
!
o.a.c.cache o.a.c.io
o.a.c.trace
#CassandraEUwww.thelastpickle.com
![Page 49: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/49.jpg)
o.a.c.concurrent.StageManager !
stages = new EnumMap<Stage, ThreadPoolExecutor>(Stage.class); !
getStage(Stage stage)
#CassandraEUwww.thelastpickle.com
![Page 50: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/50.jpg)
o.a.c.concurrent.Stage !
READ MUTATION GOSSIP REQUEST_RESPONSE ANTI_ENTROPY
(And more...)#CassandraEUwww.thelastpickle.com
![Page 51: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/51.jpg)
Database Layer. o.a.c.concurrent
o.a.c.db !
o.a.c.cache o.a.c.io
o.a.c.trace
#CassandraEUwww.thelastpickle.com
![Page 52: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/52.jpg)
o.a.c.db.Table !
// Keyspace open(String table) getColumnFamilyStore(String cfName) !
getRow(QueryFilter filter) apply(RowMutation mutation, boolean writeCommitLog)
#CassandraEUwww.thelastpickle.com
![Page 53: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/53.jpg)
o.a.c.db.ColumnFamilyStore !
// Column Family getColumnFamily(QueryFilter filter) getTopLevelColumns(...) !
apply(DecoratedKey key, ColumnFamily columnFamily, SecondaryIndexManager.Updater indexer)
#CassandraEUwww.thelastpickle.com
![Page 54: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/54.jpg)
o.a.c.db.IColumnContainer !
addColumn(IColumn column) remove(ByteBuffer columnName) !
ColumnFamily SuperColumn !
(Removed in 2.0)
#CassandraEUwww.thelastpickle.com
![Page 55: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/55.jpg)
o.a.c.db.ISortedColumns !
addColumn(IColumn column, Allocator allocator) removeColumn(ByteBuffer name) !
ArrayBackedSortedColumns AtomicSortedColumns TreeMapBackedSortedColumns
#CassandraEUwww.thelastpickle.com
![Page 56: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/56.jpg)
o.a.c.db.Memtable !
put(DecoratedKey key, ColumnFamily columnFamily, SecondaryIndexManager.Updater indexer) !
flushAndSignal(CountDownLatch latch, Future<ReplayPosition> context)
#CassandraEUwww.thelastpickle.com
![Page 57: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/57.jpg)
o.a.c.db.ReadCommand !
getRow(Table table) !
SliceByNamesReadCommand SliceFromReadCommand RangeSliceCommand
(Additional classes for paging in 2.0)
#CassandraEUwww.thelastpickle.com
![Page 58: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/58.jpg)
o.a.c.db.IDiskAtomFilter !
getMemtableColumnIterator(...) getSSTableColumnIterator(...) !
IdentityQueryFilter NamesQueryFilter SliceQueryFilter
#CassandraEUwww.thelastpickle.com
![Page 59: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/59.jpg)
Summary CustomTThreadPoolServer Message.Dispatcher
CassandraServer QueryProcessor
ReadCommand
StorageProxy
IResponseResolver
IAsyncCallback
MessagingService
IVerbHandler
Table ColumnFamilyStore IDiskAtomFilter
API
Dynamo
Database
#CassandraEUwww.thelastpickle.com
![Page 60: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/60.jpg)
Thanks. !
#CassandraEUwww.thelastpickle.com
![Page 61: C* Summit EU 2013: Cassandra Internals](https://reader035.fdocuments.net/reader035/viewer/2022062703/554fb4fbb4c90586258b53c4/html5/thumbnails/61.jpg)
Aaron Morton @aaronmorton
www.thelastpickle.com !
Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License