My First 100 days with a Cassandra Cluster

My First 100 days with a Cassandra Cluster

Presented by : Gustavo René Antúnez

February, 2017

© 2016 Pythian. Confidential

ABOUT PYTHIAN

Pythian’s 400+ IT professionals help

companies adopt and manage disruptive technologies to better compete

2

TECHNICAL EXPERTISE

Infrastructure: Transforming and managing the IT infrastructure that supports the business

DevOps: Providing critical velocity in software deployment by adoptingDevOps practices

Cloud: Using the disruptive nature of cloud for accelerated, cost-effective growth

Databases: Ensuring databasesare reliable, secure, available and continuously optimized

Big Data: Harnessing the transformative power of data on a massive scale

Advanced Analytics: Mining data for insights & business transformationusing data science

3

© 2016 Pythian. Confidential 4

Welcome to RMOUG 2017

Where do I come From–OracleDBA

• StartedwithVersion9.2in2004– SpeakeratOracleOpenWorld,OracleDevelopersDayandCollaborate

–Co-PresidentofORAMEX(MexicoOracleUserGroup)

–WebEventsChairforIOUGCloudComputingSpecialInterestGroup(SIG)

– InternationalChairRACSpecialInterestGroup(SIG);

–MovieFanatic&MusicLover–BringingthebestfromMéxico(Mexihtli)totherestoftheworldandintheprocessphotographingit:)

– rene-ace.com–@rene_ace

• #TD16

5


Where do I come From

rene-ace.com@rene_ace• #TD17


How did you get to be a DBA


6th Happiest Job of 2015!

http://www.forbes.com/sites/susanadams/2014/03/20/the-happiest-and-unhappiest-jobs-in-2014/

Work-life balance

Relationship with boss and co-workers

Daily tasksJob resources

Field will grow by 15% between

2012 and 2022

DBA can be the key driver of

success


Happiest Job of 2034?• 47percentofAmericanjobsareathighriskofbeingtakenbycomputerswithinthenexttwodecades.

– 1stWave• Computerswillstartreplacingpeopleinespeciallyvulnerablefieldsliketransportation/logistics,productionlabor,andadministrativesupport.

– 2ndWave• Dependentuponthedevelopmentofgoodartificialintelligence.Thiscouldnextputjobsinmanagement,scienceandengineering,andtheartsatrisk.


The most important questionNormalizeorDenormalize

• Goalofnormalizationistostoreafactinoneplacetominimizeupdate,deleteandinsertanomalies*

• Normalizeddata,dependingonhowcomplextheschemabecomes,oftenaffectsqueryperformance

http://blog.rdx.com/cassandra-and-relational-database-schema-comparison-query-vs-relationship-modeling



The most important questionNormalizeorDenormalize

• Normalizetoreducedataanomaliesanddenormalizetoimprovequeryperformance.

• Inrelationalsystems,administratorsmodelthedata.InCassandra,administratorsdesignschemasthatarebasedonquerypatterns.

• Denormalizationprocessisthemergingofattributesthatareoftenaccessedtogetherintoasingleschemaobject




What is Cassandra ? • NoSQLdatabase,developedinJavaOne• FullydistributedDB

• MeaningthatthereisnomasterDB,unlikeOracleorMySQL.

• Linearlyscalable• Basedon2coretechnologies,Google’sBigTableandAmazon’sDynamo

• 2versionsofCassandra• CommunityEdition.-ThisisdistributedundertheApache™License

• EnterpriseEdition.-ThisisdistributedbyDatastax

≠


CAPTheorem

• Inadistributedsystemyoucanonlyhavetwooutofthefollowingthreeguaranteesacrossawrite/readpair:

• Consistency.-Areadisguaranteedtoreturnthemostrecentwriteforagivenclient.

• Availability.-Anon-failingnodewillreturnareasonableresponsewithinareasonableamountoftime(noerrorortimeout).

• PartitionTolerance.-Thesystemwillcontinuetofunctionwhennetworkpartitionsoccur.

N1 N2

X X

N1 N2

N1 N2

What is Cassandra ?

What is Cassandra ?


CAPTheorem

• Onefallacyofdistributedcomputingisthatnetworksarereliable

• AP-Availability/PartitionTolerance-Returnthemostrecentversionofthedatayouhave,whichcouldbestale.Willalsoacceptwritesthatcanbeprocessedlaterwhenthepartitionisresolved

• CP-Consistency/PartitionTolerance-Waitforaresponsefromthepartitionednodewhichcouldresultinatimeouterror.


What is Cassandra ?

CassandraisaBASE(BasicallyAvailable,Softstate,Eventuallyconsistent)typesystem

• NotanACID(Atomicity,Consistency,Isolation,Durability)typesystem

CassandraisclassifiedasanAPsystem


It Can be as easy as … • Startyourmachineandinstallthefollowing:

• ntp(Packagesarenormallyntp,ntpdataandntp-doc)

• wget(Unlessyouhaveyourpackagescopiedoverviaothermeans)

• vim(Oryourfavoritetexteditor)• YumPackageManagement• Rootorsudoaccesstotheinstallmachine• LatestversionofOracleJavaSERuntimeEnvironment(JRE)8(recommended)orOpenJDK7.

• Python2.6+(neededifinstallingOpsCenter)


It Can be as easy as … • InstallCassandra.~$ sudo yum install dsc21-2.1.5-1 cassandra2.1.5-1

• Installoptionalutilities.~$ sudo yum install cassandra21-tools-2.1.5-1

• StartCassandraservice~$ sudo service cassandra stop

~$ sudo rm -rf /var/lib/cassandra/data/system/*

• Inthecassandra-rackdc.propertiesfile#indicatetherackanddcforthisnodedc=Pythianrack=RAC1

~$ sudo service cassandra start


Where is everything in Cassandra?Directories Description/var/lib/cassandra Datadirectories/var/log/cassandra Logdirectory/var/run/cassandra Runtimefiles/usr/share/cassandra Environmentsettings/usr/share/cassandra/lib

JARfiles/usr/bin Optionalutilities,suchassstablelevelreset,

sstablerepairedset,andsstablesplit/usr/bin Binaryfiles/usr/sbin/etc/cassandra Configurationfiles/etc/init.d Servicestartupscript/etc/security/limits.d Cassandrauserlimits/etc/default/usr/share/doc/cassandra/examples

Samplecassandra.yamlfilesforstresstesting


I come from this world…

12cVersionArchitecture…


I come from this world…Oracle…

101010

Online Redo Log10100

Data Files Control Files

Segment

Database

Tablespace

Extent

Oracle data block

Schema Data file

OS block

Logical Datafile

Physical Datafile


I come from this world…RAC-ForNodePointofFailure

RAC Cluster

Node3Node2

ASM Disks

Node1

Public Network

Storage NetworkASM Network

CSS Network

ASM ASM ASM

DBB DBBDBB

GlobalDataServices–Service Failover / Load Balancing


I come from this world…Dataguard-ForFailover

Primary

Standby

FarSyncInstance

SYNCASYNC

Zerodatalossfailover


Cassandra Architecture

CassandraCluster

N1

Node

N2

Node

Rack1

DatacenterMéxico

N3

Node

N4

Node

Rack2

DatacenterPortugal


One Ring to Rule them All• Thetotalamountofdatamanagedbytheclusterisrepresentedasaring

• Eachnodeisassignedapartofthedatabasetoholdbasedoneachtable’sprimarykey.

• Toguaranteebothavailabilityanddurabilitymultiplenodeswillbeassignedtothesamedata.

• Thereisnomasternodeallnodescanperformalloperations

1

4

3

2

A-F,T-Z,M-S

G-L,A-F,T-Z

M-S,G-L,A-F

T-Z,M-S,G-L


Gossip• Peer-to-peercommunicationprotocolinwhichnodesperiodicallyexchangestateinformation

• Runseverysecondandexchangesstatemessageswithuptothreeothernodesinthecluster

• Failuredetection• Itdetermineslocallyfromgossipstateandhistoryifanothernodeinthesystemisdownorhascomebackup.


Consistent Hashing• Ahashconsistsofoneormorearithmeticoperationsonapieceofdata

• Commonwayofloadbalancingacrossseveralnodes

• Hashfunctionmusthaveaupperandlowerboundsoobjectscanbemappedinacircle

• CommonHashalgorithms– Simplechecksums– MessageDigest(MD5)– SecureHashAlgorithm(SHA-1/2)– MurmurHash


Partitioners• Determineshowdataisdistributedacrossthenodesinthecluster

• Functionforderivingatokenrepresentingarowfromitspartitionkeybyhashing.

CassandraOffers:– Murmur3Partition– RandomPartitioner– ByteOrderedPartitioner

(NotRecommended)


Coordinators

• Actsasaproxybetweentheclientapplicationandthenodesthatownthedatabeingrequested.

• Anyclientrequestcanbesenttoanynode.


Snitch• Isresponsibleforkeepingallofthenodesuptodateonwhatnodehaswhatdata,whatnodesarecurrentlydown,whatnodesarebootstrapping,etc.

• ItInterpretsthetopology

Themostpopularare:– Gossipingpropertyfile

snitch– EC2Snitch– EC2Multi-regionsnitch– DynamicSnitch


Logical database container DataisStoredinKeyspaces


Model Around Your Queries

• DetermineWhatQueriestoSupport• Groupingbyanattribute• Orderingbyanattribute• Filteringbasedonsomesetofconditions

• Createatablewhereyoucansatisfyyourquery• generallymeansyouwilluseroughlyonetableperquerypattern


A Cassandra Table or Column Family

CoordinatorSnitchCommitlogWriterMemtablewriterMemTableFlush(Sstablewriter)ReaderMemtablesBloomFilters

CassandraNodeCommitLog

10100

SSTables


A Cassandra Table or Column Family• ConsistsofoneormoreSStablesand0ormoreMEMtables

• SStablestandsforSortedStringTable.• E.G.alloftheColumnsintheSStablearesortedinorderbykey.

• EachSStableconsistsofthedatatable,bloomfilter,indexandsomeotherminorfiles.

• SStablesareimmutable.Oncewrittentheyareneveralteredonlyreadandeventuallydeleted

videogames-events-data-jb-1.dbvideogames-events-filters-jb-1.dbvideogames-events-index-jb-1.dbvideogames-events-data-jb-2.dbvideogames-events-filters-jb-2.dbvideogames-events-index-jb-2.dbvideogames-events-data-jb-3.dbvideogames-events-filters-jb-3.dbvideogames-events-index-jb-3.dbvideogames-events-data-jb-4.dbvideogames-events-filters-jb-4.dbvideogames-events-index-jb-4.db

SStablesondisk/var/lib/cassandra


Replication Factor (RF) and Consistency

• ReplicationFactoristhenumberofcopiesofcolumnsstoredinthering

• Replicationfactorshouldnotexceedthenumberofnodesinthecluster

– RF=1isonecopythismeansthatthedataforeachcolumnisstoredonlyonceinthering.

– RF=3(default)meanseverycolumnstoredinthedatabaseisstoredthreetimes.

– Quorum.-Thereadandwritemustbeacked/returnedfromaquorumofnodes.


Replication Factor (RF) and Consistency

• Consistency– Whenwriteorreadis

performedtheapplicationcanchoosetowaitforncopiesofthedatatobewrittenorreadthisisreferredtoasconsistencyofn.

– ThereisaspecialconsistencyvaluecalledquorumwhichmeansaresponsefromRF/2+1nodesisrequired.


How to make sure we don’t loose data

• Threeanti-entropymechanismsinCassandra1)Hintedhandoff2)Readrepair3)Repair

A.K.A.Anti-Entropy


Write Path


Compactions

• SStablesareimmutable.• Deletesandupdatesarejustnew

writes• SStablesaremergedtogetherby

partitionedkey.Oldobsoletedataisdiscarded.

• LotsofSStablesbecomeafew.• Compactioncanrequirealotof

diskspace.DONOTLETyourdisksgetmorethan50%full.


CQL - Cassandra Query LanguageCQLisnotSQL

• DefaultandprimaryinterfaceintotheCassandraDatabase(since2.0)• Cassandradoesnotsupportjoinsorsubqueries• Onlywaytocreateusersanduserbasedpermissions

• Verysimilar:cqlsh> CREATE KEYSPACE sandbox WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy', DC1 : 1}; cqlsh> USE sandbox; cqlsh:sandbox>CREATE TABLE data (id uuid, data text, PRIMARY KEY (id)); cqlsh:sandbox> INSERT INTO data (id, data) values (c37d661d-7e61-49ea-96a5-68c34e83db3a, 'testing'); cqlsh:sandbox> SELECT * FROM data;


Feature/Function DSE/Cassandra OracleRDBMSCore architecture “Masterless”; peer-to-peer with

all nodes being the same Traditional standalone

High availability Continuous availability with built in redundancy and hardware rack awareness in both single and multiple data centers

Oracle Dataguard (for failover) and Oracle RAC (Node SPOF) GoldenGate

Data model Google Bigtable Relational/tabular Data consistency model Tunable consistency (CAP

theorem consistency per operation

Traditional ACID

Storage model Targeted directories with separation

Tablespaces

Logical database container

Keyspace Database

Backup/recovery Online, point-in-time restore Online, point-in-time restore

Enterprise management/monitoring

DataStax OpsCenter Oracle Enterprise Manager


Lessons Learned

• UnderstandtheDataModelDifferences• HardwareSetupdoesMatter• Grepthelogsforerrorsandwarnings• Makesureeachnodeiscreatedproperly• Knowyourtools

• nodetoolutility• Cassandrabulkloader(sstableloader)• jconsole/JavaVisualVM• Cassandra-Stress• OpsCenter


rene-ace.com

http://rend-ace.com

Thank you – Q&A CONSULTING & STRATEGY

IMPLEMENTATIONMANAGED SERVICES

To contact us [email protected]

1-877-PYTHIAN

To follow us http://www.pythian.com/blog

http://goo.gl/bImXcJ

@pythian

http://goo.gl/DMXExf

http://goo.gl/DMXExf

My First 100 days with a Cassandra Cluster

Technology

Transcript of My First 100 days with a Cassandra Cluster