Copyright©2016SplunkInc.

DhruvaKumarBhagiSr.SoftwareengineerSplunkInc.

Replicationofsummarydatainindexercluster

Disclaimer

2

Duringthecourseofthispresentation,wemaymakeforwardlookingstatementsregardingfutureeventsortheexpectedperformanceofthecompany.Wecautionyouthatsuchstatementsreflectourcurrentexpectationsandestimatesbasedonfactorscurrentlyknowntousandthatactualeventsorresultscoulddiffermaterially.Forimportantfactorsthatmaycauseactualresultstodifferfromthose

containedinourforward-lookingstatements,pleasereviewourfilingswiththeSEC.Theforward-lookingstatementsmadeinthethispresentationarebeingmadeasofthetimeanddateofitslivepresentation.Ifreviewedafteritslivepresentation,thispresentationmaynotcontaincurrentoraccurateinformation.Wedonotassumeanyobligationtoupdateanyforwardlookingstatementswemaymake.Inaddition,anyinformationaboutourroadmapoutlinesourgeneralproductdirectionandissubjecttochangeatanytimewithoutnotice.Itisforinformationalpurposesonlyandshallnot,beincorporatedintoanycontractorothercommitment.Splunkundertakesnoobligationeithertodevelopthefeaturesor

functionalitydescribedortoincludeanysuchfeatureorfunctionalityinafuturerelease.

Indexerclustertopology

3

MasterSearchhead

FORWARDERS

Indexer Indexer IndexerForReplication

Forgenerationinfo

ForSearch

Forwardingdatatoindexers

Master-slavecommunication

Forreplication

Searchhead-idx

Summaries

4

Summaries arehigherlevelrepresentationofthesearchresultsResideonindexertohelpacceleratetheexecutionoflongrunningsearchesoriginatingfromsearchheadsApplicableforstatisticalsearcheslike‘index=main|statscount’

Summaries

5

Twotypesofsummaries:• Reportaccelerationsummaries– Generatedbyacceleratedreports/searches• Datamodelsummaries– Generatedbyaccelerateddatamodelpivotsearches

ResideunderINDEX/{datamodel_summary|summary}/BID/SH_ID/summary_name/…Note:summariesaredifferentfromsummaryindex

Problem

6

SummarieswerepreviouslyneverreplicatedIfanindexerwithasummarygoesdown,bucketprimariesmovetoanothersearchablecopy,andsearcheswillnothaveaccesstothesummaries(untiltheygetregenerated),therebysearchesrunslow.Regeneratingsummariescantakelotoftime

Replicatingsummaries

7

Fromversion6.4,SplunkcanreplicateDA&RMsummariesinindexerclusterenvironmentPeersmaintainthelistoftheirsummaries(justlikebuckets)andreportsthemtoCMduringregistration(or)onanystatechangeIfanypeerismissingasummaryforabucket,masterschedulesasummaryreplicationwiththispeerasareplicationtargetOnlyreplicatesummariesforwarm/coldbuckets(nothot)services/cluster/master/buckets/BUKCETlistssummariesofeachbucket

Summariesforhotbuckets

8

Forhotbuckets,summarysearchesfromsearchheadshitallthesearchablecopies(insteadofjustprimariesfornormalsearches)therebylettingindexersbuildthesummariesindividuallyforhotbuckets.Noneedforsummaryregenerationevenonindividualnodefailures.Runningsummarygeneratingsearchesonallsearchablecopiesofahotbucketintroducesmoreresourceusage(longersearchprocess,extramemoryusage)!

VolumeRetention

9

Summariescanbesetupinavolumetomanagespaceusage

Ifindexerexceedsthevolumespace(maxVolumeDataSizeMB),splunkstartstrimmingsummaries– Atrimmedsummaryleavestombstoneindicatingthetrimmedstate– Tombstoneavoidsregeneratingsummariesinthesubsequentsearchruns– Oncewetrimasummary,wepropagatethisinfototheCM,andtheCMwill

issueatrimtoallothercopiesofthesummary.

Configuration

10

Toturnonsummaryreplication,makesummary_replication=trueunderclusteringstanzaonclustermaster.Bydefaultsummaryreplicationisturnedoff.max_peer_sum_rep_load (defaultsto5)configureshowmanymaximumsummaryreplicationsperpeerConfigchangesarereloadable(i.e.doesnotrequireasplunkrestart)

THANKYOU

Copyright©2016SplunkInc.

````

Wednesday,Sep283:00PM- 3:15PM