Copyright©2016Splunk Inc.

AlexJamesPrincipalProductManager,Splunk

SearchOptimizationKarthikSabhanatarajanSeniorSoftwareEngineer,Splunk

&

Demo

A B C D L EA B C D L EA B C D LA B C D LA B C D

A B C D L EA B C D L EA B C D LA B C D

Motivation:Taleoftwosearches

3

searchSourceType lookupL evalE searchA&L&E

Diskrawdata&index

searchSourceType&A lookupL searchL evalE searchE

• 10,000,000indexhits• 1,000,000eventscreated(i.e.extractions)• 1,000,000lookups• 1,000,000evals• 1,000,000filters• Produces100,000matchingevents

• 7,000,000fewerindexhits• 700,000 fewereventscreated• 700,000fewerlookups• 800,000fewerevals• Net500,000lessfilters• ProducesIDENTICAL100,000matchingevents

TOTALWORK

SAVINGS

OptimizationPrinciplesDoaslittleworkaspossible– Retrieveonlytherequireddata– Moveaslittledataaspossible– Parallelizeasmuchworkaspossible– Setappropriatetimewindows

ImplicationsbasedonSplunkArchitecture– Filterasmuchaspossibleintheinitialsearch– Join/Lookuponlyonrequireddata– Evalontheminimumnumberofeventspossible– Delaycommandsthatbringdatatothesearchheadasmuchaspossible.

4

NewinSplunk6.5

Howsearchworksin6.4.

6

searchsourcetype=access-*(status=401orstatus=403)|lookup usertogroupuserOUTPUTgroup|where src_category=“email_server”

search lookup

1)Spliton‘|’andcreateprocessorpipeline

2)Distributebetweenindexandsearchheads,passargumentsandexecute

search lookup whereIndexer1

search lookup whereIndexer2

combine

Searchhead

where

Howsearchworksin6.5.

7

searchsourcetype=access-*(status=401orstatus=403)|lookup usertogroupuserOUTPUTgroup|where src_category=“email_server”

search lookup

1)ParseintoAST

search lookupIndexer1

search lookupIndexer2combine

Searchhead

searchsourcetype=access-*(status=401orstatus=403)src_category=“email_server”|lookup usertogroupuserOUTPUTgroup

2)OptimizeAST

3)ConstructPipelinefromAST

JSONAST

OptimizedJSONAST

4)Distributebetweenindexandsearchheads,passargumentsandexecute

Demo

Whatoptimizationsaredone?

9

Pushingpredicatestotheleft(ordown)– For*any*streamingcommandsthatdon’tmodifyafield:

– |rangemap field=scoreF=0-64D=65-69C=70-79B=80-89A=90-100|wherehost=mail30– |wherehost=mail30|rangemap field=scoreF=0-64D=65-69C=70-79B=80-89A=90-100

– SpecialHandlingforsomecommands:ê Rename– |renamesrc asip |whereip=“192.1.2.13”– |wheresrc=“192.1.2.13”|renamesrc asip

ê Eval– |evalsrc=if(isnull(src)ORsrc=“”,“unknown”,src |wheresrc =“192.1.2.13”– |wheresrc =“192.1.2.13”|evalsrc=if(isnull(src)ORsrc=“”,“unknown”,src

ê Byclausefilters– |statscountbyclientip|searchclientip=“192.0.0.0/8”– |searchclientip=“192.0.0.0/8”|statscountbyclientip

Search/Wheremerging– searchERROR|search404|wheresourcetype=“windows”– searchERROR404sourcetype=“windows”

Whatoptimizationsarecominglater?

10

PredicateSplittingPredicateNormalizationCollapsingconsecutivecommandsConvertingEvalFunctionsintoSearchfiltersifpossibleProjectionEliminationRe-usingprevioussearchresults

Whatdoesthismeanforyou?

11

FasterSearchesUpgradeto6.5Scanfor‘inefficientsearches’– Especiallyinscheduledworkloads...

UsetheJobInspectortoseeoptimizationinactionOptimizefurthermanuallyifneeded

Q&A

12

Disclaimer

14

Duringthecourseofthispresentation,wemaymakeforwardlookingstatementsregardingfutureeventsortheexpectedperformanceofthecompany.Wecautionyouthatsuchstatementsreflectourcurrentexpectationsandestimatesbasedonfactorscurrentlyknowntousandthatactualeventsorresultscoulddiffermaterially.Forimportantfactorsthatmaycauseactualresultstodifferfromthose

containedinourforward-lookingstatements,pleasereviewourfilingswiththeSEC.Theforward-lookingstatementsmadeinthethispresentationarebeingmadeasofthetimeanddateofitslivepresentation.Ifreviewedafteritslivepresentation,thispresentationmaynotcontaincurrentoraccurateinformation.Wedonotassumeanyobligationtoupdateanyforwardlookingstatementswemaymake.Inaddition,anyinformationaboutourroadmapoutlinesourgeneralproductdirectionandissubjecttochangeatanytimewithoutnotice.Itisforinformationalpurposesonlyandshallnot,beincorporatedintoanycontractorothercommitment.Splunkundertakesnoobligationeithertodevelopthefeaturesor

functionalitydescribedortoincludeanysuchfeatureorfunctionalityinafuturerelease.

MigratingSlidesforMac1. Forbestresults,simplypasteyourslidesintothis

template.

2. ApplyslidelayoutsusingtheLayout buttonundertheFormattab.

3. IfLayoutstilldoesnotreflectthedesiredMasterLayout,chooseResetLayouttoDefaultsettings.

4. Deleteunwantedtemplateslides(anyslidesafterLastSlide).

5. ChooseSaveAstosavethefilewithoutoverwritingthetemplate.

15

MigratingSlidesforPC1. Forbestresults,simplypasteyourslidesintothistemplate.

– Pastingafterabulletslideisrecommended

2. Reviewallslidesandmakeformattingadjustmentsasneeded– OntheHome ribbon,clickLayout andselectthecorrectslidelayout– ClickReset toresetallslideelementstothedefaultsizeandposition– Checkforhiddentext,suchaswhitetextonawhitebackground

3. Deleteunnecessarytemplateslides4. SaveAstosavethefilewithoutoverwritingthetemplate

16

SlideMasters• Whenimportingslidesfromanotherpresentation,the

SlideMastersassociatedwiththoseslidesmayalsoimporttothistemplate.Thisisa‘feature’ofPPTandcannotbeturnedoff.

• TodeleteunwantedSlideMasters:– makesureallslidesinthepresentationhavethenew

templateSlideMasterLayoutsassigned(first16SlideMastersshownunderLayout)

– GotoView/MastertodeleteanyunwantedSlideMasters

• ThelastSlideMasterinthistemplateiscalledLastSlide.AnySlideMastersafterthisslidewerelikelyimportedfromanotherpresentationandcanbedeleted(ifnolongerusedbyanyslides.)

17

ImportantTips• Thistemplateusesareducedslidesize.Youmayhavetomanuallydecreasethesizeofsomeitemssuchasstrokes andfonts.

• Iffontsappearbiggerthandesired,remembertoassignaLayout toyourslideandResettoDefaultSettings.

• Ifpagenumbersdonotappearorarethewrongformatting,remembertoassignaLayout toyourslideandResettoDefaultSettings.

• Thecolorsinyourgraphicswillautomaticallybeshiftedtothenewpalette.Pleaseadjustasneeded.

18

Agenda

AgendaItemAgendaItemAgendaItem

19

2012GoalsandObjectivesExample

GoalItemGoalItemGoalItem

20

SampleTitle,66pt.Calibri

21

SampleTitle,66pt.Calibri

22

Subhead

TitleOnlySlide,60pt.Calibri

23

TitleOnlySlide,54pt.Calibri

24

Samplewithscreenshot

25

Screenshothere

SampleTwo-columnFormatSubhead

26

Sampletwo-columnformat

• Sampletwo-columnformat,sentence– Secondbullet

Sampletwo-columnformat

SplunkObjectStyleandColor

27

Hardware ProductBusiness/Corporate

HighlightOnlyGenericVirtualization

Generic

Thesearesuggestedusesforcolorsonly.

AssignDefaultObjectStyle

28

ApplyingSplunkObjectStyle

29

ToapplytheSplunk objectstyletoanyshape:1. Selecttheshapewiththedesiredstyle2. ClickonFormatPainter(paintbrush)toolintoolbar3. Applystyletoanynewshape

Logos

30

CorporateLogo ProductLogo

Logos

31

Logos

32

Splunk Icons

33

search barchart lock cloud opencloud checkmark envelope

storage- 3storageiPhoneiPadandroid

server indexer forwarder searchhead desktop laptop

datacenter

Splunk server

firewall

Splunk IconsCont’d

34

application virtualmachine virtualserver network wwworglobal tools

logfile RFID router loadbalancer script shoppingcart

user users gears/settings gear messaging tag/ticket

document

alert

gps tower

Splunk Icons

35

Checkmark InfoAlert StopiPhoneiPadAndroid

Twitter Facebook LinkedIn RSS YouTube ShoppingcartGPSTower

Healthcare Hospital Officebuilding VoIPPhone Support POSCardReader RFID

Splunk Icons

36

SecurityIcons

37

FirewallAttacker,Generic

Attacker,Insider

Attacker,Nation/State

Botnet Key

Malware MalwareDocument

MalwarePackaged

SecurityBadge

SecurityServer

Shield VirusFootsteps

TheInternetofThingsIcons

38

POSCardReader

RFIDElectricCar

EMVReaderInternetofThings Meter Factory

SignatureCapture

Arrows

39

TableExample

40

ColumnTitle ColumnTitle ColumnTitle ColumnTitle

Text Text Text Text

Text Text Text Text

Text Text Text Text

Text Text Text Text

Text Text Text Text

TableExample

41

ColumnTitle ColumnTitle ColumnTitle ColumnTitle

Text Text Text Text

Text Text Text Text

Text Text Text Text

Text Text Text Text

Text Text Text Text

SampleCustomerSuccess

42

Customerlogohere

CustomernameCustomercompany

“SplunkmakesitcheaperandeasierforHughestoanalyzenetworktrafficforenterprisecustomersaswellasmanagebandwidthforconsumerandsmallbusinesscustomers.”

BulletplaceholderBulletplaceholderBulletplaceholder

Screenshotorgraphichere

TimelineChart

43

Q1 Q2 Q3 Q4

Milestone Event

ChartExample

44

PlannedActual

Number

Number

Number

Number

Number

FY09 FY10FY08PreviousYear

N%growthoverFYxx

QuoteBox

45

“Apessimistseesthedifficultyineveryopportunity;anoptimistseestheopportunityineverydifficulty.”

-WinstonChurchill

QuoteBox

46

QuoteBox

47

WhatNow?

48

Relatedbreakoutsessionsandactivities…

THANKYOU