SplunkLive Sydney Machine Learning & Analytics

Copyright©2016SplunkInc.

MachineLearningAndrewPhillipsSr.SalesEngineer

3

DisclaimerDuringthecourseofthispresentation,wemaymakeforwardlookingstatementsregardingfuture

eventsortheexpectedperformanceofthecompany.Wecautionyouthatsuchstatementsreflectourcurrentexpectationsandestimatesbasedonfactorscurrentlyknowntousandthatactualeventsorresultscoulddiffermaterially.Forimportantfactorsthatmaycauseactualresultstodifferfromthose

containedinourforward-lookingstatements,pleasereviewourfilingswiththeSEC.Theforward-lookingstatementsmadeinthethispresentationarebeingmadeasofthetimeanddateofitslivepresentation.Ifreviewedafteritslivepresentation,thispresentationmaynotcontaincurrentoraccurateinformation.

Wedonotassumeanyobligationtoupdateanyforwardlookingstatementswemaymake.

Inaddition,anyinformationaboutourroadmapoutlinesourgeneralproductdirectionandissubjecttochangeatanytimewithoutnotice. Itisforinformationalpurposesonlyandshallnot,beincorporatedintoanycontractorothercommitment.Splunkundertakesnoobligationeithertodevelopthefeatures

orfunctionalitydescribedortoincludeanysuchfeatureorfunctionality inafuturerelease.


WhydoweneedML?

5

MLinEverydaylife


HistoricalData Real-timeData StatisticalModels

DB,Hadoop/S3/NoSQL, Splunk MachineLearning

T– afewdays T+afewdays

Whyisthissochallengingusingtraditionalmethods?

• DATAISSTILLINMOTION,stillinaBUSINESS PROCESS.• Enrichreal-timeMACHINEDATAwithstructuredHISTORICALDATA• Make decisionsINREALTIME usingALLTHEDATA• CombineLEADINGandLAGGINGINDICATORS (KPIs)

Splunk

SecurityOperationsCenter

NetworkOperationsCenter

BusinessOperationsCenter


WhatisML?

8

ML101:Whatisit?• MachineLearning(ML)isaprocessforgeneralizingfromexamples

– Examples=exampleor“training”data– Generalizing=building“statisticalmodels”tocapturecorrelations– Process=MLisneverdone,youmustkeepvalidating&refittingmodels

• SimpleMLworkflow:– Exploredata– FITmodelsbasedondata– APPLYmodelsinproduction– Keepvalidatingmodels

“Allmodelsarewrong,butsomeareuseful.”- GeorgeBox

9

TypesofMachineLearning1.Supervised Learning: generalizingfromlabeled data

10

TypesofMachineLearning2.Unsupervised Learning: generalizingfromunlabeled data

11

TypesofMachineLearning3.ReinforcementLearning:generalizingfromrewards intime

Leitner System Recommendersystems


MLUseCases

13

ITOps:PredictiveMaintenance

1. Getresourceusagedata(CPU,latency,outagereports)

2. Exploredata,andfitpredictivemodelsonpast/real-timedata

3. Apply&validatemodelsuntilpredictionsareaccurate

4. Forecastresourcesaturation,demand&usage

5. SurfaceincidentstoITOps,whoINVESTIGATES&ACTS

Problem:Networkoutagesandtruckrollscausebigtime&moneyexpenseSolution: Buildpredictivemodeltoforecastoutagescenarios,actpre-emptively&learn

14

Security:FindInsiderThreatsProblem:Securitybreachescausebigtime&moneyexpenseSolution: Buildpredictivemodeltoforecastthreatscenarios,actpre-emptively&learn

1. Getsecuritydata(datatransfers,authentication,incidents)



4. Forecastabnormalbehavior,riskscores&notableevents

5. SurfaceincidentstoSecurityOps,whoINVESTIGATES&ACTS

15

BusinessAnalytics:PredictCustomerChurnProblem:Customerchurncausesbigtime&moneyexpenseSolution: Buildpredictivemodeltoforecastpossiblechurn,actpre-emptively&learn

1. Getcustomerdata(set-topboxes,weblogs,transactionhistory)



4. Forecastchurnrate&identifycustomerslikelytochurn

5. SurfaceresultstoBusinessOps,whoINVESTIGATES&ACTS

16

Summary:TheMLProcessProblem:<Stuffintheworld>causesbigtime&moneyexpenseSolution: Buildpredictivemodeltoforecast<possibleincidents>,actpre-emptively&learn

1. Getallrelevantdatatoproblem



4. ForecastKPIs&notableeventsassociatedtousecase

5. SurfaceincidentstoXOps,whoINVESTIGATES&ACTS

Operatio

nalize


MLwithSplunk

18

Splunkbuilt-inMLcapabilities

kmeans cluster

outliers/anomalies /anomalydetection

predict

19

MachineLearninginSplunkITSIAdaptiveThresholding:• Learnbaselines&dynamicthresholds• Alert&actondeviations• Managefor1000sofKPIs&entities• Stdev/Avg,Quartile/Median,Range

AnomalyDetection:• Find“hiccups”inexpectedpatterns• Catchesdeviationsbeyondthresholds• UsesHolt-Wintersalgorithm

20

SplunkUserBehaviorAnalytics(UBA)• ~100%ofbreachesinvolvevalidcredentials(MandiantReport)• Needtounderstandnormal&anomalousbehaviorsforALLusers• UBAdetectsAdvancedCyberattacks andMaliciousInsiderThreats• LotsofMLunderthehood:

– BehaviorBaselining&Modeling– AnomalyDetection(30+models)– AdvancedThreatDetection

• E.g.,DataExfil Threat:– “Sawthisstrangelogin&datatransfer

forusermpittman at3aminChina…”– SurfacethreattoSOCAnalysts

21

MLToolkit&Showcase– DIYML

• SplunkSupportedframeworkforbuildingMLApps– Getitforfree:https://splunkbase.splunk.com/app/2890/

• LeveragesPythonforScientificComputing (PSC)add-on:– Getitforfree:refertoSplunkbase foryourOSversion

ê https://splunkbase.splunk.com/app/2881/ to/2884/– Open-sourcePythondatascienceecosystem– NumPy,SciPy,scitkit-learn,pandas,statsmodels

• Showcaseusecases:PredictHardDriveFailure,ServerPowerConsumption,ApplicationUsage,CustomerChurn&more

22

Standardalgorithms outofthebox:

Clustering:DBSCAN,KMeans,Birch,SpectralClusteringRegression: LinearRegression,RandomForestRegressor,ElasticNet,Ridge,LassoClassification: LogisticRegression,RandomForestClassifier,SVM,NaïveBayes(GaussianNB,BernoulliNB)Transformation: PCA,KernelPCA,TFIDFVectorizer,StandardScalerTextAnalytics: TF-IDFFeatureExtraction: FieldSelector (e.g.Univariate,ANOVA,K-best,etc.)

Implementoneof300+algorithmsbyeditingPythonscripts


BuildingMLApps

25

Analysts BusinessUsers

1.GetData&FindDecision-Makers

25

ITUsers

ODBCSDKAPI

DBConnectLook-Ups

AdHocSearch

MonitorandAlert

Reports/Analyze

CustomDashboards

GPS/Cellular

Devices Networks Hadoop

Servers Applications OnlineShoppingCarts

Analysts BusinessUsers

StructuredDataSources

CRM ERP HR Billing Product Finance

DataWarehouse

Clickstreams

26

2.ExploreData,BuildSearches&Dashboards• StartwiththeExploratoryDataAnalysisphase

– “80%ofdatascience issourcing,cleaning,andpreparingthedata”– Tip:leverageITSIKPIs– lotsofdomainknowledge

• Foreachdatasource,build“datadiagnostic”dashboard– What’sinteresting?Throwupsomebasiccharts.– What’srelevantforthisusecase?– Anyanomalies?Arethresholdsuseful?

• Mixdatastreams&computeaggregates– ComputeKPIs&statisticsw/stats,eventstats,etc.– Enrichdatastreamswithusefulstructureddata– statscountbyXY– whereX,Yfromdifferentsources– BuildnewKPIsfromwhatyoufind

27

3.Fit,Apply&ValidateModels• MLSPL – NewgrammarfordoingMLinSplunk• fit – fitmodelsbasedontrainingdata– [training data] | fit LinearRegression costly_KPI

from feature1 feature2 feature3 into my_model

• apply – applymodelsontestingandproductiondata– [testing/production data] | apply my_model

• ValidateYourModel (TheHardPart)– Whyhard?Becausestatistics ishard!Also:modelerror≠realworldrisk.– Analyzeresiduals,mean-squareerror,goodnessoffit,cross-validate,etc.– TakeSplunk’sAnalytics&DataScienceEducationcourse

28

4.Predict&Act• ForecastKPIs&predictnotableevents

– Whenwillmysystemhaveacriticalerror?– Inwhichserviceorprocess?– What’stheprobablerootcause?

• Howwillpeopleactonpredictions?– IsthisaSev 1/2/3event?Whoresponds?– DeliverviaNotableEventsordashboard?– Humanresponseorautomatedresponse?

• Howdoyouimprovethemodels?– Iterate,addmoredata,extractmorefeatures– Keeptrackoftrue/falsepositives


Demo


NextSteps

31

Gettingstarted• Pre-requisite: youmustberunning Splunk6.4.x

• DownloadandinstallthefreeMLToolkit&Showcase!– https://splunkbase.splunk.com/app/2890/– https://splunkbase.splunk.com/app/2881/ to/2884/

• SpeaktoyourlocalSE todiscusswaysyoucoulduseML

• JoinourlocalUserGroup– we’llberunningMLworkshops!– http://www.meetup.com/splunk-melbourne/

• Contactme!([email protected])


Q&A


ThankYou

34

ExampleSplunkSPL– ChurnUseCase|inputlookup churn.csv|samplepartitions=2seed=1234|searchpartition_number=0|fitLogisticRegression Churn?from "CustServ Calls""DayMins""EveMins"intoexample_churn_model|table*Churn*|`confusionmatrix("Churn?","predicted(Churn?)" )̀

|listmodels

|summaryexample_churn_model

|deletemodel example_churn_model

|inputlookup churn.csv|samplepartitions=2seed=1234|searchpartition_number=1|apply"example_churn_model"

|inputlookup churn.csv|samplepartitions=2seed=1234|searchpartition_number=1|apply"example_churn_model"|`confusionmatrix("Churn?","predicted(Churn?)" )̀

|inputlookup churn.csv|samplepartitions=2seed=1234|searchpartition_number=1|apply"example_churn_model"|`classificationstatistics("Churn?", "predicted(Churn?)")̀

#####exampletrainingusinglogisticregressionandrandomforestclassifierincombination

|inputlookup churn.csv|samplepartitions=2seed=1234|searchpartition_number=0|fitLogisticRegression "Churn?"from "CustServ Calls""DayMins""EveMins""Int'lPlan""IntlCalls""IntlCharge""IntlMins""NightCharge""NightMins""VMail Plan"into"LogReg_churn"|table*Churn*

|inputlookup churn.csv|samplepartitions=2seed=1234|searchpartition_number=0|fitRandomForestClassifier "Churn?" from"CustServ Calls""DayMins""EveMins""Int'lPlan""IntlCalls""IntlCharge""IntlMins""NightCharge""NightMins""VMail Plan"into"RF_churn"|table*Churn*

#####exampletestingusinglogisticregressionandrandomforestclassifierincombination

|inputlookup churn.csv|samplepartitions=2seed=1234|searchpartition_number=1|applyLogReg_churn asLogReg(Churn?)|applyRF_churn asRF(Churn?)|eval priorityscore(Churn?)=if('LogReg(Churn?)'="True.",10,0) +if('RF(Churn?)'="True.",100,0)+.1*'DayCharge'|sort- priorityscore(Churn?)|fieldspriorityscore(Churn?)*Churn?* "CustServ Calls""DayCalls""DayCharge"PhoneState|eval whattodo =if('priorityscore(Churn?)'>15,"Callthem!",null())|fieldformat "DayCharge"="$".round('DayCharge')|search"Churn?"="False."

35

ExampleSplunkSPL– MalwareUseCase|inputlookup firewall_traffic.csv

|inputlookup firewall_traffic.csv|fitLogisticRegression used_by_malware frombytes_received bytes_sent dest_port dst_iphas_known_vulnerability packets_received packets_sent receive_time serial_numbersession_id src_ip src_port intoexample_firewall_traffic_model|table*used_by_malware*|`confusionmatrix("used_by_malware","predicted(used_by_malware)")̀

|listmodels

|summaryexample_firewall_traffic_model

|deletemodel example_firewall_traffic_model

|inputlookup firewall_traffic.csv|samplepartitions=2seed=1234|searchpartition_number=1|apply"example_firewall_traffic_model”

|inputlookup firewall_traffic.csv|samplepartitions=2seed=1234|searchpartition_number=1|apply"example_firewall_traffic_model"|`confusionmatrix("used_by_malware","predicted(used_by_malware)")̀

|inputlookup firewall_traffic.csv|samplepartitions=2seed=1234|searchpartition_number=1|apply"example_firewall_traffic_model"|`classificationstatistics("used_by_malware", "predicted(used_by_malware)")̀

#####exampletrainingusinglogisticregressionandrandomforestclassifierincombination

|inputlookup firewall_traffic.csv|samplepartitions=2seed=1234|searchpartition_number=0|fitLogisticRegression used_by_malware frombytes_received bytes_sent dest_port dst_iphas_known_vulnerability packets_received packets_sent receive_time serial_numbersession_id src_ip src_port intoLogReg_used_by_malware|table*used_by_malware*|`confusionmatrix("used_by_malware","predicted(used_by_malware)")̀

|inputlookup firewall_traffic.csv|samplepartitions=2seed=1234|searchpartition_number=0|fitRandomForestClassifier used_by_malware frombytes_received bytes_sent dest_portdst_ip has_known_vulnerability packets_received packets_sent receive_time serial_numbersession_id src_ip src_port intoRF_used_by_malware|table*used_by_malware*|`confusionmatrix("used_by_malware","predicted(used_by_malware)")̀

#####exampletestingusinglogisticregressionandrandomforestclassifierincombination

|inputlookup firewall_traffic.csv|samplepartitions=2seed=1234|searchpartition_number=1|applyLogReg_used_by_malware asLogReg(used_by_malware)|applyRF_used_by_malware asRF(used_by_malware)|eval priorityscore(used_by_malware)=if('LogReg(used_by_malware)'="yes",10,0)+if('RF(used_by_malware)'="yes",100,0) +if(has_known_vulnerability="yes",50,0)|eval whattodo =if('priorityscore(used_by_malware)'>50,"Investigate!",null())|fieldswhattodo priorityscore(used_by_malware)*used_by_malware*receive_time src_ipserial_number session_id has_known_vulnerability|sortwhattodo

SplunkLive Sydney Machine Learning & Analytics

Data & Analytics

Transcript of SplunkLive Sydney Machine Learning & Analytics