Creating Your Own Threat Intel Through Hunting & Visualization

25
Creating Your Own Threat Intel Through Hunting & Visualization Raffael Marty VP Security Analytics May 11, 2016 Honeynet Workshop 2016 – San Antonio, TX

Transcript of Creating Your Own Threat Intel Through Hunting & Visualization

Page 1: Creating Your Own Threat Intel Through Hunting & Visualization

Creating Your Own Threat Intel Through Hunting & Visualization

RaffaelMartyVPSecurityAnalytics

May11,2016

HoneynetWorkshop2016– SanAntonio,TX

Page 2: Creating Your Own Threat Intel Through Hunting & Visualization

©RaffaelMarty 2

"This presentation was prepared solely by RaffaelMarty in his personal capacity. The material, views,and opinions expressed in this presentation are theauthor's own and do not reflect the views of SophosLtd. or its affiliates."

Disclaimer

Page 3: Creating Your Own Threat Intel Through Hunting & Visualization

Overview

HUNTINGAKAINTERNALTHREATINTELLIGENCE

THREAT INTELLIGENCEAPROCESSANDINFRASTRUCTUREVIEW

1

2 VISUALIZATIONATHREATINTELLIGENCEGOLDMINE

3

©RaffaelMarty 4

Page 4: Creating Your Own Threat Intel Through Hunting & Visualization

ThreatIntelligence

©RaffaelMarty 5

Page 5: Creating Your Own Threat Intel Through Hunting & Visualization

©RaffaelMarty 6

• Products / Tools

• Firewall - Blocks traffic based on pre-defined rules• Web Application Firewall - Monitors for signs of known malicious activity in Web traffic• Intrusion Prevention System - Looks for ‘signs’ of known attacks in traffic and protocol violations• Anti Virus - Looks for ‘signs’ of known attacks on the end system• Malware Sandbox - Runs new binaries and monitors their behavior for malicious signs• Security Information Management - Uses pre-defined rules to correlate signs from different data

streams to augment intelligence• Vulnerability Scanning - Searches for known vulnerabilities and vulnerable software

• Rely on pattern matching and signatures based knowledge from the past• Reactive -> always behind• Unknown and new threats -> won’t be detected• ‘Imperfect’ patterns and rules -> cause a lot of false positives

WeAreMonitoring– WithPastKnowledge

Verizon 2015 DBIR

70–90%OF MALWARE SAMPLES ARE UNIQUE TO

AN ORGANIZATION.

Page 6: Creating Your Own Threat Intel Through Hunting & Visualization

7

ANewArchitecture– TheSecurityDataLake

anydata BigDataLakeRules

contextIOCs

Page 7: Creating Your Own Threat Intel Through Hunting & Visualization

8

ExploringYOUREnvironment- Hunting

anydata Rules

IOCs

Hunting• Interactivevisualization•Analystdriven•Machineassisted

context

BigDataLake

Page 8: Creating Your Own Threat Intel Through Hunting & Visualization

9

HuntingCreatesInternalThreatIntelligence

anydata Rules

IOCs

Novel,AdvancedAttacks

internal TI

behavioralmonitoringscoringanomalydetectionmachine learningartificial intelligence

“models”

data science

x

new rules

context

BigDataLake

Page 9: Creating Your Own Threat Intel Through Hunting & Visualization

10

HowDoWeGoHunting?Inthefollowingwe’llexplorehowthisallmatters…

…butfirst,let’sseehowvisualization playsakeyrole.

Page 10: Creating Your Own Threat Intel Through Hunting & Visualization

Visualization

©RaffaelMarty 11

Page 11: Creating Your Own Threat Intel Through Hunting & Visualization

S e c u r i t y . A n a l y t i c s . I n s i g h t . 12

“HowCanWeSee,NotToConfirm- ButToLearn”

- EdwardTufte

Page 12: Creating Your Own Threat Intel Through Hunting & Visualization

13

WhyVisualization?dp

ort

time

Page 13: Creating Your Own Threat Intel Through Hunting & Visualization

14

• SELECT count(distinct protocol) FROM flows;

• SELECT count(distinct port) FROM flows;

• SELECT count(distinct src_network) FROM flows;

• SELECT count(distinct dest_network) FROM flows;

• SELECT port, count(*) FROM flows GROUP BY port;

• SELECT protocol,

count(CASE WHEN flows < 200 THEN 1 END) AS [<200],

count(CASE WHEN flows>= 201 AND flows < 300 THEN 1 END) AS [201 - 300],

count(CASE WHEN flows>= 301 AND flows < 350 THEN 1 END) AS [301 - 350],

count(CASE WHEN flows>= 351 THEN 1 END) AS [>351]

FROM flows GROUP BY protocol;

• SELECT port, count(distinct src_network) FROM flows GROUP BY port;

• SELECT src_network, count(distinct dest_network) FROM flows GROUP BY port;

• SELECT src_network, count(distinct dest_network) AS dn, sum(flows) FROM flows GROUP BY port, dn;

• SELECT port, protocol, count(*) FROM flows GROUP BY port, protocol;

• SELECT sum(flows), dest_network FROM flows GROUP BY dest_network;

• etc.

OneGraphSummarizesDozensofQueriesport dest_network

protocol src_network flows

Page 14: Creating Your Own Threat Intel Through Hunting & Visualization

Hunting

©RaffaelMarty 15

Page 15: Creating Your Own Threat Intel Through Hunting & Visualization

16

Technical

• Visualization

• Context

• DataScience

Non-Technical

• Analysts areyourbestandmostexpensiveresource

• Theyneedtherighttoolsanddata• Speed(seethedatalake)• Interaction(visual!)• Machine-assisted insight(datascience)

CoreComponents ToEnableHunting

Page 16: Creating Your Own Threat Intel Through Hunting & Visualization

17

UsersaccessingSharepointservers

User

SharepointServer

This graph of users accessingsharepoint servers, does notimmediately reveal any interestingpatterns.

data processing visualization

Page 17: Creating Your Own Threat Intel Through Hunting & Visualization

18

UsingHRdataascontext

RemoteUser

SanFranciscoOfficeUser

SharepointServer

data processing visualization

HRdata

Using color to add context to thegraph helps immediately identifyoutliers and potential problems.

Page 18: Creating Your Own Threat Intel Through Hunting & Visualization

19

• Simpleapproachesworks!• dc(dest),dc(d_port)

• Whatisnormal?

• Use data science / data mining to prepare data. Then visualize the output for human analyst.

DataScienceinSecurity- WordsofCaution

Page 19: Creating Your Own Threat Intel Through Hunting & Visualization

20

ChallengesWithClusteringNetworkTrafficThegraphshowsanabstractspacewithcolorsbeingmachineidentifiedclusters.

HardQuestions:• What aretheseclusters?• DoWebserverscluster?• Whataregood clusters?• What’sanomalous?

Page 20: Creating Your Own Threat Intel Through Hunting & Visualization

21

HBIMetricAnalysis

Visually learn, Test, Automate

Page 21: Creating Your Own Threat Intel Through Hunting & Visualization

24

• Wehavetriedmanything:o SocialNetworkAnalysiso Seasonalitydetectiono Entropyovertimeo Frequentpatternminingo Clustering

• Allkindsofchallenges• Simpleworks!

Let’sGetMathematical

Page 22: Creating Your Own Threat Intel Through Hunting & Visualization

25

Simple- DataAbstraction

Page 23: Creating Your Own Threat Intel Through Hunting & Visualization

26

LateralMovement- CrossNetworkCommunications

Challenges• Scale• Youwillfindoneofeverything• Definingwhite-listsandkeepingthemuptodate(i.e.,networkandassethygiene)

VPN

DMZ

Office

GIA

UnknownInternet

AWS

Page 24: Creating Your Own Threat Intel Through Hunting & Visualization

VisualAnalyticsDelivering Actionable Security

Intelligence

July 30,31 & August 1,2 - Las Vegas, USA

big data | analytics | visualization

http://secviz.org

WantToDiveDeeper?

©RaffaelMarty 27

Page 25: Creating Your Own Threat Intel Through Hunting & Visualization

[email protected]@raffaelmarty

©RaffaelMarty 28


Hands-on Learning Experiences for Cyber Threat Hunting Education · Cyber Hunting • Cyber threat hunting has emerged as a critical part of cyber security practice. However, –

Hands-on Learning Experiences for Cyber Threat Hunting Education · Cyber Hunting • Cyber threat hunting has emerged as a critical part of cyber security practice. However, –

Hunting Threat Actors with TLS Certificates · HUNTING THREAT ACTORS WITH TLS CERTIFICATES MARK PARSONS DEVELOPER/THREAT ANALYST KING AND UNION MARK@KINGANDUNION.COM. ... to put new

Hunting Threat Actors with TLS Certificates · HUNTING THREAT ACTORS WITH TLS CERTIFICATES MARK PARSONS DEVELOPER/THREAT ANALYST KING AND UNION [email protected]. ... to put new

Threat Hunting Using a Machine Learning Approach

Threat Hunting Using a Machine Learning Approach

IOT: EXPLORING THE THREAT SURFACEintro the big idea securing the edge ... » elasticsearch. threat hunting and response | security consulting 05 | securing the data. threat hunting

IOT: EXPLORING THE THREAT SURFACEintro the big idea securing the edge ... » elasticsearch. threat hunting and response | security consulting 05 | securing the data. threat hunting

Threat Hunting - csirt.jabarprov.go.id

Threat Hunting - csirt.jabarprov.go.id

Threat hunting workshop

Threat hunting workshop

Proactive Threat Hunting Combining the use of …Proactive Threat Hunting Combining the use of threat intelligence, analytics and automated security tools with human smarts Benefits

Proactive Threat Hunting Combining the use of …Proactive Threat Hunting Combining the use of threat intelligence, analytics and automated security tools with human smarts Benefits

Threat Hunting Report can be viewed here - News · THREAT HUNTING REPORT 3. THRET HNTING REPORT 4 KEY SURVEY FINDINGS ... Among the respondents to the threat hunting survey, six in

Threat Hunting Report can be viewed here - News · THREAT HUNTING REPORT 3. THRET HNTING REPORT 4 KEY SURVEY FINDINGS ... Among the respondents to the threat hunting survey, six in

TTPs for Threat Hunting in Refineries · TTPs for Threat Hunting in Refineries Dan Gunter | Twitter: @dan_gunter Principal Threat Analyst @ Dragos October 2018. ... •Not Red Teaming

TTPs for Threat Hunting in Refineries · TTPs for Threat Hunting in Refineries Dan Gunter | Twitter: @dan_gunter Principal Threat Analyst @ Dragos October 2018. ... •Not Red Teaming

Threat hunting with MITRE ATT&CK

Threat hunting with MITRE ATT&CK

Enabling Efficient Cyber Threat Hunting with Cyber Threat ...

Enabling Efficient Cyber Threat Hunting with Cyber Threat ...

Examples and Goals of Threat Hunting - USALearning · Examples and Goals of Threat Hunting. ... //. ... and they need to answer questions .

Examples and Goals of Threat Hunting - USALearning · Examples and Goals of Threat Hunting. ... //. ... and they need to answer questions .

Cyber Threat Hunting with Phirelight

Cyber Threat Hunting with Phirelight

Threat Hunting 102: Beyond the Basics

Threat Hunting 102: Beyond the Basics

Abstract Tools for Effective Threat Hunting

Abstract Tools for Effective Threat Hunting

The Elastic Guide to Threat Hunting - copyright

The Elastic Guide to Threat Hunting - copyright

Threat Hunting with ELK training - Intellectual Point

Threat Hunting with ELK training - Intellectual Point

SQRRL threat hunting platform

SQRRL threat hunting platform

DEMYSTIFYING THREAT HUNTING · 2021. 1. 26. · We wanted to clear up the confusion. Demystifying Threat Hunting will explain: • What threat hunting is – and what it isn’t.

DEMYSTIFYING THREAT HUNTING · 2021. 1. 26. · We wanted to clear up the confusion. Demystifying Threat Hunting will explain: • What threat hunting is – and what it isn’t.

The awakening of cyber threat hunting: Intelligence ...

The awakening of cyber threat hunting: Intelligence ...