Next generation security analytics

NEXT GENERATION SECURITY ANALYTICS

CHRISTIAN HAVE

@CKHAVE [email protected]

mailto:[email protected]

ATTACKERS NAVIGATE A GRAPH. DEFENDERS TYPICALLY THINK IN POINT-SOLUTIONS

ME!

ACADEMIC PSEUDO-INTELLECTUALISM

THE PATH OF LEAST RESISTANCE

GRAPHS?

NAVIGATING A GRAPH

▸ Expensive exploits are expensive

▸ Use the least expensive weapon

▸ Cost of 0Day exploits

▸ Use once == expensive

DEFENDERS THINK IN POINT SOLUTIONS

IMPLEMENTING MOVIE-PLOT POINTS

▸ Natural-path engineering; layout buildings, let people find paths

▸ We find the best way forward

▸ We get around controls - it’s in our nature.

MOTIVATION IS NOT IMPACTED BY CONTROLS. AS LONG AS THE ATTACK IS ECONOMICALLY FEASIBLE MOTIVATION REMAINS.

ME!

ACADEMIC PSEUDO-INTELLECTUALISM

MOTIVATION AND ECONOMIC BENEFIT

▸ The implementation of the “child pornography filter” did not change the number of convictions in Denmark

▸ It did move it out of the “open” Internet

▸ Introducing a censorship filter for enticing terrorism will not solve the problem of radicalising the youth

▸ It will move it from the “open” Internet somewhere else

▸ The clearing of pusher-street in Christiania did not stop the sale of marihuana

▸ It did move it out of the open

▸ Laws work wonders for law-abiding citizens

ECONOMIC BENEFIT

▸ Paedophiles will not stop because of a DNS block, regardless of the penalty

▸ Buying marihuana does not carry a penalty in Denmark

▸ Selling marihuana does not carry a penalty (besides whatever you have on you at the point of arrest)

▸ We can only start winning once we understand what “winning” is and what game we are actually playing

WE FIGHT HUMAN NATURE. PICKING THE RIGHT BATTLE IS KEY FOR WINNING.

ME

PICKING THE RIGHT BATTLE

WE WANT THE EASIEST ATTACKS.

NOT THE HARDEST.

ME

WINNING THE RIGHT BATTLE

EASY ATTACKS - SINCE WE CANT AVOID ATTACKS

SURRENDERING?

▸ Attackers are lazy

▸ Attackers optimise cost (0days)

▸ Controls raise the cost of attacks

▸ Making attacks hard to detect

▸ Controls are not “free”

▸ Cancer-screening has a higher mortality than not screening

▸ Anti-Virus engines are points-of-infection

▸ Accept attacks will happen

▸ Deal with the attacks when they happen

▸ Don’t screen for cancer or move the attackers away from the obvious routes

EASY ATTACKS - SINCE THEY CAN’T BE AVOIDED

NOT SURRENDERING - PICKING THE BATTLEFIELD

▸ Defensively design your security architecture

▸ Understand it’s weaknesses

▸ Exploit weaknesses

▸ Monitor and gather intelligence, and defend smarter

ECONOMY OF THE DEFENCE

SHOULDN’T WE INVEST IN CONTROLS?

▸ Of course!

▸ Controls associated with costs towards the attacks

▸ The barrier of entry (cost of attack) deters some, but typically only the lowest on the spectrum

▸ Controls as point-solutions gives way for target-fixation

▸ GDPR changes the economy of the defence

CURRENT STATE OF SECURITY ANALYTICS

INTRO


COMPONENTS OF SECURITY ANALYTICS

DATA INGEST PROCES ANALYSE PRESENT VIZ ACT.


▸ Nothing new.

▸ Everyone does syslog

▸ Everyone has an agent

▸ Some do Flow-analytics

▸ Some do application-level analytics

▸ Few do full-packet captures

DATA INGESTION DATA INGEST PROCES ANALYSE PRESENT VIZ ACT.


▸ Inbound content must be structured

▸ Structure sometimes follows a common language

▸ Taxonomy, Ontology - whatever floats your boat

▸ Some content is sometimes enriched with metadata

▸ Threat Intel, GeoIP, Asset Management DB info etc.

▸ This part has to be fast - many vendors “cheat”

PROCESSING DATA INGEST PROCES ANALYSE PRESENT VIZ ACT.

ANALYSIS

▸ Analytics today is relatively simple

▸ Simple statistics

▸ Advanced statistics

▸ Patterns

▸ Known-bad, known-good analysis on more COTS platforms

▸ Most vendors pack tons of alerts and correlations

SECURITY ANALYTICS DATA INGEST PROCES ANALYSE PRESENT VIZ ACT.

ANALYSIS

▸ Most vendors provide views on the raw data or an abstraction of the raw data

▸ Most vendors provide a relatively easy way to setup views on the raw or aggregated data for analytics

▸ Some vendors have great views when presenting alerts and important events

▸ Few if any systems are able to present hierarchies of data, the relationships between events and deviations on hierarchies

PRESENTATION DATA INGEST PROCES ANALYSE PRESENT VIZ ACT.

ANALYSIS

▸ Pie charts

▸ geo maps

▸ tables

▸ rows, columns and heatmaps

▸ Nothing you couldn't do with excel - and maybe thats ok

DATA VISUALIZATION DATA INGEST PROCES ANALYSE PRESENT VIZ ACT.

ANALYSIS

▸ We collect syslog, application data and network data

▸ We process it, transform it

▸ We enrich and present it both for the analyst and graphically

▸ Making the system provide actionable information for the analyst

▸ Some systems even go the next step and perform proactive responses on other platforms

▸ Shuts ports, adds to ACLs, disables users e.g.

ACTIONS DATA INGEST PROCES ANALYSE PRESENT VIZ ACT.


COMPONENTS OF SECURITY ANALYTICS

DATA INGEST PROCES ANALYSE PRESENT VIZ ACT.

SCALING OUT

ANCHOR IN

ORG

INVESTIN

PROJ.

IN-HOUSE?

NON-INF APPS

USING SIEM AND GETTING VALUE FROM ANALYTICS

SCALING OUT

USING AND GETTING VALUE OUT OF SIEM


SCALING OUT

▸ The only thing constant is change; 20% growth in volume

▸ Areas that we need to scale on

▸ Ingestion

▸ Processing

▸ Storage

▸ Presentation


SCALING OUT

▸ Ensure your system scales well when it comes to ingestion

▸ Everyone is doing it differently

▸ Some solutions tie together backends and presentation layers

▸ Scaling presentation is immensely important for widespread adoption in your organisation

▸ Scaling the backends should not be a concern in 2016

ORGANISATIONAL ANCHORING


SIEMS FAIL WHEN THEY ARE LEFT ALONE

ME

WHEN DO WE FAIL

SAD

ME

WHEN DO WE FAIL

IN THE CORNER. SILENTLY COLLECTING LOGS AND TRIGGERING ALERTS, NOBODY WILL EVER SEE.

ME

WHEN DO WE FAIL



▸ SIEMS fail if they are the point solution to a problem

▸ Stakeholders lose interest

▸ The value-prop was never clear

▸ A great sale but a horrible purchase



▸ Logs and network data is immensely rich in information

▸ This information can be used for much more than security

▸ Let help-desk-users use pre-defined views and prepared analytics for easier resolution (move work to 1st level support)

▸ Allow your OPs team to use analytics for root-cause analysis, statistics for predictions and forecasts

▸ Allow your management-team to view quality of infrastructure and do controls of outsourced services

▸ Liberate data from silos

INVESTING IN THE PROJECT



▸ Set expectations - understand why we use analytics

▸ Introduce the notion of the Lockheed Martin Cyber Kill Chain (see next slide)

▸ Understand the threat landscape

▸ Identify the key threats to the organisation (ext)

▸ Identify the key threats identified by the organisation (int)

▸ Bringing it all together

KILL CHAIN

SECTOR DRILL-DOWN (VERIZON DBIR - 2016)

MOTIVATION DRILL-DOWN: HEALTH-CARE

THREAT ACTOR DRILL-DOWN: HEALTH-CARE


IDENTIFYING KEY INTERNALLY IDENTIFIED THREATS

▸ Use the internal risk assessment

▸ Compare with external threat information

▸ Identify any potential gap - ask yourself why it exists


BRINGING IT ALL TOGETHER

▸ With threats and critical systems identified

▸ And with an understanding of the kill-chain and the cost of controls in mind

▸ The task of the project-team is to identify the success criteria for the project with a common acceptance and buy-in from leadership and stakeholders

THE SECURITY OPERATIONS CENTER

IN-HOUSE OR OUTSOURCED

THE SECURITY OPERATIONS CENTER

CONSIDERATIONS

▸ The 3 Ps

▸ People, Process and Technology

▸ Is it possible to retain skill

▸ With level we need

▸ In numbers sufficient to staff the SOC

▸ Are we mature enough to identify which alerts and incidents we want to act

on

▸ Can we with confidence say that we understand how to act when we

then receive the alert?

▸ Engagement models:

▸ Who takes action on our network during a breach

▸ What gets escalated back “home”

▸ Do we have sensitive data preventing a full managed SOC?

USING SIEM FOR ENTERPRISE APPLICATIONS

USING SIEM FOR ENTERPRISE APPLICATIONS

WHERE IS THE GOLD IN YOUR NETWORK

▸ Third generation SCADA

▸ Industry 4.0

▸ SOA-Enabling your ERP Platforms

▸ Federated Access with suppliers

WHERE IS THE GOLD

PATH OF LEAST RESISTANCE

▸ Vulnerabilities in SCADA

▸ Hard-coded admin-passwords

▸ non-patchable systems, “because operational IT”

▸ Non-networked mindset of admins

▸ Industry 4.0

▸ “Smart products with localisation point, status, historical positions and data points, allowing globally unique identification of all products” - Good luck with that

WHERE IS THE GOLD

PATH OF LEAST RESISTANCE

▸ SOA-Enabling your ERP Platforms

▸ % of SAP notes found externally

▸ SAP offers mobile access, organisations offers BYOD

▸ Do we trust jail-break detection?

▸ Federated Access with external suppliers

▸ Identities does not exist any longer

▸ Business rules define access to the network now

WHERE IS THE GOLD

USE-CASES FOR ENTERPRISE APPLICATION SIEM USE

▸ Changing master data records

▸ Critical transactions (payments)

▸ Changes in performance data (valve pressure)

▸ Critical changes to equipment (voltage, valve positions)

▸ Abnormality on order sizes, frequency, workflows

▸ The data is here - why not use it?

NEXT GENERATION SECURITY ANALYTICS

SORRY! IT WONT BE MINORITY REPORT

NEXT GENERATION - CHALLENGES

DIVERSITY IN DATA - VALUE IS FOUND EVERYWHERE

TWO PROBLEMS LARGE VARIATION LARGE VOLUME

NEXT GENERATION

INFORMATION OVERLOAD: OVERCOMING CHALLENGES

▸ Even with effective alerts, the amount of data is unmanageable

▸ Workflow is the key

PREPARE SITUATIONAL AWARENESS

IDENTIFY

ANALYSISREACT

INVESTIGATE

IMPROVE

COLLABORATE

NEXT GENERATION

WORKFLOW

▸ Situational awareness ▸ Identify anomalies based on what is observed ▸ Time of day deviations or time of year (scale!)

▸ Identify / analyse ▸ Based on the norm and baseline we can work on large-scale analytics ▸ Complex temporal changes in behaviours and activity

▸ React and “arm the investigators” ▸ Rapid response on what data was exposed, how and not least why

▸ Improve / Collaborate ▸ Feedback of the intelligence created in the analysis must be fed back into the system ▸ Partners and collaborators must receive the right amount of supporting information ▸ Think of this as the collective immune system

NEXT GENERATION

WORKING WITH DATA

▸ Clustering:

▸ Build a network of events and relations

NEXT GENERATION

WORKING WITH DATA

▸ Drill-down

▸ Re-draw - build hierarchies based on relationships

▸ Use gathered data, third party threat intel or collaboration data as a key to further expand on the search

▸ With our “enriched” analysis we can map a focus area

▸ Replay interactions over time, spot patterns and behaviour

▸ Login, data is moved out of network (repeat ad. inifitum)

NEXT GENERATION

LIMITATIONS OF CURRENT-GENERATION ARCHITECTURES

▸ Remember the architecture?

▸ Ingest, process, analyse, visualise, act

▸ This is inherently inefficient and a testament to legacy

▸ “NoSQL” is more part of the problem than the solution

▸ “BigData” in it’s true form is what will move us forward

▸ We spend most of the hardware available for processing data to store it and to prepare it

NEXT GENERATION

INSIGHTS INTO NEXT-GENERATION ARCHITECTURES

▸ Small hardware footprint needed for storage

▸ No processing, no normalisation, just straight to disk

▸ Use the hardware you have for analytics

▸ Towards realtime analytics and away from “Queries”

▸ Ingestion of full packet capture as an equal part to log-collections

NEXT GENERATION

ARCHITECTURE PRINCIPLES

NEXT GENERATION ARCHITECTURE

BIG DATA?

▸ Hadoop (ecosystem) is full of great and powerful tools

▸ Cluster management, realtime streaming, graph databases, distributed file systems (HDFS) etc.

▸ The technology is ready - vendors just need to get going ;)

CONCLUSIONS

ANALYTICS TRENDS

▸ Machine Learning

▸ People who bought X also looked at Y

▸ Automatic signature and pattern creation

▸ Payload analytics

▸ Deep behavioural analytics on network and log data

▸ Frameworks supports use-cases we could only dream of

▸ Online packet compression in real-time

▸ Analysis on packets to reconstruct network topologies behind NAT

CONCLUSIONS

ANALYTICS TRENDS

▸ Machine Learning Based Botnet Detection With Dynamic Adaptation

▸ Botnet beaconing based on linguistic analytics of DNS-names

▸ Detect stealthy DDoS against large-scale networks (ML)

▸ Automated discovery, attribution, analysis and risk assessment

▸ Social connectivity graphs, Machine-Learning, automatic malware reverse-engineering

CONCLUSIONS

ANALYTICS TRENDS

▸ Creation of “Social graphs” by crawling social networks and intercepting mail traffic

▸ Creation of “Social graphs” by analysing voice patterns and writing patterns regardless of from where they originate

▸ Combining social graphs and analyse sentiment

▸ (Radicalisation between actors)

NEXT GENERATION

PRODUCTISING

▸ Anomaly detection

▸ Machine learning (SparkML2.0 just released)

▸ Graph processing (All of Facebook is stored in 1 GraphDB)

▸ Scale dynamically - provision servers and services along with the processing need

▸ Scale locally or in the cloud - based on data sensitivity

NEXT GENERATION

ENRICHMENT

▸ Enriching data is possible today but sees relatively slow adoption in security

▸ STIX/TAXII/Cybox/Yara and other standards provide an ontology for attacks, actors, motives

▸ The SIEM of tomorrow will evaluate every event against internal and external threat intelligence sources

▸ The SIEM of tomorrow will forward-integrate with whatever “flavor of the month” point-solution implemented

TRANSITIONSCONCLUSIONS

CONCLUSIONS

TRANSITIONS

▸ The threat-landscape is changing

▸ The efficiency of technical controls declines in comparison to the economy of the attacker

▸ We have to level the playing field by understanding our weaknesses

▸ Ensure we have security analytics in place

▸ Ensure we have the insights and capacities to deal with it our selves or move it to a third party (responsibility not included)

CONCLUSIONS

TECHNOLOGY TRENDS

▸ We move to larger platforms

▸ Built with the tools developed at Twitter, LinkedIN and Facebook

▸ Queries, SQL and pre-processed data does not scale

▸ Imagine an out-sourced SOC with an installed capacity for the 20 of the Top2000 companies in Europe -

▸ Milions and Milions of events every second (EPS)

ANALYTICS

WRAP UP

▸ We have the technology now

▸ We have the math

▸ And we are starting to understand the threats and playing field

▸ The vendors just have to wrap everything together

▸ Few, if any, organisations have the capacity to write algorithms

Next generation security analytics

Technology

Transcript of Next generation security analytics