Securing Your Network with Anomaly Detection using ... Live Berlin 2017/BRKSEC-3056.pdf · Securing...

Securing Your Network with Anomaly Detection using Distributed Learning Architecture (Learning Networks)

Alex Honoré, CCIE #19553, Technical Leader, Engineering

BRKSEC-3056

© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public

Self Learning Networks: A terrific Journey of Innovation

BRKSEC-3056 3


What Self Learning Networks is About ...

• SLN is fundamentally a hyper-distributed analytics platform ...

• Putting together analytics and networking ... • Goldmine of untouched data on networking gear (sensing)

• Network learns and computes models on premise (analytics)

• The Network adapts, modifies its behavior (control)

• SLN for Security: attacks are incredibly sophisticated and targeted, ex-filtration of data being a major concern, requiring a next-generation approach => Stealthwatch Learning Network License

• True Technology disruption ...

BRKSEC-3056 4


Botnets and Data Ex-Filtration Techniques

• Size can range from thousands to millions of compromised hosts

• Botnet can cause DDoS & other malicious traffic (spam, ...)to originate from the inside of the corporate network

• C&C (C2) servers become increasingly evasive

• Fast Flux Service Networks (FFSN), single or double Flux

• DGA-based malware (Domain Generation Algorithms)

• DNS/NTP Tunneling

• Peer-to-Peer (P2P) protocols

• Anonymized services (Tor)

• Steganography, potentially combined with Cryptography

• Social media updates or email messages

• Mixed protocols ....

• Timing Channels

C&C Server(s)

Internet

BRKSEC-3056 5


A true paradigm shift

(Current) Generation of Security Architectures and Products

SLN is Machine Learning-based and pervasive

• Specialized security gear connected to the network (FW, IPS, ...)

• Heavily signature-based ... to detect known malwares

• Dynamic update of signatures

• Use of adaptive Machine Learning (AI) technology to detect advanced,

evasive Malware: build a model of normal patterns and detect outliers

(deviations)

• High focus on 0-day attacks

• Use every node in the network as a security engine to detect attacks

• Complementary to all other technologies (FW, IPS, ...)

BRKSEC-3056 6


Together

Learning Network License Stealthwatch

Immediate Local

Detection with Machine

Learning communication

Complete Broad and Deep

Branch Level Visibility

Netflow and Behavioral

Analytics for Branch Level

Security

Behavioral Analytics

Based on Rules and

Statistical Analysis

Central Detection with

Full Historical Data

Packet Capture

Integration with Security

Packet Analyzer

Stealthwatch: Historical/Statistical

Based Anomaly Detector

Behavioral Analytics with

Machine Learning

Packet Capture at the

Branch Level

Find zero day attacks

immediately and find

historical trends 30, 60, 90

days in the past

Learning Network License: Algorithmic

Based Anomaly Detector – ISR 4K only

Network as a Sensor in the Branch

BRKSEC-3056 7


Cisco ISEJoint Use Case: Retail Stealthwatch

Learning Manager

ISR4K with

Learning

Network License

Better Together

MPLS

Internet

Headquarters

Stealthwatch

Management

Console

ISR4K with

Learning

Network License

Stealthwatch

Flow Sensor

Retail Store

Retail Store

Complete Broad and Deep

Branch Level Visibility Netflow and Behavioral Analytics

for Branch Level Security

Integrated Threat Intelligence with

Cisco Identity Services Engine (ISE)

BRKSEC-3056 8

SLN Architecture


• Fundamentally distributed, building models for visibility and detection at edge

• Uses Machine Learning (ML)

• Context enrichment (using ISE integration, Threat Intelligence, ... )

• Ability to adapt to user feed-back (Reinforcement Learning)

• Advanced control for fine-grained mitigation

SLN Architecture Principles For Security

BRKSEC-3056 10


WAN

DLAAgent

SLN Architecture

Internet

ManagerThreat

Intel

• Orchestration of Learning Network Agents

• Advanced Visualization of anomalies

• Centralized policy for mitigation

• Interaction with other security components

such as ISE and Threat Intelligence Feeds

• North bound API to SIEM/Database (e.g.

Splunk) using CEF format

• Evaluation of anomaly relevancy

Manager

ISE

• Sensing (knowledge): granular data

collection with knowledge extraction from

NetFlow but also Deep Packet Inspection

on control and data plane & local states

• Machine Learning: real-time embedded

behavioral modeling and anomaly

detection

• Control: autonomous embedded control,

advanced networking control (police,

shaper, recoloring, redirect, ...)

Agent


Public/Private

Internet

An Open Architecture (Manager / SCA)

Internet

ISE

Agent

Threat

Intel

Manager

Identity Services Engine

Context Enrichment:

IP Address (key)

Audit session ID

User AD Domain

MAC address

NAS IP & port (!!)

Posture

TrustSec, SGT, ...

SIEM,

DB

FW,

IPS/IDS

CEF export (syslog transport)

pushing anomalies as events

into DB and SIEM

API triggering Mitigation

form external Sources

such as Firewall,

IPS/IDS, ... Abstracting

networking complexity

TALOS, potentially others


Public/Private

Internet

An Open Architecture (Agent / DLA)

Internet

ISE

ManagerThreat

Intel

Threat Grid,

OpennDNS,

WBRS, ... Other

TI feeds

BRKSEC-3056 13

Northbound API

PCM

NSC NCC

TIP DLC

Netflow DPI Local

States

Agent

AgentOther

SOLT & Traffic Modeling


Before we start ... A few (random) facts:

• Two camps ... Super Pro ML and Anti-ML, both have good arguments

• Extremely wide range of ML algorithms with no one-size-fits-all

• "No Free Lunch" theorem

• ML/AI incredibly powerful if applied to solve the right problems

• Hard to tune ? Yes if naively applied ...

Interpretability, scalability & user experience are essential

BRKSEC-3056 15


Discussing Recall, Precision, FP, ...

• Few simple notions required when discussing Machine Learning: False Positive (FP), True Positive (TP), False Negative (FN), True Negative (TN), Recall and Precision.

• Take a Classifier C trained to detect if an event E is relevant (Like) or not (irrelevant).• TP: E is classified as relevant and is indeed an relevant

• FP: E is classified as relevant and is in fact irrelevant (noise)

• TN: E is classified as irrelevant and is indeed irrelevant

• FN: E is classified as irrelevant and is in fact an relevant

• Recall = TP / ( TP + FN) (notion of sensitivity)

• Precision = TP / (TP + FP) (positive predictive value)

• Accuracy ACC = (TP + TN) / (TP + TN + FP + FN),

• Example: if a classifier that is trained to detect dogs in a picture detects 15 dogs, only 10 of them are dogs, and there are 20 dogs in the picture then the Precision = 10/15 = 0.66 and Recall = 10/20 = 0.50

BRKSEC-3056 16


Clusters, Self Organizing Learning Topology and Anomalies

Key question: how can we model host behaviors ?

Modeling mixed-behaviors unavoidably leads to hiding anomalies ...

The fundamental idea of dynamics clustering is to “group” devices according to behavioral similarity

Self Organizing Learning Topologies (SOLT): ability to build Virtual topologies used to learn models between dynamic clusters

Clusters become nodes of a graph, traffic becomes the edges

Example: find model for HTTP traffic from cluster A to cluster B

BRKSEC-3056 17


Public/Private

Internet

DLA

Branch 1

Dynamic Clustering

Internet

Branch 2

Cluster: known/internal/network

Cluster: known/internal/collab

AgentCluster: known/internal/inet::windows

BRKSEC-3056 18


Dynamic Clustering

Learning of cluster

assignment is a dynamic

task, and hosts are

allowed to transition

BRKSEC-3056 19


SOLT – Clustering StatisticsTotal # clusters quickly converges towards the 60-75 mark

Behavioral transitions keep occurring as behaviors

evolve and/or addresses get reassigned

Hosts gradually transition to “known” state as the

system collects more and more samples

BRKSEC-3056 20

Anomaly

Life

of a

n A

no

ma

ly

Agent

Manager

NSCNSC : Traffic analysis from

multiple data feeds

SOLT

Clustering: dynamic

clustering according to

behavioral degree of

similarity


Boston

Hierarchical ML Models

Voice

Collab

Printing

File Transfer

Cluster Layer

Collab File Transfer

Collab models

from C1, from

C2, from C3

Application Layer

File Transfer models

from C1, from C2,

from C3

Scr/Dest Cluster Layer

NYC

Germany

Collab models

C1-D1, C1-D2,

C1-D3, C2-D1, ...

File Transfer

models C1-D1, C1-

D2, C1-D3, C2-D1,

...

Model

BRKSEC-3056 22


Public/Private

Internet

Inside a Model ...

Internet

DLA

Germany

Multi-dimensional and Hierarchical models

using stateless/statefull features

(hundreds of dimensions) ...

High number of dimensions

extracted from multi feeds

(Netflow, DPI)

Rich DNS features: avr

names length, # of

consecutives vowels, average

entropy of characters, ...

Multi-layer: cluster-cluster-

app, cluster-app, app

BRKSEC-3056 23


Computing “SOLT” Scores

• Each scored flow update is evaluated against prior observations, computing the rank of the score over a sliding time window.

• Flow updates are then marked as anomalous or not based a set of criterion to be met (Maximum rank to be considered as anomalous, Score value, # of samples contributing to model, Maturity of the model (# of samples, time, ...).

• Boosting based on Expert knowledge (application sensitivity, # of features, ...)

• Computes an anomaly score and select TOP anomalies

BRKSEC-3056 24

Anomaly

Life

of a

n A

no

ma

ly

Agent

Manager


multiple data feeds

SOLT

Clustering: dynamic



similarity

ModelingModeling: dynamically

learned baseline with

multiple layers, high

dimensions space,

anomaly detection


In this demo, we will show - Smart Dashboard: stats on anomalies, ... - SLN System state after learning: cluster, ...- DLA states: CPU, memory, ...

Selective Anomaly Forwarder (SAF) & Selective Anomaly Pullers (SAP)


WAN

Agent

Selective Anomaly Forwarder (SAF) and Selective Anomaly Puller (SAP)

Manager

1. When an anomaly is detected by an Agent, its Selective Anomaly Forwarder decides whether this anomaly is worth being sent to the Manager (every Agent is given a "budget" of anomalies it may report)

2. If the SAF decides to forward the anomaly, a digest of the anomaly is sent to the Manager

3. When a digest of an anomaly is received by the Manager, its Selective Anomaly Puller decides whether this anomaly is worth being completely pulled

4. If the SAP decides to pull the anomaly, all the information about this anomaly is requested to the Agent

BRKSEC-3056 29


Selective Anomaly Forwarder (on the DLA)

• SAF role is to select the most interesting anomalies to be forwarded to the SCA according to Score of the anomaly, According to a forwarding Budget, with exploration

Forwarded Anomalies

Considered for

exploration

Forwarded with probability proportional to

importance and available budget

available budget

BRKSEC-3056 30


Selective Anomaly Pullers (on SCA)

• SAP role is to select the most interesting anomalies from all DLAs to be shown to the user, according to Score of the anomaly for a given DLA and across all DLAs (ensuring good diversity of anomalies), local Budget with exploration

Exploration

Puller

(importance)

Inbox Puller

(relevance)

Discarded

Puller

(-relevance)

DRL prediction

AN

OM

AL

Y S

HO

WN

TO

US

ER

ANOMALY IS NOT PULLED

dislikelike

do not pull

pull

do not pull do not pull

pull

pull

BRKSEC-3056 31

Distributed Relevance Learning explained later in great details

Anomaly

Life

of a

n A

no

ma

ly

Agent

Manager


multiple data feeds

SOLT

Clustering: dynamic



similarity




dimensions space,

anomaly detection

Scoring &

Ranking

An

om

aly

9A

no

mal

y6

An

om

al

y4

An

o

maly

5

An

o

maly

1

A n o m a l yAn

om

aly 2

A n o m al y 3A n o m a l y 7

Selective Anomaly

Forwarder: select the most

interesting anomalies

according to their score,

with exploration

Killing False Positives with Distributed Relevance Learning


Public/Private

Internet

Controller

ISE

SCA

Threat

Intel

DLA


Public/Private

Internet

Controller

ISE

SCA

Threat

Intel

DLA

Traditional Anomaly Detection Systems

• Focus on Detection (wrong)

• Core challenge is not Detection itself but Precision (avoid False

Positive / Irrelevant alarms)


Public/Private

Internet

Controller

ISE

SCA

Threat

Intel

DLA

SLN Approach

• Efficient detection and Precision

• Make the Network learn form its own mistakes and

eliminate False Positive !

• There is a notion of subjectivity too

• Not a feature but an Architecture

Traditional Anomaly Detection Systems

• Focus on Detection (wrong)

• Core challenge is not Detection itself but Precision (avoid False

Positive / Irrelevant alarms)


Reinforcement

Learning: Actor

Public/Private Network

Manager

Agent

Distributed Relevance Learning

Statistical Classifier

Optimal Forwarder

training data

predictions

BRKSEC-3056 39


HeuristicsOptimal

Forwarder

Optimal

Forwarder

Inbox

Pre-trained heuristic selects relevant events.

......

...

Up

to

50

00

dis

trib

ute

d a

ge

nts

an

aly

zin

g 9

bil

lio

n n

etw

ork

ing

eve

nts

ML Model supervised

trainingML Model

Agent

Agent

Agent

Agent

ManagerWAN


Relevance can be subjective too !

BRKSEC-3056 41

Behind the scenes ...


Challenges ...

• Design an algorithm with the following properties:

1) Remove False Positive (FP) (anomalies that are not of interest)

2) Do not remove true positive (anomalies that are relevant)

3) Learn quickly (do not require too much feed-back from the user)

4) Be consistent across data set (robustness)

5) Handle inconsistency between users, changing decisions (unlearn)

• Sophisticated architecture involving several components:

1) Deep Neural Networks (DNN)

2) Classifiers trained with Supervised Learning

3) Active Learning to request labeling of specific elements of a setbased on an importance function

BRKSEC-3056 43


SLN may improperly discard an actual anomaly ... (False Negative of the Like Class) => The user can correct mistakes too thanks to the Discarded Box.

BRKSEC-3056 44


SLN asking for help ... (remember exploration ?)

BRKSEC-3056 46

Anomaly

Life

of a

n A

no

ma

ly

Agent

Manager


multiple data feeds

SOLT

Clustering: dynamic



similarity




dimensions space,

anomaly detection

Scoring &

Ranking

An

om

aly

9A

no

mal

y6

An

om

al

y4

An

o

maly

5

An

o

maly

1

A n o m a l yAn

om

aly 2

A n o m al y 3A n o m a l y 7

Selective Anomaly

Forwarder: select the most

interesting anomalies

according to their score,

with exploration

Anomaly

Selection

Selective Anomaly Puller:

select the most interesting

anomalies according to their

score per Agent and across

all Agents, with exploration

DRLDistributed

Relevancy

Learning:

Likelihood of

relevancy (False

Positive

reduction)

Relevancy

Learning

Packet Capture & Mitigation


PCAP of Anomalous Traffic

Northbound API

PCM

NSC PBC

TIP DLC

Netflow DPI Local

States

Agent

Other

• Anomaly Detected: The DLC detects an

anomaly in the traffic and gathers all the details

to characterize it: time, IP etc.

• PBC Message: Sends a message to the PBC

with the characteristics of the anomaly

DLC

• Anomaly Message: Receives the anomaly

details from the DLC

• PBC Search and Extract: Searches for all the

packets that match the anomaly characteristics

and extracts them to a compressed PCAP file

• PCAP storage: Maintains list of files per

anomaly and purges unused files periodically

• Push files: Pushes all PCAP files for an

anomaly from the Agent when a user requests it

• Packet Details: File contains packets that have

either source or destination IP of the anomaly.

Allows to see all activity around the anomaly

• PCAP Size: Typically ~ 10KB-100KB, 10K-500K

packets

PB

C

Branch Traffic

SPAN

Traffic

Circular Buffer Compressed

PCAP Files


Controller

infrastructure

On-Premise Edge Control

Manager

Public/Private

Internet

Agent

AgentAgent

Control Policy

Smart Traffic flagging

Traffic segregation & selection

Network-centric control (shaping,

policing, divert/redirect)

Honeypot

(Forensic Analysis)

Shaping

BRKSEC-3056 51


In this demo, we will show Mitigation triggered by a user from a given anomaly

System Requirements


Stealthwatch Learning Network License Requirements

Learning Network Manager

VMWare ESXi 5.5

Memory 16 Gb

4 Virtual CPUs

1 Virtual NIC

200 Gb of hard disk

SCA Manager is Smart Enabled

Requires Smart Account on CCO

ISR 4000 (4451, 4431, and ISR 4351, 4331)

ISR 4321 and 4421 support in process for Container, Spring 2017

As a SW Only Agent we require

IOS-XE 3.16.0S / 15.4(3)S1 > LXC Container

APPX license Application Experience

ISR AX, AXV and C1 Bundle includes APPX

8 to 16G memory upgrade

(included in all ISR 4K C1 Bundles)

Option to add NIM-SSD 200Gb Storage for PCAP

Can also be deployed on UCS Blade ISR 28/38

Learning Network Agent


ISR 4K w/Learning Agent inside IOS XE

Cisco ISR 4000 Platform

Linux OS

IOSd

Control Plane

Platform-Specific Data Plane

Learning

Agent

Linux Service Container

Netflow and NBAR Data

Security monitoring now built inside your Cisco NG ISR 4K Router with dedicated core for AD Agent

Findings


Quick Status on SLN ...

Findings ?

• The system does learn, as expected

• Relevant detected anomalies (time of day, volume, unexpected flows, long live flows, ....)

• SLN detected anomalies it was not explicitly programmed for (Cognitive Computing)

• Does it detect everything without False Positive ? No, such systems simply do not exist but SLN learns and quicklyadjusts to customer relevancy learning

• The Place In the Network (PIN) is fundamental => dramatically extending the protection surface and visibility

BRKSEC-3056 57


• Tor = anonymous/tunneled browsing system based on encryption and multiple hops

• Host on Beta customer network opened SSL connection to 3 Tor nodes

• 2 are located in Europe, a 3rd one has a Japanese hostname but is geolocated in the US

Anomaly: Tor client on corporate network

BRKSEC-3056 58


• Host external to the branch performing a scan of ports TCP/22 & TCP/23

• Very subtle scan on a narrow scope and probing only two ports per host

Anomaly: retail branch subnet scanned for Telnet & SSH

BRKSEC-3056 59


• Abnormally high number of DNS requests for a printer

• Mix of UDP and TCP for DNS is also unusual

Anomaly: branch printer making numerous DNS requests over TCP & UDP

BRKSEC-3056 60


• Branch host is scanning addresses located elsewhere on the corporate network

• Wide port scan, NMAP-style

Anomaly: branch device scanning across the WAN

BRKSEC-3056 61


• New host appears on branch network and starts Windows logon sequence

• Behavior is unusual at this time of day (after 6pm local time)

Anomaly: new branch host detected at night

BRKSEC-3056 62


• Branch network device performs 280 TACACS+ requests in a few seconds

• Occurs while an SSH session to the device was active

• Most likely command authorization and/or accounting requests

Anomaly: SSH session causing a large number of TACACS+ requests

BRKSEC-3056 63


• Branch host downloads 2GB of data from an SSH server on the internet

• SSH connection terminates on port 443 which is assigned to HTTPS

• Manual check confirms port misuse, most likely to evade simple L4 firewalls

Anomaly: branch host transfers 2GB from SSH server running on HTTPS port

BRKSEC-3056 64


• Nearly a thousand incomplete TCP handshakes to a CIFS server within <1 minute; almost like a miniature SYN Flood attempt

Anomaly: branch host performs miniature SYN Flood on server

BRKSEC-3056 65


• Active malware Command & Control (C2) channel going to another country

• Using DNS as covert channel (not fully RFC compliant, but enough to be classified as DNS)

• Only detected by SLN, although FW and IPS/IDS were active on the network

Anomaly: malware Command & Control using DNS as covert channel

BRKSEC-3056 66

Conclusion


Public/Private

Internet

Agent Agent

X 1,000s

InternetController

ISE

ManagerThreat

Intel

Agent

...

(Hyper) Distributed Architecture ... ScaleThis *is* the challenge


Public/Private

Internet

Agent Agent

X 1,000s

InternetController

ISE

ManagerThreat

Intel

Agent

...


Learning ... Adaptive, Ease of UseWith dynamic False Positive Reduction


Public/Private

Internet

Agent Agent

X 1,000s

InternetController

ISE

ManagerThreat

Intel

Agent

...


Learning ... Adaptive, Ease of UseWith dynamic False Positive Reduction

Lightweight ... Pervasive

© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public* SLN DLA (Agent Arch) is specifically targeted for new NG HW from Cisco that support LXC Container, as a Cisco feature differentiator

Product Roadmap (subject to change)

3.X

Expanded footprint

• HW: ASR 1001/1002 , investigate NG

Switching

• Continue SMC Console integration

• Real-time alerting (email)*

• Mix of Manual/Automatic cluster

definition

• IPV6

• Investigate SLNL (QoS) shaping and

ACL capability

• HW: add ISR 4321 , ISR 4221, ENC

5400 w/ISRv, and CSR

• Integration with SMC (new SCA

Dashboard in SMC )

• Support for Polaris IOS XE 16.4,.5

• Reporting with email and POV

Reports

• External anomaly context

enrichment : Talos and ETTA

Extended capability& context enrichment

2.X

Enter market & gain validation

• HW ISR 4431/51 , 4351, 4331 ,and

UCS-E Blade

• ML driven detection of security

anomalies network , Reinforcement

Learning

• Initial mitigation capabilities (API)

• Central viewing of anomalies on the

Learning Manager

• Dynamic cluster creation

• PCAP

FCS 1.0 and 1.1

Q4 FY16 FY17 2H FY17

HW

SW


SLNL Part Numbers and OrderabilityPart Number Product Description

L-SW-LN-44-1Y-K9 Cisco Stealthwatch Learning Network License for Cisco 4400 Series Integrated Services Routers 1 Yr Term




L-SW-LN-UCS-1Y-K9 Cisco Stealthwatch Learning Network License for Cisco UCS 1 Yr Term

L-SW-LN-UCS-3Y-K9 Cisco Stealthwatch Learning Network License for Cisco UCS 3 Yr Term

L-SW-SCA-K9 Stealthwatch Learning Network Centralized Agent Manager

L-SW-LN-44-K9= Cisco Stealthwatch Learning Network Software for 4400 Series

L-SW-LN-43-K9= Cisco Stealthwatch Learning Network Software for 4300 Series

L-SW-LN-UCS-K9= Cisco Stealthwatch Learning Network Software for UCS Series

The 1Y and 3Y SKU’s above indicate the software term. The price for each is on Cisco Global Price List and in Cisco

Commerce (CCW). An equal sign (=) in the SKU denotes the software you download and is the master SKU for Ordering

https://cisco-apps.cisco.com/cisco/psn/commerce

BRKSEC-3056 73

https://cisco-apps.cisco.com/cisco/psn/commerce


Complete Your Online Session Evaluation

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online

• Please complete your Online Session Evaluations after each session

• Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt

• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations

74BRKSEC-3056

CiscoLive.com/Online


Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Lunch & Learn

• Meet the Engineer 1:1 meetings

• LTRSEC-2011 – SLN Deployment Lab (instructor-led)

• Thu 14:00 – 18:00 (this afternoon !)

• Hall 2 Level 1, Lab Room 601

BRKSEC-3056 75

Thank You

Securing Your Network with Anomaly Detection using ... Live Berlin 2017/BRKSEC-3056.pdf · Securing...

Documents

Transcript of Securing Your Network with Anomaly Detection using ... Live Berlin 2017/BRKSEC-3056.pdf · Securing...