© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Michael Capicotto, Solutions Architect

Matt Nowina, Solutions Architect

November 30, 2016

SAC304

Predictive SecurityUsing Big Data to Fortify Your Defenses

Cybersecurity headlines from 2015…

...Over 169 million personal records were exposed, stemming from 781

publicized breaches across the financial, business, education,

government and healthcare sectors.

...There were 38 percent more security incidents detected than in 2014.

...The median number of days that attackers stay dormant within a

network before detection is over 200.

... 81 percent reported they had neither a system nor a managed security

service in place to ensure they could self-detect data breaches, relying

instead on notification from an external party.

... Only 38 percent of global organizations claim they are prepared to

handle a sophisticated cyberattack.

You will learn how to…

Build a log analytics stack with Amazon Elasticsearch

Service

Utilize Amazon Machine Learning to predict bad actors

Perform forensic analysis on your network paths

Implement advanced options in your continuous,

predictive security stack

Big Data – Logs, logs everywhere

?Nobody looks at

them!

Big Data – Logs, logs everywhere…isn’t always good

Build a log analytics stack

Log sources in AWS

AWS CloudTrail logs OS and application

logs

VPC flow logs Amazon CloudWatch Logs

Setting up a log analytics stack

CloudWatch Logs Amazon Elasticsearch

Service

AWS Lambda

Demo #1 – Elasticsearch and Kibana

Awesome, we can see stuff!

Now we have real-time visualization of all logs

Great for risk scenarios we

already know about!

Example – Single user logging in from

several IP addresses

Not so great for unknown

scenarios

There are many of these!

How do we protect against these risks?

Integrating machine learning

Amazon Machine Learning

Easy to use,

managed machine

learning service built

for developers

Robust, powerful

machine learning

technology based on

Amazon’s internal

systems

One-click production

model deployment

Binary classification

Multiclass classification

Regression

Using Amazon Machine Learning’s real-time predictions, we

can drastically shorten how long it takes you to become aware

of a threat

Training your model (daily)

Amazon S3

Stores machine

learning dataset

AWS Lambda

Daily machine

learning model

training

Amazon Machine

Learning

Build model from

dataset

Log analytics

stack

AWS Lambda

Transform and

store logs in S3

Using Big Data – Example dataset

{

"datetime": "7/30/16 0:20",

"AWSregion": "aws-sa-east-1",

"IP": "69.90.60.155",

"protocol": "TCP",

"source": "6000",

"destination": "1433",

"country": ”BrVirginIslands",

"region": ”PricklyPear",

"postalcode": ”VG1120",

"Lat": ”18.5000",

"Long": ”64.3667”,

"Threat": 94

}

Real-time predictions

Amazon Machine

Learning

Endpoint for real-

time predictions

Log analytics

stackAWS Lambda

Trigger on each

new log entry

Amazon SNS

notification

Demo #2 – Real-time ML predictions

Security stack

Amazon Machine

Learning

Trained model and

endpoint for real-

time predictions

Log analytics

stackAWS Lambda

Trigger on each

new log entry

Amazon SNS

notification

Amazon S3

Stores machine

learning dataset

AWS Lambda

Daily machine

learning model

training

AWS Lambda

Transform and

store logs in S3

Close, but not perfect!

We still wont catch every potential breach Machine learning cannot predict every possible threat

Attackers are getting smarter and more sophisticated every day

When one does occur, we want to know why This helps us prevent it from happening again!

Forensic analysis

AWS Production Account

us-east-1a

us-east-1b

Pro

xie

s

NAT

RDS DB

DM

Z S

ub

ne

t

Priv

ate

Su

bn

et

Priv

ate

Su

bn

et

Pro

xie

s

Bastion

RDS DB

Priv

ate

Su

bn

et

Priv

ate

Su

bn

et

Virtual Private Cloud (VPC)

Network sprawl

AWS API Account

us-east-1a

us-east-1b

Priv

ate

Su

bn

et

Priv

ate

Su

bn

et

Virtual Private Cloud (VPC)

Reasoning about networks

Web service and CLI

available in private

beta

Answers questions

about your network

No packets sent

?

Demo #3 – Network reasoning

Demo

Advanced options

Evolving the practice of security architecture

Security architecture as a separate function can no longer

existStatic position papers,

architecture diagrams, and

documents

UI-dependent consoles and technologies

Auditing, assurance, and

compliance are decoupled,

separate processes

Current security

architecture

practice

Evolving the practice of security architecture

Architecture artifacts

(design choices, narrative,

etc.) committed to common

repositories

Complete solutions account for automation

Solution architectures are

living audit/compliance

artifacts and evidence in a

closed loop

Evolved security

architecture

practice

AWS

CodeCommit

AWS

CodePipeline Jenkins

Security architecture can now be part of the “maker” team

Continuous monitoring and auto-remediation

Self-managed AWS CloudTrail -> Amazon CloudWatch Logs -> Amazon CloudWatch Alerts

AWS CloudTrail -> Amazon SNS -> AWS Lambda -> Network reasoning

Compliance validation AWS Config Rules

Host-based compliance validation Amazon Inspector

Active change remediation Amazon CloudWatch Events

More sophisticated machine learning models

Train your model with your data Real-world data specific to your application

Previous threats you have dealt with

Considering modeling threats by clusters of logs Identify threats more accurately than just a single log entry

Build threat profiles that pattern typical attack stages Reconnaissance, scanning, gaining access, maintaining access, and

covering tracks

Tying it all together

Amazon Machine

Learning

Trained model and

endpoint for real-

time predictions

Log analytics

stackAWS Lambda

Trigger on each

new log entry

Amazon SNS

notification

Amazon S3

Stores machine

learning dataset

AWS Lambda

Daily machine

learning model

training

AWS Lambda

Transform and

store logs in S3

AWS Config Rules

Network

reasoning

VPC, security groups,

network ACLs

Next steps

Set up your log analytics stack: http://amzn.to/2dIZjIz Blog post and AWS CloudFormation template

Build your first Amazon ML machine learning model:

http://amzn.to/1K8HfRu

Stay tuned on the AWS Security Blog for more on this

topic

We’re here all week! Come chat with us.

Thank you!

Remember to complete

your evaluations!