THE JOY OF PROACTIVE SECURITYSCOTT BEHRENS && ANDY HOERNECKE

EHLO• Scott Behrens

• Application Security Engineer at Netflix

• Likes automation

• Torn between breaking and making

• Loves research and open source

HELO• Andy Hoernecke

• Application Security Engineer at Netflix

• App Sec

• Data Visualization

What We Will Cover• Challenges of a modern infrastructure

• Proactive security as a solution

• Examination of mature security program

• How to get started

• Conclusion

Terminology Primer• Define technology terms:

• Application

• Instance

• Autoscaling groups (ASGs) / Load balancers (ELBs)

• AMI

• SecurityGroups

• Regions / Availability Zones (AZs)

Netflix Primer• 100's of Developers

• Over 1,000 applications

• Hundreds of production pushes a day

• Over 50k instances

• No Security Gates!

Continuous Deployment• Can be fully automated

• Immutable Platform

• Looks like:

• git -> Create deb (Jenkins) -> AMI Snapshot -> Deploy

Potential Pitfalls• Multiple concurrent code-bases (A/B testing,

regional functionality, etc.)

• New applications brought online/old applications retired constantly

• Insecure third party dependencies

More Pitfalls• How do we identify and catalog assets (IP?, DNS

Name?, Application?, ELB?...)

• How do we profile and provide baseline security to new applications?

• How do we monitor security policy changes and configurations in AWS

• Monitoring the web for credential dumps, hacktivism, sensitive data exposure?

The List Goes On...• Many of these problems are not unique to

Netflix/the cloud

• Some are unique or can be solved in different ways

• We have coined our approach to these and other problems as...

Proactive Security• Dictionary.com

• Proactive - "serving to prepare for, intervene in, or control an expected occurrence or situation, especially a negative or difficult one; anticipatory"

Proactive Security• Security controls should be:

• Integrated, automated, scalable, adaptive, actionable, and intelligent

• Time is limited

• End goal: Less babysitting, more time for harder problems

Proactive Security• Find problems early and address them

• Know your weaknesses and work to improve them

• Monitor for anomalies and be prepared to respond

• Collect meaningful data and use it to improve

• Simplify make security the easy path

• Reevaluate your approach

• Share what you learn with others

Find• Identify issues early

• Old hat:

• Static asset lists, fileshare with old pentest reports

problems early and address them

Find - New Way• Define what an asset is:

• Application

• ELB

• FQDN

• IP

Find - New Way• Have an intelligent way to collect/

track assets and vulns

Monterey

Monterey• Provides way to automatically define and scan

assets

• Soon to be open sourced

• Asset groups as application name:

• Contain any ELBs or FQDN

• Do not contain instance

Monterey• Monterey queries AWS for new assets and

changes

• Monterey runs "Monklets" for things like vulnerability scanning

• Store vulns centrally in S3

Monterey Demo

Monitor• Monitor, detect, and respond to security issues

• Old hat:

• Reactionary

• New hat:

• Less reactionary aka. more proactive

• Automatic

• Intelligent

for anomalies and be prepared to respond

Simian Army• aka The Monkeys!

• Open source

• Proactively wreck your environment to simulate outages

• Take down apps, instances, ELBs, even entire regions

• Ensures developers write resilient code

"Dirty Laundry" Project• Look for assets unintentionally exposed

• Leverages Monterey for assets

• Uses Scumblr for actioning findings

• Uses Sketchy for collecting status codes, generating screenshots, and text scrapes

• Both Sketchy/Scumblr are open source

Scumblr

• Intelligent automation platform aka. our swiss army knife

• Monitor things by using plugins

• Credential Dumps

• Hacktivism

• Brand Reputation

• Full Disclosure

• Leverages Sketchy for screen shots and text scrapes

• Custom Workflows

Dirty Laundry Demo

Speedbump• Mechanism to detect attacks and enforce security policies

automatically

• You define what you want to monitor/filter

• Security policies can be time delays, blocking, routing, etc.

• A WAF, proxy and firewall on steroids because it has application intelligence (business logic)

• Ensnare

Security Monkey

• Open Source

• Monitor events/changes within AWS

• Changes to IAM users

• Security groups

• Policy changes

• Notify when things change

Have you noticed a trend?

• A more proactive approach

• Identify something interesting automatically

• Notify automatically

• Provide workflows for more complex remediations

Know• None of us have unlimited time or resources--we all have

weaknesses

• Work to automatically identify blindspots

• Leverage tools and information where useful

• Old hat:

• Spreadsheet containing static list of apps and manually created risk rankings

• Not scalable when we are talking 100's or 1000's of apps

your weaknesses and work to improve them

Penguin Shortbread• Catalog assets

• Measure attributes

• Calculate risk

• Will be open sourced if we can make it more generic (TBD)

Penguin Shortbread Demo

Case Study: Monklet Your Way To Infinity

• Shellshock

• Vulnerable vs exploitable?

• Some commercial scanners only hit root of webpage

• What about other scripts?

Case Study: Monklet Your Way To Infinity

• Monterey Monklet!

• Spiders a provided asset (gevented)

• Checks each page for exploitability of Shellshock

• Stores results in S3 centrally

• Deploy N instances of Monklet to cover 100's of apps

Collect• Collect data from intelligent systems

• Actually use it to:

• Drive initiatives

• Find what is/isn't working

meaningful data and use it to improve

Example

Simplify• If it's easy developers will do it

• Make security an enabler and not a burden

• Encourage developers to come to you

• Also: v0.1 doesn't need to be perfect

make security the easy path

Immutable Base AMI• Make your platform consistent

• For non-cloud users: system image/template

• Conformity Monkey for AWS users (open source)

Danger! Danger!• Applications often use legacy/outdated dependencies

• Make it clear to developers they are using dangerous stuff

• Netflix leverages an API based on OWASP Dependency Checker

• API will eventually be open sourced

• Provides information into our Dependency tool

• Work in progress...

Danger! Danger!

FindSecBugs• Allow users to opt-in to static analysis

• FindSecBugs is lightweight and pretty useful (for Java)

• They can still deploy regardless but no one likes seeing red

Reevaluate• Environment is always changing--what works today

may be ineffective tomorrow

• Developers are agile, which means we must be too

• Be willing to start simple, knowing the initial solution is temporary (Don't over-engineer!)

• Use the data you've collected to determine when an approach is no longer sufficient

your approach

Share• We (security professionals) are all working to solve

similar problems

• Sharing tools and information make all of our lives easier

• Allows improvement through the collective consciousness of the industry

• Prevents duplicate effort

what you learn with others

http://netflix.github.io/

(More to come!)

Proactive Security: Level 1

• Create a list of assets

• Rank by sensitivity

• Automate

Proactive Security: Level 2

• Strategize security based on asset risk. Ex:

1. Sensitive assets: Examine thoroughly

2. Frequently used components: Secure by default

3. Everything else: Find a good baseline level of due diligence

Proactive Security: Level 3• Identify weak links

• Architectural: Poor network segmentation

• Cultural: Developers don't understand XSS

• Specific: "Application X" relies on an outdated version of Struts

• Determine how to measure the weaknesses and put in place a plan to address or manage

Proactive Security: Level 4

• Monitor, alert, and gather more intel

• Detect anomalies in security relevant functionality: Login successes/failures, password resets, authorization failures...

• Dashboards are fun, but intelligent alerting is better

• "Effective Approaches to Web Application Security" by Zane Lackey

Proactive Security: All Levels

• Continually drive improvements into your tools and processes

• Be flexible and ready to adapt

• Share what you've learned and done!

Works Cited• http://www.netbraintech.com/products/feature-

guide/images/asset-report-step-5-2.png

• http://media.amazonwebservices.com/architecturecenter/AWS_ac_ra_web_01.pdf

• http://www.slideshare.net/zanelackey/effective-approaches-to-web-application-security

Tools• Netflix OSS

https://netflix.github.io/

• OWASP Dependency Check https://www.owasp.org/index.php/OWASP_Dependency_Check

• Ensnare https://github.com/ahoernecke/ensnare

• FindSecBugshttps://github.com/h3xstream/find-sec-bugs

Thanks!• Contact Us:

• sbehrens@netflix.com

• ahoernecke@netflix.com

• OSS: http://netflix.github.io/

• Want to be part of the team? We're hiring!

• Questions?