Anomaly Detection for Security
-
Upload
cody-rioux -
Category
Data & Analytics
-
view
1.445 -
download
5
Transcript of Anomaly Detection for Security
![Page 1: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/1.jpg)
Anomaly Detection for Security
Cody Rioux - @codyriouxReal-Time Analytics - Insight Engineering
![Page 2: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/2.jpg)
Overview.● Real-Time Analytics● Anomaly: Fast Incident Detection
○ Techniques○ Case Study: Detecting Phishing○ Challenges: Base Rate Fallacy
● Outlier: Identifying Rogue Agents○ Clustering○ Case Study: Cleaning Up Rogue Agents
● Recap
![Page 3: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/3.jpg)
We are drowning in information but starved for knowledge.- John Naisbitt
Real-Time Analytics
![Page 4: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/4.jpg)
Real-Time Analytics● Part of Insight Engineering.● Build systems that make intelligent decisions about our operational environment.
○ Make decisions in near real-time.○ Automate actions in the production environment.
● Support operational availability and reliability.
![Page 5: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/5.jpg)
Terminology
Outlier Anomaly
![Page 6: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/6.jpg)
Case Study: Phishing● Just hired as the only security staff at a startup.● Fell victim to a phishing attack last week.
○ They did not know it happened when it was happening.○ They did not know what to do about it
● You’re tasked with solving this problem.
![Page 7: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/7.jpg)
Incident Detection for Stats Geeks
Anomaly Detection
![Page 8: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/8.jpg)
Unexpected value for a given generating mechanism.
![Page 9: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/9.jpg)
Terminology
Outlier Anomaly
![Page 10: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/10.jpg)
![Page 11: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/11.jpg)
![Page 12: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/12.jpg)
TechniquesBasic
● Static thresholds
● Exponential Smoothing
● Three-sigma rule
Advanced● Robust Anomaly Detection (RAD) - Netflix
● Kolmogorov-Smirnov
● Highest density interval (HDI)
● t-digest
● Linear models
![Page 13: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/13.jpg)
![Page 14: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/14.jpg)
![Page 15: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/15.jpg)
![Page 16: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/16.jpg)
![Page 17: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/17.jpg)
![Page 18: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/18.jpg)
TechniquesBasic
● Static thresholds - Doesn’t play well with nonstationary signals.● Exponential Smoothing - Black Swan days like Christmas, Superbowl cause issues.● Three-sigma rule - Works (very) well only for signals drawn from a Gaussian.
![Page 19: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/19.jpg)
Show me the Money!● No threshold configuration● We require examples of normal, not examples of anomaly● Automatically adapt to moving signals● Higher accuracy enables automatic reaction● Ensemble (combination) of techniques eliminates some
downsides
![Page 20: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/20.jpg)
Base Rate FallacyIntrusion is comparatively rare which affords you many opportunities to generate a false positive.
![Page 21: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/21.jpg)
Base Rate Fallacy
● 10,000 log entries● 99% Accuracy● 0.01% Intrusions
1 Real incident
100 false + and 10% chance of false -
![Page 22: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/22.jpg)
Case StudySo far we can automatically alert interested parties to the possibility of an intrusion.
![Page 23: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/23.jpg)
Identifying Rogue Agents in a Production Environment
Outlier Detection
![Page 24: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/24.jpg)
![Page 25: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/25.jpg)
Rogue Agents?
● Identify brute force attempts on login systems● Flag cheaters in online video games● Identify participating ip addresses in a
phishing scam
![Page 26: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/26.jpg)
Terminology
Outlier Anomaly
![Page 27: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/27.jpg)
Case Study RevisitedYou’ve devised an automated technique for identifying attacks, now we require an autonomous system for remediation of attacks.
![Page 28: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/28.jpg)
Goal: identify accounts and IP Addresses that are not behaving like their peers.
![Page 29: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/29.jpg)
Clustering● DBSCAN● K-Means● Gaussian Mixture Models
Conceptually● If a point belongs to a group it should be near lots of other points as measured by
some distance function.
![Page 30: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/30.jpg)
![Page 31: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/31.jpg)
Case Study RevisitedLets cluster accounts based on their login habits and initiate an automatic password reset and notification.
![Page 32: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/32.jpg)
Case Study RevisitedLets cluster IP addresses based on their login habits and automatically ban them.
![Page 33: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/33.jpg)
Full stack autonomous incident detection and remediation.
Recap
![Page 34: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/34.jpg)
Case Study Recap● Anomaly Detection enables us to...
○ Automatically identify potential attacks in real-time.○ Notify interested parties of the attack.○ React to those attacks without user intervention.
● Outlier Detection with Clustering enables us to…○ Identify rogue agents within the environment.○ Reset customer passwords for potentially compromised accounts.○ Ban IP Addresses identified to be participating in the phishing scheme.
![Page 35: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/35.jpg)
Literature
Machine Learning: The High
Interest Credit Card of Technical
Debt (Sculley et al., 2014)
![Page 36: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/36.jpg)
Literature● The Base-Rate Fallacy and its Implications for the Difficulty of Intrusion Detection
(Alexsson, 1999)● Practical Machine Learning: A New Look at Anomaly Detection (Dunning, 2014)● ALADIN: Active Learning of Anomalies to Detect Intrusion (Stokes and Platt, 2008)● Distinguishing cause from effect using observational data: methods and benchmarks
(Mooij et al., 2014)● Enhancing Performance Prediction Robustness by Combining Analytical Modeling
and Machine Learning (Didona et al., 2015)
![Page 37: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/37.jpg)
Implementations
● Robust Anomaly Detection (RAD) - Netflix ● Seasonal Hybrid ESD - Twitter● Extendible Generic Anomaly Detection
System (EGADS) - Yahoo● Kale - Etsy
![Page 38: Anomaly Detection for Security](https://reader035.fdocuments.net/reader035/viewer/2022062412/587a25651a28abbd388b4f83/html5/thumbnails/38.jpg)
[email protected]@codyriouxlinkedin.com/in/codyrioux