Streaming Algorithms for Robust, Real-Time Detection of DDoS Attacks
Next-Gen DDoS Detection
-
Upload
alex-henthorn-iwane -
Category
Technology
-
view
259 -
download
0
Transcript of Next-Gen DDoS Detection
![Page 1: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/1.jpg)
Next-Gen DDoS Detection:Leveraging the Power of Big Data Analytics
Jim Frey, VP Product, Kentik Technologies
February 24, 2016
![Page 2: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/2.jpg)
• Context: DDoS Landscape Today
• DDoS Defense Equation: Detection + Mitigation
• Case Example: DDoS Detection
• Big Data Analytics: Key to Advanced Detection
• Kentik’s Approach: NextGen DDoS Detection• Wrap-Up / Q&A
Agenda
2
![Page 3: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/3.jpg)
3
DDoS LandscapeA Clear and Present Danger
3
![Page 4: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/4.jpg)
DDoS Landscape Today (1/6)Who is Being Targeted?
Companies surveyed were attackedin 2014 or early 2015
Of those attacked were hitrepeatedly.
Source: Neustar DDoS Attacks & Protection Report: North America & EMEA, October 2015
Being attacked at least monthly Attacks lasted > 24 hours
4
![Page 5: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/5.jpg)
DDoS Landscape Today (2/6)
Goal: Take down target with sheer massive volume of requests or activity. Can be aimed at network or server resource exhaustion.
Examples:• TCP SYN Floods• UDP Floods (NTP, DNS, SSDP)• UDP Fragments• NTP Amplification• ICMP Flood
VolumetricGoal:
Starve target’s resources by making normal exchanges…. Take.... Way.... Longer.
Examples:• Slow Loris
• Sockstress• Slow HTTP GET
• Slow HTTP POST
Low and SlowGoal: Exploit specific Layer 7 protocol and application flaws to prevent normal function
Examples:• HTTP Flood• HTTPS Flood• DNS Amplification• RegEx• Hash Collision
Application Layer
Attack Types?
5
![Page 6: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/6.jpg)
DDoS Landscape Today (3/6)Mix is broad, and heavily infrastructure-focused
Source: Akamai State of the Internet (Security) report,Q3 2015
6
![Page 7: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/7.jpg)
DDoS Landscape Today (4/6)Size/Frequency Ramping
Increased attack frequency Quarter over Quarter
Increased average attack sizeQuarter over Quarter
Source: Verisign Distributed Denial of Service Trends Report, Q3 2015
Average attack size in Gbps 1 in 5 Attacks > 10 Gbps
7
![Page 8: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/8.jpg)
DDoS Landscape Today (5/6)Sources Vary…
Source: Akamai State of the Internet (Security) report, Q3 20158
![Page 9: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/9.jpg)
DDoS Landscape Today (6/6)Reflection Attacks on the Rise
Source: Akamai State of the Internet (Security) report, Q3 2015
9
![Page 10: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/10.jpg)
10
DDoS DefenseA Two-Part Challenge: Detect + Mitigate
10
![Page 11: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/11.jpg)
DDoS Defense Architecture: Requirements
- Real-time / sub-minute
- Accurate (no false positives, no false negatives)
- Flexible (can work with multiple mitigation strategies)
- Supportive of automation/integration
- Cost Effective
Detection
- Easy to configure
- Adaptable (can support new types of attacks)
- Automated
- Deployment options (in band vs. out of band, always on vs. on demand)
- Cost Effective
Mitigation
11
![Page 12: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/12.jpg)
DDoS Defense Architecture: Tech Options
Data Source
- Stateful Packet Inspection- Flow Monitoring (NetFlow, sFlow,
IPFIX)
Platform
- Appliances
- Downloadable Software- SaaS
Detection
- BGP RTBH
- Router ACL- BGP FlowSpec
- OpenFlow
- Cloud Scrubbing Service- On-Premises Scrubbing Appliances
- No Action
Mitigation
12
![Page 13: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/13.jpg)
End to End DDoS Protection: Attack Begins
Target Servers
Internet
Detector
Attack traffic
Legit traffic
Flow data 13
![Page 14: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/14.jpg)
End to End DDoS Protection: Direct Trigger to Edge
Internet
Detector
Attack traffic
Legit traffic
ACL, Flowspec, RTBH
Flow data 14
Operator Action or automated
script/programAlert
Target Servers
![Page 15: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/15.jpg)
End to End DDoS Protection: On-Prem Scrubber
Internet
Detector
Attack traffic
Legit traffic
Redirect to Mitigation
Flow data 15
DDoS Scrubber
Target Servers
![Page 16: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/16.jpg)
End to End DDoS Protection: Cloud Mitigation
Internet
Detector
Attack traffic
Legit traffic
Redirect to Mitigation
Flow data
Cloud Mitigation
Service
16
Target Servers
![Page 17: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/17.jpg)
17
DDoS DetectionThe Common Thread
17
![Page 18: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/18.jpg)
18
Case Example: DDoS AttackThings you may find when doing forensic DDoS analysis…
18
![Page 19: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/19.jpg)
19
Seemingly Normal Variations over Several Days….?
Starting Point: Total Traffic
19
![Page 20: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/20.jpg)
20
Looking at only SRC=CN (China)
Sorting by Source Geo
20
![Page 21: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/21.jpg)
21
Zooming in time range on Second Spike
Drilling Deeper
21
![Page 22: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/22.jpg)
22
Number of Unique Source IP Addresses
Checking another Dimension
22
![Page 23: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/23.jpg)
23
Flip to: Destination Addresses
Where is the Traffic Going?
23
![Page 24: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/24.jpg)
24
Looking at all inbound traffic to the target victim Dest IP
Pulling Back to Gauge the Situation
24
![Page 25: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/25.jpg)
25
Attack details by protocol
Narrowing in on the Actual Attack
25
![Page 26: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/26.jpg)
26
Multiple simultaneous vectors at hand
The Finding: Multi-Layered Attack
26
![Page 27: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/27.jpg)
27
Finding the Necessary Details for Setting Filter Policies
The Mitigation Plan
27
![Page 28: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/28.jpg)
28
- Unusual traffic patterns from suspect Geo- Turned out to be DNS Amplification targeting a specific dest IP- But main attack was hiding other attacks/exploits- Data harvested for mitigation
- Time required to complete this analysis: 3 minutes!- How is this possible???
Case Example: Summary
28
![Page 29: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/29.jpg)
29
Big Data Analytics for DDoSKey to Advanced DDoS Detection and Forensics
29
![Page 30: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/30.jpg)
DDoS Detection Tooling – Major Decision Points1. Packet-based or Flow-based?
• Packet-based requires in-line inspection, usu. via appliances ($$)
• Flow-based can be local/appliance or SaaS
2. Fully Integrated with Mitigation, or Best of Breed?
• Fully Integrated only works when mitigation is “always on”
• Independent detection ensures mitigation flexibility
3. Next-Gen Data Architecture, or Legacy?
30
![Page 31: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/31.jpg)
DDoS Detection Tooling – Data ArchitectureKey Question
“To Summarize or Not to Summarize??”
Advantages of Summarization
- More compact long term data store
- Faster (?) searches against history
Disadvantages of Summarization- Major Loss of essential detail!!
Only Viable Answer: NO SUMMARIZATION 31
![Page 32: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/32.jpg)
Big Data for Next-Gen DDoS DetectionWhy Big Data??Network Monitoring Data IS Big Data
• Meets Volume/Variety/Velocity Test
• Billions of records/day (millions/second)Big Data architectures:
• Mature, viable for hyper-scale, real-time data sets – SCALABLE, RELIABLE
• Capable of performance at scale for analyzing ALL data – not just summaries/metadata –RESULTS IN SECONDS
![Page 33: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/33.jpg)
Big Data Analytics: The DDoS Detection PayoffWhat Do I Get by Going With Big Data?
• Accuracy
• Having ALL raw data available, not just what was pre-defined
• Essential for answering key questions like: Is this Friend or Foe?
• Flexibility
• Don’t have to wait for vendor to support new attack profiles
• Easy to add more data types/sets to enrich the story
• Can export data quickly/easily to other systems
![Page 34: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/34.jpg)
Kentik’s ApproachNext Gen Big Data NetFlow Analytics for DDoS Detection…. And more
34
![Page 35: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/35.jpg)
Kentik Detect: the first and only SaaS SolutionFor Network Ops Management & Visibility at Terabit Scale
CLOUD- BAS ED REAL- T I M E MULT I - TENANT OP EN GLOBAL
Analyze & Take Action
Big Data NetworkTelemetry Platform
in the Cloud
The Network is the Sensor
Web Portal
Real-time & historical queries
NetFlow/sFlow/IPFIX
SNMPBGP
Alerts: DDoS, Ops
E-mail / Syslog / JSON
Open API
SQL / RESTful
Kentik Data Engine
35
![Page 36: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/36.jpg)
Multi-tiered/Clustered Big Data Architecture for Scale / Load Balancing / HA
What’s Behind Kentik Detect : The Kentik (big) Data Engine
POSTGRESSERVERS
SQL
DATA STORAGE CLUSTER
NetFlowSNMPBGP
INGEST CLUSTER
CLIENTS
N M
Optimized for Massive Data Ingest & Rapid Query Response36
![Page 37: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/37.jpg)
NextGen NetFlow Analytics: Full Detail, Fast Navigation, Infinite Granularity
37
![Page 38: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/38.jpg)
NextGen NetFlow Analytics: Dashboards in Seconds
38
![Page 39: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/39.jpg)
Key Takeaways
What NextGen DDoS Detection Can (Should) Do for You: - Deliver true live monitoring & alerting
- Quickly recognize / analyze attacks
- Operate on a full data set, not just summaries or pre-defined rules
- Support multiple mitigation options
- Enable automation
39
![Page 40: Next-Gen DDoS Detection](https://reader031.fdocuments.net/reader031/viewer/2022021922/58ed26cf1a28ab0a768b4625/html5/thumbnails/40.jpg)
Network Intelligence at Exabit Scale
Thank You!
Jim FreyVP Product
Kentik [email protected]
@jfrey80