Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web...
Transcript of Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web...
![Page 1: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/1.jpg)
Preview of Vulture’s upcoming web filtering engine
Pass the SALT 2018.Security And Libre Talks.2-4 juillet 2018 - Lille, France.
![Page 2: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/2.jpg)
Vulture ?
• A brief history• 2003: Linux software (httpd / mod_perl + PHP Web UI)
• 2016: FreeBSD Cluster (pf, haproxy, httpd + Django Web UI)
• Web SSO: mod_vulture + django portal
• Web application firewall• Clustered mod_security, using hiredis [blacklisting]
• mod_defender, aka "Naxsi for Apache2" [whitelisting]
• mod_svm [machine learning]
![Page 3: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/3.jpg)
Vulture’s current filtering engine
Client
FreeBSD pf
Apache httpd
IP Reputation
GeoIP
mod_defender
mod_security
mod_vulture
mod_svm
Immediate block
Immediate block
Authentication & SSO
Request scoring ++
Request scoring ++
Request scoring ++
![Page 4: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/4.jpg)
Current limits
• Works well
• No performance issue, but we can do much better
• 3 engines => quite complex• Code overlapping
• Complex UI, httpd knowledge recommended
• Rule-based approach
• Human-based approach• Time consuming
• Need tuning
• Not mistake proof configuration
![Page 5: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/5.jpg)
The need for a better, unified engine
• Focused on performance
• High availability required
• Precision and intelligence
• No bullshit
• Simplicity -> For users AND for filter devs
Internal name: « D.A.R.W.I.N. »
![Page 6: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/6.jpg)
OverviewArchitecture
![Page 7: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/7.jpg)
Overview: XSS Filter
XSS CoreFilter
Filter
Thread 1
Thread 2
Thread 3
Thread 4
...
Thread N
Monitoring
UNIX Socket
HTTP POST BODY
XSS Score: 87%
![Page 8: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/8.jpg)
Overview: Filter Workflow
Filter 1 (ex: reputation)
Filter 2 (ex: SQLi)
Filter N
D.A.R.W.I.N.
![Page 9: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/9.jpg)
Overview: Filter Workflow
Filter 1 (ex: reputation)
Filter 2 (ex: SQLi) ManagerManagement
Socket
Filter N
D.A.R.W.I.N.
![Page 10: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/10.jpg)
Performance ?
• HAProxy asynchronous events
• C/C++14
• UNIX socket
• Shared in-memory cache (REDIS)• Context-sharing between filters among the Vulture cluster
• Used by Darwin’s Neural Networks to track events in time
• Supports GPU acceleration• TensorFlow as AI library
![Page 11: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/11.jpg)
High Availability
Filter N
Decision Manager
Management Socket
{"type":"update_filters","filters":["Decision"]}
![Page 12: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/12.jpg)
High Availability
Filter N
Decision
Decision
Manager
Management Socket
{"type":"update_filters","filters":["Decision"]}
Manager
![Page 13: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/13.jpg)
High Availability
Filter N
Decision
Decision
Manager
Management Socket
{"type":"update_filters","filters":["Decision"]}
Manager
![Page 14: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/14.jpg)
High Availability
Filter N
Decision
Manager
Management Socket
{"type":"update_filters","filters":["Decision"]}
{"s tatus":"OK"}
Manager
![Page 15: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/15.jpg)
Precision and Intelligence
• Precision• Multiple small filters
• Very efficient for one unit task
• Ability to chain filters (workflow)
• Intelligence• Decision filter based on Artificial Intelligence
• Prediction based on filters’ results
• Active learning capabilities: Interact with human to correct itself
• Human focuses on high-level “decisions”• The AI manages the technical security rules
![Page 16: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/16.jpg)
No Bullshit
• Heuristic / basic correlation in a black box is not AI
• Those methods are promising but…• We use some of them in v3 (SVM, regression…)
• Few false-positives
• Unfortunately, few false-negatives: rules still needed
• We work hard to take it to the next level!• “AI first”: by design, not an add-on component
• Excellent results so far, beta-version coming this year
![Page 17: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/17.jpg)
Simplicity
• Easy for users• Minimalist configuration
• Autonomous system
• Simple feedback (Normal or Malicious request)
• Easy for developers• Filters mostly independent
• Simple SDK
• On Github soon ;)
![Page 18: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/18.jpg)
Portable
• Replace HAProxy with anything you want• Simply develop a connector
• Not only HTTP !
• Real world example (aDvens): DARWIN + Rsyslog• mmdarwin plugin
• Real time log analysis
• Real time log enrichment
• On Github soon… ;)
![Page 19: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/19.jpg)
Questions ?
![Page 20: Preview of Vulture’s upcoming web filtering engine · Preview of Vulture’supcoming web filtering engine Pass the SALT 2018. Security And Libre Talks. 2-4 juillet 2018 -Lille,](https://reader035.fdocuments.net/reader035/viewer/2022080720/5f79ed619b03e95236455aa9/html5/thumbnails/20.jpg)
[email protected]@advens.fr
https://www.vultureproject.orghttps://github.com/VultureProject/mod_defenderhttps://github.com/VultureProject/darwin (coming soon)
Thank You !