Who am I? Past: Commercial WAF developer since 2007 ModSecurity maintainer 2007 – 2010 IDS/IPS...
-
Upload
felix-cunningham -
Category
Documents
-
view
231 -
download
1
Transcript of Who am I? Past: Commercial WAF developer since 2007 ModSecurity maintainer 2007 – 2010 IDS/IPS...
Lockdown 2013
IronBee: Open source WAF engine with a commercial offering
Brian RectanusDirector of Engineering,
WAF
Lockdown 20132
Who am I?
Past:• Commercial WAF developer since 2007• ModSecurity maintainer 2007 – 2010• IDS/IPS Developer (OISF Suricata)
Present:• Lead WAF development @ Qualys in Madison• IronBee architect and developer
Lockdown 20133
What am I covering…
• Briefly: The what and why of WAF• IronBee, modules and rules• Overview of Qualys commercial WAF (beta)– How we use IronBee– How we have simplified the process– Beta features
Lockdown 20134
WAF: What is it?
• Web Application Firewall• To many this means:
“Block web based attacks.”
• But, WAFs are known to be a pain• There must be more
Lockdown 20135
WAF: Why does it exist?
• View inside your web applications• Log (and potentially block) suspicious activity• Block known and obvious attacks and tools• Limit attack surface• Buy time to fix problems• Feed your developers with more details• Deal with legacy products
Lockdown 20136
WAF: What gets in the way?
• Different interpretations of HTTP• Document types (HTML, XML, JSON)• Encodings (URL, Base64, entities)• Different Vectors (server, browser, DB, DoS)• Evasion techniques• Application logic (auth, sessions, BI)• Encryption, compression, obfuscation
Lockdown 20137
WAF: What can go wrong?
• False Positives (oops)• False Negatives (didn't see it)• Performance cannot suffer (too much)• Device failure (site is down)
Lockdown 20138
WAF: How can we make it better?
• Easier to setup and manage– Separate server/security configs and management– Low False Positives and low tuning costs– Flexible deployments with automated updates– Manage it all centrally
• Extensible engine– Solid framework for writing security logic– Integrate with other products– Combine many advanced techniques with correlation
• Acceptable performance– Intelligent application of security logic with fast algorithms
Lockdown 20139
IronBee: What is it?
• Open Source (Apache Software License v2)
github.com/ironbee
• Framework to inspect, block, modify and log• Extremely flexible• Highly extensible• Tries not to get in your way
Lockdown 201310
IronBee: Who is involved?Christopher Alfeld, PhD Mathematics and UW alumni
Experimental projects, performance, algorithms, C++ API
Sam BaskingerData structures, Configuration, Lua API
Nick Kew, Apache FoundationServer plugins: Apache Trafficserver, Apache httpd, nginx, tserver, …
Nick LeRoyCore engine, Testing
Brian RectanusInitial IronBee author, now architect and manager
Ivan RistićSecurity Research (SSL Labs – ssllabs.com, LibHTP, ModSecurity)
Many other supporting players at Qualys – too many to name here.
Lockdown 201311
IronBee: What's the basic concept?
• Server provides HTTP data– Web server, proxy, IDS, …
• Parsers break data into fields/streams– Headers, URI, POST body, cookies, …
• Modules/Rules inspect these fields/streams– Sigs, scoring, tracking, learning, correlation, …
• Actions performed:– Log, block, modify, track, …
Lockdown 201312
IronBee: What's a server?
• Provide HTTP data to IronBee• Implement blocking, modification (if possible)• Current:– Apache Trafficserver plugin– Apache Webserver module– Nginx plugin– Tserver (nginx fork) plugin– Clipp (command line with PCAP support)
Lockdown 201313
IronBee: What's the engine do?
• Notification of events• Core HTTP fields to inspect• Rule execution• Configuration• Logging
Very minimalistic, and becoming more so.
Lockdown 201314
IronBee: What are modules?
• Dynamically loadable shared libraries in C, C++• Minimal modules in Lua, but reloadable with config• Hook into IronBee events• Extend functionality (C/C++ only), such as:
– Parsers, normalizers, operators and actions– Rule languages (and extensions)– Embed scripting languages (Lua)– Enable technologies (libinjection - SQLi detection library)– Correlation (combine sigs, scoring, tracking, learning, …)– Logging– …
Lockdown 201315
IronBee: What are rules?
• Inspect data and perform actions• Simple signature language• Complex DSL (Lua @ config time)• Full scripting language (Lua @ runtime)• Extendible via modules
Lockdown 201316
Module: Simple Rule Language
Specify fields, inspect and perform an action:
Rule <fields> <op> <meta/actions>Rule REQUEST_HEADERS \
@rx "attack|pattern" \id:ex/1 rev:1 \phase:REQUEST_HEADER \event
Lockdown 201317
Module: Simple Rule Language
Transformations and meta data:
Rule REQUEST_HEADERS.count() \@gt 15 \id:ex/2 rev:1 \phase:REQUEST_HEADER \severity:75 confidence:80 \tag:http/limits \event
Lockdown 201318
Module: Simple Rule Language
Capture potential CC#s, blocking more than 10:
StreamInspect RESPONSE_BODY_STREAM \@dfa "\d{15,16}" \id:ex/3 rev:1 \capture:CC
Rule CC.count() \@gt 10 \id:ex/4 rev:1 \phase:RESPONSE_BODY \event block:immediate
Lockdown 201319
Module: Simple Rule Language
• These are just signature rules• Simple and come with limitations– Config file syntax (single line)– Somewhat verbose (requires id/phase)– No real flow control other than phase/file order
• Other types of rules eliminate these limits
Lockdown 201320
Module: Lua
• Embedded scripting language• As a configuration DSL (config time)• As a basic module (core engine runtime)• As a rule (rule engine runtime)
Lockdown 201321
Lua: As a DSL
DSL is named "waggle" (we like Bee themes here)
Rule REQUEST_HEADERS \@rx "attack|pattern" \id:ex/1 rev:1 \phase:REQUEST_HEADER \event
Sig("ex/1w", 1):fields("REQUEST_HEADERS"):op("rx", "attack|pattern"):phase("REQUEST_HEADER"):action("event")
Lockdown 201322
Lua: Programmatic Rules Config
Lua @ config time means full support for functions, loops, etc.
-- Parameterized rule with id/regexlocal function RequestRegex(id, regex)
return Sig("test/lua/" .. id, 1): fields("REQUEST_HEADERS”): op("rx", regex): phase("REQUEST"): actions("event”)end
-- Simplify management and readabilityRequestRegex(1, [[attack|pattern]])RequestRegex(2, [[attack2|pattern2]])
Lockdown 201323
Lua: Basic Modules
Lua executed at runtime to handle core engine events.
-- Get the IronBee Module object.local ibmod = ...
-- Define a function to handle an event.local function log_event(ib) ib:logInfo("Handling event=%s”, ib.event_name) return 0end
-- Register to be called with the event.ibmod:request_header_finished_event(log_event)
Lockdown 201324
Lua: Rules
• Similar to Lua module, but less complex• Lua executed by the rule execution engine• Entire script runs vs. using event callbacks
Lockdown 201325
Rules: Scaling to the non-trivial
• Simple linear execution with basic rules– Executes a list of rules per phase– All rules are executed
• What about 1000s or 100,000s of rules?• Need a way to limit execution• Need a way to specify dependencies/order• Need a way to cache results• Need a higher level of logic and correlation
Lockdown 201326
Rules: Made to be extended
• Rule injection• Modules can take ownership of rules• Modules can decide if/when rules execute• Currently two modules use this facility– Fast rules module– Predicate rules module
Lockdown 201327
Module: Fast Rules
• Adds a fast pattern (prequalification) to rules• Rules are executed only if prequalified• All fast rules utilize modified Aho-Corasick– Extensions to utilize fixed width patterns– Speed is independent of number of patterns– Works best with large rulesets– Some limitations
Lockdown 201328
Fast Rules: An example
Utility suggests fast patterns for existing rules by adding comments to rules
# FAST RE: ^(.+),\s*max-age[^,]+,?(.*)$# FAST Suggest: "fast:max-age[^,]"Rule RESPONSE_HEADERS:Cache-Control \ @rx "^(.+),\s*max-age[^,]+,?(.*)$" …
Rule RESPONSE_HEADERS:Cache-Control \ @rx "^(.+),\s*max-age[^,]+,?(.*)$" "fast:max-age[^,]" …
Lockdown 201329
Module: Predicate Rules
• Uses Lua DSL to produce predicate expressions(and (gt (atoi (field 'Content-Length')) 0) (streq 'GET' (field 'Request-Method')) )
• Complex rules are built from simple rules• Rules form an knowledge graph– Graph optimizations performed at configuration time– Common sub-expression merging & caching– Only required rules execute, and only once
• Combines Lua DSL and runtime optimizations– Full Lua support enhances configuration– Graph optimizations enhance runtime
Lockdown 201330
Predicate Rules: Named predicates-- Parameterized named predicatelocal function header(name)
return P.Field('REQUEST_HEADERS'):sub(name)end-- Named predicateslocal range_header_too_long = P.Gt(header('Range'):length(), 1000)local host_header_too_long = P.Gt(header('Host'):length(), 100)
-- Combine named predicates into a rule/signature-- NOTE: A "/" operator is overloaded for predicates to P.Or(…)Sig(”ex/p/1", 1):
predicate( range_header_too_long / host_header_too_long ):phase([[REQUEST_HEADER]]):action([[event]]):message([[Invalid HTTP header: too long.]])
Lockdown 201331
Predicate Rules: Lua DSL in actionlocal sensitive_file_patterns = { unix = [[(?:/etc/passwd|/etc/hosts|/etc/shadow|/bin/id)$]], java = [[(?:WEB-INF/web.xml|/conf/server.xml)$]], apache = [[(?:.htaccess|.htpasswd|.meta|.web)$]]}local function contains_sensitive_files(pattern) local r = P.false for i,v in ipairs({"REQUEST_URI_PATH", "REQUEST_HEADERS", "ARGS"}) r = P.Or(r, P.rx(pattern, P.Field(v):remove_whitespace())) end return rendfor name,pattern in pairs(sensitive_file_patterns) do Sig("qrs/LFi/" .. name, "1"):
predicate(contains_sensitive_files(pattern)):phase([[REQUEST_HEADER]]):action([[event]]):message("LFi: request for sensitive " .. name .. " files.")
end
Lockdown 201332
Framework: Automata
• Iron Automata (we also like Iron themes here)• Framework and utils for building automata• Splits generation, optimization, execution• Generic execution environment, Eudoxus• Example Automata: Enhanced Aho-Corasick– Caseless matches– Fixed width patterns/sets (char sets, negation– Can be tuned for space vs time through Eudoxus
Lockdown 201333
IronAutomata: Aho-Corasick Example1
• Aho-Corasick• Unoptimized• Patterns:– he– she– his– hers
Lockdown 201334
IronAutomata: Aho-Corasick Example2
• Aho-Corasick• Speed
Optimized• Patterns:– he– she– his– hers
Lockdown 201335
IronAutomata: Optimization
• Aho-Corasick• Patterns:
~250k English Dictionary
• Data:Text of "Pride and Predjudice" novel 10x
Lockdown 201336
Module: Eudoxus Executor
Execute compiled, eudoxus automata.• Large signature database– Spam keywords– Known attack patterns– Link reputation
• Custom, auto generated automata– Based on research– Based on website traffic profiling
Lockdown 201337
Utility: Clipp
• Command line utility• Testing and rule development• HTTP data via: Raw files, PCAP, protobuf, …• Modify HTTP data via filters• Convert between formats• Highly extendible• Ruby wrapper for unit/regression testing
Lockdown 201338
IronBee: Batteries not included
• Management is not dictated, so…• No Config Management• No Rule Management• No Log Management• Must do these yourself– You should already be doing this– The point is to stay out of your way– Allow you to use your own management tools
Lockdown 201339
Qualys WAF: What will it add?
• Managed WAF appliances via cloud• Automated updates– Software– Modules– Rules
• Integration with other Qualys products– Web Application Scanning– Asset Management
Lockdown 201340
Qualys WAF Beta: What's offered?
• Initially Amazon Web Services Platform– EC2 Classic and VPC– Clustering via ELB– Auto-scaling– You decide how much power you need
• We are expanding to other platforms
Lockdown 201341
Qualys Beta WAF: What's it do?
• Manage AWS based WAF Appliances• Generic attack detection• Declarative security (fixup cookies/headers)• Data leakage detection• Reduce attack surface (HTTP limitations)• ACLs (IP and geo)
Lockdown 201342
Qualys WAF Beta: What's it look like?
• Manage AWS Appliances• Manage events• Generic attack detection• Declarative security• Data leakage detection• Reduce attack surface• Access Control
Lockdown 201343
Qualys WAF Beta: AppSec
Lockdown 201344
Qualys WAF Beta: InfoLeak
Lockdown 201345
Qualys WAF Beta: Fixups
Lockdown 201346
Qualys WAF Beta: HTTP
Lockdown 201347
Qualys WAF Beta: ACLs
Lockdown 201348
Qualys WAF: What's coming?
• QualysGuard integration– WAS scan result feedback– Shared assets
• False positive mitigation• Exception handling• Website and session profiling• Reporting
Lockdown 201349
We are Hiring in the Madison!
• Product Management• Application Security Researchers• Developers• QA
Contact me if you are interested.
Lockdown 201350
Thanks!
github.com/ironbeequalys.com/waf
qualys.com/careers
Feel free to contact me for more info.
Brian [email protected]