Adventures in Large Scale HTTP Header Abuse
description
Transcript of Adventures in Large Scale HTTP Header Abuse
![Page 1: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/1.jpg)
Adventures in Large Scale HTTP Header Abuse
Zachary Wolff
![Page 2: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/2.jpg)
About Me
• SIEM Deployments, Research Engineer for LogRhythm (Labs) | 2+ years
• Threat Research Analyst for Webroot Software | 2+ years
![Page 3: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/3.jpg)
Lets Talk About HTTP Headers
Browser
Web Server
![Page 4: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/4.jpg)
• Standard Fields for Request & Response defined in RFC- 2616 (RTFRFC)
• GET / HTTP/1.0 is a legitimate request but may not return expected results
• RFC sets no limits on size of header, field name, value or number of headers
• Most webservers now impose their own limits :
HTTP Headers Basics
IISv4 – 2MBv5 – 128K-16KB*V6 – 16 KB*V7 – 16 KB*
Apachev2.3 – 8KB*
*Per Header Field
![Page 5: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/5.jpg)
Existing/Past HTTP Header Attacks
![Page 6: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/6.jpg)
The Premise
I want to break some logging applications!
![Page 7: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/7.jpg)
par exemple, to begin
![Page 8: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/8.jpg)
Round 1: Begin
Original Premise: GET Request returns 302, 200, (valid response) then send a second GET with a malicious User Agent string* to see if we can get 500 response
1. Crawler to collect URL’s2. Python script to send attack/test UA String 3. Store results in SQLite3 DB4. Profit!
![Page 9: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/9.jpg)
Round 1: Results
Data set: 400K URL’s
• Lots of 500’s! • Lots of smaller, low traffic site, some bigger
high traffic sites• Various different errors….
![Page 10: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/10.jpg)
Round 1: Results
Custom 500’s…
![Page 11: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/11.jpg)
Regular expression parsing errors….
![Page 12: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/12.jpg)
le yawn..
Non verbose IIS Errors…
![Page 13: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/13.jpg)
Not as boring, generic apache 500
and the x.x.gov sites….?
![Page 14: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/14.jpg)
Round 1: Conclusion
What did we find?
•Some SQL injectable 500’s•Possible application level DOS•Lots of websites are not expecting malicious Header requests…•Further exploration is warranted
![Page 15: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/15.jpg)
the Question
How extensive is the problem of improper HTTP header handling?
![Page 16: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/16.jpg)
Round 2: Begin
1. Need a more effective way to identify vulnerabilities 2. Lets attack/audit more than just User-Agent Header3. Expand beyond backtick, additional attack strings4. Larger sample set, 1.6 Million URL’s5. Must be able to store and access very large set of
result data efficiently (Shodan is amazing)
![Page 17: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/17.jpg)
Round 2: Vulnerability Identification
500’s are ok, but much to broad What is a good indication of a possible SQLi vulnerability?
Run regular Expression against HTML.data response to match on, “you have an error in your sql syntax”
![Page 18: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/18.jpg)
Round 2: Vulnerability Identification
Improved error detection, basic SQLi & beyonds[0] = "you\shave\san\serror" s[1] = "Warning.*supplied\sargument\sis\snot\sa\svalid\sMySQL\sresult" s[2] = "Warning.*mysql_.*\(\)" s[4] = "microsoft\sOLE\sDB\sProvider\sfor\sODBC\sDriver" s[5] = "Microsoft\sOLE\sDB\sProvider\sfor\sSQL\sServer" s[6] = "\[Microsoft\]\[ODBC Microsoft Access Driver\] Syntax error" s[7] = "Microsoft OLE DB Provider for ODBC Drivers.*\[Microsoft\]\[ODBC SQL Server Driver\]" s[8] = "Microsoft OLE DB Provider for ODBC Drivers.*\[Microsoft\]\[ODBC Access Driver\]" s[9] = "Microsoft JET Database Engine" s[10] = "ADODB.Command.*error" s[11] = "Microsoft VBScript runtime" s[12] = "Type mismatch | VBScript / ASP error" s[13] = "Server Error.*System\.Data\.OleDb\.OleDbException" s[14] = ":\squoted\sstring\snot\sproperly\sterminated" s[15] = "ORA-[0-9][0-9][0-9][0-9]" s[16] = "Invalid SQL statement or JDBC" s[17] = "org\.apache\.jasper\.JasperException" s[18] = "Warning.*failed to open stream" s[19] = "Fatal Error.*on line" s[20] = "Fatal Error.*at line" s[21] = "\[Microsoft\]\[SQL\sNative\sClient\]\[SQL\sServer\]" s[22] = "An\sunexpected\serror\shas\soccured!.*MySQL\serror!" s[23] = "\[Microsoft\]\[ODBC\sDriver\sManager\]\sDriver" s[24] = "\[Microsoft\]\[ODBC\sSQL\sServer\sDriver\]\[SQL\sServer\]" s[25] = "\[Microsoft\]\[SQL\sServer\sNative\sClient\s10\.0\]Named\sPipes\sProvider"
*Thanks to @j0emccray for contributing to regEx list
![Page 19: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/19.jpg)
Beyond RegEx based Error Detection
Byte Anomaly Detection Added (--bad)
Compare content-length of response data from original/clean GET to data from malicious GET.
*Set margin of alert to 150 bytes above and 150 bytes below clean request, log results (including HTML response data) to file
![Page 20: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/20.jpg)
Round 2: Additional Header Fields
• Let’s test: Host, From*, X-Forwarded-For, Referer, User-Agent, Non existent Header
• Smart Mode (-s) : Will look at all Header fields returned by the server and test those (minus whitelist of rarely dynamic Headers)
• Cookies!
![Page 21: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/21.jpg)
Cookie Support
Cookie Support added. Server Sends us this:
PyLobster Responds with this:
And the server says?
![Page 22: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/22.jpg)
Round 3: Updates
Updated Testing Values: “,;,%00, %00’
![Page 23: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/23.jpg)
Round 2: Design
“I Improved the crawler to harvest 500K+ URL’s a day. You should put my picture in your whitepaper”
Output additions (beyond SQLite):• Elasticsearch Indexing support added (fast, efficient, JSON
to webinterface)• Flat File logging
Mark Vankempen, LogRhythm Labs
![Page 24: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/24.jpg)
More Improvments
Added Footprint mode (-g)1. Generate random(ish) Hash or value2. Save to key.txt file in same directory as pylobster.py3. Activate Footprint mode: ./pylobster.py –g pyLobster will now send your unique string/hash as a request like so:
Then, Wait for it… Days, Weeks, MonthsGoogle/Bing/duckduckgo your hash/string to discover unprotected Log directories ;)
![Page 25: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/25.jpg)
pyLobsters maiden voyage
Ready Begin!
pyLobster is currently a single threaded tool so I divided my 1.6 Million URL’s into 78 unique lists and spawned 78 instances
#!/bin/bashnohup python pyLobster.py -f a --bad -s -l -g &nohup python pyLobster.py -f b --bad -s -l -g &nohup python pyLobster.py -f c --bad -s -l -g &nohup python pyLobster.py -f d --bad -s -l -g &
And so on……
![Page 26: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/26.jpg)
PyLobster’s Maiden Voyage Results
• Sending a null byte in your HTTP Headers will catch a fair bit of IDS attention ;)
• Grep response HTML on regEx error match directory to find patterns & specific components/module/application/CMS vulnerabilities. (highest value finding: one vulnerable component can lead to many others, shared DB’s as well)
• Various vulnerable components identified
![Page 27: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/27.jpg)
Findings: Breakdown by RegEx #
*
Out of 1.6 Million Unique URL’s, 14,500 Error RegEx’s Matched!
*0,1 & 2 are MySQL errors, 18 & 19 are PHP
![Page 28: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/28.jpg)
Findings: Error Breakdown by Test String
Of the 14,500 Error RegEx’s Matched
![Page 29: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/29.jpg)
Findings: Error breakdown by HTTP Header
*Cookies: 1584
![Page 30: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/30.jpg)
Findings: error #0, breakdown by header field
Error #0: “you have an error in you SQL syntax"
![Page 31: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/31.jpg)
Findings: Footprint Mode
Footprint Mode
12/13/2012
02/25/2013
![Page 32: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/32.jpg)
Foot Print Mode
3/27/2013
![Page 33: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/33.jpg)
Findings: (--bad)
Byte Anomaly Detection Results
• Work to be done….• grep over dir for [wordpress|joomla|error|
pass.*=|cms|.*?|] • Sort response files by size for like errors• Sort by status code response & file size
![Page 34: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/34.jpg)
![Page 35: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/35.jpg)
Defending Against HTTP Header Attacks
• Raise developer awareness that any dynamically handled Header values need to be seen as user input and processed accordingly
• Audit your sites HTTP Header Processing (pyLobster on github, SQLmap now supports custom Header testing too. bingo!)
• Proactively review/monitor your web logs
![Page 36: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/36.jpg)
This:
Creates this Log trail:
![Page 37: Adventures in Large Scale HTTP Header Abuse](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816592550346895dd85f28/html5/thumbnails/37.jpg)
The End
Thank you!
@nopsliphttps://github.com/nopslip/pyLobster