hashdays 2011: Christian Bockermann - Protecting Databases with Trees
-
Upload
area41 -
Category
Technology
-
view
719 -
download
4
description
Transcript of hashdays 2011: Christian Bockermann - Protecting Databases with Trees
Christian Bockermann - chris @ jwall.org
A syntax-based approach to detect SQL injections
Protecting Databaseswith Trees
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
About Me
Researcher of the Artificial Intelligence Group at the University of Dortmund, Germany
Studying machine learning methods for web-security
Developer of several projects supplementoryto ModSecurity
AuditViewer, AuditConsole
Web Policy Compiler, Web Application Profiler
jwall-tools
Computer Science Department Artificial Intelligence Group
www.jwall.org@jwallorg
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Outline
Are SQL injections still a threat?
Where to fight SQL injections?
Protecting Databases with Trees
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
We start like everySQL injection talk...
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
$name = $_POST[‘name‘]; // $name = “Robert‘); DROP TABLE Students; --“
$insert = “INSERT INTO STUDENTS VALUES (‘$name‘);“;
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
$name = $_POST[‘name‘]; // $name = “Robert‘); DROP TABLE Students; --“
$insert = “INSERT INTO STUDENTS VALUES (‘$name‘);“;
INSERT INTO STUDENTS VALUES (‘Robert‘); DROP TABLE Students; -- ‘);
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
It‘s over 5 years old!
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
New Sony Hack Claims Over a Million User Passwords
2011-06-02
Sony Europe hacked by Lebanese hacker... Again
2011-06-04
LulzSec has compromised SonyPictures.RU
2011-06-05
DDoS attack takes down Atlassian's SaaS platform
2011-06-06Sony Portugal latest to fall to hackers
2011-06-09
FBI Partner Organization Website Hacked
2011-06-06
Hacker breaks into MIT website
2011-06-08
Citigroup Card Customers’ Data Hacked
2011-06-09
The Web Hacking Incident Database, June 2011
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
New Sony Hack Claims Over a Million User Passwords
2011-06-02
Sony Europe hacked by Lebanese hacker... Again
2011-06-04
LulzSec has compromised SonyPictures.RU
2011-06-05
DDoS attack takes down Atlassian's SaaS platform
2011-06-06Sony Portugal latest to fall to hackers
2011-06-09
FBI Partner Organization Website Hacked
2011-06-06
Hacker breaks into MIT website
2011-06-08
Citigroup Card Customers’ Data Hacked
2011-06-09
The Web Hacking Incident Database, June 2011
SQL Injection
SQL Injection
SQL Injection
SQL Injection
SQL Injection
SQL Injection
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
The Web Hacking Incident Database, June 2011
SQL Injectiondos/ddosother
Web Hacking Incident Database, June 2011http://projects.webappsec.org/
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Imperva Monthly Trend Report #4, September 2011http://www.imperva.com/download.asp?id=352
Monitoring 30 web applications:
on average 71 SQL injection attempts per hour
800-1300 injection attempts at peak times
Use of highly automated SQL injection tools, e.g. sqlmap, Havij,...
Imperva‘s Trend Report #4 Anatomy of a SQL attack
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Imperva‘s recent Hack-Forum Analysis
spamSQL Injectiondos/ddoszero-dayshell codebrute forceHTML injection Imperva Monthly Trend Report #5, October 2011
http://www.imperva.com/download.asp?id=327
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Just a couple of days ago...
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
OWASP - Open Web Application Security Projecthttp://www.owasp.org/
Motivation - Top 10 Attacks
The Open Web Application Security Project lists the Top-10 vulnerabilities:
1.Injection Flaws (SQL-Injection, RFI, ...)
2.Cross Site Scripting (XSS)
3.Broken Authentication / Session Management
4.Insecure Direct Object Reference
5.Cross Site Request Forgery (CSRF)
6.Security Misconfiguration
7.Malicious File Execution (Remote File Inclusion)
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Mitre Top-25
Improper Neutralization of SQL Elements
Improper Neutralization of OS commands
Buffer copy without size-check
Improper Neutralization of Input during page generation
Missing Authentication of critical functions
Missing Authorization
Use of hard-coded credentials
Missing Encryption of sensitive data
Unrestricted file uploads of dangerous file-types
...
SQL Injection
Command Injection
Buffer overflow
Cross-Site Scripting
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Not only web...
A SQL injection vulnerability in Symantec's Sygate Management Server (SMS) version 4.1, build 1417 and earlier could potentially allow a remote or local attacker to gain administrative privileges to the SMS server.
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
/query?search=security
SELECT title,abstract FROM DOCS WHERE txt LIKE ‘%security%‘
Web Server
database
Web Application
What makes a SQL injection?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
1. Attacker injects SQL code into the application
2. injection alters the statement that is executed
/query?search=security
SELECT title,abstract FROM DOCS WHERE txt LIKE ‘%security%‘
Web Server
database
Web Application
What makes a SQL injection?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
What makes a SQL injection?
Web Server
database
Web Application
/query?search=`+UNION+SELECT+LOGIN,PASSWORD+FROM+USERS; --
SELECT title,abstract FROM DOCS WHERE txt LIKE ‘%‘ UNION SELECT LOGIN,PASS FROM USERS; --%`;
1. Attacker injects SQL code into the application
2. injection alters the statement that is executed
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Where to fight SQL injections?
Web Server
database
Web Application
/query?search=security
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Where to fight SQL injections?
Web Server
database
Web Application
/query?search=security
Within the app‘s code
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Where to fight SQL injections?
Web Server
database
Web Application
/query?search=security
Within the HTTP traffic
Within the app‘s code
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Where to fight SQL injections?
Web Server
database
Web Application
/query?search=security
Within the HTTP traffic
Within the app‘s code
Within the executed SQL
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Fighting SQL injections (code)
Conceptual approaches to elevatethe security of web applications
Specifications, Developer Trainings
Penetration testing, Code Reviews
Risk Management
OWASP Guides
Code Review GuideEoin Keary et.al.
Testing GuideMatteo Meucci et.al.
Backend Security ProjectCarlo Pelliccioni et.al.
AppSensors ProjectMichael Coates, Colin Watson et.al.
Risk Management Strategies
STRIDE / DREADJ.D. Meier et. al. Microsoft 2005
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Fighting SQL injections (code)
Prepared statements can help a lot
PreparedStatement p = con.prepareStatement( “SELECT * FROM USERS WHERE login = ?“ );
p.setParameter( 0, username );p.executeQuery();
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Fighting SQL injections (code)
PreparedStatement p = con.prepareStatement( “SELECT * FROM USERS WHERE login = “ + username );p.executeQuery();
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Fighting SQL injections (code)
PreparedStatement p = con.prepareStatement( “SELECT * FROM USERS WHERE login = “ + username );p.executeQuery();
Proper use of prepared statements required!
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Yes, the whole intrusion detection (and prevention ...) game is ‚just‘ a big attempt to ‚patch‘ bugged systems...
Damiano Bolzoni, focus-ids mailing list 10/2008
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Fighting SQL injections (waf/ids)
External approaches to web security
Intrusion Detection SystemsPHPIDS, Snort
Web Application FirewallsModSecurity,... Web Server
database
Web Application
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
WAF / IDS usually check for SQL attack patterns in HTTP requests, i.e. the user input
Fighting SQL injections (waf/ids)
/query?search=`+UNION+SELECT+NAME,PASSWORD+FROM+USERS; --
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
WAF / IDS usually check for SQL attack patterns in HTTP requests, i.e. the user input
Fighting SQL injections (waf/ids)
/query?search=`+UNION+SELECT+NAME,PASSWORD+FROM+USERS; --
UNIONSELECTDROP TABLEINSERT
Any SQL keywords contained??
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Almost(?) all WAF/IDS approaches follow this black-listing or pattern based approach
The ModSecurity Core-Rules
IBM Web Application Firewall
Imperva SecureSphere
AQTRONIX Webknight
PHPIDS
Snort
Fighting SQL injections (waf/ids)
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
It‘s not just keywords, it‘s mostly regular expressions
Fighting SQL injections (waf/ids)
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
It‘s not just keywords, it‘s mostly regular expressions
"(?:\b(?:(?:s(?:elect\b(?:.{1,100}?\b(?:(?:length|count|top)\b.{1,100}?\bfrom|from\b.{1,100}?\bwhere)|.*?\b(?:d(?:ump\b.*\bfrom|ata_type)|(?:to_(?:numbe|cha)|inst)r))|p_(?:(?:addextendedpro|sqlexe)c|(?:oacreat|prepar)e|execute(?:sql)?|makewebtask)ql_(?:longvar char|variant))|xp_(?:reg(?:re(?:movemultistring|ad)|delete(?:value|key)enum(?:value|key)s|addmultistring|write)|e(?:xecresultset|numdsn)|(?:terminat|dirtre)e|availablemedia|loginconfig|cmdshell|filelist|makecab|ntsec)|u(?:nion\b.{1,100}?\bselect|tl_(?:file|http))|group\b.*\bby\b.{1,100}?\bhaving|d(?:elete\b\W*?\bfrom|bms_java)|load\b\W*?\bdata\b.*\binfile|(?:n?varcha|tbcreato)r)\b|i(?:n(?:to\b\W*?\b(?:dump|out)file|sert\b\W*?\binto|ner\b\W*?\bjoin)\b|(?:f(?:\b\W*?\(\W*?\bbenchmark|null\b)|snull\b)\W*?\()|a(?:nd\b ?(?:\d{1,10}|[\'\"][^=]{1,10}[\'\"]) ?[=<>]+|utonomous_transaction\b)|o(?:r\b ?(?:\d{1,10}|[\'\"][^=]{1,10}[\'\"]) ?[=<>]+|pen(?:rowset|query)\b)|having\b ?(?:\d{1,10}|[\'\"][^=]{1,10}[\'\"]) ?[=<>]+|print\b\W*?\@\@|cast\b\W*?\()|(?:;\W*?\b(?:shutdown|drop)|\@\@version)\b|'(?:s(?:qloledb|a)|msdasql|dbo)')"
Fighting SQL injections (waf/ids)
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Fighting SQL injections (waf/ids)
How do keyword/regex filters work with text?
„The following Cookie triggers [rule] 981248
LtpaToken2=x5Orq
(it didn't like "50r"?????)“
ModSecurity Core Rules Mailing list, 2.9.2011
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Some WAFs try to remove keywords from inputs
Replacements can easily be evaded:
id=1+UNunionION+SEselectLECT+1,2,3--
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Some WAFs try to remove keywords from inputs
Replacements can easily be evaded:
id=1+UNunionION+SEselectLECT+1,2,3--
This would likely result in a database error
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Some WAFs try to remove keywords from inputs
Replacements can easily be evaded:
id=1+UNunionION+SEselectLECT+1,2,3--
id=1+UNION+SELECT+1,2,3--
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Some WAFs try to remove keywords from inputs
Replacements can easily be evaded:
id=1+UNunionION+SEselectLECT+1,2,3--
id=1+UNION+SELECT+1,2,3--
This looks like a good candidate for a successful SQL injection
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
There are many approaches to evadepattern detection:
by encoding
by obfuscation
by HTTP parameter pollution
SQLi filter evasion and obfuscationJohannes Dahse, RUBat Conference Confidence 2.0
Beyond SQLi: Obfuscate and BypassBy CWH Undergroundwww.exploit-db.com/papers/17934
Bypassing PHPIDS 0.6.5Michael Brooks (https://sitewat.ch)
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Some WAFs do only a single decoding before filtering
double encoding your attack will bypass the WAF
http://victim.com/news.php?id=1%252f%252a*/union%252f%252a*/select%252f%252a*/1,2,3%252f%252a*/from%252f%252a*/users--
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Some WAFs do only a single decoding before filtering
double encoding your attack will bypass the WAF
http://victim.com/news.php?id=1%2f%2a*/union%2f%2a*/select%2f%2a*/1,2,3%2f%2a*/from%2f%2a*/users--
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Some WAFs do only a single decoding before filtering
double encoding your attack will bypass the WAF
http://victim.com/news.php?id=1/**/union/**/select/**/1,2,3/**/from/**/users--
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
SQL injections possible in a lot of places
Example: ASP viewState variable
Stores client-side state
base64-encoded
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
SQL injections possible in a lot of places
Example: ASP viewState variable
Stores client-side state
base64-encoded
aWQ9YCBPUiAxID4gMDtzdGF0ZT17dmFyOiJBQkMiLGNvbHVtbnM6M30
What about this?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
SQL injections possible in a lot of places
Example: ASP viewState variable
Stores client-side state
base64-encoded
id=` OR 1 > 0;state={var:"ABC",columns:3}
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Catching /* */ comments is sometimes not enough
MySQL allows for 3 different types of comments
starting with # until end-of-line
starting with -- until end-of-line
C-style comments /* */
Evading Pattern Detection
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Most DBMS are pretty tolerant regarding their SQL
MySQL does accept comments in a lot of places
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Most DBMS are pretty tolerant regarding their SQL
MySQL does accept comments in a lot of places
id=1/**/union/**/select/**/1,2/**/from/**/users--
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
The following comment based evasion was used to bypass a simple ModSecurity CRS rule
http://victim.com/news.php?id=0+div+1+union%23foo*%2F*bar%0D%0Aselect%23foo%0D%0A1%2C2%2Ccurrent_user
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
The following comment based evasion was used to bypass a simple ModSecurity CRS rule
http://victim.com/news.php?id=0 div 1 union#foo*/*barselect#foo1,2,current_user
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Special format of MySQL comments allows inline-code
Can be extended for version specific code
Evading Pattern Detection
/*! MySQL code */
CREATE /*!32302 TEMPORARY */ TABLE t ..
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
This allows for another style of comments
Used to bypass ModSecurity CRS and Wapple WAF:
/news.php?id=1/*!UnIoN*/SeLecT+1,2,3--
1 ||1=1
1 /*!order by*/ 3
1 /*!union select*/ 1,table_name from /*!information_schema.tables*/
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Patterns like ` OR 1 > 0` can be evaded by an endless repertoire of variants
` OR 2*3 > 4
` OR @@version == @@version
` OR 1
` or round(pi(),1) + 1 + 1 = version()
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Exploiting different views of WAF and Application
Web Server Interpretation Example
ASP.NET/IIS Concat by comma id=val1,val2
ASP/IIS Last parameter id=val2
PHP/Apache First parameter id=val1
JSP/Tomcat First parameter id=val1
DBMan Concat by tildes id=val1~~val2
/news.php?id=val1&id=val2
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Bypassing ModSecurity CRS with HPP
/?id=select name&id=password from users
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Bypassing ModSecurity CRS with HPP
ModSecurity filter view:
/?id=select name&id=password from users
id=select nameid=password from users
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Bypassing ModSecurity CRS with HPP
ModSecurity filter view:
Backend ASP application:
/?id=select name&id=password from users
id=select nameid=password from users
id=select name,password from users
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Parameter pollution+variants have been used to evade
ModSecurity CRS
IBM Web Application Firewall
Imperva SecureSphere
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Evading Pattern Detection
Parameter pollution+variants have been used to evade
ModSecurity CRS
IBM Web Application Firewall
Imperva SecureSphere
So, how to evade the evasion?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Taking a different perspective
Web Server
database
Web Application
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Taking a different perspective
Web Server
database
Web Application
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Where to detect injections?
Web Server
database
Web Application
At this detection point
any encodings have been decoded by the web-server and the application
no more „mangling“ is doneprior execution
any encoding-based evasionsdo not apply anymore
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
1. SQL injection needs to modify existing statement
2. modified statement needs to be valid SQL
What makes a successful SQLi?
SELECT title,abstract FROM DOCS WHERE txt LIKE ‘DROP TABLE STUDENTS‘;
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
1. SQL injection needs to modify existing statement
2. modified statement needs to be valid SQL
What makes a successful SQLi?
SELECT title,abstract FROM DOCS WHERE txt LIKE ‘%‘ UNION SELECT LOGIN,PASS FROM USERS; --%`;
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
1. SQL injection needs to modify existing statement
2. modified statement needs to be valid SQL
What makes a successful SQLi?
SELECT title,abstract FROM DOCS WHERE txt LIKE ‘%‘ UNION SELECT LOGIN,PASS FROM USERS; --%`;
How do we „capture“ such modifications?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Structure of SQL
SQL is a highly structured language (ISO SQL-92, ISO/IEC9075:2003, ...)
statements parsed to abstract syntax tree
AST presents the structure of a statement
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Structure of SQL
SQL is a highly structured language (ISO SQL-92, ISO/IEC9075:2003, ...)
statements parsed to abstract syntax tree
AST presents the structure of a statement
SELECT title,abstract FROM DOCS WHERE txt LIKE ‘%security%‘;
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Structure of SQL
SQL is a highly structured language (ISO SQL-92, ISO/IEC9075:2003, ...)
statements parsed to abstract syntax tree
AST presents the structure of a statement
SELECT title,abstract FROM DOCS WHERE txt LIKE ‘%security%‘;
SELECT
`txt``DOCS`
FROM
COL COL
COLS
`%`
LIKE
WHERE
`title` `abstr`
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
SELECT title,abstract FROM DOCS WHERE txt LIKE ‘%`;
Structure of an SQL injection
SELECT
`txt``DOCS`
TABLE_REF
COLUMN COLUMN
COLUMN_LIST
CONST
`%`
LIKE
WHERE_COND
`title` `abstr`
COLUMN
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Structure of an SQL injection
SELECT title,abstract FROM DOCS WHERE txt LIKE ‘%‘ UNION SELECT LOGIN,PASS FROM USERS; --%`;
SELECT
`txt``DOCS`
TABLE_REF
COLUMN COLUMN
COLUMN_LIST
CONST
`%`
LIKE
WHERE_COND
`title` `abstr`
COLUMN
UNION
SELECT
`USERS`
TABLE_REF COLUMN_LIST
`LOGIN` `PASS`
COLUMN COLUMN
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Structure of an SQL injection
INSERT
`Robert``STUDENTS`
TABLE_REF
COLUMN COLUMN COLUMN
COLUMN_LIST
CONST CONST
``
CONST
VALUE_LIST
```NAME` `CLASS` `GRADE`
DROP
`STUDENTS`
TABLE_REF
INSERT INTO STUDENTS (NAME,CLASS,GRADE) VALUES (`Robert`, ``, ``); DROP TABLE STUDENTS; -- `CS1`,`4`);
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Parse Tree Validation to prevent SQL-Injections
injected snippets do change overall structure of the query
compare query trees BEFORE and AFTER inserting user-data
implementation „SQLGuard“ extends Java‘s JDBC interface
Using Parse Tree Validation to Prevent SQL Injection Attacks. Gregory T. Buehrer, Bruce W. Weide, Paolo A.G. Sivilotti SEM '05: Proceedings of the 5th international workshop on Software engineering and middleware, ACM, 2005
Related Work
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Parse Tree Validation to prevent SQL-Injections
injected snippets do change overall structure of the query
compare query trees BEFORE and AFTER inserting user-data
implementation „SQLGuard“ extends Java‘s JDBC interface
Change in application code required, for checking before and after user-data insertion
Using Parse Tree Validation to Prevent SQL Injection Attacks. Gregory T. Buehrer, Bruce W. Weide, Paolo A.G. Sivilotti SEM '05: Proceedings of the 5th international workshop on Software engineering and middleware, ACM, 2005
Related Work
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Parse Tree Validation to prevent SQL-Injections
injected snippets do change overall structure of the query
compare query trees BEFORE and AFTER inserting user-data
implementation „SQLGuard“ extends Java‘s JDBC interface
Using Parse Tree Validation to Prevent SQL Injection Attacks. Gregory T. Buehrer, Bruce W. Weide, Paolo A.G. Sivilotti SEM '05: Proceedings of the 5th international workshop on Software engineering and middleware, ACM, 2005
Related Work
If you need to change the code, then switch to prepared statements!!!!
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
How to detect structural changes?
What changes as SQL snippets are inserted into SQL statements?
regular injected
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
What changes as SQL snippets are inserted intoSQL statements?
the number of inner tree nodes
How to detect structural changes?
9 : 15
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
What changes as SQL snippets are inserted intoSQL statements?
the number of inner tree nodes
the number of leave nodes
How to detect structural changes?
6 : 10
9 : 15
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
What changes as SQL snippets are inserted into SQL statements?
the number of inner tree nodes
the number of leave nodes
the height of the tree
How to detect structural changes?
6 : 10
9 : 15
4 : 5
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
How does this scale in case of other WAF evasion techniques?
Effect of Evasions?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
How does this scale in case of other WAF evasion techniques?
Effect of Evasions?
replace or 1 = 1 with
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
How does this scale in case of other WAF evasion techniques?
Effect of Evasions?
or round(pi(),1) + 1 + 1 = version()
replace or 1 = 1 with
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
How does this scale in case of other WAF evasion techniques?
Effect of Evasions?
or round(pi(),1) + 1 + 1 = version()
fn:round
fn:pi
fn:add
1
fn:add
11
fn:version
fn:equals
OR
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Effect of Evasions?
true-(mod(length(trim(leading(concat(lower(conv(version()*(true+pi()), pi()*pi(),pow(pi(),pi()))),lower(conv(pi()*pi()*pi()-pi()-pi(),pi()*pi(), pow(pi(),pi()))),lower(conv(pi()*version(),pi()*pi(),pow(pi(),pi()))), conv(version()*(true+pi()),pi()*pi(),pow(pi(),pi())),lower(conv(pi()*pi()*pi( )-pi()-pi(),pi()*pi(),pow(pi(),pi()))),lower(conv(pi()*version(),pi()*pi(), pow(pi(),pi()))),lower(conv(ceil(pi()*version())+true,pi()*pi(),pow(pi(), pi()))),lower(conv(ceil((pi()+ceil(pi()))*pi()),pi()*pi(),pow(pi(),pi()))), lower(conv(ceil(pi())*ceil(pi()+pi()),pi()*pi(),pow(pi(),pi()))), conv(ceil(pi()*version()),pi()*pi(),pow(pi(),pi())),lower(conv(ceil(pi()*pi() +pi()),pi()*pi(),pow(pi(),pi()))),lower(conv(ceil(version()*version()),pi()*pi (),pow(pi(),pi()))),lower(conv(ceil(pi()*pi()+pi()),pi()*pi(),pow(pi(),pi()))))) from(pass))),length(pass)))
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
How can we use that to detect attacks?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
A simple Demo-Shop
We implemented a simple Java Web Shop
uses MySQL backend
highly vulnerable to SQL injections
allows for simple definition of URL-to-SQL map
logs SQL statements along with request
allows logging ModSecurity anomaly scoring
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
A simple Demo-Shop
Multiple URLs that execute one or more SQL queries, defined in a URL-to-SQL map:
GET /view-product: SELECT * FROM products \ WHERE id = %{id};
POST /search: SELECT * FROM products \ WHERE name LIKE ‘%%{query}%‘ \ OR desc LIKE ‘%%{query}%‘;
GET /cart/view: SELECT * FROM cart \ WHERE id = ‘%{SESSION:ID}‘;
GET /cart/add: INSERT INTO cart VALUES \ ( %{SESSION:ID}, %{id}, 1 );
...
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
A simple Experiment
Generated some SQL logs
Generated a „normal work load“
attacked the shop with sqlmap
Recorded all HTTP traffic and SQL queries
SQLMAP Bernardo Damele, Miroslav Stampar http://www.sqlmap.org
test-client sqlmap Total
6251 147 6398
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Distribution of number of inner nodes vs. total node count for normal statements and SQL injections
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Distribution of number of inner nodes vs. total node count for normal statements and SQL injections
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Distribution of number of inner nodes vs. total node count for normal statements and SQL injections
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Training a classifier
We‘re looking for a simple binary classification
Use some training data to find a function f that will output „normal“ or „attack“ on new, unseen data
RecordedData
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Training a classifier
We‘re looking for a simple binary classification
Use some training data to find a function f that will output „normal“ or „attack“ on new, unseen data
RecordedData
Training
Test
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Training a classifier
We‘re looking for a simple binary classification
Use some training data to find a function f that will output „normal“ or „attack“ on new, unseen data
RecordedData
Training classifier
Test
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Training a classifier
We‘re looking for a simple binary classification
Use some training data to find a function f that will output „normal“ or „attack“ on new, unseen data
RecordedData
Training
classifierTest
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Training a classifier
We‘re looking for a simple binary classification
Use some training data to find a function f that will output „normal“ or „attack“ on new, unseen data
RecordedData
Training
classifierTest ??
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Watching a single URL
We trained a simple classifier on the data to distinguish a normal query and its modifications
Normal Attacks Total
514 65 579
normal sqlmap
pred normal 514 21 96,1 %
pred sqlmap 0 44 100 %
100 % 67,7 %
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Watching a single URL
We trained a simple classifier on the data to distinguish a normal query and its modifications
Normal Attacks Total
514 65 579
normal sqlmap
pred normal 514 21 96,1 %
pred sqlmap 0 44 100 %
100 % 67,7 %
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Watching a single URL
We trained a simple classifier on the data to distinguish a normal query and its modifications
Normal Attacks Total
514 65 579
normal sqlmap
pred normal 514 21 96,1 %
pred sqlmap 0 44 100 %
100 % 67,7 %
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Watching a single URL
We trained a simple classifier on the data to distinguish a normal query and its modifications
Normal Attacks Total
514 65 579
normal sqlmap
pred normal 514 21 96,1 %
pred sqlmap 0 44 100 %
100 % 67,7 %
Data labeled by User-Agent string, but sqlmap sends valid requests at initial probe phase. These are no injections.
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Watching a single URL
On a second data set with correctly labeled data, the classifier perfectly detects all attacks with no false positives
Normal Attacks Total
1245 55 1300
normal attack
pred normal 1245 0 100 %
pred attack 0 55 100 %
100 % 100 %
Results obtained by a 10-fold, stratified cross validation
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Watching multiple URLs
So we‘re able to learn how to tell a normal query and its anomalous modification apart
Most web apps use more than a single query
How does our approach scale with multiple queries?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
A more complex Experiment
In this experiment, we checked detection capabilities with multiple statements and their modified injection versions
normal attack
pred normal 6251 31 99,51 %
pred attack 0 116 100 %
100 % 78,91 %
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
A more complex Experiment
In this experiment, we checked detection capabilities with multiple statements and their modified injection versions
normal attack
pred normal 6251 31 99,51 %
pred attack 0 116 100 %
100 % 78,91 %
The results above are obtained with a Support Vector Machine (SVM) with linear kernel, C=1000.0
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
A more complex Experiment
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
The power of trees...
So far we explored classification using only the height and number of nodes of a tree
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
The power of trees...
So far we explored classification using only the height and number of nodes of a tree
What about using the complete tree?
SELECT
SUMname
SELECT name,SUM(PUNKTE) FROM STUDENTS WHERE name = 'Marcin' AND lvID = '42509'
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
The power of trees...
So far we explored classification using only the height and number of nodes of a tree
What about using the complete tree?
Start --> SELECT SELECT --> ResultCols From Where ResultCols --> ResultCol ResultCol ResultCol --> ColRef ColRef --> 'NAME' ResultCol --> ColRef AggregateNode --> SUM ColRef --> 'PUNKTE' FromList --> TableRef TableRef --> STUDENTS Where --> AndNode AndNode --> BinOp BinOp BinaryOp --> Eq ColRef Const ColRef --> `name` Const --> `Marcin` ...
SELECT
SUMname
SELECT name,SUM(PUNKTE) FROM STUDENTS WHERE name = 'Marcin' AND lvID = '42509'
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
The power of trees...
So far we explored classification using only the height and number of nodes of a tree
What about using the complete tree?
Start --> SELECT SELECT --> ResultCols From Where ResultCols --> ResultCol ResultCol ResultCol --> ColRef ColRef --> 'NAME' ResultCol --> ColRef AggregateNode --> SUM ColRef --> 'PUNKTE' FromList --> TableRef TableRef --> STUDENTS Where --> AndNode AndNode --> BinOp BinOp BinaryOp --> Eq ColRef Const ColRef --> `name` Const --> `Marcin` ...
SELECT
SUMname
SELECT name,SUM(PUNKTE) FROM STUDENTS WHERE name = 'Marcin' AND lvID = '42509'
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
The power of trees...
So far we explored classification using only the height and number of nodes of a tree
What about using the complete tree?
Start --> SELECT SELECT --> ResultCols From Where ResultCols --> ResultCol ResultCol ResultCol --> ColRef ColRef --> 'NAME' ResultCol --> ColRef AggregateNode --> SUM ColRef --> 'PUNKTE' FromList --> TableRef TableRef --> STUDENTS Where --> AndNode AndNode --> BinOp BinOp BinaryOp --> Eq ColRef Const ColRef --> `name` Const --> `Marcin` ...
.01111211111111111110.
SELECT
SUMname
SELECT name,SUM(PUNKTE) FROM STUDENTS WHERE name = 'Marcin' AND lvID = '42509'
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
The power of trees... Experiment
A high-dimensional feature space provides more chances to separate between normal and attack - so let‘s see:
normal attack
pred normal 6251 11 99,82 %
pred attack 0 136 100 %
100 % 92,52 %
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
The power of trees... Experiment
A high-dimensional feature space provides more chances to separate between normal and attack - so let‘s see:
normal attack
pred normal 6251 11 99,82 %
pred attack 0 136 100 %
100 % 92,52 %
The SVM classifier performs much better and is able to predict the attacks pretty good, with a polynomial kernel of degree 3, gamma=100.0, C=1000.0
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
11 attacks missed??
So what went wrong?
By accident labeled as „attack“ (User-Agent).
Just a „probe“ query of sqlmap
SELECT id,name,desc,price FROM products WHERE name LIKE '%secret%' OR desc LIKE '%secret%'
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
11 attacks missed??
So what else went wrong?
This one was labeled as „attack“ in the test data.
The classifier said it is „normal“.
SELECT id,name,desc,price FROM products WHERE name LIKE '%secret) AND 8579=8579 AND (7161=7161%' ORdesc LIKE '%secret) AND 8579=8579 AND (7161=7161%'
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
The power of trees... Experiment
After manual inspection - all missed attacks turned out to be normal queries or unsuccessful SQL injections
normal attack
pred normal 6262 0 100 %
pred attack 0 136 100 %
100 % 100 %
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
The power of trees... Experiment
After manual inspection - all missed attacks turned out to be normal queries or unsuccessful SQL injections
normal attack
pred normal 6262 0 100 %
pred attack 0 136 100 %
100 % 100 %
The SVM classifier perfectly distinguished attacks and normal queries with a polynomial kernel of degree 3, gamma=100.0, C=1000.0
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
So what about a real application?
Good question!
Please upload your database-logs + web-logs to my web-site and I will try :-)
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
So what about a real application?
Good question!
Please upload your database-logs + web-logs to my web-site and I will try :-)
We checked out Typo-3
1000 queries, 15 artificial attacks
about 90% detection rate
too few training data
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Trying to visualize SQL of Typo-3...
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Trying to visualize SQL of Typo-3...
ISOM created from Typo3 1000 SQL queries with 15 artificial SQL injections, a tree-kernel was used as similarity measure
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
Summary
A successful SQL injection needs to alter the query
Syntactical approach for detecting SQL injections
Escapes evasion attacks by inspecting the queries just before they hit the database
Vectorization of trees for detection using machine learning showed good results
Creating an SQL parser is the hardest part :-)
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
References
SQL parser libraries
jsqlparser (Java)(generated with javacc)
Ingres SQL parser library (Java)(based on antlr, conversion of mysql parser)
Machine Learning Tool
RapidMiner
jsqlparserhttp://jsqlparser.sf.net
Ingres SQL Parser CollectionPart of the Ingres Migration Toolsethttp://code.ingres.com/
my fork of jsqlparsergithub.com/cbockermann/jsqlparser
RapidMinerhttp://rapid-i.com/