1

Leveraging User Interactions forIn-Depth Testing of Web Applications

Sean McAllister, Engin Kirda, and Christopher KruegelRAID ’08

Seoyeon KangNovember 4, 2008

2

Introduction

Static Dynamic

Complexity ↑Bugs ↑Vulnerability ↑

3

Introduction

• Web application vulnerabilities – Result from lack of input validation– Example: Cross site scripting(XSS), SQL injection bugs

4

Review (1)

• Cross site scripting(XSS)– Malicious scripting code which transfers sensitive in-

formation to the attacker is injected into the output of an application• Stored XSS: Attacker persistently stores the malicious code

in a resource such as a database. • Reflected XSS: Script is reflected to the victim.

5

Review (2)

• SQL injection – Malicious input can reach a DB query and modify the

intended semantics of this query

6

Background (1)

• Testing tool– White box testing : the source code of the applica-

tion is analyzed to indicate errors or vulnerabilities.– Black box testing : Special input test cases are gener-

ated and sent to the application. Then, the results are analyzed.

7

Background (2)

• Testing toolWhite box Black box

Pros. • Large code coverage• In theory, can analyze all possible execution paths

• No false positives• No need knowledge about implementation-specific de-tails

Cons.

• False positives• Support limited program-ming language

• Limited code coverage

8

Background (3)

• Vulnerability scanner– An automated security program that searches for

software vulnerabilities within web applications

9

Background (4)

• Steps of vulnerability scanner– Web spider finds valid injection points– Provide a variety of specially-crafted, malformed in-

put values that trigger certain classes of vulnerabili-ties

– Analyze response that web application returns

10

Limitations

• Fail case : To leave a comment in blogging site

11

Limitations

• Code coverage– Scanner does not

have any knowledge about use cases

– Built-in web spider is insufficient

12

Goal

• Improve the effectiveness of web vulnerability scanners

• Increase test coverage

13

Approach

• Leverage input from real users as a starting point for testing– Generate test cases that can be replayed

• Interact with the web application in a more meaningful fashion– For detection of stored XSS vulnerability

14

Increasing Test Coverage

1) Increasing testing depth2) Increasing testing breadth3) Stateful Fuzzing

15

1) Increasing testing depth

• Leverage actual user input– Collecting input

• Record requests directly at the server

– Replaying input• Execute one step• Fuzzer replace the valid inputs to malformed values in data-

base• Previously recorded input values are used for the next step.

• Guided fuzzing : process of using previously collected traces

16

2) Increasing testing breadth

• Use current step as a starting point for fuzzing– For each step, only a single entry point is analyzed– Fuzzer attempts to find additional entry points

• Extended, guided fuzzing

17

3) Stateful Fuzzing

• Fail scenario : On the page of shopping cart– Delete all items in cart– The following steps of the use case fails

Show cart’s inventory Checkout

Delete all items

18

3) Stateful Fuzzing

• Stateful fuzzing – Take a snapshot of the state of the application

after each step that is replayed – Restored to the snapshot

• Application feedback– Use the mapping of URL to functions

to reuse user input for undetected pages

19

Implementation details (1)

• Require to be instrumented– To capture and restore objects– To extract the mappings between URLs and functions

• Need MVC-based web development framework– Used Django based on python

20

Implementation details (2)

• Capturing web requests– Log all incoming requests and mappings b/w URL and func-

tions through the URL dispatcher class

• Replaying use cases– Analyze a page and find the form elements

• Capturing object manipulations– Attach event listeners that wait for signals that are raised

when an object is created, updated, or deleted

• Fuzzer component– Use existing fuzzer component

21

Evaluation

• 3 publicly available, real-world web applications based on Django– Blogging application– Forum software– Web shop

• simple web spider and 3 existing web vulnerability scanners– Burp Spider– W3af spider– Acunetix Web Vulnerability Scanner

22

Test methodology

• Way to assess the effectiveness or coverage– Number of vulnerabilities– Number of locations that a scanner visits Number of vulnerable injection points Number of created different database objects

(for detecting stored XSS)

• Only tested for XSS vulnerabilities

23

Experimental results

1) Blogging application2) Forum software3) Web shop

24

1) Blogging application

25

1) Blogging application

Scanner effectiveness for blog application

26

2) Forum application

Scanner effectiveness for forum application

27

3) Online shopping application

Scanner effectiveness for online shopping application

28

3) Online shopping application

Object creation statistics (Acunetix vs. this )

29

Experimental results

• Proposed fuzzing techniques consistently find more of bugs than other scanners

• Carry out meaningful interactions with the web appli-cations, visiting many locations and creating a large variety of database objects

• Stateful fuzzing becomes useful when tested app is more complex and sensitive to the fuzzing steps

30

Related work

• Black box scanner– SecuBat - coverage limitation

• General vulnerability detection tool– Nikto, Nessus - use known velnerability DB

• Automated test case generator– Random test case generation - depth limitation

31

Summary

• The core of system is a black-box vulnerability scanner

• Proposed approaches to allow the scanner to reach deeper into application– Guided fuzzing– Extended, guided fuzzing – Stateful fuzzing

• This approach can identify more vulnerabilities than several open-source and commercial web vulnerability scanners

32

Thank you