Fuzzing

FuzzingBrute Force Vulnerability Discovery

Khalegh [email protected]

SSP, Sorena Secure Processing

A brief introduction on

mailto:[email protected]

About me

• Khalegh Salehi• Software Security & Vulnerability Assessment • http://khalegh.net• FoxFuzzing Project

– All-In-One Full Network Protocols & File Format Fuzzing.


http://khalegh.net/

http://khalegh.net/

http://foxfuzzing.com/

Software Security Analyzing• Static analysis:

– Approach for verifying software (including finding defects) without executing software• Source code vulnerability scanning tools, code inspections, etc.

• Dynamic analysis:– Approach for verifying software (including finding defects) by executing

software on specific inputs & checking results (“oracle”)• Functional testing, fuzz testing, etc.

• Hybrid analysis: – Combine above approaches

• Operational: – Tools in operational setting

• Minimize risks, report information back, etc.• Themselves may be static, dynamic, hybrid; often dynamic


Software Security Analyzing

• Static analysis: – Approach for verifying software (including finding defects) without executing software

• Source code vulnerability scanning tools, code inspections, etc.• Dynamic analysis:

– Approach for verifying software (including finding defects) by executing software on specific inputs & checking results (“oracle”)• Functional testing, fuzz testing, etc.





Why ?


https://www.youtube.com/watch?v=kYUrqdUyEpI


I see. Let's talk on business...

Software Security Analyzing• Static analysis:

– Approach for verifying software (including finding defects) without executing software• Source code vulnerability scanning tools, code inspections, etc.

• Dynamic analysis:– Approach for verifying software (including finding defects) by executing

software on specific inputs & checking results (“oracle”)• Functional testing, fuzz testing, etc.





Fuzzing in Wikipedia

“Fuzz testing or fuzzing is a software testing technique, often automated or semi-automated, that involves providing invalid, unexpected, or random data to the inputs of computer program. The program is then monitored for exceptions such as crashes, or failing built-in code assertions or for finding potential memory leaks. Fuzzing is commonly used to test for security problems in software or computer systems. It is a form of random testing which has been used for testing hardware or software”



Fuzz testing history

• Fuzz testing concept from Barton Miller’s 1988 class project University of Wisconsin– Project created “fuzzer” to test reliability of

command-line Unix programs– Repeatedly generated random data for them until

crash/hang– Later expanded for GUIs, network protocols, etc.

• Approach quickly found a number of defects• Many tools & approach variations created since


Fuzzing in brief

• A form of vulnerability analysis and testing• Many slightly anomalous test cases are input

into the target application• Application is monitored for any sign of error


Fuzz testing process

©softScheck

http://www.softscheck.com/




Fuzzing Phase

Identify inputs

Generate fuzzed data

Execute fuzzed data

Monitor for exceptions

Determine exploitability


Case Study

14

FileFuzz• Application vs. file type

– One file type multiple targets

• Vendor history– Past vulnerabilities

• High risk targets– Default file handlers

• Windows Explorer• Windows Registry

– Commonly traded file types• Media files• Office documents• Configuration files

Identify target

Identify inputs


Execute fuzzed data




15

• Proprietary vs. open formats– Vendor documents– Wotsit.org– Google

• Binary files– e.g. images, video, audio, office

documents, etc.– Headers vs. data

• Text files– e.g. *.ini, *.inf, *.xml– Name/value pairs

Identify target

Identify inputs


Execute fuzzed data




FileFuzz

16

• Binary files– Breadth (All or Range)

• Identify potential weaknesses FF FF FF FF 00 00 DB FE 0B 00 C5 00 00 01 E8 03 ; ÿÿÿÿ..Ûþ..Å...è.

D7 FF FF FF FF 00 DB FE 0B 00 C5 00 00 01 E8 03 ; ×ÿÿÿÿ.Ûþ..Å...è.

D7 CD FF FF FF FF DB FE 0B 00 C5 00 00 01 E8 03 ; ×ÍÿÿÿÿÛþ..Å...è.

– Depth• Determine level of control/influence

D7 CD FD 9A 00 00 DB FE 0B 00 C5 00 00 01 E8 03 ; ×Íýš..Ûþ..Å...è.

D7 CD FE 9A 00 00 DB FE 0B 00 C5 00 00 01 E8 03 ; ×Íþš..Ûþ..Å...è.

D7 CD FF 9A 00 00 DB FE 0B 00 C5 00 00 01 E8 03 ; ×Íÿš..Ûþ..Å...è.

• Text Files– name = value

file_size = 10file_size = AAAAAfile_size = AAAAAAAAAA

Identify target

Identify inputs


Execute fuzzed data




FileFuzz

17

• Command line arguments– Windows explorer• Tools…Folder Options…File

Types

Identify target

Identify inputs


Execute fuzzed data




FileFuzz

18

• Visual– Error messages– Blue screen

• Event logs– System logs– Application logs

• Debuggers• Return codes• Debugging API

Identify target

Identify inputs


Execute fuzzed data




FileFuzz

19

• Execute– Automated and repeated

• Monitor– Library - libdasm– Capture

• Memory location• Registry values• Exception type

• Kill– Set timeout

Identify target

Identify inputs


Execute fuzzed data



[*] "crash.exe" "C:\Program Files\WordPerfect Office 12\Programs\UA120.exe" 2000 /qt c:\fuzz\ast\8.ast[*] Access Violation[*] Exception caught at 00403f06 mov eax,[eax+edi*4][*] EAX:0014b1b8 EBX:00000005 ECX:00435c00 EDX:0012fbac[*] ESI:00435c00 EDI:cccccccc ESP:0012fab8 EBP:0012fae8


FileFuzz

20

• Skills– Disassembly ,Debugging

• Vulnerability types– Stack, Heap overflow, Integer handling,

etc.• Overflows• Signedness

– DoS• Out of bounds reads• Infinite loops• NULL pointer dereferences

– Logic errors• Windows WMF vulnerability (MS06-001)

– Format strings, Race conditions

Identify target

Identify inputs


Execute fuzzed data




FileFuzz

21SSP, Sorena Secure Processing

FileFuzz

FileFuzz is a graphical, Windows based file format fuzzing tool. FileFuzz was designed to automate the creation of abnormal file formats and the execution of applications handling these files. FileFuzz also has built in debugging capabilities to detect exceptions resulting from the fuzzed file formats.


call your guys…


Type of Fuzzers• File Fuzzers As the name implies, fuzzers that target file formats only. They do

not have the ability to speak any network protocol. • Network Fuzzers And these are fuzzers that target only network protocols.

There are allot of these as the discovery of network based vulnerabilities has always attracted allot of attention.

• General Fuzzers Following with our captain obvious theme, these fuzzers that can target a wide variety of targets, typically both file and network, and also others via custom I/O interfaces. For example: COM, shared libraries, RPC, etc.

• Custom or One-off Fuzzers These are custom written fuzzers that target a specific format or network protocol. Typically these hand written, many times by testers. Custom fuzzers vary widely on how good their data mutation/generation is. For the purposes of this document we will not examine any custom or one-off fuzzers.

• API Fuzzers, Hardware Fuzzers and dozens of Fuzzers, There is not limitations for subject… (Chapter 5, Page 161-166)


Type of Fuzzers…

There is no limitation of, Intuitively I have to say…

Where there is a Input, There’s Fuzz…There is an undeniable fact,

Before start your cool fuzzing, please formally let me know about your target.


Example

• Standard HTTP GET request– GET /index.html HTTP/1.1

• Anomalous requests– AAAAAA...AAAA /index.html HTTP/1.1– GET ///////index.html HTTP/1.1– GET %n%n%n%n%n%n.html HTTP/1.1– GET /AAAAAAAAAAAAA.html HTTP/1.1– GET /index.html HTTTTTTTTTTTTTP/1.1– GET /index.html HTTP/1.1.1.1.1.1.1.1– etc...


Example ofVulnerable Source Code

#include <stdio.h> int main( int argc, char *argv[] ) { char buffer[1024]; strcpy(buffer,argv[1]); printf("The string is a %s \n\n",buffer); return 0; }


Example ofSimple Fuzzing scheme

import subprocess,time;for i in range(1,10000): print i; subprocess.call(["./ example ","A"*i]); time.sleep(1); # figure out debugger, crash log, etc.

Go head and run the application via uninvited arguments such as and not limited to,

./example `python -c “print ‘A’*10000”`


The situation under controls...


Definition of fuzzing

“Fuzzing is a technique for intelligently and automatically generating and passing into a target system valid and invalid message sequences to see if the system breaks, and if it does, what it is that makes it break”

CODENOMICON


The Solution That Found Heartbleed

fuzzing(Defensics) was the primarysolution being used when theHeartbleed flaw was identified.A security research was runninga routine test of the Fuzzing(Defensics) feature, SafeGuard, identifying the flaw that had gone unidentified for over two years and impacted over 500,000 websites.

CODENOMICON


Fuzzing Approach

Mutation Based - “Dumb Fuzzing” Generation Based - “Smart Fuzzing” Evolutionary


Mutation Based - “Dumb Fuzzing”


Mutation Based - “Dumb Fuzzing”• Little or no knowledge of the structure of the inputs is

assumed• Anomalies are added to existing valid inputs• Anomalies may be completely random or follow some

heuristics• Requires little to no set up time• Dependent on the inputs being modified• May fail for protocols with checksums, those which depend

on challenge response, etc.Examples:• Taof, GPF, ProxyFuzz, etc.


Generation Based - “Smart Fuzzing”


Generation Based - “Smart Fuzzing”

• Test cases are generated from some description of the format: RFC, documentation, etc.

• Anomalies are added to each possible spot in the inputs

• Knowledge of protocol should give better results than random fuzzing

• Can take significant time to set up• Examples– SPIKE, Sulley, Mu-4000, Codenomicon, Bestorm


Evolutionary


Evolutionary

• Attempts to generate inputs based on the response of the program

• Autodafe– Prioritizes test cases based on which inputs have

reached dangerous API functions• EFS– Generates test cases based on code coverage

metrics (more later)• This technique is still in the alpha stage


Issues & Problems

Mutation based fuzzers can generate an infinite number of test cases... When has the fuzzer run long enough?

Generation based fuzzers generate a finite number of test cases. What happens when they’re all run and no bugs are found?

How do you monitor the target application such that you know when something “bad” has happened?


Issues with FuzzingWhat happens when you find too many bugs? Or

every anomalous test case triggers the same (boring) bug?

How do you figure out which test case caused the fault?

Given a crash, how do you find the actual vulnerability

After fuzzing, how do you know what changes to make to improve your fuzzer?

When do you give up on fuzzing an application?


Products & Frameworks


Products & FrameworksDozens of Open-Source Fuzzing Tools & Frameworks has been collected in FoxFuzzing, there is list of products with bit information of, are available by https://github.com/khaleghsalehi/FoxFuzzing/list.pdf

Not(A2Z)

https://github.com/khaleghsalehi/FoxFuzzing/list.txt




?Thank you &


References1. SWE 681 / ISA 681,Secure Software Design & Programming, Lecture 9, Analysis Approaches & Tools,

Dr. David A. Wheeler, 2014-08-172. Real World Fuzzing, Charlie Miller, Independent Security Evaluators, ctober 19, 2007,

[email protected]. Robustness Testing, Discover unknown vulnerabilities with

Testing & QA, Ari Takanen, Codenomicon Ltd.4. Michael Eddington, Leviathan Security Group, Inc. 20095. A Study of Commercially Available Fuzzers: Identification of Undisclosed Vulnerabilities with the Aid

of Commercial Fuzzing Tools. Prof. Dr. Hartmut Pohl and Daniel Baier, B.Sc. Department of Computer Sciences, Bonn-Rhein-Sieg University of Applied Sciences

6. “Fuzzing for Software Security Testing and Quality Assurance”, Ari Takanen, Jared DeMott, Charlie Miller Fuzzing for Software Security Testing and Quality Assurance (Artech House Information Security and Privacy), 2008

7. Fuzzing: Brute Force Vulnerability Discovery Paperback – July 9, 2007 by Michael Sutton, Adam Greene, Pedram Amini

8. Michael Sutton, Director, iDefense Labs, [email protected], FuzzingBrute Force Vulnerability Discovery

9. [Slide No. 11.] A Study of Commercially Available Fuzzers: Identification of Undisclosed Vulnerabilities with the Aid of Commercial Fuzzing Tools. By: Prof. Dr. Hartmut Pohl and Daniel Baier, B.Sc. Department of Computer Sciences, Bonn-Rhein-Sieg University of Applied Sciences.


http://www.amazon.com/Michael-Sutton/e/B001H6OKPM/ref=dp_byline_cont_book_1

http://www.amazon.com/s/ref=dp_byline_sr_book_2?ie=UTF8&field-author=Adam+Greene&search-alias=books&text=Adam+Greene&sort=relevancerank

http://www.amazon.com/s/ref=dp_byline_sr_book_3?ie=UTF8&field-author=Pedram+Amini&search-alias=books&text=Pedram+Amini&sort=relevancerank





Awesome Books

Fuzzing: Brute Force Vulnerability Discovery Paperback – July 9, 2007by Michael Sutton (Author), Adam Greene (Author), Pedram Amini (Author)

http://www.amazon.com/Michael-Sutton/e/B001H6OKPM/ref=dp_byline_cont_book_1

http://www.amazon.com/s/ref=dp_byline_sr_book_2?ie=UTF8&field-author=Adam+Greene&search-alias=books&text=Adam+Greene&sort=relevancerank





Awesome Books

Fuzzing for Software Security Testing and Quality Assurance (Artech House Information Security and Privacy) Hardcover – June 30, 2008by Ari Takanen (Author), Jared DeMott (Author), Charlie Miller (Author)

http://www.amazon.com/Ari-Takanen/e/B001ITXNQA/ref=dp_byline_cont_book_1

http://www.amazon.com/Ari-Takanen/e/B001ITXNQA/ref=dp_byline_cont_book_1

http://www.amazon.com/s/ref=dp_byline_sr_book_2?ie=UTF8&field-author=Jared+DeMott&search-alias=books&text=Jared+DeMott&sort=relevancerank

http://www.amazon.com/s/ref=dp_byline_sr_book_2?ie=UTF8&field-author=Jared+DeMott&search-alias=books&text=Jared+DeMott&sort=relevancerank

http://www.amazon.com/Charlie-Miller/e/B0085NZ1PS/ref=dp_byline_cont_book_3


Awesome Books

Open Source Fuzzing Tools Paperback – December 28, 2007by Noam Rathaus (Author), Gadi Evron (Author)

http://www.amazon.com/s/ref=dp_byline_sr_book_1?ie=UTF8&field-author=Noam+Rathaus&search-alias=books&text=Noam+Rathaus&sort=relevancerank

http://www.amazon.com/s/ref=dp_byline_sr_book_1?ie=UTF8&field-author=Noam+Rathaus&search-alias=books&text=Noam+Rathaus&sort=relevancerank

http://www.amazon.com/s/ref=dp_byline_sr_book_2?ie=UTF8&field-author=Gadi+Evron&search-alias=books&text=Gadi+Evron&sort=relevancerank




Awesome Books

Violent Python: A Cookbook for Hackers, Forensic Analysts, Penetration Testers and Security Engineers Paperback – August 11, 2012

& many so many books…

Fuzzing

Software

Transcript of Fuzzing