GÖDEL'S GOURD

Fuzzing for logic and state issues

Introductions

Michael Eddington CTO and Principal Consultant @ Déjà vu

Security 12+ years in security consulting Senior developer/architect in prior life Author of Peach, an open source fuzzer

Device, Kernel, User, Web, Network

DARPA Cyber Fast track

Thanks Mudge!

Defining the Problem

Fuzzers are good at findings implementation issues …that crash the target …that are generically detectable (sqli, xss)

Not good at finding design, logic, and state issues …that do not crash the target …that are not generically detectable

Examples

Buffer Overflows Memory

Corruption Resource Usage SQL Injection

Missing authentication

State corruption Incorrect logic

Easy Hard

Authentication Examples

Out of 100 admin pages, 5 are missing authentication

Microsoft SSPI skip a step auth bypass

OpenBSD IPSEC incorrect if/then logic

Authentication – Detect

Web – Missing Auth

Trigger Request page w/o

logging in

MS SSPI/OBSD IPSEC

Trigger Skip a step

Status Code (200/403)

What pages require auth

Result (Pass) Did we complete

all steps

Logic Example

Windows 95 SMB Flaw Logic error in password checking code Length of loop determined by client

input Modified SMB client, ~32 attemps

always wins

We never throw an exception or crash Typical generic fuzzer will never find this

Logic – Win95 SMB

bool CheckPw(int userdata_len, char* userdata,int sysdata_len, char* sysdata )

{for(int i=0; i<userdata_len; i++)

if(userdata[i] != sysdata[i])return false;

return true;}

Logic – Detect

Win 95 SMB Trigger

Try all chars Remove NULL

Result Does password

match

State Example

Device (phone/tablet/laptop) with theft system

Agent “heartbeats” to server Server can trigger “stolen” mode in

laptop Laptop will trigger if unable to

“heartbeat” Timer/counter runs down

State – Detect

System Server Trigger

Cause exception Flow locked Unable to

heartbeat

Can we perform state flow?

Check result of each step

How to detect?

Goal – Modify existing fuzzer to detect these issues We already produce triggers How do we add detection?

How to detect?

What do we need to detect these issues?

Provide system constraints If not authenticated result is 402 If steps 1, 2, and 3 not performed step 4 is

error Result is never 500

Verify we are still working Perform state flow w/o mutations

Proposed Solution

Gödel's Gourd

Re-use Peach fuzzing engine Mutation engine Fault detection/reporting

Constraint language Control iterations (non mutation

iterations) Mutate state model (skip, order, etc.)

Control Iterations

Goal: Verify target is working correctly

No mutations Constraints pass State model is followed

Matches recorded control iteration

How it works

R – Record iteration 1 – Fuzzing iteration C – Control iteration 2 – Fuzzing Iteration C – Control iteration 3 – Fuzzing iteration …

Remember all states/actions from record iteration

Verify on control iterations

Control iterations every N fuzzing iterations

Outcome

If control does not match record – throw fault

Identify conditions that stop normal operation

Constraints

Verify logic via simple constraint expressions

Apply constraints to state model State Action

Does not modify fuzzer state

Language Options

Existing Traditional Languages JavaScript Python Ruby etc.

Pro Well known Available via .NET

scripting interface

Cons Allows

modification of fuzzer state.

Other Options

Domain Specific Language (DSL)

Use existing

Create our own

Pros Meet all

requirements

Cons Must implement Not well known

DSL Selection

Object Constraint Language (OCL) Specification language, no side effects Developed as part of new UML standards Familiar syntax

Relatively easy to implement

Object Constraint Language (OCL) Expression types

Invariant (inv) Always true

Pre (pre) Evaluated before [ something ]

Post (post) Evaluated after [ something ] Can access state from Pre. (@pre)

OCL Examples

“Car owner must be at least 18 years old”context Carinv: self.owner.age >= 18

“If passwords match result is true”context Loginpost: result = true implies pass1 = pass2

OCL Context

Groups sets of constraints Constraints for a context are run

together Association based on context

Normal Fuzzing Iteration

Enter State Model State 1

Action 1.1 Send Data

Action 1.2 Receive Data

State N …

Fuzzing Iteration With Constraints Enter State Model State 1

Action 1.1 Send Data

Action 1.2 Receive Data

State N …

Inv(pre) Pre

EVENT

Inv(post) Post

Applying (Authentication)

Web Authentication

# Verify authentication occurredpost:

(reply = 200 && url.indexOf(‘/admin’) > -1)

implies auth.reply = 200

Applying (Authentication)

Windows SSPI

# Verify all steps completedpost: reply = true implies (

auth.step1.reply = true && auth.step2.reply = true && auth.step3.reply = true)

Applying (Logic)

Windows 95 Bug

post: reply = true implies userpw = ‘password’

Applying (State)

Antitheft System

Perform control iteration

Implementation

Technologies Used

Microsoft .NET Framework – C# Peach Fuzzer 3

Cross platform using Mono OS X Linux

Implementation Diagram

OCL Implementation

Irony .NET Language Toolkit Many differences from traditional Grammar is code Easy AST hookups

LINQ Expressions From IronPython work Last mile is already done

LINQ Expressions

Exposes language constructs for use in AST classes.

Does all the heavy lifting.

return Expression.Condition((Expression)ifNode.Evaluate(thread),

(Expression)thenNode.Evaluate(thread),

(Expression)elseNode.Evaluate(thread));

All the things that do the stuff

Gödel Usage

Peach Pit vs. Gödel Gourd

Data Model

State Model

Agents Test

Data Model OCL Definitions State Model

OCL Associations Agents Test

Gödel: Define Constraints

<Ocl><![CDATA[

context StatusCodeOkpost: context.test.publishers[self.publisher].Result = 'OK'

]]></Ocl>

Gödel: Associate Constraints

<Action type="call" method="Logout"><Ocl context="StatusCodeOk" />

</Action>

Constraints will now run with this Action.

Gödel: Control Iterations

<Test name=“Default” controlIteration=“1”>

<Agent … /><StateModel … /><Publisher … /><Logger … />

</Test>

Define how often control iterations occur.

Time and Cost

Usage Feasibility

Adding Gödel

Process: Existing Peach PIT Add OCL Constraints Test and Verify Definition

Not recreating full application logic Just our “view of the world”

Time per Protocol

Based on current experience of limited protocol set

Decent in 1 – 2 days Complete in 1 week or less

Performance

What performance impact does Gödel incur? Constraint evaluation Control iterations

No performance optimizations…yet

Performance of Constraints

1 5 10 15 20 250

10

20

30

40

50

60

Constraint Count

Tim

e f

or

10

,00

0 (

Secon

ds)

Performance Control Iterations Depends on how often, worst case half

speed

Never longer than mutation iterations

Performance Conclusions

Performance impact dependent on speed of fuzzing

Ability to scale fuzzing lowers impact

For fast fuzzers, acceptable impact For slower fuzzers, adjust control

iterations to occur less often

Conclusions

Pentesting/Quick fuzzing Reasonable for “basics” (verify state’s

work, critical logic flows)

General definition building Reasonable to implement decent coverage

1-2 days “good enough”

Wrapping it up…

Lessons Learned

Constraints applied only to control iterations

Writing good constraints that apply to all mutation cases is challenging

A few constraints can go along ways Performance overhead needs to be

lowered when many constraints used. Optimize access to most used

variables/objects

Looking towards next rev…

Can we “learn” basic constraints? Performance optimizations Shorted “name” of common objects

context.test.publishers[self.publisher].Result

self.Result

Thanks for all the fish!

Michael Eddington mike@dejavusecurity.com

http://dejavusecurity.com http://peachfuzzer.com