Fuzzing101 uvm-reporting-and-mitigation-2011-02-10

WWW.CODENOMICON.COM

UVM:Reporting and Mitigation

Ari Takanen, CTO of Codenomicon

Feb 10th, 2011

WWW.CODENOMICON.COM2

Fuzzing 101:

• The webcast series for fuzzing industry

• Vendor neutral presentations on fuzzing technologies and use-cases

• Includes invited speakers from the industry

Codenomicon:

• Fuzzing research since 1996

• 2001, Spinoff from University of Oulu

• 50-100% annual growth in number of customers and revenues in fuzzing industry

About Fuzzing 101 and Codenomicon


About Ari Takanen

The Past: Researcher and Lecturer• 1998-2002• University of Oulu• OUSPG/PROTOS research group• Software Quality related lectures

The Present: Entrepreneur and Evangelist• 2001-today• CTO of Codenomicon• Evangelist: 10 conference talks every year • Author of two books:

• VoIP Security• Fuzzing


Agenda

Introduction to UVM:

UVM Overview and Phases• Attack Surface Analysis• Testing = Fuzzing• Reporting and Mitigation

Vulnerability Reporting• Process and best practises

Mitigating Unknown Threats• Fuzzing results with security devices• Challenges

WWW.CODENOMICON.COM

Unknown Vulnerability ManagementOverview and Phases

The Only Means of Catching Zero-Day Vulnerabilities!


Challenges with Vulnerability Management

Detect Vulnerabilities as they are found• Not as they emerge, they are in the hiding already

Most costs are in patch deployment• Crisis management, each update needs immediate

attention• Ad-hoc deployment is prone to errors• Maintenance downtime can be expensive• New patches emerge several times a week• No time to test the patch


Zero-Day Vulnerability

The biggest threat to IT systems everywhere is a Zero-Day vulnerability• No patches or updates available• No security defenses

Software is released with vulnerabilities

In the past, those problems were found and disclosed by the security community = hackers


Some Helpful Definitions

Vulnerability – a weakness in software, a bug

Threat/Attack – exploit/worm/virus against a specific vulnerability

Protocol Modeling – Technique for explaining interface message sequences and message structures

Fuzzing – process and technique for security testing

Anomaly – abnormal or unexpected input

Failure – crash, busy-loop, memory corruption, or other indication of a bug in software


The Challenge: Unknown Vulnerabilities Are Everywhere


Codenomicon Labs Test Results

http://www.codenomicon.com/labs/results


Unknown Vulnerability Management (UVM)

Another name:

Zero-Day Vulnerability Management

Process of:• Detecting attack vectors• Finding zero-day vulnerabilities• Building defenses• Performing patch verification• Deployment in one big security push


UVM Phases


Phase 1: Attack Surface Analysis

Tools:• Port scanners• Resource scanners• Network analyzers

Codenomicon Network Analyzer identifies what needs to be tested within your network• Record traffic at multiple points in your network• Automatically visualize the network• You can drill up and down from looking at high-level

visualizations to inspecting the corresponding packet data

• Real time analysis• Reveal hidden interfaces and possible exploits.


Phase 2: Test

Fuzzing means crash-testing Discover both known and previously

unknown vulnerabilities with unparalleled efficiency.

Specification-based tools for over 200 protocols• Tools contain all the possible protocol messages and

structures• Genuinely interoperate with the tested system

exposing vulnerabilities even in deeper protocol layers General purpose fuzzers

• Defensics XML Fuzzer can test all XML applications. • The Traffic Capture Fuzzer uses real traffic• Generic File Format Fuzzer tests all file formats.


Phase 3: Report

Codenomicon test suites generate different reports for different audiences

Management reports provide an high-level overview of the test execution

Log files and spreadsheets help you to identify troublesome tests and to minimize false negatives

Individual tests by augmenting the already extensive test case documentation with PCAP traffic recordings

Remediation Packages can be send to third parties for automated reproduction.


Phase 4: Mitigate

Mitigation tools quickly and easily reproduce vulnerabilities, perform regression testing and verify patches

The tools automatically generate reports, which contain risk assessment and CWE values for the found vulnerabilities and direct links to the test suites that triggered the vulnerabilities

Identification of the test cases that triggered the vulnerability is critical

The test case documentation can be used to create tailored IDS rules to block possible zero-day attacks.

WWW.CODENOMICON.COM

Vulnerability Reporting

Experiences with Bug Reporting


Security View: Window of Vulnerability

SW - under vulnerability analysis

SW - after product release

SW - after the vulnerability process

TIME

BUG APPEARSRELEASEBUG FOUND

VULN FOUNDVULN REPORTVULN FIX AVAIL.

PATCH RELEASEADVISORY RELEASEPATCH INSTALL

ZeroExposure

LimitedExposure

PublicExposure


Vulnerability Reporting Process


Finding and Reporting Problems

Motivations in finding bugs in third party code:• Protecting your own system• Security research in some other component

Note that the above is one of the most important things to analyze!


How and To Who Do You Report?

To whom do you report the findings• First ask from your support contact (no details yet)• Vendor security contact (vendor CERT, PSIRT, …)• Mailing lists, support forums, …• Third party such as CERT• Remember confidentiality, encryption (PGP)


A Bug Report

What to write to a bug/incident report?• Note that first purpose is reproduction

Selected observations:• Use bug report templates• Write complete but compact bug reports• Be prepared that the report might leak or become

public• Clearly indicate what is confidential information• Consider carefully if you want to write demonstration

exploits (but be prepared for it)• Never propose a specific fix (in worst case it will just

be blindly adopted)


Share Test Environment Details

Bug reports should clearly indicate the test environment that was used• Exact software versions• Operating system versions• Network architecture (or virtual setup)

If possible, give the vendor the exact setup• This is easiest with virtual setups


Defensics Includes Test Setup Details


Collaboration and Document Sharing

If your test tools produce reports, share those with the vendor

In many cases, it might be best to actually share the fuzzing tool that you used to find the issue

Collaboration platforms and IM will make communication much easier• Codenomicon can provide the platform for you


Best Method for Reproducing with Defensics


But Often You See This…


PCAPs as Test Documentation

PCAPs are quick method for reproduction

Most fuzzing tools can save tests as PCAP, and re-execute those PCAPs• E.g. Defensics Traffic Capture Fuzzer reproduces

and tests around most PCAPs, wherever they come from

The biggest challenge with PCAPs is that they are difficult to maintain

When specification changes, the efficiency of PCAP-based test will quickly degenerate


Risk Assessment from Bug Identification


Different Types of Reports

Fuzzing tools will produce different reports for different audiences


Remediation Package

Defensics Remediation Package collects all required data

Typical difficulties with reproduction:• If only the reported test is reproduced, the next test

case can still crash the system


How to Protect Yourself

It is good to involve third party in the process:• Codenomicon (or another security company)• Internal CERT• External CERT

Remember that legal departments will almost always get involved

If something becomes difficult, let the professionals in the bug reporting process handle it

Remember: Almost always the biggest failure is in personal communications


What to Expect

Timeline:• From reporting to first acknowledgement should be

really really fast (reporters do not have patience)• From acknowledgement to first threat analysis: “we do

not think this is critical”• From start of fixing to ready patch, this is often the

longest wait, up to months or few years

Quality of the patch:• Quick and dirty patches are always suspicious• When vendor is given more time, the quality is often

better, and the patch does not break any valid functionality

What to do while waiting?

WWW.CODENOMICON.COM

Challenges in adopting Fuzzing results

with security devices

Mitigating Unknown Threats


Challenges with IDS

Although all fuzz test cases are easy to convert to IDS rules, those IDS rules are quite easy to circumvent• “Advanced Evasion Techniques” ;-)

Traditionally IDS rules detect individual attack exploits, with the same problem that variants of the same attack can easily pass through


How to Build Good IDS Rules

Be generic in the rules, detect string lengths instead of string contents, etc.

Extensively test the rules with both legal (valid case) traffic and with systematic fuzzing in the vulnerable protocol element

Note that not all fuzz tests need to be blocked by IDS -> Too complex, does not work


Patch Verification Using Fuzzers

Use the same tool and same setup immediately when you receive the first patch candidate

Also relevant to vendor, before even providing the patch candidate to the reporter• Note that it is really annoying to reporters when the

patch is bad quality, and they might just give up if they do not have motivation to continue

Some critical systems might already include the patch you are testing, so bugs in the patch are also critical, and have to be handled in such way


Conclusions

Most challenges in bug reporting are not in technology, but communication• Codenomicon provides collaboration platform to assist

Main purpose of bug reports in reproduction of the found flaws• Codenomicon Remediation Package!!

Easiest reproduction of bugs is always with exactly the same test setup, and exactly the same tool• Codenomicon provides free access to our tools for

reproduction Fuzz test cases cannot all be blocked by IDS

• Codenomicon provides extensive documentation of every test case


Summary of UVM

Security is not about security mechanisms

For full security analysis, you should study:• Threats• Attacks• Vulnerabilities• Architectures• Countermeasures

Unknown Vulnerability Management is about identification and elimination of zero-day vulnerabilities

Security is a process not a product!


Our Book On Fuzzing!

http://www.fuzz-test.com/book/

Takanen, DeMott and Miller: “Fuzzing for Software Security Testing and Quality Assurance”

Aimed at the general public, you do not need to be a security specialist to read this book

Purpose of the book is to teach next-gen testing approaches to:• Software practitioners• Security engineers• Academics


More News from Codenomicon

Facebook:• Become fan of Codenomicon and Fuzzing

Twitter:• CodenomiconLTD

Codenomicon Website:• Newsletter every second month

Customer portal• The Backstage! Contact us for access!

41

WWW.CODENOMICON.COM

PROACTIVE SECURITY AND ROBUSTNESS SOLUTIONS

THANK YOU – QUESTIONS?

“Thrill to the excitement of the chase! Stalk bugs with care, methodology, and reason. Build traps for them.

....Testers!

Break that software (as you must) anddrive it to the ultimate

- but don’t enjoy the programmer’s pain.”

[from Boris Beizer]

Fuzzing101 uvm-reporting-and-mitigation-2011-02-10

Technology

Transcript of Fuzzing101 uvm-reporting-and-mitigation-2011-02-10