Fuzzing sucks!

Click here to load reader

  • date post

    20-Aug-2015
  • Category

    Technology

  • view

    711
  • download

    3

Embed Size (px)

Transcript of Fuzzing sucks!

  1. 1. BackgroundArchitecture Usage and Demos Future Development Fuzzing Sucks!Introducing Sulley Fuzzing Framework Pedram Amini 1Aaron Portnoy2 1 pamini@tippingpoint.com 2 aportnoy@tippingpoint.com Black Hat US 2007 Amini, PortnoyFuzzing Sucks!
  2. 2. Background IntroductionArchitecture Past and Present Usage and Demos Pain Points and Solutions Future DevelopmentAbout UsWork at TippingPoints Digital Vaccine LabsResponsible for vuln-dev, patch analysis, pen-testingKeep tabs on us at http://dvlabs.tippingpoint.comLaunched OpenRCE.org over two years agoHow many here are members?Some interesting updates on the horizon after BlackHatCreators of PaiMei RE frameworkHow many here have heard of it?Lot of exciting developments coming up after BlackHatCo-authored Fuzzing: Brute Force Vulnerability Discovery Amini, PortnoyFuzzing Sucks!
  3. 3. BackgroundIntroductionArchitecturePast and Present Usage and DemosPain Points and Solutions Future DevelopmentTalk Outline Background Why does fuzzing suck? How can we make it better? Sulleys Architecture Component Breakdown Advanced Features Usage and Walkthrough Hewlett-Packard Data Protector Audit Trend Micro Server Protect Audit Future Development Whats still on the drawing board Amini, Portnoy Fuzzing Sucks!
  4. 4. Background IntroductionArchitecture Past and Present Usage and Demos Pain Points and Solutions Future DevelopmentIs Fuzzing a Dead Horse? Negative Entire BlackHat track, 3 dedicated books, more commercial vendors and still highly eective. Amini, PortnoyFuzzing Sucks!
  5. 5. BackgroundIntroduction ArchitecturePast and PresentUsage and DemosPain Points and SolutionsFuture DevelopmentOld Schoolantiparser David McKinney, Python, x-platform, API drivenDFUZ Diego Bauche, custom language, UnixSPIKE Dave Aitel, C, Unix, block basedThe list goes on ... Angel Fuzzer Framework Fuzzled Fuzzy Packet The Art of Fuzzing SPIKEle ...Amini, PortnoyFuzzing Sucks!
  6. 6. BackgroundIntroduction ArchitecturePast and PresentUsage and DemosPain Points and SolutionsFuture DevelopmentDFUZ FTP ExampleNotesThe custom language is easy to understand but very limiting.port=21/tcppeer write: @ftp:user("user")peer readpeer write: @ftp:pass("pass")peer readpeer write: "CWD /", %random:data(1024,alphanum), 0x0apeer readpeer write: @ftp:quit()peer readrepeat=1024wait=1# No OptionsAmini, PortnoyFuzzing Sucks!
  7. 7. Background IntroductionArchitecture Past and Present Usage and Demos Pain Points and Solutions Future DevelopmentSPIKE FTP ExampleNotesSPIKE data representation syntax is very simple, perhaps why its themost commonly used fuzzer?s_string("HOST ");s_string_variable("10.20.30.40");s_string("rn");s_string_variable("USER");s_string(" ");s_string_variable("bob");s_string("rn");s_string("PASS ");s_string_variable("bob");s_string("rn");s_string("SITE ");s_string_variable("SEDV");s_string("rn");s_string("CWD ");s_string_variable(".");s_string("rn");Amini, Portnoy Fuzzing Sucks!
  8. 8. Background IntroductionArchitecture Past and Present Usage and Demos Pain Points and Solutions Future DevelopmentNew SchoolPeachMichael Eddington, Python, x-platform, highly modularizedCodenomiconCommercial vendor, Java, x-platform, pre-recorded test casesGPFJared Demott, mixed, x-platform, varying fuzz modesAutodafeMartin Vuagnoux, C, Unix, next-gen SPIKEFirst fuzzer to bundle debugger functionalityEvolutionary FuzzersSideWinder, Sherri Sparks et al.EFS, Jared DemottProtocol Informatics FrameworkMarshall Beddoe, Python, x-platform, automated protocol eldidentication tool Amini, PortnoyFuzzing Sucks!
  9. 9. Background IntroductionArchitecture Past and Present Usage and Demos Pain Points and Solutions Future DevelopmentPeach FTP ExampleNotesThere is a non trivial learning curve to writing Peach fuzzers.from Peachimport *from Peach.Transformers import *from Peach.Generators import *from Peach.Protocolsimport *from Peach.Publishers import *loginGroup = group.Group()loginBlock = block.Block()loginBlock.setGenerators((static.Static("USER usernamernPASS "),dictionary.Dictionary(loginGroup, "dict.txt"),static.Static("rnQUITrn")))loginProt = null.NullStdout(ftp.BasicFtp(127.0.0.1, 21), loginBlock)script.Script(loginProt, loginGroup, 0.25).go() Amini, PortnoyFuzzing Sucks!
  10. 10. Background IntroductionArchitecture Past and Present Usage and Demos Pain Points and Solutions Future DevelopmentGPF FTP ExampleNotesData representation format is very dierent from other examples.Source:S Size:20 Data:220 (vsFTPd 1.1.3)Source:C Size:12 Data:USER jaredSource:S Size:34 Data:331 Please specify the password.Source:C Size:12 Data:PASS jaredSource:S Size:33 Data:230 Login successful. Have fun.Source:C Size:6 Data:QUITSource:S Size:14 Data:221 Goodbye....The command line can be a bit unwieldy:GPF ftp.gpf client localhost 21 ? TCP 8973987234 100000 0 + 6 6 100 100 5000 43 finsih 0 3 auto none -G b Amini, PortnoyFuzzing Sucks!
  11. 11. Background IntroductionArchitecture Past and Present Usage and Demos Pain Points and Solutions Future DevelopmentSo Why Does Fuzzing Hurt So Bad?The existing tools contribute solid ideas but are limited in usageBasically all of them are focused solely on data generationLets jump through some fuzzer requirements to get a feel forwhats missingEssentially Chapter 5 from the fuzzing bookAt each juncture well briey cover Sulleys solutionWell drill down into the specics when we cover architecture Amini, PortnoyFuzzing Sucks!
  12. 12. BackgroundIntroduction ArchitecturePast and PresentUsage and DemosPain Points and SolutionsFuture DevelopmentEasy to Use and Powerfully Flexible PainPowerful frameworks have a huge learning curveSimple frameworks quickly reach limitations RemedySulley utilizes block based data representationSulley fuzzers start simple and dont have messy syntaxOptional elements and keyword argumentsFuzzers are written in pure Python and can benet from thelanguages features and ease of useDevelopment eorts can be easily sharedCan handle challenge-response and prev-packet-length situationsAmini, PortnoyFuzzing Sucks!
  13. 13. Background IntroductionArchitecture Past and Present Usage and Demos Pain Points and Solutions Future DevelopmentReproducibility and Documentation PainIndividual test cases must be reproducibleProgress and interim results should be recorded RemedySulley can replay individual test casesSulley keeps a bi-directional PCAP of every transactionA built in web interface provides interactive feedback Amini, PortnoyFuzzing Sucks!
  14. 14. Background IntroductionArchitecture Past and Present Usage and Demos Pain Points and Solutions Future DevelopmentReusability PainNon-generic fuzzers can never be used againWidely used protocol components are re-developed all the time RemedySulley supports the creation and reuse of complex types andhelper functionsThe more Sulley is used, the smarter it gets Amini, PortnoyFuzzing Sucks!
  15. 15. Background IntroductionArchitecture Past and Present Usage and Demos Pain Points and Solutions Future DevelopmentProcess State and Process Depth PainNone of the existing fuzzers consider state pathsFuzzing A-B-D vs. A-C-DMany fuzzers can only scratch a protocol surfaceFuzzing A only RemedyIn Sulley you build fuzzers in manageable chunks called requestsThese requests are tied together in a graphThe graph is automatically walked and each state path and depthis individually fuzzed Amini, PortnoyFuzzing Sucks!
  16. 16. BackgroundIntroduction ArchitecturePast and PresentUsage and DemosPain Points and SolutionsFuture DevelopmentTracking, Code Coverage and Metrics PainHow much of the target code was exercised?What code was executed to handle a specic test case? RemedySulley supports an extensible agent modelUtilizes PaiMei/PyDbg for breakpoint-based and MSR-based codecoverage trackingAmini, PortnoyFuzzing Sucks!
  17. 17. Background IntroductionArchitecture Past and Present Usage and Demos Pain Points and Solutions Future DevelopmentFault Detection and Recovery PainMost fuzzers rely solely on a lack of response to determine whensomething bad happensahem ahem CodenomiconOnce a fault is discovered, most fuzzers simply stop!Mu Security and BreakingPoint take the interesting approach ofpower cycling RemedySulley bundles a debugger monitor agentSulley can restore target health and continue testing by:Restarting the target serviceRestoring a VMware snapshot Amini, PortnoyFuzzing Sucks!
  18. 18. Background IntroductionArchitecture Past and Present Usage and Demos Pain Points and Solutions Future DevelopmentResource Constraints PainNon-technical constraints such as time and man power often getin the way RemedySulley bundles utilities such as a PDML parser to save timeSulley is designed to allow multiple people to work together easilyThe monitoring and self recording features of the framework savea great deal of time Amini, PortnoyFuzzing Sucks!
  19. 19. BackgroundOverviewArchitecture Data Representation Usage and Demos Fuzzing Session and Agents Future DevelopmentUtilitiesSulley Architecture Diagram User DevelopedUtilities Data Generationcrashbin_explorerLegos Blocks ida_fuzz_library_extenderRequest Library pcap_cleaner PrimitivesUtils R1pdml_parser R2R3 sequence_honerDriverR1 Session ManagementR3R2 FilesystempGraphSession crashbins R4R5 pcaps... Agents TargetsVMControl NetmonProcmon ...Sulley Architecture Amini, PortnoyFuzzing Sucks!
  20. 20. BackgroundOverview Architecture Data RepresentationUsage and Demos Fuzzing Session and AgentsFuture DevelopmentUtilitiesFour Major ComponentsData Generation You build requests out of primitives and legos Legos are complex types that extend the frameworkSession Management / Driver Requests are chained together in a graph to form a session The session class exposes a standalone web interface for monitoring and control The driver ties targets, agents and requests togetherAgents Interface with the target