10YearsAfter:AutomaticSoftwareGenerationandImprovementThrough

SearchBasedTechniques

Dr.AndreaArcuriWesterdalsOsloACT

UniversityofLuxembourgarcand@westerdals.no

WhileinmyfirstyearofPhDin2006-2007

•WorkingonSBSTforunittesting• toolsandtheory

•Researchgrouponevolutionarycomputation,notsoftwareengineering(SE)•Meetingwithsupervisor/post-docs,discussionaboutco-evolution• Q:“AnyprobleminSEthatcanbeaddressedwithco-evolution???”

Literally,asolution insearchofaproblem…

Hmmmm,co-evolvingtests withwhat???Hmmmm,programs?

•UseGeneticProgramming(GP)toevolveprograms• Eg,formachine-learningclassifiers

•UseSBSTtoevolvetestcases•Butco-evolvingthemtoachievewhat?•Whatisthefitnessfunctionhere???

Whataboutautomaticallygeneratecodefromformalspecifications?

• YouuseSBSTtogeneratesometests• Formalspecificationusedasoracle forthetests• UseGPtoevolveprogramstopassalltests• Thenevolvenewtests,andkeeponasco-evolution…

Programs

Programs

Programs

Test Cases

Test Cases

…

Diditwork???

• Alreadystruggledontoyexampleslikeabubble-sort…• …andanyway,whotheheckwritesformalspecifications???• Butcouldstillpublishacoupleofpapersthough…• ASE’07andINS

• Timefor(yet)anewPhDtopic???

Fromfailureto“simpler”problems…

• Fullcodegenerationdidn’twork• ButwhatifItryfromasimplercasewhereprogramisnearlycomplete?• …actuallynotasosillyquestionasitsounds...

BugFixing

• Aprogramwithbugsisa“nearly”completeprogram• UseitasseedingforfirstgenerationofGP• Itworkedfineonsometoy examples…• Butstillissueswithformalspecifications…withoutitcannotuseco-evolution (asusedforautomatedoracle)

• Whocaresaboutthehammer/co-evolution?• …throwitawayifitdoesn’twork!!!• “repairing”aprogramonafixedsetoftestcasesisarealSEproblem• ...donemanuallybymillionsofengineerseveryday...• Alotofworkdoneinthelast10yearsbydifferentresearchers

Cangetsomethingusefuloutofco-evolution?

• DiscussionwithDavidR.White…2007-2008• BothinSEBASEproject• MePhDstudentinBirmingham,heinYork

• …afterfewbeers...whatif:• originalprogramasoracleforfunctionalcorrectness• secondobjective:optimizeforperformance• wouldco-evolutionworkhere?• canwegeneticallyimproveanexistingprogramtomakeitfaster?

MOO

Quitedifferentfromwhatcompilersdo

Diditwork?

• Small functions• Foundquiteafewnon-trivialimprovementsthatcompilerscannotdo• …so,somehowyes

InRetrospective…

• Haven’tworkedonsuchtopicssincefinishingPhDin2009• gotmorefocusedonsoftwaretesting

• Curiosity-drivenresearchonunchartedterritoriesishighrisk• Igotluckytogetpublishableresults• MaybenotreallybestforaPhDmaintopic…

• Inindustry,plentyof“problems”thatneeda“solution”• NeedtofindrightbalanceinSEresearch

Conclusion

• BugFixingandGeneticImprovementwere“lucky”accidents• Atthattime2007-2009resultsjustonsmallfunctions• Butshownitwasfeasible• Lookingforwardtonexttalkstoseethecurrentstatus;)

References

• Coevolvingprogramsandunittestsfromtheirspecification, ArcuriandYao,ASE’07• Anovelco-evolutionaryapproachtoautomaticsoftwarebugfixing,ArcuriandYao,CEC’08• Multi-objectiveimprovementofsoftwareusingco-evolutionandsmartseeding,Arcuri,White,ClarkandYao,SEAL’08• Automaticsoftwaregenerationandimprovementthroughsearchbasedtechniques,Arcuri,PhD2009• Evolutionaryrepairoffaultysoftware,Arcuri,ASoC’11• Evolutionaryimprovementofprograms,White,ArcuriandClark,TEVC’11