Test title

Analyzing Large-Scale Object-Oriented Software to Find and RemoveRuntime BloatGuoqing Xu

CSE Department Ohio State University

Ph.D. Thesis DefenseAug 3, 2011

1Rapid Heap Growth Between 2002 and 20042

Mitchell et al. LCSD052The Big Pile-up* Framework-intensive

SAP Netweaver App server3* Easy to have performance problems

* Hard to diagnose

* Millions lines of code

3The Big Pile-up Causes Runtime BloatHeaps are getting biggerGrown from 500M to 2-3G or more in the past few yearsBut not necessarily supporting more users or functions

Surprisingly common (all are from real apps):Supporting thousands of users (millions are expected)Saving 500K session state per user (2K is expected)Requiring 2M for a text index per simple documentCreating 100K temporary objects per web hit

Consequences for scalability, power usage, and performance

44Problems and Insights

Problem with compiler optimizations limited scope lack of developer insight Problem with human optimizations lack of good tools Our insight: There is a way to identify larger optimization opportunities with a small amount of developer time Our methodology: Advocate compiler-assisted manual tuning Static and dynamic analysis techniques to find and remove bloat55An Overview of Bloat AnalysisResultsDynamic: [ICSE08, PLDI09, PLDI10-a, PLDI11] Static: [PLDI10-b, Onging]Roadmap: [FoSER10]PlatformsJava Virtual Machine: IBM J9 [PLDI09, PLDI10-a] JikesRVM [PLDI11]Java Virtual Machine Tool Interface [ICSE08]Soot [PLDI10-b, Ongoing]

66Overview of TechniquesDynamic techniques Static techniques observeExecutionEvidence Lots of copies [PLDI09] Large cost/benefit [PLDI10-a] High leak confidence [ICSE 08, PLDI11]analyzeBloat/leak reportBytecodeanalyzeBloated containers [PLDI10-b] Invariant objects [Ongoing]searchBloat warning77An Overview of My WorkSoftware bloat analysis dissertation work[ICSE08, PLDI09, FoSER10, PLDI10-a, PLDI10-b, PLDI11, Ongoing]Scalable points-to and data-flow analysis[CC08, ISSTA08, ECOOP09, ISSTA11]Control- and data-flow analysis for AspectJ[ICSE07, AOSD08, ASE09, TSE11]Language-level checkpointing and replay[FSE07]88Research AreasDynamic analysisStatic analysisRuntime SystemsCompilersPLDI09, PLDI10-a, PLDI11ISSTA08, CC08, ECOOP09, PLDI10-b, ISSTA11, OngoingLanguages/Software Engineering9ICSE07, FSE07, ICSE08, AOSD08, ASE09, FoSER10, TSE119OutlineIntroductionIntroduction to bloatOverview of my thesisI: Finding low-utility data structures [PLDI10-a]Dynamic analysis to help performance tuningII: Finding loop-invariant data structures [Ongoing]Static analysis that targets a specific bloat pattern found by IFuture work and conclusions 1010Finding Low-Utility Data StructuresIdentify high-cost-low-benefit data structures that are likely to be performance bottlenecksTo achieve this, we need toCompute cost and benefit measurements for objectsPresent to users a list of data structures ranked by their cost/benefit rates

1111A Systematic Way of Detecting BloatBloat can manifest itself through many symptomstemporary objects [Dufour FSE08, Shankar OOPSLA08]excessive data copies [Xu PLDI09]highly-stale objects [Bond ASPLOS06, Novak PLDI09]problematic container behaviors [Xu ICSE08, Shaham PLDI09, Xu PLDI10]

12Is there a common way to characterize bloat? At the heart of all inefficiencies lie computations, with great expense, produce data values that have little impact on the forward progress12What is Common About Bloat e.g. java/ioboolean isPackage(String s){ List packs = directoryList(s); return packs != null;}List directoryList(String pkgName){ //find and return all files in directory pkgName // return null if nothing is found} --- from Eclipse

13High cost-benefit rate8% running time reduction achieved by specializing the method13Cost and BenefitAbsolute cost of a heap value: total number of instructions executed to produce ita = b or a = b + c, each has unit costBenefit of heap value v:

v (output) : benefit is a very large number

v (not used) : benefit is 0 v v : benefit is M (amount of work)

14

M14Cost Computation b = 10; a = b + 3;

c = b * 5;

d = a / c;

An example of a backward dynamic flow problem Requires dynamic slicing: record all mem. accesses15The cost of d is 415Abstract Dynamic SlicingTrace generated by dynamic slicing is unboundedThe amount of memory needed depends on the dynamic behavior of a programTrace contains many more details than a client would needIs it possible to capture only part of the execution relevant to the clientFor many problems, equivalence classes exist a = b.m1 a = b.m3 a = b.m67 a = b.m23 a = b.m1235

a = b.m45 a = b.m217 a = b.m29 a = b.m35 a = b.m9

16E1E216Cost ComputationUse calling contexts to define equivalence classesE102, E47, E0 are object-sensitivity-based calling contexts (i.e., chains of receiver objects)17Absolute costAbstract costa = b + c 20a = b + c E0b=e/f 34c=h/g2135b=e/f E102c=h/g E47#nodes(n.size)concrete dependence graphabstract dependence graph17Relative Abstract CostCumulative cost measures the effort made from the beginning of the execution to produce a valueIt is certain that the later a value is produced, the higher cost it has

18 Cumulative costProgram inputf g h jRelative costO3O2O1F, g,h,j are fields18Relative Abstract Benefit19Relative benefit: the effort made to transform a value read from a heap loc l to the value written into another heap loc l

19 Relative benefitProgram inputf g h j k lO3O2O1O4O519Key Problems and IdeasProblem 1: Dynamic slicing is too expensive and unnecessaryInsight 1: Abstract slicingProblem 2: How to abstract runtime instances for computing costs for object-oriented data structuresInsight 2: (Object-sensitivity-based) calling contextsProblem 3: Cumulative cost is not correlated with performance problemsInsight 3: Relative costSolution: Compute relative abstract cost

2020Performance ImprovementsImplemented in IBM J9 Java Virtual MachineCase studies on real-world large applications

2121ImpactThe first dynamic analysis targeting general bloatIdentified patterns leading to new techniquesContainer inefficiencies [PLDI10-b, context-free-language reachability formulation]Loop-invariant data structures [Ongoing, type and effect system]Problematic implementations of certain design patternsAnonymous classes2222OutlineIntroductionIntroduction to bloatOverview of my thesisI: Finding low-utility data structures [PLDI10-a]Dynamic analysis to help performance tuningII: Finding loop-invariant data structures [Ongoing]Creating objects with the same content many times in loops really hurts performance Pulling them out of loops (i.e., hoisting) would potentially save a lot of computation2323Examplefor(int i = 0; i < N; i++){ SimpleDateFormat sdf = new SimpleDateFormat(); try{ Date d = sdf.parse(date[i]); } catch(...) {...}} 24--- From applications studied in IBM T. J. Watson Research CenterHoistable logical data structureComputing hoistability measurements

OsdfO1O2OsdfOsdfOsdf24 Hoistable Data Structures25Understand how a data structure is built uppoints-to relationshipsUnderstand where the data comes fromdependence relationshipsUnderstand in which iterations objects are createdCompute iteration count abstraction (ICA) for each allocation site o : 0, 1, 0: created outside the loop (i.e., exists before the loop starts)1: inside the loop and does not escape the current iter : inside the loop and escapes to later iterations

25ICA AnnotationAnnotate points-to and dependence relationships with ICAs26A a = new A(); //O1,0String s = null;for (int i = 0; i < 100, i++){ Triple t = new Triple(); // O2,1 Integer p = new Integer(i); // O3,1 if(i < 10) s = new String(s); // O4, t.f = a; // connect O2 and O1 t.g = p; // connect O2 and O3 t.h = s; // connect O2 and O4}O2O1f, (1, 0)O3g, (1, 1)O4h, (1, )O3.valuei(1, 0)(0, 0)Annotated points-to relationshipsAnnotated dependence relationships26Detecting Hoistable Data StructuresDisjointCheck: s does not contain objects with ICAs

27O2O1f (1, 0)O3g (1, 1)O4h (1, )Annotated points-to relationships27Detecting Hoistable Data Structures (Cond)Loop-invariant fieldsCheck: No dependence chain starting from a heap location in s is annotated with Check: Nodes whose ICA are 0 are not involved in any cycle28O3.valuei(1, 0)(0, 0)Annotated dependence relationships28Hoistability MeasurementsInstead of doing transformation, we compute hoistability measurementsStructure-based hoistability (SH) How many objects in the data structure are disjointData-based hoistability (DH)How many fields in the data structure are loop-invariant

2929EvaluationThe technique was implemented using Soot 2.3 and evaluated on a set of 19 large Java programsA total 981 loop data structures considered155 are disjoint data structures(15.8%)3030Case StudiesWe studied five large applications by inspecting the top 10 data structures ranked based on hoistability measurementsLarge performance improvements achieved by manual hoistingps: 82.1%xalan: 10% (confirmed by the DaCapo team)bloat: 11.1% soot-c: 2.5%sablecc-j: 6.7%No problems that we found have been reported before3131OutlineIntroductionI: Finding low-utility data structures [PLDI10-a]II: Finding and hoisting loop-invariant data structures[Ongoing]Future Work and Conclusions 3232Future Work--Bloat AnalysisLook a bit deeper into bloat causes, we will seeObject-orientation encourages excessLeverage techniques from other fields such as systems and architecture33

33ConclusionsStatic and dynamic analysis Principles and applicationsMy dissertation Software bloat analysisDynamic analyses that can make more sense of heaps and executionsStatic analyses that can remove and prevent bloatHopesTool support: Tuning is no longer a daunting task Education: Performance is important

3434 Thank You Q & A

3535