Post on 18-Jan-2016
Generating Software Documentation in Use Case Maps
from Filtered Execution Traces
Edna Braun, Daniel Amyot, Timothy LethbridgeUniversity of Ottawa, Canada
damyot@uottawa.ca
SDL 2015, Berlin, October 14
SDL 2015 – p. 2
Motivation
• Maintenance and evolution of large legacy software
• User-friendly, automated software comprehension
• Focus on the elimination of utilities from long execution traces, with documentation and visualization based on Use Case Maps
SDL 2015 – p. 3
Program Comprehension
• Static Analysis– Based on code structure– Recovery of packages, classes, components,
attributes, associations and other relationships… • Dynamic Analysis
– Based on traces and/or other behavioral information– Recovery of invocation sequences, scenarios, state
machines… • Hybrid Approaches
– Combine (parts of) static and dynamic analyses
SDL 2015 – p. 4
Context
• For most software applications, dynamic analysis based on execution traces is difficult– Very, very lengthy traces, even for simple features
• Several approaches for component filtering and for trace compression exist– E.g., filtering out simple getters and setters in Java– But traces are still large– Need to use many approaches in practice
• Different visualizations exist– MSC / UML sequence diagrams, (business)
processes…
SDL 2015 – p. 5
Trace
Exam
ple
(Extr
act
)main 0
Entered: cruise/umple/UmpleConsoleMain.optParse 1
Entered: joptsimple/OptionParser.<init> 2
Entered: joptsimple/BuiltinHelpFormatter.<init> 3
Exited: joptsimple/BuiltinHelpFormatter.<init>
Entered: joptsimple/internal/AbbreviationMap.<init> 3
Exited: joptsimple/internal/AbbreviationMap.<init>
Entered: joptsimple/OptionParserState.moreOptions 3
Entered: joptsimple/OptionParserState$2.<init> 4
Entered: joptsimple/OptionParserState.<init> 5
Exited: joptsimple/OptionParserState.<init>
Exited: joptsimple/OptionParserState$2.<init>
Exited: joptsimple/OptionParserState.moreOptions
Exited: joptsimple/OptionParser.<init>
Entered: joptsimple/OptionParser.acceptsAll 2
Entered: joptsimple/ParserRules.<clinit> 3
Exited: joptsimple/ParserRules.<clinit>
Entered: joptsimple/ParserRules.ensureLegalOptions 3
Entered: joptsimple/ParserRules.ensureLegalOption 4
SDL 2015 – p. 6
Problem and Approach
• Can we detect and filter out utilities in software in order to shorten traces to be visualized and used for program comprehension, while preserving the traces’ meaning?
• Exploration of hybrid approaches– Different algorithms for utility detection, based on
static and dynamic information– Use of execution traces for dynamic analysis and
recovery of high-level behaviour– Visualization with Use Case Maps– Experiments with OO programs in Java
SDL 2015 – p. 7
Process Overview
Application +
Source Code GenerateLong
Trace
FilterShort
Trace
Visualize
UCMAdjust Filter
and Visualize
HelpComprehend
UCM
SDL 2015 – p. 8
Definitions
• A utility is any element of a program designed for the convenience of the designer and implementer and intended to be accessed from multiple places within a certain scope of the program
• Utilityhood is a metric defined as the extent to which a particular method can be considered a utility. • Calculated with dynamic and static variables• When the utilityhood is high, remove from the trace• Non-utility methods help describe features and
business logic
SDL 2015 – p. 9
TracetoUCMTool
SDL 2015 – p. 10
SDL 2015 – p. 11
Static Metrics
• Collected with Metrics 1.3.6– Method Lines of Code (MLOC)– Nested Block Depth (NBD)– McCabe Cyclomatic Complexity (McCabe)
SDL 2015 – p. 12
Dynamic Metrics
• Collected with Eclipse TPTP tracer and a Java profiler– UniqueFanout (array): Unique set of methods that
this method called. The length of the array is the fan-out.
– UniqueFanin (array), Fan-in, and Fan-out– NumberOfTimesCalledBy (array)– NumberOfTimesMethodsCalled (array)– TotalSegmentPresence– Depth– Method BaseTime, CumulativeCpuTime,
CpuUsageAverage– …
SDL 2015 – p. 13
SDL 2015 – p. 14
Utilityhood Metrics
• Combine basic static/dynamic metrics collected from software code and execution traces
• Combinations based on preliminary experiments and on related work– Tested on 18 simple Java open source systems– Kept the most promising combinations for further
experiments• U(r): utilityhood for method r used in a trace• RelativeMethodSize(r): size of a method r in relation
to other methods in the system• N: number of unique methods in the execution trace
SDL 2015 – p. 15
SDL 2015 – p. 16
SDL 2015 – p. 17
Use Case Maps
• Part of the User Requirements Notation (Rec. Z.151)• Combine structure with scenario-oriented behaviour• High level of abstraction, above that of messages
SDL 2015 – p. 18
Trace element
UCM element Symbol
Package Shown as a rectangle with thick border.
Class Shown as a rectangle in another rectangle (Package).
Beginning / End of trace
Start point (circle) / End point (bar) (also used as connectors for linking sub-scenarios to the parent stub)
Method Method name (shown as a X on a path)
Block of x or more instructions in the same class/object
Stub (diamond) with the name of the first instruction that is not a constructor. This stub contains a plug-in (another sub-map) showing the sub-sequence with one responsibility per instruction.The number of instructions that are placed in a block is set in the TraceToUCM tool.
Repeated method
Method name (repetition count between curly brackets) method {2}
Order of trace items
Direction of scenario flow (shown as an arrow head). We use the order of trace items in execution traces to dictate the direction of the scenario flow.
… …IN1 OUT1
… …
… …
… …
TraceUCM Mapping [Hamou-Lhadj et al., 2005]
SDL 2015 – p. 19
Example: UCEd Tool
• Use Case Editor, version 1.6.2 (Somé, 2007)• 700 Java classes, 3284 methods, 33,004 lines of code• Feature explored: “Open a project”• Trace size: 19,543 method invocations
SDL 2015 – p. 20
SDL 2015 – p. 21
Example UCM
• Algorithm 4• 40 top methods preserved• Needs another iteration, with
better parameters...
SDL 2015 – p. 22
• Same initial trace• Algorithm 4• 20 top methods
preserved• Use of UCM stubs
for hierarchical decomposition
SDL 2015 – p. 23
Validation with Experts
Software Classes Methods LOC
UCEd 700 3284 33,004
Umple 430 4457 79,459
jUCMNav 851 7406 93,919
TraceToUCM 35 96 3441
SDL 2015 – p. 24
Results of Utility Selection Algorithms
“Are the methods selected by this algorithm correct?”
Algorithms 2 (Hamou-Lhadj) and 4 (Braun) ahead of the pack
SDL 2015 – p. 25
“Assuming the correct methods had been selected, would you use UCM diagrams to document the behavior of a feature in your design documents?”
Results of Visualization
SDL 2015 – p. 26
Documentation Experiment
• Traced our own TraceToUCM tool during the analysis of the HotDraw software application
• 12 grad students and 4 professional developers• Showed UCMs of traces; some were very (too) short• “Pick the top three UCM diagrams that you fell gives
enough information to describe what the program does”• Newer algorithms scored best, but…
Subjects Alg 1 Alg 2 Alg 3 Alg 4 Alg 5 Alg 6 Alg 7 Alg 8
S1 2 3 1S2 3 2 1S3 3 2 1S4 1 3 1S5 3 2 1S6 3 2 1S7 1 2 3S8 2 1 3S9 3 2 1S10 2 3 1S11 1 1 3 (4)S12 (4) 2 2 1
S13 1 1 3 3
S14 1 3 1 3 3 (6)S15 3 2 1
S16 (4) (4) 2 2 1
Ranking Alg 1 Alg 2 Alg 3 Alg 4 Alg 5 Alg 6 Alg 7 Alg 8Small & simple
First 3 1 1 3 1 0 2 0 9Second 0 0 3 1 1 3 1 5 0Third 0 3 4 0 3 2 1 4 0Count 3 4 8 4 5 5 4 9 9
SDL 2015 – p. 27
Related Work and Threats to Validity
• Tons of them! See paper and Braun’s thesis
SDL 2015 – p. 28
Conclusions
• TraceToUCM: an environment where different algorithms targeting the filtering of utilities (exploiting dynamic, static, and CPU information) can be implemented for comparison and evaluation
• Visualization of traces with UCM (and indirectly MSC)• Comparison of 8 utilityhood functions, 4 based on
existing work and 4 new ones– Usually better results with some new functions– But: still much room for improvement
SDL 2015 – p. 29
Future Work
• Re-run study with other traces, software, parameter configurations and OO languages, with more subjects
• Improve UCM generation with better auto-layout and multiple layers of stubs
• Investigate whether utilities change from one feature to the next
• Extend the study to handle multi-threaded applications