Yuanfang Cai & Rick Kazman
May 2017 ICSE Technical Briefing
Detecting and Quantifying Architectural Debt:
Theory and Practice
1© Yuanfang Cai, Rick Kazman, 2017
The Gap between Theory and Practice
Cunningham’s
Debt Metaphor [Cunningham 1992]
Conway’s Law [Conway 1968 ]
Parnas’s Theory [Parnas 1972]
Baldwin and Clark’s
Design Rule Theory
[Baldwin and Clark 1999 ]
2 © Kazman, Cai 2017
The Gap between Theory and Practice
Manager Developer
We need to
refactor !
What is the
ROI?
We have
technical debt!
How much is
the debt?
3 © Kazman, Cai 2017
Our objective
4
Towards a Quantifiable Software Architecture Health Monitoring
System, based on principled theoretic foundations
© Kazman, Cai 2017
The Concept of Technical Debt
Software complexity accumulates slowly as part of normal
development activities.
Because it accumulates slowly, most developers are unaware that it
is occurring: they are focused on their immediate tasks.
However, quick-and-dirty changes, over time, lead to problematic
structures in a code base.
The impact of design decisions on software maintenance and
evolution have not been well studied or understood.
5 © Kazman, Cai 2017
Technical Debt vs. Real Debt
Debt in Life Technical Debt
6
Where are the debts?
How much extra maintenance cost?
How fast does penalty accumulate over time (interest rate)?
Cost: 1,500$
Interest: 14.24%
Cost: 20,000$
Interest: 4.46%
Cost: 1000,000$
Interest: 4.75%
© Kazman, Cai 2017
Design Rule Theory [Baldwin and Clark 1999]
7
Design Rules create Modules
Modules creates values in the form of options
Investing in modularization (refactoring) = Buying
options
© Kazman, Cai 2017
Design Rule Theory
8
Design Rules and True Modules
Modular Operator: Splitting Design rules split the rest of the system into true modules
True modules only depend on design rules, but not on each other
A and B-C are proto-modules A and B-C become true modules
Splittingby adding new design rules
The key idea of decoupling is to insert design rules that
(1) create true modules, so that
(2) true modules only depend on design rules, but not on each other.
© Kazman, Cai 2017
Design Rule Theory
9
Modular Operator: Substitution
A true module can be substituted with a new version
True modules can be improved, changed, or even replaced, without
influencing other parts of the system
Module-wise improvement can be accomplished independently and in
parallel, as along as design rules remain unchanged
© Kazman, Cai 2017
Design Rule Theory
10
Modules create options
A designer has the right, but not the obligation to improve a module (similar
to real options)
The value of the system increases as the value of each module increases
independently
The value of a system with n modules:
V = S0 + NOV1 + NOV2 +…+ NOVn
© Kazman, Cai 2017
Design Rule Theory
11
The value of the system:
The Net Option Value of each module
Where:
• NOVi: Net option value of module i
• i: Technical potential of module i
• n: The complexity of module i
• Q(ki): The probability of getting the highest value of module i after k experiments
• Ci(ni)ki : The cost of k experiment of module i of complexity n
• Zi: The cost of revising the modules depending on module i
V = S0 + NOV1 + NOV2 +…+ NOVn
NOVi = maxki {ini1/2 Q(ki) – Ci(ni)ki –Zi}
© Kazman, Cai 2017
12
The essence of the theory
The more independent modules are there, the higher the value of the
system
The fewer dependencies a module has, the higher the value
The option value of a module increases and then decreases as the
complexity of the module increases
The more active (technical potential) a module is, the higher its option
value.
V = S0 + NOV1 + NOV2 +…+ NOVn ;
NOVi = maxki {ini1/2 Q(ki) – Ci(ni)ki –Zi}
Design Rule Theory
© Kazman, Cai 2017
From Theory to Principles
13
The visualization and quantification of critical design principles
Information hiding
Hierarchical structure
SOLID principles
Single responsibility
Open for extension, closed for modification
Liskov substitution principle
Interface segregation principle
Dependency Inversion principle
The features of well-modularized systems explained in DRT and DSM
© Kazman, Cai 2017
Information Hiding Explained
14
Key concepts from Parnas:
Information hiding: hiding changeable decisions behind stable
interfaces
Modules: defined as independent task assignments
Explained in Design Rule Theory and DSM
Stable interfaces-> design rules
Modules-> the source of option value
Baldwin and Clark explained information hiding in design rule theory
Visualization using DSMs
© Kazman, Cai 2017
X Y Z A D G J B E H K C F I L M
X - Computer .
Y - Corpus X . X
Z - User X .
A - In Type .
D - Circ Type .
G - Alph Type .
J - Out Type .
B -In Data X X . X X
E - Circ Data X X X . X
H - Alph Data X X X X .
K - Out Data X X .
C - In Alg X X X X .
F - Circ Alg X X X X X .
I - Alph Alg X X X X X X X .
L - Out Alg X X X X X X .
M - Master X X X X X .
Sequential Design in DSMs [Sullivan et al. 2001]
15
Sequential Design
Each module is modeled as 3
elements:
Function signature
Data structure
Detailed implementation
Design Rules
Function signature and data
structure
Modules:
Detailed implementation
© Kazman, Cai 2017
X Y Z A D G J B E H K C F I L M
X - Computer .
Y - Corpus X . X
Z - User X .
A - In Type .
D - Circ Type .
G - Alph Type .
J - Out Type .
B -In Data X X . X X
E - Circ Data X X X . X
H - Alph Data X X X X .
K - Out Data X X .
C - In Alg X X X X .
F - Circ Alg X X X X X .
I - Alph Alg X X X X X X X .
L - Out Alg X X X X X X .
M - Master X X X X X .
Sequential Design in DSMs [Sullivan et al. 2001]
16
Environment Parameters
Modeling the dimensions that
are likely to change
Extracted from documentation
or added by architects
Changes start from
requirements
Same environment parameters
applied to both designs
© Kazman, Cai 2017
X Y Z N A D G J O P B C E F H I K L M
X - Computer .
Y - Corpus X . X
Z - User X .
N - Line Type .
A - In Type .
D - Circ Type .
G - Alph Type .
J - Out Type .
O - Line Data X X X . X
P - Line Alg X X X X .
B - Input Data X X X . X
C - Input Alg X X X X X .
E - Circ Data X X X X . X
F - Circ Alg X X X X X .
H - Alph Data X X X X . X
I - Alph Alg X X X X X X .
K - Out Data X X X . X
L - Out Alg X X X X X .
M - Master X X X X X X .
Information Hiding Design in DSMs [Sullivan et al. 2001]
17
Information hiding:
changeable environmental
parameters do not affect
design rules
© Kazman, Cai 2017
X Y Z A D G J B E H K C F I L M
X - Computer .
Y - Corpus X . X
Z - User X .
A - In Type .
D - Circ Type .
G - Alph Type .
J - Out Type .
B -In Data X X . X X
E - Circ Data X X X . X
H - Alph Data X X X X .
K - Out Data X X .
C - In Alg X X X X .
F - Circ Alg X X X X X .
I - Alph Alg X X X X X X X .
L - Out Alg X X X X X X .
M - Master X X X X X .
X Y Z N A D G J O P B C E F H I K L M
X - Computer .
Y - Corpus X . X
Z - User X .
N - Line Type .
A - In Type .
D - Circ Type .
G - Alph Type .
J - Out Type .
O - Line Data X X X . X
P - Line Alg X X X X .
B - Input Data X X X . X
C - Input Alg X X X X X .
E - Circ Data X X X X . X
F - Circ Alg X X X X X .
H - Alph Data X X X X . X
I - Alph Alg X X X X X X .
K - Out Data X X X . X
L - Out Alg X X X X X .
M - Master X X X X X X .
(A) Sequential Design
NOV = 0.26
(B) Information Hiding Design
NOV = 1.56
Parnas’s Two Designs in DSMs [Sullivan et al. 2001]
18
Quantifying two designs using Net Option Theory
The Information Hiding Design has much higher option values
© Kazman, Cai 2017
Pieces to move from Theory to Practice
Architectural Model [Xiao et al. 2014]
New Metric [Ran et al. 2016]
Localization [Ran el al. 2015, Xiao et al. 2016]
Option to buy
Debts to Pay
Debt Quantification [Kazman et al. 2015, Xiao et al. 2016]
Price of an option
Principal, penalty, interest rate
19 © Kazman, Cai 2017
Pieces to move from Theory to Practice
Architectural Model [Xiao et al. 2014]
New Metric [Ran et al. 2016]
Localization [Ran el al. 2015, Xiao et al. 2016]
Option to buy
Debts to Pay
Debt Quantification [Kazman et al. 2015, Xiao et al. 2016]
Price of an option
Principal, penalty, interest rate
20 © Kazman, Cai 2017
A New Architecture Model [Xiao et al. 2014]
DR-based Architectural Model
Design Rule Space (DRSpace): Modeling
software as multiple overlapping spaces
21 © Kazman, Cai 2017
Multiple Design Spaces in Software
Intuitively, a nontrivial software system must contain multiple
design spaces:
each feature implemented
each pattern applied
each concern addressed
Each file can participate in multiple design spaces
A design space may cross multiple packages
We propose that software architecture can, and should be
modeled as multiple, overlapping design spaces
22 © Kazman, Cai 2017
Multiple Design Spaces in Software
23
Each design space should have its own structure
Design Rules
Modules
Layers
Each design space may evolve independently
Each design space should be measured and managed separately
These design spaces are implemented in source code and get
mixed
Each design space can be visualized using Design Structure Matrix
(DSM)
© Kazman, Cai 2017
Design Rule Space (DRSpace) [Xiao et al. 2014]
A subset of files
Any subset of files may form a design space
Architectural connections
Structural couplings: call, inherit, aggregate, etc..
Evolutionary couplings
One or multiple types
Implicit or explicit
24
Design Rule Space A DRSpace is composed of a meaningful subset of a system’s files along with the architectural connections among these files.
© Kazman, Cai 2017
Design Rule Space (DRSpace)
A DRSpace defines the following concepts:
1. A DRSpace contains one or more leading files:
The files that all other files in the space directly or indirectly depend on
Leading files may or may not be design rule files
2. A DRSpace contains a selected set of dependency types, either
structural or evolutionary coupling
3. A DRSpace can be clustered into Design Rule Hierarchy (DRH) [Cai and
Sullivan 2006, Wong et al. 2009], using one or more architectural
relations
Primary relation: selected, DRH-clustered dependency type
Secondary relation: other dependency types
25 © Kazman, Cai 2017
The small calculator design spaces:
26
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1 mij.bnf.Node (1)
2 mij.io.Pipe (2)
3 mij.ast.Node (3)
4 mij.Filter (4)
5 mij.io.InputPipe ext (5)
6 mij.io.OutputPipe ext (6)
7 mij.io.WriterOutputPipe Impl (7)
8 mij.io.MemoryOutputPipe Impl (8)
9 mij.io.ReaderInputPipe Impl (9)
10 mij.io.MemoryInputPipe Impl (10)
11 mij.ast.TreeVisitor (11)
12 mij.Interpreter Impl Impl (12)
13 mij.parse.Parser Impl (13)
14 mij.lex.Lexer Impl (14)
15 mij.parse.Convert Impl (15)
16 mij.ast.Variable ext (16)
17 mij.ast.Number ext (17)
18 mij.ast.OperExpr ext (18)
19 mij.ast.FuncExpr ext (19)
20 mij.ast.UnaryOperExpr ext (20)
21 mij.bnf.ValueExpr ext (21)
22 mij.bnf.AddExpr ext (22)
23 mij.bnf.ParamExpr ext (23)
24 mij.bnf.UnaryExpr ext (24)
25 mij.bnf.LexExpr ext (25)
26 mij.bnf.ExponExpr ext (26)
27 mij.bnf.MultExpr ext (27)
File set
File set
File 5: mij.io.InputPipe extends File 2 mij.io.Pipe
DRSpace 1: Polymorphism DRSpace• formed by “extend” and “implement”
File 7: mij.io.WriterOutputPipe implements File 6 mij.io.OutputPipe
© Kazman, Cai 2017
Design Rule Space (DRSpace) Modeling
The small calculator example:
27
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1 mij.bnf.Node (1)
2 mij.io.Pipe (2)
3 mij.ast.Node (3)
4 mij.Filter (4)
5 mij.io.InputPipe ext (5)
6 mij.io.OutputPipe ext (6)
7 mij.io.WriterOutputPipe Impl (7)
8 mij.io.MemoryOutputPipe Impl (8)
9 mij.io.ReaderInputPipe Impl (9)
10 mij.io.MemoryInputPipe Impl (10)
11 mij.ast.TreeVisitor (11)
12 mij.Interpreter Impl Impl (12)
13 mij.parse.Parser Impl (13)
14 mij.lex.Lexer Impl (14)
15 mij.parse.Convert Impl (15)
16 mij.ast.Variable ext (16)
17 mij.ast.Number ext (17)
18 mij.ast.OperExpr ext (18)
19 mij.ast.FuncExpr ext (19)
20 mij.ast.UnaryOperExpr ext (20)
21 mij.bnf.ValueExpr ext (21)
22 mij.bnf.AddExpr ext (22)
23 mij.bnf.ParamExpr ext (23)
24 mij.bnf.UnaryExpr ext (24)
25 mij.bnf.LexExpr ext (25)
26 mij.bnf.ExponExpr ext (26)
27 mij.bnf.MultExpr ext (27)
4 leading files and Design Rules
OutputPipe Module
InputPipe Module
Visitor Module
Filter Module
Ast Node Module
Bnf Node Module
DRSpace 1: Polymorphism DRSpace• formed by “extend” and “implement”
© Kazman, Cai 2017
Design Rule Space (DRSpace) Modeling
1 2 3 4 5 6 7 8 9 10
1 mij.ast.Node (1)
2 mij.ast.TreeVisitor (2)
3 mij.ast.Variable Ext, dp dp (3)
4 mij.ast.UnaryOperExpr Ext, dp dp (4)
5 mij.ast.OperExpr Ext, dp dp (5)
6 mij.ast.Number Ext, dp dp (6)
7 mij.ast.FuncExpr Ext, dp dp (7)
8 mij.Interpreter dp Imp dp (8)
9 mij.parse.Convert dp dp dp dp dp dp dp (9)
10 mij.Console dp dp dp (10)
Design Rule Space (DRSpace) Modeling The small calculator example:
28
Ext: Extend Impl: Implement dp: General dependencies such as call, use, and create
DRSpace 2: Visitor Pattern DRSpace• formed by multiple dependency types
Visitor interface: mij.ast.TreeVisitor
Element interface: mij.ast.Node
Concrete elements of the pattern: m(rc3-7)
Concrete visitor role: Calculator
© Kazman, Cai 2017
1 2 3 4 5 6 7 8 9 10
1 mij.ast.Node (1)
2 mij.ast.TreeVisitor (2)
3 mij.ast.Variable Ext, dp dp (3)
4 mij.ast.UnaryOperExpr Ext, dp dp (4)
5 mij.ast.OperExpr Ext, dp dp (5)
6 mij.ast.Number Ext, dp dp (6)
7 mij.ast.FuncExpr Ext, dp dp (7)
8 mij.Interpreter dp Imp dp (8)
9 mij.parse.Convert dp dp dp dp dp dp dp (9)
10 mij.Console dp dp dp (10)
The small calculator example:
29
Ext: Extend Impl: Implement dp: General dependencies such as call, use, and create
DRSpace 2: Visitor Pattern DRSpace• Forming a DRH with all dependency as the primary relation
Visitor interface: mij.ast.TreeVisitor
Element interface: mij.ast.Node
Concrete elements of the pattern: m(rc3-7)
Concrete visitor role: Calculator
© Kazman, Cai 2017
Design Rule Space (DRSpace) Modeling
The small calculator example:
30
Ext: Extend Impl: Implement dp: General dependencies such as call, use, and create
DRSpace 2: Visitor Pattern DRSpace• Evolutionary coupling is added as the secondary relation
1 2 3 4 5 6 7 8 9 10
1 mij.ast.Node (1)
2 mij.ast.TreeVisitor (2)
3 mij.ast.Variable Ext, dp, 5 dp (3)
4 mij.ast.UnaryOperExpr Ext, dp, 10 dp (4)
5 mij.ast.OperExpr Ext, dp, 25 dp 5 (5)
6 mij.ast.Number Ext, dp, 8 dp 20 (6)
7 mij.ast.FuncExpr Ext, dp, 6 dp 15 (7)
8 mij.Interpreter dp, 10 Imp dp (8)
9 mij.parse.Convert dp dp dp dp dp dp dp (9)
10 mij.Console dp dp dp (10)
Visitor interface: mij.ast.TreeVisitor
Element interface: mij.ast.Node
Concrete elements of the pattern: m(rc3-7)
Concrete visitor role: Calculator
© Kazman, Cai 2017
Design Rule Space (DRSpace) Modeling
The small calculator example: There can be many more DRSpaces in this small example, formed
by each dependency type, each pattern, and each feature
31
ag: aggregation
DRSpace 3: Aggregation DRSpace
Lex Module
Memory Buffer Module
Module
communicate
through pipe
Five leading classes in first layer
Three meaningful modules in second layer
© Kazman, Cai 2017
Design Rule Space (DRSpace) Modeling
32
DRSpace 4: Dependency DRSpace
dp: dependency
Parsing Function
Console
Function
© Kazman, Cai 2017
The small calculator example: There can be many more DRSpaces in this small example, formed
by each dependency type, each pattern, and each feature
Design Rule Space (DRSpace) Modeling
DRSpaces overlap with each other
Primary: AggregationPrimary: Realize
Inherit
Aggregation
Depend
Leading Classes
33 © Kazman, Cai 2017
Design Rule Space (DRSpace) Modeling
The small calculator example:
Viewing the evolutionary coupling and the modular
structure of a DRSpace simultaneously helps to reveal
flawed architectural connections.
34
Ext: Extend Impl: Implement dp: General dependencies such as call, use, and create
1 2 3 4 5 6 7 8 9 10
1 mij.ast.Node (1)
2 mij.ast.TreeVisitor (2)
3 mij.ast.Variable Ext, dp, 5 dp (3)
4 mij.ast.UnaryOperExpr Ext, dp, 10 dp (4)
5 mij.ast.OperExpr Ext, dp, 25 dp 5 (5)
6 mij.ast.Number Ext, dp, 8 dp 20 (6)
7 mij.ast.FuncExpr Ext, dp, 6 dp 15 (7)
8 mij.Interpreter dp, 10 Imp dp (8)
9 mij.parse.Convert dp dp dp dp dp dp dp (9)
10 mij.Console dp dp dp (10)
© Kazman, Cai 2017
Design Rule Space (DRSpace) Modeling
Three primary dependency:
inherit/implement, aggregate and depend
One secondary dependency:
Evolutionary dependency (faked)
Display modularity violation
Secondary dependency: Evolutionary coupling
OperExpr change with TreeVisitor for 7 times
Evolutionary coupling without
structure dependency
35 © Kazman, Cai 2017
Design Rule Space (DRSpace) Modeling
DRSpaces in Practice
36
A DRSpace in Camel
Camel ObjectHelper DRSpace
© Kazman, Cai 2017
37
Jboss JDBCCMRFieldBridge DRSpace
A DRSpace in JBoss
© Kazman, Cai 2017
DRSpaces in Practice
A DRSpace in Hadoop
38
Hadoop FileSystem DRSpace
© Kazman, Cai 2017
DRSpaces in Practice
Pieces to move from Theory to Practice
Architectural Model [Xiao et al. 2014]
New Metric [Ran et al. 2016]
Localization [Ran el al. 2015, Xiao et al. 2016]
Option to buy
Debts to Pay
Debt Quantification [Kazman et al. 2015, xiao et al. 2016]
Price of an option
Principal, penalty, interest rate
39 © Kazman, Cai 2017
A New Metric [Ran et al. 2016]
40
DR-based Architectural Metrics
Decoupling Level (DL: an options-based metric,
measuring the system’s ability to generate options)
© Kazman, Cai 2017
41
Software “metrics”:
Lines of Code (LOC)
CK metrics
McCabe complexity
Real metrics
Meter
Pound
Kilogram
Towards a Real Metric
42
Cross project Comparison
Precisely tell maintainability
differences
Independent of project domains,
sizes, ages, etc.
Bob:
140cm
Alice:
130cm
Towards a Real Metric
Towards a Real Metric
43
Degradation Monitoring
Remain stable for non-refactored
versions
Non-trivial changes should reflect
substantial architecture variation
Architectural debt thermometer
© Kazman, Cai 2017
Towards a Real Metric
44
Maintainability Health Chart
Similar to real health chart
An industrial benchmark 50th percentileBob
© Kazman, Cai 2017
Quantification: the Essence of DRT
45
Module: A true module (with high option value) should be
Small
Independent
System: A highly modularized system should
Have large numbers of true modules…
connected by design rules
© Kazman, Cai 2017
Independence Level [Ran et al. 2016]
46
A simple measure of true modules1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
1 UI_java (1)
2 Answer_java x (2) x
3 Question_java x x (3)
4 Survey_java x (4)
5 SaveLoadFile_java x (5)
6 TextFileUI_java x (6)
7 CommandLineUI_javax (7)
8 Letters_java (8)
9 Match_java x x x x (9) x
10 MatchingAnswer_javax x x x x (10)
11 ChoiceAnswer_java x x x (11) x
12 Choice_java x x x x (12)
13 EssayAnswer_java x x x (13) x
14 Written_java x x x x (14)
15 Test_java x x x (15)
16 AnswerSheet_java x (16)
#Files in the Last Layer
#AllFilesIndependence Level (DL)
The more files are
clustered into true
module, the higher the
value
© Kazman, Cai 2017
Decoupling Level
47
Improved from Independence Level (IL).
Measuring how well a system is decoupled:
Measure a module
How small is it?
Is it independently changeable?
Measure a system
How many true modules are there?
How tightly coupled are they?
© Kazman, Cai 2017
Decoupling Level
48
Based on Design Rule Hierarchy (DRH)
Distinguish different types of modules:
Upper layer modules vs. True modules1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
1 UI_java (1)
2 Answer_java x (2) x
3 Question_java x x (3)
4 Survey_java x (4)
5 SaveLoadFile_java x (5)
6 TextFileUI_java x (6)
7 CommandLineUI_javax (7)
8 Letters_java (8)
9 Match_java x x x x (9) x
10 MatchingAnswer_javax x x x x (10)
11 ChoiceAnswer_java x x x (11) x
12 Choice_java x x x x (12)
13 EssayAnswer_java x x x (13) x
14 Written_java x x x x (14)
15 Test_java x x x (15)
16 AnswerSheet_java x (16)
Upper Layer modules:
• The fewer dependents,
the higher the value
• The larger the module,
the higher the value
𝑫𝑳𝑳𝒊 =
𝒋=𝟏
𝒌
[#𝑭𝒊𝒍𝒆𝒔 𝑴𝒋
#𝑨𝒍𝒍𝑭𝒊𝒍𝒆𝒔∗ (𝟏 −
#𝑫𝒆𝒑𝒔(𝑴𝒋)
#𝑳𝒐𝒘𝒆𝒓𝑳𝒂𝒚𝒆𝒓𝑭𝒊𝒍𝒆𝒔)]
© Kazman, Cai 2017
Decoupling Level
49
Based on Design Rule Hierarchy (DRH)
Distinguish different types of modules:
Upper layer modules vs. True modules
True modules:
• The smaller a true module,
the higher the value
• The more true modules, the
higher the value
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
1 UI_java (1)
2 Answer_java x (2) x
3 Question_java x x (3)
4 Survey_java x (4)
5 SaveLoadFile_java x (5)
6 TextFileUI_java x (6)
7 CommandLineUI_javax (7)
8 Letters_java (8)
9 Match_java x x x x (9) x
10 MatchingAnswer_javax x x x x (10)
11 ChoiceAnswer_java x x x (11) x
12 Choice_java x x x x (12)
13 EssayAnswer_java x x x (13) x
14 Written_java x x x x (14)
15 Test_java x x x (15)
16 AnswerSheet_java x (16)
True Modules
𝑫𝑳𝑳𝒊=𝒏 =
𝒋=𝟏
𝒌
𝑺𝒊𝒛𝒆𝑭𝒂𝒄𝒕𝒐𝒓(𝑴𝒋)
𝑆𝑖𝑧𝑒𝐹𝑎𝑐𝑡𝑜𝑟 𝑀𝑗 =#𝐹𝑖𝑙𝑒𝑠(𝑀𝑗)
#𝐴𝑙𝑙𝐹𝑖𝑙𝑒𝑠
𝑆𝑖𝑧𝑒𝐹𝑎𝑐𝑡𝑜𝑟 𝑀𝑗 =#𝐹𝑖𝑙𝑒𝑠(𝑀𝑗)
#𝐴𝑙𝑙𝐹𝑖𝑙𝑒𝑠∗ 𝑙𝑜𝑔5(#𝐹𝑖𝑙𝑒𝑠 𝑀𝑗 )
−1
𝒘𝒉𝒆𝒏 #𝐹𝑖𝑙𝑒𝑠 𝑀𝑗 ≤ 5
𝒘𝒉𝒆𝒏 #𝐹𝑖𝑙𝑒𝑠 𝑀𝑗 > 5
© Kazman, Cai 2017
Decoupling Level
50
Based on Design Rule Hierarchy (DRH)
The value of the system
The system:
• The more true
modules, the smaller
they are, and the less
coupled, the higher
the value
𝑫𝑳 =
𝑳𝒊=𝟏
𝒏
𝑫𝑳𝑳𝒊
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
1 UI_java (1)
2 Answer_java x (2) x
3 Question_java x x (3)
4 Survey_java x (4)
5 SaveLoadFile_java x (5)
6 TextFileUI_java x (6)
7 CommandLineUI_javax (7)
8 Letters_java (8)
9 Match_java x x x x (9) x
10 MatchingAnswer_javax x x x x (10)
11 ChoiceAnswer_java x x x (11) x
12 Choice_java x x x x (12)
13 EssayAnswer_java x x x (13) x
14 Written_java x x x x (14)
15 Test_java x x x (15)
16 AnswerSheet_java x (16)
Given a DRH with n layers, its DL is equal
to the sum of the DLs of all the layers:
© Kazman, Cai 2017
Case Study using 129 projects
51
Evaluating DL to assess if it can support:
Cross-project Comparison
Evolution Monitoring
Reflecting real maintainability
Compared with
Independence Level (IL)
Propagation Cost (PC)
© Kazman, Cai 2017
Cross-project Comparison
52
The DLs of 129 projects
108 open source, 21 industrial
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1
Decoupling Level
© Kazman, Cai 2017
Cross-project Comparison
53
The DLs of 129 projects
108 open source, 21 industrial
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1
Decoupling Level
80th percentile
DL: 75%
20th percentile
DL: 46%
60% of these
projects have DLs
between 46%
and 75%
© Kazman, Cai 2017
Cross-project Comparison
54
The DLs of 129 projects
108 open source, 21 industrial
Open Source(%) Commercial(%)
DL PC IL DL PC IL
Avg 60 20 43 54 21 35
Median 58 18 41 56 20 28
Max 92 72 100 93 50 83
Min 14 2 12 15 2 9
20th Pt 47 8 28 36 6 24
40th Pt 55 14 37 46 17 26
60th Pt 66 21 45 59 24 38
80th Pt 75 34 55 65 35 46
Pt: Percentile
• Best DL (93%) is
from industrial
• Worst DL (14%) is
from open source
© Kazman, Cai 2017
Towards a “Health Chart”
55
A potential “Maintainability Health Chart”
An industrial project:
DL: 29%, 10th percentile
Confirmed to have severe maintenance difficulty
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1
Decoupling Level
10th percentile
55© Kazman, Cai 2017
Evolution Monitoring
56
Will DL values remain stable for consecutive, non-refactoring
releases ?
Can DL indicate non-trivial architecture variation ?
Day 1 Day 6
© Kazman, Cai 2017
Evolution Monitoring
57
Will DL values remain stable for consecutive, non-refactoring
releases ?
Studied 16 projects:
13 open source, 3 commercial
Each having 4-13 releases
From 236 to 9011 files
Avg CV of DL is
only 2%
DL is more stable
than PC and IL
© Kazman, Cai 2017
Evolution Monitoring
58
Can non-trivial variation in DL faithfully indicate major architecture
variation?
Studied the evolution of one industrial project
Evolved for 6 year, 29 releases, 1082-1852 files
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Decoupling Level
Release
(1)
(2)
(3)
(4)
4 transition points
© Kazman, Cai 2017
Evolution Monitoring
59
(1) DL increases from 45% to 74%:
Transforming from a prototype to a real product
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Decoupling Level
(1)
Release
© Kazman, Cai 2017
Evolution Monitoring
60
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Decoupling Level
(1)
(2)
Release
(2) DL decreases from 78% to 68%:
New features were added
Technical debt accumulates
© Kazman, Cai 2017
Evolution Monitoring
61
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Decoupling Level
(1)
(2)
(3)
Release
(3) DL decreases from 65% to 48%:
Unsuccessful refactoring
© Kazman, Cai 2017
Evolution Monitoring
62
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Decoupling Level
(1)
(2)
(3)(4)
Release
(4) DL increases from 48% to 62%:
“Cleaning up” technical debt
© Kazman, Cai 2017
Can DL Reflect Real Maintainability?
63
Are software systems with higher DLs more maintainable ?
Studied 41 projects
38 open source, 3 commercial
All releases
63-11130 files
#Committers: 18-915
#Commits: 346-74269
© Kazman, Cai 2017
Can DL Reflect Real Maintainability?
64
The higher the DL
The more files were changed
in isolation / in parallel by different developers
The more likely developers were able to work
independently without communication overhead
CCOR BCOR CCFOR BCFOR CPCO BPCO
DL -0.74 -0.77 -0.67 -0.70 -0.61 -0.63
PC 0.68 0.65 0.62 0.59 0.54 0.53
IL -0.48 -0.48 -0.49 -0.52 -0.42 -0.45
Figure: DL shows strongly negative correlations with 3 maintainability indicators
© Kazman, Cai 2017
Pieces to move from Theory to Practice
Architectural Model [Xiao et al. 2014]
New Metric [Ran et al. 2016]
Localization [Ran el al. 2015, Xiao et al. 2016]
Option to buy
Debts to Pay
Debt Quantification [Kazman et al. 2015, xiao et al. 2016]
Price of an option
Principal, penalty, interest rate
65 © Kazman, Cai 2017
Localization
Architectural Roots [Xiao el al. 2014]
The top 5 file groups
Hotspot Pattern [Ran et al. 2015]
Architecture Issues
Architectural Debt [Xiao et al. 2016]
Consecutive Debts
66 © Kazman, Cai 2017
Localization
Architectural Roots [Xiao el al. 2014]
The top 5 file groups
Hotspot Pattern [Ran et al. 2015]
Architecture Issues
Architectural Debt [Xiao et al. 2016]
Consecutive Debts
67 © Kazman, Cai 2017
The Application of DRSpaces [Xiao et al. 2014]
What new insights can DRSpace modeling bring to
software development practice?
68 © Kazman, Cai 2017
Research Questions
RQ1: If the leading file of a DRSpace is error-prone,
are the files in the DRSpace also error-prone?
RQ2: Are the majority of the error-prone files
concentrated in a few DRSpaces?
RQ3: By combining evolution and structure
coupling, can we get more insight into architectural
problems? Can this help us find not just the
locations of errors, but also the reasons for them?
69 © Kazman, Cai 2017
Case Studies
Subject History #Files #Releases #Commits #Issues
JBoss 3.2.4 Apr 00 – Jun 04 762 11 6458 550
Hadoop 0.15 Feb 06 – Oct 07 692 15 3001 490
Eclipse 3.0.2 May 01 – Mar 05 3,704 10 27,806 3,458
Subjects
These are the first 3 projects we studied
The results hold for hundreds of projects we have
studied so far
70 © Kazman, Cai 2017
Research Question 1:
If the leading file of a DRSpace is error-prone,
are the files in the DRSpace also error-prone?
71 © Kazman, Cai 2017
RQ1: Answer We selected DRSpaces led by the top 30 most error-prone
files as design rules
We then sub-selected just DRSpaces with at least 10 files
We computed dsb and bsc for each project with respect to
Bug2, Bug5 and Bug10
72 © Kazman, Cai 2017
Conventions:
Bug Space: BugN—the files that were changed for bug fixing N times or more
Design Space bugginess: dsb---the percentage of files that are also in BugN
Bug Space coverage: bsc---the percentage of files in BugN covered by the DRSpace
RQ1: Answer
We selected DRSpaces led by the top 30 most error-prone
files as design rules
We then sub-selected just DRSpaces with at least 10 files
We computed dsb and bsc for each project with respect to
Bug2, Bug5 and Bug10
73 © Kazman, Cai 2017
Consider JBoss: 9 out of 30 DRSpaces have at least 10 files
62% of the files in each selected DRSpace have more than 2 bug fixes.
The average JBoss DRSpace contains 10% of the project's Bug2 files.
RQ1: Answer cont’dJBoss:
• Consider org.jboss.ejb.Container: • 21 bug fixes; 4th most error-prone file • Its DRSpace contains 56 files
• 32 files out of the 56 files have more than 2 bug fixes• dsb = 32/56 = 57%• bsc = 32/206 = 16%
• JBoss's Bug2 space has 206 files
74 © Kazman, Cai 2017
Research Question 2:
Are the majority of the error-prone files
concentrated in a few DRSpaces?
75 © Kazman, Cai 2017
RQ2: Answer
1) What portion of the bug space can the top 5 and top 10 largest
DRSpaces cover?
The top 5 and top 10 largest DRSpaces can cover the majority of
the files in each bug space.
76 © Kazman, Cai 2017
Architecture Roots
77 © Kazman, Cai 2017
A group of files burdening a project with
ever-increasing maintenance costs
Research Question 3:
By combining evolution and structure coupling,
can we get more insight into architectural
problems? Can this help us find not just the
locations of errors, but also the reasons for them?
78 © Kazman, Cai 2017
An Architecture Root Aggregation cycles
Shared Secrets
Figure: Jboss JDBCCMRFieldBridge DRSpace
JDBCStoreManageraggregates command classes and cache class
4 command classes and cache class also aggregate JDBCStoreManager
79 © Kazman, Cai 2017
An Architecture Root
Problematic inheritance
Figure: Hadoop FileSystem Inherit DRSpace
Unusual evolutionary coupling
80 © Kazman, Cai 2017
New Insights
A significant portion of the DRSpaces led by an error prone file is
also error-prone
A few (~5) DRSpaces capture more than half of the most error-
prone files. These are the roots of ever-increasing maintenance
costs.
Error-prone DRSpaces often have architectural flaws
81 © Kazman, Cai 2017
Localization
Architectural Roots [Xiao el al. 2014]
The top 5 file groups
Hotspot Pattern [Ran et al. 2015]
Architecture Issues
Architectural Debt [Xiao et al. 2016]
Consecutive Debts
82 © Kazman, Cai 2017
Architecture Hotspots [Ran et al. 2015]
A Group of Files with Architecture Flaw and High Maintenance Costs
Primary relationship: Inherit
Leading Class
Child modules
dp
Secondary relationship: Depend
Issue 1: Parent class depends on child class
,5
,8
,26
,7
,26
+ Evolutionary coupling
,5 ,8 ,7 ,9
,6 ,7 ,9
,4 ,6,5,5
Issue 2: Unusual evolutionary coupling between parent class and child class
Issue 3: Modularity violation
83 © Kazman, Cai 2017
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
1 config.DatabaseDescriptor (1) dp,44 ,14 ,10 ,10 ,6 ,14 ,36 ,118 ,12 , ,16 ,12 ,42 ,52 ,4 ,18 ,30
2 utils.FBUtilities dp,44 (2) ,40 ,4 ,6 ,10 ,6 ,12 ,38 ,28 ,12 ,8 ,14 ,24 ,46 ,6 ,18 ,28
3 utils.ByteBufferUtil ,14 dp,40 (3) , , , ,4 ,10 ,20 ,4 ,4 ,10 ,26 ,12 ,4
4 service.WriteResponseHandler ,10 dp,4 ,2 (4) ,4 ,6 ,18 dp,22 , ,6 , ,
5 locator.TokenMetadata ,10 ,6 ,4 (5) ,4 ,10 dp,24 , ,8 ,4 ,6 ,4 ,
6 locator.NetworkTopologyStrategy ,6 dp,10 ,2 ,6 dp,4 (6) ,10 ih,22 ,4 , , ,16 , ,8
7 service.DatacenterWriteResponseHandler dp,14 dp,6 ,2 ih,18 ,10 dp,10 (7) ,20 , , ,6 ,6 ,
8 locator.AbstractReplicationStrategy ,36 dp,12 ,4 dp,22 ag,24 ,22 dp,20 (8) ,6 ,16 ,10 , ,10
9 config.CFMetaData ,118 dp,38 dp,10 , , ,4 ,6 (9) ,16 ,36 ,46 ,56
10 dht.RandomPartitioner ,12 dp,28 dp,20 ,8 , , (10) dp,4 ,4 ,16 ,50
11 utils.GuidGenerator , dp,12 ,4 , , ,4 (11) ,4 ,
12 io.sstable.SSTable ,16 ,8 dp,4 ag,16 (12) ,4 dp,68 ,10 ,
13 utils.CLibrary ,12 dp,14 ,4 (13) ,12 ,
14 io.sstable.SSTableReader dp,42 ,24 dp,10 ,36 ,4 ih,68 dp,12 (14) ,22 ,4 , ,10
15 cli.CliClient ,52 dp,46 dp,26 ,6 ,4 ,16 ,6 ,16 ,46 ,16 ,4 ,10 , ,22 (15) ,6 ,14 ,48
16 locator.PropertyFileSnitch ,4 dp,6 , dp,6 , ,6 ,10 ,4 ,6 (16) ,4
17 dht.OrderPreservingPartitioner dp,18 dp,18 dp,12 ,4 , ,50 , , ,14 (17)
18 thrift.ThriftValidation dp,30 ,28 dp,4 , , ,8 , dp,10 dp,56 , ,10 ,48 ,4 (18)
Hotspot patterns (1): Unstable Interface
84 © Kazman, Cai 2017
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
1 config.DatabaseDescriptor (1) dp,44 ,14 ,10 ,10 ,6 ,14 ,36 ,118 ,12 , ,16 ,12 ,42 ,52 ,4 ,18 ,30
2 utils.FBUtilities dp,44 (2) ,40 ,4 ,6 ,10 ,6 ,12 ,38 ,28 ,12 ,8 ,14 ,24 ,46 ,6 ,18 ,28
3 utils.ByteBufferUtil ,14 dp,40 (3) , , , ,4 ,10 ,20 ,4 ,4 ,10 ,26 ,12 ,4
4 service.WriteResponseHandler ,10 dp,4 ,2 (4) ,4 ,6 ,18 dp,22 , ,6 , ,
5 locator.TokenMetadata ,10 ,6 ,4 (5) ,4 ,10 dp,24 , ,8 ,4 ,6 ,4 ,
6 locator.NetworkTopologyStrategy ,6 dp,10 ,2 ,6 dp,4 (6) ,10 ih,22 ,4 , , ,16 , ,8
7 service.DatacenterWriteResponseHandler dp,14 dp,6 ,2 ih,18 ,10 dp,10 (7) ,20 , , ,6 ,6 ,
8 locator.AbstractReplicationStrategy ,36 dp,12 ,4 dp,22 ag,24 ,22 dp,20 (8) ,6 ,16 ,10 , ,10
9 config.CFMetaData ,118 dp,38 dp,10 , , ,4 ,6 (9) ,16 ,36 ,46 ,56
10 dht.RandomPartitioner ,12 dp,28 dp,20 ,8 , , (10) dp,4 ,4 ,16 ,50
11 utils.GuidGenerator , dp,12 ,4 , , ,4 (11) ,4 ,
12 io.sstable.SSTable ,16 ,8 dp,4 ag,16 (12) ,4 dp,68 ,10 ,
13 utils.CLibrary ,12 dp,14 ,4 (13) ,12 ,
14 io.sstable.SSTableReader dp,42 ,24 dp,10 ,36 ,4 ih,68 dp,12 (14) ,22 ,4 , ,10
15 cli.CliClient ,52 dp,46 dp,26 ,6 ,4 ,16 ,6 ,16 ,46 ,16 ,4 ,10 , ,22 (15) ,6 ,14 ,48
16 locator.PropertyFileSnitch ,4 dp,6 , dp,6 , ,6 ,10 ,4 ,6 (16) ,4
17 dht.OrderPreservingPartitioner dp,18 dp,18 dp,12 ,4 , ,50 , , ,14 (17)
18 thrift.ThriftValidation dp,30 ,28 dp,4 , , ,8 , dp,10 dp,56 , ,10 ,48 ,4 (18)
Hotspot patterns (1): Unstable Interface
85 © Kazman, Cai 2017
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
1 config.DatabaseDescriptor (1) dp,44 ,14 ,10 ,10 ,6 ,14 ,36 ,118 ,12 , ,16 ,12 ,42 ,52 ,4 ,18 ,30
2 utils.FBUtilities dp,44 (2) ,40 ,4 ,6 ,10 ,6 ,12 ,38 ,28 ,12 ,8 ,14 ,24 ,46 ,6 ,18 ,28
3 utils.ByteBufferUtil ,14 dp,40 (3) , , , ,4 ,10 ,20 ,4 ,4 ,10 ,26 ,12 ,4
4 service.WriteResponseHandler ,10 dp,4 ,2 (4) ,4 ,6 ,18 dp,22 , ,6 , ,
5 locator.TokenMetadata ,10 ,6 ,4 (5) ,4 ,10 dp,24 , ,8 ,4 ,6 ,4 ,
6 locator.NetworkTopologyStrategy ,6 dp,10 ,2 ,6 dp,4 (6) ,10 ih,22 ,4 , , ,16 , ,8
7 service.DatacenterWriteResponseHandler dp,14 dp,6 ,2 ih,18 ,10 dp,10 (7) ,20 , , ,6 ,6 ,
8 locator.AbstractReplicationStrategy ,36 dp,12 ,4 dp,22 ag,24 ,22 dp,20 (8) ,6 ,16 ,10 , ,10
9 config.CFMetaData ,118 dp,38 dp,10 , , ,4 ,6 (9) ,16 ,36 ,46 ,56
10 dht.RandomPartitioner ,12 dp,28 dp,20 ,8 , , (10) dp,4 ,4 ,16 ,50
11 utils.GuidGenerator , dp,12 ,4 , , ,4 (11) ,4 ,
12 io.sstable.SSTable ,16 ,8 dp,4 ag,16 (12) ,4 dp,68 ,10 ,
13 utils.CLibrary ,12 dp,14 ,4 (13) ,12 ,
14 io.sstable.SSTableReader dp,42 ,24 dp,10 ,36 ,4 ih,68 dp,12 (14) ,22 ,4 , ,10
15 cli.CliClient ,52 dp,46 dp,26 ,6 ,4 ,16 ,6 ,16 ,46 ,16 ,4 ,10 , ,22 (15) ,6 ,14 ,48
16 locator.PropertyFileSnitch ,4 dp,6 , dp,6 , ,6 ,10 ,4 ,6 (16) ,4
17 dht.OrderPreservingPartitioner dp,18 dp,18 dp,12 ,4 , ,50 , , ,14 (17)
18 thrift.ThriftValidation dp,30 ,28 dp,4 , , ,8 , dp,10 dp,56 , ,10 ,48 ,4 (18)
Hotspot patterns (1): Unstable Interface
86 © Kazman, Cai 2017
Hotspot patterns (2): Modularity Violation
87
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1 path1.Bean_java (1) , , , , , , , , , , , , , , , , , , , , , , , , , ,
2 path2.Pear_java , (2) , , , , , , , , , , , , , , , , , , , , , , , , ,
3 path3.FirstFruit_java , , (3) , , , , , , , , , , , , , , , , , , , , , , , ,
4 path4.SecondFruit_java Create, , , (4) , , , , , , , , , , , , , , , , , , , , , , ,
5 path4.ThirdFruit_java Create, , , , (5) , , , , , , , , , , , , , , , , , , , , , ,
6 path4.FourthFruit_java Create, , , , , (6) , , , , , , , , , , , , , , , , , , , , ,
7 path5.FifthFruit_java , , , , , , (7) , , , , , , , , , , , , , , , , , , , ,
8 path5.Pear_java , , , , , , , (8) , , , , , , , , , , , , , , , , , , ,
9 path5.FirstJuice_java , , , , , , , , (9) , , , , , , , , , , , , , , , , , ,
10 path5.SecondJuice_java , , , , , , , , , (10) , , , , , , , , , , , , , , , , ,
11 path5.SixthFruit_java , , , , , , , , , , (11) , , , , , , , , , , , , , , , ,
12 path5.SeventhFruit_java , , , , , , , , , , , (12) , , , , , , , , , , , , , , ,
13 path5.EighthFruit_java , , , , , , , , , , , , (13) , , , , , , , , , , , , , ,
14 path5.NinethFruit_java , , , , , , , , , , , , , (14) , , , , , , , , , , , , ,
15 path5.TenthFruit_java , , , , , , , , , , , , , , (15) , , , , , , , , , , , ,
16 path5.EleventhFruit_java , , , , , , , , , , , , , , , (16) , , , , , , , , , , ,
17 path5.TwelvethFruit_java , , , , , , , , , , , , , , , , (17) , , , , , , , , , ,
18path5.ThirteenthFruit_java , , , , , , , , , , Use, , , , , , , (18) , , , , , , , , ,
19path5.FourteenthFruit_java , , , , , , , , , , , , , , , , , , (19) , , , , , , , ,
20 path6.FirstFood_java , , , , , , , , , , , , , , , , , , , (20) , , , , , , ,
21 path7.SecondFood_java , , , , , , , , , , , , , , , , , , , Use, (21) , , , , , ,
22 path8.ThirdFood_java , , , , , , , , , , , , , , , , , , , Use, , (22) , , , , ,
23 path7.Apple_java , , , , , , , , , , , , , , , , , , , Use, , , (23) , , , ,
24 path9.FourthFood_java Create, , , , , , , , , , , , , , , , , , , Use, , , , (24) , , ,
25 path9.FifthFood_java Create, , , , , , , , , , , , , , , , , , , Use, , , , , (25) , ,
26 path10.FirstFig_java Cast,Use, , , , , , , , , , , , , , , , , , , , , , , , , (26) ,
27 path2.SecondFig_java Cast,Use, , , , , , , , , , , , , , , , , , , , , , , , , , (27)
© Kazman, Cai 2017
Hotspot patterns (2): Modularity Violation
88
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1 path1.Bean_java (1) , , , , , , , , , , , , , , , , , , , , , , , , , ,
2 path2.Pear_java , (2) , , , , , , , , , , , , , , , , , , , , , , , , ,
3 path3.FirstFruit_java , , (3) , , , , , , , , , , , , , , , , , , , , , , , ,
4 path4.SecondFruit_java Create, , , (4) , , , , , , , , , , , , , , , , , , , , , , ,
5 path4.ThirdFruit_java Create, , , , (5) , , , , , , , , , , , , , , , , , , , , , ,
6 path4.FourthFruit_java Create, , , , , (6) , , , , , , , , , , , , , , , , , , , , ,
7 path5.FifthFruit_java , , , , , , (7) , , , , , , , , , , , , , , , , , , , ,
8 path5.Pear_java , , , , , , , (8) , , , , , , , , , , , , , , , , , , ,
9 path5.FirstJuice_java , , , , , , , , (9) , , , , , , , , , , , , , , , , , ,
10 path5.SecondJuice_java , , , , , , , , , (10) , , , , , , , , , , , , , , , , ,
11 path5.SixthFruit_java , , , , , , , , , , (11) , , , , , , , , , , , , , , , ,
12 path5.SeventhFruit_java , , , , , , , , , , , (12) , , , , , , , , , , , , , , ,
13 path5.EighthFruit_java , , , , , , , , , , , , (13) , , , , , , , , , , , , , ,
14 path5.NinethFruit_java , , , , , , , , , , , , , (14) , , , , , , , , , , , , ,
15 path5.TenthFruit_java , , , , , , , , , , , , , , (15) , , , , , , , , , , , ,
16 path5.EleventhFruit_java , , , , , , , , , , , , , , , (16) , , , , , , , , , , ,
17 path5.TwelvethFruit_java , , , , , , , , , , , , , , , , (17) , , , , , , , , , ,
18path5.ThirteenthFruit_java , , , , , , , , , , Use, , , , , , , (18) , , , , , , , , ,
19path5.FourteenthFruit_java , , , , , , , , , , , , , , , , , , (19) , , , , , , , ,
20 path6.FirstFood_java , , , , , , , , , , , , , , , , , , , (20) , , , , , , ,
21 path7.SecondFood_java , , , , , , , , , , , , , , , , , , , Use, (21) , , , , , ,
22 path8.ThirdFood_java , , , , , , , , , , , , , , , , , , , Use, , (22) , , , , ,
23 path7.Apple_java , , , , , , , , , , , , , , , , , , , Use, , , (23) , , , ,
24 path9.FourthFood_java Create, , , , , , , , , , , , , , , , , , , Use, , , , (24) , , ,
25 path9.FifthFood_java Create, , , , , , , , , , , , , , , , , , , Use, , , , , (25) , ,
26 path10.FirstFig_java Cast,Use, , , , , , , , , , , , , , , , , , , , , , , , , (26) ,
27 path2.SecondFig_java Cast,Use, , , , , , , , , , , , , , , , , , , , , , , , , , (27)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
1 path1.Bean_java (1) , ,10 , ,10 , , , , , , , , , ,8 ,8 , , , ,16 , , , ,8 , , ,
2 path2.Pear_java , (2) , , , , , , , , , , , , ,8 , , , , , , , , , , , ,
3 path3.FirstFruit_java ,10 , (3) , , , ,8 , , , , ,8 ,8 , ,10 ,10 ,8 , ,8 ,8 , , , , , , ,
4 path4.SecondFruit_java Create, , , (4) ,10 ,8 , , , , , , , , ,8 ,8 , , , ,10 , , , , , , ,
5 path4.ThirdFruit_java Create,10 , , ,10 (5) , , , , , , , , , , , , , , ,14 , , , , , , ,
6 path4.FourthFruit_java Create, , , ,8 , (6) , , , , , , , , , , , , , , , , , , , , ,
7 path5.FifthFruit_java , , ,8 , , , (7) , ,8 , , ,12 ,16 ,14 ,16 ,10 ,10 ,8 ,10 , , , , , , , ,
8 path5.Pear_java , , , , , , , (8) , , , , , , , ,10 , , , , , , , , , , ,
9 path5.FirstJuice_java , , , , , , ,8 , (9) ,8 , ,10 ,10 , ,12 ,8 ,10 ,12 , , , , , , , , ,
10 path5.SecondJuice_java , , , , , , , , ,8 (10) , , , , ,10 ,8 , , , , , , , , , , ,
11 path5.SixthFruit_java , , , , , , , , , , (11) , , , ,10 , , ,8 , , , , , , , , ,
12 path5.SeventhFruit_java , , ,8 , , , ,12 , ,10 , , (12) ,14 ,10 ,12 ,10 ,10 ,10 ,8 , , , , , , , ,
13 path5.EighthFruit_java , , ,8 , , , ,16 , ,10 , , ,14 (13) ,14 ,16 ,10 ,10 ,10 ,10 , , , , , , , ,
14 path5.NinethFruit_java , , , , , , ,14 , , , , ,10 ,14 (14) ,14 ,10 ,8 , ,10 , , , , , , , ,
15 path5.TenthFruit_java ,8 ,8 ,10 ,8 , , ,16 , ,12 ,10 ,10 ,12 ,16 ,14 (15) ,14 ,14 ,10 ,14 ,10 , , , , , , ,
16 path5.EleventhFruit_java ,8 , ,10 ,8 , , ,10 ,10 ,8 ,8 , ,10 ,10 ,10 ,14 (16) ,12 , ,8 ,10 , , , , ,8 , ,
17 path5.TwelvethFruit_java , , ,8 , , , ,10 , ,10 , , ,10 ,10 ,8 ,14 ,12 (17) , , , , , , , , , ,
18path5.ThirteenthFruit_java , , , , , , ,8 , ,12 , Use,8 ,10 ,10 , ,10 , , (18) , , , , , , , , ,
19path5.FourteenthFruit_java , , ,8 , , , ,10 , , , , ,8 ,10 ,10 ,14 ,8 , , (19) , , , , , , , ,
20 path6.FirstFood_java ,16 , ,8 ,10 ,14 , , , , , , , , , ,10 ,10 , , , (20) , ,8 ,10 ,8 ,8 , ,
21 path7.SecondFood_java , , , , , , , , , , , , , , , , , , , Use, (21) , ,14 , , , ,
22 path8.ThirdFood_java , , , , , , , , , , , , , , , , , , , Use,8 , (22) , , , , ,
23 path7.Apple_java , , , , , , , , , , , , , , , , , , , Use,10 ,14 , (23) , , , ,
24 path9.FourthFood_java Create,8 , , , , , , , , , , , , , , , , , , Use,8 , , , (24) ,10 , ,
25 path9.FifthFood_java Create, , , , , , , , , , , , , , , ,8 , , , Use,8 , , , ,10 (25) , ,
26 path10.FirstFig_java Cast,Use, , , , , , , , , , , , , , , , , , , , , , , , , (26) ,8
27 path2.SecondFig_java Cast,Use, , , , , , , , , , , , , , , , , , , , , , , , , ,8 (27)
© Kazman, Cai 2017
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
1 config.DatabaseDescriptor (1) dp,44 ,14 ,10 ,10 ,6 ,14 ,36 ,118 ,12 , ,16 ,12 ,42 ,52 ,4 ,18 ,30
2 utils.FBUtilities dp,44 (2) ,40 ,4 ,6 ,10 ,6 ,12 ,38 ,28 ,12 ,8 ,14 ,24 ,46 ,6 ,18 ,28
3 utils.ByteBufferUtil ,14 dp,40 (3) , , , ,4 ,10 ,20 ,4 ,4 ,10 ,26 ,12 ,4
4 service.WriteResponseHandler ,10 dp,4 ,2 (4) ,4 ,6 ,18 dp,22 , ,6 , ,
5 locator.TokenMetadata ,10 ,6 ,4 (5) ,4 ,10 dp,24 , ,8 ,4 ,6 ,4 ,
6 locator.NetworkTopologyStrategy ,6 dp,10 ,2 ,6 dp,4 (6) ,10 ih,22 ,4 , , ,16 , ,8
7 service.DatacenterWriteResponseHandler dp,14 dp,6 ,2 ih,18 ,10 dp,10 (7) ,20 , , ,6 ,6 ,
8 locator.AbstractReplicationStrategy ,36 dp,12 ,4 dp,22 ag,24 ,22 dp,20 (8) ,6 ,16 ,10 , ,10
9 config.CFMetaData ,118 dp,38 dp,10 , , ,4 ,6 (9) ,16 ,36 ,46 ,56
10 dht.RandomPartitioner ,12 dp,28 dp,20 ,8 , , (10) dp,4 ,4 ,16 ,50
11 utils.GuidGenerator , dp,12 ,4 , , ,4 (11) ,4 ,
12 io.sstable.SSTable ,16 ,8 dp,4 ag,16 (12) ,4 dp,68 ,10 ,
13 utils.CLibrary ,12 dp,14 ,4 (13) ,12 ,
14 io.sstable.SSTableReader dp,42 ,24 dp,10 ,36 ,4 ih,68 dp,12 (14) ,22 ,4 , ,10
15 cli.CliClient ,52 dp,46 dp,26 ,6 ,4 ,16 ,6 ,16 ,46 ,16 ,4 ,10 , ,22 (15) ,6 ,14 ,48
16 locator.PropertyFileSnitch ,4 dp,6 , dp,6 , ,6 ,10 ,4 ,6 (16) ,4
17 dht.OrderPreservingPartitioner dp,18 dp,18 dp,12 ,4 , ,50 , , ,14 (17)
18 thrift.ThriftValidation dp,30 ,28 dp,4 , , ,8 , dp,10 dp,56 , ,10 ,48 ,4 (18)
Hotspot patterns (3): Unhealthy Inheritance
89 © Kazman, Cai 2017
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
1 config.DatabaseDescriptor (1) dp,44 ,14 ,10 ,10 ,6 ,14 ,36 ,118 ,12 , ,16 ,12 ,42 ,52 ,4 ,18 ,30
2 utils.FBUtilities dp,44 (2) ,40 ,4 ,6 ,10 ,6 ,12 ,38 ,28 ,12 ,8 ,14 ,24 ,46 ,6 ,18 ,28
3 utils.ByteBufferUtil ,14 dp,40 (3) , , , ,4 ,10 ,20 ,4 ,4 ,10 ,26 ,12 ,4
4 service.WriteResponseHandler ,10 dp,4 ,2 (4) ,4 ,6 ,18 dp,22 , ,6 , ,
5 locator.TokenMetadata ,10 ,6 ,4 (5) ,4 ,10 dp,24 , ,8 ,4 ,6 ,4 ,
6 locator.NetworkTopologyStrategy ,6 dp,10 ,2 ,6 dp,4 (6) ,10 ih,22 ,4 , , ,16 , ,8
7 service.DatacenterWriteResponseHandler dp,14 dp,6 ,2 ih,18 ,10 dp,10 (7) ,20 , , ,6 ,6 ,
8 locator.AbstractReplicationStrategy ,36 dp,12 ,4 dp,22 ag,24 ,22 dp,20 (8) ,6 ,16 ,10 , ,10
9 config.CFMetaData ,118 dp,38 dp,10 , , ,4 ,6 (9) ,16 ,36 ,46 ,56
10 dht.RandomPartitioner ,12 dp,28 dp,20 ,8 , , (10) dp,4 ,4 ,16 ,50
11 utils.GuidGenerator , dp,12 ,4 , , ,4 (11) ,4 ,
12 io.sstable.SSTable ,16 ,8 dp,4 ag,16 (12) ,4 dp,68 ,10 ,
13 utils.CLibrary ,12 dp,14 ,4 (13) ,12 ,
14 io.sstable.SSTableReader dp,42 ,24 dp,10 ,36 ,4 ih,68 dp,12 (14) ,22 ,4 , ,10
15 cli.CliClient ,52 dp,46 dp,26 ,6 ,4 ,16 ,6 ,16 ,46 ,16 ,4 ,10 , ,22 (15) ,6 ,14 ,48
16 locator.PropertyFileSnitch ,4 dp,6 , dp,6 , ,6 ,10 ,4 ,6 (16) ,4
17 dht.OrderPreservingPartitioner dp,18 dp,18 dp,12 ,4 , ,50 , , ,14 (17)
18 thrift.ThriftValidation dp,30 ,28 dp,4 , , ,8 , dp,10 dp,56 , ,10 ,48 ,4 (18)
Hotspot patterns (4): Clique
90 © Kazman, Cai 2017
Localization
Architectural Roots [Xiao el al. 2014]
The top 5 file groups
Hotspot Pattern [Ran et al. 2015]
Architecture Issues
Architectural Debt [Xiao et al. 2016]
Consecutive Debts
© Kazman, Cai 201791
Formal Definition [Xiao et al. 2016]
92
Architectural debt (ArchDebt):
a group of architecturally connected files that incur high
maintenance costs over time due to their flawed connections.
© Kazman, Cai 2017
Subjects
93
7 Apache open source projects
311 to 9866 java files
4 to 8 years of revision history
2005 to 14673 commits
1857 to 6280 bug tickets
9 to 15 stable releases
Interest Rate of ArchDebts
94
Four types of regression models
T: ordinal number of releases
Cost(T): bug
fixing lines of
code at T
(1) Linear Regression Model - Stable Interest Rate
Interest Rate of ArchDebts
95
Four types of regression models
T: ordinal number of releases
Cost(T): bug
fixing lines of
code at T
(2) Logarithmic Regression Model – Decreasing Interest Rate
Interest Rate of ArchDebts
96
Four types of regression models
T: ordinal number of releases
Cost(T): bug
fixing lines of
code at T
(3) Exponential Regression Model – Increasing Interest Rate
Interest Rate of ArchDebts
97
Four types of regression models
T: ordinal number of releases
Cost(T): bug
fixing lines of
code at T
(4) Polynomial Regression Model – Fluctuating Interest Rate
Cost Regression Model
98
Distribution of interest rates for ArchDebts
Support decision making:
Whether, when, and where to refactor?
Linear53%Log
14%
Exp8%
Poly25%
Sample ArchDebt Interest Rate
99
R-2.0.0, Age 1, 11Files, 392 lines R-2.2.0, Age 2, 20 Files, 771 lines
R-2.12.4, Age 11, 28 Files, 2134 lines
DebtModel (T) = * T + 509.35158.75
Pieces to move from Theory to Practice
Architectural Model [Xiao et al. 2014]
New Metric [Ran et al. 2016]
Localization [Ran el al. 2015, Xiao et al. 2016]
Option to buy
Debts to Pay
Debt Quantification [Kazman et al. 2015, Xiao et al. 2016]
Price of an option
Principal, penalty, interest rate
100 © Kazman, Cai 2017
*L. Xiao, Y. Cai, R. Kazman, “Titan: A Toolset That Connects Software Architecture
with Quality Analysis”, Proceedings of FSE 2014, (Hong Kong), November 2014.© Kazman, Cai 2017101
Economic Analysis
Based on the identified DRSpaces and an identification of
their architecture flaws, we can plan refactoring strategies.
And we can make decisions about whether to refactor based
on ROI.
This analysis is entirely based on
commonly available project data.
Consider the following analysis
of a SoftServe project:
© Kazman, Cai 2017102
Economic Analysis
© Kazman, Cai 2017103
Economic Analysis
© Kazman, Cai 2017104
Economic Analysis
© Kazman, Cai 2017105
Economic Analysis
© Kazman, Cai 2017106
Economic Analysis
© Kazman, Cai 2017107
Economic Analysis
© Kazman, Cai 2017108
Economic Analysis
© Kazman, Cai 2017109
Economic Analysis
© Kazman, Cai 2017110
Economic Analysis
© Kazman, Cai 2017111
Economic Analysis
© Kazman, Cai 2017112
Economic Analysis
© Kazman, Cai 2017113
Economic Analysis
© Kazman, Cai 2017114
Economic Analysis
Result: ~300% ROI in the first year alone!
© Kazman, Cai 2017115
Take-aways
Architectural flaws lead to quality
issues.
We can locate these flaws!
We can not fix the quality
issues without fixing
the underlying flaws!
We can quantify the debt incurred
by such flaws.
© Kazman, Cai 2017116
Take-aways
This analysis allows us to plan
refactoring strategies and make
informed, economics-based
decisions about if and how to
refactor.
© Kazman, Cai 2017117
Bring the Theory to Practice
118
[xiao et al. 2015]
Titan
https://art.cs.drexel.edu:4000/
https://archdia.com
This work was supported in part by the U.S. National Science Foundation
under grants CCF-0916891, CCF-1065189, CCF-1116980 and DUE-0837665.
© Kazman, Cai 2017119
© Kazman, Cai 2017120
References[Conway 1968] Melvin Conway, (April 1968), "How do Committees Invent?",
Datamation, 14 (5): 28–31.
[Baldwin and Clark 1999] Carliss Baldwin, Kim Clark, 1999. Design Rules:
The Power of Modularity Volume 1. MIT Press, Cambridge, MA, USA.
[Cunningham 1992] Ward Cunningham (1992-03-26). "The WyCash
Portfolio Management System".
[Parnas 1972] David Parnas, On the criteria to be used in decomposing
system into modules, Communications ACM, Vol. 15, No. 12, December
1972 pp. 1053–1058.
© Kazman, Cai 2017121
References
122
[Sullivan et al 2001] Kevin J. Sullivan, William G. Griswold, YuanfangCai, Ben Hallen, "The structure and value of modularity in software design", ESEC/FSE 2001: 99-108.
[Cai 2006] Yuanfang Cai, Kevin Sullivan, "Modularity Analysis of Logical Design Models", ASE 2006: 91-102
[Wong et al. 2009] Sunny Wong, Yuanfang Cai, Giuseppe Valetto, GeorgiSimeonov, Kanwarpreet Sethi, "Design Rule Hierarchies and Parallelism in Software Development Tasks", ASE 2009: 197-208
[Wong et al. 2011] Sunny Wong, Yuanfang Cai, Miryung Kim, Michael Dalton, "Detecting software modularity violations", ICSE 2011: 411-420
© Kazman, Cai 2017
References
123
[Cai et al. 2013] Yuanfang Cai, Rick Kazman, Ciera Jaspan, Jonathan
Aldrich, "Introducing tool-supported architecture review into software
design education", CSEE&T 2013: 70-79
[Xiao et al. 2014] Lu Xiao, Yuanfang Cai, Rick Kazman, "Titan: a toolset
that connects software architecture with quality analysis", SIGSOFT FSE
2014: 763-766
[Xiao et al. 2014] Lu Xiao, Yuanfang Cai, Rick Kazman, "Design rule
spaces: a new form of architecture insight", ICSE 2014: 967-977
© Kazman, Cai 2017
References[Naedele et al. 2014] Martin Naedele, Rick Kazman, Yuanfang Cai,
"Making the case for a "manufacturing execution system" for software
development", Communations of the ACM 57(12): 33-36 (2014)
[Mo et al. 2015] Ran Mo, Yuanfang Cai, Rick Kazman, Lu Xiao, "Hotspot
Patterns: The Formal Definition and Automatic Detection of Architecture
Smells", WICSA 2015: 51-60
[Kazman et al. 2015] Rick Kazman, Yuanfang Cai, Ran Mo, Qiong Feng, Lu
Xiao, Serge Haziyev, Volodymyr Fedak, Andriy Shapochka, "A Case Study
in Locating the Architectural Roots of Technical Debt", ICSE 2015: 179-188
124
References[Xiao et al. 2016] Lu Xiao, Yuanfang Cai, Rick Kazman, Ran Mo, Qiong Feng, "Identifying and quantifying architectural debt", ICSE 2016: 488-498
[Mo et al. 2016] Ran Mo, Yuanfang Cai, Rick Kazman, Lu Xiao, Qiong Feng, "Decoupling level: a new metric for architectural maintenance complexity", ICSE 2016: 499-510
[Feng et al. 2016] Qiong Feng, Rick Kazman, Yuanfang Cai, Ran Mo, Lu Xiao, "Towards an Architecture-Centric Approach to Security Analysis", WICSA 2016: 221-230
125
Top Related