Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ......
Transcript of Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ......
![Page 1: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/1.jpg)
Automated Refactoring: Metrics are Not Enough
Chris Simons1, Jeremy Singer2, David White2
1 Department of Computer Science and Creative Technologies, University of the West of England, Bristol, BS16 1QY, United Kingdom
[email protected] 2 School of Computing Science, University of the Glasgow, G12 8RZ, United Kingdom
{jeremy.singer,david.r.white}@glasgow.ac.uk
14 September 2015
![Page 2: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/2.jpg)
Agenda
• Motivation • (semi-)automated software refactoring • Role of structural metrics in search for refactoring
• Hypotheses • Experimental Methodology
• Survey of software development practitioners
• Results • Quantitative and qualitative analysis
• Related Work • Conclusions
2
![Page 3: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/3.jpg)
3
![Page 4: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/4.jpg)
4 https://www.capgemini.com/blog/capping-it-off/2011/05/the-scrum-perspective-to-agile
![Page 5: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/5.jpg)
5
Software Evolution
“If there is one thing of which we can be sure, it is that a system of any substantial size is going to evolve. It is going to evolve even as it is
undergoing development. Later, when in use, the changing environment will call for further evolution. … the system itself should be resilient to
change, or change tolerant. Another way of putting this goal is to say that the system should be capable of evolving gracefully.”
As we iteratively engineer software, it must adapt to (changing) requirements…
Jacobson, I., Booch, G., Rumbaugh, J. (1999) The Unified Software Development Process, Addison Wesley.
![Page 6: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/6.jpg)
6
natural evolution i.e. the change in the inherited characteristics of
biological populations over successive generations.
sexual reproduction for diversity and population change
selection of fittest individuals
environment
www.flickr.com/photos/armincifuentes/
![Page 7: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/7.jpg)
7
Computational Evolution (Search)
initialise population at random while( not done ) evaluate each individual select parents recombine pairs of parents mutate new candidate individuals select candidates for next generation end while
Representation of encoded “individual” (solution) e.g. models, trees, arrays etc. etc.
Eiben, A.E., Smith, J.E. (2003) Introduction to Evolutionary Computing, Springer.
![Page 8: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/8.jpg)
8
Evolutionary Fitness Landscape
![Page 9: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/9.jpg)
• Kosa (1992) – Genetic Programming
• Holland (1975) – Genetic Algorithms
• Rechenburg (1973) – Evolutionary Strategies
• Fogel et al. (1966) – Evolutionary programming (finite state machines)
• Alex Fraser (1957) – Computational simulation of natural evolution
• Alan Turin (1952) • “Computing Machinery and Intelligence” in Mind • hints at a “…genetical programming...”
9
How old is Evolutionary Computing?
Is evolutionary computing as old as programming??
![Page 10: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/10.jpg)
– Evolving Objects (EO): an Evolutionary Computation Framework (C++)
• http://eodev.sourceforge.net/ – Open BEAGLE (C++)
• https://code.google.com/p/beagle/ – ECJ 21 (Java Evolutionary Computation)
• http://cs.gmu.edu/~eclab/projects/ecj/ – ECF (Evolutionary Computational Framework) (C++)
• http://gp.zemris.fer.hr/ecf/ – JCLEC – Java Class Library for Evolutionary Computation
• http://jclec.sourceforge.net/ – Etc. etc.
10
Some resources available
![Page 11: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/11.jpg)
11
Motivation (1) Evolutionary computation extensively applied to various aspects of software development (e.g. requirements, test case generation, architecture, design, programming)
Evolutionary computation extensively applied to refactor object-oriented software. Metrics quantify structural properties such as cohesion, coupling, module dependencies, etc.
a.k.a. Search-Based Software Engineering (SBSE)
![Page 12: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/12.jpg)
12
Motivation (2)
Barros, M.0. and Farzat, F.A.: What Can a Big Program Teach Us about Optimization? Proceedings of SSBSE 2013, LCNS 8084. Springer (2013)
O’Cinneide, M., Tratt, L., Harman, M., Counsell, S. and Moghadam, I.: Experimental Assessment of Software Metrics using Automated Refactoring. Proceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. ACM (2012)
SBSE refactoring of Apache Ant project does not improve design as assessed by an expert
When SBSE refactoring is conducted using a number of cohesion metrics, there can be disagreement between metrics
![Page 13: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/13.jpg)
13
Questions
Q: What’s the relationship between metrics and design quality?
Q: Qualitatively, how do software engineers make such judgements?
We try to answer these questions, by: - placing ultimate judgement of software quality with practising software engineers, - measuring any correlation between metrics used in search and human judgement, - examining the articulated justifications for those judgements.
![Page 14: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/14.jpg)
Signpost
• Motivation • Role of structural metrics in software refactoring
• Hypotheses • Experimental Methodology
• Survey of industrial software development practitioners
• Results • Quantitative and qualitative analysis reported
• Related Work • Conclusions
14
![Page 15: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/15.jpg)
15
H0: There is no correlation between software metric values and software engineer evaluation of quality for a given software metric.
H1: There is a correlation between software metric values and software engineer evaluation of quality for a given software metric.
We formulate our hypotheses such that the null hypothesis makes no assumption of an effect:
![Page 16: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/16.jpg)
Signpost
• Motivation • Role of structural metrics in software refactoring
• Hypotheses
• Experimental Methodology • Survey of industrial software development
practitioners • Results
• Quantitative and qualitative analysis reported
• Related Work • Conclusions
16
![Page 17: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/17.jpg)
17
Experimental Methodology
A. Survey Design
B. Questionnaire Design
C. Survey Process
Three components:
![Page 18: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/18.jpg)
18
A. Survey Design
![Page 19: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/19.jpg)
19
A. Survey Design - selection of Software Designs
Two problem domains to strengthen generality of findings - Automated Teller Machine (ATM) - Nautical Cruise Booking System
Bjork, R.C., ATM Simulation, http://www.math-cs.gordon.edu/courses/cs211/ATMExample/
Apperly, H., Simons, C.L. et al. Service- and Component-Based Development. Addison-Wesley (2003)
Five experienced software engineers produced class diagrams for each problem domain – 10 diagrams in total – and metric values calculated for each class diagram.
All designs are available at: www.cems.uwe.ac.uk/~clsimons/MetricsAreNotEnough
Balance of meaningful design versus cognitive overload
![Page 20: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/20.jpg)
20
A. Survey Design - selection of Design Qualities
Quality Model for Object-Oriented Design (QMOOD) relates six design quality attributes to corresponding properties and metrics.
Attribute Definition Reusability Reflects the presence of object-oriented design characteristics that allow
a design to be reapplied to a new problem without significant effort.
Flexibility Characteristics that allow the incorporation of changes in a design. The ability of a design to be adapted to provide functionally related capabilities.
Understandability The properties of a design that enable it to be easily learned and comprehended. This directly relates to the complexity of the design structure.
Bansiya, J. and Davis, C.G.: A Hierarchical Model for Object-Oriented Design Quality Assessment. IEEE Transactions on Software Engineering. 28(1), 4-17 (2002)
To reduce cognitive load in survey, we focussed on the most problem-domain independent i.e.
![Page 21: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/21.jpg)
21
A. Survey Design - selection of Metrics
Q: What is the distribution of software metrics among the SBSE refactoring literature?
Search query of “software metrics” in SBSE Repository… Zhang, Y.: SBSE Repository. http://crestweb.cs.ucl.ac.uk/resources/sbse_repository/
…yielded 57 papers. Excluding non-refactoring sources narrowed the list to 23.
118 different metrics used. Only 3 (LCOM, MQ, EVM) used more than once, and 1 suite (QMOOD) used more than once.
From these, we selected the QMOOD metrics Design Size in Classes (DSC), Direct Class Coupling (DCC) and Numbers of Methods (NOM).
Also selected: - elegance metric Numbers Among Classes (NAC) (used by Barros & Farzat) - Numbers of Attributes and Methods (NOAM) (to cater for attributes)
![Page 22: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/22.jpg)
22
A. Survey Design – Target Population, Sample Frame
Association of C and C++ Users www.accu.org
British Computer Society www.bcs.org
Approx. 900 members via email list
Approx. 11,000 members via LinkedIn Forum
Many, many thanks to all who responded!!!!
![Page 23: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/23.jpg)
23
B. Questionnaire Design
Designs presented at random to participants, one model per participant.
Likert Scale used to evaluate designs with seven levels: “strongly disagree”, “disagree”, “somewhat disagree”, “neutral”, “somewhat agree”, “agree” and “strongly agree”
https://en.wikipedia.org/wiki/Likert_scale
Informed consent from participants obtained. All survey data strictly confidential and published information reported either as aggregated data or anonymised.
Pretesting conducted with five experienced software engineers.
SurveyGismo used as survey platform.
Questionnaire available at: www.cems.uwe.ac.uk/~clsimons/MetricsAreNotEnough
![Page 24: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/24.jpg)
24
C. Survey Process
Survey open from 18 January to 28 February 2015.
Participants posted lively comments to the BCS LinkedIn Forum e.g. “it’s difficult to form an impression of design qualities using a class diagram in isolation of other development aspects e.g. dynamic models of behaviour, requirements, test plan etc.”
One forum contributor remarked: “it seems that your idea of what quality is and how to judge it is not the same as many of us in the industry”
![Page 25: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/25.jpg)
Signpost
• Motivation • Role of structural metrics in software refactoring
• Hypotheses • Experimental Methodology
• Survey of industrial software development practitioners
• Results • Quantitative and qualitative analysis reported
• Related Work • Conclusions
25
![Page 26: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/26.jpg)
26
Anonymised data available
50 responses received
Histogram of number of year’s experience of respondents
Respondents self-assessed expertise in software design and confidence in their ratings
Quantitative Results
http://www.cems.uwe.ac.uk/~clsimons/MetricsAreNotEnough/
![Page 27: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/27.jpg)
27
Scatter Plots and Correlation between Human Judgment of Software Qualities and Software Metrics
![Page 28: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/28.jpg)
28
Quality DSC DCC NOM NAC NOAM
Understandable -0.128 -0.271 -0.203 -0.0400 0.103
Reusable -0.0280 -0.158 -0.195 -0.200 0.0572
Flexible 0.0386 -0.0806 -0.0677 -0.0613 0.202
Quality DSC DCC NOM NAC NOAM
Understandable 0.375 0.0571 0.156 0.783 0.487
Reusable 0.847 0.272 0.174 0.163 0.693
Flexible 0.790 0.578 0.640 0.672 0.160
Spearman’s Rank Coefficients for the Correlation between Metrics and Human Judgement (to 3 s.f.)
P-Values for a Two-Sided Test of the Spearman’s Rank Correlation Coefficients (to 3 s.f.)
![Page 29: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/29.jpg)
29
We asked the engineers to justify their judgments in prose. We then coded their responses as per Grounded Theory.
Qualitative Results
Classifications for Rationale behind Judgment of “Understandable”
![Page 30: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/30.jpg)
30
Classifications for Rationale behind Judgment of “Flexible”
![Page 31: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/31.jpg)
31
Classifications for Rationale behind Judgment of “Reuseable”
![Page 32: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/32.jpg)
32
We make the following observations:
A class diagram is not enough
The Problem Domain matters
Qualities have meaning only in a given context
Good design is intuitive
Our standard metrics play a minor part
![Page 33: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/33.jpg)
Signpost
• Motivation • Role of structural metrics in software refactoring
• Hypotheses • Experimental Methodology
• Survey of industrial software development practitioners
• Results • Quantitative and qualitative analysis reported
• Related Work • Conclusions
33
![Page 34: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/34.jpg)
34
![Page 35: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/35.jpg)
35
![Page 36: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/36.jpg)
36
![Page 37: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/37.jpg)
37
![Page 38: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/38.jpg)
38
![Page 39: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/39.jpg)
39
![Page 40: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/40.jpg)
40
![Page 41: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/41.jpg)
41
![Page 42: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/42.jpg)
42
![Page 43: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/43.jpg)
43
Our Threats to Validity
Class models are small: constrained by cognitive overload and screen space
Participant understanding of qualities: we provided definitions and navigation to revisit
Bias of target population and sample frame: (as with any survey) targeted professional institutions and practitioners Pilot survey could have been more extensive
![Page 44: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/44.jpg)
Signpost
• Motivation • Role of structural metrics in software refactoring
• Hypotheses • Experimental Methodology
• Survey of industrial software development practitioners
• Results • Quantitative and qualitative analysis reported
• Related Work
• Conclusions 44
![Page 45: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/45.jpg)
45
Conclusions (1)
Refactoring metrics are not correlated with human engineer judgement
Unable to refute the null hypothesis; thus unable to support
conjecture that refactoring tools relying solely on these metrics will consistently propose useful refactored models to engineers.
![Page 46: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/46.jpg)
46
Conclusions (2) – wider lessons
Simple metrics are not able to entirely capture essential qualities of software design used by human engineers Software is inextricably connected to a problem domain
We note recent advances in machine learning and automatic programming to address such concerns…
…but without their inclusion,
human-in-the-loop automated refactoring systems may be required for meaningful solutions.
![Page 47: Automated Refactoring: Metrics are Not Enough · Automated Refactoring: Metrics are Not Enough ... Tratt, L., Harman, M., Counsell, ... Automated Refactoring: Metrics are Not Enough](https://reader035.fdocuments.net/reader035/viewer/2022062504/5b1da1057f8b9a64508ba071/html5/thumbnails/47.jpg)
47
Automated Refactoring: Metrics are Not Enough
questions?
@chrislsimons [email protected] @jsinger_compsci [email protected] @davidwhitecs [email protected]