Icsme14danieletal 150722141344-lva1-app6891
Transcript of Icsme14danieletal 150722141344-lva1-app6891
An Empirical Study of Delays In The Integration of Addressed Issues
Daniel A. Costa Shane McIntosh Uirá Kulesza Ahmed E. Hassan Surafel L. Abebe
The term issue can refer to a bug fix, a new feature or an enhancement
Bug fix
New feature
Enhancement
1
So when does this stuff get added? Will it be applied to the next FF23 beta? A 22.01 release? Otherwise?
#883554
But an addressed issue may take some time to be integrated into a release
5
3,121 issues 17 releases
An empirical study to analyze the integration delay
6
14,530 issues 15 releases
3,344 issues (JDT)
11 releases
3,121 issues 17 releases
3,121 issues 17 releases
An empirical study to analyze the integration delay
6
14,530 issues 15 releases
3,344 issues (JDT)
11 releases
3,121 issues 17 releases
20,995 issues
3,121 issues 17 releases
An empirical study to analyze the integration delay
6
14,530 issues 15 releases
3,344 issues (JDT)
11 releases
3,121 issues 17 releases
20,995 issues
Rapid Traditional Traditional
Release Notes
We collected data from release notes and issue tracking systems
7
#14 #15 #16
List of Integrated Bug Ids
extract
Release Notes
We collected data from release notes and issue tracking systems
7
#14 #15 #16
List of Integrated Bug Ids
extract
Issue Tracking System (ITS)
get info.
How long an issue was addressed before a release?
30 days
40 days 0.75
1
Time
30 40
Addressing Stage 12
Delayed issues were addressed well before the next release
Ad
dre
ssin
g St
age
1.00 0.75 0.50 0.25 0.00
13
Delayed issues were addressed well before the next release
Ad
dre
ssin
g St
age
13
1.00 0.75 0.50 0.25 0.00
Issues are usually delayed in rapid releasing cycle
14
Integration delay
Prediction models
Influential variables
Issues are adressed well before a release
14
Integration delay
Prediction models
Influential variables
Issues are usually delayed in rapid releasing cycle
14
Integration delay
Prediction models
Influential variables
Issues are adressed well before a release
Issues are usually delayed in rapid releasing cycle
We built our prediction model using random forests
It is robust to noisy data
It deals with several types of attributes
16
Studied metrics
Process metrics
Reporter metrics Issue metrics
Project metrics 18
Reporter Experience
Number of comments
Issue priority
Integration workload
Studied metrics
18 Process metrics
Reporter metrics Issue metrics
Project metrics
Reporter Experience
Number of comments
Issue priority
Integration workload
Measuring the integration workload of a given addressed issue
Time
500 ? We are still waiting as well!
19
Our models perform better than random guessing
Precision
Recall
ROC Areas 20
0.59 to 0.88
0.62 to 0.88
All above 0.74
Our models perform better than random guessing
Precision
Recall
ROC Areas 20
0.59 to 0.88
0.62 to 0.88
All above 0.74
21
Integration delay
Prediction models
Influential variables
Issues are adressed well before a release
Issues are usually delayed in rapid releasing cycle
21
Integration delay
Prediction models
Influential variables
Issues are adressed well before a release
Issues are usually delayed in rapid releasing cycle
Our models perform better than random prediction
21
Integration delay
Prediction models
Influential variables
Issues are adressed well before a release
Issues are usually delayed in rapid releasing cycle
Our models perform better than random prediction
We computed the variable importance of our methods
Experience Workload #Comments
55 340 30
66 100 70
78 540 22
Accuracy: 0.8
22
We computed the variable importance of our methods
Accuracy: 0.8
22
Experience Workload #Comments
55 340 30
66 100 70
78 540 22
We computed the variable importance of our methods
Accuracy: 0.8
Experience Workload #Comments
66 340 30
78 100 70
55 540 22
Accuracy: 0.79
22
Experience Workload #Comments
55 340 30
66 100 70
78 540 22
We computed the variable importance of our methods
Accuracy: 0.8
22
Experience Workload #Comments
55 340 30
66 100 70
78 540 22
We computed the variable importance of our methods
Accuracy: 0.8
Experience Workload #Comments
55 100 30
66 340 70
78 540 22
Accuracy: 0.3
22
Experience Workload #Comments
55 340 30
66 100 70
78 540 22
Javascript Engine is the component with the most delayed issues in firefox
0
10
20
30
40
50
60
70
80
90
Javascript Engine General
Next
After 1
After 2
>= 3
Perc
ent
of
del
ayed
is
sues
30.4 28
16
4.5
79
18 22
52
25
Javascript Engine is the component with the most delayed issues in firefox
0
10
20
30
40
50
60
70
80
90
Javascript Engine General
Next
After 1
After 2
>= 3
Perc
ent
of
del
ayed
is
sues
30.4 28
16
4.5
79
18 22
52
25
Javascript Engine is the component with the most delayed issues in firefox
0
10
20
30
40
50
60
70
80
90
Javascript Engine General
Next
After 1
After 2
>= 3
Perc
ent
of
del
ayed
is
sues
30.4 28
16
4.5
79
18 22
52
25
Surprisingly, priority and severity have little influence on our models
0
20
40
60
80
100
120
ArgoUML Eclipse Firefox
Priority 1 (P1)
Priority 2 (P2) 35 32
57 61
97 97
26
Perc
ent
of
pri
ori
ty
del
ayed
issu
es
Surprisingly, priority and severity have little influence on our models
0
20
40
60
80
100
120
ArgoUML Eclipse Firefox
Priority 1 (P1)
Priority 2 (P2) 35 32
57 61
97 97
26
Perc
ent
of
pri
ori
ty
del
ayed
issu
es
Surprisingly, priority and severity have little influence on our models
0
20
40
60
80
100
120
ArgoUML Eclipse Firefox
Priority 1 (P1)
Priority 2 (P2) 35 32
57 61
97 97
26
Perc
ent
of
pri
ori
ty
del
ayed
issu
es
Surprisingly, priority and severity have little influence on our models
0
20
40
60
80
100
120
ArgoUML Eclipse Firefox
Priority 1 (P1)
Priority 2 (P2) 35 32
57 61
97 97
26
Perc
ent
of
pri
ori
ty
del
ayed
issu
es
27
Integration delay
Prediction models
Influential variables
Issues are adressed well before a release
Issues are usually delayed in rapid releasing cycle
Our models perform better than random prediction
27
Integration delay
Prediction models
Influential variables
Issues are adressed well before a release
Issues are usually delayed in rapid releasing cycle
Our models perform better than random prediction
Integration Workload is the most influential variable in our models
So when does this stuff get added? Will it be applied to the next FF23 beta? A 22.01 release? Otherwise?
#883554
But an addressed issue may take some time to be integrated into a release
3,121 issues 17 releases
An empirical study to analyze the integration delay
14,530 issues 15 releases
3,344 issues (JDT)
11 releases
3,121 issues 17 releases
20,995 issues
Rapid Traditional Traditional
Integration delay
Prediction models
Influential variables
Issues are adressed well before a release
Issues are usually delayed in rapid releasing cycle
Our models perform better than random prediction
Integration Workload is the most influential variable in our models
An Empirical Study of Delays In The Integration of Addressed Issues
Daniel A. Costa Shane McIntosh Uirá Kulesza Ahmed E. Hassan Surafel L. Abebe