Alignment based Precision Checking
22
Alignment-based Precision Checking A. Adriansyah 1 , J. Munoz Gamma 2 , J. Carmona 2 , B.F. van Dongen 1 , W.M.P. van der Aalst 1 Tallinn, 3 September 2012 1) Department of Mathematics and Computer Science, Eindhoven University of Technology, The Netherlands 2) Software Department, Universitat Politècnica de Catalunya, Spain
-
Upload
arya-adriansyah -
Category
Documents
-
view
983 -
download
0
description
This slide tells you how to measure precision between process model and its recorded execution. Presented at BPI 2012, Tallinn, Estonia, by Arya Adriansyah. The technique is implemented in ProM 6.x, package ETConformance (see http://processmining.org).
Transcript of Alignment based Precision Checking
Alignment-based
Precision CheckingA. Adriansyah1, J. Munoz Gamma2,
J. Carmona2, B.F. van Dongen1,
W.M.P. van der Aalst1
Tallinn, 3 September 2012
1) Department of Mathematics and Computer Science,
Eindhoven University of Technology, The Netherlands
2) Software Department, Universitat Politècnica de
Catalunya, Spain
Model needs to be precise
1
Event Log
Process Models
a d
a b
b
cend
c
a b
d
end
aabd
abd
acd
aabd
c
a d
2
Event Log
Process Modelsc
a b
d
end
aabd
abd
acd
aabd
c
ad
Unfitting
d
a d
a b
b
cend
a
b
c
b
a
d
14 4
2
d
1
1
1d
2 2
0
0
b
c
0
d0
d
c0…
…
…
…
0a
0c
…
Prefix automaton
Reference: J. Muñoz-Gama, J. Carmona (2010). A
Fresh Look at Precision in Process Conformance.
8th International Conference of Business Process
Management (BPM): 211-226.
How to measure precision for unfitting
event logs?
Overview
3
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabd
aabd
acd
…
Fitting TracesAlignment
Precision
Metrics
Optimal alignment: least #deviations
4
a d
a b
b
cend
Trace adab
aAlignment
d b
a a b
a
d
Process Model
aabd Reference: W.M.P. van der Aalst, A.
Adriansyah, B.F. van Dongen (2012).
Replaying history on process models
for conformance checking and
performance analysis. WIREs Data
Mining and Knowledge
Discovery, 2(2), 182-192.
Optimal alignments can be many
5
a d
a b
b
cend
Trace ad
Alignment 1 Alignment 2
Process Model
ba
a d
d ca
a d
d
Both alignments are optimal
abd or acd
Overview
6
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments Precision
Metrics
aabd
aabd
acd
…
Fitting TracesAlignment
a22 3
db1.51.5 11
b1a
d
3
All-Alignment Automaton (Prefix-based)
7
c
Fitting Traces
a d
a b
b
cend
Process Model
All-Alignments Automaton
aabd
abd
abd or acd
aabd
1 1
0.5d
0.5
4 4
2 2 2
a
b
b
a
d
24 4
2 2 2
d2
aabd
abd
abd or acd
aabd
1-Alignment automaton (Prefix-based)
8
Fitting Traces
a d
a b
b
cend
Process Model
1-Align Automaton
aabd
abd
abd
aabd
b d
b
a
ad
4 4
2 2 2
1-Alignment automaton (Prefix-based)
9
0c
Fitting Traces
a d
a b
b
cend
Process Model
1-Align Automaton
aabd
abd
abd
aabd
Imprecision
2 2
1-Alignment
Comparison of log automata
10
aabd
abd
ad
adab
Event Log
a da b
bc
end
Process Model
Without alignment
All-Alignments
Pre
fix-b
ased
Lo
g A
uto
mata
aabd
abd
abd/acd
aabd
Fitting Traces
Overview
11
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabd
aabd
acd
…
Fitting TracesAlignment
Precision
Metrics
Reference: J. Muñoz-Gama, J. Carmona (2010). A Fresh Look at Precision in
Process Conformance. 8th International Conference of Business Process
Management (BPM): 211-226.
Overview
12
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabd
aabd
acd
…
Fitting Traces
Precision
Metrics
Alignment
Implementation in ProM 6
• Package: “ETConformance”
13
Experiment settings
14
Event Log
Flower (F)
Precise (P)
Flower’ (F’)
Precise’ (P’)
Duplicate Log Combined Log
Precise-Precise (PP’)
Precise-Flower
(PF’)
Flower -Precise
(FP’)
Flower-Flower
(FF’)
Experiment 1: Measuring precision stability
15
Perfectly fitting logs and models Non-fitting logs and models
P FP’ PF’ PP’P FP’ PF’ PP’
Experiment 2: Sensitivity to unfitting trace
•
16
Model P
0.80
0.85
0.90
0.95
1.00
0 1 2 3 4
0.00
0.20
0.40
0.60
0.80
1.00
0 1 2 3 4
Pre
cisi
on
Number of Removed Events
Noise Sensitivity - P
ETC
1-Align
All-Align
Pre
cis
ion
#Removed events
0.00
0.20
0.40
0.60
0.80
1.00
0 1 2 3 4
Model PF’
Pre
cis
ion
#Removed events
0.00
0.10
0.20
0.30
0.40
0 1 2 3 4
Pre
cis
ion
#Removed events
Model FP’
0.90
0.93
0.95
0.98
1.00
0 1 2 3 4
Pre
cis
ion
#Removed events
Model PP’
Experiment 3: Real life cases
•
17
Overview
18
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabd
aabd
acd
…
Fitting Traces
Precision
Metrics
Alignment
Conclusion and Future Work
•
19
Questions
20
Thanks!
21
