Post on 04-Jul-2015
description
Alignment-based
Precision CheckingA. Adriansyah1, J. Munoz Gamma2,
J. Carmona2, B.F. van Dongen1,
W.M.P. van der Aalst1
Tallinn, 3 September 2012
1) Department of Mathematics and Computer Science,
Eindhoven University of Technology, The Netherlands
2) Software Department, Universitat Politècnica de
Catalunya, Spain
Model needs to be precise
1
Event Log
Process Models
a d
a b
b
cend
c
a b
d
end
aabd
abd
acd
aabd
c
a d
2
Event Log
Process Modelsc
a b
d
end
aabd
abd
acd
aabd
c
ad
Unfitting
d
a d
a b
b
cend
a
b
c
b
a
d
14 4
2
d
1
1
1d
2 2
0
0
b
c
0
d0
d
c0…
…
…
…
0a
0c
…
Prefix automaton
Reference: J. Muñoz-Gama, J. Carmona (2010). A
Fresh Look at Precision in Process Conformance.
8th International Conference of Business Process
Management (BPM): 211-226.
How to measure precision for unfitting
event logs?
Overview
3
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabd
aabd
acd
…
Fitting TracesAlignment
Precision
Metrics
Optimal alignment: least #deviations
4
a d
a b
b
cend
Trace adab
aAlignment
d b
a a b
a
d
Process Model
aabd Reference: W.M.P. van der Aalst, A.
Adriansyah, B.F. van Dongen (2012).
Replaying history on process models
for conformance checking and
performance analysis. WIREs Data
Mining and Knowledge
Discovery, 2(2), 182-192.
Optimal alignments can be many
5
a d
a b
b
cend
Trace ad
Alignment 1 Alignment 2
Process Model
ba
a d
d ca
a d
d
Both alignments are optimal
abd or acd
Overview
6
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments Precision
Metrics
aabd
aabd
acd
…
Fitting TracesAlignment
a22 3
db1.51.5 11
b1a
d
3
All-Alignment Automaton (Prefix-based)
7
c
Fitting Traces
a d
a b
b
cend
Process Model
All-Alignments Automaton
aabd
abd
abd or acd
aabd
1 1
0.5d
0.5
4 4
2 2 2
a
b
b
a
d
24 4
2 2 2
d2
aabd
abd
abd or acd
aabd
1-Alignment automaton (Prefix-based)
8
Fitting Traces
a d
a b
b
cend
Process Model
1-Align Automaton
aabd
abd
abd
aabd
b d
b
a
ad
4 4
2 2 2
1-Alignment automaton (Prefix-based)
9
0c
Fitting Traces
a d
a b
b
cend
Process Model
1-Align Automaton
aabd
abd
abd
aabd
Imprecision
2 2
1-Alignment
Comparison of log automata
10
aabd
abd
ad
adab
Event Log
a da b
bc
end
Process Model
Without alignment
All-Alignments
Pre
fix-b
ased
Lo
g A
uto
mata
aabd
abd
abd/acd
aabd
Fitting Traces
Overview
11
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabd
aabd
acd
…
Fitting TracesAlignment
Precision
Metrics
Reference: J. Muñoz-Gama, J. Carmona (2010). A Fresh Look at Precision in
Process Conformance. 8th International Conference of Business Process
Management (BPM): 211-226.
Overview
12
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabd
aabd
acd
…
Fitting Traces
Precision
Metrics
Alignment
Implementation in ProM 6
• Package: “ETConformance”
13
Experiment settings
14
Event Log
Flower (F)
Precise (P)
Flower’ (F’)
Precise’ (P’)
Duplicate Log Combined Log
Precise-Precise (PP’)
Precise-Flower
(PF’)
Flower -Precise
(FP’)
Flower-Flower
(FF’)
Experiment 1: Measuring precision stability
15
Perfectly fitting logs and models Non-fitting logs and models
P FP’ PF’ PP’P FP’ PF’ PP’
Experiment 2: Sensitivity to unfitting trace
•
16
Model P
0.80
0.85
0.90
0.95
1.00
0 1 2 3 4
0.00
0.20
0.40
0.60
0.80
1.00
0 1 2 3 4
Pre
cisi
on
Number of Removed Events
Noise Sensitivity - P
ETC
1-Align
All-Align
Pre
cis
ion
#Removed events
0.00
0.20
0.40
0.60
0.80
1.00
0 1 2 3 4
Model PF’
Pre
cis
ion
#Removed events
0.00
0.10
0.20
0.30
0.40
0 1 2 3 4
Pre
cis
ion
#Removed events
Model FP’
0.90
0.93
0.95
0.98
1.00
0 1 2 3 4
Pre
cis
ion
#Removed events
Model PP’
Experiment 3: Real life cases
•
17
Overview
18
aabbd
adabd
a d
…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabd
aabd
acd
…
Fitting Traces
Precision
Metrics
Alignment
Conclusion and Future Work
•
19
Questions
20
Thanks!
21