Download - Alignment based Precision Checking

Transcript
Page 1: Alignment based Precision Checking

Alignment-based

Precision CheckingA. Adriansyah1, J. Munoz Gamma2,

J. Carmona2, B.F. van Dongen1,

W.M.P. van der Aalst1

Tallinn, 3 September 2012

1) Department of Mathematics and Computer Science,

Eindhoven University of Technology, The Netherlands

2) Software Department, Universitat Politècnica de

Catalunya, Spain

Page 2: Alignment based Precision Checking

Model needs to be precise

1

Event Log

Process Models

a d

a b

b

cend

c

a b

d

end

aabd

abd

acd

aabd

c

a d

Page 3: Alignment based Precision Checking

2

Event Log

Process Modelsc

a b

d

end

aabd

abd

acd

aabd

c

ad

Unfitting

d

a d

a b

b

cend

a

b

c

b

a

d

14 4

2

d

1

1

1d

2 2

0

0

b

c

0

d0

d

c0…

0a

0c

Prefix automaton

Reference: J. Muñoz-Gama, J. Carmona (2010). A

Fresh Look at Precision in Process Conformance.

8th International Conference of Business Process

Management (BPM): 211-226.

How to measure precision for unfitting

event logs?

Page 4: Alignment based Precision Checking

Overview

3

aabbd

adabd

a d

Unfitting Log

Process Model

Alignment Automaton

Experiments

aabd

aabd

acd

Fitting TracesAlignment

Precision

Metrics

Page 5: Alignment based Precision Checking

Optimal alignment: least #deviations

4

a d

a b

b

cend

Trace adab

aAlignment

d b

a a b

a

d

Process Model

aabd Reference: W.M.P. van der Aalst, A.

Adriansyah, B.F. van Dongen (2012).

Replaying history on process models

for conformance checking and

performance analysis. WIREs Data

Mining and Knowledge

Discovery, 2(2), 182-192.

Page 6: Alignment based Precision Checking

Optimal alignments can be many

5

a d

a b

b

cend

Trace ad

Alignment 1 Alignment 2

Process Model

ba

a d

d ca

a d

d

Both alignments are optimal

abd or acd

Page 7: Alignment based Precision Checking

Overview

6

aabbd

adabd

a d

Unfitting Log

Process Model

Alignment Automaton

Experiments Precision

Metrics

aabd

aabd

acd

Fitting TracesAlignment

Page 8: Alignment based Precision Checking

a22 3

db1.51.5 11

b1a

d

3

All-Alignment Automaton (Prefix-based)

7

c

Fitting Traces

a d

a b

b

cend

Process Model

All-Alignments Automaton

aabd

abd

abd or acd

aabd

1 1

0.5d

0.5

4 4

2 2 2

Page 9: Alignment based Precision Checking

a

b

b

a

d

24 4

2 2 2

d2

aabd

abd

abd or acd

aabd

1-Alignment automaton (Prefix-based)

8

Fitting Traces

a d

a b

b

cend

Process Model

1-Align Automaton

aabd

abd

abd

aabd

Page 10: Alignment based Precision Checking

b d

b

a

ad

4 4

2 2 2

1-Alignment automaton (Prefix-based)

9

0c

Fitting Traces

a d

a b

b

cend

Process Model

1-Align Automaton

aabd

abd

abd

aabd

Imprecision

2 2

Page 11: Alignment based Precision Checking

1-Alignment

Comparison of log automata

10

aabd

abd

ad

adab

Event Log

a da b

bc

end

Process Model

Without alignment

All-Alignments

Pre

fix-b

ased

Lo

g A

uto

mata

aabd

abd

abd/acd

aabd

Fitting Traces

Page 12: Alignment based Precision Checking

Overview

11

aabbd

adabd

a d

Unfitting Log

Process Model

Alignment Automaton

Experiments

aabd

aabd

acd

Fitting TracesAlignment

Precision

Metrics

Reference: J. Muñoz-Gama, J. Carmona (2010). A Fresh Look at Precision in

Process Conformance. 8th International Conference of Business Process

Management (BPM): 211-226.

Page 13: Alignment based Precision Checking

Overview

12

aabbd

adabd

a d

Unfitting Log

Process Model

Alignment Automaton

Experiments

aabd

aabd

acd

Fitting Traces

Precision

Metrics

Alignment

Page 14: Alignment based Precision Checking

Implementation in ProM 6

• Package: “ETConformance”

13

Page 15: Alignment based Precision Checking

Experiment settings

14

Event Log

Flower (F)

Precise (P)

Flower’ (F’)

Precise’ (P’)

Duplicate Log Combined Log

Precise-Precise (PP’)

Precise-Flower

(PF’)

Flower -Precise

(FP’)

Flower-Flower

(FF’)

Page 16: Alignment based Precision Checking

Experiment 1: Measuring precision stability

15

Perfectly fitting logs and models Non-fitting logs and models

P FP’ PF’ PP’P FP’ PF’ PP’

Page 17: Alignment based Precision Checking

Experiment 2: Sensitivity to unfitting trace

16

Model P

0.80

0.85

0.90

0.95

1.00

0 1 2 3 4

0.00

0.20

0.40

0.60

0.80

1.00

0 1 2 3 4

Pre

cisi

on

Number of Removed Events

Noise Sensitivity - P

ETC

1-Align

All-Align

Pre

cis

ion

#Removed events

0.00

0.20

0.40

0.60

0.80

1.00

0 1 2 3 4

Model PF’

Pre

cis

ion

#Removed events

0.00

0.10

0.20

0.30

0.40

0 1 2 3 4

Pre

cis

ion

#Removed events

Model FP’

0.90

0.93

0.95

0.98

1.00

0 1 2 3 4

Pre

cis

ion

#Removed events

Model PP’

Page 18: Alignment based Precision Checking

Experiment 3: Real life cases

17

Page 19: Alignment based Precision Checking

Overview

18

aabbd

adabd

a d

Unfitting Log

Process Model

Alignment Automaton

Experiments

aabd

aabd

acd

Fitting Traces

Precision

Metrics

Alignment

Page 20: Alignment based Precision Checking

Conclusion and Future Work

19

Page 21: Alignment based Precision Checking

Questions

20

Page 22: Alignment based Precision Checking

Thanks!

21