Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil...

18
Predictive Performance Monitoring of Material Handling Systems Using the Performance Spectrum Vadim Denisov, Dirk Fahland, Wil van der Aalst

Transcript of Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil...

Page 1: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Predictive Performance Monitoring of Material Handling Systems

Using the Performance Spectrum

Vadim Denisov, Dirk Fahland, Wil van der Aalst

Page 2: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Baggage Handling Systems

1

Check-InX-

Ray

Screening &identification

Finalsorting

Happy path

Page 3: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Re-Circulation Causes Significant Delays

2

Check-InX-

Ray

Screening &identification

Finalsorting

Longer path

Page 4: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Insufficient Number of Screening Machine Operators

3

Check-InX-

Ray

Screening &identification

Finalsorting

Page 5: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Unavailability of Outgoing Conveyors

4

Check-InX-

Ray

Screening &identification

Finalsorting

Page 6: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Predictive Performance Monitoring of Material Handling Processes

5

Check-InX-

Ray

Screening &identification

Finalsorting

System Behavior

Predictive Model

System Operators

Op.number,Re-circulation

Action

What is the required number of

operators in +x minutes from now?

What is the amount of re-

circulation in +x minutes from now?

Page 7: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

What Are Relevant Features?

6

• Bag color: red

• Final destination: Amsterdam

• Dimensions: 550x450x250mm

• Weight: 15kg6

• Utilization of the Loop Sorter: 90%

• Utilization of conveyors around the Sorter: 60%

• Current load on the X-Ray Screening Machine: 15 bags/min.

• Number of bags in the system: 8000

Intra-case features Inter-case features

What is the load on the screening machines? What is the amount of re-circulation in +x minutes from now?

One bag = one case

Bag properties System properties

Page 8: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Performance Spectrum [1]

7

a

b

Sub-trace: (ei(a, ta), ei+1

(b, tb))

Bins:• cases started• cases stopped• cases pending

normal speed

2 times slower

3 times slower

very slow

Time

a

b

ta tb

Step

Step

[1] V. Denisov, D. Fahland, and W. M. P. van der Aalst, “Unbiased, finegrained description of processes performance from event data,” in Business Process Management, M. Weske, M. Montali, I. Weber, and J. vomBrocke, Eds. Cham: Springer International Publishing, 2018, pp. 139–157.

Page 9: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Development of Problem Instance 2: Re-Circulation because of Unavailability of a1-a3

9

Check-InX-

Ray

Screening &identification

a1

a2

a3

b1

b2

b3

c1

c2

c3

loop

normal speed 2 times slower 3 times slower very slow

X-Ray:a2

a2:b2

b2:c2

a3:b3

b3:c3

X-Ray:a1

a1:b1

a1:loop

Severity

Time

Page 10: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Extracting Features from Performance Spectrum

10

Historic spectrum

Target spectrum

NowPrediction

horizon

Time

X-Ray:a2

a2:b2

b2:c2

a3:b3

b3:c3

X-Ray:a1

a1:b1

a1:loop

8(Y)

2(LB)

5(DB)

vij= [8, 2, 5]

Page 11: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Multi-Channel Performance Spectrum

11

NowPrediction

horizon

Time

X-Ray:a2

a2:b2

b2:c2

a3:b3

b3:c3

X-Ray:a1

a1:b1

a1:loop

Channel 1: SpeedChannel 2: Screening resultChannel 3: Sortation status

X-Ray:a2X-Ray:a2

Multi-Channel Performance Spectrum

Page 12: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Multi-Channel Performance Spectrum

12

Channel 1: SpeedChannel 2: Screening resultChannel 3: Sortation status

vns=<v1,…,vk> vn

e=<v1,…,vk>… …

v1s=<v1,…,vk> v1

e=<v1,…,vk>Se

gmen

ts

Binss e

s ns 1

………

2

1

3

In the paper: Generic definition of the Multi-Channel Performance

Spectrum Generic method for Performance Spectrum -based

feature extraction

Page 13: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Training& test sets

Model training

ML Model

13

Flatten

Sliding Window over Performance Spectrum

Page 14: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Problem Instances and Datasets

14

Check-InX-

Ray

Screening &identification

a1

a3

b1

b2

b3

c1

c2

c3

loop

Problem Instance 1Load on the X-Ray Screening Machines

Problem Instance 2Extra re-circulation because of unavailability a1-a3

Property Simulation model [2]

Time period 7 daysActivities 25Cases per day 1600Events per day 19.000Events, total 134.000

[2] https://github.com/processmining-in-logistics/psm/releases/tag/1.1.6

a2Real BHS of a major European airport

4 month85025.000-50.0001.000.000-2.000.000148.000.000

Page 15: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Implementation: Performance Spectrum Miner [3] and Scripts on Top of PyTorch

15

[3] Denisov, V., Belkina, E., Fahland, D., van der Aalst, W.M.P.: The performance spectrum miner: Visual analytics for fine-grained performance analysis of processes. In: BPM 2018 Demos. CEUR Workshop Proceedings, vol. 2196, pp. 96–100. CEUR-WS.org (2018)

https://github.com/processmining-in-logistics/psm/tree/ppm

Computational environment:• a single-machine configuration• 40 CPUs• 6 GPUs• 400GB RAM• the Performance Spectrum Miner in ProM 6.9• the PyTorch Framework

http://www.promtools.org/doku.php?id=prom69

Page 16: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Models and Baselines

16

Baselines:• Naïve baseline: current load• Data-driven inter-case feature encoding (DDICFE) [4] [5]

[4] Data-driven inter-case feature encoding, A.Senderovich, C.D.Francescomarino, and F.M.Maggi, “From knowledge driven to data-driven inter-case feature encoding in predictive process monitoring,” Information Systems, 2019.[5] https://github.com/processmining-in-logistics/psm/tree/ppm/ppm/src/main/scala/org/processmining/scala/intercase

Models:• Logistic Regression (LR)• Feedforward Neural Network (FFNN)

Page 17: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Experimental Results

17

11%

Predicted spectrum

50%

Observed spectrum

Baseline spectrum

50%

50%

7%

-30-20-10

01020

Error (%)

A

A

A

B

B

C

C

C

D

E

E

30%Predicted spectrum

Observed spectrum

Baseline spectrum

30%

30%

Experiment Approach Model RMSE, % of maxSim. model PS LR 12.1

DDICFE FFNN 21.2Naïve baseline - 35.8

Real system PS LR 7.0DDICFE FFNN 7.6Naïve baseline - 11.0

Experiment Approach Model RMSE, % of maxReal system PS FFNN 2.4

Naïve baseline - 3.0

Problem Instance 1Load on the X-Ray Screening Machines

Problem Instance 2Extra re-circulation because of exits a1-a3 unavailability

Page 18: Vadim Denisov, Dirk Fahland, Wil van der Aalst Predictive … · Vadim Denisov, Dirk Fahland, Wil van der Aalst Baggage Handling Systems 1 Check-In X-Ray Screening & identification

Conclusion and Future Work

18

• Processes: cases influence each other• Performance Spectrum:

• modelling a variety of performance features over time• capturing inter-case dependencies

• Multi-Channel PS: formulating a large class of performance prediction problems as a regression problem

• Generic methodology of solving this problem as a ML task

• Feasibility on MHS apply to Business Processes• Feature selection: domain knowledge is required include a formal process model• Validation in practice is missing higher accuracy and longer prediction horizon

Thank you!