Scott Chapman – American Electric Power Paper 9015 Session 331.
-
Upload
ruby-mcnulty -
Category
Documents
-
view
216 -
download
0
Transcript of Scott Chapman – American Electric Power Paper 9015 Session 331.
![Page 1: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/1.jpg)
Scott Chapman – American Electric Power
Paper 9015Session 331
![Page 2: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/2.jpg)
Agenda
What I mean by “critical path” My simple way of finding it Review some sample code Questions Bonus Material
![Page 3: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/3.jpg)
Geek-required xkcd reference
HTTP://XKCD.COM/399/
![Page 4: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/4.jpg)
Critical path
Simple definition: how long is this going to take? Longest sequence of activities
In a project In a batch schedule
Need to look at: Predecessor-successor relationships Durations Time dependencies
Ideas originated in Project Management
![Page 5: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/5.jpg)
Project Management History Critical Path Method (CPM) originated at
DuPont in 1950s Used to manage chemical plant
maintenance projects Critical path is the sequence of events which
determines the duration of the project Delays in tasks on the CP delay the entire
project CP must be managed to stay on schedule To finish project earlier, tasks on CP must be
somehow shortened
![Page 6: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/6.jpg)
CPM process
Identify activities Identify sequence and dependencies Draw network diagram of activities Estimate duration of activities Identify the longest path in the network
(critical path) Monitor & update as project progresses
Delays in tasks not on CP may change the CP!
![Page 7: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/7.jpg)
Batch Windows & CP Business runs on cycles
Daily, weekly, monthly processes Large applications have large batch
schedules Batch schedules can be drawn as a
network diagram Predecessor – Successor
relationships between jobs Like projects, we like our batch
windows to finish on time! Like a project, a batch schedule has a
CP
![Page 8: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/8.jpg)
CP Calculation Formally:
CP is path with no slack for any task Slack = difference between earliest & latest
start or finish time of task Latest finish = latest time task can finish
without delaying project And how do you figure that???
Informally: Find all the paths through the network
diagram Add up task duration on each path Select the longest path
![Page 9: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/9.jpg)
Simple example
Five paths through this simple example
Task A
Task D
Task B
Task C
Task E
Task F
Start
End
![Page 10: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/10.jpg)
Simple example
Five paths through this simple example
Task A
Task D
Task B
Task C
Task E
Task F
Start
End
![Page 11: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/11.jpg)
Simple example
Five paths through this simple example
Task A
Task D
Task B
Task C
Task E
Task F
Start
End
![Page 12: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/12.jpg)
Simple example
Five paths through this simple example
Task A
Task D
Task B
Task C
Task E
Task F
Start
End
![Page 13: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/13.jpg)
Simple example
Five paths through this simple example
Task A
Task D
Task B
Task C
Task E
Task F
Start
End
![Page 14: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/14.jpg)
Simple example
Five paths through this simple example
Task A
Task D
Task B
Task C
Task E
Task F
Start
End
![Page 15: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/15.jpg)
Simple example
Calculate path durations from task durations
Task A
Task D
Task B
Task C
Task E
Task F
Start
End15 mins
30 mins
20 mins
15 mins
10 mins
5 mins
A+B+E = 45A+C+E = 40A+C+F = 35D+C+E = 55D+C+F = 50
![Page 16: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/16.jpg)
Simple example
Calculate path durations from task durations
Task A
Task D
Task B
Task C
Task E
Task F
Start
End15 mins
30 mins
20 mins
15 mins
10 mins
5 mins
A+B+E = 45A+C+E = 40A+C+F = 35D+C+E = 55D+C+F = 50
![Page 17: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/17.jpg)
So that’s simple enough!
Everybody ready to leave?
![Page 18: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/18.jpg)
Real world complications Hundreds or thousands of batch jobs
Managed by a batch scheduler package
Time-of-day dependencies Extraneous dependencies
New jobs added without cleaning up obsolete dependencies
Variable execution times Variation in data to be processed Contention with other processes External waits Job failures
![Page 19: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/19.jpg)
What that might look like…
Winding your way through that mess is a bit more complicated!
![Page 20: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/20.jpg)
Tooling Options• Package from scheduler vendor
+ Should be well integrated
- Cost?
• Microsoft Project- Not really meant for this purpose
+ See CMG Proceedings: Schwarz/Aurand, 1999 and Zaslavsky, 2001
• SAS/OR- Cost and effort?
• Roll your own+ Can make output exactly what you want
- Time / effort
+ Sample code on your CD!
![Page 21: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/21.jpg)
What is the real question?1. What is the longest path through
the schedule?- prediction of the critical path- usually one-time analysis
2. Why did job X finish late last night?- an ongoing question / process- requires the CP for job X
Fortunately, #2 is much easier!
![Page 22: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/22.jpg)
Use what you know
Predecessors (from batch scheduler)
End times (from actual executions)
We are answering a question, not predicting the future
We just need to explain what happenedLook at the jobs in the critical path for job
X for anomalies
![Page 23: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/23.jpg)
Critical path simplified Start at job X Find the predecessor that ended last
– that was the critical predecessor to X Call that job W
Find last predecessor of W, call it V Repeat until:
You go back some number of levels or You reach a time dependency
Resulting list is the critical path, for the day under study, for job X
![Page 24: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/24.jpg)
Simple example
Task A
Task D
Task B
Task C
Task E
Task F
Start19:00
End15 mins
30 mins
20 mins
15 mins
10 mins
5 mins
19:15
19:30
19:35
19:45
19:55
19:50
![Page 25: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/25.jpg)
Working backwards…
Task A
Task D
Task B
Task C
Task E
Task F
Start19:00
End19:15
19:30
19:35
19:45
19:55
19:50
E ended last
![Page 26: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/26.jpg)
What is E’s last predecessor?
Task A
Task D
Task B
Task C
Task E
Task F
Start19:00
End19:15
19:30
19:35
19:45
19:55
19:50
E ended lastC ended after B
![Page 27: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/27.jpg)
What is C’s last predecessor?
Task A
Task D
Task B
Task C
Task E
Task F
Start19:00
End19:15
19:30
19:35
19:45
19:55
19:50
E ended lastC ended after BD ended after A
![Page 28: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/28.jpg)
Critical Path is E – C – D
Task A
Task D
Task B
Task C
Task E
Task F
Start19:00
End19:15
19:30
19:35
19:45
19:55
19:50
Tasks A, B, F had no direct bearing on the end time
![Page 29: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/29.jpg)
Complicating simplicity… Schedule changes every day
Weekly / monthly processing Application changes
Schedule relationships may not be pristine
Jobs may be run multiple times—be sure to use the correct instance
If you want to graph the entire schedule it gets more complicated
![Page 30: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/30.jpg)
Why bother finding the CP? Limit the data you need to look at to
investigate a late-finishing job The cause is on the CP
Find changes If the CP changes day to day:
why? Investigate impact of periodic
schedule differences Addition of monthly processing
jobs may change the CP
![Page 31: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/31.jpg)
What I do Capture job stats every day to a
performance database Standard practice Extract history daily to XML file
Extract schedule once per day and store for 45 days Saved as XML files Allows historical investigation
JavaScript browser application pulls both data sources and allows for investigation
![Page 32: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/32.jpg)
Sample application Input #1: Schedule XML file<?xml version='1.0'?><?xml-stylesheet type='text/xsl'?><opc><app id='#AMCSMISCBILL'> <op id='87' job='#AMCS331' arr='1930'><wkstn>CPUJ</wkstn> <desc>Online Bill Image xtract</desc> <pred><aid>#AMCSMISCBILL</aid><opid>81</opid></pred>… <pred><aid>#AMCSMISCBILL</aid><opid>78</opid></pred> <succ><aid>#SMCSDAILYRPTS</aid><opid>3</opid></succ>… <succ><aid>#AMCSDLYBKUP</aid><opid>6</opid></succ> </op> <op id='90' job='#AMX1358' arr='1930'><wkstn>CPUJ</wkstn> <desc>Load O/L Bill Image</desc> <pred><aid>#AMCSMISCBILL</aid><opid>87</opid></pred> <succ><aid>#AMCSMISCBILL</aid><opid>91</opid></succ> </op></app>
A grouping of jobs is an
“application”
A job is an “operation”
Each job has predecessors
and successors
Arrival time is the earliest the
job can run
![Page 33: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/33.jpg)
Sample application Input #2: Job Data XML file
<job id="#AMCS331"> <sys>COCJ</sys><cls>0</cls><desc>MCSX4000</desc><cnt>24</cnt> <acpu>2.18</acpu><aet>13.8</aet><mcpu>2.70</mcpu><met>26.7</met> <run i="2009-05-09 2:01:55" rse=" 1:59:46, 2:01:56, 2:02:09"> <et>13.7</et><cp>2.16</cp><io>21632</io> </run> <run i="2009-05-08 0:51:46" rse=" 0:51:45, 0:51:48, 0:52:02"> <et>16.5</et><cp>2.11</cp><io>21690</io> </run> <run i="2009-05-07 1:24:41" rse=" 1:22:04, 1:24:41, 1:24:52"> <et>11.4</et><cp>2.12</cp><io>21740</io><sys>CO1J</sys> </run></job>
Jobs by name here
Norms, averages, maximums
May have multiple runs per
day
Read, start, end times
Note relatively compact format designed to reduce the file size—unfortunately that increases the complexity of interpreting the data
Stats for single run
![Page 34: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/34.jpg)
Sample application output
![Page 35: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/35.jpg)
Compare days in two windows
![Page 36: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/36.jpg)
Why later on 12/29?
![Page 37: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/37.jpg)
Completely different CP here
![Page 38: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/38.jpg)
Critical path comes back together here
![Page 39: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/39.jpg)
Here’s an ET difference of >1 hour and CPU >2x increase!
![Page 40: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/40.jpg)
Sample application details On CD at back of room Very simple example coded quickly one
Sunday afternoon May not be bug free Will not satisfy all your needs For illustrative purposes only – easier to
understand than the examples in the paper HTML / JavaScript application Data is in XML files Internet Explorer only Uses XPath & XSLT
Beyond the scope of this presentation
![Page 41: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/41.jpg)
Application flow When HTML page loads
Calls init() to load the XML files Selection criteria populated in HTML
When user clicks “Find It” button Calls findIt() to find the critical path,
which in turn calls: getRunsDate(job, date) – returns array of
executions of a job on a given batch date getLatestPred(job, app) – returns
predecessor that ran last getRunsDate is an example of using DOM functions to extract data from XML
getRunsDate is an example of using DOM functions to extract data from XML
![Page 42: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/42.jpg)
Critical parts of the code
![Page 43: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/43.jpg)
findIt() Get runs for this job and date
Save stats for this job to array
Then find preds and add them to
the array
Loop through array and build
HTML table
![Page 44: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/44.jpg)
getLatestPred(cjob, capp)XPath to get
array of preds
Use DOM calls to get data from
XML
Xpath to find job name from op id
Get the runs for a pred job
Check run to see if it is the latest
pred so far
If it is, save it
![Page 45: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/45.jpg)
That’s essentially it!
getRunsDate(job, date) is nothing special – simply retrieves list of runs from the XML file
Typical housekeeping code initializing variables, etc.
Sample in the paper was much more complicated due to it being pulled from the application that does the graphing
![Page 46: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/46.jpg)
Summary
Critical Path Analysis for Performance Analysis usually involves answering why, not predicting the future
In such a case, start at the end and work backwards
That type of analysis is easy to code Record a snapshot of the schedule
daily as well as the job performance
![Page 47: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/47.jpg)
Questions / comments ?
![Page 48: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/48.jpg)
Bonus material: Export to Excel! Sometimes you need to play with
your data Copy to Excel to
Re-sort Filter Pivot tables Graph Summarize
Cut and paste HTML tables works well
Even better: automate it
![Page 49: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/49.jpg)
New & improved:
![Page 50: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/50.jpg)
![Page 51: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/51.jpg)
sendExcel() function on CD
If Excel not open, open it If previous workbook not open
Open a new workbook Add new sheet to work book with
data
Requires IE and Excel
![Page 52: Scott Chapman – American Electric Power Paper 9015 Session 331.](https://reader036.fdocuments.net/reader036/viewer/2022070307/551a3f2d5503463e778b4e97/html5/thumbnails/52.jpg)
Useful references
XPath, XSLT, XML quick reference cards http://www.mulberrytech.com/quickref/index.html
Browser Book for Web Designers http://www.visibone.com/products/browserbook.html
Humor for geeks http://xkcd.com/