© DEEDS 2008Graph Algorithms1 Application: Process Relations Some algorithms require that sub...

52
© DEEDS 2008 Graph Algorithms 1 Application: Process Relations Some algorithms require that sub processes (tasks) are executed in a certain order, because they use data from each other Example: get dressed In which order should I don my clothes? Such an order is known as a topological sort Tee-shirt before shirt Briefs before trousers Socks before shoes Pullover before coat Trousers before shoes Shirt before pullover Trousers before coat

Transcript of © DEEDS 2008Graph Algorithms1 Application: Process Relations Some algorithms require that sub...

Page 1: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 1

Application: Process Relations

Some algorithms require that sub processes (tasks) are executed in a certain order, because they use data from each other

Example: get dressed

In which order should I don my clothes? Such an order is known as a topological sort

• Tee-shirt before shirt• Briefs before trousers• Socks before shoes• Pullover before coat• Trousers before shoes• Shirt before pullover• Trousers before coat

Page 2: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 2

Application: Process Dependencies

Are these the only rules?

Tee-shirt

Briefs

PulloverShirt

Trousers Shoes

Coat

Socks • Tee-shirt before shirt• Briefs before trousers• Socks before shoes• Pullover before coat• Trousers before shoes• Shirt before pullover• Trousers before coat

Page 3: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 3

Application: Process Relations

Transitivity

Tee-shirt

Briefs

PulloverShirt

Trousers Shoes

Coat

Socks Relation??

Page 4: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 4

Partial Order

A partial order is a relation that (informally) expresses:

For any two objects holds that either... The object relate with a “greater/smaller than”,

“before/after” type relation ...or the two objects are incomparable

(shirt or socks first?)

So what is so partial about the ordering?

Page 5: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 5

Acyclic Graphs and Partial Order

Definition: A partial Order (M, ) consists of a set M and a irreflexive, transitive, antisymmetric order relation ()

Cormen’s definition is reflexive, transitive, antisymmetric

In a partial order, some elements can be incomparable

Acyclic graphs specify partial orders : Let

G = (V,E) be an acyclic, directed graph G+ = (V,E+) be the corresponding transitive closure

The partial order ( V, G ) is defined as: vi G vj ( vi, vj ) E+

p. 1075

Page 6: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 6

Transitive Closure

There are unfortunately multiple definitions for the transitive closure… There is another definition, where G* is called the transitive closure

(Cormen for instance) G* = (V,E*), where E* = E+ { (v,v) | v V } G* sometimes also known as the reflexive transitive closure

We will use: G+ = transitive closure (Wegegraph in German)

For all definitions: use the one on the slides! Also for the exam!

Page 7: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 7

Maths…

Let be a relation (on for example )

Reflexive (=,≤,≥)

Irreflexive (<)

Antisymmetric (≤,≥)

Transitive (=,<, ≤,≥)

baabba

aa

cacbba

)( aa

Page 8: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 8

Example

E+ = {

(v1, v3), (v1, v4), (v1, v5), (v1, v6), (v1, v7),

(v2, v3), (v2, v4), (v2, v5), (v2, v6), (v2, v7),

(v3, v4), (v3, v5), (v3, v6), (v3, v7),

(v4, v6),

(v5, v6), (v5, v7)

}

Graph G

Page 9: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 9

Deadlock

Test: find cycles = test for partial order How do we do this?

Example (fictive :-)

Assume: “The coat should be put on before the shirt”

Tee-shirt

Briefs

PulloverShirt

Trousers Shoes

Coat

Socks

Page 10: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 10

Topological Sort

Topological sort order: a sequence of objects that adhere to a partial order

Example:

Tee-shirt Shirt Pullover Briefs Trousers Socks Shoes Coator

Briefs Tee-shirt Trousers Shirt Socks Shoes Pullover Coat

Tee-shirt

Briefs

PulloverShirt

Trousers Shoes

Coat

Socks

p. 549

Page 11: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 11

Topological Sort

Definition: The arrangement of elements mi of a partially ordered set (M,) in a sequence m1, m2, ..., mk is called topological sort if for any two indices i and j with 1 i < j k the following holds: Either mi mj or mi and mj are incomparable

Topological Sort is a “linearization” of elements, that is compatible with partial order

Visualized as a graph: Arrangement of vertices on a horizontal line so that all edges go from left to right

v1

v3

v4

v2 v5

v6

v7

v1 v2 v3v5 v4 v7 v6

Page 12: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 12

Finding of a Topological Sort

Finding a topological sort can be done using graph algorithms

Starting point: Determine the acyclic graph G that corresponds to (M,).

Two possible approaches: (top down): As long as G is not empty...

1. Find a vertex v with incoming degree 02. Delete v from G together with its corresponding edges3. Save the deleted vertices in this order

(bottom up): 1. Starting from the vertex with incoming degree 0, start a DFS traversal. 2. Assign reverse “visit numbers” beginning with the “deepest” vertex.

Page 13: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 13

Example: top down

1. Find a vertex with d-(v) = 0

2. Delete from G vertex v and all incident edges

3. Save the deleted vertices in this order

4. Repeat steps 1 to 3 until G is empty

Graph G

Possible topological sorts:

1, 2, 3, 4, 5, 6, 7 2, 1, 3, 5, 4, 6, 7 1, 2, 3, 4, 5, 7, 6

2, 1, 3, 5, 7, 4, 6 ...

1

3

2

4 5

6 7

Page 14: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 14

Cost: top down

In worst case you always find the next vertex with in-degree 0 last

O(|V|2)

Example:

v4 v3 v2 v1

Page 15: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 15

Example: bottom up (DFS)

Given: acyclic Graph G = ( V, E )

Find: topological sort TS[v] for all v V

for ( all v V ) { v.unvisit( ); } /* initialization */

z = |V|; /* maximum number in TS */

for ( all v V ) {if ( d-(v) == 0 ) v.topSort( ); /* all starting vertices */

}

void topSort( ) { /* DFS Traversal */

this.visit( );

for ( all k‘ this.neighbors( ) ) if ( k‘.unvisited( ) ) k‘.topsort();

TS[ v ] = z; /* Assignment of TS at the end*/

z--;

}

Page 16: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 16

Example

Vertices are named “backwards” while ascending from DFS recursion.

Sorting order: 2 1 3 5 7 4 6

7

6

3

5

4

12 1

3

2

4 5

6 7

Page 17: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 17

Cost: bottom up

Consists of Search for nodes with in-degree 0

O (|V|) DFS on the corresponding subgraphs

O (|V|+|E|)

Total cost O(max (|V|,|E|)

A bit more complicated, but “better” than top down

Page 18: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 18

Side kick: Total Order

There is not only partial order Total order is a partial order where all elements in the

set M are pair wise comparable For example: ‘≤’ and ‘≥’ But not: “is a child of” over all Germans

Where could these orders be useful? Deadlock detection in concurrent programs Message ordering and consistency in distributed computing

Page 19: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 19

Application: Cable Routing

The possible cable channels are defined by the channels

Goal: every computer should be reachable

Page 20: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 20

Solution: Spanning Tree

Model the problem as a graph

Create a spanning tree As long there is a cycle

left: Remove an edge from a cycle

1 32

45

6 7 8

Page 21: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 21

Spanning Trees

Definition: A subgraph B is a spanning tree of a graph G, if there are no cycles in B and B contains all vertices in G

It is not difficult to build a spanning tree of a connected graph: If G is acyclic, then G is a spanning tree itself. If G contains cycles, delete an edge from the cycle (the graph

stays connected) and repeat the procedure until there are no more cycles.

Ch. 23

Page 22: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 22

Spanning Trees (2)

Edges in G and B are called branches Edges in G and not in B are called chords A graph with n vertices and e edges has n-1 branches

and e - n + 1 chords The spanning tree of a graph is not unique All spanning trees of a graph can be systematically

constructed by means of an acyclic exchange: Find any spanning tree B of G Add a chord to B (that leads to a cycle in B) Delete another edge from B

Page 23: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 23

Example

B1, B2, B3 - all spanning trees of G

Chords (edges in G and not in B)Branches (edges in B)

Graph G

B1 B2B3

Page 24: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 24

Application: Cable Routing

The green numbers indicate the lengths of the cables

How do we find the cheapest (shortest) routing?

Page 25: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 25

Minimum Spanning Trees

We look at spanning trees of labeled graphs where edges are labeled with c.

Definition: The weight c of a spanning tree B = (V,E) is the sum of the weights of its branches, i.e.

c(B) = eE c(e) .

Definition: A spanning tree B = ( V, E ) is called the minimum spanning tree of graph G, if for any B‘ = ( V, E‘ ) of G the following holds:

c(B) c(B‘)

Page 26: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 26

Example

c(B1) = 12, c(B2) = 15, c(B3) = 16

B1 is minimum spanning tree of G

Chord

Branch

labeled graph G

5

3

7

4

B1 B2 B3

Page 27: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 27

Finding Minimum Spanning Trees

Idea: greedy-principle, i.e. decide on the basis of the knowledge currently available and never revise the decision.

Efficient in contrast to methods that sample solutions, evaluate and revise them if necessary.

Given: Graph G = ( V, E ) labeled with cFind: minimum spanning tree T‘ = (V, E‘) of G.

E‘ = { };while (not yet finished) {

choose an appropriate edge e E;E‘ = E‘ { e }

}

Page 28: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 28

Kruskal‘s Algorithm

Sampling of edges according to the principle of the lowest weight. Sort edges according to their weights. Choose the edge with the lowest weight. If E‘ { e } results in a cycle, drop e.

e

Observation: If there is a cycle, it is pointless to leave e in the tree and to drop another edge e‘, because c(e‘) < c(e).

e‘

Page 29: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 29

Example: Kruskal

Sorting:(1, 2)(3, 4)(2, 3)(3, 6)(4, 6)(1, 5)(5, 6)(5, 7)(4, 8)(7, 8)(2, 7)(1, 7)

1 32

45

6 7 8

5 6

5229

11

7

8

1414

19

16

Needed length: 60m

Page 30: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 30

Kruskal‘s Algorithm: Complexity

The complexity comes from Cost for sorting the edges

(possible with O(|E| log |E|)) Cost for testing for cycles

(also O(|E| log |E|))

Total: O(|E| log |E|)

Page 31: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 31

Prim‘s Algorithm

Enhancement of Kruskal‘s Algorithm by Prim(sometimes Dijkstra is declared as the author):

Given: G = ( V, E ) labeled with cFind: minimum spanning tree T = ( V, E‘ ) of G

V‘ = { arbitrary start vertex v V }for (int i = 1; i |V| - 1; i++ ) {

e = edge (u,v) E with u V‘ and v V‘ and minimum weight cV‘ = V‘ { v };E‘ = E‘ { e }

}

Page 32: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 32

Example

1 32

45

6 7 8

5 6

5229

11

7

8

1414

19

16

Page 33: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 33

Analysis

(Efficient) cycle testing is an integral part of Kruskal‘s algorithm: Interconnecting two different connected components of a graph

by an edge does not lead to a cycle The vertices belonging to the new edge must be located in

different components Upon the assembly, the smaller component is added to the larger

one Works with appropriate data structures in

O( |E| log |V| ) time

Page 34: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 34

Path Finding Problems

Finding certain paths in a graph Distinct boundary conditions lead to different problems:

(un)directed graph Type of weights (only positive or also negative weights, bounded

weights?) Shortest vs. longest path Path existence (Number of Paths) Start and end vertex:

• Paths between a specified pair of vertices• Paths between a start vertex and all others• Paths between every pair of vertices

Ch. 24

Page 35: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 35

Application: Route Planning

Page 36: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 36

Abstraction: Model as a Graph

Page 37: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 37

Refinement

Page 38: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 38

1. Approach: Brute force

Brute force: The simplest way Try all possible paths

Cost: normally exponential

Worst case: Complete graph n cities (n-1)! Different paths with (n-1) edges each

complexity O(nn!)

Page 39: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 39

1. Approach: Brute force

5 Nodes: 96 paths 10 Nodes : 3.265.920 paths 100 Nodes :

92392953289504711154882246467704033485808808581737805253907034256265423993297616452852049336394953103391160941618951520668673358807695360000000000000000000000 paths

All 12903 communities in Germany:

474381088,9

Page 40: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 40

988032780296517663436322076701095155700770024537890245495168073394410613749907903334908537127328540766793107809246348588724025167815276417017650390124486547249379172139679071211781763482703236042541349862318218136576551103758850603937190932371732941574515443140722211275244104835452316865803709346211824565400461322044268326721434035983838111178657891154164927556643178656171474660541364955846571135269566089067209050083842063966215125519481522183823585218692212743733202669715476272895547369627756913232675761878439204981953372613356827581804169522488746469163622604760967219252795485480375527583094186163733617471275178950304495062900336879944902153498925439763269150239710115653805503370058652307975224136372148701221581438236021839242848128851546825711228911202757947505784345873803895182007675155066858898693737648891744866579496575892260026300294887734157699863935792018007700633163234414762019639622776185890245821775621611386383897803543489896010396675700309744434876784840088465489343324134115523556680571964252563179124505009237822399060108114358568757406253755902528547047697975637947254426283182107816551488579830240507465718633120296507369068830031103226529254096912888355394582488701708312346659355342807395570422354578245246938061310209543661790640824248864515698258916783063862709184192639909885673222152979676570965226247673732187517713838924480328589155031996896876630274080083237499467671885548338229554799947277310507760197919131028896513288374654041642193935067734583293056077975165566070718798016703959586297899227493229109065658198379525446761936503813125987314217476912540920958691501786781633016193952420445818637943831285237097681218386907496815085128670158136918224743013470148417525928758794125736083444423449517158184585748244991925372752352883903196664446123035911353975285681385557607950025161830060127387144386770890182283129433279229878044001420477388322367001685489721693216961358741984895809910057530410237766907408298984317251170185725001911792002356449462337432259915811855680138431420900271721621227629171082975740974612055073676688305916188061569085712379422149608668571756702872714717887242081282688937644017036781926791886892850699160723701808477393242501975526406247528322685597081253628903644703870145482094042563403342726628505879061682285383030772566219038403276226525576463824761999416389357888909877646081196344931614459530305303529216037971446335469336304933789696590465986538192098788718867231866722821972411202608920215463998261375226177037293246241414617886441005846443741362574589796763866647308280944923725543427360293500292357816587680690786981644763480822493581171066514641833375350953690889267118124738752384665657758279740313428551789738503345388500169962568116376501762419896564889448468299648444141841066258146163041111965515673358814164304045545169389544640456926558980709540435412249915976314003081122455954626450715899677907566902565738992731601501551892143697443236936018991215182673945313156849126844651391733253422369214395206355345371819267478091898022034500176275236986899261341308560337891814734904904095606613036861273053573490027296196740110765950770284215762304864858219405620645776247712410673816673388898455767444875353302980656538334614922173712939929382251274859196638408735513634978587075918632129610820121916947146012885960919517262526170571569545433930157932578249720066991682399275974750539324680010204734952204498414639509378962190058420679748993065899071872607045028486161438381779765629600634030830519503930480455581589235296566877393283962355565514153608617960190376427814930409939360840428953331227888642550077797950421825318235323633591523629287742602115081427067548146397849652716631746875548436220463519025875710571074416560542680266847237162982060791093099202937028778389396785579005279700409864140859641831581824389250925834691192115339791116725689447438970597317542515057363180498144026411604942744501989229304393298136629453231141812343783702413620634977019376630935205574478496273999737731565337853302481322103319295832635199091984223776614780993204045913654979139512626636535892344852600449877590541418661886923150246125633153760853409749658566380281475740120783813531809124147535433842147650806325969306425896289306258816067502103665246960074413499574332516755046107787139896686076116794787042761654341632880793384181647767253070357669481538612787731131419150022741705645646195886568331437573603682151453178761386416165152068754415078583846174012911182038337485709651924293887252362445912534437928494150975429264621005806512435045473792118831327687762824273343681145088031134009280785177950763387146501570739875072669207916022499284795689694418645469722682945507431826148379772319905972125433287055862273411891154379344506115400718996013423920293338929554843496264060069725868597164936043777557992066080293665341619550555569627806920340136483758552894707491434189865560074389908441788339034753190731362879116889273970057384244439592117251632036499360693179282953883272990501859405679435447769374160422381566623005851005999053697002552443576649735678983421516794583623454779405740274066897714110701101220388696120241269161793243130063402669177999893905252449993490729671360694698687811678700557487828520360912741855357733179540069657225768956679116405108105512772646029186261384326038422677899012589231989032333949759209992805719018714389948233473772229947524042391313292384699936213124863058560239754531694008072170962537403609125573154455224845583557874725166083947788886218817900571969902890266292923804989473982203817928217460794608293990073275755721495162223152626698507171416870748872482674239813533012940345500207082429687262161568556229858964015387763496671061890693990070602321532645396281808295496726822293511431024353505212462583756499293711574644502025540582439362382442709708040820720785306473852408425590877003045327494820078952292121765490042462300654721510962470907134479672710303215643121393792386856209991474352926024891632625911963912949760698950675785831879475331221806902023361413032724035894093608382406265646363075631840150290517680308655443891490655288584020334662252563563480232553478074003841772401043233499592980816073211103947924365003986709399355971513534579882181899224890660776147671196431079475756820250084300075966485713329896926608303268831195892023403390398201571956480739010924768475399057820310111562151832013984753479172661838243598706138112674900562277196436063702853916074454806256993211883181023010394103392204755749657217437059270699347723514366120404569826845375776357911833579006893901311020237987759426816892863346405243396486671729903709794557839038983869920701150292654535501352688647321550887477859932242159009018283207759419306998429676960247737419767268294287045224967496736554249489217942627112296625074267609629797170434356918751367330714174279615443713631038198286858979871874379934230505931480714215652572561128797518546876476647719694229519750659928176175947078135827573771830590551888363617181602532322304025441036971458937405452689583381058346598163093475461736188465611273232321263402473062885324002928680650038382449251629066689206424546771086625840288832003506763497691942032011243199214567395688362396131092129396538367456149724969170892603376840514648387321523873182682454045445638054237377459869623971425719745291960389291186598164838856133824982134315186971136160730903692192507575544951984609258319380176078686844876101949538881827257739983970316859707832074296708409948128191320865248832404816812478798673620514459354937110461331280927407051769380131550951008176222223468852541963172165865845560401793584971078384602997722900284031517819518755817404321056601424525253830584365741458632121970688160520070245189027181452190131019316944424755123067915033870052675998925476234803656560385064530519736581079453875155680244083097237666773453599814651412463303759981403737754310325380115726421703585365808129532546740694993030649545654616659748203748653884864574463322430175973529693593144370924743101825944539999533424833236794459469643732601644129783401784090869552980190302633244756382138009276788686975001957034692014158004591325414123243237892577921254977966757755838077556499093449597908470473702831048722949581723430018236809660183703719010595338608353401602305433405758470205269939757477022479613812835588929130175190092535657628325654959465075675703898262959686089032248029017714718386821673535538639264920562147171035266877039940982774185772576200558036213401066063061521600536323867176297347804263266198498698462761253817344582351270743507384640781197881322204248766526962884568279066072912733124055200693517702303669530504087231143630048100569563239738485884517422724943524264879102865381187206095961324143232692240433829445643057891542542921666797560830065071891594201604067312943657831170053048543404021182834230142711690128664776464313449666758853310977446276614333705940489350552653347749407801689009434049583625246269889769787756807987123475766789231515418890396930956743512555252972668420700156162678705486715195582806763473828179589341973822709904325983687202597536936741565553664386422712213424853031862124585251661750070508585897603803225056705517628705543945502286396346037908977248492761718998384781666563587846931160613171352511882554964493389379889185202100222662412356744314112047775423352057330756177091660771465222748562015619666650202388524047869857689586544006650409816057599659844415805603266212748474376464466977419744879701394320804990215785407712875080043130257776603411539183923340689850869715164912778237021074563986010544113682476797668900958609600430728990407331279588014337498584948058690215191832408232854214203134399617079664852458709754228660966318799807662836416934358632663692227260919187747450712347228915993614290878542292697832270368566381132277941262949296964292954283437514331515429409641204749743109012927533127763177467869898014365005754122875436659269033532755390624285034527707655342634266582209528279922113915197716479667795480433311081821252118744904524882499421361524498136699095244785942622676238750387200032737714735751017930454541332478741745363424932434030769771813581017285088662765207636043088831048835809255713327157201566864590640914013046325821926065822405759963200301106385655227249729637666701989589459407851972698421356579719454258057849958458627612166676019521804099636708887243225498506113270263800490832122689575494219686447093755252942398815682855236622454661172960863816429406116414061202191895514558390111768085751185236250731729234457468761433805141049220426761339681268680333504905612306214001964127697118097873631861172108490593685384924221132606157773164097652730989825681094721360576086652340389683516937672916138353275740135360029666778244242414560052445432011901952420822949588385982458786711722337992347637331746118797553744870716404365708562636808718884533589154058843493897340555102736783762967346995044872659713728722566751162490783900024723877040971387666820437829784466696792720985350845680104071358376986827622226195771256149302256114718902225059630452072555629690817670517473015044078146979592816090205592277956059271675552356992127856807540496349622794676168679715371937108583411189326680658825113828836354543388561439115960388530231638526226760516827053552374542833261848791831399149720945714453445699150575955112219940715228747554872258615303059155161490837381943680235804617831576030903503383322729187044111712116083001637516540871989472602179097872958825049911448067110392394987359257109464441712230324032432496777044472079462863773131538947574340914437902631884962822302973643689139002359121543748380224292212825430616216963313311975758109461647478162197534573560480522225979786091194528593909122527782886796307040736844261196132852173125067743329407365537842978056185518968496733411989711793416170812477743150664429163780480891770056253943054700632029530536294031284829899665563325106919726957801971422089883185050114762046633616503362337091304205480873436905396404091918067146510436868294865594650351135115729462869297112163090861468617241253076933549894270710238085438939800700781635052235462457775417727833210415251457180713753960480375826288185379098908266169086559883170829681071075530666205316146456803991917526572254347319903305490624971895786985012413576618668181877815739037255001596392253775490988568606899611933843019154880200156043438076078394422424088767076094704920705962578479728543764796352951011392311453297287517010918319636685083341648248203229848328576769717443453927150469654300132787983523046541710787463296935089578549413732334131888779218459234861637779701243172154788810597501621146656498378670799330418980665406082650609849386668186660711550279676724112454493260947414222537050311082447907831379090239303946407949854778837411885412725931806776123554073669251415795747611954033015280591086109368357425059056340317609912489732601781876291518746873773690528795925421174116501128887508838246773713687322642828310975975322849348942613072476784277840690913266550758805017044529389581546546050061763128537071835000318790974820036115006050965907629061526571607273935091401481071496825991951070943477536550730522827006553377270645793709209529119577316905839186323570723167834198146795301780747911411309028324132669355636287389846432665040637615565482105040115941130757590577063122879548504734507398524669122734990146306920988981865680319106150939285723954414086669573875067084031675073742991732356401395963198155207352871492231518112968858426995946050213454760412524790194980978396675119620335239973408593908577593930680601216794171641426275304989284770272031550786471337357240347688546046386316725164617102111544300485148549788117348074672157219545898535412892624085280324574748870789264755679368895824226685670906712114711568509450711266116913718620477223695350613890430492667479990297039277917753410937834381909975317851513125009557378050744879355438456086135693037875679555941746472870673314849639147160680220115231557067695370843706096810461954672077147205906428343696540454137873997377012019484849521736800717778517042700062513862701774217978637021321259923283282116874299171389758153889300634946774405778633371305448938019485022289345337447142570987863158910839511596136764799302979200653378604009432782656381540668324176678842387490882475482946051914391190466960105166312390431401166035669871704170252704917928191528224176214455126535058659283499839881829064642123939678880191709101139700381103688040678838016132444268946927572114709996483908148932750110684599117449230051410703123127433884971661035743707723581090224354993041638815761248891915272802309241808477771405913375482440491890774512171882099510602661383738043805214406259224267853462499173534023587702986314724715479380106013617393108953572394763442998024562020855883008880336654657182491912670198139545993397937838423662789812009649220242580131383872892470074715142122804716425646556773104966109877878914028847150785866591653883438860080546866646846181897796786364007772473532199565256222860817793055394680328739492828011807911542863480861150712102330001850148854780137584867912102043538086865956063065691048933999915485569909986869181752094305021707169011000286519028495616201343738956504107719747711849875545918582433626550716450935812416448670036785436699094192551004048129180737436841088889349698553137004975888080558649557198417384607021415382382738492291503234215007304427294325996752311232735401040360302093477117515818694728633959992224729282100985630305709724504333855096027018192645430719429852140720251178936062256591495066312127918199309878528076901650693049382168915700640444096940367244972698574240194898044586844507142463300770162411240430873838961883476224126978040735007580072085554824409189059031405166011370169192993345436560827314965448498952856662808405868788481610020068469728874614746145343843838245797394062648053034378216504023288428902445053029207536097823254459834114387116060781866961279245224790494755470839066500220967651811887837667895445099258991680359406757038726867848049106989464120912818309144263136721144306248226538867473659976464356102191936842502463340069440746262212029893505962849575218290814513719795951497900378366186861127980250767785986353739430916412059465844980637865165138141912151519832808268005455813332533615831612512808887498579302602647933582382604382361001221370816584929035882707022387300794039435312150112045380136382725032638347234492137755405441414366407527454494103056593678706271483261769886141994676565671952595233669769064342085383642914886198555409676151083234623923641602193133365381505010245265273071629688007826550315529329374404548643315554133998074822461027536027956441912040115705101040376883578837021436894872809828925850230582560975829151957664910355144711044455243591671869834222913679338803687208303817663910319370425293490163986553855339299464006035543245669708554902525006588651995311666906282117471923111030710753001476172377795449248078742868418801972628888426837686243677508633625117119816462167785638849758374311449582679390995961760458983703293922100661783770576571930686755329361447414136795713510855179118451796097938045450105288148009901268966632635242452607282078873943462857380766546410572212742716135296661932772631826781636106446105596889996059174092917192281153754045992818243335306492887103053742558244287434167102458996106380665633617304408683440051432082391131811826577627346778740083463868887845285305934548313486230634920213967451311625908295273048534238793221429867008312337684311521032157410775072316583639974722191384562273089318153137984916043118281369152450795784852208267064868093875437276931721425952686597736992878273478882148291921723431155553147528959935734605596939309103303343447821638437037515893093975529675051354317590549862167579433917189571665427863047574406828263031831378710692119723926689518233331992558430168046770022081216414034494382949661870181415672268759451373355067464107319320151092253240091525904076676791049761288461927075872154722215071152581353861019801291497201129991994824241306206843441657125767817797571137297521885481309782290348515162236264818955403405534059021840015537437915511815543842115693829511939471690294196133252894294421036846874440672365743536888820326539173939548314956869836891439002343632797894021522273089702890408094433348562685279840907945980569500671966610188367650084401589268338936396406993208360699565045427188707494167962065603455911892281455369336525371604557145117138354991309564153996009099258986113821380484829983225184881157732042717523638878915459479100909115639562097648788760673727373690543015516478541918204481834208980011181708450361021464095019865378740411651024127131686758542782074617794273714690464135821783852375982724499957511984271442700868563376825315135781371699589947215230002210851056134539123698288260533694188984106378441299975098622378545636063458497420228751576862833517592199170431057453565065708841201913069004447765912122175903339708461272040361738171069471084772918538752909095460954480532525312777231103697969187443286638267474624164252199840439027987066944163399069485677908221318622970553899887862388032329366876070076370385922200075787433505638471684199427910587784998717350107555051842220004433948835044720867069038582212974659470023863862891229125497541761552532241181251277578185835410284551411726315908641658100057227999451913420352086536849988886695397058222750414240095592523478243687012862085700363168532518348876766273854816392748154576643682310648165030660124130062638912547377761871375443205871053410947888202315905831681620135300070524928100684865674489303419652803025745854179688455385586264971398506112242592229689853518425585739612975173033633987838937430331978454478444804713909368303889374729443836289419052568055117607473104647069314013763271179332987137844408916445086220063579265375287245577736012523042184844570646952958909169657969187989772452812086997488219128391775456076225868914114016510623027335017633811497267923480640133063781676634498915994353483425253518615573756040583810319094553241220380880849383407339062328324852368957202339122929471762317417490704975105920367567942821891937353214725270173474365967856746775511136147678984710729040187234150723968570550055491017997609664260784346684097824809941217010980547010701348678017541802290268870892020482591910227225132499404543856060939265816740385612724722607026917702146957975510145650506861989411072818179092965069902450024385255941916237476323473006363384086672600844409846816448481986109433734471412710065808847260161675186639888312534562763868765702369730558401278975307632582832083159444992241241797606133901735461191105779964748847341613839718911348611720212563731089375969705411034651304688326166904409653077557470072072761305489436265792707169473075694217384184459641455948152122609401798233647749660562729076477835620580839881366579188787070479397978413011583089769739984234505693787942627070555615496485784557798015297826699902712795273792798656576297478683483909414190391039269050494897838859401621972962573300104769876523614631187863275372495405723039792355238027489385502879655089709373953128686057131491931181635182717725968646929863156873945945547616736054402883602194548527073944832369284081498230795992688164900372140371451828111329854455803853790099973531258879830344679458199695288344653529511104479249310029757999110494013907949429763148419434757965416330699223517992769304390949546627501286412519105423552689111278827846436126658566497340024393010034890064475857848931724748283999363251303692049970842509159373211784029855262443126595958526683037669146664274618177730852876882531030222945550469118390455092545363725680284666940436462814938405983550503710977717716879384713921311076418794384054521434154957862915389365891267671545977457484794469610370172358134555853644791320728305998716626429412013061028863646783423171204709069026042432437662226610183635372663815617559853895717150123801823164885359743792690590163402718614577852280568578099599471796387355714375430057266371759700259002695597296217098794899686157584126337655663555280176999369263144864827939248331030247105168134936037145432908235565624876414354126348102283806202130004364488158233810630295173520511798800368988562302567177528687553616348263986824874822948933829434789434067863341675354203634198330924106009524502017096777682324194274778357269840917275907068829781557205158435664528742474241747129830143266657937709501006573414305171303071803716215145053959516600067871570871689593995133140238338246468016679850840585481854714507087240223739013935987455899994229440659381788551058867934760335089193414386828118011308073836809125438440501124116893617421791310515493562659548785035325487643124860679645084774703769748619110766688750252240747950579559215229565797102800579104298045335357462839707089371963588705072791759661218480683976476264856951377028146807235348977640348319558624320374713799482466213623283854399521413951859492056972546459504702419757383830550693534024419968239453129453403062400331865893558210631779694931066460301449266265204052182718786432535959036438829425978964134155392411963538667089227891254861609349731666119688373753981263727728458366796185380957585866349319321046655106031963084085381888209405729736388892367781168240040866308715920681882136465859393804023886086720112260264853087770753080732497043552011039454915941787093135196695644942448036061740496916440466818271219539072827540017167811180346453678631504959516401454911823211202018527689119968954570455708939695776801490764313801483158498602577269634415479874715023674998423178359047783940573472413152226189330671433594104594379176902362214026960150510156999425254498559659680797766320157563015437169918840093315138966178830691559313068478044999485598624185118913474624908646258463660322229604397938242717649360231566796339992670194392709457396817227249338105171492096205890084305392568551058806390205770279604205330934402713801899985446711910701837803365717480380760196294740141579462022128795526691035648616821468490727657003800626793040969711747470414209491627611253836875260214097508350281310000544881816545376155656227592899971357318825150071172424143770345413721928308631453323603350259772846927589743234718822981631911360401060147619404317630483631265976607254759282058938458062797019707644247098663617872793624754941142511177915223282054192523138584354607263860313161628530666461291321097748522721715390402662170031825771427974140276325453782909641752866591327751224398151779175316612332251498847814113748946357235357826545777125095274038750085366606494714545519779512223221049557778069172026190485201475426404446047283618000180090153705649755325987162804660772651314504954963548607657060521482724491008143998397253768069412194903175414791316064308418884664009173290105585188616719444125986645727375117155092833295366335040702308291531770341826525294463130221677543415977633152571856529722077611249276784909399574859005600322816156356108033669422453575607232248732402299613023203837225797676648633090381891481754159002688360339665641728400890918745821544723850836594104041982774317378413851591214253108324513089581049048735777154213981662127263852752753790107981893594391172036997958222076264431419248991622761077507891462831681756922838805362819445094169721093022359145806235026531489551943765311319865895821162740318320775091662403634125342873166457923502483646967744089587755996585413678258553715687568689201392610571184839808406556210622525817949573345727136003198308838388117493079237817785698005064402496888107175579623684160712765660288510495558635256158598930187174047915171935180560226355478685178165242537177961755106596759454903884109421919506934818518650136354873771822605564836012320104422749916448599337284026816450377667156774850162222695074506225317391358063606643333507429084038126743268826727610490609839511916207524303352002796322056589342399673555960305744391317535602209845329728735788587927139810568000627809627285849542134686719382630124160173310724907802882281998593168455485106427989004629256384505377550084226106151316281154702734779515913582157894563233253078105893604302097212309220019877074277590596783985119962562997785505982692339714007103225918024517926908515345163507913165964470589568502631394778290312224645245210328585917925906609171027934252652449762790058571469331737938035073520214491313617241802496551848373635474097243234430534127675972228980830394374804137744231259283111731252607289433813609629413642617380981157882703324577601698471783969898280352966237259662153699340598342053393244793970576877122281630541724309726165450429190190928178847023223478023026767383441065105613926133386681249108510000601373011724983139107794674728156888954111052972163863909446474597908339557421744714569808479924166674120646768980349686837441595922759841030461218096712742016536407854460562638370377917910341262135461055956192775821653735213944746530554090965743685484120273109946424973181000513350958730674777419292355333082847199435867769906989052680363654827438596871817641218387413224933722962739908598696754892747608624983989517479825217326430682511949398939132322121330852179360671793748321589835052314998787262937197417986751792710981899406838342798856427468319859670910883673335114903980936080079529076086485449933329191345982943073982148841761905333960376883503920371506997949398943107894215796133428908135079797841738190066131854568744205035163540966631094307255143594807192910749942979483217733152430579387918807670749023462863945151689774589042050699873775108710337784389784261220763436566913741775660867401164533073873250550256367221332693749343768696983163279861242506336457865090122775949687080574459197382746872755458143460802004222013596768538358974651401053679020076987766094653753256120497781286429321116904906354389184992081770000513263820876148463817528592856261688833077653069326145111542557127421086587198446613103230000351246229972836611223917440398386957544983510918769431385745515776550930380695849792776427845274932535557453813770215433998952425331207677801655974531653331258105133784606115321630962789332722912988430345665541607470918035970123292845897846561432496152349026820294905056135241139711312810600783975528524101599551242477885976257412281103038921900473342420012463131591338481609931509018690094725721990085858303336656981329788305105669767844704952955228402658222560828129566310801654163220236525315464435912294253102429458439363280297042439896924855253165637266787424393277650122738212506430649563629659313981499527131193821793956544359563883731936935745546575653448279180287614383016329296168694325319990156896075728501329749054448523100625841989588315128133168543367916482017640981598618709872882080974765279474712836776957669816254811698609791759923211209398482040314660153644653311943751515661629243959461068326571557419023775959020948768521563994630939078451876913533837887016617131086499583842110277756604619626227250033822017622914883011637069154995955879087278822412361983378585407283542708645012429379377126882753826376713854984717323147409728514883530705710962263807092309447239470897075300416636715226693419231308552046146899580154937688567826003674667498845293560064403302377888099396078364484692383573642523618400187567368529308392537049856250442722110487711436254013341474344676950039361833594457395042782658933980914280864094664712236928137956984609322094170427255317726497979893444965166521326203685889375416771664237362965504001925363823006499793171722435818626373265315434817140794799897881315954507996269667515653998502246373853879261047654415131007020956332824004080873476008632038078908571692176610469430718987436390567758173038424758823264877417089516599723841776753481140924907137082103585712559026393483487044534808358627527643552565247554451874959074622201231329094148431147463149857188150579196073673243676086667555267058523638878300021089900121592431697314236149002273025609851012643724633400891906003773568053534756128200949668265529948235167029117108410788909403312273312839583222826614927371406457799063466782346379405968328375202393773820807210357515322626798991321777959415184910387382093431778009210321605506354017587026635371157757494113485541986137748429276639666347864724209532823346329942291525670520403959929408048480599630602471286761119605403643093795165452240572711340802733352242967327588920894986853855428695228123721515383372241468677676657690523495293991193089317631180401621542742144172479955727785822302523235218082574031735817953896040708276394655939154385201465486836296826369289028372746802164088832935968378376735959745107096150987973942911422182780997918635851055043101756513159623802274259910767636803740144494908735177630186599447257685338614252482048672429227311762468974925106959539790889155410962810166695278859917991221423269379338586649296931939422836226059168586858248644039055073049158874775356894150844561229068460596159498252319420048949460320707873190437521432125497124450812927473123233426112277141951169437831866251972967156030850237579207472345326122820951035140853577889069193220801277268727877462088477256728053533865379125515357874604763127646587058569449275459757946779249849668645546601682111071473490196532829054167058513717213002674859842367700971889541249200207275542206063741885115967391824802061029095778748636469234636581470557318980869354159403672935575524353178902973678630815462378998470254705860465012317151875156962180541352865250953160734816457763801035570929130202833755080099274542583432047644326117708773927117706285648517334888275025081906593432069147949245561537721980062513881595826962559270699751134974218339058877566106472400826141536944849548372026202951697896579870697551240173945154531908638275727407118177100308950492892357865845150788334764756639628320021084296795516659674021225966584349883360538948988695704332916916970968457753514626977394323235125937751596298414132821176502673559152982900907543880968559783585312849218903494924153956864751338725863748534892535808654881519748823747627172990832688322096315182171500878487858195545087822445823622172027985930981023200387487844105509751292486047537710357093034763191156956674550796847685124778495727889795469819809739211133103413434779822666526806424901626197887282974889325166305324367468486189534285230963976870315115910099770121128076828720568702348291333556553430367365014168771293106232000895389517502599856653293083823407123075654556535091268623667286509324586514944368465584850900253948687551001752212917105353667905456456778427115125964311241646439285849237135031736695597064250392688315961441452772944623933569725613347976756813338594643432630383392657599631682544985730742007858836915433604505719070855879189485197288719040021263719868370872603890471224865906247298557916928478221291167732539671661292887371135437981942779634825649139882012944572775856251041415094963377934028329521721036519200597694830700568471309673666443171259587901752003996381084283612192024173635727775849240013454410532914435555584921531777111497746804745730530095317761154643813569457717720566923931144038213744547355819912594121144910422694237154775487752005260871222729738810894646265432369005588661788919152163982140771630564900149673912974836304232189799743487901973846011704221189906187663415798978343397632890903018531294948170205920521810817079555037004170663721973573257622734385654541890774874197893194941181634994668457144887794875282066261847615572718799662464428738434438780825061929921484926915746942103848879619392172122284804702431010212519539969796788017846333873182922337676999777534812332012929564596104599508214149701145727495059306099523699510761420309048130272939065839261625847903097311300510506280343882615164671461555078160563909494052681747251161047160059699487066709910360984455010613051396660812504250678290008428852598035154540572827777776012255457008383846221905287787575146885791923472956169292751754532877537377210888668488850063956571258521640679392408898344931351638389606589417427473476507292012928999533182971060700410625023483788305882315410026995960591558583069277684801658721562872431626137159349073603480881241710404542763803861570206810973763804468626358414660740063357740456418981456129440496134941796810008809508788374342077093093032573221654574497024220255712560913428408088998336484628062602420972884881824235484997156686204850571815572289597037462797558743867558191988812172264459758021538210414051670339659254429158690879213474450218857384191333885465275086332763991715941519656460264069449678135453878298392932595928592735417547480435257809585105010218305181370464045159787719800844217944580559140987962634899939052790022722267608134919525507626667522702387995260462315343650118033684168651734549131231336977863781238099073974418036336404348251915526348716256148308366265710493939810433401574324606201851001108298603626032883931917102193786381778850221360149271446304437825797667215143815962299855108350685801519572698690875975167844559591576384803528784625173351945548820301273433263147108424812682126149704843457845771098879325188965254752647214632462274094327285939219625637908210569178906354323258068270638423718443370259173754959417280524241925413632488148322588627394310323948564813009199314311352803018221598147606559849449441930283542725117301958336300923349861886742605979336086963459746504767954407094475416276278164559258454739153194608040844670924770762639185547020009071394069165626147123832618188432405841403221478975186849192112360772169665002540901133189776403712705903170299557334238610160128352962483565719060124695974326763550084436607613043081353179500689647381215933656567158723361671674768998402424440050925248958944809638632578121251521517893047244620356007945685810267600703806696506811034616208402733862695041093087540495322984241333683745051037656990423172498225669542175893227464699035395611643202519094625652983036905179307827133463591466000260972584100751306203312729148981161886098078386987774631774628872511021482207910602594248338333653688375949911406183753947867431811617501231335840006434744147809675823583523243884079449048889645483549560682187901817756668038944630514566610136158705181573057034029514688051495727866153977779674901732493629430889653500150739041913862194790557138035834486563129700584827159926307467671039114095278994988619502912218185733880411659332876510621812940192686452614836391774232485554419353515389717178894105195097709882454365049668790154019450473729462062338175285757777220331479036285217420864441321256204989829615594911425708850538977391959700236803969002383361343180529263474633248641019728627355145215080950707142562724727379784353038457211655061801793612817865529403259118504030821365325832670300903746117708108940594398284068590231060393666902865542697619080827749742792528004130638723427802564336546966267216565663129834773737201814975996945261200646352492989680714370859969397714453559499705714752583076892375510657152127018886147639804363516993755298742787103891150700220714506301913945252708503844524188931607809291106650515823792227016427853235911041802235206685137365256401019284320832855069543469121237788941624920491426989683514552552778606044261894121946422080372039822028735770025697298646610412950573194135173996196487113793068038302423174540357048329843011515626843095296836221153859115570494563885850899357898998653920926127657581954002375699054777755793413346729815683937056235071096200948972224545441111976421949803863069791210596882786020829195672869731642969363371256235128286786494322015713897370274398431649588021987544820839753067235909062099592022980098971310059682850113483941431571980374626893492875138111858675995342924271335544904703625997990170746600001046381081394610407361858278066292733328733897503619800318544612501802739092404855756784873762845441839227696053945349373166520728784939803938687236087005436691027144039212027192562630220756241148178921471666791089733661158823965138545449464265042179903141071863860634820674693808091110040233166872929182145362742271621264848296839972006699787543846647166316832171981705345229155258853740389128472935464813551890718004522399908471460112191208294187733687208176809677440023938416823335680651720533001783199089241378188610648258925977302547087226001915784667848654569976939992800191947168992691640409363780754050616576642743116263619044683503524666271355940073703150798572195097116558255919875668443740312812885091538481696998640935296064309086938740436546158827565693737875765801134731144684944206505748478338068242223048451290109065051470567592280202113809190801118156966526164470029632830884067909579201197089984856313503391449665077302379902192608937637391519815701507521817915970925565410307046715458891394782906701634430859881733545355987595368111980381782949400354170629114341611410275358552893318152943729497103755285569915792834643963283215604286396508690418685242009489560348181982061208001219370843466155120956528522844796088562680021364428083022837355100476966979918400555873044818468991127698511726292216919632251034916167318089592699376357221088997669432517644047426564875012548960183953391444675733852462268899979284724389106189070437324879057100736377629819552912609494980475618795730599857379925133004171092622560599686218488385161665899200822216962034469209245898826036568843248370219818743511130685548894666351208734775343246725565184077878459401836822696554162390530919387039014905010977890797144485374663349962371959295723687708764704660682014885039050695390147723329421269448951708736676806237797469029409720437802364121825876745064652897513410542921498040988349394494877885632428277518099754513072624516259131382958192221132415190724677559649555157623571670834952021768305415231510484855287494795510035072937401010227794320452346929552871893336336223584538759375576318221613134409431248132145470384000289962963467709299649490623430936995709356315657925985837617281746436195424032995059682800056369803916023915466159537148171250303467970983929821699915191107892667621911098379575001256204267382073298228990277413518336720818690356696351344806740796578274286275015291663659121105329883773971968548288165166523288207939699675622974731460763361822451285568368078048908704324305227623770030375108460447355037246315094467328632985415839167120170443543183916632235096152270658581085852229519721841652317094238942912472684802688380481359624042116560637626307278223149308043230106131532057260973907559663650534631103818210924290889827147261854036977477246202584423647677302439047142421643526537980173548464474750146923183651253326207571103483020388998680803443082640787670965471378523813414600303679303337216562953584692073250178342914588457504645172582734587375353558146504645248699565474644967822969140393538048280668176906004311407520775308650197321856205962878713226540476853754333542667576418468845403155185619682605304950256773615921424671379181048784956470777782066891059790221567987244891740068853131532880945210793423034555334737660025453854812287915287367574722094737300651426629760463515778549995340105234193410522770511217230323794173125121888374664776382887409585186323113950315197022000762333919360401083374367592134434908777847625719650026220034047921321467154839628859083138802950513640019308317693786841141934540472435600534401656938443184214207263511469015649122624167982395710169899290195788102805758824444769972971040474296665346414939357948965452532159784638695555517922935930398351993771045173266825658611505177306100016129153354139892257741066374595266105688114107725172516146750920820579461545742450621314536355725031221832387628316939325052100656216406373342737248984858191108678881124576041543648200080422905476982207118772121330440288932827327761104662429036128396775803689324153970303045881083017237712821874296508805746288173415530913604617154265138730667559996605840157970824526624110875728581920378040730890453645184418402385847147461404906591404440898397760673869896656830962025631882947083824624788548861805576535186660290649531469898263555178246288102804855417041259939391122696152931052242377695915405245986156489168268299089118880311726307261356770064152744315062496052493578607239826475211106803645889616592397560778846598143629909177839671714674696881859978011676699975910610131845613802402144710006993653508488982447201590549756301533258720285122956767498690597114300602661524919958600736397491756568940198581230328419552319865354708103759124924745534158572658861446721035847887739435042246054791207348674592409522774858684924346794530034758631214817040083923430733516862901865304032040526531973673212341031144565850263327410813659547111964429870093450986111710015594788364079328252223075184368486843386067777124027372160341696042561211572266042687063746759346365928388489518829642574931002791505762214789626051856596068870227331271075371819097370083090460880663416335454807640435242289471406207247913154969701099505038724164891510218245363912903707654321966236612619581563678826136521012533151721432145118866559676620627539556685951070717238449568770629906364177302845468979642178711436187324055605425588395887773401922698736700077282274200630127966337944323612893104365337670626742723457611101848237098194971898311106575436410869149952902470469845340366098718526042450404150391366776082533569204648168467051311524402353306757499012005897084647952840926108403970191945040844688725239463949974646233136689432669059255549273794495502453404570213977395094532284190007603105269743659113235331074619837234351930892876362567606123434315167789930074131772609347015940106834280762980967616816096906230046585763659991594294849841834832414701118265758455907643539465020161044906274913352714319257921200253252062994563451532777598078633521953440917491886288492980182677564502673506001980200727765941546882052804008898327957141448899039287578436063431839085753539385850290426473722657304404328708283368990456004363462593693267233113610277404407054889020614210669381155236114153399888213821609030083962463390860046274893958743427825341822366879194590406703465400738425681537081956266857105006201896719074586131571527254876987459881263621475710581321754454449215825878353757799023776776983809184493044846165631691697022649924085999176358391111708814246789003903331956673648936259603158267583233701401520324078752883381963603200051670634155891666891140642244295146571350913425641178162777626492025465565295402949036390051839285379129822660331195521193124802122694215549198056895663010699228761735802388480000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Page 41: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 41

2. Approach: Ants

How do ants find a short path from I to O?

Solution: search concurrently!

Instead of testing, find all ways at once

Page 42: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 42

The Dijkstra Algorithm

Page 43: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 43

Shortest Path

Definition: The weight of a path p = v1, v2, ..., vk in a graph G = ( V, E ) with edges labeled with c, is the sum of the weights of its constituent edges, e.g.:

c(p) = c(( vi, vi+1 ))

Definition: The shortest path between two vertices u and v in G is the path with minimum weight

The weight of the shortest path is called the distance between u and v

In non-weighted graphs, c(p) = |p|

k-1

i=1

Page 44: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 44

Example

All paths from a to d: p1 = a, b, c, d p2 = a, b, d

Shortest path from a to d: p2 (weight 9) Shortest path from e to f: p3 = e, f (weight 2) Shortest path from a to e?

5

3

7

4

a b

cd e

f

2

Page 45: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 45

Single Source Problem

single source shortest path: Given a graph G = ( V, E ), we want to find a shortest path from a source vertex s V to all other vertices v V.

BFS-based algorithm (by Moore): uses length attribute D of a vertex to keep the distance (as number of hops).

for ( all v V ) { v.D = } /* Initialization */Queue Q = new Queue( );Q.enqueue( s ); s.D = 0; i = 0; /* s is source vertex */while ( !Q.empty( ) ) {

w = Q.dequeue( );if ( w.D == i ) i++; /* next level in BFS */for ( all v w.neighbours( ) ) {

if ( v.D == ) { v.D = i; Q.enqueue(v); }}

}

Page 46: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 46

Example

Visited vertex i v1 v2 v3 v4 v5 v6 q

0 0 v1

v1 1 1 1 1 v2,v3,v6

v2 2 v3,v6

v3 2 2 v6,v4,v5

v6 v4,v5

v4 3 v5

v5 -

“Wave propagation” in the graph

Page 47: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 47

Analysis

Moore‘s Algorithm has the same time complexity as BFS, i.e. O( max( |E|, |V| )).

The algorithm does not work for weighted edges Enhancement by Dijkstra was published 1959!

Page 48: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 48

Dijkstra‘s Algorithm

single source shortest path with arbitrary positive edge weights, start vertex s

Correctness is based on the principle of optimality: For any shortest path p = v1, ..., vk from v1 to vk, each subpath p‘ = vi, ..., vj with 1 i < j k is a shortest path between vi and vj.

Proof: Assume that this does not hold, i.e. there is a shorter path p‘‘

p‘ between vi and vj. Then, there exists a shorter path between v1 and vk by

exchanging subpath p‘ with p“.

Page 49: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 49

Dijkstra‘s Algorithm (2)

Each vertex belongs to one of the categories M1, M2, M3. Extract vertex v from M2 to which there is a minimal edge

between M1 und M2 and insert v in M1 (principle of optimality). For each vertex that was moved to M1, move its successors to

M2.

s u v

not reached vertices

marginal vertices

selected vertices

M1

M2M3

Page 50: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 50

Dijkstra‘s Algorithm (3)

More precisely: Divide the vertices in the graph into three subsets: M1 contains all vertices v, for which shortest paths between s and v are

already known. M2 contains all vertices v, that are reachable via an edge from a vertex in

M1. M3 contains all other vertices.

Invariant: denote sp(x,y) as the shortest path from x to y. Then, the following holds: For all shortest paths sp(s,u) and all edges (u,v)

c(sp(s,u)) + c((u,v)) c(sp(s,v)) For at least one shortest path sp(s,u) and one edge (u,v)

c(sp(s,u)) + c((u,v)) = c(sp(s,v))

Page 51: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 51

Dijkstra‘s Algorithm

/* M3 implicitly defined as V \ ( M1 M2 ) */for (all v V) { if ( v s ) v.L = ; } /* Initialization */

M1 = { s }; M2 = { }; s.L = 0;for (all v s.neighbors( ) ) { /* fill M2 initially */

M2 = M2 { v };v.L = c((s,v));

}while ( M2 { } ) {

v = the vertex in M2 with minimum v.L;M2 = M2 \ { v }; M1 = M1 { v };/* move v */for (all w v.neighbours( ) ) {

if ( w M3 ) { /* move neighbours and update L */M2 = M2 { w }; w.L = v.L + c((v,w));

} else if ( w.L > v.L + c((v,w)) ) /* w in M2 (or M1)*/

w.L = v.L + c((v,w));}

}

Page 52: © DEEDS 2008Graph Algorithms1 Application: Process Relations  Some algorithms require that sub processes (tasks) are executed in a certain order, because.

© DEEDS 2008 Graph Algorithms 52

(7)

Example

v M1 M2 v1 v2 v3 v4 v5 v6

{ { } 0

v1 v2,v3,v6 3 2 6

v3 v3 v2,v4,v5,v6 6 7

v2 v2 v4,v5,v6 4

v6 v6 v4,v5 5

v4 v4 V5

v5 v5}

1 2

36

5 4

3

32

6

1

5

5

2

1

4

3(6)(2)

(3)

(6)

(4)

(5)