Chapter 5Chapter 5

Decrease and ConquerDecrease and Conquer

Homework 7Homework 7

hw7 (due 3/17)hw7 (due 3/17)– page 127 question 5page 127 question 5– page 132 questions 5 and 6page 132 questions 5 and 6– page 137 questions 5 and 6page 137 questions 5 and 6– page 168 questions 1 and 4page 168 questions 1 and 4

Decrease and ConquerDecrease and ConquerAlso referred to asAlso referred to asinductiveinductive or or incremental incremental approach approach

1.1. Reduce problem instance to smaller Reduce problem instance to smaller instance of the same problem and instance of the same problem and extend solutionextend solution

2.2. Solve smaller instanceSolve smaller instance

3.3. Extend solution of smaller instance to Extend solution of smaller instance to obtain solution to original problemobtain solution to original problem

Note: We are Note: We are notnot dividing the problem dividing the problem into into twotwo smaller problems. smaller problems.

Examples of Decrease Examples of Decrease and Conquerand Conquer

Decrease by one:Decrease by one:– Insertion sortInsertion sort– Graph search Graph search

algorithms:algorithms: DFSDFS BFSBFS Topological sortingTopological sorting

– Algorithms for Algorithms for generating generating permutations, subsetspermutations, subsets

Decrease by a Decrease by a constant factorconstant factor– Binary search Binary search – Fake-coin problemsFake-coin problems– multiplication multiplication à la russeà la russe– Josephus problemJosephus problem

Variable-size decreaseVariable-size decrease– Euclid’s algorithmEuclid’s algorithm– Selection by partitionSelection by partition

What’s the difference?What’s the difference?

Consider the problem of Consider the problem of exponentiation: Compute exponentiation: Compute aann

Brute Force:Brute Force: Divide and conquer:Divide and conquer: Decrease by one:Decrease by one: Decrease by constant factor:Decrease by constant factor:

aann Brute Force:Brute Force:

a*a*a*a*a* ... *a*a*aa*a*a*a*a* ... *a*a*a Requires n multiplicationsRequires n multiplications Programmed as a loopProgrammed as a loop Obviously O(n)Obviously O(n)

aann Divide and conquer:Divide and conquer:

aan/2n/2 * a * an/2n/2

Isn’t this clever?Isn’t this clever? aa88 = =aa44 * * aa44 = = aa22*a*a22 * * aa22*a*a22 = =

aa11*a*a11*a*a11*a*a1 1 * * aa11*a*a11*a*a11*a*a11

Why is this retarded?Why is this retarded? If you compute aIf you compute a44 why do you have to why do you have to

compute it again?compute it again? Sometimes divide and conquer doesn’t Sometimes divide and conquer doesn’t

yield an advantage!yield an advantage!

aann Decrease by one:Decrease by one:

aann = a = an-1n-1 * a * a Q: Is this really any different than Q: Is this really any different than

the brute force method?the brute force method? A: No, except that it can be A: No, except that it can be

programmed recursively.programmed recursively. We still haven’t done better than We still haven’t done better than

O(n)O(n)

aann Decrease by Decrease by constant factor:constant factor: aann= (a= (an/2n/2))22

Your probably thinking: Dr.B. kidding, Your probably thinking: Dr.B. kidding, right?right?

Q: Isn’t this exactly the same as the Q: Isn’t this exactly the same as the divide an conquer approach?divide an conquer approach?

A: No, check this out.A: No, check this out. aa88 = (a = (a44))22 = ((a = ((a22))22))22 = ((a*a) = ((a*a)22))22

We actually only do 3 multiplicationsWe actually only do 3 multiplications a*a = va*a = v11; ; vv11 * v * v11 = v = v22;; vv22 * v * v22 = a = a88

aann Decrease by Decrease by constant factor:constant factor: aann= (a= (an/2n/2))22

aa1616= (((a*a)= (((a*a)22))22))22 4 multiplications 4 multiplications aa3232= ((((a*a)= ((((a*a)22))22))22))22 5 multiplications 5 multiplications …… aa10241024 10 multiplications 10 multiplications Obviously this is an O(log n) algorithmObviously this is an O(log n) algorithm What about aWhat about a4747??

aann Decrease by Decrease by constant factor:constant factor: What about aWhat about a4747?? aa4747 = a*(a = a*(a2323))22 2 multiplications 2 multiplications aa2323 = a*(a = a*(a1111))22 2 multiplications 2 multiplications aa1111 = a*(a = a*(a55))22 2 multiplications 2 multiplications aa55 = a*(a = a*(a22))22 2 multiplications 2 multiplications aa22 = a*a = a*a 1 multiplications 1 multiplications It might actually take 2*log n the worst It might actually take 2*log n the worst

case, which is still O(log n).case, which is still O(log n). Isn’t Big-O nice?Isn’t Big-O nice?

Graph TraversalGraph Traversal

Many problems require Many problems require processing all graph vertices in processing all graph vertices in systematic fashionsystematic fashion

Graph traversal algorithms:Graph traversal algorithms:– Depth-first searchDepth-first search– Breadth-first searchBreadth-first search

Depth-first searchDepth-first search Given a graph G=(V,E), explore graph always Given a graph G=(V,E), explore graph always

moving away from last visited vertexmoving away from last visited vertex G is a graph which consists of two setsG is a graph which consists of two sets

– V is a set of verticesV is a set of vertices V = {A, B, C, D, E, F}V = {A, B, C, D, E, F}

– E is a set of edgesE is a set of edges E = {(A,B), (A,C), (C,D), (D,E), (E,C), (B,F)}E = {(A,B), (A,C), (C,D), (D,E), (E,C), (B,F)}

A

B

C

DE

F

Depth-first searchDepth-first searchDFS(G)DFS(G)

count :=0count :=0mark each vertex with 0 (unvisited)mark each vertex with 0 (unvisited)for each vertex vfor each vertex v in in V doV do

if v is marked with 0 if v is marked with 0 dfs(v)dfs(v)

dfs(dfs(vv))count := count + 1count := count + 1mark mark vv with count with countfor each vertex for each vertex ww adjacent to adjacent to vv dodo

if if ww is marked with 0 is marked with 0 dfs(dfs(ww))

Types of edgesTypes of edges

Tree edges:Tree edges: edges comprising edges comprising forestforest

Back edges:Back edges: edges to ancestor edges to ancestor nodesnodes

Forward edgesForward edges: edges to : edges to descendants (digraphs only) descendants (digraphs only)

Cross edges:Cross edges: none of the above none of the above

Depth-first search: Depth-first search: NotesNotes DFS can be implemented with graphs DFS can be implemented with graphs

represented as:represented as:– Adjacency matrices: Adjacency matrices: ΘΘ((VV22))– Adjacency linked lists: Adjacency linked lists: ΘΘ((VV+E)+E)

Yields two distinct ordering of Yields two distinct ordering of vertices:vertices:– preorder: as vertices are first preorder: as vertices are first

encountered (pushed onto stack)encountered (pushed onto stack)– postorder: as vertices become dead-ends postorder: as vertices become dead-ends

(popped off stack)(popped off stack)

Depth-first search: Depth-first search: NotesNotes Applications:Applications:

– checking connectivity, finding checking connectivity, finding connected componentsconnected components

– checking acyclicitychecking acyclicity– searching state-space of problems searching state-space of problems

for solution (AI)for solution (AI)

Breadth-first searchBreadth-first search

Explore graph moving across to all the Explore graph moving across to all the neighbors of last visited vertexneighbors of last visited vertex

Similar to level-by-level tree traversals Similar to level-by-level tree traversals Instead of a stack, breadth-first uses Instead of a stack, breadth-first uses

queuequeue Applications: same as DFS, but can Applications: same as DFS, but can

also find paths from a vertex to all also find paths from a vertex to all other vertices with the smallest other vertices with the smallest number of edgesnumber of edges

Breadth-first search Breadth-first search algorithmalgorithm

bfs(bfs(vv))

count := count + 1count := count + 1

mark mark vv with count with count

initialize queue with initialize queue with vv

while queue is not empty dowhile queue is not empty do

aa := front of queue := front of queue

for each vertex for each vertex ww adjacent to adjacent to aa do do

if if ww is marked with 0 is marked with 0

count := count + 1count := count + 1

mark mark ww with count with count

add add ww to the end of the queue to the end of the queue

remove remove a a from the front of the queuefrom the front of the queue

BFS(G)BFS(G)count :=0count :=0mark each vertex with 0mark each vertex with 0for each vertex v in V dofor each vertex v in V do

bfs(v)bfs(v)

Breadth-first search: Breadth-first search: NotesNotes BFS has same efficiency as DFS and BFS has same efficiency as DFS and

can be implemented with graphs can be implemented with graphs represented as:represented as:– Adjacency matrices: Adjacency matrices: ΘΘ((VV22))– Adjacency linked lists: Adjacency linked lists: ΘΘ((VV+E)+E)

Yields single ordering of vertices Yields single ordering of vertices (order added/deleted from queue is (order added/deleted from queue is the same)the same)

Directed acyclic graph Directed acyclic graph (dag)(dag) A directed graph with no cyclesA directed graph with no cycles

Arise in modeling many problems, Arise in modeling many problems, eg:eg:– prerequisite structureprerequisite structure– food chainsfood chains

Imply partial ordering on the domainImply partial ordering on the domain

Topological Topological sortingsorting Problem: find a total order consistent Problem: find a total order consistent

with a partial orderwith a partial order Example:Example:

fish

human

shrimp

sheep

wheatplankton

tiger Order them so that they don’t have to wait for any

of their food(i.e., from lower to higher,

consistent with food chain)

Problem is solvable iff graph is dag

Topological sorting Topological sorting AlgorithmsAlgorithms1.1. DFS-based algorithm:DFS-based algorithm:

– DFS traversal noting order vertices are popped DFS traversal noting order vertices are popped off stackoff stack

– Reverse order solves topological sortingReverse order solves topological sorting– Back edges encountered?Back edges encountered?→→ NOT a dag! NOT a dag!

2.2. Source removal algorithmSource removal algorithm– Repeatedly identify and remove a Repeatedly identify and remove a sourcesource

vertex, ie, a vertex that has no incoming edgesvertex, ie, a vertex that has no incoming edges

Both Both ΘΘ((VV+E) using +E) using adjacency linked listsadjacency linked lists

Variable-size-decrease: Variable-size-decrease: Binary search treesBinary search trees

Arrange keys in a binary tree with Arrange keys in a binary tree with the the binary search tree propertybinary search tree property::

k

<k >k• What about repeated keys?What about repeated keys?

Example 1:Example 1: 5, 10, 3, 1, 7, 12, 9

Example 2:Example 2: 4, 5, 7, 2, 1, 3, 6

Searching and insertion in Searching and insertion in binary search treesbinary search trees Searching – straightforwardSearching – straightforward Insertion – search for key, insert at leaf where Insertion – search for key, insert at leaf where

search terminatedsearch terminated All operations: worst case # key comparisons = All operations: worst case # key comparisons =

hh + 1 + 1– lg lg nn ≤ ≤ h h ≤ ≤ nn – 1 with average (random files) 1.41 lg – 1 with average (random files) 1.41 lg nn– Thus all operations have:Thus all operations have:

worst case: worst case: ΘΘ((nn) ) average case: average case: ΘΘ(lg(lgnn) )

Bonus:Bonus: inorder traversal produces sorted list inorder traversal produces sorted list ((treesorttreesort))

Homework 7Homework 7

hw7 (due 3/17)hw7 (due 3/17)– page 127 question 5page 127 question 5– page 132 questions 5 and 6page 132 questions 5 and 6– page 137 questions 5 and 6page 137 questions 5 and 6– page 168 questions 1 and 4page 168 questions 1 and 4