Theory of Algorithms:Theory of Algorithms:Decrease and ConquerDecrease and ConquerTheory of Algorithms:Theory of Algorithms:Decrease and ConquerDecrease and Conquer

James Gain and Edwin Blake{jgain | edwin} @cs.uct.ac.za

Department of Computer Science

University of Cape Town

August - October 2004

ObjectivesObjectives

To introduce the decrease-and-conquer mind set

To show a variety of decrease-and-conquer solutions: Depth-First Graph Traversal

Breadth-First Graph Traversal

Fake-Coin Problem

Interpolation Search

To discuss the strengths and weaknesses of a decrease-and-conquer strategy

Decrease and ConquerDecrease and Conquer

1. Reduce problem instance to smaller instance of the same problem and extend solution

2. Solve smaller instance

3. Extend solution of smaller instance to obtain solution to original problem

Also called inductive or incremental

Unlike Divide-and-Conquer don’t work equally on both subproblems (e.g. Binary Search)

SUBPROBLEM OF SIZE n-1

A SOLUTION TO THEORIGINAL PROBLEM

A SOLUTION TO SUBPROBLEM

A PROBLEM OF SIZE n

Flavours of Decrease and ConquerFlavours of Decrease and Conquer

Decrease by a constant (usually 1): instance is reduced by the same constant on each iteration Insertion sort Graph Searching: DFS, BFS Topological sorting Generating combinatorials

Decrease by a constant factor (usually 2): instance is reduced by same multiple on each iteration Binary search Fake-coin problem

Variable-size decrease: size reduction pattern varies from one iteration to the next Euclid’s algorithm Interpolation Search

Exercise: Spot the DifferenceExercise: Spot the Difference

Problem: Derive algorithms for computing an

using:

1. Brute Force

2. Divide and conquer

3. Decrease by one

4. Decrease by constant factor (halve the problem size)

Hint: each can be described in a single line

Graph TraversalGraph Traversal

Many problems require processing all graph vertices in a systematic fashion

Data Structures Reminder:

Graph traversal strategies: Depth-first search (traversal for the Brave) Breadth-first search (traversal for the Cautious)

a b

c d

a b c d

a 0 1 1 1

b 1 0 0 1

c 1 0 0 1

d 1 1 1 0

a

b

c

d

a

b c d

d

a da b c

Depth-First SearchDepth-First Search

Explore graph always moving away from last visited vertex

Similar to preorder tree traversal

DFS(G): G = (V, E)

count 0

mark each vertex as 0

FOR each vertex v V DO

IF v is marked as 0

dfs(v)

dfs(v):

count count + 1

mark v with count

FOR each vertex w adjacent to v DO

IF w is marked as 0

dfs(w)

Example: DFSExample: DFSa b

e f

c d

g h

Traversal Stack: (pre = push, post = pop)

1a8 2b7 3f2 4e1

5g6 6c5 7d4 8h3

Push order: a b f e g c d h

Pop order: e f h d c g b a

DFS ForestDFS Forest

DFS Forest: A graph representing the traversal structure Types of Edges:

Tree edges: edge to next vertex in traversal Back edges: edge in graph to ancestor nodes Forward edges: edge in graph to descendants (digraphs only) Cross edges: none of the above

a b

e f

c d

g h

a b ef

c dg h

Backedge

Notes on Depth-First SearchNotes on Depth-First Search

Implementable with different graph structures: Adjacency matrices: (V2)

Adjacency linked lists: (V+E)

Yields two orderings: preorder: as vertices are 1st encountered (pushed)

postorder: as vertices become dead-ends (popped)

Applications: Checking connectivity, finding connected components

Checking acyclicity

Searching state-space of problems for solution (AI)

Breadth-First SearchBreadth-First Search Move across to all neighbours of the last

visited vertex Similar to level-by-level tree traversals Instead of a stack, breadth-first uses a

queue

BFS(G): G = (V, E)

count 0

mark each vertex as 0

FOR each vertex v V DO

bfs(v)

bfs(v):

count count + 1

mark v with count

initialize queue with v

WHILE queue not empty DO

a front of queue

FOR each vertex w adjacent to a DO

IF w is marked as 0

count count + 1

mark w with count

add w to queue

remove a from queue

Example: BFSExample: BFSa b

e f

c d

g h

Traversal Queue:

a1 b2 e3 f4

g5

c6 h7

d8

Order: a b e f g c h d

a

b e f

c d

g

h

Crossedge

Notes on Breadth First SearchNotes on Breadth First Search

BFS has same efficiency as DFS and can be implemented with: Adjacency matrices: (V2)

Adjacency linked lists: (V+E)

Yields single ordering of vertices

Applications: same as DFS, but can also find paths from a vertex to all other vertices with the smallest number of edges

The Fake-Coin Problem: The Fake-Coin Problem: Decrease by a Constant FactorDecrease by a Constant Factor

Problem: Among n identical looking coins, one is a fake (and weighs less)

We have a balance scale which can compare any two sets of coins

Algorithm: Divide into two size n/2 piles (keeping a coin aside if n is odd)

If they weigh the same then the extra coin is fake

Otherwise proceed recursively with the lighter pile

Efficiency: W(n) = W( n/2 ) + 1 for n > 1

W(n) = log2 n = (log2 n)

But there is a better (log3 n) algorithm

Euclid’s GCD: Euclid’s GCD: Variable-Size DecreaseVariable-Size Decrease

Problem: Greatest Common Divisor of two integers m and n is the largest

integer that divides both exactly

Alternative Solutions: Consecutive integer checking (brute force)

Identify common prime factors (transform and conquer)

Euclid’s Solution: gcd(m, n) = gcd(n, m mod n)

gcd(m, 0) = m

Right-side args are smaller by neither a constant size nor factor

Example: gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12

Interpolation Search: Interpolation Search: Variable-Size DecreaseVariable-Size Decrease

Mimics the way humans search through a phone book (look near the beginning for ‘Brown’)

Assumes that values between the leftmost (A[l]) and rightmost (A[r]) list elements increase linearly

Algorithm (key = v, find search index = i): Binary search with floating

variable at index i Setup straight line through

(l, A[l]) and (r, A[r]) Find point P = (x, y) on line

at y = v, then i = x x = l + (v - A[l])(r - l) / (A[r] - A[l])

Efficiency: Average = (log log n + 1), Worst = (n)

l i r

A[l]

v

A[r]

index

value

Strengths and Weaknesses of Strengths and Weaknesses of Decrease-and-ConquerDecrease-and-Conquer

Strengths: Can be implemented either top down (recursively) or

bottom up (without recursion)

Often very efficienct (possibly (log n) )

Leads to a powerful form of graph traversal (Breadth and Depth First Search)

Weaknesses: Less widely applicable (especially decrease by a constant

factor)