LEVEL II, TERM II CSE – 243 MD. MONJUR-UL-HASAN LECTURER DEPT OF CSE, CUET EMAIL:...

LEVEL II, TERM IICSE – 243

MD. MONJUR-UL-HASANLECTURER

DEPT OF CSE, CUETEMAIL: [email protected]

Backtracking Algorithm

An Example

Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

2

Find out all 3-bit binary numbers for which the sum of the 1's is greater than or equal to 2.

The only way to solve this problem is to check all the possibilities: (000, 001, 010, ....,111)

The 8 possibilities are called the search space of the problem. They can be organized into a tree.

An Example (cont…)


3

0 0 _

0 0 0 0 0 1

0 1 _

0 1 0 0 1 1

1 _ _0 _ _

1 0 _

1 0 0 1 0 1

1 1 _

1 1 0 1 1 1

Backtracking: Definition4


StartSuccess!

Success!

Failure•For some problems, the only way to solve is to check all possibilities.•Backtracking is a systematic way to go through all the possible configurations of a search space.•Problem space consists of states (nodes) and actions (paths that lead to new states). When in a node can only see paths to connected nodes•If a node only leads to failure go back to its "parent“ node. Try other alternatives. If these all lead to failure then more backtracking may be necessary.

Goals of Backtracking


5

Possible goals Find a path to success Find all paths to success Find the best path to success

Not all problems are exactly alike, and finding one success node may not be the end of the searchStart

Success!

Success!

Backtracking and Recursive

Backtracking is easily implemented with recursion because:

1. The run-time stack takes care of keeping track of the choices that got us to a given point.

2. Upon failure we can get to the previous choice simply by returning a failure code from the recursive call.

6


Improving Backtracking: Search Pruning


7

Search pruning will help us to reduce the search space and hence get a solution faster.

The idea is to a void those paths that may not lead to a solutions as early as possible by finding contradictions so that we can backtrack immediately without the need to build a hopeless solution vector.

A More Concrete Example: Suduku


8

8

9 by 9 matrix with some numbers filled in

all numbers must be between 1 and 9

Goal: Each row, each column, and each mini matrix must contain the numbers between 1 and 9 once each

no duplicates in rows, columns, or mini matrices

Suduku with Brut force


9

A brute force algorithm is a simple but general approach

Try all combinations until you find one that works

This approach isn’t clever, but computers are fast

Then try and improve on the brute force resuts

Sodoku (Cont…)


10

After placing a number in a cell is the remaining problem very similar to the original problem?

Brute force Sudoku Soluton if not open cells, solved scan cells from left to right, top to

bottom for first open cell When an open cell is found start

cycling through digits 1 to 9. When a digit is placed check that

the set up is legal now solve the board

1

Solving Sudoku – Later Steps


11

1 1 2 1 2 4

1 2 4 8 1 2 4 8 9

uh oh!

Solving Sudoku – Dead End


12

We have reached a dead end in our search

With the current set up none of the nine digits work in the top right corner

1 2 4 8 9

Solving Suduku - Backing Up


13

When the search reaches a dead end in backs up to the previous cell it was trying to fill and goes onto to the next digit

We would back up to the cell with a 9 and that turns out to be a dead end as well so we back up again so the algorithm needs to remember

what digit to try nextNow in the cell with the 8. We

try and 9 and move forward again.

1 2 4 8 9

1 2 4 9

Problem with Suduku


14

Brute force algorithms are slowThe don't employ a lot of logic

For example we know a 6 can't go in the last 3 columns of the first row, but the brute force algorithm will plow ahead any way

But, brute force algorithms are fairly easy to implement as a first pass solution backtracking is a form of a brute force algorithm

Solving Suduku-Search pruning


15

After trying placing a digit in a cell we want to solve the new sudoku board Isn't that a smaller (or simpler version) of the same

problem we started with?!?!?!?After placing a number in a cell the we need to

remember the next number to try in case things don't work out.

We need to know if things worked out (found a solution) or they didn't, and if they didn't try the next number

If we try all numbers and none of them work in our cell we need to report back that things didn't work

Solving Suduku-Recursive Backtracking


16

Problems such as Suduko can be solved using recursive backtracking

recursive because later versions of the problem are just slightly simpler versions of the original

backtracking because we may have to try different alternatives

General Back Tracking Algorithm


17

void checknode (node v) {

if (promising ( v ))if (aSolutionAt( v ))

write the solutionelse //expand the node for ( each child u of v )

checknode ( u )}

General Back Tracking Algorithm (Cont…)


18

Checknode uses the functions:

promising(v) which checks that the partial solution represented by v can lead to the required solution

aSolutionAt(v) which checks whether the partial solution represented by node v solves the problem.

Eight Queen Problem: A Classic Puzzle


19

Attempts to place 8 queens on a chessboard in such a way that no queen can attack any other.

A queen can attack another queen if it exists in the same row, columm or diagonal as the queen.

This problem can be solved by trying to place the first queen, then the second queen so that it cannot attack the first, and then the third so that it is not conflicting with previously placed queens.

N Queen Problem (N=8)


20

Place N Queens on an N by N chessboard so that none of them can attack each other

Number of possible placements?In 8 x 8

64 * 63 * 62 * 61 * 60 * 58 * 59 * 57 = 4,426,165,368roughly four and a half billion possible set ups

n choose k

How many ways can you choose k things from aset of n items? In this case there are 64 squares and we want to

choose 8 of them to put queens on

Solution: N Queen Puzzle


21

We know that for queens:1. each row will have exactly one queen2. each column will have exactly one queen3. each diagonal will have at most one queen

The previous calculation includes set ups like following:

Includes lots of set ups with multiple queens in the same column

Number of set ups 8 * 8 * 8 * 8 * 8 * 8 * 8 * 8 = 16,777,216

We have reduced search space by two orders of magnitude by applying some logic

N Queen Solution: Chess board Store


22

The previous calculation help us to model the chessboard not as a 2-D array, but as a set of rows, columns and diagonals.

To simplify the presentation, we will study for smaller chessboard, 4 by 4

First: we need to define an array to store the location of so far placed queens

PositionInRow

1

3

0

2

N Queen Solution: Chess board Store (Cont..)


23

We need an array to keep track of the availability status of the column when we assign queens.

FTFT

Suppose that wehave placed twoqueens



24

We have 7 left diagonals, we want to keep track of available diagonals after the so far allocated queens

T

T

F

T

F

T

T



25

We have 7 left diagonals, we want to keep track of available diagonals after the so far allocated queens

T

F

T

T

F

T

T



26

Now Run a loop from 1 to 4 with counter i and place a Queen to safe position of the ith column.

Now try to make a solution for N Queen.Try to find all possible solution.

[4,3]

[3,1]

[2,4]

[1,2]

N Queen Puzzle: Algorithm


27

putQueen(int row){ for col=1 to N if (column[col]==available &&

leftDiagonal[row+col]==available && rightDiagonal[row-col+norm]== available) { positionInRow[row]=col; column[col]=!available; leftDiagonal[row+col]=!available; rightDiagonal[row-col+norm]=!available; if (row< squares-1) putQueen(row+1); else

PRINT solution found column[col]=available; leftDiagonal[row+col]=available; rightDiagonal[row-col+norm]= available; }}

Base

Update

Forward

Recursi.

Back Trac

k

Backtracking Algorithm for optimization problems


28

Procedure checknode (node v ) {node u ;

if ( value(v) is better than best )

best = value(v);if (promising (v) )

for (each child u of v)checknode

(u );}

best is the best value so far and is initialized to a value that is equal or worse than any possible solution.

value(v) is the value of the solution at the node.

Backtracking for optimization problems (Cont…)


29

To deal with optimization we compute:best - value of best solution achieved so farvalue(v) - the value of the solution at node v

Modify promising(v)

Best is initialized to a value that is equal to a candidate solution or worse than any possible solution.

Best is updated to value(v) if the solution at v is “better”

By “better” we mean: larger in the case of maximization and smaller in the case of minimization

Modifying promising


30

A node is promising when it is feasible and can lead to a feasible solution

and “there is a chance that a better solution than best

can be achieved by expanding it”Otherwise it is nonpromising

A bound on the best solution that can be achieved by expanding the node is computed and compared to best

If the bound > best for maximization, (< best for minimization) the node is promising

Modifying promising for Maximization Problems


31

For a maximization problem the bound is an upper bound, the largest possible solution that can be

achieved by expanding the node is less or equal to the upper bound

If upper bound > best so far, a better solution may be found by expanding the node and the feasible node is promising

Modifying promising for Minimization Problems


32

For minimization the bound is a lower bound, the smallest possible solution that can be

achieved by expanding the node is less or equal to the lower bound

If lower bound < best a better solution may be found and the feasible node is promising

Optimization Example:0-1 Knapsack Problem

The 0-1 knapsack problem:N items which cost p1,p2,p3….pn and have

weight w1, w2,w3….wn. (where the i-th item is worth pi dollars and weight wi pounds.) vi and wi are integers.

A thief can carry at most C (integer) pounds.How to take as valuable a load as possible.

An item cannot be divided into pieces.The fractional knapsack problem:

The same setting, but the thief can take fractions of items.

33


Knapsack Problem: In Mathematical Term


34

Input description: A set of items 1n, where item i has size wi and value pi, and a knapsack capacity C

Problem description: Find a subset S that maximizes given that

i.e. all the items fit in a knapsack of size C

Si

ip CwSi

i

0-1 Knapsack - Example


35

50

E.g.1:

1020

30

50

Item 1

Item 2

Item 3

$60 $100 $120

10

20

$60

$100

+

$160

50

20 $100

$120

+

$220

30

$6/pound$5/pound$4/pound

W

E.g.2: Item: 1 2 3 4 5 6 7 Benefit: 5 8 3 2 7 9 4 Weight: 7 8 4 10 4 6 4 Knapsack holds a maximum of 22 pounds Fill it to get the maximum benefit

Brute force tree


36

A

B

C

D

E E

D

E E

C

D

E E

D

E E

B

C

D

E E

D

E E

C

D

E E

D

E E

In Out

Weight = 15.12Value = $315



0-1 Knapsack Problem: Brute force Algorithm


37

Let’s first solve this problem with a straightforward algorithmSince there are n items, there are 2n possible combinations of

items.We go through all combinations and find the one with the most

total value and with total weight less or equal to WRunning time will be O(2n) In the tree representation, we can think of the previous

algorithm doing a DFS in the tree If we reach a point where a solution no longer is feasible,

there is no need to continue exploring We can “backtrack” from this point and potentially cut off

much of the tree and many of the solutions We can backtrack if we know the best possible solution in

current subtree is worse than current best solution obtained so far.

0-1 Knapsack: Algorithm


38

Inputs: Positive integers n and W; arrays w and p, each indexed from 1 to n, and each containing positive integers sorted in nonincreasing order according to the values of p[i]/w[i].

Outputs: an array bestset indexed from 1 to n, where the values of bestset[i] is "yes" if the ith item is included in the optimal set and is "no" otherwise; an integer maxprofit that is the maximum profit.

void knapsack (index i, int profit, int weight) { if (weight <= W&& profit > maxprofit){ // This set is best maxprofit = profit; // so far. numbest = i; // Set numbest to bestset = include; // number of items } // considered. Set // bestset to this // solution. if (promising(i)){ include [i + 1] = "yes"; // Include w[i + 1]. knapsack(i + 1, profit + p[i + 1], weight + w[i + 1]); include [i + 1] = "no"; // Do not include knapsack (i + 1, profit, weight); // w[i + 1]. } }

0-1 Knapsack: Algorithm (Cont…)


39

bool promising (index i) { index j, k; int totweight; float bound; if (weight >= W) // Node is promising only return false; // if we should expand to else { // its children. There must j = i + 1; // be some capacity left for bound = profit; // the children. totweight = weight;

0-1 Knapsack: Algorithm (Cont…)


40

while (j <= n && totweight + w[j] < = W){ // Grab as many items as totweight = totweight + w[j]; // possible. bound = bound + p[j]; j++; } k = j; // Use k for consistency if (k <=n) // with formula in text. bound = bound + (W - totweight) * p[k]/w[k]; // Grab fraction of kth return bound > maxprofit; // item. } }

0-1 Knapsack: Implementation (Cont…)


41

numbest = 0;maxprofit = 0;knapsack(0, 0, 0);print maxprofit; // Write the maximum profit.

for (j = 1; j <= numbest; j++) // Show an optimal set of items.

print bestset[i];

Subset finding problem

We say thatrunning time of an alg. is O(f(n))

If there exists c such that for large enough n:RunningTime(n) < c·f(n)

Translation: From some point c·f(n) is an upper bound on the running time.

Why only for “large enough n”?

42


Other Problems

Do not assume that because we analyze running time asymptotically we no longer care about the constants.

If you have two algorithms with different asymptotic limits – easy to choose. (may be misleading if constants are really large).

Usually we are happy when something runs in polynomial time and not exponential time. O(n2) is much much better than O(2n).

43


LEVEL II, TERM II CSE – 243 MD. MONJUR-UL-HASAN LECTURER DEPT OF CSE, CUET EMAIL:...

Documents

Transcript of LEVEL II, TERM II CSE – 243 MD. MONJUR-UL-HASAN LECTURER DEPT OF CSE, CUET EMAIL:...