Constraint Satisfaction Problems (CSPs ) C onstraint Propagation and Local Search
C ONSTRAINT S ATISFACTION P ROBLEMS, P T 2 Instructor: Kris Hauser hauserk 1.
71
CONSTRAINT SATISFACTION PROBLEMS, PT 2 Instructor: Kris Hauser http://cs.indiana.edu/~hauserk 1
-
Upload
marian-fox -
Category
Documents
-
view
216 -
download
1
Transcript of C ONSTRAINT S ATISFACTION P ROBLEMS, P T 2 Instructor: Kris Hauser hauserk 1.
1
CONSTRAINT SATISFACTION PROBLEMS, PT 2Instructor: Kris Hauser
http://cs.indiana.edu/~hauserk
2
CONSTRAINT PROPAGATION …
… is the process of determining how the constraints and the possible values of one variable affect the possible values of other variables
It is an important form of “least-commitment” reasoning
2
3
FORWARD CHECKINGWhenever a pair (Xv) is added to assignment A do:
For each variable Y not in A do:
For every constraint C relating Y to the variables in A do:
Remove all values from Y’s domain that do not satisfy C
n = number of variables d = size of initial domains s = maximum number of constraints
involving a given variable (s n-1) Forward checking takes O(nsd) time
4
FORWARD CHECKING IN MAP COLORING
WA NT Q NSW V SA T
RGB RGB RGB RGB RGB RGB RGB
R RGB RGB RGB RGB RGB RGB
R GB G RGB RGB GB RGB
R B G RB B B RGB
Empty set: the current assignment {(WA R), (Q G), (V B)}does not lead to a solution
4
5
FORWARD CHECKING IN MAP COLORING
TWA
NT
SA
Q
NSW
V
Contradiction that forward checking did not detect
WA NT Q NSW V SA T
RGB RGB RGB RGB RGB RGB RGB
R RGB RGB RGB RGB RGB RGB
R GB G RGB RGB GB RGB
R B G RB B B RGB
5
6
FORWARD CHECKING IN MAP COLORING
TWA
NT
SA
Q
NSW
V
Contradiction that forward checking did not detect
WA NT Q NSW V SA T
RGB RGB RGB RGB RGB RGB RGB
R RGB RGB RGB RGB RGB RGB
R GB G RGB RGB GB RGB
R B G RB B B RGB
Detecting this contradiction requires a more powerful constraint propagation technique
6
7
CONSTRAINT PROPAGATION FOR BINARY CONSTRAINTS
REMOVE-VALUES(X,Y)1. removed false2. For every value v in the domain of Y do
– If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied thena. Remove v from Y‘s domainb. removed true
3. Return removed
8
CONSTRAINT PROPAGATION FOR BINARY CONSTRAINTS
AC3 1. Initialize queue Q with all variables (not yet
instantiated)
2. While Q doa. X Remove(Q)b. For every (not yet instantiated) variable Y related to
X by a (binary) constraint do– If REMOVE-VALUES(X,Y) then
i. If Y’s domain = then exitii. Insert(Y,Q)
9
EDGE LABELINGWe consider an image of a scene composed of polyhedral objects such that each vertex is the endpoint of exactly three edges
9
10
EDGE LABELINGAn “edge extractor” has accurately extracted all the visible edges in the image. The problem is to label each edge as convex (+), concave (-), or occluding () such that the complete labeling is physically possible
10
11
Convex edges
Concave edgesOccludin
gedges
11
12
+
- -
++
++
+
The arrow isoriented suchthat the object is on the right ofthe occluding edge
12
13
ONE POSSIBLE EDGE LABELING
+
++
+
+
+
+
+
++
--
13
14
JUNCTION TYPES
Fork
L
T
Y
14
15
JUNCTION LABEL SETS
+ + --
-- - + +
++ +
+
+
--
--
-+
(Waltz, 1975; Mackworth, 1977) 15
16
EDGE LABELING AS A CSP
A variable is associated with each junction The domain of a variable is the label set
associated with the junction type Constraints: The values assigned to two
adjacent junctions must give the same label to the joining edge
16
17
AC3 APPLIED TO EDGE LABELINGQ = (X1, X2, X3, ...)
X1
X5
X3
X8
X12
X2
X4
17
18
+ -
+
-+- -
++
X1
X5
Q = (X1, ...)
18
AC3 APPLIED TO EDGE LABELING
19
+ -
+
-+- -
++
X1
X5
Q = (X1, ...)
19
20
+ -
+
-+- -
++
X5
Q = (X5, ...)
20
21
+
+
+
+---
-- -
+
Q = (X5, ...)
X5
X3
21
22
+
+
+
+---
-- -
+
Q = (X5, ...)
X5
X3
22
23
+
+
+
+---
-- -
+
Q = (X3, ...)
X3
23
24
+
+
+
+
+
-- - + +
++
Q = (X3, ...)
X3
X8
+
24
25
+
+
+
+
+
-- - + +
++
Q = (X3, ...)
X3
X8
+
25
26
+
+
+
+
+
-- - + +
++
Q = (X8, ...)
X8
+
26
27
+
+
- -++
+ + --
+
X12
X8
Q = (X8, ...)
27
28
COMPLEXITY ANALYSIS OF AC3
n = number of variables d = size of initial domains s = maximum number of
constraints involving a given variable (s n-1)
Each variables is inserted in Q up to d times
REMOVE-VALUES takes O(d2) time
AC3 takes O(ndsd2) = O(nsd3) time
Usually more expensive than forward checking
AC3 1. Initialize queue Q with all variables (not
yet instantiated)
2. While Q doa. X Remove(Q)b. For every (not yet instantiated) variable Y
related to X by a (binary) constraint do– If REMOVE-VALUES(X,Y) then
i. If Y’s domain = then exitii. Insert(Y,Q)
REMOVE-VALUES(X,Y)1. removed false2. For every value v in the domain of Y do
– If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then
a. Remove v from Y‘s domainb. removed true
3. Return removed
29
IS AC3 ALL THAT WE NEED?
No !! AC3 can’t detect all contradictions among
binary constraints
X
Z
YXY
XZ YZ
{1, 2}
{1, 2}{1, 2}
29
30
IS AC3 ALL THAT WE NEED?
No !! AC3 can’t detect all contradictions among
binary constraints
X
Z
YXY
XZ YZ
{1, 2}
{1, 2}{1, 2}
30
REMOVE-VALUES(X,Y)1. removed false2. For every value v in the domain of Y do
– If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then
a. Remove v from Y‘s domainb. removed true
3. Return removed
31
IS AC3 ALL THAT WE NEED?
No !! AC3 can’t detect all contradictions among
binary constraints
X
Z
YXY
XZ YZ
{1, 2}
{1, 2}{1, 2}
31
REMOVE-VALUES(X,Y)1. removed false2. For every value v in the domain of Y do
– If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then
a. Remove v from Y‘s domainb. removed true
3. Return removed
REMOVE-VALUES(X,Y,Z)1. removed false2. For every value w in the domain of Z do
– If there is no pair (u,v) of values in the domains of X and Y verifying the constraint on (X,Y) such that the constraints on (X,Z) and (Y,Z) are satisfied thena. Remove w from Z‘s domainb. removed true
3. Return removed
32
IS AC3 ALL THAT WE NEED?
No !! AC3 can’t detect all contradictions among
binary constraints
Not all constraints are binary
X
Z
YXY
XZ YZ
{1, 2}
{1, 2}{1, 2}
32
33
TRADEOFF
Generalizing the constraint propagation algorithm increases its time complexity
Tradeoff between time spent in backtracking search and time spent in constraint propagation
A good tradeoff when all or most constraints are binary is often to combine backtracking with forward checking and/or AC3 (with REMOVE-VALUES for two variables)
33
34
MODIFIED BACKTRACKING ALGORITHM WITH AC3
CSP-BACKTRACKING(A, var-domains)1. If assignment A is complete then return A2. Run AC3 and update var-domains accordingly3. If a variable has an empty domain then return
failure4. X select a variable not in A5. D select an ordering on the domain of X6. For each value v in D do
a. Add (Xv) to Ab. var-domains forward checking(var-domains, X, v, A)c. If no variable has an empty domain then
(i) result CSP-BACKTRACKING(A, var-domains)(ii) If result failure then return result
d. Remove (Xv) from A
7. Return failure
35
A COMPLETE EXAMPLE:4-QUEENS PROBLEM
1
3
2
4
32 41
X1
{1,2,3,4}
X3
{1,2,3,4}X4
{1,2,3,4}
X2
{1,2,3,4}
1) The modified backtracking algorithm starts by calling AC3, which removes no value
35
36
4-QUEENS PROBLEM
1
3
2
4
32 41
X1
{1,2,3,4}
X3
{1,2,3,4}X4
{1,2,3,4}
X2
{1,2,3,4}
2) The backtracking algorithm then selects a variable and a value for this variable. No heuristic helps in this selection. X1 and the value 1 are arbitrarily selected
36
37
4-QUEENS PROBLEM
1
3
2
4
32 41
X1
{1,2,3,4}
X3
{1,2,3,4}X4
{1,2,3,4}
X2
{1,2,3,4}
3) The algorithm performs forward checking, which eliminates 2 values in each other variable’s domain
38
4-QUEENS PROBLEM
1
3
2
4
32 41
X1
{1,2,3,4}
X3
{1,2,3,4}X4
{1,2,3,4}
X2
{1,2,3,4}
4) The algorithm calls AC3
38
39
4-QUEENS PROBLEM
1
3
2
4
32 41
X1
{1,2,3,4}
X3
{1,2,3,4}X4
{1,2,3,4}
X2
{1,2,3,4}
4) The algorithm calls AC3, which eliminates 3 from the domain of X2
X2 = 3 isincompatiblewith any of the remaining valuesof X3
39
REMOVE-VALUES(X,Y)1. removed false2. For every value v in the domain of Y do
– If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then
a. Remove v from Y‘s domainb. removed true
3. Return removed
40
4-QUEENS PROBLEM
1
3
2
4
32 41
X1
{1,2,3,4}
X3
{1,2,3,4}X4
{1,2,3,4}
X2
{1,2,3,4}
4) The algorithm calls AC3, which eliminates 3 from the domain of X2, and 2 from the domain of X3
40
41
4-QUEENS PROBLEM
1
3
2
4
32 41
X1
{1,2,3,4}
X3
{1,2,3,4}X4
{1,2,3,4}
X2
{1,2,3,4}
4) The algorithm calls AC3, which eliminates 3 from the domain of X2, and 2 from the domain of X3, and 4 from the domain of X3
41
42
4-QUEENS PROBLEM
1
3
2
4
32 41
X1
{1,2,3,4}
X3
{1,2,3,4}X4
{1,2,3,4}
X2
{1,2,3,4}
5) The domain of X3 is empty backtracking
42
43
4-QUEENS PROBLEM
1
3
2
4
32 41
X1
{1,2,3,4}
X3
{1,2,3,4}X4
{1,2,3,4}
X2
{1,2,3,4}
6) The algorithm removes 1 from X1’s domain and assign 2 to X1 43
44
4-QUEENS PROBLEM
1
3
2
4
32 41
X1
{1,2,3,4}
X3
{1,2,3,4}X4
{1,2,3,4}
X2
{1,2,3,4}
7) The algorithm performs forward checking
44
45
4-QUEENS PROBLEM
1
3
2
4
32 41
X1
{1,2,3,4}
X3
{1,2,3,4}X4
{1,2,3,4}
X2
{1,2,3,4}
8) The algorithm calls AC3
45
46
4-QUEENS PROBLEM
1
3
2
4
32 41
X1
{1,2,3,4}
X3
{1,2,3,4}X4
{1,2,3,4}
X2
{1,2,3,4}
8) The algorithm calls AC3, which reduces the domains of X3 and X4 to a single value
46
47
EXPLOITING THE STRUCTURE OF CSP
If the constraint graph contains several components, then solve one independent CSP per component
TWA
NT
SA
Q
NSW
V
47
48
EXPLOITING THE STRUCTURE OF CSP
If the constraint graph is a tree, then :1. Order the variables from the
root to the leaves (X1, X2, …, Xn)
2. For j = n, n-1, …, 2 callREMOVE-VALUES(Xj, Xi) where Xi is the parent of Xj
3. Assign any valid value to X1
4. For j = 2, …, n doAssign any value to Xj consistent with the
value assigned to its parent Xi
X
Y Z
U V
W
(X, Y, Z, U, V, W)
48
49
EXPLOITING THE STRUCTURE OF CSP
Whenever a variable is assigned a value by the backtracking algorithm, propagate this value and remove the variable from the constraint graph
WA
NT
SA
Q
NSW
V
49
50
EXPLOITING THE STRUCTURE OF CSP
Whenever a variable is assigned a value by the backtracking algorithm, propagate this value and remove the variable from the constraint graph
WA
NTQ
NSW
V
If the graph becomes a tree, then proceed as shown in previousslide
50
51
8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
RepresentationVariables Xij for i, j in {1,..,9}
Domains {1,…,9}
Constraints:
XijXik, for jk
Xij Xkj, for ik
XijXmn, for (i,j), (m,n) in same cell
52
8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
RepresentationVariables Xij for i, j in {1,..,9}
Domains {1,…,9}
Constraints:
XijXik, for jk
Xij Xkj, for ik
XijXmn, for (i,j), (m,n) in same cell
53
8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
RepresentationVariables Xij for i, j in {1,..,9}
Domains {1,…,9}
Constraints:
XijXik, for jk
Xij Xkj, for ik
XijXmn, for (i,j), (m,n) in same cell
Can we detect this using constraint propagation?
54
8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
RepresentationVariables Xij for i, j in {1,..,9}
Domains {1,…,9}
Constraints:
XijXik, for jk
Xij Xkj, for ik
XijXmn, for (i,j), (m,n) in same cell
Must detect 9-way interactions
55
8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2VariablesRi
j in {1,…,9}Ci
j in {1,…,9}Xi
j in {(1,1),…,(3,3)}
Constraints:
???
R18=3
R22=8
R32=6
R37=9
…
C15=4
C12=9 C2
8=4 C38=1
C33=8
C39=5
…
56
8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2VariablesRi
j in {1,…,9}Ci
j in {1,…,9}Xi
j in {(1,1),…,(3,3)}
Constraints:
???
R18=3
R22=8
R32=6
R37=9
…
C15=4
C12=9 C2
8=4 C38=1
C33=8
C39=5
…
X18 = (1,3)
57
8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2VariablesRi
j in {1,…,9}Ci
j in {1,…,9}Xi
j in {(1,1),…,(3,3)}
Constraints:
???
R18=3
R22=8
R32=6
R37=9
…
C15=4
C12=9 C2
8=4 C38=1
C33=8
C39=5
…
X18 = (1,3) X2
2 = (3,3)
58
8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2VariablesRi
j in {1,…,9}Ci
j in {1,…,9}Xi
j in {(1,1),…,(3,3)}
Constraints:
???
R18=3
R22=8
R32=6
R37=9
…
C15=4
C12=9 C2
8=4 C38=1
C33=8
C39=5
…
X18 = (1,3) X2
2 = (3,3)X3
2 = (2,3)X3
7 = (3,3)
59
8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2VariablesRi
j in {1,…,9}Ci
j in {1,…,9}Xi
j in {(1,1),…,(3,3)}
Constraints:
Rij=k Ck
j=I
Similar constraints between X’s and R’s, X’s and C’s
Rij Ri
k for j k
Cij Ci
k for j k
Xij Xi
k for j k
R18=3
R22=8
R32=6
R37=9
…
C15=4
C12=9 C2
8=4 C38=1
C33=8
C39=5
…
X18 = (1,3) X2
2 = (3,3)X3
2 = (2,3)X3
7 = (3,3)
60
8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2VariablesRi
j in {1,…,9}Ci
j in {1,…,9}Xi
j in {(1,1),…,(3,3)}
Constraints:
Rij=k Ck
j=I
Similar constraints between X’s and R’s, X’s and C’s
Rij Ri
k for j k
Cij Ci
k for j k
Xij Xi
k for j k
C12 = 9
R12=
{1-9}
61
8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2VariablesRi
j in {1,…,9}Ci
j in {1,…,9}Xi
j in {(1,1),…,(3,3)}
Constraints:
Rij=k Ck
j=i
Similar constraints between X’s and R’s, X’s and C’s
Rij Ri
k for j k
Cij Ci
k for j k
Xij Xi
k for j k
C22 = {1-9}C1
2 = 9
R12=
{2-9}
62
8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2VariablesRi
j in {1,…,9}Ci
j in {1,…,9}Xi
j in {(1,1),…,(3,3)}
Constraints:
Rij=k Ck
j=I
Similar constraints between X’s and R’s, X’s and C’s
Rij Ri
k for j k
Cij Ci
k for j k
Xij Xi
k for j k
C12 = 9
R12=
{2-9}
X22 = (3,3)
X32 = (2,2)
63
8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2VariablesRi
j in {1,…,9}Ci
j in {1,…,9}Xi
j in {(1,1),…,(3,3)}
Constraints:
Rij=k Ck
j=I
Similar constraints between X’s and R’s, X’s and C’s
Rij Ri
k for j k
Cij Ci
k for j k
Xij Xi
k for j k
C12 = 9
R12=2
X22 = (3,3)
X32 = (2,2)
64
2 8
5 8
9
3
2
2
9
5
6
9
2
7
7
4
Representation 2VariablesRi
j in {1,…,9}Ci
j in {1,…,9}Xi
j in {(1,1),…,(3,3)}
Constraints:
Rij=k Ck
j=I
Similar constraints between X’s and R’s, X’s and C’s
Rij Ri
k for j k
Cij Ci
k for j k
Xij Xi
k for j k
C12 = 9
R12=2
X22 = (3,3)
X32 = (2,2)
65
LOCAL SEARCH FOR CSPS
Init: Make an arbitrary assignment Repeat: Modify some variable to reduce # of
violated constraints
66
BOOLEAN SATISFIABILITY PROBLEMS
Highly successful local search algorithms WalkSAT
See R&N 7.3
p constraints of form ui* uj* uk*= 1 where u* is either u or u
n variables ui, …, un
67
OBSERVATIONS…
If a CSP has few constraints, local search solves it quicklyRandom starting assignment not too far
from a solutionMillion-queens puzzles solved in < 1min, c.
1990 If a CSP has many constraints, local
search solves it quicklyConstraints “guide” solver to a solution (if
one exists)
68
HARD SUDOKU’S
1 . . | . . . | . . 2. 9 . | 4 . . | . 5 .. . 6 | . . . | 7 . .------+-------+------. 5 . | 9 . 3 | . . .. . . | . 7 . | . . .. . . | 8 5 . | . 4 .------+-------+------7 . . | . . . | 6 . .. 3 . | . . 9 | . 8 .. . 2 | . . . | . . 1
Human solvers: Lot of logic (deep constraint propagation)
Computer solvers: Lot of backtracking
69
HARD & EASY 3-SAT PROBLEMS Let R = # of constraints / # of
variables
As n , the fraction of hard problems reduces to 0
70
RECAP
Constraint propagation, AC3 Taking advantage of CSP structure Local search for CSPs
71
NEXT CLASS
Intro to uncertainty R&N 4.3-4, 13.1-2
