Post on 20-Feb-2020
Boolean Satisfiability!
The most fundamental NP-complete problem, and now a powerful technology
for solving many real world problems
Overview!
• CNF, SAT, 3SAT and 2SAT • Resolution and Unit Propagation • DPLL search • Conflict-driven Nogood Learning • Activity-based Search • Modelling for SAT
Conjunctive Normal Form!
• SAT solvers solve problems in – Conjunctive Normal Form (CNF)
• Boolean variable: b (true/false) or (0/1) • Literal: l variable b or its negation –b
– negating a literal: -l = -b if l = b, and -l = b if l = -b
• Clause: C disjunction or set of literals • CNF: theory T set of clauses
• Assignment: set of literals A with {b,-b} not subset: e.g. {-b1,b2} Assign b1=false, b2=true
Boolean Satisfiability (SAT)!
• This is the most basic NP-complete problem.
• SAT: Given a set of clauses T, find an assignment A such that
for each C in T
• Each clause C in T is satisfied by A
€
A ∩ C ≠ ∅
3SAT!
• 3SAT: SAT with the restriction that each clause has length at most 3.
• SAT -> 3SAT – {l1, l2, l3, l4, l5, l6} l1 ∨ l2 ∨ l3 ∨ l4 ∨ l5 ∨ l6 becomes the set of clauses
– {l1, l2, b1}, {-b1, l3, b2}, {-b2, l4, b3}, {-b3, l5, l6} where b1, b2, b3 are new Boolean variables
• 3SAT is NP-complete (by the above reduction)
2SAT!
• 2SAT: SAT with the restriction that each clause has length at most 2.
• 2SAT is decidable in polynomial time (n3)
• 2SAT is NL-complete – which is a crazy complexity class – Nondeterministic Turing machine with log writeable
memory!
Resolution!
• The fundamental inference for CNF – {l1, l2, l3, b} {-b, l4, l5, l6} implies – {l1, l2, l3, l4, l5, l6}
• Or – (l1 ∨ l2 ∨ l3 ∨ b) ∧ (-b ∨ l4 ∨ l5 ∨ l6) -> l1 ∨ l2 ∨ l3 ∨ l4 ∨ l5 ∨ l6
• One can prove unsatisfiability of a CNF formula by repeatedly applying all possible resolutions – if this generates an empty clause then UNSAT – otherwise SAT
• Exponential process!
Unit Propagation!
• Resolution = too expensive • Unit Propagation (=unit resolution)
– Restricted form of resolution = finds unit clauses • {l1, l2, …, ln, l} where –li in A forall 1 ≤ i ≤ n • Add l to assignment A
• {-b1, b2, b} A = {b1, -b2} – {-b1, b2, b} A := {b1, -b2, b}
• (-b1 ∨ b2 ∨ b) ∧ b1 ∧ -b2 -> b
Unit Propagation!
• Repeatedly apply unit propagation, – until no new unit consequences can be found – or failure detected
• A = {b1, -b2, b3, -b4} C = {-b1, -b3, b4} • {-b1, -b3, b4}
• Failure detected
Unit Propagation Example!• T = {{-e11,-e21}, {-e11,-e31},{-e11,-e41}, {e21,-b21}, {e31,-b31},{e41,-b41}, {b21,-b51},{b51,-b52,e52},{b41,-e52},
• A = {e11,b52} {-e11,-e21}
A ={e11,b52,-e21} {-e11,-e31}
A ={e11,b52,-e21,-e31} {-e11,-e41}
A ={e11,b52,-e21,-e31,-e41} {-e21,-b21}
A ={e11,b52,-e21,-e31,-e41,-b21} {-e31,-b31}
A ={e11,b52,-e21,-e31,-e41,-b21,-b31} {-e41,-b41}
A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41} {b21,-b51}
A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41,-b51} {b51,-b52,e52}
A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41,-b51,e52}
e11
-e21
-e31
-e41
-b21
-b31
-b41
-b51
b52
e52
fail
{b41,-e52} A = false
Implication Graph!• Records the reason why each Boolean literal
became true – Decision – Propagation(and why)
• Used for nogood reasoning!
e11
-e21
-e31
-e41
-b21
-b31
-b41
-b51
b52
e52
fail
Unit Propagation Exercise!• T = {{-b1,-b2,b3},{-b4,b5},{b2,-b5}, {b2,b7}, {b5,-b11,b9},{b4,-b9},{-b1,-b8,b10}, {-b5,-b10},{b5,b6,-b10} ,{-b8,b11}}
• A = {b1,-b3,b8} • Perform unit propagation and draw the implication
graph
Implication Graph!
e11
-e21
-e31
-e41
-b21
-b31
-b41
-b51
b52
-e52
DPLL search!• Davis-Putnam-Logemann-Loveland algorithm
– interleave decisions + unit propagation • dpll(A)
A' = unitprop(A) if A' == false return false else if exists Boolean variable b not appearing in A if dpll(A' union {b}) return true else if dpll(A' union {-b}) return true else return false else return true
DPLL search with nogood learning!• dpll(A)
A' = unitprop(A) if A' == false Add a clause C explaining the failure to T Backjump to the first place C can give new information else if exists Boolean variable b not appearing in A if dpll(A' union {b}) return true else if dpll(A' union {-b}) return true else return false else return true
Nogood Learning!• The implication graph shows a reason for failure • Any cut separating the fail node from decisions
– explains the failure
e11
-e21
-e31
-e41
-b21
-b31
-b41
-b51
b52
e52
fail
e52 ∧ -b41 false {b41,-e52} b52 ∧ -b51 ∧-b41 false {b41,b51,-b52} b52 ∧ -b51 ∧-e41 false {e41,b51,-b52} b52 ∧ e11 false {-e11,-b52}
Which Nogood?!
• SAT solvers almost universally use the – First Unique Implication Point (1UIP) Nogood
• Closest nogood to failure – only one literal from the last decision level
• Asserting: on backjump it will unit propagate • General: more general than many other choices • Fast: doesn’t require too much computation
– replace last literal in nogood until only one at last level
1UIP Nogood Creation!
e11
-e21
-e31
-e41
-b21
-b31
-b41
-b51 b52 e52
b22 e22
-e32
-e42
-b32
-b42
b33
b43
e33
e43 fail
e33 ∧ e43 false {-e33,-e43}
e43
e33
-b42 ∧ b43 ∧ e33 false {b42,-b43,-e33}
b43
{-b32,-b42,b33,b43} false {b32,b42,-b33,-b43}
b33
{-b21,-b32,-b42,b33} false {b21,b32,b42,-b33}
-b32
-b21
-b42
{-b21,-b32,-b42} false {b21,b32,b42} {-b21,-b41,-e42,-b32} false {b21,b41,e42,b32}
-e42 -b41
{-b21,-b31,-b41,-e32,-e42} false {b21,b31,b41,e32,e42}
-e32 -b31
{-b21,-b31,-b41,e22,-e32} false {b21,b31,b41,-e22,e32}
e22
{-b21,-b31,-b41,e22} false {b21,b31,b41,-e22}
1 UIP Nogood
Backjumping!
• Backtrack to second last level in nogood
• Nogood will propagate • e.g. {b21,b31,b41,-e22}
e11
-e21
-e31
-e41
-b21
-b31
-b41
-b51
-e22
{b21,b31,b41,-e22}
Continue unit propagation then make next choice
-b22
-b52
Why Add Nogoods!
• We will not make the same choices again – {e11,b52} leads to failure – After choosing e11 we infer –b52 – Better yet, any choice that leads to b21,b31,b41
prevents the choice b52 • Drastic reduction in search space • Faster solving
1UIP Exercise!• T = {{-b1,-b2,b3},{-b4,b5},{b2,-b5}, {b2,b7}, {b5,-b11,b9},{b4,-b9},{-b1,-b8,b10}, {-b5,-b10},{b5,b6,-b10},{-b8,b11}}
• A = {b1,-b3,b8} • Assume the order of the decisions was
b8, -b3, b1 Determine the 1UIP nogood Where does execution backjump to?
Activity!
• Each time a Boolean variable is seen during nogood creation, i.e. – appears in final nogood, or – is eliminated in the reverse propagation
• Increase activity by one • These variables are helping cause failure • Periodically divide all activities by some amount
– activity reflects helping cause recent failure
Activity-based Search!• Select the unfixed variable with highest activity • MiniSat: set to false • RSAT: set to the last value it took
– works with backjumping to recreate the same path – e.g. -b1, b3, -b11, b4, -b5, b7, fail – backjump to -b1, b3, -b11 – If b4 is now highest activity variable set it true [b4] – If b5 is next highest activity variable set it false [-b5]
• Activity-based search – concentrates on the variables causing failure – learns shorter nogoods by failing earlier
Activity-based Search!
• Works well with restart • On restart we concentrate on the now most active
variables – a new part of the search space – learn new nogoods about this
Modern SAT Solvers!
• Modern SAT solvers can handled problems with – (low) millions of clauses – millions of variables assuming the input has structure
• Random CNF is much harder (but uninteresting) • Before the advent of nogood learning
– (low) thousands of clauses – hundreds of variables
Modelling for SAT!
• Boolean constructs can all be converted to clauses • b1 = b2∨b3
– {-b2,b1}, {-b3,b1}, {b2,b3,-b1} • b1 = b2∧b3
– {-b1,b2}, {-b1,b3}, {b1,-b2,-b3} • b1 = (b2b3)
– {-b2,b3,b1}, {b2,-b3,b1}, {b2,b3,-b1}, {-b2,-b3,-b1} • etc.
Modelling Integers in SAT!
• Two main representations: i in 0..m (m = 2k-1 – 1) • Binary: i = 2k-1ik-1 + 2k-2ik-2 + … + 2i1 + i0 • Unary: i = im + im-1 + … + i1
– where ik ik-1 • Example representations
Integer Binary Unary 11 1011 0000011111111111 0 0000 0000000000000000 6 0110 0000000000111111
Pseudo-Boolean Constraints!• Psuedo-Boolean constraints: bi Boolean
• Cardinality constraints:
• Note that is b1 ∨ … ∨ bm • Straight clausal definitions are exponential • Encode: Binary Decision Diagram, Binary or
Unary arithmetic • Examine cardinality constraint encodings
€
bii=1
m
∑ ≤ k
€
bii=1
m
∑ = k
€
bii=1
m
∑ ≥ k
€
bii=1
m
∑ ≥ 1
€
aibii=1
m
∑ ≤ k
€
aibii=1
m
∑ = k
€
aibii=1
m
∑ ≥ k
Cardinality Constraints (BDD)!
a ⇔ if b then c else d
{-a,-b,c}, {-a,b,d}, {a,-b,-c}, {a,b,-d}
⇔ if then else
Cardinality Constraints (BDD)!
⇔ if then else
⇔ if then else
⇔ if then else
⇔ if then else
Cardinality Constraints (BDD)!
true
x1 = 0 x1 = 1
€
xi ≤ 1i=3
5
∑
x2
x3
€
x4 + x5 ≤ 0
€
x5 ≤ 0
€
x4 + x5 ≤ 1
• Dashed arc: xi = 0 • Full arc: xi = 1 • Number of intermediate states is (n – k) ×k
x4
x5
x1
BDD Exercise!
• Draw the BDD for the cardinality constraint
€
xi = 2i=1
4
∑
Cardinality Constraints (binary)!
∑bi
ϕ1 ∧ ϕ2 ∧ ϕ3 ⇔ ∑bi =(sk-1,…,s0)
s2 s1 s0
Sum(n)
ϕ
Cardinality Constraints (binary)!
Sum(n)
Sum(n/2)
ϕ1
ϕ1 ∧ ϕ2 ∧ ϕ3 ⇔ ∑bi =(sk-1,…,s0)
Sum(n/2)
ϕ2
s2 s1 s0
Binary adder ϕ3
Binary adder ϕ3 ∑bi
Binary Adder!
FA FA
a2 b2 a1 b1
C2 C1 Cin
y3 y2 y1
{y2,-a2,-b2,-c1}, {y2,-a2,b2,c1},{y2,a2,-b2,c1}, {y2,a2,b2,-c1}, {-y2,a2,b2,c1}, {-y2,a2,-b2,-c1}, {-y2,-a2,b2,-c1}, {-y2,-a2,-b2,c1}, {y3,-a2,-b2}, {y3,-a2,-c1}, {y3,-b2,-c1}, {-y3,a2,b2}, {-y3,a2,c1}, {-y3,b2,c1}
Cardinality Constraints (binary)!
∑bi
s2 s1 s0
Sum(n)
ϕ ∑bi ≤4
s2 → ¬s1 ∧¬s0 (s2,s1,s0) ≤4
{-s2,-s1}, {-s2,-s0} How do you encode ∑bi ≤5
Sorting Networks!
A Comparator is a device with two inputs, a and b, and two outputs, c and d
comparator a b
c=max(a,b) d=min(a,b)
A Sorting network is a composition of wires and comparators that sorts its inputs
b1 a1 a2 b2
b3 a3 a4 b4
CNF of comparator {-a,c}, {-b,c}, {-a,-b,d}, {a,-d}, {b,-d}, {a,b,-c}
Sorting Networks!
There are “handcrafted” sorting networks
And there are sorting networks based on your favorite sorting algorithm
x1 x2 x3 x4
x8
y1 y2 y3
x5 x6 x7
y5 y6 y7 y8
x9 y9 Custom sorting network, n=9
x1 x2 x3 x4
y1 y2 y3
x5 x6
y5 y6
y4
y4
“Bubble sort” network
163245
613245
631245
632145
632415
654321
Odd-Even (OE) Sorting Networks!
x1 x2 x3 x4 x5 x6 x7 x8
y1 y2 y3 y4 y5 y6 y7 y8
Sorter(n) Sorter(n/2)
Sorter(n/2) OE-
Mer
ger(
n)
Sorter(n/4)
OE-
Mer
ger(n
/2)
OE-
Mer
ger(n
/2)
Sorter(n/4)
Sorter(n/4)
Sorter(n/4)
Odd-Even Merger!
OE-
Mer
ger(4
)
x1 x2 x3 x4 x5 x6 x7 x8
y1 y2 y3
y5 y6 y7 y8
y4
Given two sorted input sequences
sorted
sorted
Merge the odd input sequences
Merge the even input sequences
OE Merger(8)
OE-
Mer
ger(4
)
Odd-Even Merger!
x1 x2 x3 x4 x5 x6 x7 x8
y1 y2 y3
y5 y6 y7 y8
y4
Given two sorted input sequences
sorted
sorted
Merge the odd input sequences
Merge the even input sequences
Combine the outputs into a sorted output
OE-
Mer
ger(4
)
OE-
Mer
ger(4
)
Sorting Networks!
• Draw the best sorting network you can for 5 inputs! Minimize levels and/or comparators
Cardinality Constraints (unary)!
• Sorting Network encodes unary sum
• Cardinality by setting output bits
0 1 1 0
1 1 0 0
1 0 1 0
1 0 1 0
1 0 1 0
x1 x2 x3 x4 x5 x6 x7 x8
y1 y2 y3 y4 Sorter(8) 0 0 0 0
€
yii=1
8
∑ ≤ 4
€
xii=1
8
∑ ≤ 4
Cardinality Constraints!
• BDD – quadratic size construction – Unit propagation on CNF enforces domain consistency
• Binary – Linear size construction – Unit Propagation on CNF does not enforce domain
consistency • Unary
– n log2 n size construction – unit propagation enforces domain consistency
Psuedo-Boolean Constraints!
• Similar to Cardinality Constraints • BDD
– exponential in size, strong propagation • Binary
– O(n log n) in size, weak propagation • Unary
– O(n log2 n) in size, medium propagation – interesting problem to find base for representation
Integer Constraints!
• BDD – can handle arbitrary formula but exponential
• Unary – addition, subtraction, multiplication by constant – multiplication is impractical
• Binary – addition, subtraction – multiplication is quadratic
Boolean Solving in MiniZinc!
• mzn –b sat invokes a SAT solver
• all constraints and variables must be Boolean • Without recursion it’s a bit hard to model
interesting problems with just MiniZinc
Example: Latin Squares!int: n; % size
array[1..n,1..n,1..n] of var bool: b;
predicate atmostone(array[int] of var bool:x) =
forall(i,j in index_set(x) where i < j)(
(not x[i]) \/ (not x[j]));
predicate exactlyone(array[int] of var bool:x) =
atmostone(x) /\ exists(x);
constraint forall(i,j in 1..n)(
exactlyone([ b[i,j,k] | k in 1..n]) /\
exactlyone([ b[i,k,j] | k in 1..n]) /\
exactlyone([ b[k,i,j] | k in 1..n]));
solve satisfy;
output [ if fix(b[i,j,k]) then
show(k) ++ if j == n then "\n" else " " endif
else "" endif | i,j,k in 1..n];
5 7 4 6 8 2 1 3 9 12 10 11 13 8 2 1 7 6 4 11 10 3 9 5 13 12 6 8 7 1 2 3 10 9 4 5 13 12 11 7 6 5 3 4 1 9 11 2 13 12 10 8 3 5 8 4 7 13 2 1 12 6 11 9 10 1 3 2 8 5 12 13 4 11 10 6 7 9 2 1 6 5 3 10 4 12 13 11 9 8 7 4 9 3 2 1 11 12 13 10 7 8 5 6 12 4 10 11 13 9 3 2 1 8 7 6 5 10 11 9 13 12 5 6 7 8 1 2 3 4 9 10 13 12 11 6 5 8 7 2 1 4 3 11 13 12 9 10 7 8 5 6 3 4 1 2 13 12 11 10 9 8 7 6 5 4 3 2 1
size 13
Summary!• Modern SAT solvers are powerful solvers
– nogood learning – activity based search – restart
• State of the art solutions to Boolean problems • Many CP problems which focus on alldifferent
and ≠ can be mapped to Boolean problems – quasi-group completion (latin squares) – graph coloring
Exercise 1: atmostone!
• There is a linear encoding of atmostone(x) – introduce a new array of sum variables s encoding s[i] = exists(j in 1..i)(x[i])
– s[1] = x[1] – if s[i] is true then x[i+1] must be false – s[i+1] can be defined in terms of s[i] and x[i+1] – its actually the sorting network circuit simplified
• Write a new predicate defn of atmostone(x) • Compare the execution speed on latin squares