Boolean Satisfiability!

The most fundamental NP-complete problem, and now a powerful technology

for solving many real world problems

Overview!

•  CNF, SAT, 3SAT and 2SAT •  Resolution and Unit Propagation •  DPLL search •  Conflict-driven Nogood Learning •  Activity-based Search •  Modelling for SAT

Conjunctive Normal Form!

•  SAT solvers solve problems in – Conjunctive Normal Form (CNF)

•  Boolean variable: b (true/false) or (0/1) •  Literal: l variable b or its negation –b

–  negating a literal: -l = -b if l = b, and -l = b if l = -b

•  Clause: C disjunction or set of literals •  CNF: theory T set of clauses

•  Assignment: set of literals A with {b,-b} not subset: e.g. {-b1,b2} Assign b1=false, b2=true

Boolean Satisfiability (SAT)!

•  This is the most basic NP-complete problem.

•  SAT: Given a set of clauses T, find an assignment A such that

for each C in T

•  Each clause C in T is satisfied by A

€

A ∩ C ≠ ∅

3SAT!

•  3SAT: SAT with the restriction that each clause has length at most 3.

•  SAT -> 3SAT –  {l1, l2, l3, l4, l5, l6} l1 ∨ l2 ∨ l3 ∨ l4 ∨ l5 ∨ l6 becomes the set of clauses

–  {l1, l2, b1}, {-b1, l3, b2}, {-b2, l4, b3}, {-b3, l5, l6} where b1, b2, b3 are new Boolean variables

•  3SAT is NP-complete (by the above reduction)

2SAT!

•  2SAT: SAT with the restriction that each clause has length at most 2.

•  2SAT is decidable in polynomial time (n3)

•  2SAT is NL-complete –  which is a crazy complexity class –  Nondeterministic Turing machine with log writeable

memory!

Resolution!

•  The fundamental inference for CNF –  {l1, l2, l3, b} {-b, l4, l5, l6} implies –  {l1, l2, l3, l4, l5, l6}

•  Or –  (l1 ∨ l2 ∨ l3 ∨ b) ∧ (-b ∨ l4 ∨ l5 ∨ l6) -> l1 ∨ l2 ∨ l3 ∨ l4 ∨ l5 ∨ l6

•  One can prove unsatisfiability of a CNF formula by repeatedly applying all possible resolutions –  if this generates an empty clause then UNSAT –  otherwise SAT

•  Exponential process!

Unit Propagation!

•  Resolution = too expensive •  Unit Propagation (=unit resolution)

–  Restricted form of resolution = finds unit clauses •  {l1, l2, …, ln, l} where –li in A forall 1 ≤ i ≤ n •  Add l to assignment A

•  {-b1, b2, b} A = {b1, -b2} –  {-b1, b2, b} A := {b1, -b2, b}

•  (-b1 ∨ b2 ∨ b) ∧ b1 ∧ -b2 -> b

Unit Propagation!

•  Repeatedly apply unit propagation, –  until no new unit consequences can be found –  or failure detected

•  A = {b1, -b2, b3, -b4} C = {-b1, -b3, b4} •  {-b1, -b3, b4}

•  Failure detected

Unit Propagation Example!•  T = {{-e11,-e21}, {-e11,-e31},{-e11,-e41}, {e21,-b21}, {e31,-b31},{e41,-b41}, {b21,-b51},{b51,-b52,e52},{b41,-e52},

•  A = {e11,b52} {-e11,-e21}

A ={e11,b52,-e21} {-e11,-e31}

A ={e11,b52,-e21,-e31} {-e11,-e41}

A ={e11,b52,-e21,-e31,-e41} {-e21,-b21}

A ={e11,b52,-e21,-e31,-e41,-b21} {-e31,-b31}

A ={e11,b52,-e21,-e31,-e41,-b21,-b31} {-e41,-b41}

A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41} {b21,-b51}

A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41,-b51} {b51,-b52,e52}

A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41,-b51,e52}

e11

-e21

-e31

-e41

-b21

-b31

-b41

-b51

b52

e52

fail

{b41,-e52} A = false

Implication Graph!•  Records the reason why each Boolean literal

became true –  Decision –  Propagation(and why)

•  Used for nogood reasoning!

e11

-e21

-e31

-e41

-b21

-b31

-b41

-b51

b52

e52

fail

Unit Propagation Exercise!•  T = {{-b1,-b2,b3},{-b4,b5},{b2,-b5}, {b2,b7}, {b5,-b11,b9},{b4,-b9},{-b1,-b8,b10}, {-b5,-b10},{b5,b6,-b10} ,{-b8,b11}}

•  A = {b1,-b3,b8} •  Perform unit propagation and draw the implication

graph

Implication Graph!

e11

-e21

-e31

-e41

-b21

-b31

-b41

-b51

b52

-e52

DPLL search!•  Davis-Putnam-Logemann-Loveland algorithm

–  interleave decisions + unit propagation •  dpll(A)

A' = unitprop(A) if A' == false return false else if exists Boolean variable b not appearing in A if dpll(A' union {b}) return true else if dpll(A' union {-b}) return true else return false else return true

DPLL search with nogood learning!•  dpll(A)

A' = unitprop(A) if A' == false Add a clause C explaining the failure to T Backjump to the first place C can give new information else if exists Boolean variable b not appearing in A if dpll(A' union {b}) return true else if dpll(A' union {-b}) return true else return false else return true

Nogood Learning!•  The implication graph shows a reason for failure •  Any cut separating the fail node from decisions

–  explains the failure

e11

-e21

-e31

-e41

-b21

-b31

-b41

-b51

b52

e52

fail

e52 ∧ -b41 false {b41,-e52} b52 ∧ -b51 ∧-b41 false {b41,b51,-b52} b52 ∧ -b51 ∧-e41 false {e41,b51,-b52} b52 ∧ e11 false {-e11,-b52}

Which Nogood?!

•  SAT solvers almost universally use the –  First Unique Implication Point (1UIP) Nogood

•  Closest nogood to failure –  only one literal from the last decision level

•  Asserting: on backjump it will unit propagate •  General: more general than many other choices •  Fast: doesn’t require too much computation

–  replace last literal in nogood until only one at last level

1UIP Nogood Creation!

e11

-e21

-e31

-e41

-b21

-b31

-b41

-b51 b52 e52

b22 e22

-e32

-e42

-b32

-b42

b33

b43

e33

e43 fail

e33 ∧ e43 false {-e33,-e43}

e43

e33

-b42 ∧ b43 ∧ e33 false {b42,-b43,-e33}

b43

{-b32,-b42,b33,b43} false {b32,b42,-b33,-b43}

b33

{-b21,-b32,-b42,b33} false {b21,b32,b42,-b33}

-b32

-b21

-b42

{-b21,-b32,-b42} false {b21,b32,b42} {-b21,-b41,-e42,-b32} false {b21,b41,e42,b32}

-e42 -b41

{-b21,-b31,-b41,-e32,-e42} false {b21,b31,b41,e32,e42}

-e32 -b31

{-b21,-b31,-b41,e22,-e32} false {b21,b31,b41,-e22,e32}

e22

{-b21,-b31,-b41,e22} false {b21,b31,b41,-e22}

1 UIP Nogood

Backjumping!

•  Backtrack to second last level in nogood

•  Nogood will propagate •  e.g. {b21,b31,b41,-e22}

e11

-e21

-e31

-e41

-b21

-b31

-b41

-b51

-e22

{b21,b31,b41,-e22}

Continue unit propagation then make next choice

-b22

-b52

Why Add Nogoods!

•  We will not make the same choices again –  {e11,b52} leads to failure –  After choosing e11 we infer –b52 –  Better yet, any choice that leads to b21,b31,b41

prevents the choice b52 •  Drastic reduction in search space •  Faster solving

1UIP Exercise!•  T = {{-b1,-b2,b3},{-b4,b5},{b2,-b5}, {b2,b7}, {b5,-b11,b9},{b4,-b9},{-b1,-b8,b10}, {-b5,-b10},{b5,b6,-b10},{-b8,b11}}

•  A = {b1,-b3,b8} •  Assume the order of the decisions was

b8, -b3, b1 Determine the 1UIP nogood Where does execution backjump to?

Activity!

•  Each time a Boolean variable is seen during nogood creation, i.e. –  appears in final nogood, or –  is eliminated in the reverse propagation

•  Increase activity by one •  These variables are helping cause failure •  Periodically divide all activities by some amount

–  activity reflects helping cause recent failure

Activity-based Search!•  Select the unfixed variable with highest activity •  MiniSat: set to false •  RSAT: set to the last value it took

–  works with backjumping to recreate the same path –  e.g. -b1, b3, -b11, b4, -b5, b7, fail –  backjump to -b1, b3, -b11 –  If b4 is now highest activity variable set it true [b4] –  If b5 is next highest activity variable set it false [-b5]

•  Activity-based search –  concentrates on the variables causing failure –  learns shorter nogoods by failing earlier

Activity-based Search!

•  Works well with restart •  On restart we concentrate on the now most active

variables –  a new part of the search space –  learn new nogoods about this

Modern SAT Solvers!

•  Modern SAT solvers can handled problems with –  (low) millions of clauses –  millions of variables assuming the input has structure

•  Random CNF is much harder (but uninteresting) •  Before the advent of nogood learning

–  (low) thousands of clauses –  hundreds of variables

Modelling for SAT!

•  Boolean constructs can all be converted to clauses •  b1 = b2∨b3

–  {-b2,b1}, {-b3,b1}, {b2,b3,-b1} •  b1 = b2∧b3

–  {-b1,b2}, {-b1,b3}, {b1,-b2,-b3} •  b1 = (b2b3)

–  {-b2,b3,b1}, {b2,-b3,b1}, {b2,b3,-b1}, {-b2,-b3,-b1} •  etc.

Modelling Integers in SAT!

•  Two main representations: i in 0..m (m = 2k-1 – 1) •  Binary: i = 2k-1ik-1 + 2k-2ik-2 + … + 2i1 + i0 •  Unary: i = im + im-1 + … + i1

–  where ik ik-1 •  Example representations

Integer Binary Unary 11 1011 0000011111111111 0 0000 0000000000000000 6 0110 0000000000111111

Pseudo-Boolean Constraints!•  Psuedo-Boolean constraints: bi Boolean

•  Cardinality constraints:

•  Note that is b1 ∨ … ∨ bm •  Straight clausal definitions are exponential •  Encode: Binary Decision Diagram, Binary or

Unary arithmetic •  Examine cardinality constraint encodings

€

bii=1

m

∑ ≤ k

€

bii=1

m

∑ = k

€

bii=1

m

∑ ≥ k

€

bii=1

m

∑ ≥ 1

€

aibii=1

m

∑ ≤ k

€

aibii=1

m

∑ = k

€

aibii=1

m

∑ ≥ k

Cardinality Constraints (BDD)!

a ⇔ if b then c else d

{-a,-b,c}, {-a,b,d}, {a,-b,-c}, {a,b,-d}

⇔ if then else

Cardinality Constraints (BDD)!

⇔ if then else

⇔ if then else

⇔ if then else

⇔ if then else

Cardinality Constraints (BDD)!

true

x1 = 0 x1 = 1

€

xi ≤ 1i=3

5

∑

x2

x3

€

x4 + x5 ≤ 0

€

x5 ≤ 0

€

x4 + x5 ≤ 1

•  Dashed arc: xi = 0 •  Full arc: xi = 1 •  Number of intermediate states is (n – k) ×k

x4

x5

x1

BDD Exercise!

•  Draw the BDD for the cardinality constraint

€

xi = 2i=1

4

∑

Cardinality Constraints (binary)!

∑bi

ϕ1 ∧ ϕ2 ∧ ϕ3 ⇔ ∑bi =(sk-1,…,s0)

s2 s1 s0

Sum(n)

ϕ

Cardinality Constraints (binary)!

Sum(n)

Sum(n/2)

ϕ1

ϕ1 ∧ ϕ2 ∧ ϕ3 ⇔ ∑bi =(sk-1,…,s0)

Sum(n/2)

ϕ2

s2 s1 s0

Binary adder ϕ3

Binary adder ϕ3 ∑bi

Binary Adder!

FA FA

a2 b2 a1 b1

C2 C1 Cin

y3 y2 y1

{y2,-a2,-b2,-c1}, {y2,-a2,b2,c1},{y2,a2,-b2,c1}, {y2,a2,b2,-c1},  {-y2,a2,b2,c1}, {-y2,a2,-b2,-c1}, {-y2,-a2,b2,-c1}, {-y2,-a2,-b2,c1}, {y3,-a2,-b2}, {y3,-a2,-c1}, {y3,-b2,-c1},                               {-y3,a2,b2}, {-y3,a2,c1}, {-y3,b2,c1}

Cardinality Constraints (binary)!

∑bi

s2 s1 s0

Sum(n)

ϕ ∑bi ≤4

s2 → ¬s1 ∧¬s0 (s2,s1,s0) ≤4

{-s2,-s1}, {-s2,-s0} How do you encode ∑bi ≤5

Sorting Networks!

A Comparator is a device with two inputs, a and b, and two outputs, c and d

comparator a b

c=max(a,b) d=min(a,b)

A Sorting network is a composition of wires and comparators that sorts its inputs

b1 a1 a2 b2

b3 a3 a4 b4

CNF of comparator {-a,c}, {-b,c}, {-a,-b,d}, {a,-d}, {b,-d}, {a,b,-c}

Sorting Networks!

There  are  “handcrafted”  sorting  networks

And  there  are  sorting  networks  based  on  your  favorite  sorting  algorithm

x1 x2 x3 x4

x8

y1 y2 y3

x5 x6 x7

y5 y6 y7 y8

x9 y9 Custom sorting network, n=9

x1 x2 x3 x4

y1 y2 y3

x5 x6

y5 y6

y4

y4

“Bubble sort” network

163245

613245

631245

632145

632415

654321

Odd-Even (OE) Sorting Networks!

x1 x2 x3 x4 x5 x6 x7 x8

y1 y2 y3 y4 y5 y6 y7 y8

Sorter(n) Sorter(n/2)

Sorter(n/2) OE-

Mer

ger(

n)

Sorter(n/4)

OE-

Mer

ger(n

/2)

OE-

Mer

ger(n

/2)

Sorter(n/4)

Sorter(n/4)

Sorter(n/4)

Odd-Even Merger!

OE-

Mer

ger(4

)

x1 x2 x3 x4 x5 x6 x7 x8

y1 y2 y3

y5 y6 y7 y8

y4

Given two sorted input sequences

sorted

sorted

Merge the odd input sequences

Merge the even input sequences

OE Merger(8)

OE-

Mer

ger(4

)

Odd-Even Merger!

x1 x2 x3 x4 x5 x6 x7 x8

y1 y2 y3

y5 y6 y7 y8

y4

Given two sorted input sequences

sorted

sorted

Merge the odd input sequences

Merge the even input sequences

Combine the outputs into a sorted output

OE-

Mer

ger(4

)

OE-

Mer

ger(4

)

Sorting Networks!

•  Draw the best sorting network you can for 5 inputs! Minimize levels and/or comparators

Cardinality Constraints (unary)!

•  Sorting Network encodes unary sum

•  Cardinality by setting output bits

0 1 1 0

1 1 0 0

1 0 1 0

1 0 1 0

1 0 1 0

x1 x2 x3 x4 x5 x6 x7 x8

y1 y2 y3 y4 Sorter(8) 0 0 0 0

€

yii=1

8

∑ ≤ 4

€

xii=1

8

∑ ≤ 4

Cardinality Constraints!

•  BDD –  quadratic size construction –  Unit propagation on CNF enforces domain consistency

•  Binary –  Linear size construction –  Unit Propagation on CNF does not enforce domain

consistency •  Unary

–  n log2 n size construction –  unit propagation enforces domain consistency

Psuedo-Boolean Constraints!

•  Similar to Cardinality Constraints •  BDD

–  exponential in size, strong propagation •  Binary

–  O(n log n) in size, weak propagation •  Unary

–  O(n log2 n) in size, medium propagation –  interesting problem to find base for representation

Integer Constraints!

•  BDD –  can handle arbitrary formula but exponential

•  Unary –  addition, subtraction, multiplication by constant –  multiplication is impractical

•  Binary –  addition, subtraction –  multiplication is quadratic

Boolean Solving in MiniZinc!

•  mzn –b sat invokes a SAT solver

•  all constraints and variables must be Boolean •  Without recursion it’s a bit hard to model

interesting problems with just MiniZinc

Example: Latin Squares!int: n; % size

array[1..n,1..n,1..n] of var bool: b;

predicate atmostone(array[int] of var bool:x) =

forall(i,j in index_set(x) where i < j)(

(not x[i]) \/ (not x[j]));

predicate exactlyone(array[int] of var bool:x) =

atmostone(x) /\ exists(x);

constraint forall(i,j in 1..n)(

exactlyone([ b[i,j,k] | k in 1..n]) /\

exactlyone([ b[i,k,j] | k in 1..n]) /\

exactlyone([ b[k,i,j] | k in 1..n]));

solve satisfy;

output [ if fix(b[i,j,k]) then

show(k) ++ if j == n then "\n" else " " endif

else "" endif | i,j,k in 1..n];

5 7 4 6 8 2 1 3 9 12 10 11 13 8 2 1 7 6 4 11 10 3 9 5 13 12 6 8 7 1 2 3 10 9 4 5 13 12 11 7 6 5 3 4 1 9 11 2 13 12 10 8 3 5 8 4 7 13 2 1 12 6 11 9 10 1 3 2 8 5 12 13 4 11 10 6 7 9 2 1 6 5 3 10 4 12 13 11 9 8 7 4 9 3 2 1 11 12 13 10 7 8 5 6 12 4 10 11 13 9 3 2 1 8 7 6 5 10 11 9 13 12 5 6 7 8 1 2 3 4 9 10 13 12 11 6 5 8 7 2 1 4 3 11 13 12 9 10 7 8 5 6 3 4 1 2 13 12 11 10 9 8 7 6 5 4 3 2 1

size 13

Summary!•  Modern SAT solvers are powerful solvers

–  nogood learning –  activity based search –  restart

•  State of the art solutions to Boolean problems •  Many CP problems which focus on alldifferent

and ≠ can be mapped to Boolean problems –  quasi-group completion (latin squares) –  graph coloring

Exercise 1: atmostone!

•  There is a linear encoding of atmostone(x) –  introduce a new array of sum variables s encoding s[i] = exists(j in 1..i)(x[i])

–  s[1] = x[1] –  if s[i] is true then x[i+1] must be false –  s[i+1] can be defined in terms of s[i] and x[i+1] –  its actually the sorting network circuit simplified

•  Write a new predicate defn of atmostone(x) •  Compare the execution speed on latin squares