Probability in the Analysis of CNF Satisfiability ...
31
Probability in the Analysis of CNF Satisfiability Algorithms and Properties John Franco Computer Science, University of Cincinnati
Transcript of Probability in the Analysis of CNF Satisfiability ...
Probability in the Analysis
of CNF Satisfiability
Algorithms and Properties
John FrancoComputer Science, University of Cincinnati
Why is it Worthwhile?
The Questions:
• Why are some problems so difficult?
• Are there algorithms that will make them easier?
Why Probability?
• Results and process tend to draw out intuition
− Identify properties that may be exploited by a fastalgorithm and properties that may prevent exploitation.
− Identify reasons for the hardness of various problems -why lots of instances are hard.
• Can explain the good or bad behavior of an algorithm
• Afford comparison of incomparable classes of formulas
• De-randomization may yield fast algorithms
TermsA variable looks like this: v9, takes a value from {0, 1}
A positive literal: v9, a negative literal: ¬v9
A clause looks like this: (¬v1 ∨ ¬v2 ∨ v5 ∨ v9)
An instance of SAT looks like this:(v1 ∨ ¬v2 ∨ v7) ∧ (¬v2 ∨ v6) ∧ (¬v2 ∨ ¬v4 ∨ ¬v5) ∧ (v10)...
Clause width: # literals; k-SAT: fixed width k
An assignment of values satisfying F is a model for F
An important splitting operation in solvers:
HHHHHj
������
tdddcc
tdddcctdddcc
(¬v1 ∨ ¬v2 ∨ vi) ∧ (¬vi ∨ v3) ∧ (¬v2 ∨ v4)
vi1 0
(v3) ∧ (¬v2 ∨ v4) (¬v1 ∨ ¬v2) ∧ (¬v2 ∨ v4)
Some problems
• Must assume an input distribution,often not reflecting reality(but sometimes we do not need to)
• Analysis can be difficult or impossible -algorithmic steps may significantly change distribution(known tools are limited)
• Can yield misleading results
A Misleading Result
Example: A probabilistic model for random formulas
Given: set L = {v1,¬v1, ..., vn,¬vn} of literals and 0 < p < 1
Construct clause c: l ∈ L independently in c with probability p
Construct formula: m independently constructed clauses
Justification: All formulas are equally likely if p = 1/3.
Applied to splitting (DPLL):
HHHHHj
������
tdddcc
tdddcctdddcc
(¬v1 ∨ ¬v2 ∨ vi) ∧ (¬vi ∨ v3) ∧ (¬v2 ∨ v4)
vi1 0
(v3) ∧ (¬v2 ∨ v4) (¬v1 ∨ ¬v2) ∧ (¬v2 ∨ v4)
A Misleading Result
Analysis sketch:
Given m clauses, the average number of clauses removedwhen a value is assigned is pm
Let Ti be the average number of clauses remainingon the ith iteration
Then T0 = m and Ti = (1 − p)Ti−1
For what i does Ti = 1? When 1 = m(1 − p)i orlg(m) = −i lg(1 − p)
i = lg(m)/ − lg(1 − p)
So, size of search space is 2i ≈ 2lg(m)/p = mc if p = 1/3
Conclusion: DPLL is fantastic, on the average!
A Misleading Result
Problems with the analysis:
• The input model is funky!The probability that a random assignment satisfiesa random formula is
(1 − (1 − p)2n)m ≈ e−me−2pn
which tends to 1 if ln(m)/pn = o(1) and means theaverage width of a clause can be at least ln(m).With p = 1/3, this holds if n > ln(m).
• If average clause width is constant (p = c/n)then the average search space size is
2− lg(m)/ lg(1−p) ≈ 2lg(m)/(c/n) = 2n lg(m)/c = mn/c
Exponential complexity!
Probabilistic Toolbox
Linearity of expectation:
E{∑
i
Xi} =∑
i
E{Xi}, Xi real valued r.v.s
First Moment Method: show Pr(P ) → 0 as n → ∞.
Let X be a count of some entitySuppose some property P holds if and only if X ≥ 1.
ThenPr(X > 1) ≤ E{X}
So, if E{X} → 0 then Pr(P ) → 0, as n → ∞.
First Moment Method
An Example:
Assume F is a random k-SAT formula, m clauses, n variables
Let P be the property that F has a model
Let X be the number of models for F
Let Xi =
{
1 if the ith assignment is a model for F0 otherwise
Pr(Xi = 1) = (1 − 2−k)m
E{X} =2n∑
i=1
Pr(Xi = 1) = 2n(1 − 2−k)m
Pr(F has a model) → 0 ifm
n>
1
lg(1 − 2−k)≈ 2k
For k = 3 F has no model w.h.p. ifm
n> 5.19
Probabilistic Toolbox
Flow Analysis:
Consider straight-line (non-backtracking) variants of DPLL
Let F be a CNF Boolean expressionSet M = ∅Repeat the following:
Choose an unassigned literal l in FIf l is a positive literal, set M = M ∪ {l}Set F = {c \ {¬l} : c ∈ F , l /∈ c}If F is satisfied then return MIf some clause in F is falsified return "give up"
When does this not give up with probability tending to 1?
Answer depends on the way literals are chosen
Flow Analysis
Ck(j) -
?
zk(j)
wk−1(j)
Ck−1(j) -
?
zk−1(j)
wk−2(j)
. . .w2(j)?
C2(j) -
?
z2(j)
w1(j)
C1(j) -
?
z1(j)
w0(j)
E{m4(j)}
j/n-0 1/4 1/2 3/4 1
m
0
E{m3(j)}
j/n-0 1/4 1/2 3/4 1
m
0
E{m2(j)}
j/n-0 1/4 1/2 3/4 1
m
0
Flow Analysis
Example:
Unit clause heuristic: When there is a clause with oneunassigned variable remaining, set the value of such avariable so as to satisfy its clause.
Intuitively: If the clause flow w1(j) < 1 then anyaccumulation of unit clauses can be prevented andno clauses will ever be eliminated
Analysis:
Write difference equations describing flows:
mi(j + 1) = mi(j) + wi(j) − wi−1(j) − zi(j), ∀ 0 ≤ i ≤ k, 1 < j < n
Take expectations and rearrange:
E{mi(j + 1) − mi(j)} = E{wi(j)} − E{wi−1(j)} − E{zi(j)}
Flow AnalysisCompute the expectations:
E{zi(j)} = E{E{zi(j)|mi(j)}} = E
{
i · mi(j)
2(n − j)
}
=i · E{mi(j)}
2(n − j)
E{wi(j)} = E{E{...}} = E
{
(i + 1)mi+1(j)
2(n − j)
}
=(i + 1)E{mi+1(j)}
2(n − j)
Substitute into difference equations:
E{mi(j + 1) − mi(j)} =(i + 1)E{mi+1(j)}
2(n − j)− i · E{mi(j)}
n − j, i < k
E{mk(j + 1) − mk(j)} = −k · E{mk(j)}n − j
Switch to differential equations (use x for j/n):
dm̄i(x)
dx=
(i + 1)m̄i+1(x)
2n(1 − x)− i · m̄i(x)
n − j; m̄k(0) = m/n, m̄i(0) = 0 for i < k
Solve: Translation:
m̄i(x) =1
2k−i
(m
n
)
(
k
i
)
(1 − x)ixk−i E{mi(j)} =m
2k−i
(
k
i
)(
1 − j
n
)i( j
n
)k−i
Flow AnalysisThe important flow:
E{w1(j)} =E{m2(j)}(n − j)
=1
2k−2
(
k
2
)(
1 − j
n
)2( j
n
)k−2
m
Find location of maximum (set derivative = 0):
j∗ =
(
k − 2
k − 1
)
n
So,
E{w1(j∗)} < 1 when
m
n<
2k−1
k
(
k − 1
k − 2
)k−1
for k = 3 this ism
n<
8
3
Conclusion: unit clause heuristic succeeds with probability
bounded by a constant whenm
n<
8
3.
Flow Analysis
What makes this work?
The clausal distribution is the same, up to parameters m and n,at each level (algorithm is myopic - no information is revealed)
There is pretty much never a sudden spurious flowPr(||Ci(j + 1)| − |Ci(j)|| > n0.2) = o(n−3)
The average flow change is pretty smoothE{|Ci(j + 1)| − |Ci(j)|} = fi(j/n, |C0(j)|/n, ..., |Ck(j)|/n) + o(1),
fi is continuous and
|fi(u1, ..., uk+2) − fi(v1, ..., vk+2)| ≤ L∑
1≤i≤j
|ui − vi|
Flow Analysis
good when m/n <
literal selection k-SAT 3-SAT
Choose from unit clause, otherwise randomly 2k/k 2.66
Choose var with maximum difference betweenoccurrences of positive and negative lits. Setvalue to maximize satisfied clauses
c · 2k/k 3.003
Choose randomly from a clause with fewestnon-falsified literals
1.125 · 2k/k 3.09
Best possible myopic algorithm c · 2k/k 3.26
literal selection, non-myopic algorithmsGreedy algorithm: maximize number of clausessatisfied, eliminate unit clauses when they exist
c · 2k/k? 3.52
Maximize the expected number of models ofthe reduced instance
c · 2k/k? > 3.6?
Flow Analysis
Pr(success)
1
0
0.46352k
k + 1
(
k − 1
k − 2
)k−2
0.77252k
k + 1
(
k − 1
k − 2
)k−2
m
n⇒
Typical probability curve of these algorithms
Is2k
kthe crossover to unsatisfiability or is it 2k?
Probabilistic Toolbox
Second Moment Method: show Pr(P ) → 1 as n → ∞Let P be some property of a formula
A witness for P : a structure whose presence in F implies P
Let W = {w : w is a witness for P in F}
Let Xi =
{
1 if structure si ∈ W
0 otherwise
Let X =∑
i
Xi and E{Xi} = q, then E{X} ∆= µ = q|W |
Suppose var(X) ∆= σ2 = o(µ2). Then, since
Pr(X = 0) ≤ σ2
µ2
we get
Pr(P ) → 1 as n → ∞
Second Moment MethodTo show σ2 = o(µ2):
Start with one witness w chosen arbitrarily
Let Aw be all witnesses sharing at least one clause with w
Let Dw be all witnesses sharing no clause with w. Then
σ2 = µ(1 − q) + µ
∑
z∈Aw
(Pr(z|w) − q) +∑
z∈Dw
(Pr(z|w) − q)
Need |Aw| ≪ |Dw| or “little” overlap among witnesses since
Pr(z|w) = Pr(z) = q if z ∈ Dw so∑
z∈Dw(Pr(z|w) − q) = 0.
If µ → ∞ and∑
z∈Aw
Pr(z|W ) → o(µ) as n → ∞ then
Pr(P ) → 1 as n → ∞
Where is the Crossover?
Cannot apply the second moment method directly to k-SAT- variance of the number of models is too highthe reason: the asymmetry of k-SAT
Let z and w be assignments agreeing in αn variables
Pr(z satisfies F | w satisfies F) =
(
1 − 1 − αk
2k − 1
)m
Variance gets too big, maximum occurs at α 6= 1/2
∑
z∈Aw
Pr(z|w) =∑
0<α≤1
( n
αn
)
(
1 − 1 − αk
2k − 1
)m
Where is the Crossover?
Analyze a different problem: Not All Equal k-SATA NAE model: one for which every clause hasat least one true and at least one false literal
Pr(∃ a model for F) > Pr(∃ a NAE-model for F)
Let X = number of models, XNAE = number of NAE models
Pr(XNAE = 0) > Pr(X = 0)
Pr(z NAE-satisfies F | w NAE-satisfies F) =
(
1 − 1 − αk − (1 − α)k
2k−1 − 1
)m
Variance is low, maximum term occurs at α = 1/2.
∑
0<α≤1
( n
αn
)
(
1 − 1 − αk − (1 − α)k
2k−1 − 1
)m
≈ 2n(1 − 21−k)m√n
≈ o(µ)
Pr(XNAE = 0) <1
µNAE
≈ 1
2n(1 − 2k−1)m→ 0 if
m
n>
1
lg(1 − 2k−1)≈ 2k−1
What About Easy Classes?
Examples:Horn, Hidden Horn, 2-SAT, Extended Horn, q-Horn,CC-balanced, SLUR, Matched, Linear Autarkies
Horn:every clause has at most one positive literalsolved in linear time by unit clause algorithm
Probabilistic analysis of polytime solvable classes can reveal:
• What critically distinguishes an easy classfrom more difficult classes
• Whether one class is much larger than anotherincomparable class in a probabilistic sense
Polynomial Time Solvable Classes
Example: q-Hornvariables
Horn
clauses
non-positive
-1 0 1 0 0 0 0 0 0 0 0
1 -1 -1 0 1 0 0 0 0 0 0
0 0 -1 -1 0 0 0 0 0 0 0
0 1 -1 0 0 0 0 0 0 0 0
0 -1 -1 0 0 0 0 1 0 1 0
-1 0 0 -1 0 1 0 0 -1 0 0
0 0 0 -1 0 0 0 0 0 -1 1
0 0 -1 -1 0 0 -1 0 0 0 0
Zero
2-SAT
If the satisfiability index of a given formula is no greaterthan 1, then the formula is q-Horn.
The class of formulas with a satisfiability index greater than1 + n−ǫ, for any ǫ, is NP-complete.
Polynomial Time Solvable Classes
Vulnerability of q-Horn to particular cycles
. . . (u2 ∨ ¬u1 ∨ ...) (u1 ∨ ¬v0 ∨ ¬u3p ∨ ...) (u3p ∨ ¬u3p−1 ∨ ...) . . .
. . . (¬up−1 ∨ up ∨ ...) (¬up ∨ ¬v0 ∨ up+1 ∨ ...) (¬up+1 ∨ up+2 ∨ ...) . . .
u1
up
v0
up+1
u3p
Pr(random formula is of specified class)
class as n → ∞ k-SAT 3-SAT
Horn 0 m > ǫ m > ǫ
Hidden Horn 0 m/n > 1/(k − lg(k + 1)) m/n > 1
q-Horn 0 m/n > 4/(k2 − k) m/n > 0.66
SLUR 0 m/n > 4/(k2 − k) m/n > 0.66
Matched 1 m/n < ck, ck → 1 m/n < 0.64
No Cycles 1 m/n < 1.36/(k2 − k) m/n < 0.226
Algorithms for Unsatisfiability
Resolution performs badly on random unsatisfiable formulas
Pr(random k-SAT formula is unsatisfiable) → 1 if mn > 2k
Pr(resolution does well) → 1 only if mn >
(
nlg(n)
)k−2
Are there better alternatives?Must avoid getting stuck on “sparse” nature of formulas
One possibility: Hitting Set
Focus attention on clauses which are all positive or negative
Let n+ = min number of 1 valued variables to satisfy positives
Let n− = min number of 0 valued variables to satisfy negatives
If n+ + n− > n then some variable must be set to 1 AND 0
That is impossible, conclude the formula is unsatisfiable
Hitting Set
Construct graphs G+, G−
Vertices are labeled as pairs of variables
Edge 〈a, b〉 ⇔ some clause contains all variables labeling a, b
(v1 ∨ v2 ∨ v3 ∨ v4) ∧ (v2 ∨ v3 ∨ v4 ∨ v5) ∧ (v1 ∨ v3 ∨ v4 ∨ v5)
v1, v2
yiihhg
v3, v4
yiihhg
AAAAAA
��
��
��
v2, v5
yiihhg
v1, v5
yiihhg
v2, v4
yiihhg
v1, v3
yiihhg
AAAAAA
��
��
��
v4, v5
yiihhg
v2, v3
yiihhg
AAAAAA
��
��
��
v1, v4
yiihhg
v3, v5
yiihhg
AAAAAA
Hitting Set
Construct graphs G+, G−
Vertices are labeled as pairs of variables
Edge 〈a, b〉 ⇔ some clause contains all variables labeling a, b
(v1 ∨ v2 ∨ v3 ∨ v4) ∧ (v2 ∨ v3 ∨ v4 ∨ v5) ∧ (v1 ∨ v3 ∨ v4 ∨ v5)
v1, v2
yiihhg
v3, v4
yiihhg
AAAAAA
��
��
��
v2, v5
yiihhg
v1, v5
yiihhg
v2, v4
yiihhg
v1, v3
yiihhg
AAAAAA
��
��
��
v4, v5
yiihhg
v2, v3
yiihhg
AAAAAA
��
��
��
v1, v4
yiihhg
v3, v5
yiihhg
AAAAAA
For k = 4, if a model exists, one graph has |IS| >n2
8
Hitting Set
Construct matrices M+,M−
Columns and rows are indexed on vertices
Mi,j =
−1−pp if there is an edge between vertices i and j
1 otherwise
where p represents a probability that can be adjusted
Exploit relationship between maximum eigenvalue and maximum IS
α(G+) < λ1(M+) and α(G−) < λ1(M
−)
For purposes of proving a bound
If p =ln7(n′)
n′then max
i{|λi(M)|} =
2n′
ln3.5(n′)(1 + o(1)) w.h.p.
This leads to Pr(HS does well) → 1 ifm
n> n(k−2)/2
Recall Pr(resolution does well) → 1 only ifm
n> nk−2.
A De-randomized Algorithm for MAXSAT
MAX k-SAT
Given: A CNF formula F with k literals per clause
Find: An assignment to the variables of F thatsatisfies a maximum number of its clauses.
Suppose F has variables v1, v2, . . . , vn. Define indicators
Ait1,...,tj
=
{
1 if clause i satisfied given v1 = t1, ..., vj = tj0 otherwise
What is the probability that Ait1,...,,tj
= 1 for random tj+1, ..., tn?
for example: Pr((¬v1 ∨ v3 ∨ ¬v5) = 1 | t1 = 1, t2 = 0) = Pr(Ai1,0) = 3/4
A De-randomized Algorithm for MAXSAT
Let N be the number of satisfied clauses in F. Then
E{N} =n∑
i=1
Pr(Ai) = (1 − 2−k)m
and
Pr(Ait1,...,tj−1
) = (Pr(Ait1,...,tj−1,1) + Pr(Ai
t1,...,tj−1,0))/2
≤ max {Pr(Ait1,...,tj
)}
so keep choosing a value tj, for j = 1, 2, ... that maximizesPr(Ai
t1,...,tj) to get
E{N} = (1 − 2−k)m =
n∑
i=1
Pr(Ai) ≤n∑
i=1
max {Pr(Ait1
)}...
≤n∑
i=1
max {Pr(Ait1,...,tn)} = # clauses satisfied given t1, ..., tn
A De-randomized Algorithm for MAXSAT
MAX k-SAT approximation algorithm uses this:
bool chooseValue (Variable *var) {double sum pos=0.0, sum neg=0.0, prob=0.5;for (int sz=1 ; sz <= k ; sz++) {
sum pos += var->no clauses as pos lit[sz]*prob;sum neg += var->no clauses as neg lit[sz]*prob;prob *= 0.5;
}return (sum pos >= sum neg) ? true : false;
}
and will always find an assignment that satisfies at least(1 − 2−k)m clauses - for any input formula!
