CNF features extraction and classi cation · portfolio-based approach for SAT: SATzilla. combines...

IntroductionDefinitions

Features ComputationClassification Models

Good Configuration ClassifierFuture Works

CNF features extraction and classification

Enrique Matos AlfonsoNorbert Manthey (Supervisor)

Technische Universitat Dresden

Status report

Enrique Matos Alfonso Norbert Manthey (Supervisor) CNF features extraction and classification




Table of contents1 Introduction

Problem description2 Definitions

Propositional LogicGraphs

3 Features ComputationSequence StatisticsGraph FeaturesOther Features

4 Classification ModelsClassification ProblemFamily ClassifierBest Configuration Classifier

5 Good Configuration Classifier6 Future Works





Problem description

CNFs

search space of SAT problem is too big

preprocessing approach can help for certain applications.

it takes time and might not guarantee that solving time isdecreased.

then normally the parameter is flagged.

what parameters should we use to guarantee the best (orgood enough) performance for a given instance of a problem?.





Problem description

Existing Approaches

winner-take-all (best solver)

portfolio-based approach for SAT: SATzilla.

combines several solverspredicts runtime for each solver

SAT as a classification problem

classifies in SAT/UNSATdoes not provides solution.






CNFs

A CNF is represented set of clauses

F = {C1, . . . ,Cn}

A clause will be represented as a set of literals Ci = {l1, . . . , ln}.a literal is either a positive (i) variable or a negatedpropositional variable (−i).

a propositional variable i ∈ N. Moreover, R = {1, ..,n} withn ∈ N






CNFs

complement operation:

l = { −v if l = vv if l = −v C = {l ∣ l ∈ C}

resolution: if Ci ∩ Cj = {l} we can produce a new clause calledresolvent Ci ⊗ Cj = (Ci ∪ Cj) ∖ {l , l}






CNFs

Such CNF representation will be semantically equivalent topropositional logic formula defined in the following way:

X =

⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩

C1 ∧ . . . ∧ Cn if X = {C1, . . . ,Cn} is a CNF.

l1 ∨ . . . ∨ ln if X = {l1, . . . , ln} is a clause¬v if X = −v is a negative literalv if X = v is a positive literal

F I = F I






Graphs

We use the notion of Directed and Bipartite graph.

Definition

A graph G = ⟨V ,E ⟩:V set of vertices.

E ⊆ V ×V

Definition

A bipartite graph G = ⟨W ,B,E ⟩:W and B disjoint set of vertices (white and black).

E ⊆ (W ×B) ∪ (B ×W )

we can also consider weights definition w ∶ E → REnrique Matos Alfonso Norbert Manthey (Supervisor) CNF features extraction and classification





Graphs

for a vertice v we define the degree

deg(v) = ∣{ w ∣ ⟨v ,w⟩ ∈ E}∣





Sequence StatisticsGraph FeaturesOther Features

Sequence Statistics

for sequence of values S = ⟨x1, . . . , xn⟩we extract from it severalparameters to be considered:

Minimum and maximum defined in the usual way.

Mode defined as the most frequent element of the sequence.

mean = 1n ∑ xi to represent the mean of the sequence.

stdev = 1n ∑ (xi −mean)2 as the standard deviation of the

sequence.

values rate 1n ∣{x1, . . . , xn}∣






Sequence Statistics

for sequence of values S = ⟨x1, . . . , xn⟩we extract from it severalparameters to be considered:

entropy = ln(n) −∑ ci ln(ci) representing the entropy of S andci being the counts each different element that appears in thesequence.

and finally Q1, . . . ,Qk−1 being the k-Quantiles dividing thesequence after being ordered in k regular intervals.

the derivative of the sorted sequence of values: ifsort(S) = ⟨x ′1, . . . , x ′n⟩ thenderivative(S) = ⟨(x ′2 − x ′1), . . . , (x ′n − x ′n−1)⟩






Problem Example of CNF for 2-digits sum

Problem: to solve 2 + 2 = x .Solution: to use binary numbers and encode the sum operations asa SAT solving problem and extract the missing value from solutionof the SAT encoding. (a2 a1) + (a4 a3) = (a6 a8 a7)






Problem Example of CNF for 2-digits sum

Encoding:

11 variables and 28 clauses

unit clauses (a2 a1) = 2 represented as [-1], [2]

and gates: a5 ≡ a1 ∧ a3 represented as [-5 1], [-5 3], [5 -1 -3]

xor gates: a6 ≡ a1 ⊗ a3 represented as[-6 -1 -3], [-6 1 3], [6 -1 3], [6 1 -3]

or gates : a7 ≡ a10 ∨ a11 represented as[-7 10 11], [7 -10], [7 -11]






Clauses-Variable Graph

We compute the positive (CV +) and negative (CV −) versionof this graph as described:

Nodes The indices of the clauses of the formula(B = {1, . . .n} for F = {C1, . . . ,Cn}) and thevariables present in the formula W = {1, . . . ,m}.

Edges there will be an edge between nodes i and j iffj ∈ Ci (−j ∈ Ci ).

Weights not considered.The following sequences are then considered:

⟨ deg±(i) ∣ i ∈W ⟩⟨ deg±(i) ∣ i ∈ B⟩⟨ deg(i) ∣ i ∈ B⟩⟨ deg(i) ∣ i ∈W ⟩

deg(i) = degCV+(i) + degCV−(i)

deg±(i) = max(degCV+(i),degCV−(i))deg(i)






Clauses Variables (+) Graph (xor gate)

[-6 -1 -3] [6 -1 3] [6 1 -3] [-6 1 3]

136

++

+

+

++






Clauses Variables (-) Graph (xor gate)

[-6 -1 -3] [6 -1 3] [6 1 -3] [-6 1 3]

136

--

--

- -






Variables Graph

Two variables will be connected in this graph if they appear asliterals in the same clause:

Nodes The variables of the formula (V = {1, ..n}).Edges there will be an edge between nodes i and j iff

∃k ∣ {i , j} ⊆ (Ck ∪ Ck).Weights with the clauses Ck that share the two variables

we do ∑k 2−∣Ck ∣.

The following sequences are then considered:

degrees ⟨ deg(i) ∣ i ∈ V ⟩weights ⟨ w(⟨i , j⟩) ∣ ⟨i , j⟩ ∈ E ⟩






Variables Graph (and gate)

[-5 1], [-5 3], [5 -1 -3]

51

3

122+ 1

23

122+ 1

23

123






Variables Graph (degree) for a particular instance

0

5

10

15

20

25

variable

degre

e

variable

variable degree






Clauses Graph

Two clauses will be connected in this graph if they share aliteral:

Nodes The indices of clauses in the formula(V = {1, . . . ,n}).

Edges there will be an edge between nodes i and j iffCj ∩ Ci ≠ ∅.

Weights the number of common literals ∣Cj ∩ Ci ∣.The following sequences are then considered:







Clauses Graph (xor gate)

[-6 -1 -3] [6 -1 3]

[6 1 -3][-6 1 3]






Clauses Graph (degree) for a particular instance

0

10

20

30

40

50

60

70

80

90

cla

use d

egre

e

clause

clause degree






Resolution Graph

Two clauses will be connected in this graph if they can beresolved upon one variable:

Nodes The indices of clauses in the formula(V = {1, . . . ,n}).

Edges edge between nodes i and j iff Cj ∩ Ci = {l},where l is the literal to resolve upon.

Weights With the size of the resolvent between the twoclauses do 2−(∣(Cj∪Ci)∣−2).

Note

CV + and CV − are used to build the resolution graph.

The following sequences are then considered:degrees ⟨ deg(i) ∣ i ∈ V ⟩weights ⟨ w(⟨i , j⟩) ∣ ⟨i , j⟩ ∈ E ⟩






Resolution Graph (xor gate)

[-6 -1 -3] [6 -1 3]

[6 1 -3][-6 1 3][-1]

[-3]

122

122

122

122






Resolution Graph (degree) for a particular instance

0

10

20

30

40

50

60

70

cla

use d

egre

e

clause

clause degree






Resolution Graph (weights) for a particular instance

1

2

3

4

5

6

resolu

tion w

eig

hts

edges

resolution weights






AND Graph

We search for AND gates l0 ↔ l1 ∧ . . . ∧ lk or in blocked ANDgates l0 → l1 ∧ . . . ∧ lk

Nodes The literals of the formula (V = {±1, . . . ± n}).Edges for each AND gate we add edges ⟨l0, li ⟩.

Weights 2−k








EX1L Graph

We search for Exactly one Literal gates EX1L(l1, . . . , lk)Nodes The literals of the formula (V = {±1, . . . ± n}).Edges for each i ≠ j add edges ⟨li , lj⟩ and ⟨lj , li⟩ for

{i , j} ⊆ {1, . . . , k}.Weights Not considered.


degrees ⟨ deg(i) ∣ i ∈ V ⟩






Other Features

Additionally we also consider:

Number of clauses, unit (2 lit, 3,...8, 9+ lit) clauses, hornclauses.

Symmetry:

Some sequences Si of hash codes based on literals in a clausesand in previous sequence Si = fhash(Si−1) is computed.gives a measure of how symmetric are the clauses of theformula.Then we consider the concatenation of the elements of thesequences Si






Other Features

Additionally we also consider:

Recursive Weight Heuristic (RWH): To estimate probability ofeach literal being in the final model. The number ofhigh-ranked literals gives an estimate on the size of thebackbone (= literals that have to be set).

CPU time to compute each graph features (not meant forclassification).






Features Runtimes

0.001

0.01

0.1

1

10

100

0 200 400 600 800 1000 1200 1400 1600

tim

e in s

econds

instance

Resolution.and.Clause.graphs.timeBin.Implication.graph.time

Constraints.recognition.timeSymmetry.computation.time

Clause-Var.and.Var.graphs.time






Total Runtimes

0

100

200

300

400

500

600

700

800

900

0 500 1000 1500 2000 2500 3000 3500

tim

e in s

econds

instances

ourfeaturesNoCG.time

ourfeaturesNoCGRes.time

ourfeaturesNoRes.time

SATzillaAll.time

SATzillaBase.time

ourfeatures.time





Classification Problem

Desired Model

a function ff ∶ CNFs → F that maps a given CNF to a vectorof features v ∈ F (usually F = Rn).

an internal function ffc ∶ F → C that maps a vector of featuresto a class c ∈ C .

then we take the composition fc = ff ○ ffc ∶ CNFs → C thatmaps a given CNF to a class c ∈ C .





Classification Problem

Classifier

a function ffc ∶ F → C that maps a vector of features to a classc ∈ C .

built using a training dataset Train = F × C where for theelements ⟨f , c⟩ ∈ F × C the value c is said to be thecorresponding class of the features vector f . (Benchmark from2011, 2011-2012, 2012).

tested using Test = F × C (Benchmark from 2012 (whentraining on 2011) and 2013).

accuracy ∣{ ⟨f ,c⟩ ∣ ffc(f )=c}∣∣Test∣





Good Configuration Classifier

For each parameter configuration Ci :

compute which instance is ”good” to be solved with it.build a classifier to predict how good is this configuration for agiven CNF.

then we do the prediction for each configuration on a givenCNF and we take the “best” (max probability) one.

we compute on a test dataset how many instances are solvedwithin the timeout with the prediction of good configurations.





Good Configuration Classifier

CNF f ∈ F . . .

Cn,pn

C1,p1

pbest = max(pi)

sat(CNF,Cbest)





Good/Bad criteria

Timeout criteria:classi(x) = good ≡ solving timei(x) < timeout

improvement relative to configuration k : for i ≠ k classi(x) =good ≡ solving timei(x) < min(timeout, solving timek(x))complement relative to configuration k : for i = kclassk(x) = good ≡ solving timek(x) < timeoutand for i ≠ k classi(x) = good ≡solving timei(x) < timeout ∧ classk(x) = bad





Experiments on Good Configuration Classifier (Training)

2011 2012 2011-12Configuration Inst acc Inst acc Inst acc

ourf 324/570 100 917/1500 99 1241/2070 99

ourf−[CG] 360/658 100 980/1582 99 1340/2240 99

ourf−[CG ,R] 372/705 100 1001/1609 100 1373/2314 99

ourf−[R] 363/663 100 977/1579 99 1340/2242 99zillaBase 422/807 100 1117/1757 100 1539/2564 100zillaAll 245/492 100 598/1084 100 843/1576 100





Experiments on Good Configuration Classifier (Testing)

11-12 11-13 1112 -13 12-13

Config Inst acc Inst acc Inst acc Inst acc

ourf 8191500 92 188

466 79 174466 80 199

466 68

ourf −[CG] 8671582 92 186

495 86 192495 84 210

495 73

ourf−[CG ,R] 8741609 93 167

510 75 207510 83 212

510 80

ourf−[R] 8601579 91 185

496 85 219496 88 189

496 702012 dataset has 1789 instances2013 dataset has 600 instances





Experiments on Good Configuration Classifier (Testing)

11-12 11-13 1112 -13 12-13

Config Inst acc Inst acc Inst acc Inst acc

ourf−[R] 8601579 91 185

496 85 219496 88 189

496 70

ourf−[CG ,R] 8741609 93 167

510 75 207510 83 212

510 80

zillaBase 9841757 94 190

557 87 222557 90 214

557 90

zillaAll 5131084 95 70

260 90 89260 91 89

260 912012 dataset has 1789 instances2013 dataset has 600 instances





Experiments on Good Configuration Classifier (our-Res)

0

100

200

300

400

500

600

700

800

900

0 50 100 150 200 250 300 350

tim

e in s

econds

solved instances

RealTime.glucose22RealTime.lgl

RealTime.riss-vsbpred-1-3pred-2-3

pred-12-3bestRealTime.riss3g

worstRealTime.LABSextANYALL





Experiments on Good Configuration Classifier (ZillaBase)

0

100

200

300

400

500

600

700

800

900

0 50 100 150 200 250 300 350

tim

e in s

econds

solved instances

RealTime.glucose22RealTime.lgl

RealTime.riss-vsbpred-1-3pred-2-3

pred-12-3bestRealTime.riss3g

worstRealTime.LABSextANYALL





Future Tasks

to decrease the computation time of the clause graph.

to compute and try some other cheap features.

to increase the number of configurations (to deal withunsolved CNFs)

try another approach training and testing datasets.

to try to obtain more significant results





The End

Thanks...


CNF features extraction and classi cation · portfolio-based approach for SAT: SATzilla. combines...

Documents

Transcript of CNF features extraction and classi cation · portfolio-based approach for SAT: SATzilla. combines...