Download - CSCE496/896, Fall 2011 cse.unl/~choueiry/F11-496-896 Questions : [email protected]

Problem Solving with Constraints, CSCE 496/896

Overview 1 1

CSCE496/896, Fall 2011

www.cse.unl.edu/~choueiry/F11-496-896Questions: [email protected]

Broadcast: [email protected]

Berthe Y. Choueiry (Shu-we-ri)AVH 360

Constraint Satisfaction 101


Overview 1 2

Outline

Motivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal characterizationBasic solving techniques

(Implementing backtrack search)

Advanced solving techniques

Issues & research directions


Overview 1 3

Motivating example• Context: You are a senior in college

• Problem: You need to register in 4 courses for the Spring semester

• Possibilities: Many courses offered in Math, CSE, EE, CBA, etc.

• Constraints: restrict the choices you can make– Unary: Courses have prerequisites you have/don't have

Courses/instructors you like/dislike

– Binary: Courses are scheduled at the same time– n-ary: In CE: 4 courses from 5 tracks such as at least 3 tracks are covered

• You have choices, but are restricted by constraints– Make the right decisions!!


Overview 1 4

Motivating example (cont’d)• Given

– A set of variables: 4 courses at UNL– For each variable, a set of choices (values)– A set of constraints that restrict the combinations

of values the variables can take at the same time

• Questions– Does a solution exist? (classical decision problem)– How two or more solutions differ? How to change

specific choices without perturbing the solution?– If there is no solution, what are the sources of

conflicts? Which constraints should be retracted?– etc.


Overview 1 5

Practical applications

• Radio resource management (RRM)• Databases (computing joins, view updates)• Temporal and spatial reasoning • Planning, scheduling, resource allocation • Design and configuration • Graphics, visualization, interfaces • Hardware verification and software engineering• HC Interaction and decision support • Molecular biology • Robotics, machine vision and computational linguistics • Transportation • Qualitative and diagnostic reasoning

Adapted from E.C. Freuder


Overview 1 6

Constraint Processing

• is about ...– Solving a decision problem…– … While allowing the user to state arbitrary

constraints in an expressive way and– Providing concise and high-level feedback about

alternatives and conflicts

• Related areas: – AI, OR, Algorithmic, DB, TCS, Prog. Languages, etc.


Overview 1 7

• Flexibility & expressiveness of representations

• Interactivity

users can constraints

Power of Constraints Processing

relax

reinforce


Overview 1 8

Outline

Motivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal

characterizationBasic solving techniques


Overview 1 9

Defining a problem

• General template of any computational problem– Given:

• Example: a set of objects, their relations, etc.

– Query/Question: • Example: Find x such that the condition y is

satisfied

• How about the Constraint Satisfaction Problem?


Overview 1 10

Definition of a CSP• Given P = (V, D, C )

– V is a set of variables,

– D is a set of variable domains (domain values)

– C is a set of constraints,

• Query: can we find a value for each variable such that all constraints are satisfied?

nVVV ,,, 21 V

nVVV DDD ,,, 21 D

€

withRVa ,Vb ,...,Vi= (x,y,K ,z){ }⊆DV a × DV b ×K × DV i

lCCC ,,, 21 C


Overview 1 11

Different Queries Yield Different Problems

• Find a solution decision problem• Find number of/all solutions counting problem• Find a set of constraints that can be removed so

that a solution exists optimization problem• Etc.


Overview 1 12

Domain Types• P = (V, D, C ) where

• Domains:– Restricted to {0,1}: Boolean CSPs– Finite (discrete), enumeration works– Continuous, sophisticated algebraic techniques are

needed• Consistency techniques on domain bounds

nVVV ,,, 21 V nVVV DDD ,,, 21 D

mVlVkVVVV DDDzyxmlk

),,,(,...,,C with

lCCC ,,, 21 C


Overview 1 13

CSP Representation (I)• Given P = (V, D, C ), where

• Find a consistent assignment for variables

nVVV ,,, 21 V nVVV DDD ,,, 21 D

Constraint Network (graph, hypergraph)

• Variable node (vertex)

• Domain node label

• Constraint arc (edge) between nodes

lCCC ,,, 21 C


CSP Representation (II)

Overview 1 14

{ 3, 6, 7 }{ 1, 2, 3, 4 }

{ 3, 5, 7 }{ 3, 4, 9 }

V3

V1 V2

v2 > v4

V4

v1 < v2

v2 < v3 v1+v3 < 9

v1+v2+V4 < 10

{1, 2, 3, 4}

{ 3, 5, 7 }{ 3, 4, 9 }

{ 3, 6, 7 }

v2 > v4

V4

V2

v1+v3 < 9

V3

V1

v2 < v3

v1 < v2Graph• Vertices: variables

• Edges: binary constraints

Hypergraph• Vertices: variables

• Hyperedges: constraints


Constraint Definition

Overview 1 15

• A constraint C is defined by− A scope, the set of variables on which the constraint applies

Notation: SCOPE(C), scope(C), scp(C)

− A relation, a subset of the Cartesian product of the domains of the variables in the scope of the constraint

Notation: RELATION(C), rel(C)

• Arity, cardinality of the constraint’s scope− Unary, binary, ternary,…, global− Universal constraint


Relation Definition

Overview 1 16

• Extension, all tuples are enumerated− As a list of allowed tuples (supports, positive table)− As a list of forbidden tuples (conflicts, no-goods)

• Intension, given by a set builder− When it is not practical or possible to list all tuples− Define types/templates of common constraints to be

used repeatedly− Examples: linear constraints, All-Diff (mutex), Atmost,

TSP-constraint, cycle-constraint, etc.


Overview 1 17

Constraint Implementation

• Predicate function• Set of tuples (list or table) • Binary matrix (bit-matrix)• Constrained Decision Diagrams ([Cheng & Yap, AAAI 05])

• etc.


Overview 1 18

Outline




Overview 1 19

Example II: Temporal reasoning

• Give one solution: …….• Satisfaction, yes/no: decision problem

[ 5.... 18]

[ 4.... 15]

[ 1.... 10 ] B < C

A < B

B

A

2 < C - A < 5C


Overview 1 20

Example III: Map coloring

Using 3 colors (R, G, & B), color the US map such that no two adjacent states have the same color

• Variables?

• Domains?

• Constraints?


Overview 1 21

Example III: Map coloring (cont’d)

Using 3 colors (R, G, & B), color the US map such that no two adjacent states have the same color

{ red, green, blue }


{ red, green, blue }{ red, green, blue }{ red, green, blue }

WY

NE

KS

OKNM

TX

LA

CO

UT

AZ

AR


Overview 1 22

Example IV: Resource Allocation

What is the CSP formulation?

{ a, b, c }

{ a, b } { a, c, d } { b, c, d }


Overview 1 23

Example IV: RA (cont’d)

{ a, b, c }

{ a, b } { a, c, d } { b, c, d }

{ R1, R3 }T1

{ R1, R3 }

{ R1, R3 } { R1, R3, R4 }

{ R1, R2, R3 }

{ R2, R4 }

{ R2, R4 }

T2

T3 T4

T5

T6

T7

Interval Order

T2

T1Constraint Graph

T4

{ R1, R3 }T3

{ R2, R4 }

{ R2, R4 }

T6

T7T5

{ R1, R2, R3 }

{ R1, R3 }

{ R1, R3 }

{ R1, R2, R3 }


Overview 1 24

Example V: Cryptarithmetic puzzles

• DX1 = DX2 = DX3 = {0,1}• DF=DT=DU=DW=DR=DO=[0,9]

• O+O = R+10X1• X1+W+W = U+10X2• X2+ T+T = O + 10X3• X3=F• Alldiff({F,D,U,V,R,O})


Overview 1 25

Example VI: Product Configuration

Train, elevator, car, etc.

Given: • Components and their attributes (variables)• Domain covered by each characteristic (values)• Relations among the components (constraints) • A set of required functionalities (more constraints)

Find: a product configuration i.e., an acceptable combination of components that realizes the required functionalities


Overview 1 26

Example VII: PuzzleGiven: • Four musicians: Fred, Ike, Mike, and Sal, play bass, drums,

guitar and keyboard, not necessarily in that order.

• They have 4 successful songs, ‘Blue Sky,’ ‘Happy Song,’ ‘Gentle Rhythm,’ and ‘Nice Melody.’

• Ike and Mike are, in one order or the other, the composer of ‘Nice Melody’ and the keyboardist.

• etc ...

Query: Who plays which instrument and who composed which song?


Overview 1 27

Example VII: Puzzle (cont’d)

Formulation 1: • Variables: Bass, Drums, Guitar, Keyboard, Blue Sky, Happy Song Gentle Rhythm and Nice Melody.• Domains: Fred, Ike, Mike, Sal • Constraints: …

Formulation 2:• Variables: Fred's-instrument, Ike's-instrument, …,

Fred's-song, Ikes's-song, Mike’s-song, …, etc.• Domains:

{ bass, drums, guitar, keyboard } { Blue Sky, Happy Song, Gentle Rhythm, Nice Melody}

• Constraints: …


Overview 1 28

Constraint types: examples• Example I: algebraic constraints

• Example II:

(algebraic) constraints

of bounded difference

• Example III & IV: coloring, mutual exclusion, difference constraints

• Example V & VI: elements of C must be made explicit

{1, 2, 3, 4}

{ 3, 5, 7 }{ 3, 4, 9 }

{ 3, 6, 7 }

v2 > v4

V4

V2

v1+v3 < 9

V3

V1

v2 < v3

v1 < v2

[ 5.... 18]

[ 4.... 15]

[ 1.... 10 ] B < C

A < B

B

A

2 < C - A < 5C



{ red, green, blue }{ red, green, blue }{ red, green, blue }

WY

NE

KS

OKNM

TX

LA

CO

UT

AZ

AR

T1

Constraint Graph

T4

{ R1, R3 }T3

{ R2, R4 }

{ R2, R4 }

T6

T7T5

{ R1, R2, R3 }

{ R1, R3 }

{ R1, R3 }

{ R1, R2, R3 }


Overview 1 29

More examples

• Example VII: Databases– Join operation in relational DB is a CSP– View materialization is a CSP

• Example VIII: Interactive systems– Data-flow constraints– Spreadsheets– Graphical layout systems and animation– Graphical user interfaces

• Example IX: Molecular biology (bioinformatics)– Threading, etc


Overview 1 30

Outline




Overview 1 31

Representation (again)Macrostructure G(P):- constraint graph for

binary constraints - constraint network for non-binary constraints

Micro-structure (P):

Co-microstructure co-(P):

a, b

a, c b, c

V1

V2 V3

(V1, a ) (V1, b)

(V2, a ) (V2, c) (V3, b ) (V3, c)

(V1, a ) (V1, b)

(V2, a ) (V2, c) (V3, b ) (V3, c)

no goods


Overview 1 32

Complexity of CSP

Characterization• Decision problem

• In general, NP-complete by reduction from 3SAT


Overview 1 33

Proving NP-completeness

1. Show that 1 is in NP

2. Given a problem 1 in NP, show that an known NP-complete problem 2 can be efficiently reduced to 1

1. Select a known NP-complete problem 2 (e.g., SAT)

2. Construct a transformation f from 2 to 1

3. Prove that f is a polynomial transformation

(Check Chapter 3 of Garey & Johnson)


Overview 1 34

What is SAT?Given a sentence:

– Sentence: conjunction of clauses

– Clause: disjunction of literals

– Literal: a term or its negation

– Term: Boolean variable

Question: Find an assignment of truth values to the Boolean variables such the sentence is satisfied.

4326541 ccccccc

32 cc

61, cc

01 11 cc


Overview 1 35

CSP is NP-Complete• Verifying that an assignment for all

variables is a solution – Provided constraints can be checked in

polynomial time

• Reduction from 3SAT to CSP– Many such reductions exist in the literature

(perhaps 7 of them)


Overview 1 36

Problem ReductionExample: CSP into SAT (proves nothing, just an exercise)

Notation: variable-value pair = vvp

• vvp term– V1 = {a, b, c, d} yields x1 = (V1, a), x2 = (V1, b), x3 = (V1, c), x4 = (V1, d),

– V2 = {a, b, c} yields x5 = (V2, a), x6 = (V2, b), x7 = (V2,c).

• The vvp’s of a variable disjunction of terms– V1 = {a, b, c, d} yields

• (Optional) At most one VVP per variable

4321 xxxx

43214321

43214321

xxxxxxxx

xxxxxxxx


Overview 1 37

CSP into SAT (cont.)Constraint:

1. Way 1: Each inconsistent tuple one disjunctive clause– For example: how many?

2. Way 2:a) Consistent tuple conjunction of termsb) Each constraint disjunction of these conjunctions

transform into conjunctive normal form (CNF)

Question: find a truth assignment of the Boolean variables such that the sentence is satisfied

)},(),,(),,(),,(),,{(21

adbccbbaaaC VV

71 xx

51 xx

5463

726151

xxxx

xxxxxx


Overview 1 38

Outline

Motivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal characterizationBasic solving techniques

• Modeling and consistency checking• Constructive, systematic search• Iterative improvement, local search


Overview 1 39

How to solve a CSP?

Search

1. Constructive, systematic

2. Iterative repair, local search


Overview 1 40

Before starting search!

Consider:• Importance of modeling/formulation:

– To control the size of the search space

• Preprocessing – A.k.a. constraint filtering/propagation, consistency

checking– reduces size of search space


Overview 1 41

4 Rows

V1

V2

V3

V4

Importance of Modeling

• N-queen: formulation 1

– Variables? – Domains?– Size of CSP?

• N-queens: formulation 2

– Variables?– Domains?– Size of CSP?

4 Column positions

1 2 3 4

84 244444

{0,1}162

16 Cells

V11 V12 V13 V14

V21 V22 V23 V24

V31 V32 V33 V34

V41 V42 V43 V44


Overview 1 42

Constraint Checking Arc-consistency

[ 5.... 18]

[ 4.... 15]

[ 1.... 10 ]B < C

A < B

B

A

2 < C - A < 5

C

2- A: [ 2 .. 10 ]

C: [ 6 .. 14 ]

3- B: [ 5 .. 13 ]

C: [ 6 .. 15 ]

1- B: [ 5 .. 14 ]1413

6

2

14


Overview 1 43

Constraint Checking Arc-consistency: every combination of two adjacent variables 3-consistency, k-consistency (k n)

Constraint filtering, constraint checking, etc.. Eliminate non-acceptable tuples prior to search Warning: arc-consistency does not solve the problem

{ 1, 2, 3 } { 2, 3, 4 }

BA BA

{ 2, 3 } { 2, 3 }

)3()2( BA still is not a solution!


Overview 1 44

Systematic Search Starting from a root node Consider all values for a variable V1

For every value for V1, consider all values for V2

etc..

For n variables, each of domain size d Maximum depth? fixed! Maximum number of paths? size of search space, size of CSP

S

v1 v4Var 1

Var 2

v3v2


Overview 1 45

Systematic search: Back-checking

• Systematic search generates dn possibilities• Are all possibilities acceptable?

Expand a partial solution only when it is consistent This yields early pruning of inconsistent paths

S

v1 v4Var 1

Var 2

v3v2

S

v1 v4Var 1

Var 2

v3v2


Overview 1 46

Systematic search: Chronological backtracking

What if only one solution is needed?

• Depth-first search & Chronological backtracking

• DFS: Soundness? Completeness?

Var 1 v1 v2

S

S

v1 v4Var 1

Var 2

v3v2


Overview 1 47

Systematic search: Intelligent backtracking

What if the reason for failure was higher up in the tree?

Backtrack to source of conflict !!

Backjumping, conflict-directed backjumping, etc.

Var 1 v1 v2

S S

v1 v2Var 1


Overview 1 48

Systematic search: Ordering heuristics

• Which variable to expand first?

• Heuristics:– most constrained variable first (reduce branching factor)

– most promising value first (find quickly first solution)

},{},,,,{,,:2121 baDdcbaDVV VV Exp

)}(),{()}(),{(: 2121 bVcVaVcV andSol

dynamic

staticbecould

ordering value

ordering variableforStrategies

s

c

aV2 b a b a b

ba dV1

V1

s

V2 ba

ba c d


Overview 1 49

Systematic search: Back-checking• Search tree with only backtrack search?

Root node

1 2 4

1 3 41 2 3 4

1 2 3 4

1Q

3Q Q

2Q

1 2 3

1 2

2Q

4Q

1Q

3Q

Solution!

26 nodes visited.


Overview 1 50

Systematic search: Forward checking

Root node

1Q

3Q Q 4

Domain Wipe Out V3

2Q

Domain Wipe Out V4

2Q

4Q

1Q

3Q

Solution!

8 nodes visited.

Search Tree with domains filter by Forward Check


Overview 1 51

Improving BT search

• General purpose methods for1. Variable, value ordering

2. Improving backtracking: intelligent backtracking avoids repeating failure

3. Look-ahead techniques: propagate constraints as variables are instantiated


Overview 1 52

Search-tree branching

• K-way branching– One branch for every value in the domain

• Binary branching– One branch for the first value– One branch for all the remaining values in the domain– Used in commercial constraint solvers: ILOG, Eclipse

• Have different behaviors (e.g., WRT value ordering

heuristics [Smith, IJCAI 95])


Overview 1 53

Summary of backtrack search• Constructive, systematic, exhaustive

– In general sound and complete

• Back-checking: expands nodes consistent with past• Backtracking: Chronological vs. intelligent• Ordering heuristics:

– Static– Dynamic variable– Dynamic variable-value pairs

• Looking ahead:– Partial look-ahead

• Forward checking (FC)• Directional arc-consistency (DAC)

– Full (a.k.a. Maintaining Arc-consistency or MAC)


Overview 1 54

CSP: a decision problem (NP-complete)

1. Modeling: abstraction and reformulation2. Preprocessing techniques:

• eliminate non-acceptable tuples prior to search

3. Systematic search:• potentially dn paths of fixed lengths• chronological backtracking• intelligent backtracking• variable/value ordering heuristics

4. Search ‘hybrids:’• Mixing consistency checking with search: look-ahead

strategies


Overview 1 55

Non-Systematic Search• Iterative repair, local search: modifies a

global but inconsistent solution to decrease the number of violated constraints

• Examples: Hill climbing, taboo search, simulated annealing, GSAT, WalkSAT, Genetic Algorithms, Swarm intelligence, etc.

• Features: Incomplete & not sound

– Advantage: anytime algorithm

– Shortcoming: cannot tell that no solution exists


Overview 1 56

Outline of CSP 101

• We have seenMotivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal characterizationBasic solving techniques

• We will move to (Implementing backtrack search)

Advanced solving techniquesIssues & research directions