Modeling Qualitative Preferences Using the CP-net Model

Modeling Qualitative Preferences

Using the CP-net Model

To make good decision, we must be able to assess and compare different

alternatives.

The ability to make decisions is a corner-stone of many AI applications:

• Decision-support expert systems

• Autonomous agents

• Configuration software

• …

Assessing Alternatives

We compare alternatives based on their:

• Likelihood

• Desirability

Our Focus:

Assessing/specifying outcome desirability

Specifying Preferences

UncertaintyInvolved?

Utility functiondesirable

Enough torank potential

outcomes

yes

no

Need to weigh the contributions of different outcome

Need to recognize the best feasible alternative

Utility functions

• Qantify outcome desirability– Capture the difference in desirability

between outcomes

• Necessary when:– Uncertainty is involved, or

• Drawbacks:– Complicated preference elicitation.– Generally hard optimization.

Serious practical concern

Serious comput. concern

When a utility function cannot be and/or need not be obtained, one should resort to other, more qualitative forms of preference representation.

What We Want from a Qualitative Preference Model

• Simple elicitation process based on intuitive and natural statements about preferences– No need for an expert decision analyst– Makes automatic online elicitation feasible

• As expressive as possible, subject to above

• Supports an efficient optimization process

Overview

• CP-net model for qualitative preferences.• Preference Optimization.• Complexity analysis of outcome comparisons.• Consistency testing.• Various Enhancements.

• Potential applications.• Future research directions and open problems.

Ceteris Paribus (cp) Statemenents

Ceteris Paribus (Lat.) – all else being equal

“ I prefer to have wine with my meal, all else being equal”

That is: given two identical meals, one with wine and one without, I prefer the former.

Conditional CP Statements

“ I prefer red wine to white wine with my meal, ceteris paribus, given that meat is served”

That is: given two identical meals in which meat is served, I prefer red wine to white wine.

Tells us nothing about two identical meals in which meat is NOT served.

Outcomes and Preferences1. Domain variables .

2. Outcome space .

3. Preferences over the outcome space.

1{ , , }nV

1( ) ( )nDom Dom

Example on Product (Computer System) Configuration:

• Domain variables are the system’s properties:

Processor Speed, Processor Manufacturer, Screen Size, etc.

• Dom(Screen Size) = {15in, 17in, 19in, 21in}

• Possible preferential statements of a customer:

I prefer 1000 MHz on 800 MHz

I prefer 19in screen on 17in screen if video card is Sony’s

Preferential Independence

If my preferences over the values of a variable v does not depends on the values of some other variables, then v is preferentially independent of all other variables.

I prefer 1000 MHz to 800 MHz (all else being equal)

If my preferences over the values of a variable v depends on the values of some other variables v1, …,vk , then v is conditionally preferentially independent of all other variables V-{v1, …,vk }, given an assignment on v1, …,vk .

I prefer 19in screen to 17in screen if video card is Sony’s (all else being equal)

Preferential Independence

A subset of variables is preferentially independent of its complement if and only if, for all assignments holds

X Y V X

1 2 1 2, , ,x x y y

1 1 2 1 1 2 2 2x y x y iff x y x y

Let be a partition of into three disjoint non-empty sets. and are conditionally preferentially independent given if and only if, for all holds

, ,X Y Z VX Y

z1 2 1 2, , ,x x y y

1 1 2 1 1 2 2 2x y z x y z iff x y z x y z

CP-nets (Boutilier, Brafman, Hoos, Poole, UAI ‘99)

An intuitive, qualitative, graphical model of An intuitive, qualitative, graphical model of preferences, that captures statements of preferences, that captures statements of conditional preferential independence.conditional preferential independence.

• DAG in which each node represents a domain variable.

• The immediate parents P(v) of a variable v in the network are those variables that affect user’s preference over the values of v.

P(screen size) = { video card manuf. }P(operating system) = { processor speed, screen

size }

A

C

D E

F

B b b

( ) ( ) :

( ) ( ) :

a b a b c c

a b a b c c

:

:

c e e

c e e

a a

:

:

c d d

c d d

:

:

d f f

d f f

Example of a CP-net

Any CP-net defines a partial order over the outcome space.

A

C

B b b

( ) ( ) :

( ) ( ) :

a b a b c c

a b a b c c

a aabc

abc abc abc

abc abc abc

abc

Consistency

worst

best

Overview



Preferential Optimization

Finding the preferentially optimal outcome is straightforward!

A

C

D E

F

B b b

( ) ( ) :

( ) ( ) :

a b a b c c

a b a b c c

:

:

c e e

c e e

a a

:

:

c d d

c d d

:

:

d f f

d f f

A

C

D E

F

Ba b

c

d e

f

Adding Constraints1. Domain variables .

2. Outcome space .

3. Preferences over the outcome space.

4. Constraints on the domain variables.

1{ , , }nV

1( ) ( )nDom Dom

Constraints in Product (Computer System) Configuration:

17in screens are currently unavailable.

15in screens of NEC are incompatible with the Sony’s graphical card.

Windows XP requires at least 800MHz processor speed.

Constraint-based Preferential Optimization (Boutilier et al. 97’)

Branch & Bound algorithm for determining the set of feasible, preferentially non-dominated outcomes was

suggested.• Starting with an empty set of solutions, the algorithm

continuously extends it by adding new non-dominated solutions.

• The current set of the solutions serves a lower bound for the forthcoming candidates.

>>

Constraint-based Preferential Optimization

E A B F

GC

DH

X Y Z

Initialize the set of local results R = .

Choose any variable v with no parents in G.

A Let v1 >… > vk be the preference ordering of D(v) given the assignment on P(v) in K.

K(x,y) : a1 > … > ak

E ai B F

GC

DH

X Y Z for i = 1 to k do v = vi

K(x,y) : a1 > …> ai > …> ak

Strengthen the set of constraints C by v = vi to obtain Ci

if Ci is inconsistent or exist j<i s.t. Cj Ci continue with next iteration

Reduce G to G’ by removing the variables assigned by K’.

E B F

GC

DH

Let G’1,…, G’m be the strongly connected components of G’

for j= 1 to m do

Sj = SearchSearch (G’j, KK’, Ci)

elseLet K’ be the partial

assignment induced by v = vi and Ci .

ai

for i = 1 to k do v = vi

Strengthen the set of constraints C by v = vi to obtain Ci

if Ci is inconsistent or exist j<i s.t. Cj Ci continue with next iteration

Reduce G to G’ by removing the variables assigned by K’.

E B F

GC

DH

Let G’1,…, G’m be the strongly connected components of G’

for j= 1 to m do

Sj = SearchSearch (G’j, KK’, Ci)

elseLet K’ be the partial assignment

induced by v = vi and Ci .

E ai B F

GC

DH

X Y Z

ai

Add o to R

return R

if Sj for all j m

if for each o’ R holds Ko’ Ko

foreach o K’ S1 … Sm do

Constraint-based Preferential Optimization

So what is the price of comparison (dominance testingdominance testing)

between two outcomes?

Overview



abc

abc abc abc

abc abc abc

abc

Flipping Sequence - Example

abc

abcabc

abc

check ( )abc abc

Dominance TestingIn (Boutilier et al.) dominance testing was treated as a search for a flipping sequence from the (purported) less preferred outcome to the (purported) more preferred outcome through a sequence of more preferred outcomes:

0 1 1m m

Where, for , 0 1i m • outcome differs from the outcome in

the value of exactly one variablei 1i

j

• , given the values of in (and )

1[ ] [ ]i ij j ( )jP i

1i

Dominance Testing for CP-nets with Binary Variables.

Backtrack free algorithm in BBHP ‘99

RemarksRemarks

Linear?Tree

ComplexityComplexityCP-net grapCP-net graphh

Complexity of Dominance Testing

First result (Boutilier et al.):

Dominance testing for binary, tree CP-nets is backtrack free.

Example:

A

C

D E

B:

:

a c c

a c c

a a

:

:

c d d

c d d

:

:

d f f

d f f

:

:

a b b

a b b

abcde abcde

a

b c

d e

abcde

cabcde

Example:

A

C

D E

B:

:

a c c

a c c

a a

:

:

c d d

c d d

:

:

d f f

d f f

:

:

a b b

a b b

abcde abcde

a

b c

d e

abcde

abcdeb

abcde

Example:

A

C

D E

B:

:

a c c

a c c

a a

:

:

c d d

c d d

:

:

d f f

d f f

:

:

a b b

a b b

abcde abcde

a

b c

d e

abcde

abcdeb

abcded

abcde

Example:

A

C

D E

B:

:

a c c

a c c

a a

:

:

c d d

c d d

:

:

d f f

d f f

:

:

a b b

a b b

abcde abcde

a

b c

d e

abcde

abcdeb

abcded

abcde

a

abcde

Example:

A

C

D E

B:

:

a c c

a c c

a a

:

:

c d d

c d d

:

:

d f f

d f f

:

:

a b b

a b b

abcde abcde

a

b c

d e

abcde

abcdeb

abcded

abcde

a

abcde

c

abcde

The algorithm for trees is not good for polytrees

A

C

B b b

( ) ( ) :

( ) ( ) :

a b a b c c

a b a b c c

a a

abc abc

c

baabc

a

abc

abc

abc


Lower bound

RemarksRemarks

Tree


2( )O n


Lower bound

RemarksRemarks

Tree

Polytree


2( )O n2 2 3(2 )k kO n

k - maximal indegree

Start of Analysis

Denote by the maximal number of times that a variable may be required to flip its value on a irreducible flipping sequence from to .

Lemma 1Lemma 1: Given a dominance testing problem where is a singly connected, binary CP-net, for each variable we have thatolds:

MaxReq( )ii

, ,N

i

MaxReq( )i n

N

Framework for Polytrees

• Using the upper bound established by Lemma 1, we provide a polynomial time procedure that determines the maximal number of feasible, possibly required value flips for a given variable.

• We provide a polynomial time algorithm that determines whether or not exist a flipping sequence from to . This algorithm is based on top-down execution of the previously defined procedure on the variables of the CP-net.

MaxReq

FeasibleMaxReq( ) min(MaxCan( ),MaxReq( ))i i i


k - maximal indegree

Lower bound

RemarksRemarks

Tree

Singly-connected DAG

Polytree


2( )O n2 2 3(2 )k kO n

NP-complete Reduction from 3SAT


Minimal flipping sequences are polynomially bounded

Reduction from 3SAT

- maximal indegree

Lower bound

RemarksRemarks

Tree

NP-complete -connected DAG

NP-completeSingly-connected DAG

Polytree


2( )O n2 2 3(2 )kO n


EXPTIME or in NP?

Minimal flipping sequences are polynomially bounded

Reduction from 3SAT

- maximal indegree

Lower bound

RemarksRemarks

Tree

?DAG

NP-complete -connected DAG

NP-completeSingly-connected DAG

Polytree


2( )O n2 2 3(2 )kO n

Recall that ...

• Starting with an empty set of solutions, the B&B algorithm continuously extends it by adding new non-dominated solutions.

• The current set of the solutions serves a lower bound for the forthcoming candidates.

• New candidate is compared to all solutions generated until now.

The Big Picture• Generally, COP is much harder than CSP.

• If any non-dominated solution is enough, then the CP-net based COP is not harder than the corresponding CSP.

• If some non-dominated solutions are required, and the CP-net is reasonably restricted, then the CP-net based COP is still not harder than the corresponding CSP.

• In general, if some (even just 2) non-dominated solutions are required, then the CP-net based COP may be much harder.

Overview



Any CP-net defines a partial order over the outcomes.

A

C

B b b

( ) ( ) :

( ) ( ) :

a b a b c c

a b a b c c

a aabc

abc abc abc

abc abc abc

abc

Recall that …

worst

best

acyclic

Cyclic CP-nets are not necessarily consistent …

Consistency of Cyclic CP-nets

A

B

:

:

a b b

a b b

:

:

b a a

b a a

ab ab

ab ab

:

:

b a a

b a a

ab ab

ab ab

Localizing the Analysis

Strongly Connected Components

Context variables of a SCC

Contexts of a SCC =

assignments to its context variables

Localizing the Analysis

Theorem: A cyclic CP-net is consistent if and only if for every strongly connected component and for every context of , we have that is consistent given .

Conclusion:We can start by consistency analysis of

only strongly connected CP-nets.

Basic SCCs – Simple Cycles, Binary Variables

a

c d

b

Simple cycle

Surprisingly …

Theorem: A consistent specification of a binary, simple cycle CP-net over more than 2 variables is impossible!

A

B:

:

a b b

a b b

:

:

b a a

b a a

ab ab

ab ab

Putting Things Together

Theorem: A consistent specification of a binary, simple cycle CP-net over more than 2 variables is impossible!

Surprising observation:There are many cyclic binary CP-nets with

simple cycles SCCs that are consistent!

Theorem: A cyclic binary CP-net is consistent if and only if for every its strongly connected component and for every context of , we have that is consistent given .

Disappointing?

Specific contexts break the cycles!

Overview


– Importance tradeoffs– Adding utilities


Adding Variable Importance into CP-networks

Relative Importance

If it is more important to me that the value of X be high than the value of Y be high, then X is more important than Y.

Processor type is more important to me than operating system (all else being equal).

Operating system is more important than processor type (all else being equal), if the PC is used primarily for

graphical applications.If, given zDom(Z), it is more important to me that the value of X be high than the value of Y be high, then X is conditionally more important than Y.

X Y

X Yz

A

C

B

b b

( ) ( ) :

( ) ( ) :

a b a b c c

a b a b c c

a a

abc

abc

abc

abc

abc

abc

abc

abc

worst

best

A

B

C D

E

:

:

b c c

b c c

e e

a a

:

:

b d d

b d d

:

:

a b b

a b b

:

:

:

be C D

be D C

be D C

nodes variablescp-arcs (directed)

i-arcs (directed)

ci-arcs (undirected)cp-tables

ci-tables

( , ) { , }C D B ES

The same “forward sweep” algorithm works as is for TCP-nets!

Outcome Optimization in TCP-nets

1. The relative importance relations do not play a role in this case.

2. The network is traversed according to a topological ordering induced by the CP-net part of the given TCP-net.

– Any acyclic CP-net induce a partial order over the variables, according to which the variables should be processed.

– Any conditionally acyclic TCP-net induce a hierarchical set of partial orders over the variables.

The central difference between CP-nets and TCP-nets with respect to

constraint-based outcome optimization

Extension to conditionally acyclic TCP-nets

• If any non-dominated solution is enough, then the CP-net based COP is not harder than the corresponding CSP.

• “As is”, the Branch & Bound algorithm for CP-nets, executed on a conditionally acyclic TCP-net, loses its anytime behavior.

• We extended this algorithm to process TCP-net based constraint outcome optimization in an anytime fashion.

Introducing Utilities into CP-networks

Utility Functions

• Utility functions assign a real number to each outcome: u(o1)=0.4, u(o2)=4.3, …

• Actual u-values are significant:– U-values can be used to rank outcomes:

u(o1) < u(o2) implies that o1 is preferred to o2

– But much more! we can assess any uncertain combination of outcomes: the utility of an action yielding with probability 0.4 and with probability 0.6 is 0.4*u(o1) + 0.6*u(o1)

Dominance queries are easy

Optimiziation may be difficult

Generalized Additive Independ.

• The (possibly overlapping) sets X1,...,Xk of variables

are generalized additive independent (GAI) if:

– for any distributions P1, P2, with identical marginals over

the Xi, the expected utility w.r.t. U is the same for P1 and P2

• X1,...,Xk are GAI iff U can be written as

U(v) = i fi(xi )

for some component utility factors fi

UCP-nets – CP nets + Utilities• Utilities permit fine value tradeoffs and effective dominance tests• CP-nets allows effective outcome optimization

• UCP-nets use the graphical structure of a CP-net, but quantify conditional utilities

• U(x) is sum of utility factors; U(abcd) = fA(a)+fB(b)+fC(abc)+fD(cd) =

5 + 4 + .2 + .9 = 10.1

• Utility computation is linear

A B

C

D

a > a5 2

b > b4 3

a b .6 .1a b .2 .8a b .3 .8a b .9 .3

c .9 .8c .2 .3

c c

d d

The CPI-Restriction• So, far, directionality is not exploited

– This is just a GAI decomposition with redundant factors (no need for factors fA or fB; can be incorporated into fC)

• We also require:

DAG G is a valid CP-network for the preference order induced by U

• UCP-net is a specific form of GAI decomposition– Not all GAI-decompositions correspond to a UCP-net

topology

– Not all utility functions can be captured by UCP-nets: the CPI-conditions may not be satisfied

Testing CP-Ind. Condition• CP-Ind conditions implies that not all quantifications lead to

legitimate UCP-nets• A local test exists to verify CPI conditions

• X dominates its children if for all x1,x2 s.t. fX(x1,u) fX(x2,u),u, z, y :

fX(x1,u) - fX(x2,u) i fYi(yix1uizi) - fYi(yix2uizi)

• G is a UCP-net iff each variable dominates its children

• Domination is a local test

• Simpler sufficient conditions exist

X

Y1 Y2

U U2U1

Z1 Z2

Tradeoff Weight Representation

• Let’s normalize utilities to the [0,1] range

• For each X with parents U and factor fX,

– normalize each row of conditional “utility function” fX(—,u) to obtain local value function vu(X)

– for each u Dom(U), specify mulitplicative tradeoff

weight u and additive tradeoff weight u

– utility factor fX(x,u) = u vu(x) + u

• Every UCP-net has a normalized equivalent

Overview



Preference-based Preference-based ConfigurationConfiguration

of Web Page Contentof Web Page Content

Our Goal

Model for presenting the content of a Model for presenting the content of a web-page that:web-page that:

• Reflects the preferences of the web-page designer.

• Adjusts dynamically to the viewer’s current interests.

State of the Art – Categorization

Web page

Component: ArticleArticlePresentation alternatives:

1. Full content2. Partial content3. Heading only4. Link only5. Invisible6. …

Today:Today: Any web page is a collection of components, each one with its static, static, predefined appearance.predefined appearance.

Preference-based Configuration

• Multi-valued components of a web-page create a configuration spaceconfiguration space .

• Preference rankingPreference ranking is a total preorder over : means that configuration is subjectively

equally or more preferred to .

C

C

1 2c c 1c2c

The preference order represents the The preference order represents the subjective preferences of the web page subjective preferences of the web page

designer, not of its viewer.designer, not of its viewer.

However, the actual choices of the viewer affect the choice of configuration.

Why qualitative decision making?

• Utility assessments unlikely to be intuitive in our setting.

• Ordering different options for each component is likely to be a relatively easy task.

Looking for a Model - Objectives

• Explicit ranking of all options – infeasible.

• Desiderata:– Intuitive.– Relatively efficient reasoning.– Capture conditional preference dependencies.

A

B

C D

v ha a

:

:h

v

h

h

v

v

a

a b

b b

b

:

:v v h

h h v

b d d

b d d

:

:v h v

h v h

b c c

b c c

Each component of a web page can be Each component of a web page can be represented by a variable in the CP-represented by a variable in the CP-

network.network.

Authoring ToolHTML

DHTML

Preference specification

Web Browser

1. Interaction1. Interaction

3. Presentation3. Presentation

2. Preference-2. Preference-based based

reconfiguration of reconfiguration of the web-page.the web-page.

Plain HTML (DIV tags)

Parsed HTML

CP-network definition

Preference definition

(1 - present, 0 - hide)

Authoring ToolHTML

DHTML

Preference specification

Web Browser

1. Interaction1. Interaction

3. Presentation3. Presentation

2. Preference-2. Preference-based based

reconfiguration of reconfiguration of the web-page.the web-page.

- CP-network of the web page's components.

- Queue of the recent events (component + value).

Let be the recent event: an interaction of the viewer with

component (setting on/i

i

e

C

Reconfiguration

G

E

off). Then

1. Switch the value of and add into .

If is bigger than some threshold - remove the oldest event from .

2. Project on (specify values of recently observed components wrt. ).

i iC e E

E E

E G E

3. Traverse in a topological order and set each unspecified component

to its most preferred value wrt. the values of its predecessors in .

G

G.

Example• Elections - Presenting is unconditionally preferred to hiding.

• Traffic Accident – Hiding is unconditionally preferred to presenting.

• New Airbag – Presenting is preferred only if Traffic Accident is presented, and Elections is hidden.

• NBA – Presenting is preferred only if Traffic Accident is not presented.

• NY Times comm. – Presenting is preferred only if both Elections and Traffic Accident are presented.

• Volvo comm. – Presenting is preferred only if either New Airbag or Traffic Accident is presented.

• Nike comm. – Presenting is preferred only if NBA is presented.

TrafficAccident

Elections

New Airbag NBA

Volvo Nike

NY Times

Corresponding CP-network

TrafficAccident

Elections

New Airbag NBA

Volvo Nike

NY Times

Initially optimal presentation

TrafficAccident

Elections

New Airbag NBA

Volvo Nike

NY Times

Viewer opened “Traffic Accident”

TrafficAccident

Elections

New Airbag NBA

Volvo Nike

NY Times

Viewer closed “Elections”

TrafficAccident

Elections

New Airbag NBA

Volvo Nike

NY Times

TrafficAccident

Elections

New Airbag NBA

Volvo Nike

NY Times

What if the event queue is bigger?

TrafficAccident

Elections

New Airbag NBA

Volvo Nike

NY Times

TrafficAccident

Elections

New Airbag NBA

Volvo Nike

NY Times

Integrating Global Constraints

What if the constraints and the preferences are not defined over the same set of variables?

No obvious way to utilize preferences for search space pruning …

• Example - User Interface Configuration– Preferences are defined over the content

alternatives for the different UI components.– Geometric layout constraints.

Integrating Layout Constraints

• Layout constraints are represented as an integer linear program.

• Reconfiguration is adopted to look for a feasible, Pareto-optimal presentation.

Generally, this task is computationally hard.

Highly Constrained Problems

Reasonable approach:

• Forget about the CP-net, and generate all the solutions for the CSP.

• Using the CP-net, filter all the dominated solutions.

Generally, filtering may be painful ...

Given an outcome a, it is easy to determine the complete set of outcomes {a1 , …, am } that are (preferentially) minimally worse than a .

Preferentially optimal outcome

Underconstrained Problems

Potential Applications

• Multi-agent meeting scheduler– CP-nets support efficient computation of Pareto-

optimal solutions

• Filtering the results of search– Context can be use to disambiguate words, focus on the

more important terms

• Scheduling problems– Various constraints are more important than others

Overview



• Possible connections with non-monotonic reasoning.

• Model acquisition from a restricted natural language text.– Possibility for an intuitive closed loop system …

• Analyzing domains in which qualitative modeling of preferences seem to be required.– Product configuration (!)– Computational Biology (?)

Additional directions

Modeling Qualitative Preferences Using the CP-net Model

Documents

Transcript of Modeling Qualitative Preferences Using the CP-net Model