CS483/683 Multi-Agent Systems
description
Transcript of CS483/683 Multi-Agent Systems
CS483/683 Multi-Agent Systems
Lectures 5-6:From Satisfaction to Optimization
ADOPT: Asynchronous Distributed Optimization
2-4 February 2010Instructor: Kostas Bekris
Computer Science & Engineering, University of Nevada, Reno
CS483
Distributed Constrained Optimization
DCOP:
•A set of n variables V={X1, ..., Xn}
•Each variable has a discrete domain: D1, ..., Dn
•Each variable is assigned to an agent
- Only the agent who is assigned a variable has control over its value
- And knowledge of its domain
•Assign values to variable so as to optimize a global objective function
- The optimization of the function satisfies a set of constraints
Requirements:
•Distributed computation using only local communication
•Fast, asynchronous computation - agents should work in parallel
•Quality guarantees are needed
- Provably optimal solutions whenever possible
- Trade-off between computation and solution quality
✓Bounded-error approximation: Guarantee solution within a distance from optimal, less time than the optimal
CS483 ADOPT: Asynchronous Distributed Optimization
The objective function is a summation over a set of cost functions
F(A)=∑Xi,Xj∈V fij(di,dj)
Xi←di, Xj←dj in A
We want to find A* that minimizes F(A)
e.g. F( { (X1,0), (X2,0), (X3,0), (X4,0) } ) = 4
F( { (X1,1), (X2,1), (X3,1), (X4,1) } ) = 0
XX11
XX22
XX33 XX44
di djfij(di,dj
)
0 0 1
0 1 2
1 0 2
1 1 0Constraint Graph
CS483
Assumptions
1. Summation operation over cost function
•Associative
•Commutative
•Monotonic
- Cost of a solution can only increase as more costs are aggregated (i.e., we cannot have negative cost)
2. Constraints are at most binary
•There are ways to extend to constraints that involve a larger number of variables
3. Each agent is assigned a single variable
•There is a way to extend to the case that an agent must handle multiple variables
CS483
Key Ideas in ADOPT
1. Opportunistic best-first search
•Agents are prioritized in a tree structure
- an agent has a single parent and multiple children
•Each agent keeps on choosing the best value based on the current available information
- i.e., chooses the variable which implies the smallest lower bound
- lower bounds do not need global information to be estimated
•Each agent maintains a lower and an upper bound for the cost of its subtrees
- and informs its parent about its own bounds
•Strategy allows agents to abandon partial solutions which have not been proven to be suboptimal
- they may have to reconsider the same assignments into the future
XX11
XX22
XX33 XX44
Communication Graph
CS483
Key Ideas in ADOPT
2. Backtrack Threshold
•When an agent knows from previous search experiences that lb is a lower bound for its subtree
- inform the subtree agents not to bother searching for a solution whose cost is lower than lb
•In the general case, remembering these lower bounds for past assignments requires exponential space
- Approach remembers only one value and then cost is subdivided to children arbitrarily and adapted on the fly as new computations are executed
3. Built-in Termination Detection
•Keeping track of bounds (lower and upper bound for the cost function) on each agent
- allows to keep track of the progress towards the optimum solution
- and automatically terminates when necessary
CS483
Messages and Data Structures
VALUE message (like ok?)
Send selected value to children along the
constraint graph
COST message (like NoGood)
Send to parents along the communication graph
THRESHOLD message
Send to children along the communication graph
Each agent maintains the “context”
(like the “agent_view”)
•A recorf of higher priority neighbors’ current variable assignment
Two contexts are compatible if they do not disagree on any variable assignment
XX11
XX22
XX33 XX44
Constraint Graph
XX11
XX22
XX33 XX44
Communication Graph
CS483
COST message
Xk transmits COST message to Xi
Message contains
•context of Xk
•lb of Xk
•up of Xk
When Xi receives the message it stores
•lb(d,Xk)
•ub(d,Xk)
where d is the assignment of Xi in Xk’s context
If context of Xi is incompatible with the context of Xk:
•lb(d,Xk) = 0
•ub(d,Xk) = ∞
XXii
XXkk
CS483
Costs and Bounds
Local cost: δ(di) = ∑(Xj,dj) fij(di,dj)
di : assignment of agent Xi
Xj : higher priority neighbors than Xi
Lower bound for value d:
∀ d ∈ Di:
LB(d) = δ(d) + ∑ Xk ∈ children lb(d,Xk)
similarly for the upper bound for value d
Lower bound:
LB = min d ∈ Di LB(d)
similarly for the upper bound
For leaves: LB(d) = UB(d) = δ(d)
If not a leaf but has not get received a COST message: LB = δ(d) and UB = ∞
XX11
XX22
XX33 XX44
Constraint Graph
XX11
XX22
XX33 XX44
Communication Graph
CS483
When does Xi change value?
Whenever LB(di) exceeds the backtrack threshold value, Xi changes its variable value to one with smaller lower bound
The threshold is updated with the following three ways:
•Its value can increase whenever Xi determines that LB is greater than the current threshold
- guarantees that there is always a variable with a lower bound than the threshold
•Its value can decrease whenever Xi determines that UB is lower than the current threshold
Invariant: LB ≤ threshold ≤ UB
•Its value is also updated whenever a THRESHOLD message is received from a parent
- a parent subdivides its own threshold value among its children
- t(d,Xk): the threshold on cost allocated by parent Xi to child Xk
- then the value of t(d,Xk) respects the following invariants:
✓threshold = δ(di) + ∑ Xk ∈ children t(di,Xk)
✓∀ d ∈ Di, ∀ Xk ∈ children: lb(d,Xk) ≤ t(d,Xk) ≤ ub(d,Xk)
CS483
Example
All agents begin concurrently choosing 0.
Each agents send a VALUE message to lower priority agents along the constraint graph
We will follow one specific execution path - there are many possible
XX11
XX22
XX33 XX44
Communication Graph
di djfij(di,dj
)
0 0 1
0 1 2
1 0 2
1 1 0
XX11
XX22
XX33 XX44
Constraint Graph
CS483
Example
X2 receives X1’s VALUE message
and records this value to its context
X2’s context: {X1=0}
Then it computes bounds:
•LB(0) = δ(0) + lb(0,X3) + lb(0,X4) = 1
•LB(1) = δ(1) + lb(1,X3) + lb(1,X4) = 2
•LB(0) < LB(1) ⇒ LB = LB(0) = 1
Similarly: UB = ∞
threshold was set to LB(0), equal to 1 so the invariant holds
Transmits a COST message to X1:
COST( {X1=0}, 1, ∞ )
XX11=0=0
XX22=0=0
XX44=0=0XX33=0=0
di djfij(di,dj
)
0 0 1
0 1 2
1 0 2
1 1 0
CS483
Example
X3 receives X1’s and X2’s VALUE messages
and records these values to its context
X3’s context: {X1=0, X2=0}
Then it computes bounds:
•LB(0) = δ(0) = 1 + 1 = 2
•LB(1) = δ(1) = 2 + 2 = 4
•LB(0) < LB(1) ⇒ LB = LB(0) = 2
Similarly: UB = 2
threshold was LB(0) = 2 so the invariant holds
Transmits a COST message to X2:
COST( {X1=0, X2=0}, 2, 2 )
Similarly with X4... but no reference to X1
XX11=0=0
XX22=0=0
XX44=0=0XX33=0=0
di djfij(di,dj
)
0 0 1
0 1 2
1 0 2
1 1 0
CS483
Example
X1 receives X2’s COST message
COST( {X1=0}, 1, ∞ )
•test if compatible with its current context
•store: lb(0,X2) = 1 and ub(0,X2) = ∞
Then it computes bounds:
•LB(0) = δ(0) + lb(0,X2) = 0 + 1 = 1
•LB(1) = δ(1) + lb(1,X2) = 0 + 0 = 0
•LB(1) < LB(0) ⇒ LB = LB(1) = 0
Similarly: UB = ∞
threshold was 0, but LB(0) = 1:
•violation of the invariant, change assignment
Send VALUE messages to children
XX11=1=1
XX22=0=0
XX44=0=0XX33=0=0
di djfij(di,dj
)
0 0 1
0 1 2
1 0 2
1 1 0
CS483
Example
Assume COST messages from X3 and X4 are delayed... instead VALUE message from X1 arrives first at X2
•Current context at X2: {X1=1}
When X2 receives the COST messages from X3 its context will be incompatible with X2’s
•the bounds in the message will not be stored
The message from X4 is not incompatible:
•store lb(0,x4) = 1 and up(0,x4) = 1
Then it computes bounds:
•LB(0) = δ(0) + lb(0,X3) + lb(0,X4) = 2+ 0 +1 = 3
•LB(1) = δ(1) + lb(1,X3) + lb(1,X4) = 0 +0 +0 = 0
•LB(1) < LB(0) ⇒ LB = LB(1) = 1
Similarly: UB = ∞
XX11=1=1
XX22=1=1
XX44=0=0XX33=0=0
di djfij(di,dj
)
0 0 1
0 1 2
1 0 2
1 1 0
CS483
Example
X2 will inform X3 and X4 about the changes.
Similar changes will take place on X3 and X4:
•1 will be selected as the value
COST messages will be transmitted:
•from X4 to X2: ( {X2=1}, 0, 0 )
•from X3 to X1 and X2: ( {X1=1,X2=1}, 0, 0 )
•from X2 to X1: ( {X1=1}, 0, 0)
- this is after receiving the COST messages from X3 and X4
Upon receipt of the COST message of X2 at X1:
•LB = UB = threshold = 0
X1 sends TERMINATE messages to other agents.
XX11=1=1
XX22=1=1
XX44=1=1XX33=1=1
di djfij(di,dj
)
0 0 1
0 1 2
1 0 2
1 1 0