A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R....
-
date post
19-Dec-2015 -
Category
Documents
-
view
216 -
download
0
Transcript of A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R....
![Page 1: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/1.jpg)
A Decision-Theoretic Approach to A Decision-Theoretic Approach to Designing Proactive Communication in Designing Proactive Communication in
Multi-Agent TeamworkMulti-Agent Teamwork
Thomas R. Ioerger, Yu Zhang,
Richard Volz, John Yen (PSU-IST)
Dept. of Computer Science
Texas A&M University
![Page 2: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/2.jpg)
2
MotivationMotivation
AgentMulti-Agent
Team Agents share a large amount of knowledge aboutthe teamwork.Hard coded Interactions amongparticipants.High-frequency message exchange.Communication risk.
![Page 3: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/3.jpg)
3
Challenging Issues in Designing Challenging Issues in Designing Communication ProtocolsCommunication Protocols
Each agent has incomplete information from which uncertainties arise.
Each agent has different problem solving capabilities.
Data are decentralized and lack systems’ global control.
Excessive/unrestricted communication leads to lack of scalability
![Page 4: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/4.jpg)
4
Our Approach and Its ContributionsOur Approach and Its Contributions
Proactive CommunicationOBPC: Reduction of communication load
through OBservations.
DIP: Dynamic estimation of the probability distribution of Information Production and need.
DTPC: Decision-Theoretic determination of communication strategies.
![Page 5: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/5.jpg)
5
BackgroundBackground CAST (Collab. Agents for Simulating Teamwork) MALLET (Multi-Agent Logic-based Language for
Encoding Teamwork)
(team-plan killwumpus(?w) (process (seq (agent-bind ?ca (constraint (play-role ?ca scout))) (DO ?ca (findwumpus ?w))) (agent-bind ?fi (constraint ((play-role ?fi fighter)
(closest-to-wumpus ?fi ?w)))) (DO ?fi (movetowumpus ?w)) (DO ?fi (shootwumpus ?w))))))
(ioper shootwumpus (?w) (pre-cond (wumpus ?w) (location ?w ?x ?y) (dead ?w false)) (effect (dead ?w true)))
![Page 6: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/6.jpg)
6
OverviewOverview
CASTCASTKB
KB
KB
KB
KB
KBProactive Communication
Proactive Communication
OBPCOBPC
DIP DIP DTPCDTPC
Optimal Communication Strategy
Team Structure & Teamwork Procedure
![Page 7: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/7.jpg)
7
Agent Execution CycleAgent Execution Cycle
ObserveSense Predict
Info. need and production
DecideStrategyCommunicate
Information
ActEffect
ExecutionCycle
![Page 8: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/8.jpg)
8
Syntax of ObservabilitySyntax of Observability
<observability> ::= (CanSee <viewing>)* (BelieveCanSee <believer><viewing>)*
<viewing> ::= <observer><observable> <cond><believer> ::= <agent><observer> ::= <agent><observable> ::= <property>|<action><cond> ::= (<property>)*<property> ::= (<property-name> <object> <args>)<action> ::= (DO <doer> (<operator-name> <args>))<object> ::= <agent>|<non-agent><doer> ::= <agent>
![Page 9: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/9.jpg)
9
Example Observability RulesExample Observability Rules(CanSee ca (location ?o ?x ?y) (location ca ?xc ?yc) (location ?o ?x ?y) (inradius ?x ?y ?xc ?yc rca)) //The carrier can see the location property of an object.
(CanSee ca (DO ?fi (shootwumpus ?w)) (play-role fighter ?fi) (location ca ?xc ?yc) (location ?fi ?x ?y) (adjacent ?xc ?yc ?x ?y) ) //The carrier can see the shootwumpus action of a fighter.
(BelieveCanSee ca fi (location ?o ?x ?y) (location fi ?xi ?yi) (location ?o ?x ?y) (inradius ?x ?y ?xi ?yi rfi)) //The carrier believes the fighter is able to see the location property of an object.
(BelieveCanSee ca fi (DO ?f (shootwumpus ?w)) (play-role fighter ?f) ( ?f fi) (location ca ?xc ?yc) (location fi ?xi ?yi) (location ?f ?x ?
y) (inradius ?xi ?yi ?xc ?yc rca) (inradius ?x ?y ?xc ?yc rca) (adjacent ?x ?y ?xi ?yi)) //The carrier believes the fighter is able to see the shootwumpus action of another
fighter.
![Page 10: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/10.jpg)
10
Proactive Communication Based Proactive Communication Based on Observationon Observation
ProactiveTell– A provider reasons about what information it will have.– A provider reasons about whether to deliver a piece of
information when having the information.
ActiveAsk– A needer reasons about what information it will need.
– A needer reasons about whether to ask for a piece of information when needing the information.
![Page 11: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/11.jpg)
11
EvaluationEvaluation
20 wumpuses, 8 pits, and 20 piles of gold per world.
1 carrier and 3 fighters compose a team.
The team goal is to kill wumpuses and get the gold without being killed.
5 randomly generated worlds with 20×20 cells.
Multi-Agent Wumpus World
![Page 12: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/12.jpg)
12
Decision-Theoretic Proactive Decision-Theoretic Proactive CommunicationCommunication
StrategiesUtility FunctionCost FunctionValue FunctionDecision-Making
![Page 13: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/13.jpg)
13
Decision-Making on Situation PADecision-Making on Situation PA
0
1
2
e
ea-b: ProactiveTell
a-b: Silence
b-a: Accept
b-a: Wait
b-a: Silence
e
e
b-a: ActiveAsk
Situation PA: Provider produces a new piece of information
a: provider b: needer e: end
![Page 14: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/14.jpg)
14
DM on Situation PBDM on Situation PB
0
a-b: Reply
ea-b: WaitUntilNext
Situation PB: Provider receives a request for a piece of information
e
![Page 15: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/15.jpg)
15
DM on Situation NADM on Situation NA
b-a: ActiveAsk
b-a: Silence
b-a: Wait
a-b: Reply
a-b: WaitUntilNext
a-b: Silence
a-b: ProactiveTell
Situation NA: Needer needs a piece of information
01
0
t
t
e
e
e
t: transfer
![Page 16: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/16.jpg)
16
DM on Situation NBDM on Situation NB
Situation NB: Needer receives a piece of information
t
0 eb-a: Accept
![Page 17: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/17.jpg)
17
Utility FunctionUtility Function
Parameters in utility function:
– I: information about which communication occurs
– t: time of decision-making
– t1: time at which I is needed
– t2: time at which the value for I used is produced
– SU: situation at t
– S: strategy available at SU
– M: a set of messages involving in obtaining I
– E: environment state at t
U(I, t, t1, t2, SU, S, M, E)
=V(I, t, t1, t2, SU, S)–C(M)
![Page 18: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/18.jpg)
18
Value FunctionValue Function
V(I, t, t1, t2, SU, S)
=T(I, t, t1, t2, SU, S)//Timeliness
+R(I, t, t1, t2, SU, S)//Relevance
![Page 19: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/19.jpg)
19
Timeliness– Whether agents use a value that can be produced in
time when they need I.
d(I, t, t1, t2, SU, S) = max(0, t2–t1)
ft(d(I, t, t1, t2, SU, S))s.t. ft(x) < ft(y) if y < x
T(I, t, t1, t2, SU, S) = ft(d(I, t, t1, t2, SU, S))
Timeliness FunctionTimeliness Function
![Page 20: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/20.jpg)
20
Relevance FunctionRelevance Function
Relevance– Unprocessed, Most recent, Important
P(I, t, t1, t2, SU, S) = Pr(I t t1 t2 no other value for I was produced
between Int[t1,t2] | S SU)
frI(P(I, t, t1, t2, SU, S))s.t. frI(x) < frI(y) if x < y
R(I, t, t1, t2, SU, S) = frI(P(I, t, t1, t2, SU, S))
![Page 21: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/21.jpg)
21
Cost FunctionCost Function
0 if Mi=
C(Mi) =
k1 + k2 × len(Mi) otherwise
![Page 22: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/22.jpg)
22
Expected UtilityExpected Utility
E(U) =
Time
Strategy
t1 t2
P.ProactiveTell
P.Silence +T
P.Reply
P.WaitUntilNext
N.ActiveAsk if a Reply
if a WaitUnitlNext
N.Silence
N.Wait if a ProactiveTell
+T if a Silence
N.Accept
ufNbT ,
0,PaTuf
NbT ,0,PaT
qbT ,1
,q
PaT
0,q
PaT
qbT ,
0,NbT
rbT ,
0,a
PaT1
,a
PaT0,NbT
0,NbT
1,a
PaT
gbT ,1
,gNbT
0,NbT
1s 2st t 2121r )t,U(t)t,t(P
![Page 23: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/23.jpg)
23
StrategiesStrategies
xNbT ,
ufNbT ,
nsPaT ,
0,PaT
1,PaT
lsPaT ,
t
Current time
Unknown
Known
Next production
Last sent
Last notsent
Last need aware of
Unfulfilled need
Situation PA: Situation PA: provider produces I
ProactiveTell?Silence?
![Page 24: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/24.jpg)
24
StrategiesStrategies
qbT ,
1,q
PaT
0,q
PaTt
Current time
Unknown
Known
Next production
Last production
Situation PB:Situation PB: provider receives a request for I
Reply? WaitUntilNext?
![Page 25: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/25.jpg)
25
StrategiesStrategies
0,NbTrbT ,
0,a
PaT 1,a
PaT
t
Current time
Unknown
Known
Next production
Last I received
Most recentproduction
Situation NA: Situation NA: needer needs I
ActiveAsk?Wait?
Silence?
![Page 26: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/26.jpg)
26
StrategiesStrategiesSituation NB: Situation NB: needer receives I
Accept
![Page 27: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/27.jpg)
27
Summary• Advantages of Approach: allows agents to
make intelligent choices of communication policy based on:– frequencies: of needs, of sensing, of info. change– costs: of messages, plus penalities for delays in
action, or acting with incorrect information
![Page 28: A Decision-Theoretic Approach to Designing Proactive Communication in Multi-Agent Teamwork Thomas R. Ioerger, Yu Zhang, Richard Volz, John Yen (PSU-IST)](https://reader036.fdocuments.net/reader036/viewer/2022062516/56649d3f5503460f94a189bc/html5/thumbnails/28.jpg)
28
CriteriaCriteria for for Applicable DomainsApplicable DomainsThere are information needs among the team.
Agents can communicate.
There is uncertainty in the environment. – Stochastic properties of teamwork process.– Agents have incomplete/disjoint knowledge
about the world.
The team acts under critical time constraints, so proactive assistance becomes important.