Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings
description
Transcript of Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings
![Page 1: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/1.jpg)
Decentralised Coordination of Continuously Valued Control Parameters using the
Max-Sum Algorithm
Ruben Stranders, Alessandro Farinelli, Alex Rogers, Nick JenningsSchool of Electronics and Computer Science
University of Southampton{rs06r, af2, acr, nrj}@ecs.soton.ac.uk
![Page 2: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/2.jpg)
2
This presentation focuses on the use of Max-Sum in coordination problems with continuous parameters
From Discrete to Continuous
Max-Sum for Decentralised Coordination
Empirical Evaluation
![Page 3: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/3.jpg)
3
Max-Sum is a powerful algorithm for solving DCOPs
Complete Algorithms
DPOPOptAPOADOPT
Communication Cost
Iterative AlgorithmsBest Response (BR)
Distributed Stochastic Algorithm (DSA)
Fictitious Play (FP)
Max-SumAlgorithm
Optimality
![Page 4: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/4.jpg)
Max-Sum solves the social welfare maximisation problem in a decentralised way
Agents
![Page 5: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/5.jpg)
Max-Sum solves the social welfare maximisation problem in a decentralised way
1x
2x
3x
4x
5x
6x
7x8x
Control Parameters
![Page 6: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/6.jpg)
Max-Sum solves the social welfare maximisation problem in a decentralised way
1U
2U
3U
4U
5U
6U
7U8U
Utility Functions
![Page 7: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/7.jpg)
Max-Sum solves the social welfare maximisation problem in a decentralised way
)( 33 xU
Localised Interaction
},,,{ 54313 xxxxx
![Page 8: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/8.jpg)
Max-Sum solves the social welfare maximisation problem in a decentralised way
Agents
M
iiiU
1
)(maxarg xx
Social welfare:
![Page 9: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/9.jpg)
The input for the Max-Sum algorithm is a graphical representation of the problem: a Factor Graph
Variable nodes Function nodes
1x
2x
3x
1U
2U
3U
Agent 1Agent 2
Agent 3
![Page 10: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/10.jpg)
Max-Sum solves the social welfare maximisation problem by message passing
1x
2x
3x
1U
2U
3U
Variable nodes Function nodes
Agent 1Agent 2
Agent 3
![Page 11: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/11.jpg)
Max-Sum solves the social welfare maximisation problem by message passing
jiadjk
iikiji xrxq\)(
)()(
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
From variable i to function j
From function j to variable i
![Page 12: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/12.jpg)
Until now, Max-Sum was only defined for discretely valued variables
Graph Colouring)( iji xq )( iij xr
![Page 13: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/13.jpg)
However, many problems are inherently continuous. Heading
andVelocity
Unattended Ground Sensor
ActivationTime
Autonomous Ground Robot
Thermostat
Preferred RoomTemperature
![Page 14: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/14.jpg)
So, we extended the Max-Sum algorithm to operate in continuous action spaces
Discrete Continuous
![Page 15: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/15.jpg)
We focussed on utility functions that are Continuous Piecewise Linear Functions (CPLFs)
![Page 16: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/16.jpg)
We focussed on utility functions that are Continuous Piecewise Linear Functions (CPLFs)
)( iji xq )( iij xr
“Continuous” Graph Colouring
![Page 17: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/17.jpg)
A CPLF is defined by a domain partitioning followed by value assignment
![Page 18: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/18.jpg)
A CPLF is defined by a domain partitioning followed by value assignment
![Page 19: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/19.jpg)
A CPLF is defined by a domain partitioning followed by value assignment
![Page 20: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/20.jpg)
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
To make Max-Sum work on CPLFs, we need to define key two operations on them
jiadjk
iikiji xrxq\)(
)()(From variable i to function j
From function j to variable i
![Page 21: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/21.jpg)
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
To make Max-Sum work on CPLFs, we need to define key two operations on them
jiadjk
iikiji xrxq\)(
)()(From variable i to function j
From function j to variable i1. Addition
of two CPLFs
![Page 22: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/22.jpg)
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
To make Max-Sum work on CPLFs, we need to define key two operations on them
jiadjk
iikiji xrxq\)(
)()(From variable i to function j
From function j to variable i
2. Marginal Maximisation to a single variable
![Page 23: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/23.jpg)
Addition of two CPLFs involves merging their domains, and then summing their values
1x
2x
1x
2x
1f 2f
![Page 24: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/24.jpg)
Addition of two CPLFs involves merging their domains, and then summing their values
1x
2x
1x
2x
1x
2x
1. Merge domains
1f 2f
21 ff
![Page 25: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/25.jpg)
Addition of two CPLFs involves merging their domains, and then summing their values
![Page 26: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/26.jpg)
Addition of two CPLFs involves merging their domains, and then summing their values
2. Sum Values
![Page 27: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/27.jpg)
Marginal maximisation is the operation of finding the maximum value of a function, if we fix all but one variable
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
From function j to variable i:
![Page 28: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/28.jpg)
Marginal maximisation involves finding the maximum value of a function, if we fix all but one variable
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
)(max)(\ jjii fxgj
xx
![Page 29: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/29.jpg)
Marginal maximisation involves finding the maximum value of a function, if we fix all but one variable
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
)(max)(\ jii fxgj
xx
),(max)( 2112
xxfxgx
Example: bivariate function:
![Page 30: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/30.jpg)
Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction
),( 1 fxProject onto
axis
![Page 31: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/31.jpg)
Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction
Project onto
axis),( 1 fx
![Page 32: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/32.jpg)
Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction
Project onto
axis),( 1 fx
Result of projection
![Page 33: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/33.jpg)
Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction
Extract UpperEnvelope
![Page 34: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/34.jpg)
Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction
Extract UpperEnvelope
),(max)( 2112
xxfxgx
)( 1xg
![Page 35: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/35.jpg)
We empirically evaluated this algorithm in a wide-area surveillance scenario
Dense deployment of sensors to detect activity within an urban environment.
Unattended GroundSensor
![Page 36: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/36.jpg)
Sensors adapt their duty cycles to maximise event detection by coordinating with overlapping sensors
time
duty cycleDiscrete
time
duty cycle
time
duty cycle
Discretised time
![Page 37: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/37.jpg)
Sensors adapt their duty cycles to maximise event detection by coordinating with overlapping sensors
time
duty cycleDiscrete Continuous
time
duty cycle
time
duty cycle
time
duty cycle
time
duty cycle
time
duty cycle
![Page 38: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/38.jpg)
Continuous Max-Sum outperforms Discrete Max-Sum by up to 10%
0 5 10 15 20 25 300.75
0.80
0.85
0.90
0.95
1.00
ContinuousDiscrete
Discretisation
Solu
tion
Qua
lity
(as f
racti
on o
f opti
mal
)
Average Solution Quality over 25 Iterations
![Page 39: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/39.jpg)
Tota
l Mes
sage
Size
Continuous Max-Sum leads to more effective use of communication resources than Discrete Max-Sum
0 5 10 15 20 25 300
5000
10000
15000
20000
ContinuousDiscrete
Discretisation
Total number of values exchanged between agents
![Page 40: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/40.jpg)
In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum
1. No artificial discretisation
time
time
![Page 41: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/41.jpg)
In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum
1. No artificial discretisation Continuous
Discrete
2. Better solutions
time
timeSo
lutio
n Q
ualit
y
![Page 42: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/42.jpg)
In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum
time
time1. No artificial discretisation Continuous
Discrete
2. Better solutions ContinuousDiscrete
3. Effective communication
Solu
tion
Qua
lity
Mes
sage
Size
![Page 43: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/43.jpg)
For future work, we wish to extend the algorithm to arbitrary continuous functions
For example, using Gaussian Processes
![Page 44: Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings](https://reader035.fdocuments.net/reader035/viewer/2022062501/568161ec550346895dd2217f/html5/thumbnails/44.jpg)
In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum
time
time1. No artificial discretisation Continuous
Discrete
2. Better solutions ContinuousDiscrete
3. Effective communication
Solu
tion
Qua
lity
Mes
sage
Size
Questions?