Optimizing Robotic Team Performance with Probabilistic Model … · 2015-04-28 · 2. Observe the...

© 2014 Carnegie Mellon University

Optimizing Robotic Team

Performance with

Probabilistic Model Checking

Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Sagar Chaki, Joseph Giampapa, David Kyle (presenting), John Lehoczky October 22nd, 2014

Report Documentation Page Form ApprovedOMB No. 0704-0188

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.

1. REPORT DATE 22 OCT 2014

2. REPORT TYPE N/A

3. DATES COVERED

4. TITLE AND SUBTITLE Optimizing Robotic Team Performance with Probabilistic Model Checking

5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) Kyle /David S.

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Software Engineering Institute Carnegie Mellon University Pittsburgh,PA 15213

8. PERFORMING ORGANIZATIONREPORT NUMBER

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)

11. SPONSOR/MONITOR’S REPORT NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release, distribution unlimited.

13. SUPPLEMENTARY NOTES

14. ABSTRACT

15. SUBJECT TERMS

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT

SAR

18. NUMBEROF PAGES

19

19a. NAME OFRESPONSIBLE PERSON

a. REPORT unclassified

b. ABSTRACT unclassified

c. THIS PAGE unclassified

Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18

2

SIMPAR 2014, Bergamo, Italy

David Kyle, 0ct. 22, 2014


Copyright 2014 Carnegie Mellon University This material is based upon work funded and supported by the Department of Defense under Contract No. FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Department of Defense. NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN “AS-IS” BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. This material has been approved for public release and unlimited distribution except as restricted below. This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at [email protected]. DM-0001796

3




Model Checking

Pentium floating point bug (1995): inspired Intel to model check chips

Now being applied to software, as well

System of Interest (e.g., Code or Chip Design)

Model

Properties

of Interest

Model

Checker (typically based on

boolean-satisfiability)

Proof of Correctness

Counterexample

4




Probabilistic Model Checking

Model Checking is purely boolean; a property is true or false.

For some systems, we want probabilities

System of Interest (e.g., Code or Chip Design)

Model (e.g., Markov chain)

Properties

of Interest

Probabilistic

Model

Checker

Analysis Output Probability of each Property

Many kinds exist;

we use Discrete Time

Markov Chains (DTMC)

5




DTMCs and Multi-Agent Robotic Systems

• Benefits:

1. Performance vs physics-based simulation

2. Exact results. Given a model, probabilities are calculated exactly

• Essential problems:

1. Modelling physical systems is difficult

– Can’t just extract from a design or program code; must observe system to model it

– Physical systems are continuous. Probabilistic Model Checking relies on discrete states

– Given an imperfect model based on finite observations, how does that impact predictions?

2. Robots interact. Modelling an entire system of multiple robots is hard.

6




Our Contributions

1. Model robots individually:

1. observe and measure individual behavior

2. discretize observations in time and space, create Markov models

3. compose these models into a Markov model of the whole system

2. Use known statistical error on the measurements made of the individual robots to produce estimates of error of the outputs of model checking the whole system.

7




S2 S1

Scenario

For our experiments, and as an illustration, we imagine a mine sweeping scenario. Objective: find a mine/IED in a constrained space (i.e., a drainage culvert under a road).

M B

8




Discrete Time Markov Chains

DTMCs have:

• A set of states, each representing a discrete point in time

• Transitions between states, with probabilities associated. Probabilities are based only on the current state.

Searching

Waiting

Found, Returning Not found, Returning

Not found, Returned Found, Returned

0.8

0.3 0.7

0.2

0.5 0.4

Lost

0.5 0.6

9




Modal DTMCs

We contribute a variant:

• States can also have “mode change” transitions.

• Mode changes can represent interaction between robots, or the environment.

• In our paper, we show how to convert to a basic DTMC for use with existing model checkers.

Searching

Waiting

Found, Returning Not found, Returning

Not found, Returned Found, Returned

GO

MINE TIMEOUT

0.5 0.4

Lost

0.5 0.6

10




Discrete Time and Space

We represent robot position with states representing different grid positions, and different times.

R2, C1, T1

Waiting

R2, C2, T2 R1, C2, T2

R1, C3, T3 R2, C3, T3

R1

R3

R2

C1 C2 C3 C4 C5 C6 C7 C8

R2, C4, T4 R2, C2, T4 …

R2, C5, T5

…

R2, C1, T5

Found, and Returned

MINE

GO

REPORTED

0.5 0.5

1 1

1 1

1

1

1

11




Composing Modal DTMCs

• Modal DTMCs allow us to model individual robots, then easily compose them together.

• To create an individual model:

1. Run the robots individually, with pre-planned mode changes

2. Observe the robot’s behavior

3. Create a Modal DTMC with transition probabilities based on observation, and mode changes as pre-planned

• Then, collect the individual modal DTMCs into a whole-system modal DTMC, and convert it to a non-modal DTMC

• Details of this construction, and correctness proof, are in the paper.

12




Error Estimation

• Probabilistic Model Checking itself has no error; given a model, it finds exact probabilities.

• However, modeling a robotic system will certainly not be perfect.

• Many kinds of error might appear causing a model to not reflect reality. We looked at handling one: the statistical errors due to observing the individual robots only a finite number of times.

• To examine this specific kind of error, we assume:

• That the system can be fully described by a DTMC

• That we have figured out the states of that DTMC

• But the transition probabilities are observed over the course of finite trials

13




Dirichlet-based Distribution of DTMCs

To analyze error, we create a random distribution of DTMCs.

For each transition, we use the counts of the times that transition was observed to describe the Dirichlet distribution of transition probabilities for that state, which includes a variance which shrinks with more observations.

State 0

State 1 State 2 State 3

12 3 8

14




Model Checking a distribution of DTMCs

We randomly generate a large number of DTMCs from the distribution we created. We model check each DTMC, which gives us probabilities for our properties of interest. This gives us a mean and standard deviation for those outputs.

State 0


12 3

8

State 0


0.52 0.13

0.35 State 0


0.52 0.13

0.38 State 0


0.52 0.13

0.32 State 0


0.52 0.13

0.35

15




Experiment

• Used the simulator V-REP, with Kilobot models based on observations of real Kilobots

• Simulated Kilobots individually, used observations to create models for various team configurations (using Modal DTMCs), and predicted outcomes, using our Dirichlet sampling technique. Our metrics:

• Probability base learns of mine (SUCCESS)

• Expected number of bots that return to base (RETURNED)

• Simulated those teams in V-REP, and compared those outcomes to predicted outcomes.

16




Experiment Results – SUCCESS metric

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

3-2-1 4-6-1 4-6-2 5-6-2 5-6-7 6-1-7 6-5-7 7-3-5 7-3-6 7-6-1

95-%ile

5-%ile

Mean

Observed

'1111 r ~ , ; ; 0 ~ .. u • • ~~ • l ~

~ ~

u u

n ~ ~

~ ~ ~

~~ • I .,

0 •

• I I I I

...... ..... I I • I I I I

-- Software Engineering Institute Carnegie l\ lellon UniYt~rsity

17




0

0.5

1

1.5

2

2.5

3-2-1 4-6-1 4-6-2 5-6-2 5-6-7 6-1-7 6-5-7 7-3-5 7-3-6 7-6-1

95-%ile

5-%ile

Mean

Observed

Experiment Results – RETURNED metric

--

l • • ' :

' • ' i I I I

• ' :

! I I

1 I I

• • ~

Software Engineering Institute Carnegie l\ lellon UniYt~rsity

•

I I

• ' • • I

•

• I

18




Questions?

19




Contact Information

Presenter

David Kyle

CPS/ULS initiative, DART project

Telephone: +1 412-268-9636

Email: [email protected]

Software Engineering Institute

4500 Fifth Avenue

Pittsburgh, PA 15213-2612

USA

Web

www.sei.cmu.edu

www.sei.cmu.edu/contact.cfm

SEI Email: [email protected]

SEI Phone: +1 412-268-5800

SEI Fax: +1 412-268-6257

Optimizing Robotic Team Performance with Probabilistic Model … · 2015-04-28 · 2. Observe the...

Documents

Transcript of Optimizing Robotic Team Performance with Probabilistic Model … · 2015-04-28 · 2. Observe the...