1

AM3 Task 1.3

Navigation Using Spatio-Temporal Gaussian Processes

Songhwai Oh(Presented by Sam Burden)

MAST Annual ReviewUniversity of PennsylvaniaMarch 8-9, 2010

2

Overview

• Songhwai Oh– Professor, Seoul National University– In collaboration with Prof. Shankar Sastry (UC Berkeley)

• Goal: control and navigation in unstructured and uncertain environments– Model environment as a Gaussian Process (GP)– Efficiently incorporate new data– Use GP for control and navigation

• Expected Results at End of Fiscal Year: multiple agents learn GP model, navigate new environments

Navigation Using Spatio-Temporal Gaussian Processes

3

Technical Relevance

• Current techniques in GP regression– Offline

• Global gradient descent for parameter tuning• Incorporating new observations requires full computation

– Centralized• Estimation decoupled from control and navigation

• No natural way to integrate data from multiple agents

• Our approach to GPs– Real-time– Distributed– Integrate estimation with navigation

4

Relevance to MAST

• Our approach to GPs– Real-time– Distributed– Integrate estimation with navigation

Enables MAST platforms to navigate in unstructured and uncertain environments

5

• 1Q09 (A-09-1.3a): Spatio-temporal GPs: done• 2Q09 (A-09-1.3b): Integrate GP with MPC: ongoing• 3Q09 (A-09-1.3c): Assumed-Density GP

– Developed a distributed GP learning algorithm instead

• 4Q09 (A-09-1.3d): Assumed-Density GP with MPC– Ongoing; developed bio-inspired navigation strategies [2]

• 1Q10 (A-10-1.3a): Approximation– Developed rigorous approximation for truncated Gaussian Process

• 2Q10 (A-10-1.3b): Multi-Agent Integration– Share observations among agents, coordinate navigation

• 3Q10 (A-10-1.3c): Learning GP Kernel• 4Q10 (A-10-1.3d): Implement on MAST Platform

• 1Q09 (A-09-1.3a): Spatio-temporal GPs:• 2Q09 (A-09-1.3b): Integrate GP with MPC:• 3Q09 (A-09-1.3c): Assumed-Density GP

• 4Q09 (A-09-1.3d): Assumed-Density GP with MPC

• 1Q10 (A-10-1.3a): Approximation

• 2Q10 (A-10-1.3b): Multi-Agent Integration

Technical Accomplishments

6

Gaussian Process Estimation

• Unknown / uncertain environment modeled as a random process specified by mean and covariance

• Gaussian Process Regression provides straightforward but costly way to estimate process

• How can we reduce GP computation without degrading performance?

7

Truncated Gaussian Process

• Idea: recent measurements are more informative• Formalize, provide bounds to quantify tradeoff

between accuracy and computationAp

prox

imati

on E

rror

Truncation Size

8

Collaborations and Future Plans

• Prof. Herbert Tanner (Autonomy, UDelaware)– Model Predictive Navigation using GPs– Helps avoid local minima

• Possible new collaborative efforts / experiments– Use GPs to map occupancy and wireless signal strength

• Ideas going forward– Nonstationary GP kernels for mapping– Efficient estimation of GP parameters– Mixture-of-GPs to estimate multiscale phenomena

9

Collaborations and Future Plans

• Prof. Herbert Tanner (Autonomy, UDelaware)– Model Predictive Navigation using GPs– Helps avoid local minima

• Possible new collaborative efforts / experiments– Use GPs to map occupancy and wireless signal strength

• Ideas going forward– Nonstationary GP kernels for mapping– Efficient estimation of GP parameters– Mixture-of-GPs to estimate multiscale phenomena

10

Metrics

• [1] Jongeun Choi, Songhwai Oh, and Roberto Horowitz, Distributed Learning and Cooperative Control for Multi-Agent Systems. Automatica, vol. 45, no. 12, pp. 2802-2814, Dec. 2009.

• [2] Jongeun Choi, Joonho Lee, and Songhwai Oh, Navigation Strategies for Swarm Intelligence using Spatio-Temporal Gaussian Processes. Submitted to Neural Computing and Applications.

• [3] Yunfei Xu, Jongeun Choi and Songhwai Oh, Mobile Sensor Network Coordination Using Gaussian Processes with Truncated Observations. Submitted to IEEE Transactions on Robotics.

11

AM3 Task 1.4

Stochastic Hybrid Models for Aerial and Ground Vehicles

Sam BurdenMAST Annual ReviewUniversity of PennsylvaniaMarch 8-9, 2010

12

Overview

• Sam Burden– Ph.D. Student, UC Berkeley– Advised by Prof. Shankar Sastry

• Collaborators:– Aaron Hoover (Prof. Ronald Fearing, Micromechanics, UC Berkeley)– Prof. Robert Full (Micromechanics, UC Berkeley)

• Goal: develop general principles for hybrid system identification, apply to MAST platforms– High-fidelity 3D motion capture system– Theoretical principles for identification of stochastic hybrid systems– Apply model to aid control and improve design of the platforms

• Expected Results at End of Fiscal Year: model for terrestrial platform, controller to execute beacon following behavior

Stochastic Hybrid Models for Aerial and Ground Vehicles

13

• High-fidelity 3D Motion Capture

• System Identification

• State Estimation

– Camera calibration via bundle adjustment (Argyros, 2009)– Open source library developed with Python

– Linear systems (Ljung, 1987)– Continuum dynamics / PDEs (Tomlin, 2006)– Piecewise affine hybrid systems (Niessen, 2005)– Stochastic hybrid systems (Lygeros, 2008)

– Linear systems (Kalman Filter (KF); Rauch, Tung, Striebel, 1965)– Nonlinear Systems (Unscented KF; Julier and Uhlmann, 1995)

We are using nonlinear geometric theory to identify stochastic hybrid dynamics

Technical Relevance

14

Relevance to MAST

• Experimental tools (http://eecs.berkeley.edu/~sburden/python)– High-fidelity 3D motion capture– State estimation for stochastic nonlinear and hybrid systems

• Theoretical / Modeling tools– Unified theoretical framework for stochastic systems– System identification in this general framework

• Experimental outcomes– Develop empirically-validated model for terrestrial platform– Execute useful low-level behavior, e.g. beacon following

15

• 1Q10 (A-10-1.4a): Experimentation– Evaluated fidelity of VICON; inadequate for present application– Developed high-speed camera calibration software suite– Tools available at http://eecs.berkeley.edu/~sburden/python

• 2Q10 (A-10-1.4b): Hybrid System Identification– Creating geometric framework for ID problem– Applying framework to abstract mathematical models– Implementing estimation and identification

• 3Q10 (A-10-1.4c): Characterize System Noise• 4Q10 (A-10-1.4d): Control MAST Platform

• 1Q10 (A-10-1.4a): Experimentation– Evaluated fidelity of VICON; inadequate for present application– Software for camera calibration and nonlinear state estimation– Tools available at http://eecs.berkeley.edu/~sburden/python

• 2Q10 (A-10-1.4b): Hybrid System Identification– Creating geometric framework for ID problem– Applying framework to abstract models for locomotion– Implementation for empirical robot data

• 3Q10 (A-10-1.4c): Characterize System Noise• 4Q10 (A-10-1.4d): Control MAST Platform

Technical Accomplishments

16

HS: Intuitive Picture

Natural abstraction for running, flapping, & climbing robots

17

HS ID: Illustrative Example

Inelastic Bouncing Ball

• Velocity vector `jumps’ discontinuously• Kalman filter, particle filter will fail to estimate state

18

HS ID: Illustrative Example

Inelastic Bouncing Ball

• How can we identify the dynamics?• We can solve this problem in a geometric framework

– i.e. for running, flapping, climbing robots

19

Collaborations

• Weekly Meetings– Aaron Hoover (Prof. Ronald Fearing, Micromechanics, UC Berkeley)

• Design of terrestrial platform & control of platform• Empirical evaluation of dynamical abstractions

– Prof. Robert Full (Micromechanics, UC Berkeley)• Experiment design and resources (force platform, HS cameras)• HS ID for running cockroaches

• Monthly Meetings: Berkeley MAST Group– Autonomy, Micromechanics, Integration, Microelectronics

• Possible new collaborative efforts / experiments– Identification for other MAST platforms with hybrid dynamics– Implement controllers, improve design using identified dynamics

20

Future Plans

• 3Q—4Q 2010: Plans– Develop, validate open-loop model for terrestrial robot– Characterize dynamical uncertainty with respect to model– Design control scheme using stochastic hybrid model

• 2011—2013: Ideas, Goals– Improve estimation and identification for hybrid systems using

e.g. particle filters in abstract geometric spaces– Model the effect of varying terrain and morphology

esp. as a means to decrease dynamical uncertainty– Closed-loop model for robot dynamics by explicitly considering

dynamical effect of control effort

21

Future Plans

• 3Q—4Q 2010: Plans– Develop, validate open-loop model for terrestrial robot– Characterize dynamical uncertainty with respect to model– Design control scheme using stochastic hybrid model

• 2011—2013: Ideas, Goals– Improve estimation and identification for hybrid systems using

e.g. particle filters in abstract geometric spaces– Model the effect of varying terrain and morphology

esp. as a means to decrease dynamical uncertainty– Closed-loop model for robot dynamics by explicitly considering

dynamical effect of control effort

22

Discussion & Questions

• Metrics– Presenting Hybrid System ID work at HSCC in April, 2010

(HSCC: Hybrid Systems, Computation, and Control)

Thank you for your time

Collaborators– Aaron Hoover

(Prof. Ronald Fearing)– Prof. Robert Full

Support– ARL MAST

(Autonomy Center)– NSF GRF– Prof. Shankar Sastry

• Acknowledgements

23

Technical Slides

• Hybrid System Formal Framework• HS Identification Problem Statement• HS ID Intuitive Example• HS ID Recap

24

HS: Formal Framework

• Consider hybrid dynamical systems1 H := (Q, D, F, R):

1. Bernardo et. al. 2007

• Nice properties– Determinism– Existence & Uniqueness– Structural Stability

25

• Given output from the discrete-time stochastic model

estimate the state.

HS ID: Problem Statement

26

HS ID: Illustrative Example

Inelastic Bouncing Ball

• Velocity is discontinuous when ball bounces

• Kalman filter, Particle filter will give poor estimates after the bounce

• We can solve this problem • We can identify general hybrid dynamical systems

27

HS ID: Recap

• We consider a general class of hybrid systems

• We wish to estimate the state in the presence of uncertainty and noisy measurements

• We can solve this problem in general– i.e. for walking, flapping, climbing robots

ex:

28

Technical Slides

• Gaussian Processes• Spatio-Temporal• Conditional Distribution• Dynamics and Sensing• Navigation Strategies• Path Planning• Switched Path Planning• Truncated Gaussian Processes• Navigation using Truncated Observations

29

Gaussian Process

• A Gaussian process (GP) is a stochastic process. Any finite number of samples from a GP has a Gaussian distribution

• Regression using Gaussian processes:– Widely used in geostatistics (Kriging), statistics, machine learning

• GP can be used to estimate a field such as – mapping1

– radio signal strength (WiFi-SLAM2)– danger level or stealthiness– others: temperature, lighting, noise-level, etc.

1 [O'Callaghan, Ramos, Durrant-Whyte, 2009]2 [Ferris, Fox, Lawrence, 2007]

30

Recap: Spatio-Temporal Gaussian Processes

31

Recap: Conditional Distribution

32

Recap: Agent Dynamics and Sensing

33

Recap: Navigation Strategies

34

Path Planning

1/25/2010

Flocking

Consensus

NavigationGoal

35

Switching Path Planning

• Assumption: Stationary field, ¾max2 > k(s,s)

• Algorithm:1. Starts with exploration strategy2. If (max(prediction error) < ²)

Switch to exploitation (tracing) strategy

• Theorem [2]: Under some smoothness conditions, with probability at least 1-

¾max2 / ²2, |z(s,t)-z*(s,¿|t)| < ² at ¿ > t+1, for all s.

35

36

Switching Path Planning

36

Field to be estimated Trajectories of agents and estimated field

37

Switching Path Planning

37

RMS over simulation time (100 Monte Carlo runs)

Predicted variance, RMS, maximum errors of agents performing switching path planning.

38

Truncated Gaussian Processes

38

Motivation• Keeping all measurements up to time t requires large memory and

computation time. • Most recent measurements are more informative.

• Kernel function of spatio-temporal Kalman filter

Difference in prediction error variance as a function of the truncation size. (Blue: ¾t

2 = 5, red: ¾t

2 = 10.)

39

Truncated Gaussian Processes

39

Theorem [3]: