Post on 11-Aug-2020
Guest Lecture
METR 4202: Robotics & Automation
Dr Surya Singh -- Lecture # 12 (Week 13) October 21, 2019
metr4202-staff@itee.uq.edu.au http://robotics.itee.uq.edu.au/~metr4202/ [http://metr4202.com]
© 2019 School of Information Technology and Electrical Engineering at the University of Queensland
Probabilistic Robotics: Control (LQR, Value Functions, Q-Learning, Deep Reinforcement Learning, etc.)
& The Future of Robotics/Automation (Open Challenges)
METR 4202: Robotics & Automation
Dr Surya Singh -- Lecture # 12 (Week 13) October 21, 2019
metr4202-staff@itee.uq.edu.au http://robotics.itee.uq.edu.au/~metr4202/ [http://metr4202.com]
© 2019 School of Information Technology and Electrical Engineering at the University of Queensland
Reference Material
UQ Library / Online (PDF)
UQ Library(TJ211.4 .L38 1991)
October 21, 2019 -METR 4202: Robotics 3
Probabilistic RoboticsWhat about variation?
October 21, 2019 -METR 4202: Robotics 4
Inherent Sources of UncertaintyThere are some common sources of uncertainty:• Inherent stochasticity in the system being modeled
• Incomplete modeling
• Incomplete observability
• Incompetent control (or action)
October 21, 2019 -METR 4202: Robotics 5
Two Views of Probability
Frequentist probability• Relates directly to the rates at
which events occur• Basically, things that are
directly “repeatable”• e.g. Drawing a hand of cards
Belief (Bayesian) probability• Degree of a Belief• Qualitative levels of uncertainty• For more details about why a
small set of common sense• Ramsey (1926) –
The same set of axioms control both kinds of probability
October 21, 2019 -METR 4202: Robotics 6
II.1 Basic Probability Theory
• Probability Theory– Given a data generating process, what are the properties of
the outcome?
• Statistical Inference– Given the outcome, what can we say about the process that
generated the data?– How can we generalize these observations and make
predictions about future outcomes?
October 21, 2019 -METR 4202: Robotics II.7
In Particular: Bayes’ Rule
• What does Bayes’ Formula helps to find?– Helps us to find:
– By having already known:
( )ABP |
( )|P A B
October 21, 2019 -METR 4202: Robotics 8
Example: Inference / Generative Models
October 21, 2019 -METR 4202: Robotics 9
Robot ControlsControls Example I: Cart & Pole
(& Obstacles)
October 21, 2019 -METR 4202: Robotics 10
Classic State-Space Problem: Cart & Pole• Equations of State:
(" +$) '̈ + $()̈cos) − $()̇/sin) = 3$('̈cos) + $(/)̈ − $4(sin) = 0
• Solution:
'̈ = 63 + $sin)(()̇/ − 4cos)" +$sin/)
)̈ = −3cos) − $()̇/sin)cos) + (" +$)4sin))((" +$sin/)
7̇ ='̇'̈)̇)̈=
'̇63 + $sin)(()̇/ − 4cos)
" +$sin/))̇
−3cos) − $()̇/sin)cos) + (" +$)4sin))((" +$sin/)
=
'̇3 − $)4
")̇
−3 + (" +$)4)("
=
0 1 0 00 0 −$4" 00 0 0 10 0 (" +$)4
(" 0
''̇))̇+
01"0
− 1("
3
See Also: Tutorial 12: Cart-Pole Inverted Pendulum (State-Space)
October 21, 2019 -METR 4202: Robotics 11
Motion Planning & Control: a Unified Design Tool?
Atkeson, C. & Stephens, B.“Random sampling of states in dynamic programming” IEEE T. Syst. Man Cybern, 2008
Maeda, Singh, Durrant-Whyte, ICRA 2011
GPS tree
Belief tree
Seiler, et al. ICRA, May 2015, 2290-2297
⇡⇤(b) = argmax
a2A
Z
s2SR(s, a)b(s)ds+ E
" 1X
t=1
�tR(st, at)|⌧(b, a,O), ⇡⇤
#!POMDP Framework:
12October 21, 2019 -METR 4202: Robotics 12
Why? Bellman’s Principle of Optimality• Suppose we are given a DT ODE
Δ" = $ ", &" ∈ ℝ), & ∈ ℝ*
• Optimal Control:Find &: 0, - → ℝ* that minimizes the cost (or said more optimistically: maximizes the reward):
/ ", & = 0 -, " - + 2345
678ℒ :, " : , & :
• Bellman’s insight (1957):* the optimal control at time s depends only on ; <
(i.e. not any prior states!)∴ It’s an MDP!
Final cost
“Running” cost
October 21, 2019 -METR 4202: Robotics 13
Motion Planning & Control: Trajectory Generation with Constraints
Steering/Feedback ControlMethods:
Motion Planning Methods:
14October 21, 2019 -METR 4202: Robotics 14
Feedback controller
Motion planner
Parameterized control policy(PID, LQR, …)
One Approach: Gain-Scheduled RRTCore Idea: • Basic approach: decoupling à tractable• Integrated approach: use feedback to shortcut the planning phase
xgoal
xinit* Maeda, G.J; Singh, S.P.N; Durrant-Whyte, H.
“Feedback Motion Planning Approach for Nonlinear Control using Gain Scheduled RRTs”, IROS 2010
15October 21, 2019 -METR 4202: Robotics 15
Gain-Scheduled RRT: Algorithm
Initialize tree(s)
Design feedback at the goal
Estimate the RoA
Extend tree
Try to reach RoA
Finish
yes
no
16October 21, 2019 -METR 4202: Robotics 16
Gain-Scheduled RRT
Holonomic case
q(m)
q(m
/s)
Under differential constraints
xinit s(m)
r(m)
xgoalxrand
Core Idea: • Basic approach: decoupling à tractable• Integrated approach: use feedback to shortcut the planning phase
* Maeda, G.J; Singh, S.P.N; Durrant-Whyte, H. “Feedback Motion Planning Approach for Nonlinear Control using Gain Scheduled RRTs”, IROS 2010
17October 21, 2019 -METR 4202: Robotics 17
A RRT solution rarely reaches the goal (or connect the two trees) with zero error
How large?
Gain-Scheduled RRT: RRT Connection Gap
Connection gap
18October 21, 2019 -METR 4202: Robotics 18
Region searchRRT connection is relaxed
Gain-Scheduled RRT: Search
Backwards tree
Forward tree
goal
Single state search:Extensive exploration
Feedback system
19October 21, 2019 -METR 4202: Robotics 19
Sum of squares relaxation
GS-RRT: RoA & Verification• Find a candidate
• Maximize candidate ( ρ )
• Verify candidate **R. Tedrake, “LQR-Trees: Feedback Motion Planning on Sparse Randomized Trees”, RSS 2009
V(x) =In the LQR case: J: optimal cost-to-go S: Algebraic Ricatti Eq.
goal
October 21, 2019 -METR 4202: Robotics 21
Gain-Scheduled RRT:
22October 21, 2019 -METR 4202: Robotics 22
Automatic Robot Controls
Controls Example II: Underactuated Robotics
October 21, 2019 -METR 4202: Robotics 23
The Jitterbug Problem
October 21, 2019 -METR 4202: Robotics 24
Solving the Jitterbug Problem: Continuous Action Deep Reinforcement Learning
October 21, 2019 -METR 4202: Robotics 25
Solving the Jitterbug Problem:
October 21, 2019 -METR 4202: Robotics 26
Future of RoboticsSome Research Examples J
October 21, 2019 -METR 4202: Robotics 27
Computer Aided Surgery: R/C Toolholders?
èMove in tandem with heart: Cardiac procedures without stopping it• Unstructured environment (patient) makes this harder
October 21, 2019 -METR 4202: Robotics 28
• Biomechanics approach: Predict expected tissue trajectories• (Stochastic) Robot Motion Planning / Control Methods!
Modern (Tele)Surgical Robotics:
ARC DP160100714
October 21, 2019 -METR 4202: Robotics 29
Computer Aided Surgery: “Soft” is “Hard”!
October 21, 2019 -METR 4202: Robotics 30
Research: Incorporating Stiffness (Haptics): Visual Deformable Object Analysis
Dansereau, Singh, Leitner, ICRA 2016
October 21, 2019 -METR 4202: Robotics 31
Iceberg to Titanic: Take Advantage of Information
• 30 Min/Day Talking on Phone – 5.5 days/year of audio samples
– Track this (notably the pauses) over time to detect onset of dimentia
• 150 Photos/Month– Time history for detecting
precursors – Skin cancer monitoring
October 21, 2019 -METR 4202: Robotics 32
How?
• More Signals • Stochastic Processing(Think TAPIR!)
October 21, 2019 -METR 4202: Robotics 33
Robotics & Health: A Friendly Touch!
October 21, 2019 -METR 4202: Robotics 34
∴ Planning to Inform “Novel Robot” Design Strategy
Morley-Drabble & Singh, AIM 2018 Ham, Singh & Lucey, WACV 2017
October 21, 2019 -METR 4202: Robotics 35
THE Future of Robotics…
October 21, 2019 -METR 4202: Robotics 36
You! J
October 21, 2019 -METR 4202: Robotics 37
It’s all up to you!
If you want to build a ship, don't drum up the men to gather wood, divide the work and give orders. Instead, teach them to yearn for the vast and endless sea.
Antoine de Saint-Exupery, "The Wisdom of the Sands"
© National Geographic. Suruga Bay, Japan“The Magic Starts Here: Kenji’s Workshop of Camera Wizardry”, December 4, 2014
October 21, 2019 -METR 4202: Robotics 38
UQ Robotics: Dynamic Systems in Motion
Decision-Making/Planning
Mechanicsof motion
Aerial Systems
Systems
Pauline Pounds (ANU/Yale)Surya Singh (Stanford/Syd)
Diverse international research group
October 21, 2019 -METR 4202: Robotics 39
October 21, 2019 -METR 4202: Robotics 40