High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon...
-
Upload
roland-rich -
Category
Documents
-
view
223 -
download
2
Transcript of High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon...
High-level robot behavior control using POMDPsJoelle Pineau and Sebastian Thrun
Carnegie Mellon University
CogRob2002 workshop Joelle Pineau
Introducing Pearl
• Pearl is a prototype nursing robot, providing assistance to both nurses and elderly people.
“thinkers”
eyes with cameras
sonar sensors
handlebars
wheeled base
carrying tray
LCD smile/frown
CogRob2002 workshop Joelle Pineau
Profile of our aging population
450,000 more nurses needed by 2008 campaign to recruit and retain nurses
and other health care providers
CogRob2002 workshop Joelle Pineau
Moving thingsaround
Moving thingsaround
Enabling use of remote
health services
Enabling use of remote
health services
Supportinginter-personal
communication
Supportinginter-personal
communication
Calling for helpin emergencies
Calling for helpin emergencies
Monitoring Rx adherence
& safety
Monitoring Rx adherence
& safety
Providinginformation
(TV, weather)
Providinginformation
(TV, weather)
Management support of
ADLs
Management support of
ADLsReminding
to eat, drink, & take meds
Reminding to eat, drink, & take meds
Providing physical
assistance
Providing physical
assistance
Linking the caregiver to resources
Linking the caregiver to resources
Our vision of robotic-assisted health-care
CogRob2002 workshop Joelle Pineau
The Nursebot project in its early days
CogRob2002 workshop Joelle Pineau
We need a high-level controller that can:
• prioritize goals from specialized modules
• trade-off goals with widely different costs/rewards
• trade-off between information-gathering and goal- satisfaction
• switch between tasks and share sensory information
• handle uncertainty
Speech recognition&synthesis(Ravishankar, 1996; Black et al., 1999)
Autonomous navigation(Burgard et al., 1999)
Autominder(Pollack et al., 2002)
People tracking/following(Montemerlo et al., 2002)
Top controller
CogRob2002 workshop Joelle Pineau
High-level robot behavior control usingPartially Observable Markov Decision Processes
USER + WORLD + ROBOT
ACTIONS
OBSERVATIONSBELIEF STATE
STATE
CogRob2002 workshop Joelle Pineau
What are POMDPs?
POMDP is n-tuple { S, A, , b, T, O, R }
POMDP task 1: State tracking:
After an action, what is the state of the world?
POMDP task 2: Computing a policy:
Which action should the controller apply next?
Very hard!
Not so hard.
Robot belief:
World state:
Control layer:
bt-1 ??
at-1
ot
st-1 st
...
...
??ot-1
...
...
rt-1 rt
CogRob2002 workshop Joelle Pineau
Our approach: Hierarchical POMDPs
• Key Idea: Exploit hierarchical structure in the problem domain to break a problem into many “related” POMDPs.
Act
InvestigateHealth
Move
NavigateCheckPulse
AskWhere
North South East West
CheckMeds
subtask
abstract action
primitive action
CogRob2002 workshop Joelle Pineau
Planning with Hierarchical POMDPs
• Given POMDP model M = { S, A, , b, T, O, R } and hierarchy H
• For each subtask h H:1) Set components
Ah children nodes
Sh S
h bh, Th, Oh, Rh
2) Minimize modelSh {zh(s0), …, zh(sn)}
h {yh(o0), …, yh(op)}
3) Solve subtask hh {bh, Th, Oh, Rh}
Move
Navigate AskWhere
South East WestNorth
ANav ={N,S,E,W}SNav ={X,Y}Nav ={o0,…,om}
AMove ={AskWhere,Navigate}SMove ={X’,Y’,Destination}Move ={o0,…,op}
CogRob2002 workshop Joelle Pineau
Execution with Hierarchical POMDPs
• Step 1 - Update belief:
• Step 2 - Traversing hierarchy top-down, for each subtask:1) Get local belief.
2) Consult local policy.
3) If a is leaf node, terminate.
Else, go to that subtask.
Act
InvestigateHealth Move
NavigateCheckPulse
AskWhere
North South East West
CheckMeds
baoP
sbassPasoP
sbSs
jjii
ij
t
t
,|
,|,| 1
CogRob2002 workshop Joelle Pineau
Experimental Setup
• Task: Robot provides reminders and guidance to elderly user.
• Action hierarchy:
CogRob2002 workshop Joelle Pineau
Sample Scenario
CogRob2002 workshop Joelle Pineau
Still unconvinced about the importance of uncertainty?
CogRob2002 workshop Joelle Pineau
On the question of numericanumerical vs logicallogical representations...
• We haven’t tried using logical representations to control the robot…
• But our experience tells us that:
Uncertainty is crucial when dealing with people.
Probabilistic techniques are necessary to reason about uncertainty.
Real belief tracking and planning really matters!