Scaling Human Robot Teams
description
Transcript of Scaling Human Robot Teams
Scaling Human Robot Teams
Prasanna VelagapudiPaul Scerri
Katia SycaraMike Lewis
Robotics InstituteCarnegie Mellon University
Pittsburgh, PA
Large Multiagent Teams
• 1000s of robots, agents, and people
• Must collaborate to complete complex tasks
Large Multiagent Teams
Large Multiagent Teams
• Network Constraints
Large Multiagent Teams
• Human Information Needs
Network Constraints
• Networks affect human interface design– Limited bandwidth– Significant latency– Lossy transmission– Partial/transient connectivity
Network Constraints
• How can we design robust tasks?– Feasible under network constraints– Tolerant of latency– Within bandwidth constraints– Robust to changes in information
Network Constraints
• Humans are a limited resource
Network Constraints
• Humans are a limited resource– Centralized, expensive– Limited attention and workload– Penalties for context switching– Necessary for certain tasks
• Complex visual perception• Meta-knowledge
Network Constraints
• How do we maximize the effectiveness of humans in these systems with respect to network constraints?
MrCSMulti-robot Control System
MrCSMulti-robot Control System
Waypoint Waypoint NavigationNavigation
TeleoperationTeleoperation
Video/ Video/ Image Image ViewerViewer
Status Status WindowWindow
Map Map OverviewOverview
Victims Found in USAR Task
Number of
Victims
[Velagapudi et al, IROS ’08]
Task decomposition[Velagapudi et al, IROS ’08]
Network Constraints
• How we divide tasks between agents may affect performance– What is the best way to factor tasks?– Where should we focus autonomy?
Large Multiagent Teams
• Human Information Needs
Human Information Needs
• Human operators need information to make good decisions
• In small teams, send everyone everything
• This doesn’t work in large systems
Human Information Needs
• Sensor raw datarates – Proprioception
• < 1kbps
– RADAR/LIDAR• 100kbps – 20Mbps
– Video• 300kbps – 80Mbps
Human Information Needs
• Can’t transmit every bit of information– Selectively forward data
• How do agents decide which pieces of information are important?
– Fuse the data• What information are we losing when we fuse
data?
Asynchronous Imagery
• Inspired by planetary robotic solutions– Limited bandwidth– High latency
• Multiple photographs from single location– Maximizes coverage– Can be mapped to virtual pan-tilt-zoom camera
Asynchronous Imagery
• Streaming Mode • Panorama Mode
Panoramas stored for later viewingPanoramas stored for later viewingStreaming live videoStreaming live video
[Velagapudi et al, ACHI ’08]
Victims Found[Velagapudi et al, ACHI ’08]
Average Average # of # of
victims victims foundfound
Accuracy ThresholdAccuracy Threshold
11
22
33
44
55
66
Within Within 0.75m0.75m
Within 1mWithin 1m Within 1.5mWithin 1.5m Within 2mWithin 2m00
PanoramaStreaming
Environmental Factors
• Colocated operators get extra information– Exocentric view of other agents– Ecological cues– Positional and scale cues
Conclusion
• Need to consider the practicalities of large network systems when designing for humans.
• Need to consider human needs when designing algorithms for large network systems.
Our Work
Cognitive modeling
• ACT-R models of user data
• Determine– What pieces of information users are using?– Where are the bottlenecks of the system?
Environmental Factors
• Colocated operators get extra information– Exocentric view of other agents– Ecological cues– Positional and scale cues
Utility-based information sharing
• It is hard to describe user information needs
• Agents often don’t know how useful information will be
• Many effective algorithms use information gain or probabilistic mass
• Can we compute utility for information used by people
MrCSMulti-robot Control System
MrCSMulti-robot Control System
Waypoint Waypoint NavigationNavigation
TeleoperationTeleoperation
Video/ Video/ Image Image ViewerViewer
Status Status WindowWindow
Map Map OverviewOverview
Victims Found
Number of
Victims
Task decomposition
NavigationNavigation
SearchSearch
Task decomposition
Asynchronous Data
• One way to address the latency of networks is to transition to asynchronous methods of perception and control.
• Asynchronous imagery– Decouples users from time constraints in
control
Asynchronous Imagery
• Inspired by planetary robotic solutions– Limited bandwidth– High latency
• Multiple photographs from single location– Maximizes coverage– Can be mapped to virtual pan-tilt-zoom camera
Asynchronous Imagery
• Streaming Mode • Panorama Mode
Panoramas stored for later viewingPanoramas stored for later viewingStreaming live videoStreaming live video
Victims Found
Average Average # of # of
victims victims foundfound
Accuracy ThresholdAccuracy Threshold
11
22
33
44
55
66
Within Within 0.75m0.75m
Within 1mWithin 1m Within 1.5mWithin 1.5m Within 2mWithin 2m00
PanoramaStreaming
Tools
• USARSim/MrCS
• VBS2
• Procerus UAVs
• LANdroids
• ACT-R
USARSim
[http://www.sourceforge.net/projects/usarsim]
• Based on UnrealEngine2
• High-fidelity physics• Realistic rendering
– Camera– Laser scanner
(LIDAR)
MrCSMulti-robot Control System
MrCSMulti-robot Control System
Waypoint Waypoint NavigationNavigation
TeleoperationTeleoperation
Video/ Video/ Image Image ViewerViewer
Status Status WindowWindow
Map Map OverviewOverview
VBS2
[http://www.vbs2.com]
• Based on Armed Assault and Operation Flashpoint
• Large scale agent simulation
• “Realistic” rendering– Cameras– Unit movements
Procerus UAVs
• Unicorn UAV• Developed at BYU• Foam EPP flying wing• Fixed and gimbaled
cameras• Integrated with
Machinetta agent middleware for full autonomy
LANdroids Prototype
• Based on iRobot Create platform
• Integrated 5GHz 802.11a based MANET
• Designed for warfighter networking
• Video capable
ACT-R
• Cognitive modeling framework
• Able to create generative models for testing