Modelovanie uchopovanie objektov pre humanoidného robota iCub
Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale...
Transcript of Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale...
![Page 1: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/1.jpg)
Lorenzo NataleiCub Facility
Istituto Italiano di Tecnologia, Genova
VVV13, 7th iCub Summer School
July 2013, Sestri Levante
![Page 2: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/2.jpg)
Motivations
![Page 3: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/3.jpg)
The iCub Facility: the ingredients
research/science
research in e.g. cognition, vision, mechatronics , learning, materials, speech…
engineering/community
iCub & friends
Program complex behaviors: integration
![Page 4: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/4.jpg)
The iCub Facility: the ingredients
research/science
research in e.g. cognition, vision, mechatronics , learning, materials, speech…
engineering/community
iCub & friends
Software integration: programming complex behaviors
![Page 5: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/5.jpg)
Complex behaviors
![Page 6: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/6.jpg)
Complex behaviors
![Page 7: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/7.jpg)
Complex behaviors
![Page 8: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/8.jpg)
iCub software architecture
ROBOT
ROBOT
ROBOT
ROBOT
…
/arm/encs
/arm/trqs
/hand/touch
/arm/cmd
/camera
![Page 9: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/9.jpg)
ReachingReachingReachingReaching
...
...
... ...
iCub software architecture
ROBOT
ROBOT
ROBOT
ROBOT
…
Hand DetectionHand DetectionHand DetectionHand Detection
GraspingGraspingGraspingGrasping
Object rec.Object rec.Object rec.Object rec.
/arm/encs
/arm/trqs
/hand/touch
/arm/cmd
/camera
...
...
...
...
...
...
![Page 10: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/10.jpg)
ReachingReachingReachingReaching
...
...
... ...
iCub software architecture
ROBOT
ROBOT
ROBOT
ROBOT
…
Hand DetectionHand DetectionHand DetectionHand Detection
GraspingGraspingGraspingGrasping
Object rec.Object rec.Object rec.Object rec.
/arm/encs
/arm/trqs
/hand/touch
/arm/cmd
/camera
...
...
...
...
...
...
![Page 11: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/11.jpg)
ReachingReachingReachingReaching
...
...
... ...
iCub software architecture
ROBOT
ROBOT
ROBOT
ROBOT
…
Hand DetectionHand DetectionHand DetectionHand Detection
GraspingGraspingGraspingGrasping
Object rec.Object rec.Object rec.Object rec.
/arm/encs
/arm/trqs
/hand/touch
/arm/cmd
/camera
...
...
...
...
...
...
![Page 12: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/12.jpg)
ReachingReachingReachingReaching
...
...
... ...
iCub software architecture
ROBOT
ROBOT
ROBOT
ROBOT
…
Hand DetectionHand DetectionHand DetectionHand Detection
GraspingGraspingGraspingGrasping
Object rec.Object rec.Object rec.Object rec.
/arm/encs
/arm/trqs
/hand/touch
/arm/cmd
/camera
...
...
...
...
...
...
![Page 13: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/13.jpg)
ReachingReachingReachingReaching
...
...
... ...
iCub software architecture
ROBOT
ROBOT
ROBOT
ROBOT
…
Hand DetectionHand DetectionHand DetectionHand Detection
GraspingGraspingGraspingGrasping
Object rec.Object rec.Object rec.Object rec.
/arm/encs
/arm/trqs
/hand/touch
/arm/cmd
/camera
...
...
...
...
...
...
![Page 14: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/14.jpg)
YARP• Peer-to-peer, loosely coupled, communication
• Very stable code base >10 years old
• Flexibility and minimal dependencies, fits well with other systems
• Easy install with binaries on many OSes/distributions (Ubuntu, Debian, Windows, MacOs), see: http://wiki.icub.org/wiki/YARP_Supported_Distributions
• Several protocols:
– Built-in: tcp/udp/mcast
– Plug-ins: ROS tcp, xml rpc, mjpg etc..
![Page 15: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/15.jpg)
Interfaces
• Define interfaces for motor control, device drivers so to minimize the impact of changes in the hardware
• Also: network stubs allow remotization
![Page 16: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/16.jpg)
Interfaces
IPosition
Control
FrameGrabbers
IIVelocity
Control
…getImage(); …lookAt();reach();grasp();…
.
User Code:
ICartesian
Control
![Page 17: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/17.jpg)
Interfaces
IPosition
Control
FrameGrabbers
IIVelocity
Control
…getImage(); …lookAt();reach();grasp();…
.
User Code:
ICartesian
Control
Network Stub
Network Stub
Network Stub
network
![Page 18: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/18.jpg)
YARP plugins
• YARP includes a plugin system for drivers and protocols (carriers)
• Interchangeable carriers allow:
– interfacing existing software with ports (without bridges)
– change significantly port behavior
• Examples:
– ROS, mjpeg, xml rpc, etc…
– bayer carrier, priority based communication
![Page 19: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/19.jpg)
Tools: working offline
• Data collector/player
• Simulator
• GUIs
![Page 20: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/20.jpg)
Repositories and online resources
• YARP online documentation: www.yarp.it• iCub: http://wiki.icub.org• Github: https://github.com/robotology
– yarp– codyco
• Sourceforge:– Robotcub (iCub: main and contrib, poeticon++, emorph, darwin)
– efaa– xperience
![Page 21: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/21.jpg)
Binaries
wiki.icub.org/iCub� Software Installation� Tutorials
Libraries and modules for motor control, machine learning, vision…
![Page 22: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/22.jpg)
Online resources, documentation,
manual and dashboard
http://icub.org � iCub software http://icub.org � iCub Manual
http://dashboard.icub.org
![Page 23: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/23.jpg)
Modularity: where to draw the line?
• Libraries
• Components (processes)
• Functionalities (group of components)
![Page 24: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/24.jpg)
Managing complexity
In a modular system integration becomes an issue:• Execution and monitoring• Development• Coordination
![Page 25: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/25.jpg)
Execution and monitoring:
yarp manager
Available resources
Requiredmodules connectionsnodesresources
![Page 26: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/26.jpg)
Tools for rapid development
• YARP builder: graphical tool to design application
• Interface Definition Language (IDL):
– formalization of types and interfaces between modules
– automatic generation of message handlers
![Page 27: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/27.jpg)
Coordinating modules
![Page 28: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/28.jpg)
Take & return
Look for object
Reach for object
Look for face
Grasp Object
Give object
Release object
![Page 29: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/29.jpg)
Take & return
Look for object
Reach for object
Look for face
Grasp Object
Give object
Release object
reaching fails
![Page 30: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/30.jpg)
Take & return
Look for object
Reach for object
Look for face
Grasp Object
Give object
Release object
reaching fails
grasp fails
![Page 31: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/31.jpg)
Take & return
Look for object
Reach for object
Look for face
Grasp Object
Give object
Release object
reaching fails
grasp fails
object is dropped
![Page 32: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/32.jpg)
Take & return
Look for object
Reach for object
Look for face
Grasp Object
Give object
Release object
reaching fails
grasp fails
object is dropped
somebody gives the object to the robot
![Page 33: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/33.jpg)
Take & return
Look for object
Reach for object
Look for face
Grasp Object
Give object
Release object
reaching fails
grasp fails
object is dropped
somebody gives the object to the robot
boring, error prone, task specific…
![Page 34: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/34.jpg)
A different approach
• no central coordinator
• all modules can be re-used
• no need for extra connections
![Page 35: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/35.jpg)
See Ali Paikan’s presentation during the school
![Page 36: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/36.jpg)
See Ali Paikan’s presentation during the school
![Page 37: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/37.jpg)
People
Marco Accame
Valentina Gaggero
Alberto Cardellino
Alessandro Scalzo
Daniele Domenichelli
Paul Fitzpatrick
Ali Paikan
Elena Ceseracciu
Vadim Tikhanoff
Ugo Pattacini
Marco Randazzo
… and many others
at IIT and in the iCub community
![Page 38: Lorenzo Natale - iCubwiki.icub.org/images/d/d4/Vvv13-Lorenzo-introduction.pdf · Lorenzo Natale iCub Facility Istituto Italiano di Tecnologia, Genova VVV13, 7 th iCub Summer School](https://reader033.fdocuments.net/reader033/viewer/2022042323/5f0d8df27e708231d43aef64/html5/thumbnails/38.jpg)
Thank you!