Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for...
-
Upload
nguyendien -
Category
Documents
-
view
225 -
download
4
Transcript of Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for...
![Page 1: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/1.jpg)
Using ROS2 for Vision-Based Manipulation with Industrial Robots
Adam [email protected]
Karsten [email protected]
![Page 2: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/2.jpg)
![Page 3: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/3.jpg)
![Page 4: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/4.jpg)
DDSAuto-Discovery
ROS1 Bridge
ament_resource_index
Lifecycle Nodes
Cross Platform
Namespacing
Cross Language
Intra ProcessCommunication
Parameters
Multi-NodeProcesses
Composition
Security
QoS
![Page 5: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/5.jpg)
DDSAuto-Discovery
ROS1 Bridge
ament_resource_index
Lifecycle Nodes
Cross Platform
Namespacing
Cross Language
Intra ProcessCommunication
Parameters
Multi-NodeProcesses
Composition
Security
QoS
![Page 7: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/7.jpg)
The big picture
Joint State Controller
// publish joint statesupdate()
Joint Trajectory Controller
// listen to traj msg// set joint statesupdate()
Robot HAL
while(true){ read_joints() controller_manager.update()
write_joints()}
More details from Jackie Kay and Adolfo Rodriguez Tsouroukdissian at ROScon 2015https://roscon.ros.org/2015/presentations/RealtimeROS2.pdf
![Page 8: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/8.jpg)
Controller ManagerControllerManager::load_controller(package_name, class_name, controller_name){ auto library_path = get_ament_resource(package_name, class_name);
auto loader = std::make_shared<class_loader::ClassLoader>(library_path);
auto controller =
loader->createInstance<controller_interface::ControllerInterface>(class_name);
controller->init(controller_name);
executor_->add_node(controller->get_node_base_interface());}
int main(int argc, char ** argv){ cm.load_controller( “ros_controllers”, “ros_controllers::JointTrajectoryController”, “trajectory_controller/left”);
cm.load_controller( “ros_controllers”, “ros_controllers::JointTrajectoryController”, “trajectory_controller/right”);
}
HAL
Controller
https://github.com/ros2/ros2/wiki/Composition
![Page 9: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/9.jpg)
Ament Resource Index<ros2_ws/install>
`-- share
| `-- ... # Other, non-plugin related, stuff
|-- ament_index
|-- resource_index
|-- packages
| `-- controller_manager
| `-- ...
|-- controllers
| `-- ros_controllers # containing controller info
$ cat /ros2_ws/install/share/ament_index/resource_index/controllers/ros_controllersros_controllers::JointStateController;bin/default_controllers.dllros_controllers::JointTrajectoryController;bin/libdefault_controllers.dll
https://github.com/ament/ament_cmake/blob/master/ament_cmake_core/doc/resource_index.md
HAL
Controller
![Page 10: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/10.jpg)
Lifecycle Controller Nodes
http://design.ros2.org/articles/node_lifecycle.html
class ControllerInterface : public LifecycleNodeInterface{ virtual rcl_lifecycle_transition_key_t on_configure(const State & previous_state); virtual rcl_lifecycle_transition_key_t on_activate(const State & previous_state); virtual rcl_lifecycle_transition_key_t on_deactivate(const State & previous_state); virtual rcl_lifecycle_transition_key_t on_cleanup(const State & previous_state);
virtual rcl_lifecycle_transition_key_t on_shutdown(const State & previous_state); virtual rcl_lifecycle_transition_key_t on_error(const State & previous_state);};
HAL
Controller
![Page 11: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/11.jpg)
Lifecycle Controller Nodes
https://github.com/ros2/ros2/wiki/Managed-Nodes
class MyController : public ControllerInterface{ // do lifecycle controller stuff on_activate()...};
$ ros2 service list/my_controller/change_state/my_controller/get_available_states/my_controller/get_available_transitions/my_controller/get_state
$ ros2 service call /my_controller/get_state lifecycle_msgs/GetStatelifecycle_msgs.srv.GetState_Response(current_state=lifecycle_msgs.msg.State(id=1, label='unconfigured'))
HAL
Controller
![Page 13: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/13.jpg)
The Picky Robot: Motivation● Show that ROS2's core features are in place● Do some integration but also add some new functionality
![Page 14: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/14.jpg)
Let's build a Pickand Place demo!
![Page 15: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/15.jpg)
Object Detector (LINEMOD)
Astra Depth Camera
Astra Camera Driver
Architecture
![Page 16: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/16.jpg)
Shared Memory for Intraprocess Communication
https://github.com/ros2/ros2/wiki/Intra-Process-Communication
![Page 17: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/17.jpg)
IntraprocessCommunication
Object Detector (LINEMOD)
Astra Depth Camera
Astra Camera Driver
Architecture
![Page 18: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/18.jpg)
IntraprocessCommunication
UR5 RobotObject
Detector (LINEMOD)
Astra Depth Camera
Astra Camera Driver
Robot Interface
Architecture
6DOF Pose
![Page 19: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/19.jpg)
ROS 2 launch file (Python)
IntraprocessCommunication
UR5 RobotObject
Detector (LINEMOD)
Astra Depth Camera
Astra Camera Driver
Robot Interface
Architecture
![Page 20: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/20.jpg)
ROS 2 launch file (Python)
IntraprocessCommunication
UR5 RobotObject
Detector (LINEMOD)
ROS2 bridgeRViz
Astra Depth Camera
Astra Camera Driver
Robot Interface
Architecture
![Page 21: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/21.jpg)
ROS 2 launch file (Python)
IntraprocessCommunication
UR5 RobotObject
Detector (LINEMOD)
ROS2 bridgeRViz
Astra Depth Camera
Astra Camera Driver
Robot Interface
Architecture
Docker Container
![Page 22: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/22.jpg)
http://www.trossenrobotics.com/Shared/images/PImages/widegripper/rg-wide-gripper-a.jpg
![Page 23: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/23.jpg)
Xhttp://www.trossenrobotics.com/Shared/images/PImages/widegripper/rg-wide-gripper-a.jpg
![Page 25: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/25.jpg)
DDSAuto-Discovery
ROS1 Bridge
ament_resource_index
Lifecycle Nodes
Cross Platform
Namespacing
Cross Language
Intra ProcessCommunication
Parameters
Multi-NodeProcesses
Composition
Security
QoS
![Page 26: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/26.jpg)
DDSAuto-Discovery
ROS1 Bridge
ament_resource_index
Lifecycle Nodes
Cross Platform
Namespacing
Cross Language
Intra ProcessCommunication
Parameters
Multi-NodeProcesses
Composition
Security
QoS
![Page 27: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/27.jpg)
Special Thanks to ...
Calvin Ngan - Karsten's Bosch summer intern
Mikael Arguedas - Adam's Open Robotics summer mentor
![Page 28: Using ROS2 for Vision-Based Manipulation with Industrial ... 2017 ROS2 Vision... · Using ROS2 for Vision-Based Manipulation with Industrial Robots Adam Allevato ... More details](https://reader034.fdocuments.net/reader034/viewer/2022051602/5aefb9bf7f8b9ad0618d2f3a/html5/thumbnails/28.jpg)
Thank you!
Adam [email protected]
Karsten [email protected]
Code available:● https://github.com/kukanani/picky_robot● https://github.com/ros-controls/ros2_control● https://github.com/ros-controls/ros2_controllers