3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles
-
Upload
justas-miseikis -
Category
Technology
-
view
12 -
download
1
Transcript of 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles
![Page 1: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/1.jpg)
3D Vision Guided Robotic Charging Station
for Electric and Plug-in Hybrid Vehicles
J. Mišeikis, M. Rüther, B. Walzel, M. Hirz, H. Brunner
![Page 2: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/2.jpg)
Electric Vehicles are becoming great!
![Page 3: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/3.jpg)
… but EV owners still have plenty to
complain about!
Electric Vehicles are becoming great!
![Page 4: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/4.jpg)
… but EV owners still have plenty to
complain about!
Electric Vehicles are becoming great!
Charging
![Page 5: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/5.jpg)
Image source: http://www.electriccarpledge.com/electric-vehicle-resources/electric-car-plug-types/
![Page 6: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/6.jpg)
Image source: http://www.quick220.com/page/cp-product-solutions-2.html
![Page 7: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/7.jpg)
Image source: https://twitter.com/dunneautomotive/status/780996480417034241
![Page 8: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/8.jpg)
Image source: https://twitter.com/dunneautomotive/status/780996480417034241
![Page 9: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/9.jpg)
Image source: https://twitter.com/dunneautomotive/status/780996480417034241
But the charging cable becomes very thick and heavy!
![Page 10: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/10.jpg)
Let’s allow the robot to
do the dirty work!
![Page 11: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/11.jpg)
Use Case
The car is parked in the designated parking spot
and the robot handles the whole charging
process
![Page 12: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/12.jpg)
Related Work
![Page 13: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/13.jpg)
Our Concept: Robotic EV / PHEV Charging Station
![Page 14: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/14.jpg)
Our System Setup
![Page 15: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/15.jpg)
Algorithm
Detection of the
charging port
3D Pose Estimation
Robot Motion
Planning
3-Step Plug-In
Procedure
Charging!
![Page 16: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/16.jpg)
Algorithm
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Step-by-step
![Page 17: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/17.jpg)
Shape-Based Template Matching: Templates
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
![Page 18: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/18.jpg)
Shape-Based Template Matching: Evaluation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
![Page 19: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/19.jpg)
Shape-Based Template Matching: Process
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Left Cam Right Cam
![Page 20: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/20.jpg)
Shape-Based Template Matching: Process
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Left Cam Right Cam
![Page 21: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/21.jpg)
Shape-Based Template Matching: Process
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Left Cam Right Cam
![Page 22: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/22.jpg)
Left Cam Right Cam
3D Pose Estimation: Stereo Triangulation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
![Page 23: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/23.jpg)
3D Pose Estimation: Stereo Triangulation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Point 1 (X, Y)
Point 2 (X, Y)
Point 3 (X, Y)
Point 1 (X, Y)
Point 2 (X, Y)
Point 3 (X, Y)
Left Cam Right Cam
![Page 24: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/24.jpg)
3D Pose Estimation: Stereo Triangulation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
We use stereo triangulation to calculate depth
Left Cam Right Cam
![Page 25: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/25.jpg)
3D Pose Estimation: Stereo Triangulation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Real-World Coordinates
Point 1 (X, Y, Z)
Point 2 (X, Y, Z)
Point 3 (X, Y, Z)
Left Cam Right Cam
![Page 26: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/26.jpg)
3D Pose Estimation: Find the Relative Orientation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Roll
Pitch
Yaw
![Page 27: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/27.jpg)
3D Pose Estimation: Find the Relative Orientation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Roll
Pitch
Yaw
We can think about this as 3D plane fitting on selected points
![Page 28: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/28.jpg)
3D Pose Estimation: Find the Relative Orientation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Roll
Pitch
Yaw
We can think about this as 3D plane fitting on selected points
![Page 29: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/29.jpg)
3D Pose Estimation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Charging Port Position in
Real-World Coordinates
(X, Y, Z, roll, pitch, yaw)
Left Cam Right Cam
![Page 30: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/30.jpg)
Marker-less Eye-to-Hand Calibration
Cam to Plug
Transform
End-Effector to
Plug
Transform
![Page 31: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/31.jpg)
Marker-less Eye-to-Hand Calibration
![Page 32: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/32.jpg)
3D Pose Estimation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Now we have the following information:
● Distance to the port from the vision sensor in X, Y and Z
● Orientation of the port in relation to the vision sensor
● Location of the vision sensor in relation to the robot
● Location of the charging plug in relation to the robot
![Page 33: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/33.jpg)
3D Pose Estimation
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Let’s move the robot!
Now we have the following information:
● Distance to the port from the vision sensor in X, Y and Z
● Orientation of the port in relation to the vision sensor
● Location of the vision sensor in relation to the robot
● Location of the charging plug in relation to the robot
![Page 34: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/34.jpg)
Robot Motion Planning: RRT-Connect
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
![Page 35: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/35.jpg)
Plug-In Procedure: 3 Steps
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
![Page 36: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/36.jpg)
Charging! And Unplugging Afterwards
Detection of the
charging port
3D Pose
Estimation
Robot Motion
Planning
3-Step Plug-In
ProcedureCharging!
Simply track-back the same trajectory!
![Page 37: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/37.jpg)
![Page 38: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/38.jpg)
Results
Exp Charging Port at 10° angle Charging Port at 30° angle
1 Success Success: Misalignment
2 Success: Misalignment Failed: Missed rotation
3 Success Success
4 Failed: Missed rotation Success: Misalignment
5 Success: Misalignment Success: Misalignment
Attempts: 10
Successful Charging: 8
Small alignment issues: 5
Failed connection: 2
Due to initial calibration issues
![Page 39: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/39.jpg)
Future Work
● Test on a real vehicle
● Automatically open charging port lid
● Add more safety features
● Automatic communication between
the charger and the vehicle
● Incorporate automatic parking
![Page 40: 3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Vehicles](https://reader031.fdocuments.net/reader031/viewer/2022022415/5a6485ce7f8b9a36568b4c5d/html5/thumbnails/40.jpg)
Thank You!
Any Questions?