GESTURE BASED WIRELESS SHADOW ROBOT !
-
Upload
sharif-raihan-anik -
Category
Education
-
view
353 -
download
1
Transcript of GESTURE BASED WIRELESS SHADOW ROBOT !
![Page 1: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/1.jpg)
GESTURE BASED WIRELESS SHADOW ROBOTGroup# 40
A Project SubmittedBy
1. Ratul, Ahasan Ulla ID: 13-23070-12. Sumnoon, Sharif Md. ID: 11-19449-23. Kabir, Sharif Raihan ID: 12-21365-2
4. Islam, Md. Nazrul ID: 12-21730-2
Under the Supervision ofDr. M. Tanseer AliAssistant Professor
American International University - Bangladesh
![Page 2: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/2.jpg)
INTRODUCTION In the field of robotics, gesture controlling is becoming
popular lately. People need more user friendly and interactive robots for their work. Gesture detection makes thing easy for the users.
The project proposes a robotic body which mimics the action of human body
![Page 3: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/3.jpg)
PROJECT GOALS
Develop the shadow code to read the sensor readings. Design and develop a skeleton structure for robot frame Implementation of control mechanism or Actuators and Drive
Systems. Using of Kinect Sensor data for the real time movement of
upper portion of body. Basic non-feedback walking sequence for the forward walking
of the Shadow Robot. Create save and replay mode the shadow commands . Implement a feedback IP camera at robot head.
![Page 4: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/4.jpg)
PROJECT BLOCK DIAGRAM
![Page 5: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/5.jpg)
THE KINECT SENSOR
1. Infrared (IR) projector.2. IR depth sensor.3. RGB camera.4. Group of tilt motors.5. Three-axis accelerometer.6. Microphone array.
![Page 6: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/6.jpg)
KINECT SDK
• The Kinect SDK is synthesized set of libraries of software’s and the tools that is helpful for us for using the Kinect-based natural input.
![Page 7: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/7.jpg)
COLOR STREAM Color image
formatResolution FPS Data
InfraredResol
uzion640x480
Fps30
640 x 480 30 Pixel format
is gray16
RawBayerRes
oluzion1280x9
60Fps12
1280 x 960 12 Bayer data
RawBayerRes
oluzion640x48
0Fps30
640 x 480 30 Bayer data
RawYuvResol
uzion640x480
Fps15
640 x 480 15 Raw YUV
RgbResoluzio
n1280x960Fps
12
1280 x 960 12 RGB
(X8R8G8B8)
RgbResoluzio
n640x480Fps1
5
640 x 480 15 Raw YUV
![Page 8: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/8.jpg)
DEPTH STREAM
Depth image format Resolution Frame rate
Resoluzion640x480Fps30 640 x 480 30 FPS
Resoluzion320x240Fps30 320 x 240 30 FPS
Resolution80x60Fps 80 x 60 30 FPS
![Page 9: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/9.jpg)
DEPTH IMAGE
![Page 10: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/10.jpg)
THE BODY POSITIONING
The body positioning system incorporates depth mapping and decision tree analysis.
![Page 11: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/11.jpg)
SKELETON IMAGE
![Page 12: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/12.jpg)
JOINT ANGLES CALCULATION FROM SKELETON IMAGE
Skeleton joints tracking from Skeleton Image. Skeleton frame data to skeleton Arrays. Angle Calculation (ex: Inverse Kinematic
approach) using X,Y,Z co-ordinates of the joints that is saved in the arrays.
![Page 13: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/13.jpg)
INVERSE KINEMATIC ANGLE CALCULATION
![Page 14: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/14.jpg)
ARDUINO AND SERVO MOTORS
The Arduino Mega is used in our project for two purposes.
To generate servo signals from the received strings.
To calibrate the servo motors for initial position making and for creating walking sequence
![Page 15: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/15.jpg)
GENERATING SERVO SIGNAL USING ARDUINO MEGA
END
START
If string Available
Read HC-05
If the slave String is entirely Complete
Break the string until “&” sign
Convert the rest of the piece into servo angle
Compare servo Id if it matches with saved Ids
Break it further until “:” for servo ID
NOYES
YES
NO
NO YES
![Page 16: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/16.jpg)
CALIBRATION OF JOINT SERVOS USING ARDUINO
Arduino can be used to control up to 12 servos with minimum jitter. Two major parts are there in the application
The first part is the firmware, that needs to upload in the arduino. The second part is to install a simple software named “Serial servo
controller”.
![Page 17: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/17.jpg)
COMMUNICATION BETWEEN MASTER AND SLAVE BLUETOOTH
To enable Wireless connectivity that enables windows pc bluetooth , C# language is used. Bluetooth can be programmed by C# in two ways.
Using Windows Sockets. That is programmed under windows sockets class.
Using Computer’s Serial Comport.
![Page 18: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/18.jpg)
SENDING STRING DATA FROM PC BLUETOOTH
if (serial.IsOpen){try {String hexstring = ("1" + ":" + LShoulder + "&" + "2" + ":" + LElbow + "&" + "3" + ":" + RShoulder + "&" + "4" + ":" + RElbow);;serial.Writeln(hexstring);Thread.Sleep(1); }catch (Exception ex){para.Inlines.Add("Failed to SEND" + data + "\n" + ex + "\n"); }}
![Page 19: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/19.jpg)
NON-FEEDBACK WALKING OF BIPED
The walking of the Shadow Robot follows 8 repetitive steps.
I. Stand still II. Lean Left III. Right Step forward
IV. Stand stillV. Lean RightVI. Left step forward
![Page 20: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/20.jpg)
DESIGN PROTOTYPE AND SIMULATION
![Page 21: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/21.jpg)
DESIGN PROTOTYPE AND SIMULATION
![Page 22: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/22.jpg)
DESIGN PROTOTYPE AND SIMULATION
![Page 23: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/23.jpg)
APPLICATIONS OF THE PROJECT
Artificial Intelligence. Industrial use. Simulation. Training & Education. Assistive living.Entertainment.
![Page 24: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/24.jpg)
FUTURE WORK
Design A Dynamic Walk sequence with feedback from Gyro and accelerometer.
Finger and Face tracking. Making an Trained and Intelligent
Humanoid .
![Page 25: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/25.jpg)
REFERENCES
[1] Jungong Han; Ling Shao; Dong Xu; Shotton, J., "Enhanced Computer Vision With Microsoft Kinect Sensor: A Review," in Cybernetics, IEEE Transactions on, vol.43, no.5, pp.1318-1334, Oct. 2013 doi: 10.1109/TCYB.2013.2265378.
[2] P. Baerlocher and R. Boilic, Inverse Kinematics Techniques for the Interactive Posture Control of Articulated Figures, PhD thesis, EcolePolytechniqueFederale de Lausanne, 2001.
[3] Machida, E.; Meifen Cao; Murao, T.; Hashimoto, H., "Human motion tracking of mobile robot with Kinect 3D sensor," in SICE Annual Conference (SICE), 2012 Proceedings of , vol., no.,pp.2207-2211,20-23Aug.2012.
[4] Xu, D. and Acosta, A. (2005) An Analysis of the Inverse Kinematics for a 5-DOF Manipulator. International Journal of Automation and Computing, 2, 114-124. http://dx.doi.org/10.1007/s11633-005-0114-1.
[5] Stephan Waldherr; Roseli Romero and Sebastian Thrun, A Gesture Based Interface for Human-Robot Interaction. , Autonomous Robots, September 2000, Volume 9, Issue 2, pp 151-173.
[6] W. Xu and E. J. Lee, “Continuous gesture recognition system using improved HMM algorithm based on 2D and 3D space”, International Journal of Multimedia and Ubiquitous Engineering, vol. 7, no. 2, (2012), pp. 335-340.
[7] Bhuiyan , M. & Picking, R. (2009). Gesture-controlled user interfaces, what have we done and what's next?, Proceedings of the Fifth Collaborative Research Symposium on Security, E-Learning, Internet and Networking (SEIN 2009), Darmstadt, Germany, 26-27 November 2009, pp59-60.
![Page 27: GESTURE BASED WIRELESS SHADOW ROBOT !](https://reader034.fdocuments.net/reader034/viewer/2022042619/5873dbf11a28abd72e8b4c73/html5/thumbnails/27.jpg)
Thank You !