PyCon2013

Application of Python in Robotics

Lentin Josephwww.lentinjoseph.com

PyCon2013

About Speaker

• Former Robotics Engineer @ASIMOV Robotics

• Founder of www.technolabsz.com• Robotics, Machine Vision for last 3 years• Python and GNU/Linux for last 5 years • Website : www.lentinjoseph.com

PyCon2013

Agenda

• Introduction to Robotics• Designing an intelligent ball tracking robot• Block Diagram explanations• Technology explanations• Role of Python• Robot demos

PyCon2013

Robotics in a Nutshell

• What is Robotics

• What is a Robot

• Types of Robot

PyCon2013

Robotics and Robot

• Robotics deals with design, construction, operation and application of robots

• Robot is a machine build for doing real world functions

PyCon2013

Types of Robots

Manipulators Mobile Robots Hybrid Robots

PyCon2013

Designing a ball tracking robot

PyCon2013

What this Robot Can do??• It will track coloured objects like balls

Add-on :• It can respond to human voice commands

using Artificial Intelligence

• It can detect obstacles

PyCon2013

What are the Features

• Locomotion• Image Processing capability• Speech synthesis and recognition• Basic Artificial Intelligence• Obstacle avoidance

PyCon2013

Block Diagram

Arduino

Raspberry Pi Camera

Motor Control board

Motors

Speaker and Mic

Ultrasonic sound sensors

PyCon2013

How this components look like??

PyCon2013

Block Diagram Explanation

Arduino

Raspberry Pi Camera

Motor Control board

Motors

Speaker and Mic

Ultrasonic sound sensors

PyCon2013

Motors• DC Geared Motor

• Specification• 12 volt• 60 rpm

PyCon2013

Block Diagram

Arduino

Raspberry Pi Camera

Motor Control board

Motors

Speaker and Mic

Ultrasonic sound sensors

PyCon2013

Motor Driver• Name: MotoMama• L298 H-Bridge• 2 Motor output port• Arduino Compatible• Zigbee, Bluetooth

ready• LED Indicators

PyCon2013

Block Diagram

Arduino

Raspberry Pi Camera

Motor Control board

Motors

Speaker and Mic

Ultrasonic sound sensors

PyCon2013

Arduino• Opensource hardware• AtMega 328

micro-controller• GPIO and Analog pins• USB Compatible• 5volt supply• Easy programming

interface

PyCon2013

Block Diagram

Arduino

Raspberry Pi Camera

Motor Control board

Motors

Speaker and Mic

Ultrasonic sound sensors

PyCon2013

Ultra Sonic Sound Sensor• Obstacle detection• Compatible with

Arduino• Detect collision up to 2

cm to 3 m• TTL Compatible

PyCon2013

Block Diagram

Arduino

Raspberry Pi Camera

Motor Control board

Motors

Speaker and Mic

Ultrasonic sound sensors

PyCon2013

Raspberry Pi• Mini Computer• Running GNU/Linux• Specification:

• 700Mhz Processor• 512 MB of RAM• USB Ports• GPIO Pins• Ethernet• Audio Jack

PyCon2013

Block Diagram

Arduino

Raspberry Pi Camera

Motor Control board

Motors

Speaker and Mic

Ultrasonic sound sensors

PyCon2013

How it Works??Grabbing frames

from cameraProcess Each

frameFind position of ball in the frame

If ball is left side of the image, calculate

how much pixels from center, send

this control signals to arduino and vice

versa

Arduino ProcessComands from

Raspberry Pi and sent commands to motor driver

Motor controller board sent PWM

to MotorsMotors

PyCon2013

Grab frames and Process??

• What is a Digital Image• What is Digital Image Processing• How to do color detection using Image

processing• What is the role of python in image

processing• Which library used for image processing

PyCon2013

Open Computer Vision(Open-CV)

• Developed by Intel maintaining by Willogarage , Itseez

• Designed to do real time computer vision• Written in C++ ,support for C,C++,Python,

Java

PyCon2013

How colour tracking code works??

• Colour tracker uses Python wrapper of Open CV

Grabbing frames from

cameraSmoothing BGR->HSV

ThresholdFinding momentsFinding centroid of moments

Compare center pixel of images

and centroid

Send the left and right offset from center to arduino

PyCon2013

How colour tracking code works??

• Demo Code

PyCon2013

How to send commands to arduino??

• Python Serial Port Communication

• >>import serial• >>ser=serial.Serial(‘/dev/ttyUSB0’,57600)• >>ser.write(“L60”)• >>print ser.read();• >>ser.close()

PyCon2013

Working of Arduino Firmware

Initialize Serial Communication

Receive Control Signal from Rpi

NodeSplit Direction

and Value

Scale the value and sent to motor

controller

Receive Distance data from

Ultrasound sensor

Stop rover if obstacle

PyCon2013

Working of Ping Sensor

PyCon2013

How to do communication between nodes??

• What is Robotic Operating System(ROS)• What is ROS and its concepts• ROS Topics, Nodes, ROS Core, ROS

Package

PyCon2013

Communication between nodes??

• Two Nodes• Arduino node and Raspberry Pi node

ROS MasterRaspberry Pi

NodeArduino Node

Camera Arduino

Publish Topic Subscribe Topic

PyCon2013

Speech Processing and AI

• Speech Synthesis : Festival and Pyfestival• Speech Recognition : GStreamer ,Pocket

Sphinx• Artificial Intelligence: AIML and PyAIML• GUI :Qt and PyQT

PyCon2013

Some of the projects !!

PyCon2013

Demo

PyCon2013

Reference●www.opencv.org●www.ros.com●www.arduino.cc

PyCon2013

Questions

PyCon2013

Thanks

Lentin Joseph

www.lentinjoseph.comwww.technolabsz.com