Autonomous Vehicle For Object Autonomous Vehicle For Object TrackingTracking

Group Members:- Prathamesh Joshi [15]Group Members:- Prathamesh Joshi [15]

Anirudh Panchal [31]Anirudh Panchal [31]

Project Guide:- Mr Kiran BhandariProject Guide:- Mr Kiran Bhandari

Goal Goal

““Build a mobile robot platform, including Build a mobile robot platform, including mechanics and electronics, implement and mechanics and electronics, implement and test a purely vision based image matching test a purely vision based image matching algorithm under real-time constraints.”algorithm under real-time constraints.”

Object matching using SIFTObject matching using SIFT

SIFT- Scale Invariant Feature TransformSIFT- Scale Invariant Feature Transform

To extract distinctive invariant features for reliable To extract distinctive invariant features for reliable matching –object recognition matching –object recognition

The features areThe features are ––invariant to scaleinvariant to scale ––invariant to rotationinvariant to rotation ––robust to (partial invariant to) affine distortionrobust to (partial invariant to) affine distortion ––robust to change in 3D viewpointrobust to change in 3D viewpoint ––robust to noiserobust to noise ––robust to change in illuminationrobust to change in illumination

SIFT SystemSIFT System

Extract features from reference images in Extract features from reference images in a databasea database

Extract features from a new given imageExtract features from a new given image

Select candidates -the new image key-Select candidates -the new image key-points (features) and match them with the points (features) and match them with the features in the databasefeatures in the database

Major Stages of Computing Major Stages of Computing Image FeaturesImage Features

1.1. Scale Space Extrema DetectionScale Space Extrema Detection

22. Key-point Localization. Key-point Localization

3.3. Orientation assignment of key-points Orientation assignment of key-points

44. Calculation of Descriptor vector of key-. Calculation of Descriptor vector of key-pointspoints

Convolving the image I(x,y) with a variable scale Convolving the image I(x,y) with a variable scale Gaussian Gaussian GG((xx,,yy,σ ).,σ ).

LL((xx, , yy,σ ) =,σ ) =GG((xx,,yy,σ )*,σ )*II((xx, , yy)………………………(1))………………………(1)

To efficiently detect stable key-points difference To efficiently detect stable key-points difference of Gaussians are calculated of Gaussians are calculated

DoGDoG((xx, , yy,σ ) = ,σ ) = LL((xx, , yy,,kkσ)− σ)− LL((xx,,yy,σ)………….. (2),σ)………….. (2)

Scale space imagesScale space images

…

first octave

…

…

second octave

…

…

third octave

…

fourth octave

…

…

Difference-of-Gaussian imagesDifference-of-Gaussian images

…

first octave

…

…

second octave

…

…

third octave

…

fourth octave

…

…

Finding extremaFinding extrema

Sample point is selected only if it is a Sample point is selected only if it is a minimum or a maximum of these pointsminimum or a maximum of these points

DoG scale spaceExtrema in this image

FilteringFiltering

For each candidate keypoint: Keypoints with low contrast are removed Responses along edges are eliminated

Orientation assignmentOrientation assignment

))),1(),1(/())1,()1,(((2tan),(

))1,()1,(()),1(),1((),( 22

yxLyxLyxLyxLayx

yxLyxLyxLyxLyxm

−−+−−+=−−++−−+=

θ

DescriptorDescriptor

Descriptor has 3 dimensions Descriptor has 3 dimensions (x,y,(x,y,θθ)) Orientation histogram of gradient magnitudesOrientation histogram of gradient magnitudes Position and orientation of each gradient Position and orientation of each gradient

sample rotated relative to keypoint orientationsample rotated relative to keypoint orientation

Recognition using SIFT features

Compute SIFT features on the input image

Match these features to the SIFT feature database

Each keypoint species 4 parameters: 2D location,scale, and orientation.

Depth measurement in real-time frames

Using Lagrange’s InterpolationUsing Lagrange’s Interpolation

Observation Table for Depth Observation Table for Depth DetectionDetection

The Lagrange’s interpolation Polynomial reduces to the following equation

F(x) = 0.0004 x2-0.2283 x+44.3429

This method has some advantages : a) Using only a single camera for the depth

finding. b) Having no direct dependency on the

camera parameters like focal length and etc. c) Having uncomplicated calculations. d) Requiring no auxiliary devices. d) Having a constant response time,

because of having a fixed amount of calculations;

f) This method can be used for both stationary and moving targets.

HARDWAREHARDWARE

The hardware consists of :-The hardware consists of :-

Wireless Video Camera.Wireless Video Camera. RS-232 cable.RS-232 cable. PCB board.PCB board. Robot Platform equiped with DC motor.Robot Platform equiped with DC motor.

Wireless Video CameraWireless Video Camera

Consists of Tx and Rx unit.Consists of Tx and Rx unit. Operates at 1.2Ghz.Operates at 1.2Ghz. Compatible with PC via TV tuner card.Compatible with PC via TV tuner card. Requires 9V for camera and 12V for Rx module.Requires 9V for camera and 12V for Rx module.

CIRCUIT DIAGRAMCIRCUIT DIAGRAM

PCB BOARD AND LAYOUTPCB BOARD AND LAYOUT

OperationOperation

Data is sent by serial module of Data is sent by serial module of Roborealm to the PCB board via RS-232.Roborealm to the PCB board via RS-232.

Data is processed by the microcontroller Data is processed by the microcontroller and control signals are given to the motors and control signals are given to the motors depending upon the input data.depending upon the input data.

Snap of VehicleSnap of Vehicle PlatformPlatform

ConclusionConclusion

Drawbacks:Drawbacks: Algorithm is computationally intensive.

False triggering.

Varying speed of the moving object.

Object may go outside the vision range of the camera.

Lack of integration between RoboRealm and Matlab.

ReferencesReferences A Moving Object Tracked by A Mobile Robot with Real-Time A Moving Object Tracked by A Mobile Robot with Real-Time

Obstacles Avoidance Capacity --Chung-Hao Chen, Chang Cheng, Obstacles Avoidance Capacity --Chung-Hao Chen, Chang Cheng, David Page, Andreas Koschan, and Mongi Abidi. David Page, Andreas Koschan, and Mongi Abidi.

A Vision System for IIT Kanpur Mirosot Robot Soccer Team CS497 A Vision System for IIT Kanpur Mirosot Robot Soccer Team CS497 Special Topics in Computer Science Semester II, 2002-03 Manu Special Topics in Computer Science Semester II, 2002-03 Manu Chhabra-99211.Chhabra-99211.

AA New Method for Depth Detection Using Interpolation Functions New Method for Depth Detection Using Interpolation Functions Mahdi Mirzabaki Azad University of Tabriz, Computer Engineering Mahdi Mirzabaki Azad University of Tabriz, Computer Engineering DepartementDepartement

DepthFinder, A Real-time Depth Detection System for Aided DepthFinder, A Real-time Depth Detection System for Aided DrivingDriving

Yi Lu Murphey, Jie Chen1, Jacob Crossman, Jianxin Zhang, Paul Yi Lu Murphey, Jie Chen1, Jacob Crossman, Jianxin Zhang, Paul Richardson and Larry. Richardson and Larry.

REFERENCESREFERENCES

A Real-time Image Recognition System for Tiny A Real-time Image Recognition System for Tiny Autonomous Mobile RobotsAutonomous Mobile Robots

Stefan Mahlknecht ,Roland Oberhammer ,Gregor Novak.Stefan Mahlknecht ,Roland Oberhammer ,Gregor Novak. Multipurpose Control System and Mobile Robot Multipurpose Control System and Mobile Robot

Development, for Control Development, for Control Motion Detection and Object Tracking in Image Motion Detection and Object Tracking in Image

Sequences---Zoran ˇZivkovi´.Sequences---Zoran ˇZivkovi´. Object Recognition from Local Scale-Invariant Object Recognition from Local Scale-Invariant

Features--David G. Lowe.Features--David G. Lowe. An Image Identification Algorithm using Scale Invariant An Image Identification Algorithm using Scale Invariant

Feature Detection Feature Detection

THANK YOUTHANK YOU