Vision Control of Mobile Robots
-
Upload
pulkit-shah -
Category
Documents
-
view
230 -
download
0
Transcript of Vision Control of Mobile Robots
-
7/29/2019 Vision Control of Mobile Robots
1/40
P R E S E N T E D B Y P U L K I T S H A H( 2 0 1 1 E E A 2 2 3 8 )
VISION CONTROL OF A
MOBILE ROBOT
Guide:Dr. Shubhendu Bhasin
-
7/29/2019 Vision Control of Mobile Robots
2/40
PROBLEM STATEMENT
Mobilerobotwith
camera
Designing a control law todrive the robot from positionA to position B using onlyimages as inputs
A B
-
7/29/2019 Vision Control of Mobile Robots
3/40
PROBLEM STATEMENT
Feature points
Mobilerobotwithcamera Epipoles
-
7/29/2019 Vision Control of Mobile Robots
4/40
COMMON APPROACH TO SOLVING
PROBLEM
Pose based visual servoing
Velocity controller
-
7/29/2019 Vision Control of Mobile Robots
5/40
A BRIEF LOOK INTO THE PBVS
APPROACH: HOMOGRAPHY
*
cos sin 0
sin cos 00 0 1
( ) [ ( ) ( ) 0]
* ( *) **
( *)*
i i
T
x y
T
i
i i
T
m Rm q
R
q t q t q t
d n mm Hm
qH R n
d
Sovle forR ,
, n*
-
7/29/2019 Vision Control of Mobile Robots
6/40
AMBIGUITY IN HOMOGRAPHY
Solution to H is not unique, we get 8solutions.
We can eliminate four of them by putting constraint that the robot must
always be in front of the plane.
From the remaining four we can eliminate two by putting a constraint on
it must be between 0 and 2 Now we are left with two solutions, both of which satisfy all the above
constraints.
For resolving this ambiguity we must have some prior knowledge of the 3-
D scene in which our robot is operating
For example, if all the feature points are taken on the z-plane then the
normal vectorn* would be [0 0 1]T
-
7/29/2019 Vision Control of Mobile Robots
7/40
THE IBVS APPROACH
Image based visual servoing
-
7/29/2019 Vision Control of Mobile Robots
8/40
SOME BASIC CONCEPTS IN VISION
-
7/29/2019 Vision Control of Mobile Robots
9/40
PERSPECTIVE CAMERA MODEL
x
y
Xp f
Z
Yp fZ
*
0 0
0 0
0 0 1
p K P
f
K f
( , )x yp p
-
7/29/2019 Vision Control of Mobile Robots
10/40
EPIPOLES
-
7/29/2019 Vision Control of Mobile Robots
11/40
HOW DO WE FIND EPIPOLES?
-
7/29/2019 Vision Control of Mobile Robots
12/40
EPIPOLAR CONSTRAINT
-
7/29/2019 Vision Control of Mobile Robots
13/40
EPIPOLES FROM ESSENTIAL MATRIX
Epipoles are just the right and left nullspaces of the Essential matrix
00
T
d
a
e EEe
-
7/29/2019 Vision Control of Mobile Robots
14/40
COMPUTING THE ESSENTIAL MATRIX, E
Computing E from Point Matches Assume that you have m
correspondences Each correspondence satisfies:
E is a 3x3 matrix (9 entries) We get a HOMOGENEOUS linearsystem with 9 unknowns
0, 1,...,Tri lip Ep i m
B
-
7/29/2019 Vision Control of Mobile Robots
15/40
11 12 13
21 22 23
31 32 33
11 21 31 12
22 32 13 23 33
( , ,1)
( ', ',1)
0; 1,...,
' ' 1
1
' ' '
' ' ' 0
T
li i i
T
ri i i
Tri li
i
i i i
i i i i i i i
i i i i i
p u v
p u v
p Ep i m
e e e u
u v e e e v
e e e
u u e u v e u e v u e
v v e v e u e v e e
COMPUTING E
where m>7
-
7/29/2019 Vision Control of Mobile Robots
16/40
COMPUTING E
11
1 1 1 1 1 1 1 1 1 1 1 1
12
13
21
22
23
31
32
33
' ' ' ' ' 1
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .0
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .' ' ' ' ' 1m m m m m m m m m m m m
eu u u u u v u v v v u v
e
e
ee
e
e
eu u u u u v u v v v u v
e
A minimum least squares problem
-
7/29/2019 Vision Control of Mobile Robots
17/40
WHY EPIPOLES?
When the actual and desired configurations are
aligned, both epipoles coincide with the origin ofthe corresponding image frame.
Epipoles =0Robots have same orientation!
-
7/29/2019 Vision Control of Mobile Robots
18/40
INTUITIVE OUTLINE OF CONTROL
STRATEGY
Step 1:Use acontrollaw todrive therobotsuch thatepipoles
go tozero
Step 2:Use afeature-basedcontroller toeliminate thetranslationerror.
A
-
7/29/2019 Vision Control of Mobile Robots
19/40
STRATEGY
Mobilerobotwithcamera
Actualposition Desired
position
Step 2:
Translating todesired position
Intermediateposition
Step 1: Aligning theactual and desiredviews
-
7/29/2019 Vision Control of Mobile Robots
20/40
cos
sin
x v
y v
s = [x y ]T
PROBLEM FORMULATION: DEFINING A
NONHOLONOMIC KINEMATIC MODEL
-
7/29/2019 Vision Control of Mobile Robots
21/40
FIRST STEP: ZEROING THE EPIPOLES
Deriving epipole kinematics:
( , ) ( , , )au due e f x y
( , ) ( , )au due e f v
Note that the v component of epipole coordinate (u,v)will be zero at all times as the robot is moving in aplane
-
7/29/2019 Vision Control of Mobile Robots
22/40
GEOMETRICAL SETUP
sin cos
cos sin
du
au
x
e fy
x ye f
x y
-
7/29/2019 Vision Control of Mobile Robots
23/40
EPIPOLE KINEMATICS
2 2 2 2
2 2
2 2
( )
( )
au auau du auau
au duau duau
au
e e fsign e e e fe v
d f f
e e fsign e ee vd f e f
We need to design v and such that eauand edu go to zero, the obvious choice is:
1 au
du
evD
e
au
du
e v
De
-
7/29/2019 Vision Control of Mobile Robots
24/40
SOLUTION APPROACH: ROADBLOCK
au
du
e vD
e
2 2 2 2
2 2
2 2
( )
( )
0
au auau du au
au duau du
au
e e fsign e e e f
d f fD
e e fsign e e
d f e f
Distance between the actual and thedesired robot position) is unknown in apurely image-based control setting.
A
-
7/29/2019 Vision Control of Mobile Robots
25/40
SOLUTION APPROACH
It is possible to us an approximate inverse of D bysetting
2 2
2 2
1
2 2 2 2
0 ( )
( )
au
au du
au du
au du
df e f sign e e
e e fDf f
e f e f
11
2
vv
D v
Where an estimate ofd,
has been used
A
-
7/29/2019 Vision Control of Mobile Robots
26/40
SOLUTION APPROACH
2 2
2 21 11
2 2
1 1
0
au
au du
du
e fde v vv d e f
D DDe v v
d
d
The resulting epipole velocities are:
-
7/29/2019 Vision Control of Mobile Robots
27/40
DESIGNING CONTROL INPUTS
2 2/
1 22 2
1 auau au dudu
e fde k e k e
d e f
/
2
du du
de k e
d
11
/
22 .
au
du
k ev
k ev
Let:
Closed loopepipole
dynamicsare:
-
7/29/2019 Vision Control of Mobile Robots
28/40
CHOOSING UPDATE EQUATION FOR
ESTIMATE
2 2/
1 22 2
1 auau au dudu
e fde k e k e
d e f
Closed loopepipole
dynamicsare:
/
2
du du
de k e
d
Needs to be positive in order for edu to converge to zero
A
B
-
7/29/2019 Vision Control of Mobile Robots
29/40
CHOOSING UPDATE EQUATION FOR
ESTIMATE
xx yyd
d
/2
2 2 2
( )
du
au du
ed k f d e e f
B
0
0 0 0
0
0
. 1 1
.
t
t
d d dt d dd
d dd d dt
We can choose any
value of 0>d
0
B
-
7/29/2019 Vision Control of Mobile Robots
30/40
CHOOSING UPDATE EQUATION FOR
ESTIMATE
B
Epipoles converge to zero in finite time !
/2
2 2 2
( )
du
au du
ed k f d
e e f
B
-
7/29/2019 Vision Control of Mobile Robots
31/40
SECOND STEP: MATCHING THE
FEATURES
the desired and the actual epipoles are zero andthe intermediate robot configuration qi is alignedwith the desired configuration
We need a control law which will make thetranslational error to zero.
Let be the norm difference betweenthe actual and the desired projection of the featurepoint and
B
2 2|| || || ' ||D p p
0
tv K D
-
7/29/2019 Vision Control of Mobile Robots
32/40
PROOF: SECOND STEP
'
TX Y
p f fZ Z
TX Y
p f fZ d Z d
2 2 2 2 2
2 2
(2 )|| || || ' || ( )
( )
d Z dD p p f X Y d
Z Z d
2 21 12 2
V d d
2. t tV d v k dD k d
Globally
Exponentially
Stable!
-
7/29/2019 Vision Control of Mobile Robots
33/40
CONTRIBUTION
-
7/29/2019 Vision Control of Mobile Robots
34/40
SIMULATION AND RESULTS
Step 1
Step 2
-
7/29/2019 Vision Control of Mobile Robots
35/40
SIMULATION: ERROR IN X AND Y
COORDINATES
xe(t)ye(t)
-
7/29/2019 Vision Control of Mobile Robots
36/40
SIMULATION: ERROR IN ORIENTATION
AND DEPTH ESTIMATE
Error in(t)
-
7/29/2019 Vision Control of Mobile Robots
37/40
SIMULATION: LINEAR AND ANGULAR
VELOCITIES
Step 2Step 1
Linear velocity, v Angular velocity,
B
-
7/29/2019 Vision Control of Mobile Robots
38/40
ADVANTAGES OF THIS APPROACH
It is an IBVS hence no prior info needed of the 3-Dscene
As we use epipoles singularities stemming from
inversion on Jacobian are avoided As we do not need to decompose a homography
matrix, ambiguity in solutions is avoided
IBVS more robust to camera calibration than PBVS
where we do not rely on the calibration matrix toextract epipoles from the images.
B
-
7/29/2019 Vision Control of Mobile Robots
39/40
FUTURE WORK
Extend the algorithm to discrete time system
to account for the sampling rate of camera
acquiring images(in real world scenario)
Extracting the feature points from a givenimage
To implement the strategy on a mobile robot
Combining IBVS and PBVS methods toeliminate the issue of singular configurations
in IBVS (2 D visual servoing)
-
7/29/2019 Vision Control of Mobile Robots
40/40
QUESTIONS