Knowdiff visiting lecturer 140 (Azad Shademan): Uncalibrated Image-Based Robotic Visual Servoing

Uncalibrated Image-based Uncalibrated Image-based Control of RobotsControl of Robots

Azad ShademanPhD Candidate

Computing Science, University of AlbertaEdmonton, Alberta, CANADA

[email protected]

Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008

Vision-Based Control

current

desired

Left Image Right Image

A

A

A

B

BB


Vision-Based Control

Left Image Right Image

B

BB


Where is the camera located? Eye-to-Hand

e.g.,hand/eye coordination

Eye-in-Hand


Vision-Based Control Feedback from visual sensor (camera) to

control a robot Also called visual servoing Visual servoing is the task of minimizing a

visually specified objective by giving appropriate control commands to a robot

Is it any difficult?Images are 2D, the robot workspace is 3D 2D data 3D geometry


Visual Servo Control law Position-Based:

Robust and real-time pose estimation + robot’s world-space (Cartesian) controller

Image-Based:Desired image features seen from cameraControl law entirely based on image features

Hybrid:Depth information is added to image data to

increase stability


Position-Based Robust and real-time relative pose

estimation Extended Kalman Filter to solve the

nonlinear relative pose equations. Cons:

EKF is not the optimal estimator.Performance and the convergence of pose

estimates are highly sensitive to EKF parameters.


Position-Based

Desired pose

Estimated pose


Position-Based

Measurement noise

State variable

Process noise

yaw pitch roll

Measurement equation (projection) is nonlinear and must be linearized.


K

xk-1,k-1

zk

Rk

Pk,k

Pk,k-1

Ck

Qk-1

xk,k

xk,k-1

Pk-1,k-1

Kalman Gain

Measurement noise covariance

A priori error cov. @ k-1

Process noise covariance

Initial/previous state

Linearization

Measurement

State update

State prediction Error cov. prediction

Error cov. update


Image-Based

Desired Image feature

Extracted image feature


Visual-motor Equation

x1

x2

x3

x4

q=[q1 … q6]

Visual-Motor Equation

This Jacobian is important for motion control.


Visual-motor JacobianImage spacevelocity

Joint spacevelocity

A

A

B

B


Image-Based Control Law

1. Measure the error in image space

2. Calculate/Estimate the inverse Jacobian

3. Update new joint values


Image-Based Control Law

Desired Image feature

Extracted image feature


Jacobian calculation Analytic form available if model is known.

Known model Calibrated

Must be estimated if model is not known

Unknown model Uncalibrated


Calibrated: Interaction Matrix

Analytic form depends on depth estimates.

Camera/Robot transform required. No flexibility.

CameraVelocity


Uncalibrated: Visual-Motor Jacobian

A naïve method: Orthogonal projections



A naïve method: Orthogonal projections

…



A popular local estimator:

Recursive secant method (Broyden update):


Relaxed model assumptions

Traditionally: Local methods No global planning (-) Difficult to show

asymptotic stability condition is ensured (-)

Main problem of traditional methods is the locality.

Calibrated vs. Uncalibrated

Model derived analytically Global asymptotic

stability (+) Optimal planning is

possible (+) A lot of prior knowledge

on the model (-)

Global Model Estimation (Research result)

Optimal trajectory planning (+)

Global stability guarantee (+)


Synopsis of Global Visual Servoing Model Estimation (Uncalibrated) Visual-Motor Kinematics Model Global Model

Extending Linear Estimation (Visual-Motor Jacobian) to Nonlinear Estimation

Our contributions:K-NN Regression-Based EstimationLocally Least Squares Estimation


Local vs. Global

Key idea: using only the previous estimation to estimate the Jacobian

1. RLS with forgetting factor Hosoda and Asada ’94

2. 1st Rank Broyden update: Jägersand et al. ’97

3. Exploratory motion: Sutanto et al. ‘98

4. Quasi-Newton Jacobian estimation of moving object: Piepmeier et al. ‘04

Key idea: using all of the interaction history to estimate the Jacobian

Globally-Stable controller design

Optimal path planning Local methods don’t!


3 NN

K-NN Regression-based Method

q1

q2

x1

q1

q2

?


q1

q2

x1 ?

K-neighbour(q)

(X,q)

Locally Least Squares Method


Eye-to-hand Experiments

Puma 560 Stereo vision Features: projection of the end-effector’s

position on image planes (4-dim) 3 DOF for control


Measuring the Estimation Error


Global Estimation Error


Visual Task Specification Image Features:

Geometric primitives (points, lines, etc.) Higher order image moments Shape parameters …

Visual Tasks Point-to-point (point alignment) Point-to-line (colinearity) Point-to-plane (coplanarity) …


Eye-in-hand


Eye-in-Hand Experiments


Mean-Squared-Error

Task 1

Task 2


Task Errors


Conclusions Reviewed position-based and image-based visual

servoing schemes. Presented two global methods to learn the visual-

motor function. KNN suffers from the bias in local estimations. LLS (global) works better than the KNN (global) and

local updates. The Jacobian of more complex visual tasks can also

be learned using LLS method.

[email protected]


Thank you!


Visual Ambiguity: Single Camera


Visual Ambiguity: Stereo

Knowdiff visiting lecturer 140 (Azad Shademan): Uncalibrated Image-Based Robotic Visual Servoing

Technology

Transcript of Knowdiff visiting lecturer 140 (Azad Shademan): Uncalibrated Image-Based Robotic Visual Servoing