Yanfei LiuCLEMSON U N I V E R S I T Y Dynamic Workcell for Industrial Robots Dept. of Electrical and...
-
date post
21-Dec-2015 -
Category
Documents
-
view
224 -
download
2
Transcript of Yanfei LiuCLEMSON U N I V E R S I T Y Dynamic Workcell for Industrial Robots Dept. of Electrical and...
Yanfei Liu
CLEMSONCLEMSONU N I V E R S I T Y
Dynamic Workcell for Industrial Robots
Dept. of Electrical and Computer EngineeringClemson University, SC
Clemson University
Outline
• Motivation for this research– Current status of vision in industrial workcells
– A novel industrial workcell with continuous visual guidance
• Work that has been done– Our prototype: camera network based industrial workcell
– A new generic timing model for vision-based robotic systems
– Dynamic intercept and manipulation of objects under semi-structured motion
– Grasping research using a novel flexible pneumatic end-effector
Clemson University
Motivation for this research
• Current industrial workcells– No vision or a single snapshot in certain locations
– Disadvantages
• Cannot deal with flexible parts
• Cannot deal with uncertainty
Clemson University
Motivation for this research• Our novel dynamic workcell design
– Manipulation is integrated with visual sensing– Applications ( reduce fixtures, handle objects on the ship)
Clemson University
System architecture
– A set of cameras embedded into the workcell
– An industrial manipulator with its conventional controller
Clemson University
Experimental platform• Our prototype
– Staubli RX130 manipulator with its conventional controller
– Six cameras, wired to two PC-RGB framegrabbers mounted in a Compaq Proliant 8500 computer
– V+ Operating systems and language
– Alter command to accomplish real time motion
Clemson University
Tracking experiments
Clemson University
First part:
A new generic timing model for vision-based robotic system
Clemson University
Introduction
• Classical visual servoing structure
– eye-in-hand systems• Corke (1996), an eye-in-hand manipulator to fixate on a
thrown ping-pong ball
• Gangloff (2002), a 6-DOF manipulator to follow unknown but structured 3-D profiles.
– part-in-hand systems• Stanvnitzky (2000), align a metal part with another fixed part
– mobile robot systems• Kim (2000), a mobile robot system to play soccer
controlpower
amplifiers robot
camera
desired position
+-
e
Clemson University
Introduction
• Vision guided control structure– Allen (1993), a PUMA-560 tracking and grasping a moving model
train which moved around a circular railway.
– Nakai (1998), a robot system to play volleyball with human beings.
– Miyazaki (2002), a robot accomplished a ping pong task based on virtual targets
camera
desiredposition
control jointcontroller
robot encoders+-
e
Clemson University
Introduction
• Three common problems in visual systems– Maximum possible rate for complex visual sensing and
processing is much slower than the minimum required rate for mechanical control.
– Complex visual processing introduces a significant lag (processing lag) between when reality is sensed and when the result from processing a measurement of the object state is available.
– A lag (motion lag) is produced when the mechanical system takes time to complete the desired motion.
Clemson University
Previous work
• the first two of the three problems have been addressed to some extent in previous works. All of these works neglect the motion time (motion lag) of the robot.
• Corke and Kim, presented timing diagram to describe time delay, used discrete time models to model the systems and simplified these asynchronous systems to single-rate systems.
Work Image processing rate
(HZ)
Control rate
(HZ)
Processing lag
(ms)
Motion lag
(ms)
Corke, Good 50 70 48 --
Stavnitzky, Capson 30 1000 -- --
Kim et. al. 30 30 90 --
Allen et. al. 10 50 100 --
Nakai et. al. 60 500 -- --
this work 23 250 151 130
Clemson University
Timing Model: notation
processinglag
s1 s2
u1
kq
c1 … cN
motion lag
u2
f
sensing
image processing
synchronizing tracking
controlling
finishingmotion
Clemson University
Timing Model: our prototype
• Inherent values (obtained by analysis/measurement) s = 33ms u = 19+30+14 = 63ms wm = 39ms wf = (5+16+27)/3 = 16ms w = 39+16 = 55ms
l = s + u + w =151ms f = 130ms
• User-variable values c = 4ms q = 40ms
Clemson University
Experiments• Problem description
– The most recently measured position and velocity of the object is where the object was (l+k) ms before, xt-l- k, vt-l- k
– The current position, xt
– N, d?
)( klvxx kltkltt
Clemson University
Experiments
• Solutions ))1(( fincivixix ttit
dinixix qtit
ivcd
ixixivcfin
t
qttt
)(
c
divt
Constraint:
N
ixixid qtit
Clemson University
Experiments: model validation
• Setup
– A small cylindrical object is dragged by a string tied to a belt moving at a constant velocity.
– The robot will lunge and cover the object on the table with a modified end-effector, a small plastic bowl.
Clemson University
Experiments (video)
Clemson University
Experiments
q=40 q=80Velocity
range(mm/s)
Stdev
range
Catch percentage
Velocity
range(mm/s)
Stdev
range
Catch
percentage
84.4 – 97.4 1.3 –3.8 100% 85.9 – 95.1 2.5 - 3.7 100%
129.8-146.7 1.7 - 3.2 100% 126.1 – 137.7 1.7 – 3.3 100%
177.6 – 195.1 0.5 – 2.6 100% 175.8 – 192.8 1.1 – 2.7 100%
• Experiment description– We set q to two different values, 40 and 80, in these two sets of
experiments. We let the object move at three different velocities. For each velocity, we ran the experiment ten times.
• Results
Clemson University
Experiments (video)
Clemson University
Second part:
Dynamic Intercept and Manipulation of Objects under Semi-Structured Motion
Clemson University
Scooping balls (video)
Clemson University
Scooping balls: problem description
Start tracking Make prediction (t)
Open loopClosed loop
Impact (t+i)
x
y
xt , yt : object position at time tvx , vy: object velocity at time txr , yr: initial robot positionxf , yf : final impact position
robot
Unknown variables: yf , i
Clemson University
Scooping balls: solution
• Solutions
rf xx
fyt yivy
fxt xivx
x
tr
v
xxi
trx
ytf xx
v
vyy
c
fim
• Object unsensed time– Time between the last instant when reality is sensed
and the final impact time
– Delay between visual sensing and manipulation
Clemson University
Timeline description: object unsensed timet
processing lag(l) + k
synchronizing tracking
q q
controlling
finishing motion
closed loop open loop
…c1 cN…c1 cN
m 20
motion lag (f)
t = l + k + 4m+ f m < N = 10, k < 30 + 14 = 44ms
t = 151 + ( 40 + 44 ) / 2 +115 = 308ms
…
Clemson University
Impact point
20 alters
impact point
10 alters
20 alters
impact point
10 alters
y
z
Clemson University
Equations
tvxx
tvxx
if
iif
ˆ
t
xxvv
ff
i
ˆ
3082
wvvi
• Implementation– Predict the maximum acceleration of the object motion that the
robot still can achieve a successful catch
– Calculate the size of the end-effector in order to overcome the maximum acceleration of the moving objects
• Solutions
Clemson University
Experimental Validation
Bowl Scooper1 scooper2
Catch Miss Catch Miss Catch miss
Catch 88.6% 1.4% 84.3% 2.9% 95.7% 0.0%
Miss 1.4% 1.4% 2.9% 4.3% 0.0% 0.0%
Too fast 7.1% 5.1% 4.3%
Total failure 2.8% 5.8% 0.0%
• Setup– Two types of end-effector (bowl, two scoopers with different width).– Three types of interference (wind, bump, ramp)
• Results– With wind interference
Clemson University
Experimental Validation
Bowl Scooper1 scooper2
Catch Miss Catch Miss Catch miss
Catch 2.9% 0.0% 2.9% 7.1% 1.4% 4.3%
Miss 0.0% 84.3% 0.0% 80.0% 2.9% 85.7%
Too fast 12.9% 10.0% 5.7%
Total failure 0.0% 7.1% 7.2%
Bowl Scooper1 scooper2
Catch Miss Catch Miss Catch miss
Catch 37.1% 7.1% 31.4% 0.0% 50.0% 2.9%
Miss 1.4% 45.7% 2.9% 52.9% 0.0% 45.7%
Too fast 8.6% 12.9% 1.4%
Total failure 8.5% 2.9% 2.9%
– with bump interference, weighted corner
– with bump interference, balanced
Clemson University
Experimental Validation
Bowl Scooper1 scooper2
Catch Miss Catch Miss Catch miss
Catch 0.0% 4.3% 0.0% 8.6% 1.4% 8.6%
Miss 2.9% 82.9% 4.3% 78.6% 2.9% 77.1%
Too fast 10.0% 8.6% 10.0%
Total failure 7.2% 12.9% 11.5%
Bowl Scooper1 scooper2
Catch Miss Catch Miss Catch miss
Catch 60.0% 7.1% 50.0% 5.7% 68.6% 8.6%
Miss 4.3% 21.4% 14.3% 30.0% 4.3% 11.4%
Too fast 7.1% 0.0% 7.1%
Total failure 11.4% 20.0% 12.9%
– with ramp interference, weighted corner
– with ramp interference, balanced
Clemson University
Third part:
A Novel Pneumatic Three-finger Robot Hand
Clemson University
Related work• Three different types of robot hands
– Electric motor powered hands, for example:• A. Ramos et. al. Goldfinger
• C. Lovchik et. al. The robonaut hand
• J. Butterfa et. al. DLR-Hand
• Barrett hand
– Pneumatically driven hands:• S. Jacobsen et. al. UTAH/M.I.T. hand
– Hydraulically driven hands:• D. Schmidt et. al. Hydraulically actuated finger
• Vision-based robot hand research– A. Morales et. al. presented a vision-based strategy for computing three-
finger grasp on unknown planar objects
– A. Hauck et. al. Determine 3D grasps on unknown, non-polyhedral objects using a parallel jaw gripper
Clemson University
Novel pneumatic hand
• Novel hand architecture– build-in pneumatic line in
Staubli RX130
– Paper tube, music steel wire embedded inside
– Camera mount adjusting “finger” spread angle
– 120 degrees between each other
• Disadvantages of current robot hands– Most robot hands are heavy
– Even with visual guidance, the robot hand can only grasp stationary objects
Clemson University
Novel pneumatic hand
• Close position Open position
• Our research here is to demonstrate that we use a novel idea to built a flexible end effector and it can grasp semi-randomly moving objects. This is not a new type of complex research tool-type robot hands.
Clemson University
Grasping research• Problem statement
robot
ball track
x
y
initial hand position final hand
position
Clemson University
Grasping research• Position prediction
– Same as the method in the second part work of this research
• Orientation adjustment– Line fitting to get the final “roll” angle
– equations
n
i
n
i ii
n
i
n
i
n
i iiii
n
i
n
i ii
n
i
n
i
n
i
n
i iiiii
xxn
yxyxnb
xxn
yxxxya
1 1
22
1 1 1
1 1
22
1 1 1 1
2
)(
)(
bxay
)arctan(b
Clemson University
Grasping experiments (video)
Clemson University
Conclusions: timing model
• A generic timing model for a robotic system using visual sensing, where the camera provides the desired position to the robot controller.
• We demonstrate how to obtain the values of the parameters in the model, using our camera network workcell as an example.
• Implementation to let our industrial manipulator intercept a moving object.
• Experimental results indicate that our model is highly effective, and generalizable.
Clemson University
Conclusions: dynamic manipulation
• Based on the timing model, we present a novel generic and simple theory to quantify the dynamic intercept ability of vision based robotic systems.
• We validate the theory by designing 15 sets of experiments (1050 runs), using two different end effectors under three different interference.
• The experimental results demonstrate that our theory is effective.
Clemson University
Conclusions: novel pneumatic hand
• A novel pneumatic three-finger hand is designed and demonstrated.
• It is simple, light and effective.
• Experimental results demonstrate that this novel pneumatic hand can grasp semi-randomly moving objects.
• Advantages– The compliance from pneumatics will allow the three-finger hand
to manipulate more delicate and fragile objects.
– In the experiments of grasping moving objects, unlike the traditional gripper, the contact position for this continuous finger is not very critical, which leaves more room for sensing error.
Clemson University
Sponsors
• The South Carolina Commission on Higher Education
• The Staubli Corporation
• The U.S. Office of Naval Research
Clemson University
Thanks
Clemson University
Conclusions: different manipulations
Manipulation
name
Basic robot motion
Impact time Success definition
Scoop Changing position and yaw orientation
Half way of scoop motion
Scoop the object out of table
Catch Changing position Finishing whole motion
Grab the object
Trap Changing position Half way of trap motion
Cover the object under the bowl
Ensnare Changing position and roll orientation
Finishing whole motion
Trap and then grasp the object
Clemson University
bump interference ramp interference
The distribution of |vi – vavg | in the balance ramp and bump cases.
Clemson University
Determining the Values
• An external camera to observe operation
• A conveyor moving in a fixed path at a constant velocity
• A light bulb as a tracking object
• A laser mounted in the end effector of the robot