Columbia

GraspIt!: A Versatile Simulator for Robotic Grasping

Andrew T. Miller

Columbia University

Columbia

Outline

• Motivation• Simulator Overview• Simulator Front-end• Grasp Analysis• Dynamic Simulation• Performing a Grasping Task• Future Work

Columbia

Simulation for Hand Design

• Mechanical prototypes are costly– Spend time and money

• Evaluate design in simulation– Examine range of motion, finger placement,

kinematics, link geometry– Ensure it can grasp desired objects– Change design easily

Columbia

Simulation for Grasp Planning

• Integrated grasp analysis– Grasp quality, weak point, force optimization

• Visualize grasp from all angles

• Perform many grasps quickly– Faster than using a real arm and hand

• Build a library of saved grasps– Recall grasp when object is encountered again

Columbia

• Library of hands and objects

• Intuitive interface allows many grasps to be tested quickly

• Visualize grasp wrench space

• Quality measures evaluate grasp

GraspIt: A Tool for Grasping Research

Columbia

GraspIt Components

Hand/ObjectConstruction

• read object model• read link models• read kinematics• assemble hand

UserInterface

• view 3D scene• change hand pose• auto-grip• manually move joints

ContactDetermination

• detect collisions• adjust contact to object surface• find contact area• add friction cones

GraspAnalysis

• compute grasp wrench space• use metrics on space• grasp force optimization

Wrench SpaceVisualization

• create 3D projections of GWS

Front end

Back end

ObjectDynamics

• compute object motion

Columbia

Hand Construction

• The hand kinematics (in D-H notation) specify the transforms between links, and can handle coupled joints.

• The 3D link geometries are accurately described in CAD model files.

• The format is flexible and can easily model many of the available complex articulated hands.

Columbia

Hand Configuration File… Header info and palm description ...

#-----------------------------------f0---------------------------------------#number of joints3

#number of links3

#Transforms from palm origin to base of the finger#EXAMPLE:#r 90 x (rotation by 90 degrees about x axis)#rr 3.0368729 y (rotation by 3.0368729 radians about y axis)#t 1.92188 134.5 39.0717 (translation in mm)t 25.0 0 -1.0r 180 y

#Joint Descriptions (1 joint per line)#DOFNum theta d a alpha DOFminval DOFmaxval#(joints are ordered from closest to palm outward)#(linear equations are of the form: q#*k+c [no spaces!])#(coupled joints must come after joints they are coupled to)d0+90 0 50 -90 0 180d1+5 0 70 0 0 144d1*0.333333333+40 0 55 0 0 144

#Link Descriptions (1 link per line)#filename material lastJoint#(links are ordered from closest to palm outward)#(lastJoint is the last joint in the chain which can affect this link)link1 plastic 0link2 plastic 1link3 plastic 2

#-----------------------------------f1---------------------------------------etc...

Columbia

Hand Library

Barrett Rutgers Robonaut

Parallel Jaw DLR

Columbia

User Interaction• Inventor 3D viewer• Rotate hand• Translate hand• Auto-grasp• Manipulate individual

DOF’s

• Simple and intuitive• Direct interaction gives

sense of hand’s capabilities

Columbia

Collision Detection

• PQP system (Larsen et al.)• Each body is a polygonal soup• Creates bounding volume hierarchies• Fast recursive algorithm tests for overlap, or

provides minimum distance

Columbia

Finding Contacts

• Binary search until link is < 0.1mm away• Record all triangle pairs within this distance• Compute points of overlap between each pair• Only keep contact points on region boundary• Mark points with red friction cones

Collision detected Search for instant of contact

Determine contact region

Columbia

Grasp Analysis

• Occurs when a contact is formed or broken• Computes space of forces and torques that can be

applied by the grasp• Quality measures numerically evaluate grasp• Provides a means to evaluate grasps

– Compare grasps of one hand, one object

– Compare grasps of many hands, one object

– Compare grasps of many hands, across a task specified object set.

Columbia

Wrench Spaces

• In 3-space, a wrench is a 6D vector composed of a force and a torque:

• The space of wrenches that may need to be applied during a task is the task wrench space.

• The space of wrenches that can be applied by a grasp is the grasp wrench space.

• A possible quality measure:

τ

fw

GWS

TWS

Columbia

Special Types of Grasps

• A force-closure grasp completely restrains the object.– Origin is contained within grasp wrench space.

• A manipulable grasp can impart arbitrary velocities on the object without breaking contact.

Columbia

Friction Cones

• Friction at a contact point allows forces in directions other than the contact normal

• COF, is determined by the contacting materials• Estimate friction cone as convex sum of a force

vectors on the boundary assuming a unit normal force, .

m

j

jj

1

ff 1f

Columbia

Contact Wrenches

• Each force acts at a position on object.

• Compute the corresponding wrench with respect to object’s center of gravity:

• - torque scale factor,

jii

jiji

,

,, fd

fw

max1 d

i: contact numberj: (1-8) around cone boundary

Columbia

• Objective: find total space of wrenches that can be applied by a grasp of unit magnitude.

• Grasp vector:

• Define with norm: – Sum magnitude of contact normal forces is 1.

• Compute grasp wrench space using qhull:

n

imiiL ConvexHullW

1,1, ,,

1ww

Grasp Wrench Space

1,21 gfffgT

n

1Lg

Columbia

Two Measures of Quality

• Assume task wrench space is unknown.

• Estimate with wrench space ball - good grasps will resist all wrenches equally well.

• Previously proposed measures of quality:– Radius, , of the largest wrench space ball that

can fit within the unit grasp wrench space.– Volume, v, of unit grasp wrench space.

Columbia

Grasp Wrench Space Projections

• To visualize the 6D grasp wrench space project it to 3-space by fixing three coordinates.

• A useful choice:– See the forces that can be applied without

applying a net torque (or vice versa).

Columbia

Visualizing the Results

Numeric quality &Worst case indicator

Wrench SpaceProjection

Columbia

Comparing Grasps

Columbia

Comparing Grasps

Columbia

Comparing Grasps

Columbia

Comparing Grasps

Columbia

Grasp Force Optimization

• Given a specific wrench to resist, find the optimal force to apply at contact points

• Forces should obey torque limit constraints and avoid friction cone boundaries

• Previously done by using linearized friction cones and linear programming [Kerr & Roth ’86]– overly conservative

0 extc wGf λNwGf extc

Columbia

Grasp Force Optimization

• GraspIt implements LMI technique by Han et al. ’00, • Does not require friction cone approximation• Feasible Set: • Minimize:

• Determinant Maximization with LMI constraints• Using hand Jacobian, it can also compute optimal

joint torque values

)(detlog)( 1 xxwx PT

),0)(,0)(|{ FGxxxx CPx m

Contact forces Inverse friction cone LMIWeighting factor

Columbia

Grasp Force Optimization

Columbia

Simulating Object Dynamics

• Added realism• Can examine grasp formation process• Formulate non-penetration and friction cone constraints as a

Linear Complementarity Problem [Stewart & Trinkle ’00]

• Lemke’s algorithm solves LCP to find contact forces that satisfy constraints.

0 bAfa

0f0af T force OR acceleration must be 0

Columbia

Simulating Object Dynamics

Columbia

Simulating Object Dynamics

Columbia

Connecting GraspIt to theReal World

Object PoseEstimation

FrameGrabber

GraspExecution

TaskMonitoring

Robot/HandControl

Presented atICRA ‘01

Columbia

Environment Modeling• Easy to import

environmental objects like workbench.

• Puma arm with inverse kinematics module also supplied.

• Prevents user from planning infeasible grasps.

Columbia

Real-Time Vision System

• Supply GraspIt with object pose.

• Needs to track object that is partially occluded by the hand.

• Since we have an object model, why not use model-based pose estimation and tracking?

• Single camera located 2m from workspace, calibrated wrt robot coordinate system.

Columbia

Real-Time Pose Estimationand Tracking

• Position estimation based on Dementhon and Davis.• Tracking based on Araujo and Brown’s extension of Lowe.

3D ProjectionMatrix

Fitting 3DModel

POSE

AffineMotion

NormalFlow

LocateVisible Edges

RenderModel

CameraParameters

ManualInitialization

Columbia

Tracking System Example

Wireframe modelSelected points are

used to determine pose

Pose estimated,and model overlaid

Normal flow estimation

Columbia

Grasp Execution Experiments

Pose estimated User plans grasp Grasp performed

Columbia

Grasp Execution Experiments

Pose estimated User plans grasp Grasp performed

Columbia

Grasp Execution Experiments

Pose estimated User plans grasp Grasp performed

Columbia

Grasping and Visual Servoing

Once pose has been estimated,grasp may be planned,

then executed.

Columbia

Grasping and Visual Servoing

Destination plannedin GraspIt.

Goal position isprojected into image.

Image based servoing drives robot to goal.

Columbia

Planning a Grasping Task in GraspIt

Columbia

Planning a Grasping Task in GraspIt

Columbia

Planning a Grasping Task in GraspIt

Columbia

Contributions• Grasping simulator

– Flexible method to specify hand construction– Interactive interface – Contact determination system– Inclusion of material properties

• Integrated grasp analysis theory into simulation system– GWS, quality measures, GFO

• Visualization techniques• Object Dynamics: First implementation of S-T algorithm

with large scale and interactive bodies• Integration of simulation system into grasping task• A useful tool for research in grasping and hand design:

– Currently being used by a number of research groups for design and synthesis: NASA, Sandia, Rutgers, U of MD

Columbia

Future Work

• Evaluate manipulable grasps• Provide method of specifying task wrench space• Human hand model• Full dynamic simulation• Automatic grasp selection

– Combine high-level and low-level synthesis techniques

– Learning techniques