Internship Presentation

Marius Muja

August 26, 2009

What’s been keeping me busy?

Milestone 2

Far outlet detectionDoor handle detection

Tabletop object detection

2D Chamfer matching3D Model fitting

Integrating FLANN in OpenCV

PR2 Challenge

Far outlet detection

Uses the stereo camera andbase laser to identify locationand 3D pose of power outlets

Outlet candidates obtained bysegmenting the disparity image

Wall pose computed form thebase laser scan

List of outlet candidates isfiltered using position, size andorientation priors

Far outlet detection

Outlet candidate patches arerectified to obtain a frontal view

Outlet identity is confirmedusing a template matchingalgorithm

Door handle detection

Initial detection using a boostedcascaded classifier of haarfeatures

False positives eliminated usingposition and size priors

Detections clustered acrossseveral frames for increasedrobustness

3D location of handle fromstereo

Tabletop object detection

Manipulating tabletop objects: one of the main tasks of arobot in a household environment

Setting/cleaning the tableServing drinks

Usual tabletop objects are difficult (no texture, transparent)

Techniques based on local features fail

Ikea objects dataset

2D Chamfer matching

Approach for finding the bestmatch of a contour model to anedge image

Contours can be strongindicator of object’s identity

2D Chamfer matching

3D model fitting

Grasping of delicate (glass)objects requires precise objectlocalization

Just a bounding box aroundthe object is not enoughWe must also know the grasplocation for a specific object

Extended the 2D chamfermatching approach to 3D

3D model fitting

Two stage approach

Bottom-up object localization

Determines probable objectlocationsTable plane detection andremovalPoint cloud clustering

Top down model fitting

Determines exact object poseand identityFind the model with the bestcorrespondence to the pointcloudICP-like (Iterative ClosestPoint) algorithm

3D model fitting

“tabletop objects” package

The 3D model fitter determines

Object identityObject poseGrasp poseObject mesh - used in theplaning stage for a moreprecise collision map

Integrated with the planingpipeline

PR2 challenge

Applied the 3D model fittingapproach for detectingjuice/watter bottles

Used the tilting laser pointcloud (much more sparse thanthe laser point cloud)

Model capture

Others

Integrated FLANN in OpenCV

FLANN - Fast Library for Approximate Nearest Neighborsfast nearest-neighbor searching in high dimensional spaces

PR2 teleoperation using a phone

based on Asterisk open source PBXcall the robot on the phone and send it to a specific officeother use cases possible: eg. deliver a message

Work in progress

Improving bottom-up part of object detection pipeline

3D features voting for object pose

3D object tracking

Neigborhood indexing for large point cloud structures

Sparse voxel grid indexingBenchmarks for the different index types

Thank you!