#DWwebinar

Innovative Trends in Robotics

#DWwebinar

Thank You To Our Sponsors

#DWwebinar

q  This webinar will be available afterwards at www.designworldonline.com & email

q  Q&A at the end of the presentation q  Hashtag for this webinar: #DWwebinar

Before We Start

#DWwebinar

Innovative Trends in Robotics Meet your speakers

Paul Heney Editorial Director – Moderator Design World Magazine

Ilia Baranov Senior Electrical Designer Clearpath Robotics

Mark Bünger VP Research Lux Research

Gary McMurray Principle Research Engineer & Division Chief Georgia Tech

#DWwebinar Clearpath Robotics, Inc. ©2015 #DWwebinar

Trends in Flexible R&D Robots

ABOUT CLEARPATH

Located in Kitchener, Canada, Clearpath Robotics develops self-driving vehicles for research and industrial applications. The company provides hardware, software and services to enable robotic development, deployment and operation. Clearpath is driven to automate the world’s dullest, dirtiest, and deadliest jobs.

#DWwebinar

ADAPTABLE TECHNOLOGY

ACCESSORIES ON-BOARD COMPUTER

#DWwebinar

HUSKY

MARKET NEEDS

Cost

Performance

Flexibility

Industry

Students

(Doesn’t yet exist!)

Research

#DWwebinar

RESEARCH TRENDS

HARDWARE TRENDS •  Lidar mapping •  Vision •  Depth sensors (Msoft, Apple, Intel) •  Computing is fast enough/$ to fit

into public facing robots •  General purpose hardware

SOFTWARE TRENDS •  ROS •  Open source •  Multi-Agent Systems •  Less structured spaces

#DWwebinar

RESEARCH TRENDS

RESEARCH TOPICS •  Vision •  Human-robot interaction •  Manipulation •  Unstructured path planning

#DWwebinar

DATABASE REPORT

References to “software” and “mobile robot” in IEEE Xplore database.

0

100

200

300

400

500

600

700

ROS C++ MATLAB Simulink

2009

2010

2011

2012

2013

2014

#DWwebinar

ACCESSORIES

Flexible platforms enable a diversity of payloads…

DEPTH SENSORS

GPS SOLUTIONS

HIGH PERFORMANCE CAMERAS

STEREO CAMERAS INERTIAL

MEASUREMENT UNITS

COMPUTING AND VISION PROCESSING

LASER RANGEFINDERS

#DWwebinar

R&D à COMMERCIAL

TRADITIONAL FACTORY VEHICLES (AMRs) INDUSTRIAL RESEARCH ADVANCED

MANUFACTURING

Research robotics play a critical role in the evolution of commercial applications.

#DWwebinar

#DWwebinar

Vision Guided Robotics:

We are Closer to the Revolution than You Think!

Gary McMurray

Associate Director for Industry, Institute for Robotics and Intelligent Machines, Georgia Tech

#DWwebinar

Introduction

•  Pick-and-place task

•  Robots moving from controlled settings to unstructured environments

• Robust object perception is crucial

#DWwebinar

Real-time 3D Model-based Tracking Using Edge and Keypoint Featuresfor Robotic Manipulation

Changhyun Choi and Henrik I. ChristensenRobotics & Intelligent Machines, College of Computing

Georgia Institute of TechnologyAtlanta, GA 30332, USA

{cchoi,hic}@cc.gatech.edu

Abstract— We propose a combined approach for 3D real-timeobject recognition and tracking, which is directly applicable torobotic manipulation. We use keypoints features for the initialpose estimation. This pose estimate serves as an initial estimatefor edge-based tracking. The combination of these two comple-mentary methods provides an efficient and robust tracking so-lution. The main contributions of this paper includes: 1) Whilemost of the RAPiD style tracking methods have used simplifiedCAD models or at least manually well designed models, oursystem can handle any form of polygon mesh model. To achievethe generality of object shapes, salient edges are automaticallyidentified during an offline stage. Dull edges usually invisible inimages are maintained as well for the cases when they constitutethe object boundaries. 2) Our system provides a fully automaticrecognition and tracking solution, unlike most of the previousedge-based tracking that require a manual pose initializationscheme. Since the edge-based tracking sometimes drift becauseof edge ambiguity, the proposed system monitors the trackingresults and occasionally re-initialize when the tracking resultsare inconsistent. Experimental results demonstrate our system’sefficiency as well as robustness.

I. INTRODUCTION

As robots moves from industrial to daily environments, themost important problem robots face is to recognize objectsand estimate 6-DOF pose parameters in less constrainedenvironments. For the last decade, computer vision, robotics,and augmented reality have all addressed this as a model-based tracking issue. Most of the work has been based on 3DCAD models or keypoint metric models. The former modelscorrespond to edges in an image, which can be efficientlycomputed, while the latter models match with keypoints in animage which are suitable for robust wide baseline matching.A strategy for using keypoint for pose initialization anddifferential methods for pose tracking is presented.

II. RELATED WORK

For the 6-DOF pose tracking, robotics and augmented re-ality areas have employed a number of different approaches.One of the easiest way is through use of fiducial markers.Artificial markers are attached to the object or environmentas camera targets. Although the method provides an easyand robust solution for real-time pose estimation, attachingmarkers has been regarded as a major limitation. Hence,researchers have focused on tracking using natural features.For several decades methods, which employ natural fea-tures, have been proposed: edge-based, optical flow-based,

ImageAcquisition

ModelRendering

EdgeDetection

Pose Update

with IRLS

Error Calculation

CAD Model

Keyframes

KeypointMatching

Pose Estimation

!"#$%&'()*$+, -+)".+'/),$'0,12.1)34)%.+'/),$'0,12.1)3

()3)%5+.6'7.2$6.

Fig. 1: Overall system flow. We use a monocular camera. Theinitial pose of the object is estimated by using the SURF keypointmatching in the Global Pose Estimation (GPE). Using the initialpose, the Local Pose Estimation (LPE) consecutively estimatesposes of the object utilizing RAPiD style tracking. keyframesand CAD model are employed as models by the GPE and LPE,respectively. The model are generated offline.

template-based, and keypoint-based. Each method has itsown pros and cons, but surveying every methods in thispaper is out of scope. For an in-depth study of the differentmethods, we refer the interested reader to the survey [1].

Among the various methods, we focus on two methods:edge-based and keypoint-based. The edge features are easy tocompute and computationally cheap. Since the edge is usu-ally computed by image gradients, it is moderately invariantto illumination and viewpoint. The keypoint features are alsocapable of being invariant to illumination, orientation, scale,and partially viewpoint. But the keypoints requires relativelycomputationally expensive descriptors which maintain localtexture or orientation information around stable points to bedistinctive.

In edge-based methods, a 3D CAD model is usuallyemployed to estimate the full pose using a monocular cam-era. Harris [2] established RAPiD (Real-time Attitude andPosition Determination) which was one of the first marker-less 3D model-based real-time tracking system. It tracks anobject by comparing projected CAD model edges to edgesdetected in a gray-scale image. To project the model close

Overview

2D Monocular > Combining Keypoint and Edge Features

#DWwebinar

Videos

•  Oneofthemostimportantproblemistorecognizeobjectsandes6mate6-DOFposeparameters.

•  Weadoptacombinedapproachinwhichkeypoint-basedmatchingandedge-basedtrackingareemployed.

•  Oursystemcanrobustly(re-)localizeanobjectandefficientlytracktheobject.

#DWwebinar

Robotic Assembly

#DWwebinar

Uncalibrated Visual Servoing•  Plant: Robot (Unmodeled) •  Feedback: Vision System

(Uncalibrated) •  Target: Unmodeled •  Controller: Quasi-Newton

Optimization with Jacobian Estimation

or

#DWwebinar

Examples of Visual Servoing

#DWwebinar

Conclusion

•  Ability of robots to work in unstructured environments has dramatically increased in recent years

•  Vision guided robotics is key enabling technology•  Demonstrated two approached to accomplishing this

•  Based on availability of CAD model•  Based on identification of target using vision

•  The application of this technology will revolutionize the manufacturing and service sectors as robots will be able to work in environments without significant alternations (jigs, fixtures, hard stops, special tooling, etc.).

#DWwebinar

Overview •  Robots have long been hard, in both senses of difficult to make, and of painful when

they hit you. But now, a growing crowd of material, electronics, software, and design innovations is bringing the new paradigm of soft robotics to market.

•  Startups, corporations, and venture investors are working to build and deploy robots that are lighter, safer, smaller, and cheaper – without sacrificing precision or strength.

•  Biomimicry, novel polymers, advanced textiles, and flexible electronics are some of the key enabling material technologies, while artificial intelligence, machine vision, and computer-aided design complement them; investment and advances are needed in each.

•  Users in industry are looking to deploy soft robotics in manufacturing, medicine, agriculture, logistics, and many other fields where hard robots face shortcomings in weight, safety, cost, and other respects.

22

#DWwebinar Source: Lux Research, “Autonomous Systems 2.0: A Taxonomy” June 15, 2014

Soft Robotics in an

overall taxonomy

#DWwebinar

Robotics experts

Applications and users

Advanced materials

developers

24

Soft Robotics – a confluence of three communities

Softroboticsishere

“The best way to predict the future is to invent it.” - Alan Kay, Xerox PARC

#DWwebinar

Robobusiness Panel “New Materials for Building Robots” Sept 23, 2015

•  Bob Christopher o  Director, Innovation o  Panasonic R&D Company of America

•  John Suh o  VP - Head of Office o  Hyundai Motor Company

•  Matthieu Repellin o  Senior Manager - Corporate Development o  Airbus Group

•  Mark Bünger o  VP of Research o  Lux Research, Inc.

25

#DWwebinar

Soft Robotics Foresight Workshop June 10, 2015

26

(now a LinkedIn group)

#DWwebinar

SRI – legendary source of robotics innovation spins out robotics startups

27

#DWwebinar

Grabit: Electroadhesion grippers for material handling

•  Electroadhesion pads comprised of printed conductive traces on flexible substrate with a thin protective coating

•  Operates at 1 kV to 3 kV, but with low power and current, using a step-up transformer that comprises most of its current cost structure

•  Recently raised $6 million series A, which included strategic investment from ABB and Nike

•  Needs to continue to verify product safety and compatibility with sensitive goods

•  Looking to get into gripper, conveyer belt, and 3D part handing applications

•  Electroadhesion offers advantages over competitive vacuum and mechanical approaches; clients should seek partnership opportunities for materials, contract manufacturing, or for use in their own plants

28

#DWwebinar

Soft Robotics (company name)

squishy grippers for unexpected shapes

29

Location: Cambridge, MA

Founded: 2012Stage: IntroductionEmployees: 5Partnerships:•  None announced•  Revenue: $150,000 (estimate)•  Cash: $500,000 (estimate)

  Technology

  Soft, flexible robotic grippers based on the soft robotic actuators developed by the Whitesides Research Group at Harvard University

  Company is able to design and build customized grippers within seven business days; currently demo kits cost $15,000 which is inclusive of both the gripper and the control system

  Grippers offer a long life cycle – upwards of 10 million cycles

  Lux Take: Positive

  Control system that does not require vision libraries but rather uses a stereo-vision camera that calculates the center of mass of a given object

  System currently offers a valuable solution to pick and place tasks but may still be limited in bin picking applications

#DWwebinar

Pneubotics

textile structures with gas/fluid-pressure actuation •  Textile, pneumatic robots •  Own words: “A new class of all-fluidic,

membrane based robotics that are entirely constructed out of compliant skins and filled with pressurized fluids to create structure and movement.”

•  Safe to operate near humans •  New hardware-software integration •  Reduces cost and time to design and build •  Greatly increases the number of robot

developers and applications

30

#DWwebinar

Exoskeletons -

the augmented reality of robots

31

#DWwebinar

Soft robotics science keeps advancing

32

#DWwebinar

Vytas SunSpiral, NASA Ames

Dynamic Tensegrity Robotics Project

•  Robots use both rigid and soft materials to absorb forces and traverse uneven/unknown surfaces

•  Building and testing at several scales from small to large •  Complex software simulator – available via NASA Tensegrity

Robotics Toolkit (https://github.com/NASA-Tensegrity-Robotics-Toolkit/NTRTsim/releases)

•  http://ti.arc.nasa.gov/tech/asr/intelligent-robotics/tensegrity/

33

#DWwebinar

Science today, commercial tomorrow:

Bio-inspired connected materials and soft machines •  The convergence of matter and information will continue even when IoT is everyday •  Smart materials, devices and systems being developed today will bring the next wave:

•  Computing composites – intelligent polymers •  Chemical computing •  Tangible interfaces •  4D printing (3D printed objects that change with time) and 5D printing (voxel

manufacturing) •  Soft robotics •  Hybrid bio/artificial systems

34

A tissue-engineered jellyfish with biomimetic propulsionhdp://www.nature.com/nbt/journal/v30/n8/full/nbt.2269.html

Development of Miniaturized Walking Biological Machineshdp://www.nature.com/srep/2012/121115/srep00857/full/srep00857.html

Soft robots in DARPA’s Maximum Mobility and Manipulation (M3) program use microfluidics and chemofluorescence for motion and camouflage

Active materials by four-dimension printing

hdp://scitation.aip.org/content/aip/journal/apl/103/13/10.1063/1.4819837

#DWwebinar

Innovative Trends in Robotics Questions?

Paul Heney Editorial Director – Moderator Design World Magazine pheney@wtwhmedia.com

Ilia Baranov Senior Electrical Designer Clearpath Robotics ibaranov@clearpathrobotics.com

Mark Bünger VP Research Lux Research Mark.bunger@luxresearchinc.com

Gary McMurray Principle Research Engineer & Division Chief Georgia Tech Gary.mcmurray@gtri-gatech.edu

#DWwebinar

Thank You q  This webinar will be available at

designworldonline.com & email

q  Tweet with hashtag #DWwebinar

q  Connect with Design World

q  Discuss this on EngineeringExchange.com