Baxter Robots are Learning to Think, See, and Cookbengfort/papers/baxter-robots-learning.pdf ·...

Baxter Robots are Learningto Think, See, and CookBenjamin Bengfort, Victoria Cepeda, Huijing Gong, Nicholas Labich, &Mahmoud Sayed University of Maryland, College Park

Thursday, February 26, 2015

Baxters are low cost humanoid robotsmeant for adaptable manufacturingpurposes by using long nimble arms

and a suite of visual and tactile sen-sors. At the University of Maryland, re-searchers are exploring computer vision,machine learning, and artificial intelli-gence by training the Baxters to pour wa-ter into a moving jar, learn to cook bywatching YouTube, and work with otherrobots.

Introduction

In April 2014, researchers at the University ofMaryland acquired two Baxter Intera 3 humanoidrobots manufactured by Rethink Robotics [1].Baxter robots are well known for their use inmicro-manufacturing as they are specially de-signed for repeated grasping and moving tasksin a stationary position. While a Baxter’s multi-dimensional range of motion is important, whatmakes them especially interesting is their adapt-ability. Baxter robots are trained not pro-grammed on work sites. Human workers movethe Baxters incredibly flexible robotic arms toshow Baxter how to do a particular task. Thisability, combined with Baxter’s powerful onboardprocessing, a suite of sensors, and smooth operat-ing mechanisms, makes Baxter ideal for research.

Figure 1: Baxter stirs a bowl

In the Computer Science department, researchgroups have already begun to explore computervision, machine learning, and artificial intelligencewithin a robotic context. Three research groupsthe Computer Vision Labratory (CFAR), theMetacognitive Lab (MCL), and the MarylandRobotics Center have been cooperating to pro-duce practical results from their individual theo-retical fields. Simple tasks like picking up utensilsand stirring bowls, pouring water in a movingcontainer, and building structures from blockshave been quickly achieved thanks to this col-laboration. By studying the difficulties involvedin teaching Baxter to perform these tasks, theresearch groups hope to solve larger theoretical

of 3

Figure 2: UMD researchers apply vision and AI research to practical robots.

challenges.

MetacognitionThe primary research goal of the MetacognitionLab (MCL) at University of Maryland is to createan enduring artificial agent: one which persistsover time, learns from its mistakes, interacts withthe environment, and constantly acquires newknowledge. An agent of this kind specificallyrequires metacognition thinking about think-ing to analyze its own cognitive processes andadapt how it thinks to prevent future failures.It primarily does this by attempting to under-stand anomalies in reasoning, asking itself whatwent wrong?. An enduring agent requires thearchitectural integration or unification of severalcomponents including vision, language process-ing, reasoning, a knowledge base, goals, and amethod of interacting with the world. Using theBaxter platform, the MCL group hopes to puttogether multi-agent or mixed initiative systems.

Professor Don Perlis and MCL are pursuingmore direct artificial intelligence research and us-ing the Baxter as a physical realization of theirwork. One example is the BaxnBuzz project,

which hopes to create a multi-agent system bypairingd the immobile Baxter with a Parrot ARquadcopter. The quadcopter will be used to ex-tend Baxters line of sight and help the Baxtersolve problems such as locating a dropped item.The group is also researching mixed-initiative sys-tems that have both a human and the Baxterrobot participating in a single scenario. For exam-ple, the group is currently exploring a construc-tion domain where a human foreman instructs theBaxter using natural language to build a struc-ture using blocks or other building materials.

Computer VisionHuman beings, when learning to cook, maywatch others with cooking experience and mimictheir actions. With the advent of YouTube andother video-sharing websites, there are significantamounts of training data available; perhaps arobot can learn to cook by watching YouTubevideos and mimicking actions. A group led byProfessor Yiannis Aloimonos has developed avisual learning system to teach a Baxter robotto cook by watching YouTube [3]. The team ap-proached the problem from two levels: first, using

of 3

Figure 3: Baxter pours water into a moving jar

a low-level visual system which recognizes objects,hands, and types of manipulations from video,and second, a high-level action system whichtranslates the information from the video intoactions to be performed by the robot. This sys-tem allowed them to use unconstrained YouTubevideos as their data source. Much like the pushin virtual reality to be able to generate renderedmodels of real-world environments, the ability totrain intelligent systems with real-world sensoryinput is an important goal for artificial intelli-gence research.

Dynamic Pouring

Professor James Reggia and his group in theMaryland Robotics Center focus on developingimitation learning approaches for motion. In adynamic fluid pouring task, humans train a Bax-ter robot to learn by observing errors made byhumans and modeling how humans recover fromfailed action [2]. Using a supervised model offailure-to-success transitions and imitation learn-ing, the group showed that a robot can success-fully perform the same task under different vari-ations and conditions without a sophisticatedplanning system. By restricting a set of taskparameters, such as pouring speed, the imple-mentation of their imitation learning approach isable to succeed at this dynamic pouring task.

ConclusionThe collaboration of vision, robotics, and AI withpractical applications using the Baxter robot isallowing research groups to understand visualscenes more thoroughly and to act upon it. TheBaxter’s unique hardware and programming in-terface has provided smaller startup costs andreduced the barrier of entry for groups who tra-ditionally don’t use robotics platforms in theirwork. Because of this work, we expect to seerobotic behaviors that mimic and learn from hu-mans to do chores or follow directions withoutexternal programming. The potential for robotsto learn much faster and at a much lower cost,and to share that knowledge with other robots, isa significant step towards developing technologiesthat could have benefits in areas such as militaryrepair and logistics, smart manufacturing environ-ments, and completely automated warehouses.

For videos showing the Baxter at work, pleasesee: http://ter.ps/8i9 and http://ter.ps/8ia.

References[1] C. Fitzgerald. Developing baxter. In

Technologies for Practical Robot Applications(TePRA), 2013 IEEE International Confer-ence on, pages 1–6. IEEE, 2013.

[2] Joshua D. Langsfeld, Krishnanand N. Kaipa,Rodolphe J. Gentili, James A. Reggia, andSatyandra K. Gupta. Incorporating Failure-to-Success Transitions in Imitation Learningfor a Dynamic Pouring Task. In Workshopon Compliant Manipulation: Challenges andControl, Chicago, IL, September 2014.

[3] Yezhou Yang, Yi Li, and Yiannis Aloimonos.Robot Learning Manipulation Action Plansby Watching Unconstrained Videos from theWorld Wide Web. Under Review, 2015.

ContributorsBenjamin Bengfort ([email protected])Huijing Gong ([email protected])Nicholas Labich ([email protected])Mahmoud Sayed ([email protected])Victoria Cepeda ([email protected])

of 3

Baxter Robots are Learning to Think, See, and Cookbengfort/papers/baxter-robots-learning.pdf ·...

Documents

Transcript of Baxter Robots are Learning to Think, See, and Cookbengfort/papers/baxter-robots-learning.pdf ·...