Preface

- Edward Gibbon

My motivation for writing this book grew out of a perceivedneed for an . integratedexplanatorytext on the subject This perceptioncame from the Robots I havehadin finding a goodbookfor a coursein Autonomous difficulty I havebeenteachingfor the pastten yearsor so. Although an extensive body of literaturehas beenproducedon behaviorbasedrobots the lack of sucha , to text madeit hardto introducestudents this field without throwing theminto technicalliterature generallymakingit accessible , only to advanced fairly deep students Thoughthere are severalgood books of collectedoriginal . graduate . as , papers they provedto be only partially adequate an introduction and This book' s intended audienceincludes upperlevel undergraduates studentsstudying artificial intellegence(AI ) and robotics as well , graduate the . in as thoseinterested learningmore aboutroboticsin general It assumes in college level artificial intelligence The text . a ability to comprehend course robotics or to could be usedto supporta coursein AI -basedor autonomous . a generalAI course supplement . , , Acknowledgmentsof course area necessityBeing a religiousman, I' d first like to thank God andJesusChrist for enablingme to completethis long and \\10biblical passages haveinspired me as a roboticist can that arduoustask. 1 be found in Matthew3:9 andLuke 19:37- 40. in generated My family hasbeenmostgracious putting up with the hardships . My wife, to my amazement , proofreadthe entirework. Sheand by authorship and by my four childrenput up with th~ inevitableabsences strainscaused this . . time-sappingprocessI love themdearlyfor their support

Preface

Earlier tutorialson behaviorbased robotics presented , initially by myself (at the 1991IEEE InternationalConference Systems Man, and Cybernetics on , ) and later with Rod Grupen(at the 1993lntemationalConference Robotics on and Automation and the 1993 InternationalJoint Conferenceon Artificial , . ) Intelligence helpedcoalesce manyof the ideascontainedherein I thank Rod for working with me on these . A large number of studentsat Georgia Tech over many years have contributed in a wide rangeof invaluableways. I ' d expresslylike to thank Robin , Murphy, Doug MacKenzie Thcker Balch, Khaled Ali , Zhong Chen Russ , Clark, ElizabethNitz, David Hobbs WarrenGardner William Carter Gary , , , Boone Michael PearceJuanCarlosSantamariaDavid Cardoze Bill Wester , , , , , Keith Ward David Vaughn Mark Pearson and others who have madethis , , , book possible I ' d alsolike to thankBrandonRhodes pointing out the short . for story that leadsinto chapter10. Interactionswith manyof my professional in at colleagues residence Georgia Tech aswell asvisitors, havealsohelpedto generate the ideasfound in of , this text. AmongthemareProf. ChrisAtkeson Dr. Jonathan CameronDr. Tom , , Collins, Prof. Ashok Ooel, Prof. Jessica , Hodgins Dr. Daryl Lawton, Dr. John Pani Prof. T. M . Rao, andProf. Ashwin Ram. My thanksto y ' all. , This book would never have been possiblewithout the researchfunding . ' eachof supportthat camefrom a variety of sources I d like to acknowledge theseagencies the cognizantfunding agent the National Science and : Foundation Khosla the Office (HowardMoraff), DARPA (Eric Mettala andPradeep ), of Naval Research Teresa McMullen), andthe Westing houseSavannah River ( Center(Clyde Ward . ) Technology The folks at MIT Presshave been great to work with from the book' s . , inception in particularthe late Harry Stantonand Jerry Weinstein I am also indebtedto Prof. Michael Arbib for so graciouslycontributingthe forewordto this book. Specialthanksgo to Jim Hendierfor piloting a draft versionof the book for a courseat the University of Maryland. Finally, it is impossibleto list all the people within the greaterresearch , support and insight , community who have contributedwith encouragement into this endeavorTo all of you I am deeplyindebtedand can only hopethat . this text serves someway ascompensation thoseefforts. in for

Foreword Michael Arbib

Had he turnedfrom politics to robotics John FitzgeraldKennedymight well , have said "Ask not what robotics can do for you, ask what you can do for , robotics" Ronald Arkin has a more balancedview, however for his book . , makesus vividly awarethat the interplaybetweenroboticsanda host of other . richly andfruitfully in both directions disciplinesproceeds " " Theemphasis on the" brains of robotsratherthanon their " bodies - thus is " which movesthe detailsof robot sensors " BehaviorBased and thetitle robots , to actuators , firmly into the backgroundmakingthem secondary the main aim what behaviorswe shouldexpectrobotsto exhibit, of the book: to understand . can and which computationalmechanisms serveto achievethesebehaviors featuresof the book that contributegreatly to its However I shouldnote two , of liveliness a fine selectionof epigramsanda superbgallery of photographs : robotsfrom all aroundthe world. Thus evenwhile the book focuseson robot . to brains we havemanyopportunities admirehumanwit androbot bodies , RonaldArkin hasalwaysbeenfascinated parallelsbetweenanimals(including by herein his marshalingof . humans and robots This is fully expressed ) datafrom biology and psychologyto showthat much of the behaviorof animals from a in andthusof robotscanbe understood termsof patterns emerging . basicsetof reactivemodules However Arkin also showsus that manybehaviors , of canbe achieved only by detailedanalysisof representations the world, 's -term experienceWe are . andof the animal placewithin it , built up from long that thusofferedan insightful analysisof robot architectures placesthemalong . a continuumfrom reactiveto deliberative a powerful frameworkexploitedhereto createbehaviors Schema theory provides that are distributed in their control structures integrativeof action , can . and perception and open to learning Theseschemas in turn be implemented , conventionalcomputerprograms finite stateacceptors neural , , using

Foreword networks or genetic algorithms In building our appreciationof this framework . , , Arkin usesroboticsto illuminate basic issuesin computerscienceand artificial intelligence andto feednewinsightsbackinto our readingof biology , andpsychology . The book' s chapteron social behaviorpresentswhat is itself a relatively recentchapterof robotics- socioroboticsWhile in its infancy we can seein . , the studiesof robot teams inter-robot communication and social lerning the , , , beginningsnot only of a powerful new technology but also of a new science of experimental . sociology Finally, we are takento that meetingplacebetweensciencefiction, philos, ophy and technologythat attractedmany of us to wonderaboutrobotsin the first place The final chapter " Fringe Robotics BeyondBehavior (a nod to . : " , ' " the 1960s British review " Beyondthe Fringe ?), debates issuesof robot the ' , , emotion and imagination returns to Arkin s longstanding , , thought consciousness concernwith the possibleutility to robots of analogsof hormones andhomeostasis closeswith an all too brief glimpseof nanotechnology . , and In this way we aregiven a tour that impress with the depthof its analysis es of the schemas , underlying robot behavior while continually illustrating the betweenrobotics and biology, psychology sociology and , , deep reciprocity betweenrobotics and many other , philosophy and the important connections areas computerscienceThis is a subjectwhosefascination only increase of . can in the decades aheadas many researchers build on the framework so ably here presented .

Chapter Whence

1 Behavior ?

Chapte Object 1thecon .unders Tospec what robot are .deve intellig 2andof a review recent that ledb tome the beh histor robotic . the.system .learn 3apprec To of wide robo1.1 TOWARD ROBOTS INTELLIGENTthe : Perhaps bestway to begin our study is with a question If we could create robots what shouldthey be like, and what shouldthey be able to , intelligent do? Answeringthe first part of this question " What shouldthey be like?" 's ) requiresa descriptionof both the robot physical structure(appearanceand its performance of the question " What , (behavior. However the second ) part should they be able to do?" - frames the answerfor the first part. Robots that need to move objects must be able to grasp them; robots that have to traverserugged outdoor terrain need locomotion systemscapableof moving in adverseconditions robots that must function at night need sensors ; of operatingunder thoseconditions A guiding principle in robotic . capable the , , design whetherstructuralor behavioral involvesunderstanding environment within which the robot operatesand the task(s) it is required to undertake . This ecologicalapproach in which the robot' s goals and surroundings , influenceits design will be a recurring theme throughoutthis , heavily book. But what is a robot? Accordingto the RoboticsIndustryAssociation(RIA ), " a robot is a reto , programmablemultifunctional , manipulator designed move material parts tools, or specializeddevicesthrough variable programmed , , " motionsfor the performance a varietyof tasks (JablonskiandPosey1985 . of )

Chapterl

(A) Figure1.1 robots U musician of Anthropomorphic . (A) WAS BaT, a keyboard capable reading music (B) WlnrI , a robotthat walkedin excess 65 kin duringoneexhibition . of . of of .) ( photographs courtesy AtsuoTakanishi Waseda University This definition is quite restrictive excluding mobile robots among other , , ' robotics as the intelligent , things. On the other exb eme anotherdefinition describes connectionof perceptionto action ( Brady 1985 . This seems ) overly inclusive but doesacknowledge necessary the relationshipbetweentheseessential of . ingredients robotic systems In anycase our working definition will be: An intelligentrobot is a machine , able to extractinformation from its environmentand useknowledgeaboutits world to move safely in a meaningfuland purposivemanner Hollywood has .

? WhenceBehavior

(B) ) Figure 1.1 (continued

creaturesfashionedin the image often depictedrobots as anthropomorphic . two arms a torso, and a head Indeedrobots have of man having two legs , , , : e structure Figure 1.1 illustrates actually been createdthat have a hu~ . two such robots Robots have often been modeledafter animals other than and are commercially . humans however Insectlike robots are now commonplace , otherslook more like horses spiders or octopi, as figure 1.2 , available , ; . shows , Robotsalso often look like vehiclescapableof operatingon the ground in of classes . the air, or underseasThe examplesshownin figure 1.3 represent aerial : vehicles UAV (unmanned referredto as unmanned robots generically vehicle UGV (unmannedground vehicle and UUV (unmannedundersea ), ), vehicle . )

5

Figure 1.2

? WhenceBehavior

(C) ) Figure1.2(continued of robot (Photograph . crablike . Animallike robots (A) Ariel, a hexapod courtesy IS and robot , Robotics , , MA. ) , Somerville ) ( B Quadruped thattrots paces boundsbuilt at . . (Q MarcRaibert . in theCMUleglaboratory 1984(Photograph Jack Bingham 1992 by of robot .) All rightsreserved(C) HEGI060 courtesy California Hexapod . (Photograph Co. , CA. Cybernetics , Thjunga ) Robotscanbe differentiatedin termsof their size the materialsfrom which , aremade the way they arejoined together the actuators , they use(motors , they andtransmissions the typesof sensing , their locomotion theypossess ), systems . , system andtheir onboardcomputersystemsBut a physicalstructureis clearly . not enough Robotsmustbe animate so they musthavean underlyingcontrol , to providethe ability to movein a coordinated . This book focuses way system of roboticsandthe designof control and on theperformance behavioral aspects that systems allow themto performthe way we would like. The physicaldesign : Many goodsources of robotsis not addressed alreadycoverthat material(e.g., , ) Craig 1989 McKerrow 1991 .

Chapter1

(A)

(B ) 1.3 Figure

? WhenceRehavinr

(C) ) Figure 1. 3 (continued Search vehicle: the Advanced Unmanned vehicles (A ) Unmanned . undersea Unmanned lanceandSecurity : aerialvehicle MultipurposeSurveil ) System(AUSS . ( B) Unmanned : Mission Platform (MSSMP . (C) UnmannedGround Vehicle Ground Surveillance ) of U.S. Navy.) Robot (GSR . ( Photographs ) courtesy How do we realize the goal of intelligent robotic behavior? What basic science and technology is needed to achieve this goal ? This book attempts to answer these questions by studying the basis and organization of behavior and the related roles of knowledge and perception , learning and adaptation, and teamwork .

1.2 PRECURSORSPeoplethat are really weird can get into sensitivepositions and have a tremendous impacton history. - J. DanforthQuayle

ChapterI To inventyou needa goodimaginationanda pile of junk . - ThomasAlva Edison

The significanthistory associated with the origins of modembehaviorbased roboticsis importantin understanding appreciating currentstateof the and the art. We now review important historical developments threerelatedareas in : . , , cyberneticsartificial intelligence androbotics

1.2.1

Cybernetics

Norbert Wiener is generallycreditedwith leading in the late 1940s the development , , of cybernetics a marriageof control theory information science : , , and biology that seeksto explain the commonprinciplesof control and communication in both animalsand machines( Wiener1948. Ashby ( 1952 and ) ) Wiener furtheredthis view of an organismas a machineby using the mathematics for . developed feedbackcontrol systemsto expressnatural behavior This affirmed the notion of situatednessthat is, a strong two-way coupling , betweenan organismandits environment . In 1953 W. Grey Walter applied theseprinciples in the creationof a precursor , robotic design termedMachina Speculatrix which was subsequently ' transformedinto hardwareform as Grey Walters tortoise Someof the principles . that werecapturedin his designinclude: 1. Parsimony Simple is better Simple reflexes can serveas the basisfor behavior : . . "The variationsof behaviorpatterns exhibitedevenwith sucheconomy " of structurearecomplexandunpredictable ( Walter 1953 p. 126 . , ) 2. Exploration or speculation The systemnever remainsstill except when : undernonnal circumstances ) feeding(recharging. This constantmotion is adequate to keepit from being trapped " In its explorationof any ordinaryroom . it inevitablyencounters ; manyobstaclesbut apartfrom stairsandfur rugs, there arefew situationsfrom which it cannotextricateitself ' ( Walker 1953 p. 126 . , ) 3. Attraction (positive tropism : The systemis motivatedto move towards ) someenvironmental of , object. In thecase thetortoise this is a light of moderate . intensity 4. Aversion(negativetropism : The systemmovesawayfrom certainnegative ) stimuli, for example avoidingheavyobstacles slopes and . , 5. Discernment The systemhasthe ability to distinguishbetweenproductive : andunproductive behavior adaptingitself to the situationat hand . ,

WhenceBehavior ?

. .

Figure 1.4 Circuit of MachinaSpeculatrix ( FromTheliving Brain by W. Grey Walter Copyright . . 1953@ 1963and renewed 1981 1991by W. Grey Walter Reprintedby permission @ . , of W. W. Norton andCompanyInc.) ,

The tortoise itself, constructedas an analogdevice (figure 1.4), consisted of two sensorstwo actuators and two " nervecells" or vacuumtubes A directional . , , for detectinglight and a bumpcontactsensor photocell providedthe environmental feedback One motor steered single front driving . the requisite wheel The photocell alwayspointed in the direction of this wheel and thus . could scanthe environment The driving motor poweredthe wheel and provided . locomotion . The tortoiseexhibitedthe following behaviors : . Seeking light : The sensorrotated until a weak light sourcewas detected while thedrive motorcontinuouslymovedtherobot to explorethe environment at the sametime. . Head toward weak light : Once a weak light was detected the tortoise , movedin its direction. . Back away from bright light : An aversivebehaviorrepelledthe tortoise from bright light sourcesThis behaviorwasusedto particularadvantage . when the tortoisewasrecharging . . Thm and push: Used to avoid obstacles this behavioroverrodethe light , . response . Rechargebattery : When the onboardbattery power was low, the tortoise the a perceived stronglight sourceas weak. Because rechargingstationhad a

I

1

. \

. 1 . ' : . .

Chapter1

Figure 1. 5 ' , Grey Walter s tortoise recently restoredto working order by Owen Holland., ( photo . graphcourtesyof OwenHolland, The University of the Westof England)

. , strong light over it , the robot movedtoward it and docked After recharging the light sourcewas perceivedas strong and the robot was repelledfrom the , . rechargingstation The behaviorswere prioritized from lowest to highest order: seekinglight, moveto/from light, andavoid obstacleThe tortoisealwaysactedon the highest . over , priority behaviorapplicable for examplechoosingto avoid obstacles toward a light . Behaviorbasedroboticsstill usesthis basicprinciple, moving referredto as an arbitration coordinationmechanism section3.4.3), widely. ( Walter' s tortoiseexhibitedmoderately : complexbehavior moving safelyabout a room and rechargingitself as needed(figure 1.5). One recent architecture ' in (Agah andBekey 1997 described section9.8.3, employsWalter s ideason ), positiveand negativetropismsas a basisfor creatingadaptivebehaviorbased robotic systems . Valentino Braitenberg revived this tradition three decadesafter Walter , ) (Braitenberg1984 . Taking the vantagepoint of a psychologist he extended the principles of analogcircuit behaviorto a seriesof gedankenexperiments . usedinhibitory involving the designof a collection of vehicles Thesesystems

? WhenceBehavior

. to and excitatoryinfluences directly coupling the sensors the motors As before , , seemingly complexbehaviorresultedfrom relativelysimplesensorimotor createda wide rangeof vehicles including those . transformationsBraitenberg , to exhibit cowardice aggressionandevenlove (figure 1.6). As with , , imagined Walter' s tortoise thesesystemsare inflexible, custommachinesand are not , . , reprogrammableNonethelessthe variability of their overt behavioris com pelling. , , Eventually scientistscreatedBraitenbergcreaturesthat were true robots ' at . not merely thoughtexperimentsIn one sucheffort, scientists MIT s Media Lab ( Hogg Martin, and Resnick 1991 usedspeciallymodified LEGO bricks ) , ' creaturevehiclesusing Braitenbergs principles to build twelve autonomous , a timid shadowseekeran indecisiveshadowedgefinder, a paranoid , including shadowfearing robot, a doggedobstacleavoider an insecurewall follower, , havebeenassembled seekerEven morecomplexcreatures . and a driven light ,

\" '.\ ., -I,,'"\ 0I I-

"

(A )

(B)

Figure 1.6 Vehicles Braitenberg / ) (A ) Vehicle1 (Singlemotor singlesensor: Motion is alwaysforward in the directionof . the sensorstalk, with the speedcontrolled by the sensorEnvironmentalperturbations in terrain producechanges direction. , ) (slippage rough /two motors : The photophobeon the left is aversiveto ) (B) Vehicle 2 ( Two sensors " ) light (exhibiting fear" by fleeing since the motor closestto the light sourcemoves . fasterthan the one farther away This resultsin a net motion awayfrom the light . The and sensors motors to on the left is attracted light whenthe wires connecting photovore " " from the photophobe aremerelyreversed (exhibiting aggression by charginginto the attractor . )

Chapter1

-'.\ II-: .0I\" I -

(C)Figure 1.6 (continued ) (C) Vehicle3: Samewiring as for Vehicle2 but now with inhibitory connectionsThe . vehiclesslow down in the presence a stronglight sourceandgo fast in the presence of of weaklight . In both casesthe vehicleapproach and stopsby ilie light source(with es , one facing the light and one with the light sourceto the rear . The vehicle on the left ) is said to " love" the light sourcesinceit will stay thereindefinitely while the vehicle , on the right exploresthe world, liking to be nearits currentattractor but alwayson the , lookout for something . else to ( D Vehicle 4: By adding variousnonlinearspeeddependencies Vehicle 3, where ) the speedpeaks somewhere betweenthe maximum and minimum intensities other , . interestingmotor behaviorscan be observedThis can result in oscillatory navigation between two different light sources traced (top) or by circular or otherunusualpatterns arounda single source( bottom. ) from Braitenberg1984 Reprintedwith permission) . . ( Figures

~

Whence Behavior ?

,, (;Y - - - - - - ...\,. I""' ,' I /-~ .....4 ""/' /' / ......... -, , '',II-, -"5"( --- - - - - - - .." ,'" @ II'~........ /' ....... """, ~ ' ~ , , \ \

/ I I , , \ \ \

/ /'

/ . \

\II '.:\"\\ -1\:: J ~/ 11/

- "

\ \

\ \I / / / / / .......

\ I I

'") (D ) Figure1.6 (continued

/ / .

\

'

ChapterI

attributedwith personalitytraits suchaspersistenceconsistencyinhumanity , , , or a frantic or observant nature Granted it is quite a leapto attributethese . traits , to robotsbuilt from suchextremelysimplecircuits and plastic toy blocks, but the merefact that an observer perceive can thesequalities evenmildly, in such , creatures notable is . simple

1.2.2 Artificial IntelligenceThe birth of artificial intelligence(AI ) as a distinct field is generallyassociated with the DartmouthSummerResearch Conference held in August 1955 . 's This conference goalsinvolvedthe studyof a wide rangeof topics including use , language , neuralnets complexity theory self-improvement abstractions , , , and creativity In the original proposal( McCarthyet al. 1955 Marvin Min . ), " sky indicatesthat an intelligent machine would tend to build up within itself an abstractmodel of the environmentin which it is placed If it were given a . it could first explore solutionswithin the internal abstractmodel of problem the environment then attemptexternalexperiments This approach and ." dominated roboticsresearch the nextthirty years during which time AI research for , developeda strong dependence upon the use of representational knowledge and deliberativereasoningmethodsfor robotic planning Hierarchicalorganization . for planningwasalsomainstreamA plan is any hierarchicalprocess : in the organismthat can control the order in which a sequence operationsis of . , , performed ( Miller, Galanter andPribram 1960 p.16 . ) Someof the betterknown examples the AI planningtradition include: of . STRIPS This theoremproving systemusedfirst-order logic to developa : navigationalplan ( FikesandNilsson 1971 . ) . ABSTRIPS This refinementof the STRIPS systemused a hierarchy of : abstraction to spaces improvethe efficiencyof a STRIPStypeplanner refining , the detailsof a plan asthey becomeimportant(Sacerdoti1974 . ) . HACKER: This systemsearch througha library of procedures propose es to a . Jomain plan, which it later debugsThe blocks world < (toy blocksmovedabout by a simulatedoversimplifiedrobotic arm) servedasa primary demonstration venue(Sussman 1975 . ) . NOAH: This hierarchicalrobotic assembly plannerusesproblemdecomposition and then criticizes the potentially interactingsubproblemsreordering , their plannedexecutionasnecessarySacerdoti1975 . ( ) The classical AI methodologyhas two important characteristics( Boden 1995 : the ability to representhierarchical structureby abstractionand the )

~ WhenceRehavinr

" use of " strong knowledgethat employs explicit symbolic representational assertions aboutthe world. AI ' s influenceon roboticsup to this point wasin the idea that knowledgeand knowledgerepresentation centralto intelligence are , and that robotics was no exception Perhaps . this was a consequence AI ' s of . preoccupationwith humm-level intelligence Consideringlower life forms seemed . uninteresting Behaviorbasedrobotics systemsreactedagainstthesetraditions Perhaps . " Brooks ( 1987a said it best Planningis just a way of avoiding figuring out : ) what to do next." Although initially resisted as paradigmshifts often are the , , ' notion of sensingand acting within the environmentstartedto take preeminence in AI -relatedrobotics researchover the previousfocus on knowledge and . in hardware representation planning Enablingadvances robotic andsensor ' roboticscommunitys hypotheses ,had madeit feasibleto testthe behaviorbased . The resultscaptured imaginationof AI researchers the aroundthe world. The inception and growth of distributed artificial intelligence (OAI) paralleled thesedevelopmentsBeginning as early as the Pandemonium . system (SelfridgeandNeisser1960 the notion beganto takeroot that multiple competing ), or cooperatingprocess (referredto initially as demonsand later as es are capableof generatingcoherentbehavior Early blackboardbased . ) agents II suchasHearsay ( Erman al. 1980 referred et ) speech understanding systems to theseasynchronousindependent , , agentsas knowledgesources communicating with each other through a global data structurecalled a blackboard . 's of Mind Theory ( Minsky 1986 forwardedmultiagentsystems ) Minsky Society as the basisfor all intelligence claiming that althougheachagentis as , and interactionbetween simpleasit canbe, throughthe coordinated concerted thesesimple agents highly complex intelligencecan emerge Individual behaviors . , canoften be viewedasindependent in behaviorbased robotics , agents it closely to OAI. relating1.2. 3 Robotics

Mainstreamroboticistshaveby necessity with generallybeenmoreconcerned and action than their classicalartificial intelligencecounterparts . perception To conductrobotics researchrobots are needed Thosewho only work with . , simulationsoften ignore this seeminglyobvious point. Robots can be complex to build and difficult to maintain To position currentresearch . relativeto ' them, it is worth briefly reviewing someof roboticists earliestefforts, bearing constrained these in mind that technologyin the 1960sand 1970sseverely . today projectscomparedto the computationalluxuries afforded researchers

Chapter1

Figure1.7 . (Photograph . Shakey courtesyof SRI International) Many other robots will be discussed throughoutthis book, but thesesystems arenotableaspioneers thosethat followed. for . Shakey: Oneof the first mobilerobots Shakey(figure 1.7), wasconstructed , in the late 1960s the StanfordResearch at Institute( Nilsson1969 . It inhabited ) an artificial world, an office areawith objects specially colored and shaped to assistit in recognizingan obj.ectusing vision. It plannedan action suchas , pushingthe recognizedobject from one place to another and then executed the plan. The STRIPSplanning systemmentionedearlier was developed for

Whence Behavior ?

usein this system The robot itself wasconstructed two independently . of con trolled steppermotors and had a vidicon televisioncameraand optical range finder mountedat the top. The camera motor-controlled tilt , focus, andiris had . were mountedat the peripheryof the capabilities Whiskerlike bump sensors robot for protection The plannerusedinformation storedwithin a symbolic . : world model to determinewhat actionsto take to achievethe robot' s goal at a given time. In this system perceptionprovidedthe information to maintain , andmodify the world model' s representations . . Hlli A RE: This project beganaround 1977 at Laboratoired' Automatique et d' Analysedes System ( LAAS) in Toulouse France(Giralt, Chatila and es , , Vaisset1984 . Therobot Hll. .ARE (figure 1.8) wasequipped with threewheels : ) two drive andone caster It wasratherheavy weighingin at 400 kg. Its world . ,

Figure1.8 Hn.ARE ( photograph . . of . , , courtesy LAAS CNRSToulouseFrance)

Chapter1

(A) Figure1.9 . contained smoothflat floorsfoundin a typical office environmentFor sensing the . , , , it useda videocamerafourteenultrasonicsensorsanda laserrangefinder : within a multilevel representational space Geometric Planningwasconducted of and the modelsrepresented actualdistances measurements the worlds, anda . the relationalmodelexpressed connectivityof roomsandcorridors Of special 's for experimentation . noteis Hll..AR E longevity. Therobot wasstill beingused after its initial construction( NoreilsandChatila 1989 . well over a decade ) . Stanford Cart/ CMU Rover: The StanfordCart (figure 1.9, A ) was aminimal robotic platform used by Moravec to test stereovision as a meansfor aboutonemeter 1977 . It wasquite slow, lurching ahead ) navigation( Moravec . , every ten to fifteen minutes with a full run lasting about five hours The vi -

Behavior ~ Whence

(B)) Figure 1.9 (continued . courtesyof HansMoravecand the (A ) StanfordCart. (B) CMU Rover (Photographs . RoboticsInstitute CarnegieMellon University) ,

was sualprocessing the mosttime-consumingaspect but the cart success , fully obstacles , twenty metercoursesavoidingvisually detected fairly complex navigated were addedto its internal world map as detected as it went. Obstacles . as andwererepresented enclosingspheresThe cart useda graphsearch algorithm . to find the shortest paththroughthis abstractmodel Around 1980 Moravecleft for CarnegieMellon University (CMU) where , he led the effort in constructingthe CMU Rover (Moravec 1983 a smaller , ), robot with three independentlypoweredand steeredwheel pairs cylindrical / capableof carrying a cameramountedon a pan tilt mechanismas well as infrared and ultrasonicsensors (figure 1.9, B). This robot was followed by a

Chapter1

1.10 FigureRobotcontrol systemspectrum . long succession of other CMU robots , several of which are described in other portions of this book . These and other robotic precursors set the stage for the advancesand controversies to come as behavior -based robotic systems appeared in the mid - 1980s.

1.3 TIlE SPECTRUM ROBOT OF CONTROLand es Many differenttechniques approach for robotic control havebeendeveloped . Figure 1.10 depictsa spectrumof currentrobot control strategiesThe . left side represents methodsthat employ deliberativereasoningand the right reactivecontrol. A robot employingdeliberative represents reasoning requires relatively completeknowledgeabout the world and usesthis knowledgeto it , predictthe outcomeof its actions an ability that enables to optimizeits performance oftenrequires relativeto its modelof the world. Deliberatereasoning aboutthis world model primarily that the knowledge , upon strongassumptions is is . which reasoning based consistentreliable, andcertain If the information , usesis inaccurate haschanged or sinceobtained the outcomeof the reasoner , . reasoning mayerr seriously In a dynamicworld, whereobjectsmaybe moving arbitrarily (e.g., in a battlefieldor a crowdedcorridor), it is potentiallydangerous to rely on pastinformation that may no longer be valid. Representational from both prior knowledge world modelsare thereforegenerallyconstructed datain supportof deliberation . and aboutthe environment incomingsensor

? WhenceBehavior

Level 7 Levele Level 5 Level4 Level3 Level 2 Level 1

1.11 Figure '

. Albus s hierarchicalintelligent control system Deliberative reasoning systems often have several common characteristics: . They are hierarchical in structure with a clearly identifiable subdivision of es functionality , similar to the organization of commercial business or military command. . Communication and control occurs in a predictable and predetermined manner , flowing up and down the hierarchy , with little if any lateral movement. . Higher levels in the hierarchy provide subgoals for lower subordinate levels. . Planning scope, both spatial and temporal , changes during descent in the hierarchy . Time requirements are shorter and spatial considerations are more local at the lower levels. . They rely heavily on symbolic representation world models. 1. 3.1 Deliberative / Hierarchicai Control

thoseof reactive The intelligent control roboticscommunity whoseroots precede , -based as methods its principal behavior , reasoning systemsusesdeliberative and . , paradigm Albus, at the National Institute of Standards Technology ' ' . is one of this philosophys leading proponentsHis methodsattemptto integrate both natural and artificial reasoning(Albus 1991 . Figure 1.11 depicts )

Chapter1

a jukebox-like hierarchicalmodel with eachlayer consistingof four components , asoutlined in Albus' s theoryof intelligence sensory : ; , processingworld task decompositionand valuejudgment All layersarejoined by a . , , modeling . global memorythroughwhich representational knowledgeis shared Perhaps the most telling assertionthat represents heavyrelianceon world models the is reflectedin Albus' s views regardingthe role of perception Perception the : is establishment maintenance correspondence and of betweenthe internal world modelandthe externalreal world. Consequentlyactionresultsfrom reasoning , over the world model. Perception thus is not tied directly to action . In the mid- 1980s this view so dominatedrobotics that the government , a architecture reflectedthis model. Figure 1.12 shows that developed standard the NASA NIST( NBS standard / reference modelfor TelerobotControl System ) Architecture or NASREM (Albus, McCain, and Lumia 1987 . Despite the ) ' s endorsementNASREM hashad limited acceptance is but , government only still beingusedfor taskssuchascreatinga flight teleroboticservicercapable of and tasksfor NA SA' s spacestation performingmaintenance simpleassembly Freedom(Lumia 1994 . The six levelsembodiedon this systemeachcapture ) a specificfunctionality. Simply put, from the lowestlevel to the highest : 1. Servo providesservocontrol (position velocity, and force management : , ) for all the robot' s actuators . 2. Primitive: determines motion primitives to generate smoothtrajectories . 3. Elementalmove definesandplansfor therobot pathsfreeof collisionswith : environmental obstacles . 4. Task converts : desiredactionson a singleobjectin the world into sequences of elementalmovesthat canaccomplish them. 5. Service bay: convertsactions on groups of objects into tasks to be performed on individual objects scheduling taskswithin the servicebay area . , 6. Servicemission decomposes overall high-level missionplan into service : the . bay commands for . Higher levelsin the hierarchycreatesubgoals lower levels Anotherarchitecturalembodiment thesesameideas RCS (the Real time of , Control Systemreferencemodel architecture, hasthe samebasiclayering as ) NASREM but morefaithfully embeds components the outlinedin Albus' s theory of intelligence This approach . wastestedin simulationfor an autonomous submarine Huang 1996 but hasnot yet beenfieldedon the actualvehicle. ( ) In other work alongthesesamelines, researchers Drexel University have at focusedon the theory of intelligent hierarchicalcontrol and createda control modelpossessing following characteristics the :

Whence Behavior ?

SENSORY PROCESSING

WORLD MODELING

TASK DE MPOSmON CO

SENSEfigure 1.12 . architecture NASREM. . .

ACT

It correlates human teams and robotic control structures: ' A hierarchy of decision makers implements this idea. Autonomous control systems are organized as teams of decision makers. . It assumesthat the task is decomposable, that is , it can result in structured subtasks. . Hierarchies are generated by recursion using a generalized controller . . Preconditions are established at each level of recursion to ensure proper execution.

Chapter1

. Figure 1.13 showsa mobile robot control systemconsistingof six levels The setof nested hierarchicalcontrollersconsists a high-levelplanner navigator of , , . , pilot, path monitor, controller andlow-level control system In yet anotherrepresentative the intelligent controlscommunity research of , at the Rensselaer Institute ( Lefebvreand Saridis 1992 restricted ) PolYtechnic the hierarchyto three primary levels organizationlevel (conductshigh-level : ) planning and reasoning, coordinationlevel (providesintegrationacrossvarious hardwaresubsystems and executionlevel (supportsbasic control and ), hardware. This approachimplementsthe principle of increasingprecision ) with decreasing . intelligenceasonedescends throughthehierarchyFigure 1.14 . depictsa logical model of this architecturalframework Note the clear and restrictive- flow of control and communicationbetweenlevels within the . hierarchy Hierarchicalcontrol is seeminglywell suitedfor structuredandhighly predictable environmentse.g., manufacturing. Reactivesystemshowever were ( , , ) in response severalof the apparentdrawbacksassociated to with developed the hierarchicaldesignparadigmincluding a perceived lack of responsiveness in unstructured and uncertainenvironments due both to the requirements of world modeling and the limited communication ; pathways and the difficulty in engineering completesystemsas incrementalcompetency proveddifficult to achieve that is, virtually the entire systemneeded be built beforetesting to , wasfeasible .

1.3.2 Reactive SystemsThe right side of the spectrum reactivesystems depictedin figure 1.10 represents . Simplyput, reactivecontrol is a technique tightly couplingperception for and action, typically in the contextof motor behaviors to producetimely , robotic response dynamicand unstructured in worlds. We further definethe following: . An individual behavior a stimulusresponse : / pair for a given environmental n. settingthat is modulated attentionanddetennined intentio, by by . Attention prioritizes tasksandfocusessensory : resources is detennined and context . by the currentenvironmental . Intention: determines which set of behaviorsshouldbe activebasedon the ' robotic agents internal goalsandobjectives . . Overt or emergent : behavior the global behaviorof the robot or organismas a consequence the interactionof the activeindividual behaviors of .

? WhenceBehavior

Figure 1.13 . Nestedhierarchicalintelligent controller

Chapter1

Figure 1.14 J modelof hierarchicalintelligent robot. Logica . Reflexive behavior (alternatively , purely reactive behavior ) : behavior that is generated by hardwired reactive behaviors with tight sensor effector arcs, where sensory information is not persistent and no world models are used whatsoever. Several key aspects of this behavior-based methodology include (Brooks 1991b) : . Situatedness: The robot is an entity situated and surrounded by the real world . It does not operate upon abstract representations of reality , but rather reality itself . . Embodiment : A robot has a physical presence (a body ) . This spatial reality has consequences in its dynamic interactions with the world that cannot be simulated faithfully .

? WhenceBehavior

. EmergenceIntelligencearises : from theinteractions therobotic agentwith of its environment It is not a propertyof either the agentor the environmentin . isolationbut is rathera result of the interplaybetweenthem. This book focuses reactiverobotic systemswhose , prim~ ly on behaviorbased 4 describein moredetail. Hierarchi structureandorganization 3 chapters and further in the contextof hybrid robotic cal control, however is also discussed , in architectures presented chapter6.

RELATED ISSUESand the A few importantissues centralto understanding appreciating behavior beforeheadinginto the core of basedparadigmwarrantadditionaldiscussion , this book. . Groundingin reality: A chroniccriticism of traditionalartificial intelligence researchis that it suffers from the symbol grounding problem that is, the , with which the systemreasonsoften have no physical correlation symbols . with reality; they are not groundedby perceptualor motor acts In a sense can : ungrounded systems be saidto be delusional Their world is an artifactual . hallucination Robotic simulationsare often the most insidious examplesof " this problem with " robots purporting to be sensingand acting but instead , to just creatingnew symbolsfrom old, noneof which truly corresponds actual . events Embodiment as statedearlier forces a robot to function within its , , of : environment sensing acting, and sufferingdirectly from the consequences , ." and its misperceptions misconceptions Building robotsthat aresituatedin the " world crystallizesthe hard issues (Flynn and Brooks 1989 . For that reason ) in this book focusesprimarily on real robotic systems implemented hardware for asexemplars robotic control. . Ecological dynamics A physical agentdoesnot residein a vacuumbut is : immersedin a highly dynamicenvironmentthat varies significantly typically in both spaceand time. Further theseenvironmentaldynamics except for , , , highly structuredworkplaces are very difficult if not impossibleto characterize . Nonethelessif a situatedrobotic agentis to be designed , , properly it must and within its designtheopportunities perils thatthe environment acknowledge es . affordsit . This is mucheasiersaidthandone In nature evolutionaryprocess , shapeagentsto fit their ecologicalniche; thesetime scalesunfortunatelyare not availableto the practicingroboticist. Adaptation howevercanbe crucially , ; chapter8 exploresthis further. important

Chapter1 . Scalability : Scalability of the behavior-based approach has been a major question from its inception . Although these methods are clearly well suited for low -level tasks requiring the competence of creatures such as insects, it has

beenunclearwhethertheywould scaleto conformto humanlevel intelligence . " the strict behaviorist Tsotsos( 1995 for example argues that for the , ), position modeling of intelligencedoesnot scaleto humanlike problemsand performance ." Section7.1 considers point further. Many of the strict behaviorists this has , persistin their view that the approach no limits ; notably Brooks ( 1990b ) statesthat " we believe that in principle we haveuncoveredthe fundamental foundationof intelligence" Othersadvocate hybrid approach . a betweensymbolic andbehavioralmethodsarguingthat thesetwo approach are es , reasoning :" hierarchical control fully compatible The falsedichotomythatexistsbetween " and reactivesystems shouldbe dropped (Arkin 1989d. (Seealsochapter6.) ) Much currentresearch focuses testingthe limits of behaviorbased on methods , andthis themewill recur throughoutthis book.

'S 1.5 WHAT AHEADThis book consistsof the following chapters : 1. Introduction : highlightsthe core issues intelligent roboticsandreviews of the history of cyberneticsartificial intelligence androboticsthat led up to the , , of behaviorbased robotic systems . development 2. Animal behavior: studiesthe basis for intelligence biological systems , , , , throughthe eyesof psychologistsneuroscientistsand ethologistsand examines several robotic systems . representative inspiredby animalbehavior 3. Robot behavior: describes basisfor behaviorbased the robotics including , the notation expressionencoding assemblingandcoordinationof behaviors . , , , , -based architectures: presentsa range of robotic architectures 4. Behavior . employingthe behaviorbased paradigm 5. Representationalissuesfor behavioral systems questionsand explores : the role of representational within the contextof a behaviorbased knowledge . system 6. Hybrid deUberativeireactive architectures: evaluatesrobotic architectures that couplemore traditional artificial intelligenceplanningsystems with reactivecontrol systems an effort to extendfurther the utility of behavior in based control. 7. Perceptual basis for behavior-based control: considersthe issuesconcerning the connection perception action sensor of to mod, types perceptual

Whence Behavior ? utes expectationsattention and so on- and presents , , , perceptual designfor a . of specificapplications , rangeof robotic tasks including descriptions es 8. Adaptive behavior: address how robotscancopewith a changingworld mechanisms , including reinforcement througha variety of learningandadaptation . neuralnet Worksfuzzy logic, evolutionarymethodsandothers , , , learning of 9. Social behavior: opensup behaviorbasedroboticsto the consideration how teamsand societiesof robots can function togethereffectively raising new issuessuchas communicationinterference and multiagentcompetition , , , a , andlearning andpresents casestudyillustratingmanyof these cooperationconcepts. 10. Open issues: explores some open questions and philosophical issues regarding intelligence within artificial systems in general and behavior based robots in particular .

Chapter 2Animal Behavior

Animals, in their generation are wiser than the sons of men; but their wisdom is , . , . confinedto a few particulars and lies in a very narrowcompass - Joseph Addison

Chapterofpossible Objectives between 1 develop .Torobot an animal understanding and .background control ,psychology behavior therelationships ethol 2fora . .To reasonable ,and provide in the of neuroscience roboticist systems ogya range robotic . 3 examine motivated .To wide biologically2.1 WHAT DOES ANIMAL BEHAVIOR OFFER ROBOTICS? in The possibility of intelligent behavioris indicatedby its manifestation biological thenthat a suitablestartingpoint for the study . It seems systems logical of behaviorbasedrobotics should begin with an overview of biological behavior . . First, animal behaviordefinesintelligence Where intelligencebegins and endsis an open -endedquestion but we will concedein this text that intelligence , animals Our working definition will be that . canresidein subhuman endowsa system(biologicalor otherwise with the ability to improve ) intelligence to its likelihood of survival within the real world and whereappropriate success , fully with other agentsto do so. Second animal competeor cooperate

Chapter2

behaviorprovidesan existence . proof that intelligenceis achievableIt is not a a poorly understood , it is a concrete mysticalconcept reality, although phenomena . Third, the study of animal behaviorcan provide modelsthat a roboticist canoperationalize within a robotic system Thesemodelsmaybe implemented . with high fidelity to their animalcounterparts may serveonly asan inspiration or for the roboticsresearcher . ' Roboticistshavestruggled providetheir machines to with animals simplest : capabilities the ability to perceiveand act within the environmentin a mean . ingful and purposivemanner Although a studyof existingbiological systems that alreadypossess ability to conductthesetaskssuccess the obviously fully seems a reasonable methodto achieve that goal, the roboticscommunityhas it . historically resisted for two principal reasonsFirst, the underlyinghardware is fundamentallydifferent. Biological systems bring a large amountof evolutionary to supportintelligent behaviorin their siliconbaggage unnecessary basedcounterpartsSecond our knowledgeof the functioning of biological . , hardwareis often inadequate supportits migration from one systemto an to other For theseandother reasonsmanyroboticistsignore biological realities . , andseekpurely engineering solutions . Behaviorbasedroboticistsarguethat there is much that can be gainedfor roboticsthroughthe studyof neuroscience . , psychology andethology ,

The behaviorbasedroboticist needsto decidehow to use resultsfrom these otherdisciplines Somescientists . attemptto implementtheseresultsasclosely as possibly concerningthemselves , primarily with testing the underlying hypotheses of the biological modelsin question Otherschooseto abstractthe . underlying details and use thesemodelsfor inspiration to createmore intelligent robots unconcerned with any impact within the disciplinesfrom which , the original modelsarose Wewill seeexamples both approach within this . of es book. To appreciate behaviorbasedrobotics it is important to have somebackground , in biological behavior which this chapterattempts provide. We first to ,

Animal Behavior

of overviewthe importantconcepts neuroscience , psychologyandethology in , , . with several robotic systems whose that order The chapterconcludes exemplar , goalshavedrawn heavily on biological modelsfor robotic implementation a of themethat continues varying degrees to throughoutthe remainder the book.

VI I BASIS FORBEHA 0 R 2.2 NE ROSCffi NTI F C UThe centralnervoussystem(CNS) is a highly complexsubjectwhosediscussion warrantsat least a separate textbook This sectionattemptsonly a gross . . overview First, it highlights the componenttechnologyof neural circuitry. of Next, it introducesthe readerto the most basic aspects brain function and that translatestimulusinto restructureand the neurophysiological pathways models . abstract , computational . sponsethat is, producebehavior Last, it presents as within braintheorythathaveserved a basisfor behaviorbased developed robotic systems .

2.2.1

Neural Circuitry Eachoutcry of the huntedhare . A fibre from the brain doestear - William Blake

' is The nervoussystems elementalcellular component the neuron(figure 2.1). " " canonical neuron and : Theycomein manydifferentshapes Thereis no single . a sizes but they do possess commonstructure Emanatingfrom the cell body , es at the axonhillock is the axon which after a traversalof somelengthbranch , . off into a collection of synapticterminalsor bulbs This branchingis referred in . to as axonal arborealization The axon is often sheathed myelin, which of facilitatesthe transmission the neuralimpulsealongthe fiber. The boundary betweenneural interconnectionsreferredto as a synapseis wherechemical , , the diffuse across synapticcleft whenthe cell " fires." At the neurotransmitters ' from the cell body neurons receivingend a collection of dendritesemanates , , . from the other sideof the synapse continuing of the occursacross neuronby the conveyance an electrical Signaltransmission ' chargefrom the dendrites input surfacesthrough the cell body. If the total amountof electricity impinging upon the cell is below a certain threshold , , the currentis passivelypropagated throughthe cell up the axon becoming the es. weakeras it progress If , however it exceeds thresholdat the axon , without significant loss and hillock, a spike is generated actively propagated

2 Chapter~Dendri

8m ....

-

~

Axon Axonhitockrborization I r . Axon

Figure2.1 of . (From Metaphorical The Brainby M. Arbib Copyright . Stylized representationa neuron 1972 WileyInterscience . Reprinted permission JohnWileyandSons of by , by Inc.)of current up the axon to the synaptic bulbs , causing the release of neurotransmitters across the synaptic cleft . The cell must then wait a finite amount of time (the refractory period) before it can generate another electrical spike. This spike is also referred to as the action potential . Basic neurotransmitters are of two principal types : excitatory , adding to the probability of the receiving cell ' s firing ; and inhibitory , decreasing the likelihood of the receiving cell ' s firing . Combinations of neurons give rise to ever- increasing complex neural circuitry . There are many examples of specialized small systems of neurons whose function neuroscientists have elucidated. These special purpose systems include (from Arbib 1995b) : . . . . . . . . scratch reflex es in turtles bat sonar stomatogastric control in lobsters locomotion pattern generation in lampreys wiping reflex in frogs cockroach locomotion location of objects with electricity in electric fish visuomotor coordination in flies and frogs

Animal Behavior

Often roboticists can draw from these neural models to create similar fonD S of behavior in machines. In Section 2.5 , we will study a few examples of this

eachdedicatedto a spec function. Thesephysically parallel columnsalso "ific processinformation in parallel. This is of particular importancefor space from sensory , inputs. For example in touch, space mapsgenerated preserving ' s embeddedneural tactile sensors project preserving maps from the skin cortex This is also the case for visual input . to the brain' s somatosensory ' . from the eyes retinasthat ultimately projects onto the visual cortex Parallel of pathwaysare naturally presentfor the processing spatially distributed information. One model related to the inherentparallelism in neural processingis referred ' to aslateral inhibition, whereinhibition of a neurons firing arisesfrom . the activity of its neural neighbors Lateral inhibition can yield asingledom . inant pathway even when multiple concurrentactive pathwaysare present This results from amplification of the variationsin activity betweendifferent . neuronsor neuralpathways Throughstronglateral inhibition, one choice . from many can be selectedin a winner-take all manner This is of particular value in taskssuchas competitivelearningfor patternclassificationtasks , in of word sense language solving the correspondence , , prey recognition disambiguation problem in stereovision (finding matching featuresin two or moreseparate ), images or selectingonefrom amongmanypossiblebehavioral . responses

2.2.2 Brain StructureandFunction' It is said that the Limbic systemof the brain controlsthe four F s: Feeding Fighting, , . , Fleeing andReproduction - Karl Pribram

. Animal brains obviously comein a very wide rangeof sizes Simple invertebrates - 1Q4neurons whereas brain the havenervoussystems , consistingof 103 . suchasa mousecontainsapproximately107neuronsThe of a smallvertebrate . individual neurons Despite to humanbrain hasbeenestimated contain 101- 1011 in brain size we can say severalthings generallyabout a largevariation , . . brains First, locality is a commonfeature Brains are not a homogeneous vertebrate into different massof neurons rather they are structurallyorganized ; , . Next, animalbrains eachof which containsspecialized , functionality regions

Chapter2

generallyhavethree major subdivisions(figure 2.2). For mammalianbrains , thesegenerallyconsistof ( 1) the forebrain which comprises the , . Neocortex associated : with higher level cognition. . Limbic system(bet ween neocortexand cerebrum: the ; ) providing basicbehavioral survivalresponses . . Thalamus mediating incoming sensoryinformation and : outgoing motor . responses . Hypothalamus managinghomeostasisthat is, maintaininga safe internal : , state(temperaturehunger respiration and the like). , , , the (2) the brainstem which comprises , . Midbrain: concerned with the processing incoming sensoryinformation of , (sight, sound touch, and so forth) and control of primitive motor response . systems . Hindbrain which consists the of , . Pons projectingacross brain : the . carryinginformationto the cortex . Cerebellum maintainingthe tone of muscle : for groups necessary coordinated motion. . Medulla oblongata connecting brain andthe : the spinalcord. and (3) the spinal cord, containingreflexive pathwaysfor control of various motor systemsFinally, afferent inputs conveysignals(typically sensory toward . ) the brain, whereasefferentsignalsconveycommands from the brain to the body. Invertebrate neuralstructureis highly variableandthusfewer gener alizationscanbe made . Mammaliancortex hasregionsassociated with specificsensoryinputs and motor commandoutputs (figure 2.3). In humans the visual cortex (sight) is , toward the rear of the brain, the auditory cortex (sound is to the side and , ) the somatosensory cortex (touch is midbrain, adjacentto the motor cortex ) from the input sensory (locomotion . Spacepreservingtopographicmappings ) within all theseregions It is interestingto note . organsto the cortexarepresent that thesemappingsare plastic in the sense that they can be reorganized after . This has been shownfor both the somatosensory damage system(Florence and Kaas 1995 andfor visual cortex(Kaaset al. 1990 . ) ) occurs within the brain as well. At this level neuroscientific Subspecialization modelshaveoften had an impact on behaviorbasedrobot design For . : example " " . In section7.2.2 we encounter what" and" where corticalregionsassociated with visual processing .

Animal Behavior

Brain Primitive Stem

::~ ; ~ ; ~~ db. forebrain mlraIn .Brain Reptilian

~ :~:::. ~ ~Brain Chimpanzee

RatBrain ) (notto scale

cerebra hemisp ~ml ~'tJ ~. Hforebra ~iraIn =~ hindbrain 'd) bbrain stem cerebellum

::~ml:2db ~ ~SZforebrain ~.: raInMammalian Brain

Human Brainleft cerebralhere

Figure2.2 .) of . (Figure Animalbrainstructures courtesy RodGropen . Section6.2 discuss distinctionsbetweendeliberative(willed) and automatic es motor actionsbased for behavioralcontrol systems managing uponneu . rophysiologicalevidence . Evidencefor parallelmechanisms with both long- andshort term associated , ) (Miller andDesimone1994 . For both cases memoryhasalsobeenuncovered : onefor factsandevents are , two distinctprocessing present systems seemingly es skills. Section5.2 discuss therole the otherfor learningmotor andperceptual . robotic systems of thesetwo different typesof memoryfor behaviorbased

2 Chaptersomatosensory cortexmotor cortex

auditory

cortexprimary cortical secondary cortical association

Fiaure 2.3Regions of sensory and motor process in the human cortex . The general flow of infor -

mationwithin brain indicaterlby arrows ( Figurecourtesyof Rod Grope ) the is . D.

for of Neurobiologyoften argues the hypothesis a vectorialbasisfor motor control, somethingthat can be readily ttanslatedinto robotic control systems at (section3.3.2). Research MIT ( Bizzi, MussaIvaldi, and Giszter 1991 has ) shown that a neural encodingof potential limb motion encompassing direction , amplitude and velocity existswithin the spinal cord of the deafferented , frog. Microstimulation of different regionsof the spinal cord generates specific force fields directingthe forelimb to specificlocations Theseconvergent . force fields move the limb towardsan equilibrium point specifiedby the region stimulated(figure 2.4) . The limb itself canbe considered set of tunable a springsasit movestowardsits restposition(equilibrium point). Thusthe planning of the aspects the CNS ttanslateinto establishing equilibrium points that a desiredmotion. Of particular interest is the observation implicitly specifyFigure 2.4 (A ) Forcefields generated microstimulationof lumbarregions(A -D) of frog spinal by cord (shownat left). of ( B) Superposition multiple stimuli. C denotes of simple vector summation independent fields A and B. D represents actualfield evokedby microstimulationof regionsA and B conc ' ently. (Reprintedwith permissionfrom Bizzi, MussaIvaldi, and Giszter Un . 1991 Copyright 1991by AmericanAssociationfor the Advancement Science of .)

Animal Behavior

A .,"" It '.~t',," \-,~ ,." ,'" '" "" ~ ,'.'\" , " /'.......... ..... .......... ....... , I - . ---. -- _ . . - . ~ . .. ~ . . .

-

ID ,, ~J ~~ ' ...'\\I;. ' ' " " \ \ \ ' ' \ \ \ \ \ \ \ ~ , ..

(A)

IVif ~" ~ " ,~ ,.," I' ~ ; ' '

"t""~ """"" ,,,',\'~ t~~ ,\' , :~ ,:~, . . . , . . .

"

.

.

.

.

.

"

"

.

"

"

"

"

" " " " " " ... -- " - ~ .. ,-

" 111 " 11 " 11 ' & 1 " , & II \ " I ' I ' . . , . ... .

C

.--_.'~ ---.,'"-~"'" .,~ .,I\~ ~ """"" , I ' ,-" ,

\ \ , I , ~ ,. " ' \ \ I / ' ~ ", I " ' \ \ 1 " -" ,,, ' " \ . , , ,. " ' " \ . " " -'

. " " " "

(B)

~,.','," ~", ", --",,.., --,. , -""" " " " .1

\ I I I ' ~ ' \ I I I ' ~ ~ \ \ III " \ III ~ " \ 11 / -' "

2 Chapter Box2.1 Memory Typesover , " as by Information . that to STM . hours to temporary transfer actions in retention refer " viewed distinction acting the , process a memory is minutes " in inferior with to is requiring usually measured appropriate we is significant memory working tasks a " involved , is seconds what accuracy as guide LTM tenn information is to to for is appears several recall Long memory . of scale its performing ) LTM " infonnation memory . referred brain but tenn for time , 1981 information the tenn also LTM periods of of is The holding to involves . Short for : Long ) for capacity : ) definition that STM STM its region . , ( McFarland ) STM in ( persists LTM . " ( from scale 1995 capacity manipulation conversation working STM its hours the in time in limitless memory memory 24 One memory hippocampal . Bumod and everyday tenn information stored than The term and almost in term years limited . of as long or storage intennediate longer STM quite Short Long Guigon ( an is from memory many . days of of process .

that multiple stimulationsgive rise to new spatial equilibrium points generated -based . by simple vector addition This sameprinciple is usedin schema robotic control (section4.4). New experiments humans(intact fortunately in ), havebeenshownto be consistent with this force-field model when appliedto and reachingtasks(Shadmehr MussaIvaldi 1994 . ) Othercompellingexamples that computations motion within the of arguing brain should be consideredas vectorsinclude researchfrom the New York University Medical Center( pellioniszand ilinas 1980 . The authorscontend ) that activity within the brain is vectorial Intendedmotion is generated an . as -dimensional . Brain function is considered activity vectorwithin a three space " of ." geometricwherethe language the brain is vectorial The authorsexplain , however that simple reflex are not adequate explain the entire rangeof es to , . complexityevidenced the actionsthe brain generates by Another exampleforwarding spatial vectorsas an underlying representational mediumfor neuralspecification motorbehaviorcomesfrom theJohns of " Schoolof Medicine (Georgopoulos 1986 . The " vector hypothesis Hopkins ) asserts the changes the activity of specificpopulationsof neuronsgenerates that in a neural coding in the fonD of a spatial vector for primate reaching . resultshave been shown to be consistentwith this underlying Experimental . hypothesis has Finally, vector based trajectorygeneration servedasan accountfor certain ' fonDS of animal navigation Arbib and Houses model ( 1987 explains . )

Animal Behavior

detourbehaviorin toads(i .e., their circumnavigation obstaclesby describing of ) the animal' s path planningin tenD S the generation divergence of of fields ' directional vectors basedon the animal s perceivedenvironment Inparticular . ( ) , repulsivefields surroundingobstacles attractiveforces leading to food , sourcesanddirectionalvectorsbased the frog' s spatialorientationgenerate on , a computational model of path planningin toadsconsistent with observed experimental -based data This modelhasbeeninfluential in the designof schema . robot controllers(section3.3.2).

2.2.3

Abstract Neurosci enti:fi(: Models

' , Unfortunately our knowledgeof the brain s function is still largely superficial in is at (literally). Progress neuroscience proceeding a rapid paceas new tools . brain function becomeavailable Nonethelessit has been for understanding . , said that even if we possessed completeroad map of the brain' s neural a structure (all of its neuronsand their interconnections our understanding ), would still be inadequateBrain activity over the neural substrateis highly . and control would still need , dynamic and information regardingprocessing to be elaborated . What then should a brain theorist do? The key for many scientistslies in their first formulating an abstractionof brain function and then looking for . neuralconfirmation This top-down approach characterizes many researchers in neuroscienceand has potentially high payoff for roboticists as abstract , , modelsof brain function hypothesized theseneuroscientists potentially can by leadto robotic control systems useful in their own right. Abstract computationalmodels used to expressbrain behavior have two forms: schema . es mainstream theoryandneuralnetworks Thesetwo approach arefully compatible(figure2.5). Schema is a higher level abstraction theory by . which behaviorcan be expressed modularly Neural networksprovide a basis occursat a lower for modelingat a finer granularity whereparallelprocessing , level. Schematheory is currently more adept at expressingbrain function, . whereasneural networkscan more closely reflect brain structure Schemas , onceformulated may be translatedinto neural network modelsif desiredor , deemed . In necessary this book we studyboth methodsin the contextof what . they offer behaviorbasedrobotic control systems

. 2.2.3 Schema .1

: Methods

The use of schemas a philosophicalmodel for the explanationof behavior as . datesas far back as Immanuel Kant in the eighteenthcentury Schemas

Chapter2

Figure 2. 5 Abstract behavioral models. Schemas or neural networks by themselves can be used to represent overt agent behavior , or schemascan be used as a higher - level abstraction that is in turn decomposed into a collection of neural networks .

weredefinedas a meansby which understanding ableto categorize is sensory of . perceptionin the process realizingknowledgeor experienceNeurophysio . logical schema theory emerged early in the twentiethcentury The first application was an effort to explainposturalcontrol mechanisms humans( Head in and Holmes 1911 . Schematheory has influencedpsychologyas well, serving ) asa bridging abstraction between brain andmind. Work by Bartlett ( 1932 ) and Piaget( 1971 usedschematheory as a mechanism expressing for models ) of memory and learning Neisser( 1976 presented cognitive model of . a ) interactionbetweenmotor behaviors the form of schemas in interlocking with in the contextof the perceptual . Normanand Shallice( 1986 perception ) cycle usedschemas a meansfor differentiatingbetweentwo classes behavior as of , willed and automatic and proposeda cognitive model that usescontention , as and competitionbetween schedulingmechanisms a meansfor cooperation behaviors Sections6.2 and 7.2.3 discusstheselast two examples . further. Arbib ( 1981 wasthe first to considerthe applications schema of ) theoryto robotic

+

)

\

Neural

"

function

\

Networks

avior structure \ ( Modeling Behavioral ) Models Schema functional ( decomposition

Animal Behavior

sion systems RisemanandHanson1987 . ( ) " " Many definitionsexist for the term schema, often strongly influencedby its applicationarea(e.g., computationalneuroscientificpsychological. Some , , ) : representative examples . a patternof action aswell asa pattern action ( Neisser .for 1976 ) . an adaptivecontroller that uses an identification procedureto updateits of ) representation the objectbeingcontrolled (Arbib 1981 . a functional unit receiving specialinformation, anticipatinga possibleperceptual content matchingitself to the perceivedinformation (Koy-Oberthur , 1989 ) . a perceptual whole corresponding a mentalentity ( Piaget1971 to ) Our working definition is as follows: A schemais the basic unit of behavior from which complex actions can be constructed it consistsof the ; of how to act or perceiveas well as the computational knowledge by process which it is enacted Schematheory providesa methodfor encodingrobotic . behaviorat a coarsergranularitythan neuralnetworkswhile retainingthe aspects of concurrent cooperativecompetitivecontrolcommonto neuroscientific models . Variousneurocomputational architectures have beencreatedthat incorporate theseideas For example work at the University of Genova(Morasso and . , of es ) Sanguineti1994 hasled to the development a model that encompass the vector basedmotion-planning strategies described earlier within the posterior . are into , parietalcortex Here multiple sensorimotor mappings integrated a unitary -orientedmovements for . , body schema necessary the generationof goal Vectorbased potentialfields (section3.3.2) providethe currencyof task specification for this integrationof taskintentions Later, section4.4 exploresother . modelsand associated methodsfor operationalizing schema theory as a basis -based control. for behavior

2.2.3.2 Neural Networks, Computationalmodelsfor neural networks also referredto as connectionist of , systemshavea rich historyparallelingthe development traditionalsymbolic AI . Someof the earliestwork in the areacanbe tracedto the McCulloch-Pitts model of neurons( 1943 . McCulloch and Pitts used a simple linear threshold ) to unit with synapticweightsassociated eachsynapticinput. If a threshold . was exceededthe neuronfired, carrying its output to the next neuron This ,

Chapter2 synaptic weights

Xl w1 --" X----~--"'~ 2- -~-. -"---- :~X- -!3_ ~ .3 - -_ -: . . Xn Input ( Vector ) b2 inary 6A of is each Figure inputs compone 's a A .ronresult by then . vectormultiplied assoc percept Xitogether binary (:)and to .The 's output is binary then then Wito unit subjec synapticsummedis weight the 1 .This ) thresholding output operation sentcell . on indetermine to (the the8network nextOutput

simplemodel gaverise to networkscapableof learningsimple patternrecognition tasks Rosenblatt( 1958 later introduceda formal neural model called . ) a percept (figure 2.6) andthe associated ron ron percept convergence proof that established of thesenetworksystemsIn the 1960s . provablelearningproperties and 1970s however neural network researchwent into a decline for avariety , , of reasons including the publication of the book Perceptrons Minsky , ( andPapert1969 which provedthe limitations of single layer percept networks ron ), . In the 1980s however the field resurgedwith the advent of multilayer , , neural networks and the use of backpropagation Rumelhart Hinton, and , ( Williams 1986 as a meansfor training these systems Many other notable . ) efforts within connectionism , during the last decade far too numerousto review here have yielded highly significantresults It shouldbe remembered . , , however that mostneuralnetworksareonly inspiredby actualbiological neurons , and provide poor fidelity regardingbrain function. Nonethelessthese , abstract modelshaverelevance the behaviorbased to roboticist computational andhavebeenusedwidely in tasksrangingfrom visual road following strategies section7.6.1) to adaptationand learning in behavioralcontrol systems ( 1989provides a more generaltreatmentof neural (section 8.4). Wasserman . networks

Animal Behavior

2.3 PSYCHOLOGICAL BASIS FORBEHAVIORis is , Psychology brainless neuroscience mind1e~~ - Les Karlovits

has Psychology traditionallyfocusedon the functioningof the mind, lesssothe brain. It is not our intent to revisit the classicalmonistdualist debateof mind / ' andbrain herebut ratherto look at what a psychologists perspective offer canrobotics . Certainly psychology is preoccupied with behavior. Within that scope, we focus on perception and action , as these issues are of primary concern for the roboticist , and provide a brief history of the field , beginning with the twentieth century. Sensory psychophysics was the first to relate stimulationintensityto perception

. Weberand Fechnerdeveloped the physicallaws that described relationships ' betweena stimuluss physicalintensity and its intensity asperceived by an observer Pani 1996 . ( ) Behaviorismburst upon psychologyin the early 1910s Behavioristsdiscarded . all mentalisticconcepts sensationperception image desire purpose : , , , , , 1925 . Behaviorwasdefinedby , andemotion amongothers( Watson , ) thinking observation only; datawas obtainedfrom observingwhat an organismdid or 's said. Everythingwas castin termsof stimulusand responseThis approach . main benefitwas making the field more scientificallyobjective moving away , from the use of introspectionas the primary basisfor the study of mind. Its mainclaim was" that thereis a response everyeffectivestimulusandthatresponse to " is immediate ( Watson 1925 . As behaviorism , ) progressedpsychology moreandmorescientificandlessphilosophical sociological asa field became , , andtheologicalby relying heavily uponempiricaldata(Hull 1943 . B.F. Skin) ' ner ( 1974 eventuallybecame behaviorisms bestknown proponent . ) Kohler 1947 brought physics into the fray, drawing Gestaltpsychology( ) ' from the tradition of sensorypsychophysics while broadening behaviorisms basis This form of psychologyinvertedbehaviorism . somewhatconcerning itself , with sensory visual) andhow behaviorarises heavily input (predominantly of asa direct consequence the structure the physicalenvironment of interacting " wasderivedfrom the Germanwhereit " with the agentitself. The term gestalt . enabled certainbehaviors referredto form or shape anattribute Certaingestalts as based the physicsof retinal projectionandthe ability of the perceiver upon to organizethe incoming stimuli. Gestaltpsychologyfocusedon perception

Chapter2

whereasbehaviorismprincipally concerned itself with action ( Neumann and Prinz 1990 . Gestalters however felt that behaviorismwas limited, arguing , , ) that levelsof organizationexist abovethe sensation itself, which an organism could useto its advantage . a , ), Ecological psychology as advocated J. J. Gibson ( 1979 demanded by of the environmentin which the organismwas situated deepunderstanding and how evolution affectedits developmentThe notion of affordances discussed . ( ' further in section7.2. 3) providesa meansfor explainingperceptions roots in behavior This psychologicaltheory saysthat things are perceivedin . termsof the opportunitiesthey afford an agentto act. All actionsare a direct of consequence sensorypickup. This resultsfrom the tuning by evolution of an organismsituatedin the world to its availablestimuli. Significantassertions (Gibson 1979 include: ) . The environmentis what organismsperceive The physical world differs . from the environmentthat is, it is morethan the world described physics . , by . The observer the environment and eachother . complement . Perceptionof surfacesis a powerful meansof understanding environment the . ' . Information is inherentin the ambientlight and is picked up by the agents . optic array Later, cognitive psychologyemerged paralleling the adventof computer , science defining cognition as "the activity of knowing: the acquisition organization , , " and , and use of knowledge ( Neisser1976 . Information processing ) modelsof the mind beganto play an ever increasingrole. Behaviorism computational wasrelegated the role of explaininganimalbehaviorandbecame to far less influential in studying humanintelligence Unifying methodsof explaining . the relationshipbetweenaction and perception(section7.2. 3) were ) developedunder the banner of cognitive psychology( Neisser1976 . Mentalistic terms previously abandoned could now be consideredusing compu tational process or metaphorsSomeof the underlying assumptions the es . of information processing ( ) approach Eysenck1993 include . A seriesof subsystems es information (e.g., stimulus process environmental ~ attention~ perception~ thoughtprocess ~ decision~ response es ). . The individual subsystems transformthe datasystematically . . Informationprocessing peoplestronglycorrelates in with that in computers . . Bottom-up processing initiated by stimuli, top- down processing intentions is by andexpectationssection7.5.4). (

Animal Behavior

Connectionism the associated and of development neuralnetwork technology (seeSection2.2.3) offer anotheralternativecomputationalmodel to explain mentalprocessingavailableto be exploitedby psychologists . , hasfluctuatedsignificantly depending the current on Although psychology , school of thought roboticistscan derive considerable benefitfrom an understanding , of thesedifferent perspectivesThe roboticist' s goals are generally . different: Machineintelligencedoesnot necessarily explanation requirea satisfactory of humanlevel intelligence Indeed evenpasse . theories , psychological canbe of valueasinspirationin building behaviorbasedautomatons .

LOGICAL 2.4 Emo ~ BASIS FORBEHAVIORare characters a kindof old sagastylizedbecause themost in . Animals stylized even acute themhave of little leeway theyplayouttheirparts as . - Edward Hoagland . Ed1ologyis die study of animal behaviorin its natural environment To die ' strict ed1ologistbehavioralstudiesmust be undertaken die wild ; animals in , haveno meaningoutsided1eirnatural setting The animal itself is . responses of , only onecomponent die overallsystem which mustincludedie environment in which it resides . Konrad Lorenz and Niko Tinbergen are widely acknowledgedas die foundersof die field. Tinbergen considered studiesto focuson four ed1ological areasof behavior( McFarland 1981 : causationsurvivalvalue development , ) , primary . into , andevolution Animal behavioritself canbe roughly categorized threemajor classesBeer Chiel, andSterling 1990 McFarland1981 : , , ( ) . Reftexesare rapid, automaticinvoluntary responses triggeredby a certain environmental stimuli. The reflexiveresponse persistsonly as long as die duration of die stimulus Further die response . wid1die stim, intensitycorrelates 's ulus strength Reflex areusedfor locomotionandod1er . es highly coordinated activities Certainescape . behaviors suchas d1ose found in snails and bristle , worms involvereflexiveactionthatresultsin rapidcontraction specificmuscles of , relatedto the flight response . . Taxesare behavioralresponses orient die animal towardor away from d1at a stimulus (attractiveor aversive. Taxesoccur in response visual, chemical to ) in . , mechanical and electromagnetic , phenomena a wide rangeof animals is Chemotaxis evidentin response chemicalstimuli asfound in die ttail following to of ants Klinotaxis occursin fly maggotsmoving towarda light source .

Chapter2

, by comparingthe intensityof the light from eachsideof their bodies resulting in a wavy course Tropotaxisexhibitedby wood lousesresultsin their heading . . directly towardsa light sourcethroughthe useof their compound eyes . Fixed-action patterns are time-extendedresponse patternsbiggered by a stimulus but persistingfor longer than the stimulus itself. The intensity and durationof the response not governed the strengthand durationof the are by stimulus unlike a reflexivebehavior Fixed-action patternsmay be motivated . , , unlike reflex , andthey may resultfrom a muchbroaderrangeof stimuli than es thosethat govern a simple reflex. Examplesinclude egg rebieving behavior of the greyling goose the songof crickets locust flight patterns and crayfish , , , . escape Motivated behaviorsare governednot only by environmentalstimuli but also by the internal stateof the animal being influencedby such things as , . appetite as Ethologistssuchas Lorenz adoptedthe notion of schema well. Schemas of es in , capturecomplicatedcombinations reflex , taxes and patternsreleased to a suitablecombinationof stimuli. A sign stimulusis the particular response external stimulus that releasesthe stereotypicalresponse Schemas which . , werelater renamed innate releasingmechanismsI RMs in an effort to clarify ( ) their meaning( Lorenz1981 havethe following traits (LorenzandLeyhausen ), 1973 : ) . An IRM is a simplified renderingof a combinationof stimuli eliciting a in . , , specific perhaps complex response a particularbiological situation . One IRM belongsto one reactionto a given situation attunedto relatively , few distinctivefeaturesof the environment oblivious to the rest. and . Every action dependent its own releasingschema on may be elicited completely of independently all otherreactionsintendedfor the sameobject. . The innate releasingmechanism providesthe overall meansfor a specific stimulusto release stereotypical a within a given environmental sign response context . For Lorenz and Tinbergen(Lorenz 1981 complex systemsof behavioral ), mechanisms had a hierarchicalcomponent although Tmbergenconsidered , this a weak commitmentuseful principally only for organizational . purposes Figure 2.7 shows an examplefor the display behavior sticklebackfish use in protecting territory. This notion of hierarchicalgrouping has parallels in serveas aggregates component of schema theory as well, whereassemblages schemassection3.4.4). (

" ". 0 1 ,) ' 01 6 ' ~ ~ ~ . O11 ' ~ O e ' 1 , .

'll,QiJ iPI -rr-" "e6 '(ch .'A ,"6, r:Q vqr CP ~I: QlJ o~o ~< ~-il'~ IqC ;.:peilo ..~ O ~ '

2 ChapterMaximum Selecting System Sensor

Sensor

Sensor

Sensor

Figure 2.8 . Model of maximum selectionsystemproviding lateral inhibition betweenbehaviors a a Whenever behavioris readyand its sensorystimuluspresent it generates positive , . . while inhibiting otherpotentiallyactivebehaviors After Lorenz 1981 response

[~ ~ [~

, Reciprocalinhibition of parallel behaviors a form of lateral inhibition, is also available (figure 2.8). In one model a maximumselecting systemen , and ablesoneof manybehaviorsto dominatebasedon its readiness incoming stimuli. This winner-take all strategyis commonwith many of the arbitration roboticssystems section3.4.3). All behaviors usedin behaviorbased methods ( . not active at a particular momentare inhibited centrally Supportingexperimental existsfor this locusof superior commandin animalsranging evidence to from invertebrates primates( Lorenz1981 . Hybrid behaviorbasedrobotic ) in chapter6, exploit the utility of this organi at architecturesdiscussed length , . zationalconcept are studiesin animal communicationmechanisms highly relevant Ethological for multiagentrobotic systemsArkin andHobbs1992 . Displaybehavior , ) ( in in particular involves the signaling of information by changes postureor ,

Animal Behavior

and activity. Thesestereotyped often highly unusualdisplaysare most often -action patterns(Smith 1977 and may be visible, audible , ) by generated fixed tactile, chemical or evenelectrical as in the caseof the electric eel. The displays , , themselves includebirdsong raisingof a dog' s hackles courtingdisplays , , in ducks color changes fish, leg waving in spiders etc. Suchdisplayshave in , , behavior evolutionarybenefitsfor suchactivitiesasindirectly invoking escape in thepresence predatorsreducingthe likelihood of fighting, andfacilitating of , . themselves , may be behavioralselection mating amongmanyothers The messages that enablethe recipientto respondappropriatelyfor a given messages . situation (i .e., " what" to do) such as flee, in the caseof an alarm message " " , suchas who messages messages They may alsobe so-callednonbehavioral " for kin or sex recognition or " where messagesproviding location information still from the senderThesenonbehavioral . messages may ultimately affect ' s behavior but not so . Ethologists asbehavioral the recipient , directly messages for havedeveloped methods analyzingandrepresenting complexandritualistic . interactions suchascourtshipand greeting . One of the most important conceptsfor behaviorbasedroboticistsdrawn the field of ethology is the ecological niche. As definedby McFarland from " , , ( 1981 p. 411), The statusof an animal in its community in terms of its relationsto food and enemies is generallycalled its niche." Animals survive , in naturebecause ably stableniche: a place where they have found a reason this . can coexistwith their environment Gibson( 1979 strongly asserted ) they mutuality of animal and environmentas a tenet of his school of ecological psychology(seesection2.3). Evolution hasmoldedanimalsto fit their niche. Further as the environmentis always to some degreein flux , a successful , or to animal mustbe capableof adaptingto somedegree thesechanges it will in habitat climate, food asserted changes . , by perish Environmentalpressures ' . sourcesandthe like, canprofoundly influencethe species survivability , of . This conceptof niche is important to roboticistsbecause their goals If and that is autonomous cansuccess theroboticistintendsto build a system fully inhabitants that systemmust find a stable , competewith other environmental the . nicheor it (asan application will be unsuccessfulThis promulgates view ) with mustfind their placewithin the world ascompetitors that robotic systems other ecological counterparts e.g., people . For robots to be commonplace , ) ( / mustfind the ecologicalnichesthat allow themto surviveand or dominate they or their competitors whetherthey be mechanical biological. Often economic , . are pressures sufficientto preventthe fielding of a robotic system If humans taskasa robot (e.g., vacuuming at a lower cost arewilling to performthe same ) and or with greaterreliability, the robot will be unableto displacethe human /

Chapter2worker from the niche he already occupies. Thus , for a roboticist to design effective real world systems, he must be able to characterize the environment effectively . The system must be targeted towards some niche. Often this implies a high degree of specialization . These same arguments are often used in economics and marketing and are generalizable to behavior-based robotics ( McFarland and Bosser 1993) . Section 4.5.7 presents one example of a nichebased robotic architecture.

2.5 REPRESENTATIVE EXAMPLF~ OFBIOROBOTS ~Let us begin our discussion of biorobots by summarizing some important lessons animal behavior affords the roboticist : . Complex behaviors can be constructed from simpler ones (e. ., g through hierarchies or sequentially, as in fixed -action patterns) . . Perceptual strategies should be tuned to respond only to the specific environmental stimuli relevant for situation - specific responses . . Competing behaviors must be coordinated by selection, arbitration , or some other means. . Robotic behaviors should match their environment well , that is , fit aparticular ecological niche. We now turn to five representative examples of robotic systems heavily motivated by animal studies. The first two focus on perceptual aspects that , is , sensory devices mimicking chemotaxis in ants and the compound eye of the fly . A pair of examples then illustrates the problem of producing coordinated locomotion for a robotic cockroach and a primate swinging from trees. The last case concerns interagent communication for a robotic honeybee. These examples are but interesting pieces of the puzzle of building robots ; subsequent chapters explore complete behavior-based robot design. 2. 5.1 Ant Chemotaxis Go to the ant, thou sluggard considerher ways andbe wise. , ,- Proverbs 6:6

Ant behavioris of keeninterestto roboticistsbecause arerelativelysimple ants creatures of complexactionsthroughtheir social behaviorand biologists capable havestudiedthemextensivelyExcellentreference . works areavailableon antbehavior(e.g., Holldobler andWIlson 1990 . Much animalresearch ) signif-

Animal Behavior J

: until icantly influencedmultiagentrobotic systems We defer that discussion ' 9. For the momentwe considerhow chemicalsensinginspiredby ants , chapter behavior canbe usedfor pathfollowing in robots . , Ant communicationis predominantlychemical Visited paths are marked . a volatile b ail pheromoneAll antstraversinga useful path continually ' . using add this odor to the trail, strengthening reinforcing it for future use The and . variationsin foraging strategies result in a wide rangeof speciesspecificcollective that of patterns haveevolvedto fit the ecologicalneeds the environment to which they are adaptedIt could be useful for one interested developing . in robots capableof foraging over long distances to considerthe models forwarded , . by ant entomologists Simulationstudiesconducted the University of Brusselshaveshownthe at ' spontaneous developmentof biologically plausible b ails using mathematical behavior models Intemest traffic for the Argentine ant has been simulated . ) using pheromonemodels (Aron et al. 1990 . Deneubourgand Goss ( 1989 have reproducedspeciesspecific foraging patternsfor three different ) . ) army ant species Goss et al. ( 1990 have likewise emulatedcomputation ally the rotation of foraging trails observedin the harvesterant. Thesesimulation studies although encouraging still require implementationon real , , robots to gain widespreadacceptance useful models for robot foraging as behavior . Researchers Australia havetaken a step forward towardsmore directly in ant behavior by creating robotic systemscapableof both laying emulating down and detectingchemicaltrails (Russell Thiel, and Mackay SiI D 1994. , ) exhibit chemotaxisdetectingandorientingthemselves : Thesesystems alonga chemicaltrail. Camphor a volatile chemicalusedin mothballs servesas the , , ' : chemicalscent The applicationmethodis sbaightforward the robot dragsa . felt-tipped pen containingcamphoracrossthe floor as it moves depositinga , is . trail.onecentimeter wide (figure2.9a). Sensing morecomplex The detection devicecontainstwo sensorheadsseparated 50 mm (figure 2.9b). An inlet by drawsin air from immediatelybelow the sensoracrossa gravimetricdetector crystal. An air downflow surroundingthe inlet insuresthat the inlet air is . arriving from directly below the sensor The detectorcrystal is treatedwith a coating that absorbscamphor and as massis added the crystal' s resonant , , . in proportionto the amountof camphorabsorbedWhen frequencychanges to this chemotactic hasbeenattached a trackedmobile robot provided system ' with an algorithm that strivesto keep the odor b ail betweenthe two sensor b inlets, the robot hasbeenable to follow the c