Oasis: Onboard autonomous science investigation system for opportunistic rover science

OASIS: Onboard AutonomousScience Investigation System

for Opportunistic RoverScience

Rebecca Castano*Machine Learning and Instrument AutonomyJet Propulsion Laboratory4800 Oak Grove Drive – MS 126-347Pasadena, California 91109e-mail: [email protected]

Tara EstlinArtificial IntelligenceJet Propulsion Laboratory4800 Oak Grove Drive – MS 126-347Pasadena, California 91109e-mail: [email protected]

Robert C. AndersonGeophysics and Planetary GeosciencesJet Propulsion Laboratory4800 Oak Grove Drive – MS 264-621Pasadena, California 91109e-mail: [email protected]

Daniel M. GainesArtificial IntelligenceJet Propulsion Laboratory4800 Oak Grove Drive – MS 126-347Pasadena, California 91109e-mail: [email protected]

Andres CastanoMachine VisionJet Propulsion Laboratory4800 Oak Grove Drive – MS 125-209Pasadena, California 91109e-mail: [email protected]

• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •

Journal of Field Robotics 24(5), 379–397 (2007) © 2007 Wiley Periodicals, Inc.Published online in Wiley InterScience (www.interscience.wiley.com). • DOI: 10.1002/rob.20192

Benjamin BornsteinMachine Learning and Instrument AutonomyJet Propulsion Laboratory4800 Oak Grove Drive – MS 126-347Pasadena, California 91109e-mail: [email protected]

Caroline ChouinardArtificial IntelligenceJet Propulsion Laboratory4800 Oak Grove Drive – MS 125-209Pasadena, California 91109e-mail: [email protected]

Michele JuddScience DivisionJet Propulsion Laboratory4800 Oak Grove Drive – MS 183-335Pasadena, California 91109e-mail: [email protected]

Received 12 February 2006; accepted 12 February 2007

The Onboard Autonomous Science Investigation System has been developed to enable arover to identify and react to serendipitous science opportunities. Using the FIDO roverin the Mars Yard at JPL, we have successfully demonstrated a fully autonomous oppor-tunistic science system. The closed loop system tests included the rover acquiring imagedata, finding rocks in the image, analyzing rock properties and identifying rocks thatmerit further investigation. When the system on the rover alerts the rover to take addi-tional measurements of interesting rocks, the planning and scheduling component de-termines if there are enough resources to meet this additional science data request. Therover is then instructed to either turn toward the rock, or to actually move closer to therock to take an additional, close-up image. Prototype dust devil and cloud detection al-gorithms were delivered to an infusion task which refined the algorithms specifically forMars Exploration Rovers �MER�. These algorithms have been integrated into the MERflight software and were recently uploaded to the rovers on Mars. © 2007 Wiley Periodicals,Inc.

1. INTRODUCTION

The Mars Pathfinder and Mars Exploration Rover�MER� missions have demonstrated that mobile rov-ers are a viable and extremely useful option for ex-ploring the surface of other planets. The MER rovershave traveled across kilometers of terrain and gath-

ered extensive scientific data, uncovering profoundnew insights into Mars’ current and past environ-ment, the history of its rocks, and the various rolesand abundances of water. As a result of past suc-cesses, future missions are being planned that willsend additional robotic explorers to Mars as well as tothe moon and outer planets.

Surface rovers offer scientists the ability to movearound a planetary surface and explore different ar-eas of interest. The farther the rover can travel, the*To whom correspondence should be addressed.

380 • Journal of Field Robotics—2007

Journal of Field Robotics DOI 10.1002/rob

greater the opportunity exists for increased scientificdiscovery. Most mobile robot efforts at JPL and NASAhave concentrated on navigation, manipulation, andcontrol. For the MER mission, process automationhas already proven valuable in engineering areas. Forexample, the rovers can drive autonomously usingGESTALT �Maimone, Biesiadecki, Tunstel, Cheng, &Leger, 2006�, a stereo-based hazard avoidance pro-gram that steers the rover away from rocks and steephills. It can also keep track of its position using on-board visual odometry during some drives, which ismore robust to slippage than wheel odometry�Cheng, Maimone, & Matthies, 2005�. Due to ad-vances in rover navigation, traverse ranges are in-creasing at a rate much faster than communicationsbandwidth. While the Sojourner rover traveledaround 100 m in the entire mission, the drive recordfor the most distance covered in a single sol �Martianday� is over 220 m set by the MER Opportunity rover.As this trend in increased mobility continues, thequantity of data that can be returned to Earth permeter traversed is reduced. Thus, much of the terrainthe rover observes on a long traverse may never beobserved or examined by scientists. We present a sys-tem developed to maximize the quality of the sciencedata transmitted to Earth through the use of onboardscience. This system expands onboard automationbeyond the engineering domain to the science do-main.

The Onboard Autonomous Science InvestigationSystem �OASIS� system has been developed to evalu-ate, and autonomously act upon, science data gath-ered by in situ spacecraft such as planetary landersand rovers �Castano et al., 2003; Castano et al., 2004;Castano et al., 2005; Castano et al., 2006�. OASIS ana-lyzes the geologic data gathered by the rover on-board. This analysis is used to identify terrain fea-tures of interest and additional science gatheringopportunities. A planning and scheduling compo-nent of the system enables the rover to take advan-tage of the identified science opportunity by updat-ing the command sequence to include theopportunistic measurements. OASIS currently worksin a closed loop fashion with onboard control soft-ware �e.g., navigation and vision� and has the abilityto autonomously perform the following sequence ofsteps: analyze gray scale images to find rocks, extractthe properties of the rocks, identify rocks of interest,retask the rover to take additional imagery of theidentified target and then allow the rover to continueon its original mission. We have conducted a number

of tests of the combined system and individual com-ponents. We describe results for the system in detect-ing and reacting to a science alert �identified scienceopportunity�.

Several systems and components have been de-veloped and demonstrated for autonomous rover sci-ence operations. Wagner et al. �2001� and Pedersen�2001� describe a system that was successful at au-tonomously identifying meteorites in Antarctica. Gu-lick, Morris, Ruzon, & Roush �2001� described tech-niques for analyzing field test data for theMarsrokhod rover. Gilmore et al. �2000� also pre-sented several methods developed specifically for au-tonomous rover science. More recently, there hasbeen a development of methods for autonomous sci-ence including classification of features and survey inassociation with the automated identification of lifein the Atacama dessert �Smith, Niekum, Thompson,& Wettergreen, 2005; Thompson, Smith, & Wetter-green, 2005a; Thompson, Niekum, Smith, & Wetter-green, 2005b�.

In this paper, we first discuss several motivatingscenarios for on-board science and the OASIS system.We then explain the components of OASIS and thesystem itself after which we describe experimentaltesting with the system and assess the results. Finally,we address future work.

2. OVERVIEW OF THE OASIS SYSTEM

The OASIS system �Figure 1� consists of an analysis

Figure 1. This diagram shows an overview of the OASISsystem and how data flows between different systemcomponents.

Castano et al.: OASIS: Onboard Science System • 381


capability for identification and prioritization of dataand a planning and scheduling component to enableresponse to unanticipated opportunities. The analysiscapability includes feature extraction as well as pri-oritization.

• Feature detection: Enables extraction of fea-tures of interest from collected images of thesurrounding terrain. This module both lo-cates rocks in the images and extracts rockproperties �features� including shape, tex-ture, and albedo. It also includes atmosphericanalysis.

• Data analysis and prioritization: Analyzesthe extracted features to assess the scientificvalue of the data and to generate new sciencegoals.

• Planning and scheduling: Enables dynamicmodification of the current rover commandsequence �or plan� to accommodate new sci-ence requests from the data analysis and pri-oritization module. A continuous planningapproach is used to iteratively adjust the planas new goals arise. Opportunistic goals maybe added to the plan as long as resource andother operational constraints are still met andall higher priority goals can be achieved.

2.1. Feature Detection

In this work, we will describe our methods for theanalysis of scientific data acquired by a rover. Ourtechniques are applicable to a wide range of datamodalities; however, our initial focus is on imageanalysis as images are commonly available and pro-vide significant information about a scene.

The first step in image evaluation is the extrac-tion of features of interest from the scene depicted.Features may be extracted from the original imageor from a region after segmentation. Currentlywithin OASIS there are two image segmentationmodules and three feature extraction modules. Eachof these modules applies general data analysis prin-cipals to identify and characterize image featuresthat are representative of distinct scientific phenom-ena.

2.1.1. Image Segmentation

2.1.1.1. Sky detector

Both to constrain geologic analysis to the groundand atmospheric analysis to the sky, it is beneficial tobe able to distinguish the sky from the ground, i.e.,to identify the horizon. Our approach to sky detec-tion is based on region growing and is composed offour steps �Figure 2�. First, we assess whether sky ispresent in the image or not. If so, seeds, areas of lowvariance, are identified. The lower bound of the skyis delineated by performing edge detection on thevariance image. The seeds are then grown down tothe variance edge. Finally, gaps in the sky region arefilled. During this process we determine the skyline�the pixels at the interface of sky and ground� andthe horizon �the image row below which there areno sky pixels�. This algorithm had approximately a90% accuracy in testing on 301 MER Opportunityrover images �errors were evenly distributed be-tween false positives and misses�.

2.1.1.2. Rock Detector

Detecting rocks in images is a valuable capability forautonomous planetary science. Rocks are excellenttargets for compositional analysis with spectrom-eters. Their shape, size, and texture hold a wealth ofgeologic information. Computing the locations anddistributions of rocks facilitates autonomous roverfunctions like adaptive target selection �Castanoet al., 2006�, selective image return �Castano et al.,2003�, and autonomous site characterization �Th-ompson et al., 2005a�.

Research in the past decade has produced a va-riety of strategies for detecting rocks. These include

Figure 2. Flow diagram for sky detection algorithm.



stereo based techniques for finding rocks based ontheir protrusion from the ground plane, edge-basedmethods that find closed contours, template-basedmethods that look for characteristic pixel patterns,and methods that detect rocks using their shadows.Each approach has advantages for different condi-tions and mission requirements. For a formal com-parison of the performance of these algorithms, seeThompson & Castano �2007�.

From the many sensor modalities expected to beavailable on a rover, the rock detection algorithm ini-tially used by the OASIS system is based on analysisof intensity of single grayscale images. The use ofstereo is highly desirable for detection of large rocksbut is unsuitable for detection of smaller rocks; fur-ther, it cannot be applied to images that do not comein stereo pairs, as is the case for the MER micro-scopic imager �MI�. In contrast, rock detection onsingle grayscale images applies directly to analysisof low resolution hazard camera �hazcam�, highresolution navigation camera �navcam�, any spectralband �or combination� of the Panoramic camera�Pancam�, and the MI camera. The detection of rocksis carried out by finding closed shapes in the image.The image is initially normalized, filtered with anedge preserving smoother �Tomasi & Manduchi,1998� and its edges are enhanced using unmasksharpening. The edges of the resulting image are de-tected using both a Sobel and a Canny edge detec-tors �Trucco & Verri, 1998�. For each result, we searchfor closed shapes �which presumably correspond torelatively small homogeneous regions� using anedgewalker. The results from both detectors arecombined and output as a list of contours of theidentified shapes. A flow diagram is shown in Figure3. In testing on a set of 65 MER Spirit Pancam im-

ages, 92% of the regions identified as rocks correctlywere rocks. In a direct comparison to several othermethods using MER Navcam images, this algorithmwas at 90% while other rock identification algo-rithms were predominantly in the 65%–75% range.Testing with the FIDO field test rover in the JPLMars Yard has yielded similar results. While havinga very low false alarm rate, making it useful for au-tomated target selection �see Section 3.3�, it has ahigh miss rate �one of the lowest recall rates� and isbest not used for estimating rock abundances.

2.1.2. Feature Extraction

2.1.2.1. Cloud Detector

Clouds are an atmospheric phenomena observed bythe MER rovers on Mars. In detecting clouds, it isassumed that large variations in the intensity of thesky in the image correspond to clouds. Our ap-proach to automating the detection of clouds is tofirst locate the sky �equivalently, the horizon� in animage and then determine if there are high varianceregions in the sky �Figure 4�. This algorithm, whichoperates on individual images �Figure 5�, achievedover 93% accuracy in testing on 210 hand-labeledimages taken by the Mars Exploration Rover Oppor-tunity. There were three misses �false negatives� andeleven false positives. All of the three misses werelabeled as a possible cloud �low confidence� by thescientist performing the labeling. No high confi-dence clouds were missed. For more details on thealgorithms and experimental testing, see Castano

Figure 3. Flow diagram for rock finder algorithm.

Figure 4. Flow diagram for cloud detection algorithm.



et al. �2006� and Castano et al. �2007�. Both the clouddetector and the dust devil detector described in thenext section were uploaded on to the Mars Explora-tion Rovers. Preliminary experiments with the algo-rithms onboard have been successful.

2.1.2.2. Dust Devil Detector

A second type of dynamic atmospheric phenomenaof interest on Mars is dust devils. The two mostcommon methods for detecting dust devils are thecomparison of two or more spectral bands of thescene and the motion detection using a temporal se-quence. We selected the latter as it has application tograyscale as well as color imagery. In theory, detect-ing rapid motion in the scene is not equal to detect-ing dust devils. In practice, changes in a sequence ofimages taken over a short time period time at ascene on Mars are from dust devils. Dust devils arehigh dust opacity features on a dusty backgroundand often have a faint signature in an image �Figure6�. The main challenge is to detect these often subtlefeatures in the presence of significant image noise.

The algorithm �Figure 7� consists of a prepro-cessing step to reduce image noise followed by animage averaging. The difference between the aver-age image and the input or test image is then com-puted. Noise effects are removed from the differenceimage and blob detection is performed on the re-maining differences. A buffered bounding box isformed around each detection to ensure the full dustdevil is captured. The dust devil algorithm wastested on 385 images, divided into 25 image se-quences, acquired by the MER Spirit rover. The se-quence lengths varied between 6 and 20 images. Thealgorithm achieved an 85% accuracy rate when the

average image was determined using sets of fourcontiguous images. For more details on the algo-rithms and experimental testing, including resultswith different window sizes for the average image�i.e., different number of images used to form theaverage image�, see Castano et al. �2006� andCastano et al. �2007�.

2.1.2.3. Rock Properties

The primary focus of OASIS feature extraction andsystem testing has been on geological features, spe-

Figure 5. An example of cloud detection. Left image isthe original image and right image is the result of thecloud detection algorithm. Figure 6. �Top� result of motion detection in an image.

Two of the dust devils are observable �third and fifth box�,while the other three occur later in the sequence.�Bottom� contrast adjusted image highlighting dustdevils in scene. �enhanced image source:http://www.lpl.arizona.edu/ lemmon/mer_dd.html�.

Figure 7. Flow diagram for dust devil detectionalgorithm.



cifically rock properties, rather than atmosphericphenomena. In terrestrial studies, the physical prop-erties of rocks and soils are analyzed for informationregarding their geologic history. Physical propertiessuch as rock/grain size, shape, composition, sorting,and texture are common features of rocks and soilsthat can be used to infer their origin and transporta-tion history. Using image data, the properties thatOASIS currently estimates are albedo, texture, size,and shape.

The albedo of a rock is an indicator of the reflec-tance properties of a surface. OASIS measures al-bedo by computing the average gray-scale value ofthe pixels that comprise the image of the rock. Thereflectance properties of a rock provide informationabout its mineralogical composition. Shadows andsun angle can both affect the gray-scale value of apixel. Although this can be corrected by using therange data along with knowledge of both the sunangle and the camera orientation, the current systemdoes not address these specific issues.

OASIS uses Gabor filters to estimate the visualtexture of observed rocks �Castano, Mann, & Mjol-sness, 1999�. Visual texture can provide valuableclues to both the mineral composition and geologicalhistory of a rock as well as soil �Mahmood, Mitchell,& Carrier, 1974�.

One of the important properties of rocks on thesurface is their size. Size can be used to identify sort-ing and geologic contacts. We model rocks as el-lipses �if no range data is provided� or ellipsoids �ifrange data is available�.

Inherent shape is another important and geo-logically informative feature of rocks. Although theshape of a rock is complex and often difficult to de-scribe, significant geologic information can be ex-tracted to better understand provenance �source ofmaterial� and environmental conditions. Variousshape parameters are used to classify rocks in terres-trial studies �Wadell, 1932�, including elongation �oraspect ratio�, ruggedness �or angularity�, and surfacearea. In OASIS, an ellipse is fit to the outline of therock �Halir & Flusser, 1998�. The eccentricity of thefit ellipse, as well as the error, is computed �Fox,Castano, & Anderson, 2002�. In addition, the angu-larity of each rock is assessed using a measure ofruggedness �Hentschel & Page, 2002�.

2.1.2.4. Boundary Detection

A rover on a planetary surface, such as Mars, repre-sents a modern version of a field geologist on Earth.

One of the most important tasks for a geologist is todevelop an understanding �spatial and temporal� ofthe field area. For most mapping projects, this in-volves going into the field area, identifying whattypes of rocks are present, identifying key landformsand landscapes, mapping geological contacts orboundaries, developing a geologic map, and creat-ing a model based on historical interpretations andthe dynamic processes that have shaped the land-scape.

In the field, rocks exposed at the surface are theonly record of the surface history; their physical ap-pearance and location testify to their environmentalsettings in which they formed. Therefore, to gain anunderstanding of the basic geologic history of a re-gion, one must be able to identify where the rocks onthe surface originated from. To do this, one mustidentify and map geological contacts/boundaries inthe field. The identification and mapping of geologi-cal boundaries can range from simplistic boundarydetection, such as hills, plains, and river channels, tocomplex identification of different rock and clasttypes, to erosional and depositional histories of thelandscape. For the field geologists, detail examina-tion of units provides the data for mapping; for therover, it is the critical hardware interface in conjunc-tion with the scientist back on Earth that collects thedata. For future long-range rovers on a planetarysurface, it is critical for the rover’s on-board soft-ware to be capable of identifying simple boundariestransitions during long traverses.

2.2. Data Analysis/Prioritization

After extraction of features, the information is usedto effect a change either in the downlink data queueor in the rover activities. In this section, we describefour methods used to assess the priority of the dataand the information it contains. The first method isused for clouds and dust devils while the remainingmethods are currently used with rocks in OASIS.

2.2.1. Detected Event

The first type of reaction is to set a flag when anevent of interest is detected. In the case of dust dev-ils and clouds, the detection of an event in an imageis used to flag that image or image sequence fordownlink. Cloud detection images without cloudsneed not take up downlink bandwidth. Similarly,dust devil image sequences that do not contain dust



devils do not need to be transmitted to Earth. Otherpotential reactions include collecting an image se-quence of the cloud or collecting additional types ofdata when a dust devil is detected.

2.2.2. Key Target Signature

The second technique recognizes pre-specified targetsignatures that have been identified by the scienceteam as data of high interest. This prioritizationmethod enables scientists to efficiently and easilystipulate the value and importance to assign to eachfeature. In OASIS, this method is used to prioritizerocks as a function of the distance of their extractedfeature vector from the specified weighted featurevector. Scientists are given two ways to set the targetsignatures that will determine how the rocks areranked. In the first method, the scientist can directlyset the importance of specific feature values. For ex-ample, the scientist may choose to prioritize rocksbased on two aspects of their shape, such as eccen-tricity and ellipse fit. The second manner in whichscientists can specify a target signature is by select-ing a rock with interesting properties from the set ofalready identified rocks. Rocks that resemble thisparticular rock in the selected properties are given ahigh priority. While we have emphasized rocks, onecould envision using this method with other fea-tures, including identification of clouds or dust dev-ils with particular properties.

2.2.3. Novelty Detection

The third technique, novelty detection, identifies un-usual signatures that do not conform to the statisti-cal norm for the region. We have developed threemethods for detecting and prioritizing novel signa-tures, representing the three dominant flavors of ma-chine learning approaches to novelty detection:

• distance-based,• probability-based �i.e., “generative”�,• and discriminative.

The first novelty detection method is a distance-based k-means clustering approach. Again, we haveapplied this to rock data. Initially, all available rockdata is clustered into a specified number �k� ofclasses. The novelty of any rock is then the distanceof the rock feature vector to the nearest center of any

of the k clusters. The greater the rock’s distance is tothe nearest center, the higher the novelty ranking as-signed to the rock.

The second technique is a probability-basedGaussian mixture model, which attempts to modelthe probability density over the feature space. In thisapproach, the novelty of a rock is inversely propor-tional to the resulting probability of that rock beinggenerated by the model learned on previous rockdata.

The final method is a discrimination-based ker-nel one-class classifier approach. Here we treat allprevious rock data as the “positive class” and learnthe discriminant boundary that encloses all that datain the feature space. We essentially consider the pre-vious rock data as a cloud scatter in someD-dimensional space, where D is the number of fea-tures. The algorithm learns the boundary of thatcloud, so that future rock data that falls farther out-side the cloud boundary is considered more novel.

2.2.4. Representative Sampling

The last prioritization algorithm, known as represen-tative sampling, prioritizes data for downlink by en-suring that representative rocks of the traversed re-gion are returned. One of the objectives for rovertraverse science is to gain an understanding of theregion being traversed. To meet this objective, thedownlink back to Earth should include informationon rocks that are typical for a region, and not justinformation on interesting and unusual rocks. A re-gion is likely populated by several types of rockswith each rock type having a different abundance. Ifuniform sampling is employed for downlink imageselection, as opposed to our autonomous on-boardselection process, the downlinked set will be biasedtowards the dominant class of rock present. Thissituation may result in smaller classes not being rep-resented at all in the downlinked data.

To provide an understanding of the typical char-acteristics of a region, rocks are first clustered intogroups with similar properties. The data is then pri-oritized to ensure that representative rocks fromeach class are sampled. The rocks are clustered intogroups based on the features extracted from the im-age data for each rock. To determine the classes, theproperty values are concatenated together to form afeature vector, and a weight is assigned to the im-portance of each property. Different weight assign-ments can be used as a function of the particular



properties that are of interest. For example, albedoand texture are typically used to distinguish types ofrocks, but rock size may be used if sorting is of in-terest. Unsupervised clustering is then used to sepa-rate the feature vectors into similar classes. We cur-rently employ k-means due to its relatively lowcomputational requirements, although any unsuper-vised method could be used. For each class of rocks,we find the most representative rock in the class, i.e.,the single rock in any image that is closest to themean of the set. We give a high priority to the imagecontaining this rock. The optimal number of classescan be determined using cross-validation techniques�Smyth, 1996�.

2.3. Planning and Execution

When the data analysis software identifies sciencetargets of interest �e.g., a novel rock�, a science alertis generated. This results in a new science goal beingpassed to the planning and scheduling modulewhich determines if the new goal can be accommo-dated. If it can be, the current rover command se-quence is modified to collect new science data.

The OASIS planning and execution module �Es-tlin et al., 2007� is intended to run with little commu-nication with ground. It accepts new science goalsand then modifies the current rover command se-quence �or plan� to try and achieve as many of thegoals as possible while still respecting relevant stateand resource constraints. This module also executesthe current rover plan by dispatching commands tothe rover’s low-level control software and monitor-ing relevant state and resource information to iden-tify potential problems or opportunities. If problemsor new opportunities are detected, the system is de-signed to handle such situations by using replanningtechniques to add, move, or delete plan activities.

2.3.1. Challenges for Onboard Planning andExecution

Rovers equipped with on-board planning and ex-ecution capabilities have the potential for increasingscience return not only by dynamically handling op-portunistic science events but also by making se-quencing decisions on-board with access to currentstate and environment information. There are, how-ever, a number of challenges in providing softwareto support these types of autonomous operations.

To generate and/or modify its own command

sequence for carrying out a set of science goals, theonboard software must reason about a rich model ofscience and engineering activities and the corre-sponding resource and temporal constraints. For ex-ample, the software has to predict power consump-tion of variable duration activities such asdownlinks and traverses, keep track of availablepower levels, and ensure that generated plans donot exceed power limitations. Science activities mayalso have varying priorities that indicate their over-all mission value. On-board planning and executionsoftware must reason about these priorities andhandle new science opportunities in a dynamic andefficient manner.

Sequence generation for rover surface missionsalso raises a number of interesting challenges re-garding spatial reasoning capabilities. One of thedominating characteristics of rover operations istraverses to designated waypoints and science tar-gets. This element is especially important in futuremissions that intend to explore large geographic ar-eas. On-board planning and execution softwareneeds to coordinate with several levels of rover navi-gation software to generate an efficient and achiev-able rover plan. In addition, it is difficult to predictduration and resource usage of rover navigation op-erations since rovers are traveling over unknownterrain. A large part of a rover schedule consists ofrover moving to different target locations. If timeand resource predictions for these moves are inaccu-rate, the onboard software must be able to continu-ously modify the schedule in order to accommodatethe new information.

Further, on-board autonomy software must ro-bustly handle the inherent uncertainty in rover poseestimation. The estimation of rover position is oftena constant source of error. The Sojourner Rover pro-duced a position error of roughly 5%–10% of dis-tance traveled and an average heading drift of 13°per day of traverse �Mishkin et al., 1998�. The MERrovers have seen large variations in position error,including Opportunity experiencing a slippage ofclose to 100% when the rover was caught in a sanddune �but thought it drove 40 m�. Though visualodometry software has helped the MER rovers per-form more accurate pose estimation than wheelodometry alone, it is typically only run in a limitedfashion due to the additional time requirements.



2.3.2. Planning and Execution System Description

Planning and execution capabilities in OASIS areprovided by an integration of the CASPER �Continu-ous Activity Scheduling, Planning, Execution andRe-planning� continuous planner and the TDL �TaskDescription Language� executive system �Estlinet al., 2007�.

In our system framework, CASPER and TDLhandle the following functionality:

• Creating an initial plan based on an input setof goals,

• maintaining resource, temporal and otherrover operability constraints,

• executing a plan by interacting with basicand low-level rover control functionality�e.g., navigation, vision�,

• monitoring plan execution to track plan ac-tivity and goal status,

• dynamically modifying the current planbased on plan activity, state, and resource up-dates,

• performing plan optimization to reasonabout soft constraints and goal priorities,

• handling dynamically identified sciencegoals �called science alerts� that are generatedthrough onboard data analysis.

Planning and scheduling capabilities for OASISare provided by the CASPER continuous planningsystem �Chien, Knight, Stechert, Sherwood, & Ra-bideau, 2000�. Based on an input set of science goalsand the rover’s current state, CASPER generates asequence of activities that satisfies the goals whileobeying relevant resource, state and temporal con-straints, as well as operation �or flight� rules. Plansare produced using an iterative repair algorithm thatclassifies plan conflicts and resolves them individu-ally by performing one or more plan modifications.1

CASPER also monitors current rover state and theexecution status of plan activities. As this informa-tion is acquired, CASPER updates future-plan pro-jections. Based on this new information, new con-flicts and/or opportunities may arise, requiring theplanner to replan in order to accommodate the un-expected events. An example of a plan in the

CASPER GUI that was executed during a roverdemonstration is shown in Figure 8.

The executive functionality in OASIS is per-formed by the TDL executive system �Simmons &Apfelbaum, 1998�. TDL was designed to performtask-level control for a robotic system and to medi-ate between a planning system and low-level robotcontrol software. It expands abstract tasks into low-level commands, executes the commands, and moni-tors their execution. TDL also provides direct sup-port for exception handling and fine-grainedsynchronization of subtasks. TDL is implemented asan extension of C++ that simplifies the developmentof robot control programs by including explicit syn-tactic support for task-level control capabilities. Ituses a construct called a task tree to describe the treestructure that is produced when tasks are brokendown into low-level commands.

Currently, OASIS has a separate planner and ex-ecutive and, thus, this framework does share simi-larities to other three-layer architecture approaches�Gat, 1991; Bonasso et al., 1997; Alami, Chatila, Fl-eury, Ghallab, & Ingrand, 1998�. However, as com-pared to these approaches where planning is typi-cally done in a batch fashion and takes on the orderof minutes to hours, this integration uses a continu-ous planning approach, where plans are updatedand repaired in a matter of seconds. This enables theuse of planning techniques at a finer timescale fortracking the progress of plan execution, quicklyidentifying potential problems in future parts of the

1Definitions of plan activities, goals, resources, and states, as wellas the aforementioned constraints and rules, are encoded in aCASPER plan model.

Figure 8. Sample rover plan displayed in planner GUI.Plan activities are shown in upper portion of window,where bars represent the start and end time of each activ-ity. State and resource timelines are shown in bottom por-tion of the screen and show the effects of the plan as timeprogress. Time is depicted as advancing from left to right.



plan, and responding accordingly. As we expect mi-nor portions of the plan to change frequently, we usea lightweight plan runner to dispatch activities tothe executive a few seconds before the task’s sched-uled start time. This approach differs from the morecommon batch approach of turning the entire planover to the executive for execution. Executive tech-niques are then used in only reactive situations or attimes where procedural reasoning is preferred.

Another related approach directed towardsrover command generation uses a contingency plan-ner �CPS� to schedule rover operations where thegenerated plan allows both temporal flexibility andcontains contingency branches at points where therover activity is predicted to have above a certainprobability of failure. �Bresina, Golden, Smith, &Washington, 1999�. Contingent command sequences�or plans� are produced on the ground and only theexecutive, which executes the sequence, is on-boardthe rover. This approach enables planning to be per-formed without the strict processing constraints thatare often present when running on-board a space-craft. However, typically, only a limited number ofcontingencies can be preplanned due to the size andcomplexity issues. Further, it may be difficult to pre-dict probable points of plan failure, especially whenrovers drive through unknown terrain. The OASISsystem approach provides more flexibility to handlenew situations since the planner is on-board and canrespond to a much large number of problem �or for-tuitous� situations. Further, other work has shownthe feasibility of using an automated planner on-board a spacecraft, even when only very limited pro-cessing and memory are available �Chien et al.,2005�.

2.3.2.1. Science Alerts

To handle opportunistic science, we enabled the OA-SIS planning and execution module to recognize andrespond to science alerts, which are new science op-portunities detected by on-board data-analysis soft-ware. For example, if a rock is detected in navigationimagery that has a previously unseen shape or tex-ture, a science alert may be generated to take addi-tional measurements of that rock.

Science alerts can have different levels of reac-tion from the planning and execution system. Themost basic reaction is to adjust the rover plan so thatthe rover holds at the current position and theflagged data is sent back to Earth for further analysis

at the next communication opportunity. The nextlevel of reaction is to collect additional data at thecurrent site before transmitting back to Earth. A fur-ther step is to have the rover alter its path to getcloser to objects of interest before taking additionalmeasurements. These operations provide new datathat could not be obtained through analysis of theoriginal image. Each of these levels of reaction hasbeen demonstrated for the OASIS system in simula-tion and with rover hardware �more details aregiven in Section 4�.

How the plan is modified to accommodate thealert depends on the type of alert that is considered.When a science alert is received that requires hold-ing at the current position until data is communi-cated with Earth �called a stop and call home alert�, thesystem alters the plan to remove or abort any activi-ties that are not critical and wait for the next com-munication opportunity. If activities are currently ex-ecuting, the planner requests the executivecomponents of OASIS to abort them. If activities arescheduled in the future, the planner deletes themand resolves any inconsistencies created by these de-letions.

To handle a science alert that requests additionalmeasurements �called a data sample request alert�, theplanner must generate a plan that achieves the newgoals without deleting existing activities or causingconflicts that cannot be resolved �e.g., schedulingmore activities than can be executive over a certaintime window�. Data sample requests are also as-sumed to have an assigned priority, which is repre-sentative of their scientific value. This priority is cur-rently assigned by the onboard data analysis system.A simple linear priority scheme is used to reflect arequest’s value and is chosen by the data prioritiza-tion algorithm being employed. For example, if theanalysis algorithm is looking for rocks with a certainset of properties, then rocks with features thatclosely match the target property set will be as-signed high priorities.

To handle a data sample request, the plannermust be able to add a new science observation and anew move command to correctly place the rover inposition to take the observation. This process is donethrough the use of planning optimization capabili-ties. Because it may not be possible to accommodate



all alerts,2 a science alert is represented as an op-tional planning goal with a particular priority. As anoptional goal, its achievement is not mandatory butmay improve the plan’s optimization score if in-cluded in the plan. Before attempting to handle ascience alert, CASPER protects the current plan bysaving a copy before optimization. If CASPER canhandle a new science alert �e.g., by adding addi-tional science measurements� without causing othernegative affects, such as resource over-subscriptionsor the deletion of ground-specified �mandatory� sci-ence goals, then the new plans optimization score isevaluated. If the new plan has an optimization scorethat is higher than the original plan, then the newplan that accommodates the science alert is used. Formore information on how CASPER reasons aboutplan optimization please see �Rabideau, Engelhardt,& Chien, 2000�. CASPER also is given a time limit tosearch for a new plan that handles the science alert.If a new plan is not found under the time limit, theprevious plan is restored and the science alert re-mains unsatisfied. This time limit can be adjustedbased on user preferences and available processingtime.

2.3.3. CLARAty Robotic Architecture

In order to test OASIS with rover hardware, theplanning, scheduling, and execution component isintegrated with the Coupled Layered Architecturefor Robotic Autonomy �CLARAty� �Nesnas et al.,2003�, which is being developed at JPL in response to

the need for a robotic control architecture that cansupport future mission autonomy requirements.CLARAty is a unified and reusable robotic softwarearchitecture that provides a large range of basic ro-botic functionality and simplifies the integration ofnew technologies on different robotic platforms.Through CLARAty, various OASIS componentshave been tested with several hardware platformsincluding Rocky 7, Rocky 8, and FIDO �shown inFigure 9�. For the testing reported in this paper, theFIDO rover was primarily used and the OASIS rock-finder and planning and execution software were di-rectly integrated with CLARAty and handled inter-action with other key CLARAty elements that wererequired to run with rover hardware. Other pieces ofOASIS �such as feature extraction� are in the processof being integrated but were not integrated for thesetests. Through this integration, the OASIS system in-teracted with several technologies provided byCLARAty, which operate onboard the rover. Thesesoftware components are locomotion, position esti-mation, navigation, and stereo vision processing.

The locomotion module coordinates a set of mo-tors to steer and drive the rover’s wheels, enablingthe rover to move across the terrain. The FIDO rover,which was used in these tests, is a six-wheeled ve-hicle with all-wheel steering and driving enablingthe rover to turn in place, drive in straight lines,drive in arcs, and perform crab maneuvers in whichthe rover moves in one direction while facing an-other.

As the rover traverses across the landscape, po-sition estimation software attempts to estimate therover’s current location and orientation relative to aglobal and a local reference frame. Varying texture inthe sand results in different amounts of wheel sink-

2In system testing, a number of different situations have beenevaluated including allowing the data analysis component to gen-erate a large set of data sample requests that cannot all be accom-modated due to plan limitations on resources and time.

Figure 9. Rocky 8 rover �left�, FIDO rover �middle�, Rocky 7 rover �right�.



age which, in turn, results in variance in the effectivewheel radius of the rover. In addition, the rover of-ten encounters rocks of varying sizes, shapes, andtextures which results in wheel slippage as it drivesover the rocks. The position estimation algorithm inthe current system uses an IMU �Inertial MeasuringUnit� to estimate rover attitude �roll, pitch, andheading� and wheel odometry to estimate linear ve-locity. The IMU helps detect slippage that could notbe detected with wheel odometry alone.

Navigation software is used to guide the roverto a goal location while avoiding obstacles. OASISuses the Morphin navigation system developed atCMU �Urmson et al., 2003�. As part of the hazardavoidance algorithm, the navigator acquires stereoimages from the rover hazard cameras to detect ob-stacles in the rover’s path. The front camera imagesare also passed to the OASIS rock detection process,running in a separate thread.

Both the navigator and the OASIS rock detectionalgorithm make use of the JPL stereo vision packageto acquire range data for the hazard camera images.The navigator uses stereo processing to generate apoint cloud representing the terrain around therover. OASIS uses range data to identify the loca-tions of identified rocks. Rocks are first located in acoordinate frame relative to the center of the roverand then translated into the global reference frameusing the current position estimate.

3. ONBOARD SCIENCE FOR MAXIMIZINGMISSION SCIENCE RETURN

On-board science or science autonomy refers to thecapability of analyzing the scientific content of dataon-board a spacecraft and using this information toautonomously take intelligent actions. The capabilitycan be used to increase the science return of a missionby ensuring that high science content data reachesscientists on Earth. With the constrained computingresources, time, power, and downlink bandwidth onplanetary rovers, onboard science can be used for ef-ficient allocation of a rover’s time and bandwidth re-sources. The OASIS system for on-board science canachieve this through opportunistic science during atraverse, campaign science, end-of-day science, andprioritization of data for downlink.

3.1. Opportunistic Traverse Science

One mission concept for future rover exploration isto focus the scientific investigation on multiple sci-ence sites that are located several rover traverse daysaway from each other. This scenario allows for adeep and concentrated exploration of each sciencesite; however, in order to expeditiously drive therover to each of the sites, scientists cannot performdetailed examinations of the terrain between eachsite. This is where traverse science, the capability ofautonomously studying the terrain during the longtraverse, can be beneficial.

OASIS can be used to identify and react to sci-ence opportunities along a traverse. As explained inthe previous sections, the system can identify fea-tures that scientists have specified as important aswell as novel features �see, for example, Figure 10�and then, if resources are available, autonomouslytake additional measurements of that feature.

Figure 10. Image credit NASA/JPL/Cornell, JPL archivenumber PIA07269. On January 2, 2005, NASA’s Mars Ex-ploration Rover Opportunity found this iron meteorite onthe surface of Mars—close to the heat shield that thespacecraft dropped on its descent to the surface. OASIScan train the rover to look for meteorites, or other types ofrock, using the key target signature data analysistechnique.



3.2. Campaign Science

OASIS can be used not only during a traverse, butalso for campaigns to monitor for science events orfeatures of interest. As previously described, proto-type algorithms for dust-devil detection and clouddetection that were developed for OASIS have beenrefined for MER and integrated into the MER flightsoftware. These algorithms allow selective transmis-sion of data containing the features of interest.Ground testing indicates that a 4X increase in thenumber of images with science features can be ex-pected through the use of these algorithms.

3.3. Automated Target Selection/End-of-DayScience

A third benefit of onboard science data analysis isthe identification of targets both at the end of theday and during the course of a drive. A number ofrover remote sensing instruments have a very nar-row field-of-view and, thus, require selection of spe-cific focused targets for sampling. Such instrumentsinclude mini-TES �thermal emission spectrometer�,LIBS �laser induced breakdown spectrometer�, andinfrared point spectrometers. The typical scenariofor selecting targets for these instruments is tomanually identify the targets using data that hasbeen previously downloaded. This means that tar-gets can only be selected from the site for which datahas already been downloaded the night before �thus,at the end of the day, the rover sits and waits untilthe next morning for instructions on which rocks tosample�.

Rather than waiting until the next day, somesamples could be collected by “blindly” targetingthe instrument after a traverse day. However, byanalyzing image data on-board, targets for these in-struments can be identified automatically. As part ofthe OASIS system, we have implemented a methodfor automatically selecting rock targets for samplingat the end of a traverse. This could be used, for ex-ample, on the Mars Science Laboratory �MSL� to se-lect targets for the ChemCam instrument �which in-cludes a LIBS� to sample.

The approach is to first identify the rocks in thescene using the rock finder in the feature extractioncomponent. Points on these identified rocks are thenselected for targeting �Figure 11�.

3.4. Onboard Prioritization of Data

Finally, one of the primary motivations for the de-velopment of OASIS was to prioritize data fordownlink. In this application, OASIS analyzes datathe rover gathers, and then prioritizes the data basedon criteria set by the science team. Three prioritiza-tion methods have been developed:

• Representative sampling �this method ensuresthat at least one image of every rock type en-countered that sol is returned to Earth�,

• Key target signature �the rover looks forscience-specified rock types and gives imageswith those rocks a high priority for down-link�, and

• Novelty detection �if the rover sees a rock typethat it has not seen before, it assigns thatrock’s image a high downlink priority�.

At the next opportunity for transmitting databack to Earth, the data is already prioritized—ensuring that the most valuable data is sent first.

4. OASIS SYSTEM TESTING

To evaluate our system, we have performed a seriesof tests over several years both in simulation and us-ing rover hardware in the JPL Mars Yard. These tests

Figure 11. OASIS selects five potential targets for theChemCam instrument to sample. Autonomously selectingtargets vs blind sampling should greatly increase thechances of accurately targeting a rock.



covered a wide range of scenarios that included thehandling of multiple, prioritized science targets, lim-ited time and resources, opportunistic science events,resource usage uncertainty causing under- or over-subscriptions of power and memory, large variationsin traverse time, and unexpected obstacles blockingthe rover’s path.

Our testing scenarios typically consisted of anumber of science targets specified at certain loca-tions. A map was used that would represent a samplemission-site location where data would be gatheredusing multiple instruments at a number of locations.Figure 12 shows a sample scenario that was run aspart of these tests. This particular map is of the JPLMars Yard. The prespecified science targets repre-sented targets that would be communicated by sci-entists on Earth. These targets were typically priori-tized and for many scenarios constraints on time,power, or memory would limit the number of sciencetargets that could be handled. A large focus of thesetests was to improve system robustness and flexibil-ity in a realistic environment. Towards that goal, weused a variety of target locations and consistently se-lected new science targets and/or new science targetcombinations that had not been previously tested.

Another primary scenario element was dynami-cally identifying and handling opportunistic scienceevents. For these tests, we have concentrated on aparticular type of event, which was finding rocks

with distinct features. Specifically, we have tested thefeature detection of rocks with certain albedo levelsand shape features. These settings were an exampleof using the data analysis algorithm for target signa-ture, where a particular terrain signature is identifiedas having a high interest level. If rocks were identifiedin hazard camera imagery that had a certain interestscore, then a science alert was created and sent to theplanner. Science alerts would typically come in dur-ing rover traverses to new locations, but it was alsopossible for them to come in while the rover was at ascience target location due to a small lag caused byimage processing time. If a science alert was detected,the planner attempted to modify the plan so an ad-ditional image of the rock of interest was acquired. Asample image that was taken in response to a sciencealert is shown in Figure 13.

4.1. Testing in Simulation

Since testing with rover hardware can be an expen-sive and time-intensive process, we ran a large num-ber of tests for the planning and execution modulein simulation using a relatively simple simulator.This simulator could execute rover sequence com-mands and simulate their effects at a coarse level ofgranularity. For instance, the simulator handleditems such as rover position changes and energy us-age over straight-line movements, but did not simu-late obstacle avoidance or rover kinematics. Anothercapability that was used in simulation was trigger-ing science alerts at preset or random times. This

Figure 12. Sample plan shown in the Grid VisualizationTool �GriViT�. Green lines show the planned path of therover, blue lines show the real path, and pink lines showthe path that is currently executing.

Figure 13. Sample image that was taken in response to ascience alert indication identification of a rock with whitealbedo on the JPL FIDO rover.



capability helped in evaluating the software’s capac-ity to correctly handle different opportunistic sciencescenarios.

To easily run and evaluate large numbers oftests, we also invested in a testing infrastructure,which allowed tests to be run offline and automati-cally gathered statistics, including items such asnumber of plan conflicts found and resolved, plangeneration and replanning time, number of goalssatisfied, overall plan traverse distance, and plan op-timization scores. This testing infrastructure also en-abled the automatic creation of mpeg movies thatshowed plan changes using snapshots of a plan vi-sualization tool. This tool showed the results of plangeneration and execution on an overhead map of theworld, and could be used for both simulated andhardware testing. An example plan snapshot dis-played by this tool is shown in Figure 12. Planningand execution results were evaluated by examininggathered statistics and by viewing created mpegs toflag incorrect or nonoptimal behavior.

4.2. Testing with Rover Hardware

In addition to testing in simulation, a large numberof tests with all OASIS software modules were runin the JPL Mars Yard �shown in Figure 14� using dif-ferent rover hardware platforms. For the past yearand for the final demonstration, the FIDO rover�shown in Figure 9� was used for the majority oftests. System setup and the specific rover controlsoftware used for capabilities such as navigation andvision was described in Section 2.3.3.

Tests in the Mars Yard typically consisted of20–50 m runs over a 100 m2 area with many ob-stacles that cause deviations in the rover’s path.Most rocks in the Mars Yard are dark in color, thus,for our albedo testing, we brought in a number ofwhiter rocks to trigger science alerts during rover

traverses. Science measurements using rover hard-ware were always images, since other instrumentswere not readily available �such as a spectrometer�.However, different types of measurements were in-cluded when testing in simulation.

4.3. Lessons Learned

Integrating and testing with hardware and requiredcontrol software �e.g., for navigation� introduced anumber of challenges. In this section, we will brieflyoutline a few of the key lessons learned during thisprocess. One primary challenge was that the testingand demonstration of our on-board science systemrequired a number of underlying components toproperly perform and support our software. An im-portant lesson for running testing with rover hard-ware is that a significant amount of time must beallocated to tune and test supporting software andhardware components. Though the majority of sup-porting components were provided through CLAR-Aty, many had not been run through extensive test-ing in the environment that we wished todemonstrate OASIS. Some key components that wespent a large amount of time testing for our use in-clude the FIDO rover cameras, navigation software,and position estimation software. This process wasfurther complicated when supporting software hadlarge numbers of tunable parameters and, sincesome supporting software had only been tested incertain environments or exercised in unit testing, ourtesting process also uncovered several problems �orbugs� in supporting software that had not been pre-viously discovered. In the future, we hope to havemore access to the people who developed the sup-porting technology and have direct support in cor-rectly tuning their application for our needs. How-ever, since this will not always be the case, we expectfuture work with hardware testing to consistentlyrequire time for tuning supporting technology.

Another lesson learned was the value of per-forming a large number of tests in simulation, evenwith a relatively simple simulator. To test the abilityof the planning system to handle science alerts un-der varying conditions, we invested in a testing in-frastructure that allowed large numbers of tests to berun offline using a simple simulator. This simulatortracked items such as resource usage and rover po-sition, but did not simulate higher fidelity itemssuch as obstacle avoidance or stereo image process-ing. During these tests certain factors were varied

Figure 14. The JPL Mars Yard with terrain of variousdifficulties.



such as the number of science alerts, available re-sources, rover speed, etc. This allowed the planningand execution software to be exercised on a largevariety of situations and allowed many bugs to becaught before hardware testing was even performed.Furthermore, problematic situations that were dis-covered in hardware testing could often be easilyreproduced and debugged by using a simulator.Simulated testing also provided an easy way to runlarge numbers of tests that would have been impos-sible to run exclusively on hardware, due to the timerequired to complete them. Overall, our testing insimulation made our system significantly more ro-bust to variations in resource usage, rover position,activity execution time, and frequency of sciencealerts. Such robustness is particularly importantwhen handling rover operations since factors suchas traverse time, power usage, and even possible sci-ence opportunities are difficult to predict. We shouldnote that, although testing using a simple simulatorwas very valuable, it would have also been valuableto perform some testing with higher fidelity simula-tion. For instance, when testing with rover hard-ware, obstacle avoidance software often performeddifferently depending on variations in rover posi-tion, sun angle, etc. Since we could not easily repro-duce this behavior in our simple simulator, we hadto rely on our hardware tests to ensure this behaviordid not cause problems or unexpected situations forthe OASIS software.

Furthermore, running with hardware often al-lowed a perspective that was difficult to attainthrough simulated testing. For example, the accu-racy of rover turns towards new science opportuni-ties was much easier to judge when running withhardware.

5. TESTING RESULTS

Here we report results from a set of ten runs with theFIDO rover that were conducted on four different�nonconsecutive� days. The runs had an average of 13images. The criteria specified was based on albedo—identify rocks that were white. An example is shownin Figure 13. The goal was to detect targets if they arewithin approximately 3 m of the forward field ofview of the rover as it proceeds along the traverse.Each run had between one and seven targets that therover encountered within the goal detection range.Over the ten traverse test runs, this consisted of 40

total target rocks. Thirty-six of these targets were de-tected �85% detection rate�. There were no falsealarms during any of the ten runs.

The 85% detection rate indicates how many of thetargets were identified at some time over the courseof the traverse. A target may be imaged multipletimes during the traverse as the rover approaches anddrives by the rock. Over the ten runs, there were 82instances of targets appearing in an image within thedetection range. Individual targets were imaged be-tween one and six times. Of the six targets that weremissed, three of these appeared in a single image andthree appeared in two images. No target that ap-peared in more than two images was missed.

5.1. OASIS Component Run Times on FIDO

As part of our testing, we gathered preliminary sta-tistics on the run time of key OASIS components.While we have not spent significant time optimizingthe performance of these components, the numbersprovide a general idea of current performance andprovide a reference to track future improvements. Ina representative run, the OASIS rockfinder pro-cessed 11 hazcam images resulting in five sciencealerts being sent to the planner. The rockfinder wasrun on a 233 Mhz Pentium processor running Vx-Works 5.5 with 128 MB of RAM. Rockfinder took anaverage of 53 s to process each image and foundabout nine rocks per image with a total of 103 rocksbeing identified for the 11 images. The OASIS fea-ture extraction component was run on a 930 MhzPentium processor running Linux 2.4 with 256 MBof RAM. For these tests, albedo and shape informa-tion were extracted for each rock. For the 11 imagesprocessed, feature extraction averaged 0.5 s per im-age. The planning and scheduling component ran ona 2.5 GHz Pentium processor running Linux 2.4 with1 GB of RAM. In handling the five science alerts, theplanner spent an average of 6 s generating a plan foreach alert.

6. CONCLUSIONS AND FUTURE WORK

We have demonstrated an autonomous science sys-tem in the field conducting opportunistic science. Byintegrating data analysis and planning capabilities,the resulting system can operate in a closed-loopfashion. This framework enables new science targetsto be addressed onboard with little or no communi-



cation with Earth. An important contribution of thiswork is closing the loop between the sensor data col-lection, science goal selection, and activity planningand scheduling. Current approaches require humananalysis to determine goals and to manually convertthe set of high-level science goals into low-level rovercommand sequences. By integrating these compo-nents onboard, we enable a rover to function autono-mously, as if a scientist were always in communica-tion. This type of capability should dramaticallyincrease the science return of future rover missions.

In the future we will expand the features that thesystem can recognize. We also would like to incorpo-rate information from other sources such as from or-bital data in assessing the surface features such asboundaries. Currently, the system does not explicitlyrecognize the same target or feature from differentviewpoints. This is an aspect of information thatcould be accumulated onboard. For example, obser-vations of a feature as the rover approaches it couldbe used to update the information on the interestlevel of the feature. Similarly, when a feature isviewed from a different angle, this would be valuableinformation. In addition, information from differentinstruments could be incorporated into the onboardanalysis to identify regions and individual features ofhigh science interest.

The Deep Space Network will remain a con-straining resource for future deep space missions asthe number of high bandwidth missions increases.Traditional data compression can provide a valuablemechanism for increasing the amount of useful datareturned; however, a limited amount of compressionis possible before distortion levels become intolerablyhigh. Science return can be maximized by returningthe data with the highest science content possible.The use of onboard analysis to identify opportunitiesand select the data with the highest scientific interestwill be a critical functionality to maximize science re-turn on future deep space missions with high datavolume instruments.

ACKNOWLEDGMENTS

The research described in this paper was carried outat the Jet Propulsion Laboratory, California Instituteof Technology, under a contract with the NationalAeronautics and Space Administration. This work

was sponsored by the Mars Technology Program,the Intelligent Systems Program and the Interplan-etary Network Technology Program.

Finally, our work this year could not proceedwithout the support and advice of scientists outsideof the team, particularly Albert Haldemann �JPL�and Matt Golombek �JPL�.

REFERENCES

Alami, R., Chatila, R., Fleury, S., Ghallab, M., & Ingrand, F.�1998�. An architecture for autonomy. InternationalJournal of Robotics Research 17�4�, 315–337.

Bonasso, R., Firby, R., Gat, E., Kortenkamp, D., Miller, D.,& Slack, M. �1997�. Experiences with an architecturefor intelligent, reactive agents. Journal of Experimen-tal and Theoretical Artificial Intelligence Research9�2/3�, 237–256.

Bresina, J., Golden, K., Smith, D., & Washington, R. �1999,June�. Increased flexibility and robustness of Marsrovers. Proceedings of the International Symposiumon AI, Robotics, and Autonomous for Space�iSAIRAS�, Noordwijk, Netherlands.

Castano, A., Fukunaga, A., Biesiadecki, J., Neakrase, L.,Whelley, P., Greeley, R. et al. �2007�. Autonomous de-tection of dust devils and clouds at Mars. To appearMachine Vision and Applications.

Castano, A., Fukunaga, A., Biesiadecki, J., Neakrase, L.,Whelley, P., Greeley, et al. �2006, October�. Autono-mous detection of dust devils and clouds on Mars.International Conference on Image Processing, At-lanta, GA.

Castano, R., Mann, T., & Mjolsness, E. �1999, July� Textureanalysis for Mars rover images. In Applications ofDigital Image Processing XXII, Proceedings of SPIE,Vol. 3808, pp. 162–173, Denver, CO.

Castano, R., Anderson, R.C., Estlin, T., Decoste, D., Fisher,F., Gaines, D. et al. �2003, March�. Rover traverse sci-ence for increased mission science return. Proceedingsof the 2003 IEEE Aerospace Conference, Big Sky, MT.

Castano, R., Judd, M., Estlin, T., Anderson, R.C., Scharen-broich, L., Song, L. et al. �2004, March�. Autonomousonboard traverse science system. Proceedings of the2004 IEEE Aerospace Conference, Big Sky, MT.

Castano, R., Judd, M., Estlin, T., Anderson, R.C., Gaines,G., Castaño, A. et al. �2005, March�. Current resultsfrom a rover science data analysis system. Proceed-ings of the 2005 IEEE Aerospace Conference, Big Sky,MT.

Castano, R., Estlin, T., Gaines, G., Castaño, A., Chouinard,C., Bornstein, B. et al. �2006, March�. Opportunisticrover science: finding and reacting to rocks, cloudsand dust devils. Proceedings of the 2006 IEEE Aero-space Conference, Big Sky, MT.

Cheng, Y., Maimone, M., & Matthies, L. �2005�. Visualodometry on the Mars exploration rovers. Proceed-



ings of International Conference Systems, Man, andCybernetics, Vol. 1, pp. 903–910, Hawaii, HI.

Chien, S., Sherwood, R., Tran, D., Cichy, B., Rabideau, G.,Castano, R. et al. �2005�. Using autonomy flight soft-ware to improve science return on Earth ObservingOne. Journal of Aerospace Computing, Information,and Communication 2�4�, 196–216.

Chien, S., Knight, R., Stechert, A., Sherwood, R., & Ra-bideau, G. �2000, April�. Using iterative repair to im-prove the responsiveness of planning and scheduling.Proceedings of the Fifth International Conference onArtificial Intelligence Planning and Scheduling, Breck-enridge, CO.

Estlin, T., Gaines, D., Chouinard, C., Castano, R., Born-stein, B., Judd, M. et al. �2007, April�. Increased Marsrover autonomy using AI planning, scheduling andexecution. Proceedings of the IEEE International Con-ference on Robotics and Automation �ICRA 2007�,Rome, Italy.

Fox, J., Castano, R., & Anderson, R.C. �2002, March�. On-board autonomous rock shape analysis for Mars rov-ers. Proceedings of the 2002 IEEE Aerospace Confer-ence, Big Sky, MT.

Gat, E. �1991�. Integrating planning and reacting in a het-erogeneous asynchronous architecture for mobile ro-bots. SIGART Bulletin 2, 70–74.

Gilmore, M., Castano, R., Mann, T., Anderson, R.C., Mjol-sness, E., Manduchi, R. et al. �2000�. Strategies for au-tonomous rovers at Mars. Journal of Geophysical Re-search 105�E12�, 29223–29237.

Gulick, V.C., Morris, R.L., Ruzon, M.A., & Roush, T.L.�2001�. Autonomous image analysis during the 1999Marsokhod rover field test. Journal of GeophysicalResearch 106�E4�, 7745–7764.

Halir, R., & Flusser, J. �1998�. Numerically stable directleast-squares fitting of ellipses. Proceedings of the 6thInternational Conference in Central Europe on Com-puter Graphics, Visualization and Interactive DigitalMedia �WSCG�, Plzen, Czech Republic, 125–132.

Hentschel, M.L., & Page, N.W. �2002�. Selection of descrip-tors for particle shape characterization. Particle & Par-ticle Systems Characterization,” 20�1�, 25–38.

Mahmood, A., Mitchell, J.K., & Carrier, W.D. �1974�. Grainorientation in lunar soil. Lunar and Planetary ScienceConference, 5, Houston, TX, 2347–2354.

Maimone, M., Biesiadecki, J., Tunstel, E., Cheng, Y., &Leger, C. �2006� Surface navigation and mobility intel-ligence on the Mars Exploration Rovers. In Intelli-gence for Space Robotics, �pp. 45–69�. Albuquerque,NM: TSI Press.

Mishkin, A., Morrison, J., Nguyen, T., Stone, H., Cooper,B., & Wilcox, B. �1998, Mar.�. Experiences with opera-tions and autonomy of the Mars Pathfinder mi-crorover. Proceedings of the 1998 IEEE AerospaceConference, Aspen, CO.

Nesnas, I.A., Wright, A., Bajracharya, M., Simmons, R., Es-tlin, T., & Kim, W.S. �2003, Apr.� CLARAty: an archi-tecture for reusable robotic software. SPIE AerosenseConference, Orlando, FL.

Pedersen, L. �2001�. Robotic Rock Classification and Au-tonomous Exploration, Ph.D. thesis, Robotics Insti-tute, Carnegie Mellon University �CMU-RI-TR-01-14�.Pittsburgh, PA.

Rabideau, G., Engelhardt, B., & Chien, S. �2000, Apr.�. Us-ing generic preferences to incrementally improve planquality. Proceedings of the International Conferenceon Artificial Intelligence Planning Systems �AIPS2000�, Breckenridge, CO.

Simmons, R. & Apfelbaum, D. �1998, October�. A task de-scription language for robot control. Proceedings ofthe Intelligent Robots and Systems Conference, Van-couver, Canada.

Smith, T., Niekum, S., Thompson, D., & Wettergreen, D.�2005, March� Concepts for science autonomy duringrobotic traverse and survey. Proceedings of IEEEAerospace, Big Sky, MT.

Smyth, P. �1996�. Clustering using Monte-Carlo cross-validation. Proceedings of the 2nd International Con-ference on Knowledge Discovery and Data Mining,AAAI Press.

Thompson, D.R., Smith, T., & Wettergreen, D. �2005a�.Data mining during rover traverse: from images togeologic signatures. International Symposium on Ar-tificial Intelligence, Robotics and Automation in Space�iSAIRAS�, Darmstadt, Germany.

Thompson, D., Niekum, S., Smith, T., & Wettergreen, D.,�2005b�. Automatic detection and classification of fea-tures of geologic interest. Proceedings of the 2005IEEE Aerospace Conference, Big Sky, MT.

Thompson, D.R., & Castano, R. �2007, March� A perfor-mance comparison of rock detection algorithms forautonomous planetary geology. Proceedings of the2007 IEEE Aerospace Conference, Big Sky, MT.

Tomasi, C., & Manduchi, R. �1998, January� Bilateral Filter-ing for Gray and Color Images. In International Con-ference on Computer Vision �ICCV�, pp. 839–846,Bombay, India.

Trucco, E., & Verri, A. �1998�. Introductory Techniques for3D Computer Vision, p. 343, Upper Saddle River, NJ:Prentice-Hall.

Wadell, H. �1932�. Volume, shape and roundness of rockparticles. Journal of Geology 40: 443–451.

Wagner, M.D., Apostolopoulos, D., Shillcutt, K., Shamah,B., Simmons, R.G., & Whittaker, W. �2001, May�. Thescience autonomy system of the Nomad Robot. Pro-ceedings of the International Conference on Roboticsand Automation �ICRA 2001�, pp. 1742–1749, Seoul,Korea.



Oasis: Onboard autonomous science investigation system for opportunistic rover science

Documents

Transcript of Oasis: Onboard autonomous science investigation system for opportunistic rover science