1

Towards robotic assistants in nursing homes: challenges and resultsJoelle Pineau

�

, Michael Montemerlo�

, Martha Pollack�

, Nicholas Roy�

, Sebastian Thrun�

�

Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, 15232, USA�

Dept. of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, 48019, USA

This paper describes a mobile robotic assistant, developed to assist elderly individuals with mild cognitive and physicalimpairments, as well as support nurses in their daily activities. We present three software modules relevant to ensure successfulhuman-robot interaction: an automated reminder system; a people-tracking and detection system; and finally a high-levelrobot controller which performs planning under uncertainty by incorporating knowledge from low-level modules, and selectingappropriate courses of actions. During the course of experiments conducted in an assisted living facility, the robot successfullydemonstrated that it could autonomously provide reminders and guidance for elderly residents.

1. Introduction

The US population is aging at an alarming rate. Atpresent, 12.5% of the US population is of age 65 orolder (30). It is widely recognized that this ratio willincrease as the baby-boomer generation moves intoretirement age. Meanwhile, the nation faces a sig-nificant shortage of nursing professionals. The Fed-eration of Nurses and Health Care Professionals hasprojected a need for 450,000 additional nurses by theyear 2008.

This acute need provides significant opportunitiesfor roboticists and AI researchers to develop assis-tive technology that can improve the quality of life ofour aging population, and help nurses become moreeffective in their activities. The Nursebot Projectwas conceived in response to this challenge. It isformed by a multi-disciplinary team of investigatorsfrom the fields of health-care, HCI/psychology, andAI/robotics. The overall goal of the project is to de-velop mobile robotic assistants that can assist nursesand elderly people in their daily activities.

To this aim, the team has developed two pro-totype autonomous mobile robots, shown in Fig-ure 1 (22). These robots primarily interact with theworld through speech, visual displays, facial expres-sions and physical motion. They differ from earlierworkplace robots in that they go beyond simply inter-acting with an (often static) environment, to interact-ing with human users and bystanders. Thus we lever-age earlier technology for navigation, localization andmapping, and specifically focus on developing new

algorithmic approaches to track people, predict theirbehavior, and react appropriately.

From the many services a nursing-assistant robotcould provide (11; 18), the work reported here con-siders the task of reminding people of events andguiding them through their environments. Both ofthese tasks are particularly relevant with the elderlycommunity. Decreased memory is a common effectof age-related cognitive decline, which often leadsto forgetfulness about routine daily activities (e.g.taking medications, attending appointments, eating,drinking, bathing, toileting) thus the need for a robotthat can offer cognitive reminders. In addition, nurs-ing staff in assisted living facilities frequently need toescort elderly people walking, either to get exercise,or to attend meals, appointments or social events. Thefact that many elderly people move at extremely slowspeeds (e.g. 5 cm/sec) makes this one of the mostlabor-intensive tasks in assisted living facilities. It isalso important to note that the help provided is of-ten not strictly of a physical nature, as many elderlypeople select walking aids over physical assistance bynurses. Rather, nurses often provide important cog-nitive help, in the form of reminders, guidance andmotivation, in addition to valuable social interaction.

From an AI point of view, several factors make thistask a challenging one for a robot to accomplish suc-cessfully. Many elderly have difficulty understandingthe robot’s synthesized speech, as well as articulatingan appropriate response in a computer-understandableway. In addition, walking abilities vary drastically be-

2

Figure 1. Nursebots Flo (left) and Pearl (right)

tween individuals. People with walking aids are usu-ally an order of magnitude slower than people with-out, and people often stop to chat or catch breathalong the way. It is therefore imperative that the robotadapt to individuals—an aspect of interaction that hasbeen poorly explored in AI and robotics.

The work presented in this paper seeks to addressthese challenges, focusing on three software compo-nents most pertinent to human-robot interaction: anautomated reminder system that incorporates knowl-edge of a person’s typical schedule with observationsof recent activities, and issues pertinent remindersabout upcoming events; a module which uses efficientparticle filter techniques to detect and track people;and finally a high-level robot controller which usesprobabilistic reasoning techniques to arbitrate be-tween information-gatheringand performance-relatedactions, as well as incorporate information obtainedthrough both navigation sensors (e.g. laser range-finder) and interaction sensors (e.g. speech recogni-tion and touchscreen).

In systematic experiments conducted at a nursinghome, we found the combination of techniques tobe highly effective in dealing with elderly test sub-jects. In particular, during a sequence of one-one-onescenarios between Pearl and residents of the nursinghome, the robot demonstrated the ability to contact aresident, remind them of an appointment, accompanythem to that appointment, as well as provide informa-tion of interest to that person, for example weatherreports or television schedules.

2. Hardware and Software Description

Figure 1 shows images of the robots Flo (first pro-totype, now retired) and Pearl (the present robot).Each robot is equipped with a differential drive sys-tem, two on-board PCs, wireless ethernet, laser rangefinders, sonar sensors, microphones for speech recog-nition, speakers for speech synthesis, touch-sensitivegraphical displays, actuated head units, and stereocamera systems. As a result of feedback from nursesand medical experts following deployment of the firstrobot, Flo, the second robot Pearl also features an im-proved visual appearance, two sturdy handle-bars, amore compact design that allows for cargo space anda removable tray, doubled battery capacity, a secondlaser range finder, and a significantly more sophisti-cated head unit.

On the software side, both robots feature off-the-shelf autonomous mobile robot navigation system (4;28), speech recognition software (24), speech synthe-sis software (3), fast image capture and compressionsoftware for online video streaming, face detectiontracking software (25), as well as the three major newsoftware modules described in this paper. These mod-ules are principally concerned with people interactionand control. They overcome important deficiencies ofthe work described by (4; 28), which had only rudi-mentary abilities to interact with people.

3. Plan management with Autominder

The Autominder software component is designedas an intelligent cognitive orthotic system, providingelderly people with reminders about their daily activ-ities (23). The idea of using computer technology toenhance the performance of cognitively disabled peo-ple dates back nearly forty years (12). More recently,cognitive orthotics have enabled reminders to be pro-vided using the telephone (13), personal digital assis-tants (10), and pagers (15). Work has also been doneon improved modelling of users’ activities (17; 20),where in one case a hand-device uses AI planningtechnology to model the user’s plans, and provide vi-sual and audible cues about its execution.

In the Nursebot project, the goal of this softwaresystem is to make principled decisions about what re-minders to issue and when, balancing the followingpotentially competing objectives: (i) ensure that the

3

Figure 2. Autominder Architecture

user is aware of activities s/he is expected to perform,(ii) increase the likelihood that s/he will perform atleast the required activities (e.g. taking medicine),(iii) avoid annoying the user, and (iv) avoid makingthe user overly reliant on the system. To attain thesegoals, the system must be flexible and adaptive, re-sponding to the actions taken by the user.

The Autominder architecture is shown in figure 2.As depicted, the system maintains an accurate modelof a user’s daily schedule, monitors performance ofactivities, and plans reminders accordingly. The threemain components are: a Plan Manager (PM), whichstores the user’s plan of daily activities in the ClientPlan, and is responsible for updating it and iden-tifying any potential conflicts in it; a Client Mod-eler (CM), which uses information about the user’sobservable activities to track the execution of theplan, storing its beliefs about the execution status inthe Client Model; and a Personal Cognitive Orthotic(PCO), which reasons about any disparities betweenwhat the user is supposed to do and what s/he is doing,and makes decisions about when to issue reminders.

To initialize the system, the caregiver for an elderlyuser inputs a description of the user’s daily activities,as well as any constraints on, or preferences regard-ing, the time or manner of their performance. Thisplan may then be changed in one of four ways: (i)the user or caregiver may add new activities; (ii) theuser or caregiver may modify or delete activities al-

ready in the plan; (iii) the user may execute one ofthe planned activities; or (iv) the simple passage oftime may cause automatic changes to be made in theplan. Whenever a change occurs, the PM updatesthe user plan, performing plan merging and constraintpropagation as needed. To adequately represent userplans, it essential to support a rich set of temporalconstraints; we achieve this goal by modelling userplans as Disjunctive Temporal Problems (DTPs) andreasoning about them using efficient algorithms (29).

Throughout the day, sensor information is gatheredby the robot and sent to the CM, which uses this in-formation to try to infer what activities the user isperforming. If the likelihood is high that a plannedactivity has been executed, the CM reports this tothe PM, which can then update the user’s plan byrecording the time of execution, and propagate anyaffected constraints accordingly. The user model isrepresented using a Quantitative Temporal Bayes Net(QTBN), which was developed to handle the needboth to reason about fluents and about probabilistictemporal constraints (5).

The final component of the Autominder is thePCO (21), which uses both the user plan and theuser model to determine what reminders should beissued and when. The PCO identifies activities thatmay require reminders based on their importance andtheir likelihood of being executed on time as mod-eled in the CM. It also determines the most effec-tive times to issue each required reminder, taking ac-count of the expected user behavior, and any prefer-ences explicitly provided by the user and the care-giver. Finally, the PCO provides justifications as towhy particular activities warrant a reminder. ThePCO treats the generation of a reminder plan as asatisficing problem and uses a local-search approachcalled Planning-by-Rewriting (PbR) (1) to producea high-quality reminder plan that takes into accountthe user’s expected behavior, preferences, and inter-actions amongst planned activities.

4. Locating People

In order to issue reminders and, when appropri-ate, guide users to their activities, it is necessary tointeract with people spatially, and most specificallyto be able to locate people in their living environ-ment. The problem of locating people is the prob-

4

(a) (b) (c) (d)

Figure 3. (a)-(d) Evolution of the conditional particle filter from global uncertainty to successful localizationand tracking. (d) The tracker continues to track a person even as that person is occluded repeatedly by a secondindividual.

lem of determining their � - � -location relative to therobot. Previous approaches to people tracking inrobotics are feature-based: they analyze sensor mea-surements (images, range scans) for the presence offeatures (14; 26) as the basis of tracking. In our case,the diversity of the environment mandates a differ-ent approach. Pearl detects people using map differ-encing: the robot learns a map, and people are de-tected by significant deviations from the map. Fig-ure 3 shows an example map acquired using preexist-ing software (28).

Mathematically, the problem of people tracking is acombined posterior estimation problem and model se-lection problem. Let � be the number of people nearthe robot. The posterior over the people’s positions isgiven by��� ���� � ����� ������� � � ��� ����� (1)

where ����� with ����� ��� is the location of a per-son at time ! , � the sequence of all sensor measure-ments, � the sequence of all robot controls, and � isthe environment map. However, to use map differenc-ing, the robot has to know its own location. The loca-tion and total number of nearby people detected by therobot is clearly dependent on the robot’s estimate ofits own location and heading direction. Hence, Pearlestimates a posterior of the type:��� � �� � ����� ��� ��� � � � � ��� �"�#� (2)

where � denotes the sequence of robot poses (thepath) up to time ! . If � was known, estimat-ing this posterior would be a high-dimensional es-timation problem, with complexity cubic in � for

Kalman filters (2), or exponential in � with particlefilters (8). Neither of these approaches is, thus, ap-plicable: Kalman filters cannot globally localize therobot, and particle filters would be computationallyprohibitive.

Luckily, under mildly restrictive conditions (dis-cussed below) the posterior (2) can be factored into�%$&� conditionally independent estimates:

�'�(� � � ��� �"�#�*) ��'� ����� +� � �"� ����� (3)

This factorization opens the door for a particle filterthat scales linearly in � . Our approach is similar (butnot identical) to the Rao-Blackwellized particle filterdescribed in (9). First, the robot path � is estimatedusing a particle filter, as in the Monte Carlo local-ization (MCL) algorithm for mobile robot localiza-tion (6). Each particle in this filter is associated witha set of � particle filters, each representing one ofthe people position estimates ��� � �,� � � ��� ����� . Theseconditional particle filters represent people positionestimates conditioned on robot path estimates—hencecapturing the inherent dependence of people androbot location estimates. The data association be-tween measurements and people is done using max-imum likelihood, as in (2). Under the (false) assump-tion that this maximum likelihood estimator is alwayscorrect, our approach can be shown to converge to thecorrect posterior, and it does so with update time lin-ear in � . In practice, we found that the data associa-tion is correct in the vast majority of situations. Thenested particle filter formulation has a secondary ad-

5

vantage that the number of people � can be madedependent on individual robot path particles. Our ap-proach for estimating � uses the classical AIC crite-rion for model selection, with a prior that imposes acomplexity penalty exponential in � .

Figure 3 shows results of the filter in action. In Fig-ure 3a, the robot is globally uncertain, and the numberand location of the corresponding people estimatesvaries drastically. As the robot reduces its uncer-tainty, the number of modes in the robot pose poste-rior quickly becomes finite, and each such mode has adistinct set of people estimates, as shown in Figure 3b.Finally, as the robot is localized, so is the person (Fig-ure 3c). When guiding people, the localization esti-mate of the person is used to determine the velocity ofthe robot, so that the robot maintains roughly a con-stant distance to the person. In our experiments inthe target facility, we found the adaptive velocity con-trol to be absolutely essential for the robot’s ability tocope with the huge range of walking paces found inthe elderly population. Initial experiments with fixedvelocity led almost always to frustration on the peo-ple’s side, in that the robot was either too slow or toofast.

Finally, Figure 3d illustrates the robustness of thefilter to interfering people. Here another person stepsbetween the robot and its target subject. The fil-ter obtains its robustness to occlusion from a care-fully crafted probabilistic model of people’s motion�'� � �,� � � � � �,� � . This enables the conditional particlefilters to maintain tight estimates while the occlusiontakes place, as shown in Figure 3d. During in-labexperiments involving 31 tracking instances with upto five people at a time, the error in determining thenumber of people was 9.6%. The error in the robotposition was

� ������� ��� cm, and the people positionerror was as low as �������*� � cm, when comparedto measurements obtained with a carefully calibratedstatic sensor with � � cm error.

5. High Level Robot Control and Dialog Manage-ment

The most central module in Pearl’s software is aprobabilistic algorithm for high-level control and di-alog management. This module integrates observa-tions from lower-level modules (e.g. the Autominder,the people tracker, the speech recognizer, etc.) and

uses this information to select appropriate behaviorsand responses.

Pearl’s high-level control architecture is a hierar-chical variant of a partially observable Markov deci-sion process (POMDP) (16). The POMDP is a modelfor calculating optimal control actions under uncer-tainty. The control decision is based on a probabilisticbelief over possible states.

In Pearl’s case, this distribution is defined over acollection of multi-valued state variables:� robot location (discrete approximation)� person’s location (discrete approximation)� person’s status (inferred from speech recog-

nizer)� motion goal (where to move)� reminder goal (what to inform the user of)� user initiated goal (e.g., an information request)

The value of the person’s location variable is ob-served through the people tracker, and similarly thereminder goal variable is set by the Autominder mod-ule. Overall, there are 516 possible states. The inputto the POMDP is a factored probability distributionover these states, generated by a state estimator, suchas in Equation (2). Uncertainty over the current statearises predominantly from the localization modulesand the speech recognition system. The considerationof uncertainty is especially important in this domain,as the costs of giving a reminder to the wrong person,or unnecessarily sending the robot to a location canbe large.

Unfortunately, POMDPs of the size encounteredhere are an order of magnitude larger than today’sbest exact POMDP algorithms can tackle (16). How-ever, Pearl’s domain is highly structured, since cer-tain actions are only applicable in certain situations.To exploit this structure, we developed a hierarchicalversion of POMDPs, which breaks down the decisionmaking problem into a collection of smaller problemsthat can be solved more efficiently. Our approach issimilar to the MAX-Q decomposition for MDPs (7),but defined over POMDPs (where states are unob-served).

The basic idea of the hierarchical POMDP is topartition the action space—not the state space, sincethe state is not fully observable—into smaller chunks.For Pearl’s guidance task the action hierarchy isshown in Figure 4, where abstract actions (shown in

6

Assist

Act

Remind

Contact

RemindPhysioPublishStatus

RingBellGotoRoom

VerifyRelease

Inform

SayTimeSayWeatherVerifyRequest

RestRechargeGotoHome

Move

BringtoPhysioCheckUserPresentDeliverUser

VerifyBring

Figure 4. Dialog Problem Action Hierarchy

circles) are introduced to subsume logical subgroupsof lower-level actions. This action hierarchy induces adecomposition of the control problem, where at eachnode all lower-level actions, if any, are considered inthe context of a local sub-controller. At the lowestlevel, the control problem is a regular POMDP, witha reduced action space. At higher levels, the controlproblem is also a POMDP, yet involves a mixture ofphysical and abstract actions (where abstract actionscorrespond to lower level POMDPs.)

It is important to notice that such a decomposi-tion is especially appropriate when the optimal con-trol transgresses down along a single path in the hi-erarchy to receive its reward. This is approximatelythe case in the Pearl domain, where goals are satisfiedupon successfully delivering a person, or successfullygathering information through communication.

Using the hierarchical POMDP, the high-level de-cision making problem in Pearl is tractable, and anear-optimal control policy can be computed off-line.Thus, during execution time the controller simplymonitors the state (calculates the posterior) and looksup the appropriate control. Table 1 shows an exampledialog between the robot and a test subject. Becauseof the uncertainty management in POMDPs, the robotchooses to ask a clarification question at three occa-sions. The number of such questions depends on theclarity of a person’s speech, as detected by the Sphinxspeech recognition system.

An important remaining question concerns the im-portance of handling uncertainty in high-level con-

Observation True State Action Rewardpearl hello request begun say hello 100pearl what is like start meds ask repeat -100pearl what time is it

for will the want time say time 100pearl was on abc want tv ask which station -1pearl was on abc want abc say abc 100pearl what is on nbc want nbc confirm channel nbc -1pearl yes want nbc say nbc 100pearl go to the that

pretty good what send robot ask robot where -1pearl that that hello be send robot bedroom confirm robot place -1pearl the bedroom any i send robot bedroom go to bedroom 100pearl go it eight a hello send robot ask robot where -1pearl the kitchen hello send robot kitchen go to kitchen 100

Table 1Sample dialog demonstrating the role of clarificationactions. Actions in bold font are clarification actions,chosen by the POMDP because of high uncertainty inthe speech signal.

trol. To investigate this, we ran a series of compar-ative experiments, all involving real data collected inour lab. In one series of experiments, we investigatedthe importance of considering the uncertainty arisingfrom the speech interface. In particular, we com-pared Pearl’s performance to a system that ignoresthat uncertainty, but is otherwise identical. The result-ing approach is an MDP, similar to the one describedin (27). Figure 5 shows results for three differentperformance measures, and three different users (indecreasing order of speech recognition performance).For poor speakers, the MDP requires less time to “sat-isfy” a request due to the lack of clarification ques-tions (Figure 5a). However, its error rate is muchhigher (Figure 5b), which negatively affects the over-all reward received by the robot (Figure 5c). Theseresults clearly demonstrate the importance of consid-ering uncertainty at the highest robot control level,specifically with poor speech recognition.

In the second series of experiments, we investi-gated the importance of uncertainty management inthe context of highly imbalanced costs and rewards.For example, in Pearl’s case, asking a clarificationquestion is in fact much cheaper than accidentally de-livering a person to a wrong location, or guiding aperson who does not want to be walked. We there-fore compared performance using two POMDP mod-els which differed only in their cost models. Onemodel assumed uniform costs for all actions, whereasthe second model assumed a more discriminative cost

7

0

0.5

1

1.5

2

2.5

3

User 1 User 2 User 3

Ave

rage

# o

f ac

tions

to s

atis

fy r

eque

st

User Data -- Time to Satisfy RequestPOMDP Policy

ConventionalMDP Policy

2.0

2.5

1.631.9 1.86

1.42

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

User 1 User 2 User 3

Ave

rage

Err

ors

per

Act

ion

User Data -- Error Performance

ConventionalMDP Policy

POMDP Policy

0.1 0.10.18

0.55

0.825

0.36

0

10

20

30

40

50

60

User 1 User 2 User 3

Ave

rage

Rew

ard

per

Act

ion

User Data -- Reward AccumulationPOMDP Policy

Conventional MDP Policy52.2

36.95

49.72

24.8

6.19

44.95

(a) (b) (c)

Figure 5. Empirical comparison between POMDPs (with uncertainty, shown in gray) and MDPs (no uncertainty,shown in black) for high-level robot control, evaluated on data collected in the assisted living facility. Shown arethe average time to task completion (a), the average number of errors (b), and the average user-assigned (not modelassigned) reward (c), for the MDP and POMDP. The data is shown for three users, with good, average and poorspeech recognition.

model in which the cost of verbal questions was lowerthan the cost of performing the wrong motion actions.A POMDP policy was learned for each of these mod-els, and then tested experimentally in our laboratory.The results presented in figure 6 show that the non-uniform model makes more judicious use of confir-mation actions, thus leading to a significantly lowererror rate, especially for users with low recognitionaccuracy.

6. Results

Following integration of the three software mod-ules onto Pearl, the robot was deployed in a retirementcommunity located near Pittsburgh, PA. This sectiondescribes experiments involving elderly residents ofthis facility, with mild cognitive, perceptual, or phys-ical limitations.

We tested the robot in five separate experiments,each lasting one full day. The first three days focusedon open-ended interactions with a large number of el-derly users, during which the robot interacted verballyand spatially with elderly people with the specific taskof delivered sweets. This allowed us to gauge peo-ple’s initial reactions to the robot.

Following this, we performed two days of formalexperiments during which the robot autonomously led12 full guidances, involving 6 different elderly peo-ple. Figure 7 shows an example guidance experiment,

0

0.5

1

1.5

2

User 1 User 2 User 3

Err

ors

per

task

User Data -- Error Performance

Uniform cost modelNon-uniform cost model

0.3

1.2

1.6

0.2

0.6

0.1

Figure 6. Empirical comparison between uniform andnon-uniform cost models. Results are an average over10 tasks. Depicted are 3 example users, with vary-ing levels of speech recognition accuracy. Users 2& 3 had the lowest recognition accuracy, and con-sequently more errors when using the uniform costmodel.

involving an elderly person who uses a walking aid.The sequence of images illustrates the major stagesof a successful delivery: from contacting the person,delivering the reminder, walking her through the fa-cility, and providing information after the successfuldelivery—in this case on the weather.

In all trials, the task was performed to completion.Post-experimental debriefings illustrated a uniform

8

(a) (b)

(c) (d)

(e) (f)

Figure 7. Example of a successful guidance experi-ment: a) Pearl picks up the patient outside her room,b) reminds her of a physiotherapy appointment, c)guides the person to the physiotherapy department,d) enters the department, e) satisfies a request for theweather report, and f) terminates the interaction andleaves.

high level of excitement on the side of the elderly.Overall, only a few problems were detected duringthe operation. None of the test subjects showed diffi-culties understanding the major functions of the robot.They all were able to operate the robot after less thanfive minutes of introduction. Earlier trials with apoorly adjusted speech recognition system, and fixed-velocity robot motion, both caused difficulties. Thesewere addressed early on by increasing the role of thetouchscreen, and including adaptable velocities.

7. Discussion

This paper described a mobile robotic assistant fornurses and elderly residents in assisted living facili-ties. The system has been tested successfully in exper-iments in an assisted living facility. The experimentswere successful in two main dimensions. First, theyprovided some evidence towards the feasibility of us-ing autonomous mobile robots as assistants to nursesand institutionalized elderly. Second, they demon-strated that various probabilistic tracking and plan-ning techniques are well-suited to solve problems per-taining to human-robot interactions.

One of the key lessons learned while developingthis robot is that the elderly population requires tech-niques that can cope with individual differences (e.g.walking speed), age-related decline (e.g. memoryloss) and noisy perception (e.g. poor speech recog-nition). We view the area of assistive technology as aprime source for great AI problems in the future.

References

[1] J.L. Ambite and C.A. Knoblock. Planning by rewriting. JAIR,15:207-261. 2001.

[2] Y. Bar-Shalom and T. E. Fortmann. Tracking and Data Asso-ciation. Academic Press. 1998.

[3] A.W. Black, P. Taylor, and R. Caley. The Festival SpeechSynthesis System. University of Edinburgh. 1999.

[4] W. Burgard, A.B., Cremers, D. Fox, D. Hahnel, G. Lake-meyer, D. Schulz, W. Steiner, and S. Thrun. The interactivemuseum tour-guide robot. AAAI-98.

[5] D. Colbry, B. Peintner and M.E. Pollack. Execution monitor-ing with quantitative temporal bayesian networks. AIPS-02.

[6] F. Dellaert, D. Fox, W. Burgard, and S. Thrun. Monte Carlolocalization for mobile robots. ICRA-99.

[7] T. Dietterich. The MAXQ method for hierarchical reinforce-ment learning. ICML-98.

[8] A. Doucet, N. de Freitas, and N.J. Gordon, editors. SequentialMonte Carlo Methods In Practice. Springer. 2001.

[9] A. Doucet, N. de Freitas, K. Murphy, and S. Russell. Rao-Blackwellised particle filtering for dynamic bayesian net-works. UAI-2000.

[10] M.M. Dowds, and K. Robinson Assistive technology formemory impairment: Palmtop computers and TBI. In Brain-tree Hospital Traumatic Brain Injury NeurorehabilitationConference. 1996.

[11] G. Engelberger. Services. In Handbook of IndustrialRobotics,. John Wiley and Sons. 1999.

9

[12] D.C. Englebart. A conceptual framework for the augmen-tation of man’s intellect. In Vistas in Information Handling.Spartan Books. 1963.

[13] R.H. Friendman. Automated telephone conversation to assesshealth behavior and deliver behavioral interventions. Journalof Medical Systems. 22:95-101. 1998.

[14] D. M. Gavrila. The visual analysis of human movement: Asurvey. Computer Vision and Image Understanding. 73(1).1999.

[15] N.A. Hersh and L. Treadgold. Neuropage: The rehabilita-tion of memory dysfunction by prosthetic memory and cue-ing. NeroRehabilitation. 4:187-197. 1994.

[16] L.P. Kaelbling, M.L. Littman, and A.R. Cassandra. Planningand acting in partially observable stochastic domains. Artifi-cial Intelligence. 101. 1998.

[17] N. Kirsch, S.P. Levine, M. Fallon-Kureger, L.A. Jaros. Themicrocomputer as an ’orthotic’ device for pations with cogni-tive deficits. Journal of Head Trauma Rehabilitation. 2:77-86.

[18] G. Lacey and K.M. Dawson-Howe. The application ofrobotics to a mobility aid for the elderly blind. Robotics andAutonomous Systems. 23. 1998.

[19] G. Lakemeyer, editor. Notes Second International Workshopon Cognitive Robotics. Berlin. 2000

[20] R. Levinson. Peat-the lanning and execution assistant andtrainer. Robotics and Autonomous systems. 23:245-252.

[21] C.E. McCarthy, and M. Pollack. A Plan-Based PersonalizedCognitive Orthotic. AIPS-2002.

[22] M. Montemerlo, J. Pineau, N. Roy, S. Thrun and V. VermaExperients with a Mobile Robotic Guide for the Elderly AAAI-2002

[23] M. Pollack Planning technology for intelligent cognitive or-thotics AIPS-2002.

[24] M. Ravishankar. Efficient algorithms for speech recognition.1996. Internal Report.

[25] H.A. Rowley, S. Baluja, and T. Kanade. Neural network-based face detection. IEEE Transactions on Pattern Analysisand Machine Intelligence. 20(1). 1998.

[26] D. Schulz, W. Burgard, D. Fox, and A. Cremers. Trackingmultiple moving targets with a mobile robot using particlesfilters and statistical data association. ICRA-2001.

[27] S. Singh, M. Kearns, D. Litman, and M. Walker. Reinforce-ment learning for spoken dialogue systems. NIPS-2000.

[28] S. Thrun, M. Beetz, M. Bennewitz, W. Burgard, A.B. Cre-mers, F. Dellaert, D. Fox, D. Hahnel, C. Rosenberg, N. Roy,J. Schulte, and D. Schulz. Probabilistic algorithms and theinteractive museum tour-guide robot Minerva. InternationalJournal of Robotics Research. 19(11). 2000.

[29] I. Tsamardinos. Constraint-Based Temporal Reasoning Al-gorithsm with Applications to Planning. Ph.D. Dissertation.University of Pittsburgh Intelligent Systems Program. 2001.

[30] US Department of Health and Human Services. Health,United states, 1999. Health and aging chartbook, 1999.