Developing Ambient Intelligence Applications for the...

8
Developing Ambient Intelligence Applications for the Assisted Living of the Elderly Edgardo Avilés-López, J. Antonio García-Macías, and Ismael Villanueva-Miranda CICESE Research Center, Mexico {avilesl, jagm, ivillanu}@cicese.mx ABSTRACT Ambient Assisted Living is a research area which has emerged to assist individuals in performing their everyday activities with the help of ambient intelligence technology. To ac- complish this, technological aspects such as context acquisi- tion, device integration, networking, and social aspects such as accessibility, usability, privacy, and others, play an im- portant role. These aspects become crucial when assisting the elderly, because they present very specific and demand- ing requirements. In this paper, we present work in some application scenarios conducted within our research group, focused on supporting the elderly. The implementation of these scenarios is possible through the use of our proposed technological infrastructure consisting of two main parts: (1) a physical platform constituted by sensors, actuators, RFID tags, and other devices, and (2) a software platform pro- viding Web services with basic blocks such as voice recogni- tion, indoor localization, and others, that can be composed to construct more sophisticated services. Categories and Subject Descriptors D.2.6 [Software Engineering]: Programming Environments; D.2.11 [Software Engineering]: Software Architectures; J.3 [Computer Applications]: Life and Medical Sciences Keywords ambient assisted living, sentient visors, ubiquitous comput- ing, smart home, mobility assistance, audio sensor networks, pervasive health, ubiquitous mashups 1. INTRODUCTION The ubiquitous computing vision [22] describes an aug- mented world full of intelligent devices that comprises a fine grained distributed network. Ambient intelligence (AmI) goes further by studying how to allow environments to be sensitive, adaptive, and responsive to the presence of people, in order to support them in their day to day experiences [2]. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ANT ’2010 Paris, France Copyright 20XX ACM X-XXXXX-XX-X/XX/XX ...$10.00. We are interested in ambient assisted living (AAL) systems with a special focus on supporting the elderly support (also known as Home Care Systems [6]). The physical and men- tal declination that comes with the natural aging process is one of the reasons why ambient intelligence aims to come up with new alternatives and innovative tools to improve their lifestyle by helping them to maintain certain degree of independence. The rest of the paper is organized as follows. In section 2 we talk about the motivation of our work and explain why the support for the elders is a good case study for ambi- ent intelligence. Next, in section 3 we discuss some of the related work. In section 4, we present our technological in- frastructure, which comprises both a physical and a software platform to build services on top of it. Then, in section 5, we discuss the cases of study developed with our contributions and talk about how they were used to successfully accom- plish the specific requirements of each case. In section 6, we talk about the lessons learned and finally, we conclude with some final remarks and outline the future work. 2. MOTIVATION Augmented spaces are particularly helpful for the elders, and, as average life expectancy increases worldwide with an aging population constantly growing [20], we can expect a greater demand for services and applications oriented to- ward assisting them. These demands include providing ser- vices for those who suffer from various illnesses (both mental and physical) such as: diabetes, arthritis, senile dementia, alzheimer, heart-related diseases among many others. The emergence of new types of mobile and embedded comput- ing devices, developments in wireless networking and com- munications, smart sensors, and others, gives us the tools and methods to come up with innovative services and appli- cations to better assist users, and therefore, improve their lifestyle. Examples of those can be seen in related areas such as tele-homecare [11], pervasive health (or ubi-health) [15], and mobile tele-monitorization [13]. We believe that support for the elderly through AAL in- frastructures is the most naturally appropriate. Not just because of its evident social impact, but because of the char- acteristics of its special requirements. We summarize this by analysing the characteristics shown by a traditional ambient intelligence system, according to Oppermann et al. [16]: Invisible. Since there is the need to track and auto- matically identify the users of an augmented space, they are required to wear a digital tag that can take the form of a watch, a purse, glasses, be embedded in

Transcript of Developing Ambient Intelligence Applications for the...

Page 1: Developing Ambient Intelligence Applications for the ...usuario.cicese.mx/~jagm/docs/aviles-ant10acc.pdf · Developing Ambient Intelligence Applications for the Assisted Living of

Developing Ambient Intelligence Applicationsfor the Assisted Living of the Elderly

Edgardo Avilés-López, J. Antonio García-Macías, and Ismael Villanueva-MirandaCICESE Research Center, Mexico{avilesl, jagm, ivillanu}@cicese.mx

ABSTRACTAmbient Assisted Living is a research area which has emergedto assist individuals in performing their everyday activitieswith the help of ambient intelligence technology. To ac-complish this, technological aspects such as context acquisi-tion, device integration, networking, and social aspects suchas accessibility, usability, privacy, and others, play an im-portant role. These aspects become crucial when assistingthe elderly, because they present very specific and demand-ing requirements. In this paper, we present work in someapplication scenarios conducted within our research group,focused on supporting the elderly. The implementation ofthese scenarios is possible through the use of our proposedtechnological infrastructure consisting of two main parts: (1)a physical platform constituted by sensors, actuators, RFIDtags, and other devices, and (2) a software platform pro-viding Web services with basic blocks such as voice recogni-tion, indoor localization, and others, that can be composedto construct more sophisticated services.

Categories and Subject DescriptorsD.2.6 [Software Engineering]: Programming Environments;D.2.11 [Software Engineering]: Software Architectures;J.3 [Computer Applications]: Life and Medical Sciences

Keywordsambient assisted living, sentient visors, ubiquitous comput-ing, smart home, mobility assistance, audio sensor networks,pervasive health, ubiquitous mashups

1. INTRODUCTIONThe ubiquitous computing vision [22] describes an aug-

mented world full of intelligent devices that comprises a finegrained distributed network. Ambient intelligence (AmI)goes further by studying how to allow environments to besensitive, adaptive, and responsive to the presence of people,in order to support them in their day to day experiences [2].

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.ANT ’2010 Paris, FranceCopyright 20XX ACM X-XXXXX-XX-X/XX/XX ...$10.00.

We are interested in ambient assisted living (AAL) systemswith a special focus on supporting the elderly support (alsoknown as Home Care Systems [6]). The physical and men-tal declination that comes with the natural aging process isone of the reasons why ambient intelligence aims to comeup with new alternatives and innovative tools to improvetheir lifestyle by helping them to maintain certain degree ofindependence.

The rest of the paper is organized as follows. In section 2we talk about the motivation of our work and explain whythe support for the elders is a good case study for ambi-ent intelligence. Next, in section 3 we discuss some of therelated work. In section 4, we present our technological in-frastructure, which comprises both a physical and a softwareplatform to build services on top of it. Then, in section 5, wediscuss the cases of study developed with our contributionsand talk about how they were used to successfully accom-plish the specific requirements of each case. In section 6, wetalk about the lessons learned and finally, we conclude withsome final remarks and outline the future work.

2. MOTIVATIONAugmented spaces are particularly helpful for the elders,

and, as average life expectancy increases worldwide with anaging population constantly growing [20], we can expect agreater demand for services and applications oriented to-ward assisting them. These demands include providing ser-vices for those who suffer from various illnesses (both mentaland physical) such as: diabetes, arthritis, senile dementia,alzheimer, heart-related diseases among many others. Theemergence of new types of mobile and embedded comput-ing devices, developments in wireless networking and com-munications, smart sensors, and others, gives us the toolsand methods to come up with innovative services and appli-cations to better assist users, and therefore, improve theirlifestyle. Examples of those can be seen in related areas suchas tele-homecare [11], pervasive health (or ubi-health) [15],and mobile tele-monitorization [13].

We believe that support for the elderly through AAL in-frastructures is the most naturally appropriate. Not justbecause of its evident social impact, but because of the char-acteristics of its special requirements. We summarize this byanalysing the characteristics shown by a traditional ambientintelligence system, according to Oppermann et al. [16]:

• Invisible. Since there is the need to track and auto-matically identify the users of an augmented space,they are required to wear a digital tag that can takethe form of a watch, a purse, glasses, be embedded in

Page 2: Developing Ambient Intelligence Applications for the ...usuario.cicese.mx/~jagm/docs/aviles-ant10acc.pdf · Developing Ambient Intelligence Applications for the Assisted Living of

clothes, and others. In some nursing homes (like theone we applied our cases of study in), residents are re-quired to wear an identification purse, allowing us toreplace it by a digitally augmented version.

• Mobile. As users move around their homes, or nursingrooms, they need to carry with them some of the com-putation or sensing needed to support their activities.This becomes crucial when dealing with an applicationthat helps the user to take medications, or one in whichfactors as blood pressure, sugar levels, an others, mustbe constantly monitored and controlled.

• Spontaneous communication. In a scenario in whichconstant connectivity is required and the number ofnetworked elements is high, an ad hoc network infras-tructure is needed, in order to guarantee connectivityin spite of node failures.

• Heterogeneous an hierarchical Augmenting a space usu-ally involves the integration and coordination of sev-eral different kind of system nodes and devices. Ahierarchical organization of them can help in buildinga richer encapsulated functionality. For example, asensor node can provide raw sensorial data, a sensornetwork on the other hand, can constitute a sort of“macroscope” we can use to identify composite activi-ties, while hiding us the individual nodes coordination.

• Context-aware. Three steps are involved: first, rawcontext data such as temperature, identity, localiza-tion, and others, is acquired; second, context is mod-eled, communicated through the system, and then in-terpreted; third, context-inferred data is used to per-form an action or cast a change in the system. Thisrequirement becomes really important for the elder be-cause some systems usually rely on personal contextsuch as blood pressure, sugar levels, and others. Con-text in such situations should be communicated in fast,reliable and opportune ways.

• Anticipatory. The system must take actions on behalfof the user displaying proactive behaviour. For exam-ple, if the elder user has suffered a fall, the systemmust alert the nursing home personnel.

• Natural communication with users. This requirementis specially important with the elderly. This kind ofuser presents interaction problems with traditional mech-anisms such as mouse, keyboard, text on screens, etc.A new approach must be taken when displaying data,selecting between options, and requesting input data.Possible approaches could be the use of voice com-mands and visual components. This is a topic of ac-tive research (for instance, see [12]). Also, additionalresearch is needed to accomplish proper usability.

• Natural interaction with users by means of devices theyare used to. Users of ambient intelligent systems shouldconsider the use of everyday items as ways of interac-tion with the users.

• Adaptive. Systems must be designed to be able to re-solve abnormal or exceptional situations autonomously(if possible), and be prepared to support other scenar-ios the user could need to incorporate or change after

time. Also, as technology rapidly evolves, we must de-sign system architectures in such way that new devicesand communication technologies can be exploited (asa research instance, see [10]).

Along with the above listed characteristics, there are otherfeatures:

• Elder users, and specially, elders hosted in a nursinghome are used to stay in well known spaces, for in-stance, their bedrooms, a game area, a TV room, andso on. Augmentation of their spaces can, therefore, becarefully planned.

• Detractors of ambient assisted living usually mentionprivacy as a major concern on supporting scenarios.The elderly, in cases where they are being hosted inhomes for their care taking, are constantly being mon-itored. Their activities are recorded (usually videotaped), their localisation tracked, and so on. Also,as the augmented spaces are basically confined spaces,privacy can be relaxed to some extent and not takeprimary importance on research designs for the elders.

3. RELATED WORKAlarmNET is an AAL system designed to provide assis-

tance through a smart healthcare environment. It specifi-cally targets assisted-living residents and others whom maybenefit from continuous remote healthcare monitoring [21].AlarmNET’s network manages an audit trail and continuousmedical history while preserving resident’s autonomy, com-fort and privacy, thus improving security and quality of life.The AlarmNET system [24] is mainly based on a wirelesssensor network that provides readings of temperature, light,pulse and blood oxygenation. It also features a circadianactivity rhythm analysis program. Similar to AlarmNET,our platform offers a number of different services to supportthe elder, while we also offer a sensorial infrastructure, weprovide other services and tools such as physical actuators,an augmented reality framework, and others, that allow theimplementation of many different application scenarios.

CenceMe, is a personal sensing system that enables usersto share their activities through the Web, particularly throughthe use of social networks [7]. The CenceMe system can iden-tify the status of a user in terms of his activity (sitting, walk-ing, etc.), disposition (happy, sad, etc.), habits (at work, athe gym, etc.), and surroundings (environmental conditions).While our infrastructure also includes a behavioral analysisservice that can be used to infer if a user is seated, walk-ing, or if he has fallen, we are taking this further by offeringa complete solution for ambient assisted living applicationsthat use that information to enable further support.

EMI2lets [8] is a platform that facilitates the developmentand deployment of AmI scenarios. Users of the platform in-teract with smart objects in the environment through theuse of mobile devices. The platform helps on discoveringlocally available services, deploying new software represen-tatives of everyday objects, and creating software controllersto be run on mobile devices. Our contributions also considerthe interaction with augmented objects through the use ofthe sentient visor toolkit, which is not just focused on theinteraction with the object controllers, but on providing anatural way for the user to interact with them. Or focus isalso more general, to provide a platform for the description,

Page 3: Developing Ambient Intelligence Applications for the ...usuario.cicese.mx/~jagm/docs/aviles-ant10acc.pdf · Developing Ambient Intelligence Applications for the Assisted Living of

discovery, and integration of services that produce compo-sitions to facilitate the development of AmI applications,independently of aspects such as the targeted platform, theprogramming language, or the scenario where the final ap-plication is deployed.

4. TECHNOLOGICAL INFRASTRUCTUREBefore presenting and discussing the different scenarios we

have explored to support the elders, we have to talk aboutour technological infrastructure. We have designed and im-plemented a platform featuring basic services and tools toallow the easy composition of hardware and software func-tionality. In the following subsections we introduce and de-tail each of these services and tools.

4.1 ServicesA service in our platform consists of a RESTful [9] Web

service which exposes its functionality by describing whatcommon interfaces it implements, and that notifies its avail-ability within the environment by using the discovery mech-anism provided by the platform. We have designed a numberof common interfaces for those services, including context-acquisition, behaviour analysis, indoor localization, meta-data management, and others.

Once the services are registered in the platform, a devel-oper can create compositions by defining which services areinvolved and the application flow between them. The com-position files are sent to one of the execution engines avail-able in the environment, which are in charge of handling therequests and data involved on the execution of the describedfunctionality. The developer can alternatively use our plat-form framework to directly interact with the discovery anddescription mechanisms to handle its own composition im-plementation.

Some services in our platform have to notify data as ithappens, i.e., they follow an event-based communication. Tosupport this, we have included the publish/subscribe model.For instance, if you want to be notified about new tag read-ings of an RFID service, you must first subscribe to it andspecify the parameters of how and when you will be alerted.As the components of our platform are mostly RESTful Webservices, a notification of data usually consists of a POSTrequest to a target subscribed URI. The notification can beconfigured with aggregation operators.

We have implemented a working prototype of our platformthat includes server, desktop, and mobile versions of theexecution engine. To conduct our cases of study, we alsoimplemented the next basic services:

Indoor Localization This service estimates the user’s po-sition based on the signal’s intensity of surroundingWi-Fi access points. This approach has been widelyused in other work that requires knowledge of user/objectlocation in an indoor environment [14]. The servicehas to be trained before the location can be estimated.Training is done by measuring Wi-Fi signal strengthsat different points of the area to be covered and asso-ciating the measured signal strength with a specific lo-cation. This process is also associated to the platformbeing used, additional training is required for each ofthe different platforms to be used. When an estimationrequest is issued, the service looks into the measure-ments stored during training and estimates a position

by using a custom heuristic based on the similitude ofsignal strengths. During experimental tests and usingbetween 3 and 10 access points, we managed to getan average error of 4 meters, which is enough to sup-port the localization needed in our cases of study (asexplained in Section 5).

Wireless Sensor Networks This service provides a highlevel abstraction to different wireless sensor platforms,allowing the easy acquisition of raw context. The typeof context being acquired depends on the sensorial ca-pability of the nodes in the network. Our setup con-sisted of 8 MicaZ motes and 10 Telos B motes (bothfrom Crossbow, Inc.) which provided us with envi-ronmental (light, temperature, humidity, noise) anduser (audio capture, accelerometer) context. In somespecial cases we attached a sensor node to a user tomonitor his behaviour by tracking the changes of theaccelerometer data. The current implementation ofthis service is based on a previous work [3]. This ser-vice also offers asynchronous notifications following thepublish/subscribe model. An entity should first sub-scribe itself by describing the desired sensor type, thesensing area of interest, the time window, and the ag-gregation operator to be applied to sensor readings be-tween that window (e.g., the hourly average tempera-ture of a room).

RFID Readers This service provides notifications of RFIDtag readings. The intention of this service is to serve asa way for automatic identification of objects and users.We use this service in the taking medications applica-tion scenario (see subsection 5.1) where the medicinebottles should be identified.

Video Feeds The current implementation of this servicetakes the form of a Web widget, a visual componentthat can be easily embedded in other Web pages orsystems. However, the design of the service specifiesthe delivering of a video stream that can be used inconjunction with image processing techniques to offerricher behavioral analysis.

Voice Commands Voice commands are a very natural wayof interaction where user effort is minimized. Withthis service, we enable voice commands on an environ-ment by provisioning it with a wireless sensor networkwhere nodes are equipped with a constantly open mi-crophone. The reliability of the voice recognition net-work is extremely important, as the sensor nodes arehighly prone to failures. We have deployed nodes re-dundantly, often overlapping coverage areas, in thisway, if a node fails, another one can perform its func-tions without disrupting overall system functionality.The main functionality that this service exposes is thenotification of a new command identified by the net-work. To detect a command, the sensor nodes firstcapture sound, and if it is within the human voicerange then they forward it to a base station, wherethe sound is analyzed to determine if it is a command.The base station is the service provider as well (pleaserefer to [17] for in-depth details).

Speech Synthesis Each instance of a provider for this ser-vice is directly associated with a speaker. On a request,

Page 4: Developing Ambient Intelligence Applications for the ...usuario.cicese.mx/~jagm/docs/aviles-ant10acc.pdf · Developing Ambient Intelligence Applications for the Assisted Living of

the message received is synthesized to an audible formand played on the service’s speaker. The synthesizedspeech is another very natural way of interaction withthe users, as we don’t require them to look a displayto acknowledge information.

Actuators This kind of services allows the developers toperform a physical reaction in the environment. Thisreaction can take the form of lights, motors, sounds,and other actuators. We have implemented services totrigger single servo-motors that can be used to opendoors or windows. We designed these services in ageneric way so the developers can easily adapt themto their own application scenarios.

Input Devices We have implemented services to allow sim-ple input, such as buttons, sliders, touch events, rota-tors, and others using Phidget hardware [1]. As statedbefore, one of the common characteristics of an AmIsystem is to provide natural interaction with the usersby means they are used to. We designed the input de-vice services with a general focus, so they can be easilyextended to include other types of devices, for exam-ple, infrared sensors, electric scales, and others. Thistype of services are event-based, any entity interestedon receiving the event of a new user input must firstsubscribe.

Other Services There are other services implemented toallow specialized actions. One example of these ser-vices is the context-processor which receives sensorreading sent by a wireless sensor service and then, af-ter applying a certain logic (through scripts), gener-ates new events. We have used the context-processorservice in the intelligent care monitoring applicationscenario. More services can easily be added to theplatform. The necessary steps are: first, develop iton any programming language (as long as it exposesits functionality through the RESTful approach), sec-ond, describe which platform interfaces are being im-plemented by the service in a service description file,and third, to register it on the service discovery mech-anism (by using one of the many different providedtools).

4.2 ToolsAlong with the design and implementation of the services

in the platform, we have a couple of tools built on top of ourinfrastructure to better support the development of AAL ap-plications for the elderly. The most important one is a sen-tient visor toolkit that can be used to develop applicationsthat allow the interaction of users with augmented objectsin a very natural way.

4.2.1 Sentient VisorsUser interaction on augmented assisted living environ-

ments is intended to be the most natural possible, ideallynot requiring the use of traditional means such as displays,keyboards, and so on. When designing interaction of elderusers with augmented environments, we must think on newapproaches that offer users of advanced age with alternativesthat require no previous knowledge nor training to use. El-ders need a clear and simple human-interaction model that

Virtual Object Representation

PhysicalObject

Automatic Object Identi!cation

ContextAcquisition

AugmentedReality Scenario

AdditionalServices

Figure 1: The application scenario of a sentient vi-sor.

presents them filtered, condensed, and summarized informa-tion based on their personal profile and preferences. Follow-ing the idea introduced by the ambient devices concept [23]we have been working on what we call sentient visors [5]. Asentient visor is a device carried by the user that basicallypresents three characteristics: a screen, a back-facing cam-era (pointing to the back of the screen), and some sensingcapabilities. A tablet computer or a modern smartphonecan be currently used as sentient visors [25].

The scenario of use for a sentient visor is as follows (seeFig. 1). To start the interaction, the user points the camera(and the device) to the augmented object of interest or to anarea in the environment according to what the applicationscenario involves. The device then automatically identifieswhat is being focused, captures whatever additional contextis required by the application, and then sends a request foran augmented reality scenario to a service in our platformto be shown on the device. The augmented reality scenariocould include some sort of basic interaction such as touchingone of the objects on the screen. The device then sends asmany additional requests to the data providers as the appli-cation flow needs. The final scenario is that the user sees anaugmented version of the physical reality and in this way, heis able to interact with the objects he uses in his everydaylife just by pointing the sentient device to it. We use thistool on the taking medications scenario (see subsection 5.1).

4.2.2 Other Development ToolsWe have developed other tools aimed to better assisting

the developers of ambient intelligence systems by provid-ing higher abstraction layers on top of our platform. Forinstance, while the services are already designed to facili-tate the development, the Mashup Editor (see [4] for moredetails) allows the development of applications just by drag-ging graphical representations of the services to form com-positions to be run on the execution engines, which is alsohandled by the editor. Other tools provide Web widgets topresent and manage information, they are also ready to beupdated as new notifications arrive, automatically handlingthe low level communication.

5. APPLICATION SCENARIOSWe now present some scenarios that we came across dur-

ing our work with ambient-assisted living technologies to

Page 5: Developing Ambient Intelligence Applications for the ...usuario.cicese.mx/~jagm/docs/aviles-ant10acc.pdf · Developing Ambient Intelligence Applications for the Assisted Living of

Figure 2: The taking medications application show-ing the two possible results.

promote safer and healthier lifestyles for elders. These sce-narios have also been very useful for highlighting the benefitsof our proposed platform for developing AmI applications.

5.1 Taking MedicationsMr. Smith is a senior of advanced age who needs to

take many different medications on a daily basis. Some-times he forgets to take the medications, and maybe worse,he takes the same dosis of pills twice or thrice because hejust wants to be sure he took it. This situation can leadto adverse, sometimes very serious consequences and it isknown to cause many problems worldwide [18]. Mr. Smithis physically impaired, so it is impossible for him to visithis physician except on certain special occasions. Withoutfrequent visits, the medical examinations are conducted viaphone calls and sometimes his physician changes the dosesby altering the number of pills to be taken or by setting adifferent intaking schedule, which makes the problem evenworse because Mr. Smith must remember the proper dosesof each of the many bottles of pills he takes during the day.

To support this scenario, we first identified which objectsin Mr. Smith’s daily life we could augment to allow him touse the system the way he is used to. We decided to usethe bottles of pills. We attached an RFID tag to each of thebottles to be able to individually identify them. Later, weanalyzed which services were already available in our plat-form so we could use them. They were the RFID reader (forbottle identification) and the speech synthesis services (toalert the user). Then, we identified what custom serviceswere needed to complete the functionality for this scenario.Two services were specially implemented: a service that pro-vides a Web page to input the doses specifications to be usedby the physician, and a service that keeps record of the dosesand that sends the alerts to the user when the schedule ofa doses is met. Finally, to wrap all the services and imple-ment the final application, we used the sentient visor tool tocreate an augmented reality scenario that shows if the dosesfrom a bottle should be taken or if the user should wait untilthe next doses.

The new scenario with the support of our contributionsis as follows: Mr. Smith takes many different medications

Figure 3: Desktop version of the intelligent videomonitoring application.

during the day. When he wants to be sure he is on schedulewith the doses, he takes his mobile device (e.g., smartphone)and points it to each of the bottles of pills. In the screen,a green or red glow appears around the bottle indicatingwhether he needs to take or to wait for a doses; if he needsto take it, the number of pills also appears on the screen, ifhe must wait, an animated countdown appears telling himhow many minutes he should wait until the next doses (seeFig. 2). When he must take the doses, a button appears,to be pushed after he takes it. When Mr. Smith hasn’tused the smartphone and the schedule is not met, the alertservice notifies the voice synthesis service to send an usernotification telling him the number of pills he must imme-diately take. Sometimes, Mr. Smith’s physician logs in intothe doses specification Web page to change the schedule orto check the progress of his patient, but this is all transpar-ent to Mr. Smith, as the system takes care of updates andother details in the background.

5.2 Intelligent Care MonitoringMrs. Jones is a caregiver at a nursing facility. As part of

her activities, she must regularly check the residents to lookif they require assistance. To avoid her to walk around themany different rooms, they installed a closed circuit camerasystem for her. Now, when she is in the office she can keepan eye on the residents by watching a monitor that showsall the camera feeds in the building. Sometimes, one ofthe residents falls, if she is not careful with the inspection,the elder could spent a long time until she realized of theaccident. Given the fact that during the aging process thereis a considerable decline in motor capabilities, older adultsbecome more prone to falls. While Mrs. Jones is not theonly caregiver, they have many other activities, and aftersome time, the camera feeds are no longer being used formonitoring, but for having a record of what happens insidethe building.

To implement this scenario, we provided each of the resi-dents with wearable sensor nodes, in this way, we are awareof their location by using the perceived Wi-Fi signal strengthsand infer their behaviour by analyzing the readings of anembedded accelerometer. We get the locations of the usersthanks to the indoor localization service which we had to

Page 6: Developing Ambient Intelligence Applications for the ...usuario.cicese.mx/~jagm/docs/aviles-ant10acc.pdf · Developing Ambient Intelligence Applications for the Assisted Living of

Wi-Fi Access Points

Wireless Sensor Network

User Controller and Localization Badge

Door Controller

Figure 4: Infrastructure implemented for the voice-enabled assistance scenario.

train before using it. We took readings in 2,500 positions,in each location we took 10 signal strength readings. Weequipped the nursing home with 5 Wi-Fi extra access pointsto enhance the location estimation. Each of the nodes inter-acts directly with the indoor localization service to estimatetheir individual position and then reports it to a centralizedservice. The behaviour analysis is provided by accelerome-ter readings, each sensor reports segments of data each 10seconds to a centralized service, data is analyzed there toconclude if the user is walking, seated, or if he has fallen.The implementation of the centralized service in both casesis a specialization of the context-processor service providedby our platform. We use the video feeds of each camera toprovide a way for the caregivers to confirm that the alertsare real events and avoid false positives. Finally we imple-mented an application, with desktop and mobile versions,that uses the functionality provided by the centralized loca-tion and behavior analysis services to support this scenario.

The new scenario is as follows: Mrs. Jones must checkthe residents regularly. When she is in the office, in herdesktop computer she can see a map of the building showingthe activity and location of each resident. She can click onone of the resident icons and then the system shows thevideo feed of the nearest camera that has that location onsight. When one of the residents suffers a fall, the desktopsystem shows her a visual and audible alert highlighting thelocation and the video feed so she can be sure the alert is nota false positive and then go directly to attend the resident(see Fig. 3). When she and other caregivers are not in theoffice, they receive a fall alert containing the location andan image captured from the nearest camera. Residents canalso use a button in the sensor node to trigger an alert.

5.3 Voice-Enabled AssistanceMr. Brown is a patient who uses an electric-powered

wheelchair to move himself around the building. Very often,he requires assistance to open doors or when he needs to goto the bathroom. When nobody is around he usually findshimself waiting until a caregiver or another patient assistshim.

To support this scenario, we are using the indoor local-ization, voice commands, input device, and actuators ser-vices. We designed a custom input device (and service) thatpresents two buttons for the use of the elder. The applica-

tion (a service composition) is running in the environment,when the user pushes the “command” button, the applica-tion identifies him with its location and waits for the voicecommand service to detect a command, when it receives therequest for an action, it acts accordingly. When the userpresses the “help” button, the system sends a notification toa caregiver in the form of a message containing the locationof the patient (see Fig. 4).

The voice-powered scenario is as follows: Mr. Brown is anelder who presents physical disadvantages that makes himuse a wheelchair. When he needs to open a door, he pressesthe “command” button and says “open”, then the door infront of him opens to let him enter. Once inside, he pressesthe button again and says “close”, then the door closes be-hind him. When Mr. Brown requires assistance from acaregiver, he presses the “help” button, then a caregiver re-ceives his request containing his location so he can be foundrapidly.

6. LESSONS LEARNEDWorking with assisted living systems for supporting the

elderly is a very interesting and challenging task; while theresearch offers a good degree of complexity, the results havean important and quick social impact. When designing theapplication scenarios, we found many issues that requiredvery well planed privacy management, for instance, the in-telligent care monitoring scenario, where the activities ofthe residents of a nursing house are always being tracked.We think that, as long as the application is managed lo-cally, privacy of users in such cases is not so crucial. Onthe other hand, there are scenarios which are more orientedtowards healthcare, where the processing and transmissionof confidential patient data requires important security andprivacy considerations [19]. We faced another problem whendealing with the precision of the indoor localization service.We needed a mechanism to get localization at the lowest costpossible (in terms of computation, infrastructure, and price)which led us to base it on Wi-Fi signal strength readings.Wi-Fi based techniques manage to get estimations withinan approximate error of 20 meters, we needed a better oneand considered a 5 meters error window. The precision wefinally got was satisfactorily enough but this resulted par-ticularly difficult on the voice-enabled assistance scenario,where the user activates the doors in front of him. Luckily,doors in the nursing room in which we conducted the ex-perimentation were spread in far longer distances than ourerror window.

7. CONCLUSIONS AND FUTURE WORKIn this article, we approached the use of ambient intelli-

gence systems in the context of ambient assisted living forsupporting the elder. This is a topic that has been pursuedrecently and it has gained greater importance because of theworldwide increase in average life expectancy [20]. We alsopresented our hardware and software infrastructure to sup-port this kind of applications and discussed the implemen-tation of a number of scenarios conducted in our researchgroup. We believe our proposals will contribute in improvingthe lifestyle of the elderly by allowing the rapid constructionof prototypes for usability, and application-driven research.

As stated before, the elder users have quite specific re-quirements, and their relationship with augmented objects

Page 7: Developing Ambient Intelligence Applications for the ...usuario.cicese.mx/~jagm/docs/aviles-ant10acc.pdf · Developing Ambient Intelligence Applications for the Assisted Living of

and modern interface technologies have yet to be carefullyanalyzed. We plan to conduct an extensive evaluation of ourcontributions from two approaches. First, to evaluate theconstruction of ambient assisted living applications for theelderly with the use of our platform from the perspective ofthe developers of such applications. Second, to evaluate theusability and acceptance of technology of the applicationsfrom the perspective of the elderly user. We have success-fully tested the implemented applications in our laboratory,the next step is to conduct the usability studies by using ourprototypes in the nursing home in which we conducted ouranalysis.

8. ACKNOWLEDGMENTSThe work presented in this paper was funded with grants

from CONACyT, the Mexican Council for Science and Tech-nology.

9. REFERENCES[1] Phidgets, Inc. http://www.phidgets.com/.

[2] E. Aarts, H. Harwig, and M. Schuurmans. Theinvisible future: the seamless integration of technologyinto everyday life. McGraw-Hill, Inc., New York, NY,USA, 2002.

[3] E. Aviles-Lopez and J. A. Garcıa-Macıas. TinySOA: aservice-oriented architecture for wireless sensornetworks. Service Oriented Computing andApplications, 3(2):99–108, 2009.

[4] E. Aviles-Lopez and J. A. Garcıa-Macıas. UbiSOADashboard: Integrating the physical and digitaldomains through mashups. In Human Interface andthe Management of Information. DesigningInformation Environments, pages 466–474.Springer-Verlag, Berlin, Heidelberg, 2009.

[5] E. Aviles-Lopez, I. Villanueva-Miranda, J. A.Garcıa-Macıas, and L. E. Palafox-Maestre. Takingcare of our elders through augmented spaces. InProceedings of the Latin-American Conference onHuman-Computer Interaction (CLIHC ’09), Nov 2009.

[6] R. Bellazzi, S. Montani, A. Riva, and M. Stefanelli.Web-based telemedicine systems for home-care:technical issues and experiences. Computer Methodsand Programs in Biomedicine, 64(3):175–187, 2001.

[7] A. T. Campbell, S. B. Eisenman, K. Fodor, N. D.Lane, H. Lu, E. Miluzzo, M. Musolesi, R. A. Peterson,and X. Zheng. CenceMe: Injecting sensing presenceinto social network applications using mobile phones.In Proc. of ACM 9th International Symposium onMobile Ad Hoc Networking and Computing(MobiHoc’08), Hong Kong, May 2008.

[8] D. L. de Ipina, J. Vazquez, D. Garcıa, J. Fernandez,I. Garcıa, D. Sainz, and A. Almeida. Emi2lets: Areflective framework for enabling AmI. Journal ofUniversal Computer Science, 12(3):297–314, 2006.

[9] R. T. Fielding. Architectural styles and the design ofnetwork-based software architectures. PhD thesis,University of California, Irvine, 2000.

[10] L. Fuentes, D. Jimenez, and M. Pinto. Development ofambient intelligence applications using componentsand aspects. Journal of Universal Computer Science,12(3):236–251, 2006.

[11] B. Goncalves, J. G. P. Filho, R. V. Andreao, andG. Guizzardi. ECG data provisioning for telehomecaremonitoring. In SAC ’08: Proceedings of the 2008 ACMsymposium on Applied computing, pages 1374–1379,New York, NY, USA, 2008. ACM.

[12] T. Kleinberger, M. Becker, E. Ras, A. Holzinger, andP. Muller. Ambient intelligence in assisted living:Enable elderly people to handle future interfaces. InUniversal Access in Human-Computer Interaction.Ambient Interaction, pages 103–112. 2007.

[13] I. Korhonen. Health monitoring in the home of thefuture. IEEE Eng. Med. Biol. Mag., 22(3):66–73, 2003.

[14] J. Krumm and E. Horvitz. LOCADIO: Inferringmotion and location from Wi-Fi signal strengths.Mobile and Ubiquitous Systems, Annual InternationalConference on, 0:4–13, 2004.

[15] D. Malan, T. Fulford-Jones, M. Welsh, andS. Moulton. CodeBlue: An ad hoc sensor networkinfrastructure for emergency medical care. In InInternational Workshop on Wearable and ImplantableBody Sensor Networks, 2004.

[16] R. Oppermann, R. Rashev, and K. Kinshuk.Adaptability and adaptivity in learning systems.Knowledge Transfer, 2:173–179, 1997.

[17] L. E. Palafox and J. A. Garcıa-Macias. Wireless sensornetworks for voice capture in ubiquitous homeenvironments. In ISWPC’09: Proceedings of the 4thinternational conference on Wireless pervasivecomputing, pages 332–336, Piscataway, NJ, USA,2009. IEEE Press.

[18] P. A. Routledge, M. S. O’Mahony, and K. W.Woodhouse. Adverse drug reactions in elderlypatients. British journal of clinical pharmacology,57(2):121–126, 2004.

[19] C. Ruan and S.-S. Yeo. Modeling of an intelligente-consent system in a healthcare domain. Journal ofUniversal Computer Science, 15(12):2429–2444, 2009.

[20] United Nations. World population prospects the 2008revisions. http://www.un.org/esa/population/-publications/wpp2008/wpp2008 highlights.pdf.

[21] G. Virone, A. Wood, L. Selavo, Q. Cao, L. Fang,T. Doan, Z. He, R. Stoleru, S. Lin, and J. A.Stankovic. An assisted living oriented informationsystem based on a residential wireless sensor network.In Proc. 1st Transdisciplinary Conference onDistributed Diagnosis and Home Healthcare (D2H22006), pages 95–100, Arlington, VA, USA, 2006.

[22] M. Weiser and J. S. Brown. The coming age of calmtechnology. Beyond Calculation: The Next Fifty Yearsof Computing, pages 75–85, 1997.

[23] C. Wisneski, H. Ishii, A. Dahley, M. G. Gorbet,S. Brave, B. Ullmer, and P. Yarin. Ambient displays:Turning architectural space into an interface betweenpeople and digital information. In CoBuild ’98:Proceedings of the First International Workshop onCooperative Buildings, Integrating Information,Organization, and Architecture, pages 22–32, London,UK, 1998. Springer-Verlag.

[24] A. Wood, G. Virone, T. Doan, Q. Cao, L. Selavo,Y. Wu, L. Fang, Z. He, S. Lin, and J. Stankovic.ALARM-NET: Wireless sensor networks forassisted-living and residential monitoring. Technical

Page 8: Developing Ambient Intelligence Applications for the ...usuario.cicese.mx/~jagm/docs/aviles-ant10acc.pdf · Developing Ambient Intelligence Applications for the Assisted Living of

Report CS-2006-11, Department of Computer Science,University of Virginia, 2006.

[25] A. Wright. Get smart. Communications of the ACM,52(1):15–16, 2009.