Automated Activity Recognition and Monitoring ofElderly Using Wireless Sensors: Research Challenges

Damith C. Ranasinghe, Roberto L. Shinmoto Torres, Asanga WickramasingheAuto-ID Lab

School of Computer ScienceThe University of AdelaideSouth Australia, Australia

Emails: {damith.ranasinghe,roberto.shinmototorres,asanga.wickramasinghe}@adelaide.edu.au

I. INTRODUCTION

A rapidly growing aging population presents many chal-lenges to health and aged care services around the world.Recognising and understanding the activities performed byelderly is an important research area that has the potentialto address these challenges and healthcare needs of the 21st

century by enabling a wide range of valuable applications suchas remote health monitoring. A key enabling technology forsuch applications is wireless sensors. However we must firstovercome a number of challenges that are technological, socialand economic, before being able to realize such applicationsusing pervasive technologies.

II. CHALLENGES

Difficulties in human activity monitoring in older popula-tions arise from many domain specific challenges. With anymonitoring application the person has to interact with sensorsto enable the collection of their movement data. Body wornsensors [1], [2], [3], [4] ensure that the readings obtainedcorrespond to the person only. In contrast, environmentalsensors [5], [6], where unrelated sensor readings can confusethe system. Such ambient intelligence based approaches limitmonitoring to environments with the capability to gather therequired sensory data. However, body worn sensors enablesmonitoring of the subject irrespective of the location [7]; whichis essential for prolong observations[8].

In particular, body worn sensors have taken a renewedfocus recently with the rapidly increasing interest in body areanetworks [9]. Firstly, activity recognition system employingwearable sensors must be acceptable to the user and the usershould feel comfortable [7], [10], [11] but this is not thecase for many existing wireless sensors platforms[1], [2], [3].Therefore any sensor deployment must be clinically evaluatednot only from the patients’ perspective, but also from thecarers’ perspective [7], [8], [12].

Secondly, whatever the approach chosen, all researchersmust consider privacy related when monitoring peoples’ ac-tivities (such as going to the rest room although relevantfrom a safety and medical care perspective [13] to ensurethe translation of research into practice. The result of the thestudy by Govercin et. al [7] reveals the dislike among elderlyfor video monitoring and their concerns regarding privacyviolations.

Thirdly, robust classification systems are required to dealwith sensorial input data in order to recognize activities.Accurate results are important but so is the responsivenessof the system, as promptness to recognize a high risk activityand, for example, seek an intervention in the event of a highfalls risk activity (such as getting out of a bed) is crucialfor any monitoring system. A key issue in the developmentof classifiers is that the recognition models developed forone population sample may not work interchangeably in adifferent population [14]. In particular, classification modelsbased on empirical studies, such as threshold based approachesdo not generalize to larger populations and in fact personalizedmodels may be required [15]. Thus, researchers are lured intothe machine learning domain in learning generalized classifi-cation models. Most of the machine learning approaches relyon annotated recordings (ground truth) of activities. Thoughwearable sensors make human activity data abundant [16], datawith sufficiently detailed annotations are scares due to: i) theinherent issues in the annotation process [17], [18]; and ii)the difficulty in collecting data from target domains (such aselderly volunteers). This leads to the requirement for moreadvanced machine learning paradigms [16]. Moreover, moststudies are performed with young healthy volunteers but bodymechanics are different for young, older people and peoplewith dementia or other cognitive impairments [19]. Thesedifferences may result that recognition models often developedin young adult volunteers do not work in elderly populations[14].

Finally, there are challenges imposed by the characteristicsof sensor data, the nature of the wireless medium (wireless sig-nal occlusion by interveining objects, noise in sensor data), andmore recenly the passive powering of sensors [20]. Interleavingand overlapping activities further increases the complexity [16]and the streaming nature of the data imposes further challengesin data segmenting (defining windows) for feature extraction[21] for classification algorithms as well as the responsivenessof the system (the need to be real-time).

In the following sections we will describe our prelimi-nary investigations and results into developing an interventionaimed at reducing the falls rate in older people with cogni-tive impairments and dementia. The intervention is enabledby the development of a wearable passive Radio FrequencyIdentification (RFID) based sensor [20] to enable the real-timeclassification of activities recognized as leading to falls [4],[22].

TABLE I. RESULTS FROM THE PILOT STUDY

High Risk Activity Sensitivity SpecificitySitting down on a chair 92.2% 97.9%

Getting up from a chair 90.4% 94.0%

Getting into bed 100% 100%

Getting out of bed 100% 100%

Entering a room/restroom 100% 100%

Leaving a room/restroom 100% 100%

Walking without a walking aid 100% 100%

Ceiling Mounted

Antenna

Alarms/Response Team

Sensor (S)enabled RFID tag

Who

Where

What

When

Real-Time Monitoring EnvironmentEvent Driven

Patient

Monitoring

Software

Other Wards

Fig. 1. An overview of the architecture of the MSA intervention.

III. MOVEMENT SENSOR ALARM INTERVENTION

Falls in hospital is a major safety and quality issue. Fallsprevention is a national priority. It is established that despitebest clinical practice (relying on staff) falls rate in hospitalsremain worryingly high. Our innovative approach [23] auto-matically and in real-time: i) monitor patients activities; ii)identify high falls risk related activities (see Table I); and iii)alert clinical staff in the event of a high falls risk activityallowing for pre-emptive supervision to mitigate the risk ofa fall (see Figure 1). Furthermore, in the event of a falla caregiver is able to provide immediate relief and executeemergency protocols.

The Movement-Sensor-Alarm (MSA) intervention is ex-pected to reduce falls and therefore healthcare costs, injuryand trauma, a common sequela of falls. In-hospital fallsoccur far too commonly in older people and have long termdevastating impacts not only on functional independence butalso on healthcare costs. It has been estimated that fallscontributes to an additional 886,000 bed days per year inAustralia (population 23 million, Australian Commission onSafety and Quality in Health Care 2009). Clearly, the costimpact will be greater in countries with larger populationsor with greater population ageing. Falls prevention will avoidthe negative health outcomes arising from injury and fractureswhile large gains will be seen as a result of health and agedcare cost savings. Table I outlines the results of a study withhealthy adult volunteers to investigate the effectiveness of ourtechnological innovation. Here the activities are identified inisolation to inform our future research.

Sensitivity and specificity estimates above 90% were ob-tained across the board indicating satisfactory performance.The researchers are currently: i) developing a new wearablesensor; ii) developing classifiers cable of automatically recon-ising multiple activities; iii) developing a middleware extensionfiltering and collection of sensor data based on sensor enabledRFID tags; iii) undertaking older volunteers trial to evaluatethe performance of the classification algorithms (first results

Fig. 2. Wearable wireless RFID based sensor with flexible antenna.

are published in [24]); iv) developing automated methods foraddressing the data imbalance issue (inherent to this domain)in training activity recognition classifiers.

IV. RELATED WORKS

To date, there has only been one clinical study confirmingthe benefits of a technological intervention (Health InformationToolkit) in acute care. However the technology only involveda tool to aid the implementation of best clinical practice (careplanning, risk patient education, etc. [25]). In contrast, thenovel MSA intervention aims to prevent falls among patients inacute hospitals using a wearable, single sensor (accelerometer)enabled, passive Radio Frequency Identification (RFID) tagsfor inferring patient activities (Figure 2) [20]. Therefore, amajor innovation of this research is the development andapplication, for the first time, of low cost passive (battery freeand maintenance free) and wearable RFID tags for elder carein hospitals as well as real time interpretation to alert cliniciansin a timely manner. The MSA intervention, in contrast to usingcameras, preserves a patients privacy since patient activities areinferred indirectly.

Previous studies focused on the monitoring of single highrisk activities such as bed exiting, as in the case of Hilbe et al.[26] and Bruyneel et al. [27]. These methods relied on the useof different types of sensors such as pressure sensors on railsand presence sensors in bed mats. These studies achieved highperformance results. However their methods were constrainedto a single activity; moreover, required constant maintenancedue to the mechanical nature of the devices and high risk ofcontact with contaminated fluids. Other studies focused on themonitoring of human activities by instrumenting the patientas is our case, studies by Karantonis et al. [3] and Godfreyet al. [2] achieved good results, the former even resolved realtime monitoring; however, the use of bulky sensors and data-loggers or transponders make this technology not applicable toelderly patients. Other studies focused on self monitoring ofthe subject with prevention routines such as Timed up-and-gotest (TUGT) and Alternate step test (AST) [28], [29]. Thesemethods achieved high performance results; however, are notdesigned for long term monitoring and require the subject tobe cognitively healthy. Finally, other studies [30], [31], [17],[32] have considered the use of machine learning techniquesto build strong classifiers achieving high performance results;however the use of bulky sensors and equipment as in [30]

and multiple sensors on body [31], [33] make these studiesdifficult to deploy in an elderly care environment.

V. SUMMARY

The proliferation of pervasive technologies into the health-care space is now a reality, however, there are significantchallenges to be overcome before we will see the widespreaduse of such technologies in practice.

The MSA intervention, in contrast to using cameras, pre-serves a patients privacy since patient activities are inferredindirectly. The innovative system has wide ranging application:i) application in nursing homes; ii) automatic prediction ordetection of medical symptoms using data mining and; iii)tailoring of treatments such as monitoring and medication doseadjustment to minimise symptoms and disability in people withhealth conditions such as Parkinson’s Disease.

Finally as with other healthcare related technologies, inorder to make an impact we must first show evidence thatsuch technologies can indeed make a real impact. This willrequire that we undertake controlled trials (such as randomisedcontrolled trials) to show the effectiveness of the technologyin practice.

REFERENCES

[1] B. Najafi, K. Aminian, A. Paraschiv-Ionescu, F. Loew, C. Bula, andP. Robert, “Ambulatory system for human motion analysis using akinematic sensor: monitoring of daily physical activity in the elderly,”IEEE Trans. Biomed. Eng., vol. 50, no. 6, pp. 711–723, 2003.

[2] A. Godfrey, A. Bourke, G. Olaighin, P. van de Ven, and J. Nelson,“Activity classification using a single chest mounted tri-axial accelerom-eter,” Medical engineering & physics, vol. 33, no. 9, pp. 1127–1135,2011.

[3] D. Karantonis, M. Narayanan, M. Mathie, N. Lovell, and B. Celler,“Implementation of a real-time human movement classifier using atriaxial accelerometer for ambulatory monitoring,” IEEE Trans. Inf.Technol. Biomed., vol. 10, no. 1, pp. 156–167, 2006.

[4] D. C. Ranasinghe, R. L. Shinmoto Torres, K. D. Hill, and R. Vis-vanathan, “Low cost and batteryless sensor-enabled radio frequencyidentification tag based approaches to identify patient bed entry andexit posture transitions,” Gait & Posture, in press.

[5] D. J. Cook, “Learning setting-generalized activity models for smartspaces,” IEEE Intell. Syst., vol. 27, no. 1, pp. 32–38, 2012.

[6] M. Buettner, R. Prasad, M. Philipose, and D. Wetherall, “Recognizingdaily activities with RFID-based sensors,” in Proceedings of the 11thinternational conference on Ubiquitous computing. ACM, 2009, pp.51–60.

[7] M. Govercin, Y. Kltzsch, M. Meis, S. Wegel, M. Gietzelt, J. Spehr,S. Winkelbach, M. Marschollek, and E. Steinhagen-Thiessen, “Definingthe user requirements for wearable and optical fall prediction and falldetection devices for home use,” Informatics for Health & Social Care,vol. 35, no. 3/4, pp. 177–187, Sep. 2010.

[8] R. Paoli, F. J. Fernndez-Luque, G. Domnech, F. Martnez, J. Zapata,and R. Ruiz, “A system for ubiquitous fall monitoring at home via awireless sensor network and a wearable mote,” Expert Systems withApplications, vol. 39, no. 5, pp. 5566–5575, Apr. 2012.

[9] B. Calhoun, J. Lach, J. Stankovic, D. Wentzloff, K. Whitehouse,A. Barth, J. Brown, Q. Li, S. Oh, N. Roberts, and Y. Zhang, “Bodysensor networks: A holistic approach from silicon to users,” Proceedingsof the IEEE, vol. 100, no. 1, pp. 91–106, 2012.

[10] A. K. Bourke, P. W. van de Ven, A. E. Chaya, G. M. OLaighin, andJ. Nelson, “The design and development of a long-term fall detectionsystem incorporated into a custom vest for the elderly.” in Engineeringin Medicine and Biology Society, 2008. EMBS 2008. 30th AnnualInternational Conference of the IEEE, 2008, p. 28362839.

[11] A. K. Bourke, P. W. van de Ven, A. E. Chaya, G. M. Olaighin, andJ. Nelson, “Testing of a long-term fall detection system incorporatedinto a custom vest for the elderly,” in Engineering in Medicine and Bi-ology Society, 2008. EMBS 2008. 30th Annual International Conferenceof the IEEE, 2008, p. 28442847.

[12] M. Ferrari, B. Harrison, O. Rawashdeh, R. Hammond,Y. Avery, M. Rawashdeh, W. Sadeh, and M. Maddens,“Clinical feasibility trial of a motion detection system for fallprevention in hospitalized older adult patients,” Geriatric Nursing,vol. 33, no. 3, pp. 177–183, May 2012. [Online]. Available:http://www.sciencedirect.com/science/article/pii/S0197457211006045

[13] S. Londei, J. Rousseau, F. Ducharme, A. St-Arnaud, J. Meunier, J. Saint-Arnaud, and F. Giroux, “An intelligent videomonitoring system forfall detection at home: perceptions of elderly people,” Journal ofTelemedicine and Telecare, vol. 15, no. 8, pp. 383–390, 2009.

[14] Y.-D. Lee and W.-Y. Chung, “Wireless sensor networkbased wearable smart shirt for ubiquitous health andactivity monitoring,” Sensors and Actuators B: Chemical, vol.140, no. 2, pp. 390–395, Jul. 2009. [Online]. Available:http://www.sciencedirect.com/science/article/pii/S0925400509003724

[15] R. Y. W. Lee and A. J. Carlisle, “Detection of falls using accelerometersand mobile phone technology,” Age and Ageing, May 2011.

[16] M. Stikic, “Towards less supervision for scalable recognition ofdaily activities,” phd, TU Darmstadt, May 2010. [Online]. Available:http://tuprints.ulb.tu-darmstadt.de/2174/

[17] M. Stikic, K. Van Laerhoven, and B. Schiele, “Exploring semi-supervised and active learning for activity recognition,” in 12th IEEEInternational Symposium on Wearable Computers, 2008. ISWC 2008,Oct. 2008, pp. 81 –88.

[18] D. T. G. Huynh, “Human activity recognition with wearable sensors,”phd, TU Darmstadt, Sep. 2008.

[19] J. Wang, R. Chen, X. Sun, M. F. She, and Y. Wu, “Recognizinghuman daily activities from accelerometer signal,” ProcediaEngineering, vol. 15, pp. 1780–1786, 2011. [Online]. Available:http://www.sciencedirect.com/science/article/pii/S1877705811018327

[20] T. Kaufmann, D. C. Ranasinghe, M. Zhou, and C. Fumeaux, “Wearablequarter-wave microstrip antenna for passive UHF RFID applications,”International Journal of Antennas and Propagation, 2013, available on-line.

[21] N. C. Krishnan and D. J. Cook, “Activity recognition on streamingsensor data,” Pervasive and Mobile Computing, no. 0, 2012.

[22] D. C. Ranasinghe, R. L. Shinmoto Torres, A. P. Sample, J. R. Smith,K. Hill, and R. Visvanathan, “Towards falls prevention: A wearablewireless and battery-less sensing and automatic identification tag for realtime monitoring of human movements,” in 2012 Annual InternationalConference of the IEEE Engineering in Medicine and Biology Society(EMBC), 2012, pp. 6402–6405.

[23] R. Visvanathan, D. C. Ranasinghe, and R. L. S. Torres, “Framework forpreventing falls in acute hospitals using passive sensor enabled radiofrequency identification technology,” in 34th International IEEE EMBSConference (EMBC 2012), San Diego, USA., 2012.

[24] R. L. Shinmoto Torres, D. C. Ranasinghe, Q. Shi, and A. P. Sample,“Sensor enabled wearable rfid technology for mitigating the risk of fallsnear beds,” in 2013 IEEE International Conference on RFID, 2013.

[25] P. C. Dykes, D. L. Carroll, A. Hurley, S. Lipsitz, A. Benoit, F. Chang,S. Meltzer, R. Tsurikova, L. Zuyov, and B. Middleton, “Fall preventionin acute care hospitals,” JAMA: the journal of the American MedicalAssociation, vol. 304, no. 17, pp. 1912–1918, 2010.

[26] J. Hilbe, E. Schulc, B. Linder, and C. Them, “Development and alarmthreshold evaluation of a side rail integrated sensor technology forthe prevention of falls,” International journal of medical informatics,vol. 79, no. 3, pp. 173–180, 2010.

[27] M. Bruyneel, W. Libert, and V. Ninane, “Detection of bed-exit eventsusing a new wireless bed monitoring assistance,” International Journalof Medical Informatics, vol. 80, no. 2, pp. 127–132, 2011.

[28] Y. Liu, S. Redmond, N. Wang, F. Blumenkron, M. Narayanan, andN. Lovell, “Spectral analysis of accelerometry signals from a directed-routine for falls-risk estimation,” IEEE Trans. Biomed. Eng., vol. 58,no. 8, pp. 2308–2315, 2011.

[29] S. Redmond, M. Scalzi, M. Narayanan, S. Lord, S. Cerutti, and

N. Lovell, “Automatic segmentation of triaxial accelerometry signalsfor falls risk estimation,” in 2010 Annual International Conference ofthe IEEE Engineering in Medicine and Biology Society (EMBC), 2010,pp. 2234–2237.

[30] G. Cohn, D. Morris, S. Patel, and D. Tan, “Humantenna: using the bodyas an antenna for real-time whole-body interaction,” in Proceedings ofthe 2012 ACM annual conference on Human Factors in ComputingSystems, ser. CHI ’12. New York, NY, USA: ACM, 2012, pp. 1901–1910.

[31] C. Doukas and I. Maglogiannis, “Emergency fall incidents detectionin assisted living environments utilizing motion, sound, and visualperceptual components,” IEEE Trans. Inf. Technol. Biomed., vol. 15,no. 2, pp. 277–289, 2011.

[32] M. Yuwono, B. D. Moulton, S. W. Su, B. G. Celler, and H. T. Nguyen,“Unsupervised machine-learning method for improving the performanceof ambulatory fall-detection systems,” BioMedical Engineering OnLine,vol. 11, no. 1, pp. 1–11, Dec. 2012.

[33] J. P. Varkey, D. Pompili, and T. A. Walls, “Human motion recognitionusing a wireless sensor-based wearable system,” Personal and Ubiqui-tous Computing, vol. 16, no. 7, pp. 897–910, Oct. 2012.