Vision-based monitoring of people in private spaces
-
Upload
francisco-paco-florez-revuelta -
Category
Technology
-
view
161 -
download
1
description
Transcript of Vision-based monitoring of people in private spaces
Francisco Flórez-Revuelta Digital Imaging Research Centre
Vision-based monitoring of people in private spaces
Presentation Architecture Human action recognition Optimisation Privacy by the context Discussion
Overview
Associate Professor (now on leave) at the University of Alicante (Spain) Member of the "Domotics and Ambient Intelligence" research group Multi-disciplinary research: computer vision, computational intelligence (artificial neural networks, evolutionary computation), home automation… Research on ICT for smart environments (services at home, system integration, efficient use of energy, ambient assisted living, person-environment interaction…) Development of person-environment interfaces for the interaction of people with their environment, paying particular attention to the support of the independent living of elderly and disabled people
My background
Joined Kingston University in 2011 as Marie Curie fellow The Robot Vision Team at the Digital Imaging Research Centre had a huge experience on video-surveillance Combine their expertise on video-surveillance with mine on ambient assisted living
My background
A study on people activity in private spaces: towards a multisensor network that meets privacy requirements Project funded by the European Commission – 2011-2013
Objectives Study, design and development of a distributed vision system for human motion analysis in a private environment Development of appropriate techniques for maintaining the privacy of the individuals being monitored Integration of the vision system with other sensors forming a perceptual, unobtrusive and interoperable network Integration and validation of the system in a smart home environment
Senior Researcher since 2013 Strengthen research on ambient assisted living and intelligent environments at Kingston University
Current position
Platform for self-assessment and efficient management for informal caregivers Project funded by the European Commission Ambient Assisted Living Joint Programme (call 5) – 2013-2015
Activities and events to be detected: Recognition of activities of daily living (ADL): wake-up, going to the toilet, sleeping… Activity level: light, moderate, intensive Location and tracking of the person inside the home Risky situations: fall detection, dangerous objects; fire, flood, gas alarms… Social activity: leaving home, receiving visits, use of telephone… Detection of basic and advanced activities: cooking, watching TV…
Architecture
Camera 1
Motion Detection
Human Behaviour Analysis
Multi-view Human Behaviour Analysis
Privacy
Reasoning System
Alarm Actuators
Camera 2
Motion Detection
Human Behaviour Analysis
Camera N
Motion Detection
Human Behaviour Analysis
...
Setup and Profiles DB (Activities, Inhabitants, Objects, ...)
Log
Event
Long-term analysis
...
...
Environmental Sensor Information
Caregiver
Human behaviour analysis
Feature extraction
Bag of key poses
Multiple views
Results
Weizmann MuHAVi-‐8
MuHAVi-‐14
IXMAS
Use of RGB-D cameras
Optimisation
We need to recognise the action/activity in order to apply an appropiate privacy filter A high success rate is compulsory
Evolutionary feature subset selection
Some features are not needed or they introduce noise Use of an evolutionary algorithm as a wrapper Chromosomes are encoded as binary vectors, each gene representing whether a feature is or is not used for recognition
Results
Weizmann MuHAVi-‐8
MuHAVi-‐14
DHA
IXMAS
Evolutionary feature subset selection
Chromosome
Normalisa=on
Crossover
Results
Results
Cooperative coevolutionary optimisation
Simultaneous selection of the training set, feature set and parameters of the clustering algorithm to improve the success rate
Cooperative coevolutionary optimisation
Cooperative coevolutionary optimisation
Cooperative coevolutionary optimisation
Privacy preservation
Privacy by the context Context:
identity of the subject appearance location on-going activity identity of the observer
Privacy filters
Current work
Fusion of multiple vision devices: multiple RGB-D devices, RGB and RGB-D cameras… Feature selection from a large bank of features Incremental and adaptive learning Continuous activity recognition Validation in real scenarios Monitoring people with all kind of sensors, fusion with environmental sensors Anything related to intelligent environments
Papers
Human activity recognition Chaaraoui, A.A.; Padilla-López, J.R.; Ferrández-Pastor, F.J.; Nieto-Hidalgo, M.; Flórez-Revuelta, F.: A Vision-based System for Intelligent Monitoring: Human Behaviour Analysis and Privacy by Context, Sensors, 14(5):8895-8925, 2014. Chaaraoui, A.A.; Flórez-Revuelta, F.: Adaptive Human Action Recognition With an Evolving Bag of Key Poses, IEEE Transactions on Autonomous Mental Development (accepted for publication) Chaaraoui, A.A.; Padilla-López, J.R.; Flórez-Revuelta, F.: Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices, 3rd Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV), ICCV 2013 Workshop, Sydney (Australia), 2013. Chaaraoui, A.A.; Climent-Pérez, P.; Flórez-Revuelta, F.: Silhouette-based Human Action Recognition using Sequences of Key Poses, Pattern Recognition Letters, 34(15):1799–1807, 2013. Chaaraoui, A.A.; Climent-Pérez, P.; Flórez-Revuelta, F.: An Efficient Approach for Multi-view Human Action Recognition Based on Bag-of-Key-Poses, Lecture Notes in Computer Science, 7559:29-40, 2012. Chaaraoui, A.A.; Climent-Pérez, P.; Flórez-Revuelta, F., A review on vision techniques applied to Human Behaviour Analysis for Ambient-Assisted Living. Expert Systems with Applications, 39(12): 10873-10888, 2012.
Papers
Evolutionary optimisation Chaaraoui, A.A.; Padilla-López, J.R.; Climent-Pérez, P.; Flórez-Revuelta, F.: Evolutionary joint selection to improve human action recognition with RGB-D devices, Expert Systems with Applications, 41(3):786-794, 2014. Chaaraoui, A.A.; Flórez-Revuelta, F.: Optimizing human action recognition based on a cooperative coevolutionary algorithm, Engineering Applications of Artificial Intelligence, Volume 31:116–125, 2014. Chaaraoui, A.A.; Flórez-Revuelta, F.: Human action recognition optimization based on evolutionary feature subset selection, Proceeding of the 15th Annual Genetic and Evolutionary Computation Conference, Amsterdam (The Netherlands), 2013. Climent-Pérez, P.; Chaaraoui, A.A.; Padilla-López, J.R.; Flórez-Revuelta, F.: Optimal Feature Selection for Skeletal Data from RGB-D Devices using a Genetic Algorithm, 11th Mexican International Conference on Artificial Intelligence, San Luis Potosí (Mexico), 2012.
Papers
Privacy preservation Padilla-López, J.R.; Chaaraoui, A.A.; Flórez-Revuelta, F.: Visual privacy by context: A level-based visualisation scheme (submitted to conference) Padilla-López, J.R.; Chaaraoui, A.A.; Flórez-Revuelta, F.: Visual privacy protection methods: a survey (submitted to journal) Padilla-López, J.R.; Flórez-Revuelta, F.; Monekosso, D.N.; Remagnino, P.: The "Good" Brother: Monitoring People Activity in Private Spaces, International Symposium on Distributed Computing and Artificial Intelligence (DCAI 2012), Salamanca (Spain), pp. 49-56, 2012.
Papers
Home automation and intelligent environments Ferrández-Pastor, F.J.; García-Chamizo, J.M.; Romacho-Agud, V.; Flórez-Revuelta, F.: Using the wavelet transform to disaggregate electrical power consumption into the major end-uses (submitted to journal) Ferrández-Pastor, F.J.; Nieto-Hidalgo, M.; García-Chamizo, J.M.; Flórez-Revuelta, F.: DAI Virtual Lab: a Virtual Laboratory for Testing Ambient Intelligence Digital Services, V Congreso Internacional de Diseño, Redes de Investigación y Tecnología para Todos (DRT4ALL 2013), Madrid (Spain), 2013. Climent-Pérez, P.; Chaaraoui, A.A.; Flórez-Revuelta, F.: Useful Research Tools for Human Behaviour Understanding in the Context of Ambient Assisted Living, International Symposium on Ambient Intelligence (ISAmI 2012), Salamanca (Spain), pp. 201-205, 2012. Nieto-Hidalgo, M.; Limiñana-Benito, A.; Torregrosa-Jorqués, R.; Padilla-López, J.R.; Cabo-Díez, M.; Ferrández-Pastor, F.J.; Romacho-Agud, V.; Climent-Pérez, P.; Chaaraoui, A.; Flórez-Revuelta, F.: DAI Virtual Lab: a virtual lab for the co-design, development and validation of AmI services and technologies, International Joint Conference on Ambient Intelligence (AmI-11), Amsterdam (Nederlands), 2011.
Francisco (Paco) Flórez-Revuelta [email protected] @fflorezrevuelta www.dtic.ua.es/~florez franciscoflorezrevuelta