Activity recognition based on a multi-sensor meta-classifier
-
Upload
oresti-banos -
Category
Science
-
view
229 -
download
2
description
Transcript of Activity recognition based on a multi-sensor meta-classifier
Activity recognition based on a multi-sensor hierarchical-
classifier
IWANN 2013, 12-14 June, Tenerife (Spain)
Oresti Baños, Miguel Damas, Héctor Pomares and Ignacio Rojas Department of Computer Architecture and Computer Technology, CITIC-UGR,
University of Granada, SPAIN
DG-Research Grant #228398
Introduction
• Activity recognition concept
– “Recognize the actions and goals of one or more agents from a series of observations on the agents' actions and the environmental conditions”
• Applications (among others)
– eHealth (AAL, telerehabilation)
– Sports (performance improvement, injury-free pose)
– Industrial (assembly tasks, avoidance of risk situations)
– Gaming (Kinect, Wii Mote, PlayStationMove)
• Categorization by sensor modality
– Ambient
– On-body
2
Sensing Activity
3
• Ambient sensors
Sensing Activity
• Ambient sensors
Limitations*
3rd Generation (and beyond…)
2nd Generation 1st Generation
Sensing Activity
5
• On-body sensors
Activity Recognition Chain (ARC)
6
Activity Recognition Chain (ARC)
7
Activity Recognition Chain (ARC)
8
Activity Recognition Chain (ARC)
9
Activity Recognition Chain (ARC)
10
Activity Recognition Chain (ARC)
11
Activity Recognition Chain (ARC)
12
Activity Recognition Chain (ARC)
13
[-0
.14
, 3.4
1, 4
,21
, … ,
6.1
1]
[-0
.84
, 3.2
1, 4
.21
, … ,
6.1
1]
[-0
.81
, 5.7
1, 4
.21
, … ,
6.2
2]
[-0
.14
, 3.9
2, 4
.23
, … ,
7.8
2]
S u p s1,s2,…,sk fℝ(s1,s2,…,sk) c
Activity Recognition Chain (ARC)
14
Activity Recognition Chain (ARC)
15
Activity Recognition Chain (ARC)
16
SENSOR FUSION
ARC Fusion: Feature Fusion
17
[-0
.14
, 3.4
1, 4
,21
, … ,
6.1
1]
[-0
.84
, 3.2
1, 4
.21
, … ,
6.1
1]
[-0
.81
, 5.7
1, 4
.21
, … ,
6.2
2]
[-0
.14
, 3.9
2, 4
.23
, … ,
7.8
2]
S1
S2
SM
u1 p1 s11,s12,…,s1k fℝ(s11,s12,…,s1k)
c u2 p2 s21,s22,…,s2k fℝ(s21,s22,…,s2k)
uM pM sM1,sM2,…,sMk fℝ(sM1,sM2,…,sMk)
fℝ(s11,s12,…,s1k,
s21,s22,…,s2k,…,
sM1,sM2,…,sMk)
ARC Fusion: Decision Fusion
18
[-0
.14
, 3.4
1, 4
,21
, … ,
6.1
1]
[-0
.84
, 3.2
1, 4
.21
, … ,
6.1
1]
[-0
.81
, 5.7
1, 4
.21
, … ,
6.2
2]
[-0
.14
, 3.9
2, 4
.23
, … ,
7.8
2]
S1
S2
SM
u1 p1 s11,s12,…,s1k fℝ(s11,s12,…,s1k) c1
c=φ(c1,c2,…,cM)
u2 p2 s21,s22,…,s2k fℝ(s21,s22,…,s2k) c2
uM pM sM1,sM2,…,sMk fℝ(sM1,sM2,…,sMk) cM
Multi-Sensor Hierarchical Classifier
19
SM
S2
S1 α11
∑ C12
C1N
C11
∑
C21
C22
C2N
∑
CM1
CM2
CMN
∑
Decisio
n
Class level Source level Fusion
β11
α12 β12
α1N β1N
α21 β21
α22 β22
α2N β2N
αM1 βM1
αM2 βM2
αMN βMN
γ11,…,1N δ11,…,1N
γ21,…,2N δ21,…,2N
γM1,…,MN δM1,…,MN
[-0
.14
, 3.4
1, 4
,21
, … ,
6.1
1]
[-0
.84
, 3.2
1, 4
.21
, … ,
6.1
1]
[-0
.81
, 5.7
1, 4
.21
, … ,
6.2
2]
[-0
.14
, 3.9
2, 4
.23
, … ,
7.8
2]
S1
S2
SM
u1 p1 s11,s12,…,s1k fℝ(s11,s12,…,s1k)
u2 p2 s21,s22,…,s2k fℝ(s21,s22,…,s2k)
uM pM sM1,sM2,…,sMk fℝ(sM1,sM2,…,sMk)
Multi-Sensor Hierarchical Classifier
20
N activities M sensors & Class level Source level Fusion
Multi-Sensor Hierarchical Classifier
21
N activities M sensors & Class level Source level Fusion
Multi-Sensor Hierarchical Classifier
22
N activities M sensors & Class level Source level Fusion
Multi-Sensor Hierarchical Classifier
23
N activities M sensors & Class level Source level Fusion
Experimental setup: dataset
• Fitness benchmark dataset
• Up to 33 activities
• 9 IMUs (XSENS) ACC, GYR, MAG
• 17 subjects
24 Baños, O., Toth M. A., Damas, M., Pomares, H., Rojas, I., Amft, O.: A benchmark dataset to evaluate sensor displacement in activity recognition. In: 14th International Conference on Ubiquitous Computing (Ubicomp 2012), Pittsburgh, USA, September 5-8, (2012)
Results
• Segmentation: sliding window (6 seconds) • Feature extraction: FS1={mean}, FS2={mean,std}, FS3={mean,std,max,min,cr} • Classification: Decision tree (C4.5) (10-fold cross-validated, 100 repetitions)
25 10 activities 20 activities 33 activities
FS1 FS2 FS3 FS1 FS2 FS3 FS1 FS2 FS360
65
70
75
80
85
90
95
100
Accura
cy (
%)
Feature Fusion Weighted Majority Voting Multi-Sensor Hierarchical Classifier
Experimental Parameters
Conclusions
• We propose a multi-sensor hierarchical classifier that allows data fusion of multiple sensors
– Its assymetric decision weighting (SEinsertions/SPrejections) leverages the potential of the classifiers either for classification/rejection or both
– Specially suited for complex scenarios
• Feature Fusion and MSHC are quite in line in terms of performance however
– Our method outperforms the former when a more informative feature set is used
– Particularly notable for complex recognition scenarios
• Our model is expected to be particularly suited to deal with sensor anomalies (work-in-progress)
26
On-going work…
• Our model is expected to be particularly suited to deal with sensor anomalies (work-in-progress)
27
FEAT-FUSION MSHC0
20
40
60
80
100
Accura
cy (
%)
Ideal Self Induced
Thank you for your attention. Questions?
Oresti Baños Legrán Dep. Computer Architecture & Computer Technology
Faculty of Computer & Electrical Engineering (ETSIIT) University of Granada, Granada (SPAIN)
Email: [email protected] Phone: +34 958 241 516 Fax: +34 958 248 993
Work supported in part by the HPC-Europa2 project funded by the European Commission - DG Research in the Seventh Framework Programme under grant agreement no. 228398, the Spanish CICYT Project SAF2010-20558, Junta de Andalucia Project P09-TIC-175476 and the FPU Spanish grant AP2009-2244.
28