Understanding and Modeling of WiFi Signal Based Human ... · Understanding and Modeling of WiFi...

Understanding and Modeling of WiFi SignalBased Human Activity Recognition

Wei Wang†, Alex X. Liu†‡, Muhammad Shahzad‡,Kang Ling†, Sanglu Lu†

†Nanjing University, ‡Michigan State University

September 8, 2015

1/24

Motivation Modeling Design Experiments Conclusions

Motivation

• WiFi signals are available almost everywhere and they areable to monitor surrounding activities.

2/24


Problem Statment

WiFi based Activity Recognition

• Using commercial WiFi devices to recognize human activities.

Wireless router

Laptop

Wireless signal reflection

AdvantagesX Work in darkX Better coverageX Less intrusive to user privacyX No need to wear sensors

3/24


Limitations of Prior Arts

Limitations of Prior Arts: no model (signal, activity)• So, have to rely on statistical characteristics of WiFi signals• So, sensitive to environmental changes signals

Our Approach: model(signal, speed)+model(signal, activity)• Robust to environmental changes• High recognition accuracy

11 11.5 12 12.560

65

70

75

CS

I

Time (seconds)

4/24


Understanding Multipath

Key observations

• Multipaths contain both staticcomponent and dynamic com-ponent

• Each path has different phase• Phases determine the ampli-

tude of the combined signal

Sender

ReceiverWall

Reflected by

body

Reflected by

wall

LoS path

dk(0)

dk(0)

5/24



Sender

ReceiverWall

Reflected by

body

Reflected by

wall

LoS path

dk(0)

dk(0)

I

Q

Combined

Static

component

Dynamic

Component

6/24



Sender

Receiver

dk(t)

Wall

Reflected by

body

Reflected by

wall

LoS path

LoS path

I

Q

Combined

Static

component

Dynamic

Component

6/24



Sender

Receiver

dk(t)

Wall

Reflected by

body

Reflected by

wall

LoS path

I

QCombined

Static

component

Dynamic

Component

6/24



Interpreting CSI amplitude• Phases of paths are deter-

mined by path length• Path length change of one

wavelength gives phasechange of 2π

• Frequency of amplitudechange can be converted tomovement speed I

Q

Combined

Static

component

Dynamic

Component

7/24


CSI-Speed Model

How accurate is it?

• Wave length→ 5 ∼ 6cm in 5 GHz band

2.5 3 3.5 4 4.5 5 5.5 6 6.5 750

100

150

200

Time (seconds)

CS

I p

ow

er

Waveform with regular moving speed

CSI amplitude changes areclose to sinusoids

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.080

0.2

0.4

0.6

0.8

1

Measurement error (meters)

CD

F

Moving distance measurement error

Average distance measurementerror of 2.86 cm

8/24


CSI-Speed Model

How robust is it?

• Robust over different multipath conditions and movementdirections

• Linear combination of multipath do not change frequency

0.2 0.4 0.6 0.8 1 1.2 1.40

0.05

0.1

0.15

0.2

0.25

Estimated speed (m/s)

Pro

bab

ilit

y

running

walking

sitting down

Speed distribution of different activities in different environments

9/24


CSI-Activity Model

Activities are characterized by

• Movement speeds• Change in movement speeds• Speeds of different body components

2 2.5 3 3.5 4−15

−10

−5

0

5

10

15

Time (seconds)

CS

I

Walking

0.5 1 1.5 2−40

−20

0

20

40

Time (seconds)

CS

I

Falling

0 0.5 1 1.5 2−40

−20

0

20

Time (seconds)

CS

I

Sitting down

10/24


CSI-Activity Model

• Use time-frequency analysis to extract features• Use HMM to characterize the state transitions of movements

Walking Falling

Sitting down

11/24


CSI-Activity Model

• Build one HMM model for each activity• Determine states based on observations in waveform pat-

terns• State durations and relationships are captured by transition

probabilities

State 3State 2State 1 State 4

12/24


System Architecture

CSI measurement

collection

Noise reduction

HMM training

HMM

Model

Feature extraction

HMM based activity

recognition

Activity

data

collection

Model

generationActivity detection and

segmenting

Online

monitoring

Monitoring records

13/24


Data Collection

N

M

30 subcarriersN ×M × 30 CSI streams

11 11.5 12 12.560

65

70

75

CS

I

Time (seconds)11 11.5 12 12.5

60

65

70

75

CS

I

Time (seconds)

11 11.5 12 12.560

65

70

75

CS

I

Time (seconds)11 11.5 12 12.5

60

65

70

75

CS

I

Time (seconds)

11 11.5 12 12.560

65

70

75

CS

I

Time (seconds)11 11.5 12 12.5

60

65

70

75

CS

I

Time (seconds)

11 11.5 12 12.560

65

70

75

CS

I

Time (seconds)11 11.5 12 12.5

60

65

70

75

CS

I

Time (seconds)

11 11.5 12 12.560

65

70

75

CS

I

Time (seconds)11 11.5 12 12.5

60

65

70

75

CS

I

Time (seconds)

14/24


Noise Reduction

Correlation of CSI on different subcarriers• Subcarriers only differ slightly in wavelength• Subcarriers have the same set of paths, with different phases

Frequency

312.5kHz

Wave length= 5.150214 cm

Wave length= 5.149662 cm

15/24


Correlation in CSI Streams

16/24


Noise Reduction

Combines N × M × 30 subcarriers using PCA to detect time-varying correlations in signal

11 11.5 12 12.560

65

70

75

CS

I

Time (seconds)

Original

11 11.5 12 12.5

65

70

75

CS

I

Time (seconds)

Low-pass filter

11 11.5 12 12.5−10

−5

0

5

10

Time (seconds)

CSI

PCA

17/24


Real-time Recognition

• Activity detection- Use both the signal variance and correlation to detect pres-

ence of activities

• Feature extraction- Time-frequency analysis (DWT)

• HMM model building- Eight activities

Walking, running, falling, brushing teeth, sitting down, opening refrigerator,pushing, boxing

- More than 1,400 samples from 25 persons as the trainingset

18/24


Evaluation Setup

• Commercial hardware with no modification- Transmitter: NetGEAR JR6100 Wireless Router- Receiver: Thinkpad X200 with Intel 5300 NIC

• A single communicating pair is enough to monitor 450 m2

open area

• Measurement on UDP packets sent between the pair• Sampling rate 2,500 samples per second

19/24


Evaluation Results

Activity recognized

True

activ

ity

R W S O F B P T ERunning 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000Walking 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000Sitting 0.000 0.000 0.947 0.030 0.011 0.000 0.012 0.000 0.000

Opening 0.000 0.005 0.150 0.803 0.042 0.000 0.000 0.000 0.000Falling 0.000 0.010 0.041 0.010 0.939 0.000 0.000 0.000 0.000Boxing 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000 0.000

Pushing 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000Brushing 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000

Empty 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000

• Ten-fold validation accuracy: 96.5%• Detects human movements at 14 meters• Real-time recognition on laptops• Packet sending rate can be as low as 800 frames per sec-

ond

20/24


Evaluation on Robustness

• Models are robust to environmentchanges

• Train once, apply to different sce-narios

• Training use database collected inlab with different users

• Test in with users not in the train-ing set

• Open lobby• Apartment (NLOS)• Small office

Table

TableWalking/running route

7.7 m

6.5

m

1.6

m

Fridge

Training location

Tx

Rx

Lab

Table

Fridge

Kitchen

Tx

Bath

room

Rx

Testing location

Appartment

21/24


Evaluation on Robustness

• Consistent performance in unknown environments, with morethan 80% average accuracy

lab lobby apartment office0

0.5

1

Environments

Accu

racy

R W S O F B P T

22/24


Conclusions

• CSI measurements contains fine-grained movement infor-mations

• CSI-Speed modelquantifies the correlation between CSI value dynamics and human movement

speeds

• CSI-Activity modelquantifies the correlation between the movement speeds of different human body

parts and a specific human activity

• Our models are robust to environment changes

23/24


Q & A

Thank you!

Questions?

24/24

Understanding and Modeling of WiFi Signal Based Human ... · Understanding and Modeling of WiFi...

Documents

Transcript of Understanding and Modeling of WiFi Signal Based Human ... · Understanding and Modeling of WiFi...