Kalman Filters and Adaptive Windows for Learning in Data Streams
Adaptive Cleaning for RFID Data Streams. RFID: Radio Frequency IDentification.
-
Upload
arron-dickerson -
Category
Documents
-
view
229 -
download
0
description
Transcript of Adaptive Cleaning for RFID Data Streams. RFID: Radio Frequency IDentification.
Adaptive Cleaning for Adaptive Cleaning for RFID Data StreamsRFID Data Streams
RFID: Radio Frequency RFID: Radio Frequency IDentificationIDentification
RFID data is dirtyRFID data is dirtyShelf 0 Shelf 1
RFIDReaders
StaticTags
Mobile Tags
15ft1.5ft
3ft9ft
3ft
3ft
3ft
A simple experiment:•2 RFID-enabled shelves•10 static tags•5 mobile tags
RFID Data CleaningRFID Data Cleaning
Time
Raw readings
Smoothed output
• RFID data has many dropped readings• Typically, use a smoothing filter to
interpolateSELECT distinct tag_idFROM RFID_stream [RANGE ‘5 sec’]GROUP BY tag_id
Smoothing Filter
Smoothing filter Smoothing filter
Middleware
Clean RFID
Completeness
Tag dynamics
Read all tags in range
RFID Data CleaningRFID Data Cleaning
Time
Raw readings
Smoothed output
• RFID data has many dropped readings• Typically, use a smoothing filter to
interpolateSELECT distinct tag_idFROM RFID_stream [RANGE ‘5 sec’]GROUP BY tag_idBut, how to set the size
of the window?
Smoothing Filter
Window Size for RFID Window Size for RFID SmoothingSmoothing
Fido moving Fido resting
Small windowRealityRaw readings
Large window
Need to balance completeness vs. capturing tag movement
Truly Declarative Truly Declarative SmoothingSmoothing
• Problem: window size non-declarative Application wants a clean stream
of data Window size is how to get it
• Solution: adapt the window size in response to data
RFIDRFID
Epoch TagID ReadRate0 1 .90 2 .60 3 .3
Tag 1
Tag 2
Tag 3
Tag 4
Antenna & readerTags
E1 E2 E3 E4 E5 E6 E7 E8 E9E0
Read Cycle (Epoch)
(For Alien readers)
Tag List
1. Interrogation cycle2. Epoch
Controlled condition real condition
SMURFSMURF• Statistical Smoothing for Unreliable RFID
Data• Adapts window based on statistical
properties• Mechanisms for:
• Per-tag and multi-tag cleaning
Multi-tagCleaning
SMURFPer-tag
Cleaning
raw RFID streams
cleanedcount readings
cleanedper-tag readings
Application(s) Application(s)
Per-Tag Smoothing: Model and Per-Tag Smoothing: Model and BackgroundBackground
• Epoch t, Tag population Nt
• pi,t: Per epoch sampling prob.Response count of tag i per epoch
(total interrogation cycle)
Epoch TagID ReadRate0 1 .90 2 .60 3 .3
• Smoothing window size wi epoch • Per epoch sampling prob: pi
• Number of successful observations of tag i Binominal distribution B(wi,pi)
Per-Tag Smoothing: Model and Per-Tag Smoothing: Model and BackgroundBackground
Per-Tag Smoothing: Model and Per-Tag Smoothing: Model and BackgroundBackground
• Use a binomial sampling model
Time (epochs)
pi
1
0
Smoothing Window
wi Bernoulli trials
piavg
Si
(Read rate of tag i)
E1 E2 E3 E4 E5 E6 E7 E8 E9E0
Set of epochs where tag i can be seen
• We want to ensure that there are enough epochs in Wi such that tag i is observed (if it exists within the reader’s range) Completeness
Per-Tag Smoothing: CompletenessPer-Tag Smoothing: Completeness
Per-Tag Smoothing: Per-Tag Smoothing: CompletenessCompleteness
• If the tag is there, read it with high probability
Want a large window
pi
1
0
Reading with a low pi
Expand the window
Time (epochs)E1 E2 E3 E4 E5 E6 E7 E8 E9E0
Per-Tag Smoothing: CompletenessPer-Tag Smoothing: Completeness
Per-Tag Smoothing: Per-Tag Smoothing: CompletenessCompleteness
Expected epochs needed to read
With probability 1-
Desired window size for tag i
1ln*1
avgi
ip
w
Per-Tag Smoothing: Per-Tag Smoothing: TransitionsTransitions• Detect transitions as statistically
significant changes in the data
pi
1
0
Statistically significant difference Flag a transition and
shrink the window
The tag has likely left by this point
Time (epochs)E1 E2 E3 E4 E5 E6 E7 E8 E9E0
• Significant difference between mean observed sample size Si and expected size
• Find outlier (2)
Number of successful epochs in a window
Si
Mean
Per-Tag Smoothing: Per-Tag Smoothing: TransitionsTransitions
# expected readings Is the difference
“statistically significant”?# observed
readings
)1(**2|*||| avgi
avgii
avgiii ppwpwS
•Statistically significantStatistically significant
Algorithm Algorithm
SMURF in ActionSMURF in ActionFido moving Fido resting
SMURF
Experiments with real and simulated data show similar results
Normal sliding window Completeness Transition