Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information...
Transcript of Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information...
![Page 1: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/1.jpg)
Lecture 4: People-centric Sensing Cristian Borcea
Department of Computer Science
NJIT
![Page 2: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/2.jpg)
2
Enable accurate real-time monitoring of the physical world
Consist of homogeneous, static, tiny sensors
Sink Internet, Satellite
Task Manager
End user
Sensor node
Water pollution in the river Fire in the forest Structural integrity of the bridge
How can we benefit from sensing in our
daily routine?
![Page 3: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/3.jpg)
Leverage existing “infrastructure”: smart phones and vehicular systems No deployment cost
Excellent coverage: billions of devices distributed everywhere
3
Tornado
approaching!
Highway
Parking Area
Shopping
mall
Traffic jam!
Local fog
patches!
Free parking spot!
50%
discount!
People-centric: people are both consumers and providers of sensed data
![Page 4: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/4.jpg)
On phones: camera, microphone, GPS, accelerometer, digital compass, light sensor, Bluetooth, [future: pollution sensors, medical sensors]
On cars: all car sensors can be made available to the vehicular system
Owners can provide information as well (i.e., human sensors)
4
![Page 5: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/5.jpg)
Monitoring pollution in urban areas
Beneficiary: environment protection agencies
Monitoring vehicular traffic to detect/prevent traffic jams
Beneficiary: departments of transportation for planning, all of us for
improved traffic conditions
Finding the locations of potholes on the roads
Beneficiary: municipalities or departments of transportation
Determining the effect of lifestyle (meeting people, level
of exercise) on people’s wellbeing
Beneficiary: medical/social researchers, all of us
Documenting public events in real-time
Beneficiary: news outlets, all of us 5
![Page 6: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/6.jpg)
On phones: free SDKs for development & app stores for deployment Could easily deploy sensing applications at scale
On cars: no standard vehicular system yet Care must be taken to ensure apps do not interfere with
car safety 6
![Page 7: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/7.jpg)
Introduction
Smart phone sensing
Micro-Blog @ Duke
CenceMe @ Dartmouth
Activity recognition on smart phones
Vehicle sensing
Future challenges
7
![Page 8: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/8.jpg)
Internet
Micro-Blog vision: virtual information telescope [Ref. 4]
8
![Page 9: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/9.jpg)
Users encouraged to blog on mobile phones Video, audio, pictures, text, etc.
Micro-Blog phone client geo-tags blog
Uploads to server over WiFi/GPRS
Micro-Blog server positions blog on Google Maps
Internet users zoom into maps
Witness blog streams across the world
9
![Page 10: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/10.jpg)
Virtual Telescope
Cellular, WiFi Visualization Service
Web Service
Phones People
Physical Space
Some queries participatory Is beach parking available?
Others are not Is there WiFi at the beach café?
Phones reply to query & reply posted on Google Map as new micro-blog
10
![Page 11: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/11.jpg)
Content generated at a certain region is automatically
downloaded on user smart phone when user enters region
Example
User X creates micro-blog about restaurant food
“Floats” micro-blog at the restaurant
User Y arrives at restaurant
X’s micro-blog downloaded onto Y’s phone
Y can modify content and “re-float”
11
![Page 12: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/12.jpg)
Continuous GPS (8 hours battery life) WiFi, GSM localization improves energy (16, 40 hours)
Degrades localization accuracy (40, 500m)
Solution: switch between them as function of app needs
WiFi
GSM GPS
Time (in minutes) 12
![Page 13: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/13.jpg)
Phones need to continuously update their location Poses privacy risks
Pseudonyms insufficient
Proposed 3 blogging modes
Public, Social, Private
Users set privacy policy
In social mode, only those in social network view blogs
For querying
Privacy feasible through K-anonymity based solutions
13
![Page 14: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/14.jpg)
14
Leverage sensors on the phones and social networks to provide context/activity inferences
![Page 15: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/15.jpg)
15
![Page 16: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/16.jpg)
16
![Page 17: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/17.jpg)
Introduction
Smart phone sensing
Activity recognition on smart phones
Darwin phones @ Dartmouth
Vehicle sensing
Future challenges
17
![Page 18: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/18.jpg)
Machine learning techniques applied on phone sensor data to
determine user activity
E.g., sitting, walking, running, in a car, etc.
Done on phone or server (on-line or off-line) function of amount of
resources required
Data exchange between nearby phones could enable group activity recognition
Examples of applications:
Is the user exercising enough? (sensors: GPS & accelerometer)
Has the user a good social life? (sensors: Bluetooth and microphone)
Deriving higher-level context information
▪ Is the user in an important meeting? (sensors: Bluetooth,
localization + social network) 18
![Page 19: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/19.jpg)
19
![Page 20: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/20.jpg)
Calibration, uncontrolled phone context, computational efficiency, battery power
Example of uncontrolled phone context: one activity, two different patterns
Cycling while phone in pants pocket (A) and backpack (B)
20
![Page 21: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/21.jpg)
microphone
camera
GPS/WiFi/
cellular
air quality
pollution
social context
audio / pollution / RF
fingerprinting
image / video
manipulation
classification model evolution
classification model pooling
collaborative inference
Darwin
Applies distributed computing and collaborative inference to activity recognition
Why Darwin? Same classifier model for all users doesn’t work; even same classifier
for same user doesn’t work when changing context
Creating one classifier model per user doesn’t scale
21
![Page 22: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/22.jpg)
4 phases
initial training (derive model seed)
classification model evolution
classification model pooling
collaborative inference
supervised
unsupervised
Darwin was tested on voice recognition 22
![Page 23: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/23.jpg)
23
phone: feature extraction (low computation)
backend
backend: model training (high computation)
![Page 24: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/24.jpg)
24
phone determines when to evolve
match?
NO
evolve (train new model using
backend as before)
training sampled
![Page 25: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/25.jpg)
Speaker A’s model
Phone A Phone B
Phone C
Speaker B’s model
Speaker C’s model
Speaker C’s model
Speaker A’s model
Speaker B’s model
Speaker B’s model
Speaker A’s model
Speaker C’s model
Re-use already available classifiers instead of training a new model for each speaker Saves battery power and reduces delay
25
![Page 26: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/26.jpg)
Phone A Phone B
Phone C
User A speaking!!!
A’s LI results:
Prob(A speaking) = 0.65
Prob(B speaking) = 0.25
Prob(C speaking) = 0.10
C’s LI results:
Prob(A speaking) = 0.30
Prob(B speaking) = 0.67
Prob(C speaking) = 0.03
B’s LI results:
Prob(A speaking) = 0.79
Prob(B speaking) = 0.11
Prob(C speaking) = 0.10
Local inference can be misleading
Use collaborative inference to reach common decision
26
![Page 27: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/27.jpg)
Collaborative inference & classification model evolution boost performance The more phones the better 27
• Indoor, quiet scenario • 8 people around a table
![Page 28: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/28.jpg)
Introduction
Smart phone sensing
Activity recognition on smart phones
Vehicle sensing
Pothole Patrol @ MIT
ParkNet @ Rutgers
Future challenges
28
![Page 29: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/29.jpg)
Determine which roads need to be fixed Challenges
Differentiate potholes from other road anomalies (railroad crossings, expansion joints)
Cope with variations in detecting same pothole (speed, sensor orientation)
Vehicles have GPS and 3-axis accelerometer <time,location,speed,heading,3-axis acceleration>
Detection based on accelerometer data
Embedded Linux system + WiFi/Cellular installed in 7 taxis
29
![Page 30: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/30.jpg)
Pothole Record
Clustering Algorithm
Cab 1
GPS
3 Axis Accelerometer
Location Interpolator
Pothole Detector
Cab 2
GPS
3 Axis Accelerometer
Location Interpolator
Pothole Detector
Central Server
30
![Page 31: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/31.jpg)
Use classifier Hand-labeled data for training
Extend training set with loosely-labeled training data (type & frequencies, but not exact locations)
Clustering (at least K events must happen at same location with small error)
31
![Page 32: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/32.jpg)
On labeled data:
32
On unlabeled data (with spot verification):
![Page 33: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/33.jpg)
Provide a high-level view of parking availability to drivers on the road Single sensor vehicle senses multiple parking spots
33
![Page 34: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/34.jpg)
34
Sensor vehicle instrumented with Computer & WiFi/cellular communication
Ultrasonic sensor: 20 readings per sec; range: 6.5m
GPS: 5 readings per sec
Webcam: capturing ground truth for evaluation
![Page 35: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/35.jpg)
Obstacles: ‘dips’ in ultrasonic sensor readings Use classifier to differentiate parked cars from
other objects
35
![Page 36: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/36.jpg)
36
Slotted (marked) spots
Unslotted spots Difficult to estimate if a car fits in the sensed spot
![Page 37: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/37.jpg)
Introduction
Smart phone sensing
Activity recognition on smart phones
Vehicle sensing
Future challenges
37
![Page 38: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/38.jpg)
Dependability How can “clients” trust data received from mobile sensors?
▪ How to validate these data? Should use reputation mechanism?
▪ How to determine ground truth?
Sensor calibration?
Privacy How to provide anonymity?
▪ How to balance the need for anonymity with the need for reputation?
Incentives Users motivated by self-interest for some apps (e.g., traffic jams)
Why should they provide data if there is no clear self-interest?
▪ Get paid? How to get paid while maintaining anonymity?
▪ Need for fair exchange protocol 38
![Page 39: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/39.jpg)
Programmability Current systems are application-specific; How to quickly develop and
deploy new applications?
How to decompose global distributed sensing tasks into individual tasks?
▪ E.g., ensure spatial and temporal distribution of sensed data?
How can device owners control where, when, and how much resources are consumed for sensing?
Scheduling and real-time constraints What parameters should be considered in task scheduling?
▪ Payment, real-time constraints, spatio-temporal distribution of data providers?
▪ Should a provider be paid if the answer comes late? 39
![Page 40: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/40.jpg)
1. Urban sensing @ UCLA: http://urban.cens.ucla.edu/
2. MetroSense project @ Dartmouth:
http://metrosense.cs.dartmouth.edu/
3. Urbanets: http://cs.njit.edu/~borcea/papers/ieee-pervasive07.pdf
4. Micro-Blog: http://synrg.ee.duke.edu/microblog.html
5. CenceMe:
http://www.cs.dartmouth.edu/~sensorlab/pubs/cenceme_sensys08.pdf
6. Darwin phones: http://www.ists.dartmouth.edu/library/478.pdf
7. Pothole patrol: http://db.csail.mit.edu/pubs/mobisys08.pdf
8. ParkNet:
http://www.winlab.rutgers.edu/~suhas/SuhasMathur_Mobisys2010.pdf
40
![Page 41: Lecture 4: People-centric Sensing · 2017-04-26 · vehicular system Owners can provide information as well (i.e., human sensors) 4 Monitoring pollution in urban areas Beneficiary:](https://reader034.fdocuments.net/reader034/viewer/2022042908/5f38765a80a7f111b931fa3d/html5/thumbnails/41.jpg)
1. Two decades of mobile computing
2. Infrastructure support for mobility
3. Mobile social computing
4. People-centric sensing
5. Programming mobile ad hoc networks Migratory Services: context-aware client-service programming
Spatial Programming: location-aware imperative programming
Contory: SQL-like declarative programming
6. Vehicular computing and networking
7. Privacy and security in mobile computing
41