by Muthukumar V. Ramaswamy An Applied Project Presented in ...
DARWIN PHONES: THE EVOLUTION OF SENSING AND INFERENCE ON MOBILE PHONES PRESENTED BY: BRANDON OCHS...
-
Upload
dustin-greene -
Category
Documents
-
view
216 -
download
2
Transcript of DARWIN PHONES: THE EVOLUTION OF SENSING AND INFERENCE ON MOBILE PHONES PRESENTED BY: BRANDON OCHS...
DARWIN PHONES: THE EVOLUTION OF SENSING AND INFERENCE ON MOBILE PHONES
PRESENTED BY: BRANDON OCHS
Emiliano Miluzzo, Cory T. Cornelius, Ashwin Ramaswamy, Tanzeem Choudhury, Zhigang Liu, Andrew T. Campbell, "Darwin phones: the evolution of sensing and inference on mobile phones," In Proc. of 8th ACM Conference on Mobile Systems, Applications, and Services (MobiSys), 2010, pp. 5-20.
What does Darwin do?
A Smartphone platform for urban sensing
Proof of concept model uses microphone Communicates with other local devices
to improve inference accuracy (collaborative inference)
Framework can be expanded to gatherinformation using a range of sensor data
What about battery life?
Communicates with backend server to do the CPU-intensive machine learning algorithms
Local devices share models rather than re-computing them
Sensing is enabled/disabled as the system sees fit
Common Urban Sensing Challenges Human burden of training classifiers Ability to perform reliably in different
environments (indoor vs outdoor) The ability to scale to a large number of
phones without hurting usability and battery life.
Darwin overcomes all of these through classifier/model evolution, model pooling, and collaborative inference
Types of Learning
Supervised: Given a fully-labeled training set
Semi-Supervised: Given a small training set that is evolved
Unsupervised: No training set is given
Darwin Steps
Evolution, Pooling, and Collaborative Inference
These represent Darwin’s novel evolve-pool-collaborate model implemented on mobile phones
Classifier Evolution
Automated approach to updating models over time
Needs to account for variability in sensing conditions and settings
Variability in background noise and phone location require separate models
Model Pooling
Reuses models that have already been built and evolved on other phones
Exchange classification models whenever the model is available from another phone
Classifiers do not need to be retrained, which increases scalability
Can pool models from backend servers
Collaborative Inference
Combines results from multiple phones
Run inference algorithms in parallel on the same classifiers
System is more robust to degradation in sensing quality
Increases accuracy
Darwin Design: Computation Reduces the on-the-phone computation
by offloading some of the work to backend servers
Backend server uses a machine learning algorithm to compute a Gaussian Mixture Model (2 hours for 15 seconds of audio)
Feature vectors are computedlocally
Darwin Design: Context
Context (in/out of pocket, in/out of bag) will impact the sensing and inference capability
Classifier evolution makes sure the classifier of an event is robust across different environments
Darwin Design: Co-location
Accounts for a group of co-located phones running the same classification algorithm and sensing the same event but computing different inference results
Phones pool classification models when collocated or from backend servers
Compares against its own model and the co-located model
Drastically reduces classification latency Exploits diversity of different phone
sensing context viewpoints
Speaker Recognition
Attempts to identify a speaker by analyzing the microphone’s audio stream
Suppresses silence, low amplitude audio, and chunks that do not contain human voice
Reduce false positives by pre-processing in 32ms blocks
Speaker Modeling
Feature vector consisting ofMel Frequency Cepstral Coefficients
Each speaker is modeled with 20 Gaussians
An initial speaker model is built by collecting a short training sample
Classifier Evolution: Training Step Short training phase (30 seconds) used
to build a model which is later evolved First 15 seconds used as the training set Last 15 seconds used as baseline for
evolution
Classifier Evolution: Evolution Step Semi-supervised learning strategy If the likelihood of the incoming audio
stream is much lower than any of the baselines then a new model is evolved
Collaborative Inference
Local inference phase can be broken into three steps: Local inference operated by each individual
phone Propagation of the result of the local
inference to the neighboring phones Final inference based on the neighboring
mobile phones local inference results Each node individually operates
inference on the sensed event Results and confidence broadcasted
Privacy and Trust
Raw sensor data is not stored on or leaves the mobile phone
The content of a conversation or raw audio data is never disclosed
Users can choose to opt out of Darwin
Experimental Results
Tested using a mixture of five N97 and iPhones used by eight people over a period of two weeks
Audio recorded in different locations
Classifier trained indoors
Experiment 1 Parameters
Three people walk along a sidewalk of a busy road and engage in conversation
The speaker recognition application without the Darwin components runs on each of the phones carried by the people
Experiment 2 Parameters
Meeting setting in an office environment where 8 people are involved in conversation
The phones are located at different distances from people in the meeting, some on the table and some in people’s pockets
Experiment 3 Parameters
Five phones in a noisy restaurant Three of the five people are engaged in
conversation Two of the five phones are placed on the
table Phone 4 Is the closest phone to speaker
4 and also the closest phone to another group of people having a loud conversation
Experiment 4 Parameters
Five people walk along a sidewalk and three of them are talking
The greatest improvement is observed by speaker 1, whose phone is clipped to their belt
Time and Energy Measurements Baselines for power use determined Measurements performed using the
Nokia Energy Profiler tool No data gathered for the iPhone Smart duty cycling required later to save
battery life
Possible Applications
Virtual square application Social application for a group of friends
Place discovery application Use collaborative inference to determine
location Friend Tagging application
Exploit face recognition to tag friends on pictures
Improvements On The Paper Studies don’t show conclusive evidence;
there should be separate control models for each of the scenarios
Conclusion
The Darwin system combines classifier evolution, model pooling, and collaborative inference
Results indicate that the performance boost offered by Darwin off sets problems with sensing context
The Darwin system provides a scalable framework that can be used for other urban sensing applications
References
[1] Emiliano Miluzzo, Cory T. Cornelius, Ashwin Ramaswamy, Tanzeem Choudhury, Zhigang Liu, Andrew T. Campbell, "Darwin phones: the evolution of sensing and inference on mobile phones," In Proc. of 8th ACM Conference on Mobile Systems, Applications, and Services (MobiSys), 2010, pp. 5-20.
[2] H. Ezzaidi and J. Rouat. Pitch and MFCC Dependent GMM Models for Speaker Identification systems. In Electrical and Computer Engineering, 2004. Canadian Conference on, volume 1, 2004
[3] H. Ezzaidi and J. Rouat. Pitch and MFCC Dependent GMM Models for Speaker Identification systems. In Electrical and Computer Engineering, 2004. Canadian Conference on, volume 1, 2004.