Donnie H. Kim, Nicolai M. Petersen, Mohammad Rahimi, Jeff Burke, Deborh Estrin
Center for Embedded Networked Sensing, UCLALenore Arab
David Geffen School of Medicine, UCLA
Presented by Reg Arvidson
Phones are everywhere! Increasingly carry imaging and
location capabilities Creation of assisted recall systems
• Record aspects of the environment for later playback
Rewind supports data collection on specific behaviors and situations• As opposed to life blogging systems
Scalable system of everyday mobile phones and supporting web services
Explore how client/server-side image processing can…• lower bandwidth needs• streamline user navigation
Original pilot was to assist in recall of dietary intake
Other short/exploratory trials developed
Take advantage of the application’s constraints in…• capturing images• presenting images• processing images• and uploading images
Manages data flow among system components• Interacts using
SSL/TLS encryption and authenticated transmission
Provides a user interface to view captured images• Password protected
web interface
Web Interface – communication over HTTPS through URLs, stored in secure file system and relation database
Image Handling Services – checks if processing is required for each image• Resizing – generates thumbnail for web
interface• Reaping – deletes images marked by user or
filters• Image processing – pushes images to IPS
and retrieves results
Can potentially capture significant personal data• Family members, computer screen contents…• Even occasional image inside a restroom
Privacy addressed at earliest phases of prototyping
Secure HTTP over SSL with a X.509 public key certificate for the web server
Images viewable only by the individual owner, no identifiable information stored
Users given authenticated access to Image Viewer with a User ID/Password
Shown a subset of images (thumbnails) based on time clustering and quality rank
Tasked by DMS to…• Process images• Classify images• Annotate images
Time-consuming and application-dependent image processing tasks separated from core data flow services• Easily scalable through
addition of IPSs
Matlab used as computational engine Extended with an internal TCP/IP
server to provide an interface for external applications
Can handle image processing as a scheduled task or in a FIFO manner
IPS classifies image into four categories…• Clear• Blurred• Exposed• BlurExposed
Class determined from four well-known features• Mean of Intensity• Standard Deviation of Intensity• Number of Edges• Sum of High Frequency Coefs. Of Discreate
Cosine Transform (DCT)
Nokia N80 S60 3rd Edition User Interface Symbian v9.1 Operating System 3 megapixel camera Both 802.11 b/g and GSM connectivity Runs Campaignr
• Acquires data from hardware/software sensors• Immediately stores to internal memory, queues
for upload to DMS• Application-specific XML file specifies which
sensors to collect data from
Clear Clear BlurredMotion of Carrier
BlurredMotion of Subject
ExposedPoor Lighting
BlurExposed
FullDayDietary pilot provided a large and varying data set to work with
Many images were blurred due to motion of individual of subjects of image
Additional images were either over- or under-exposed
Individuals marked the classification of images to produce groundtruth data
83% of images correctly classified93% of clear images correctly classified2% of low quality images were incorrectly classified
Latency dominated by image processing computation
Very small deviation in processing latency per image• Roughly predetermine time to process
images
DietaryRecall• 10 users – 11,090 images uploaded• Device turned on only during meals• Experience kept simple, 35 images max per
episode FullDayDietary
• 14 users – 14,958 images released (6 users)• Ran while outside home, 6 images/minute
PosterSessionCapture• 15 users – visitors to a research conference
poster session• Tested system with many simultaneous users
Pilots showed large number of filtered images
Wireless upload channel became congested with presence of co-located users
Image processing became non-negligible with many users
Added extensions to the system to support local image processing
Early pilots resulted in many low quality images and congestion on the upload link• 33% of DietaryRecall images were marked low qual
Filter out extremely low quality images that would be filtered by back-end server anyways
Also resulted in interesting requirements for prioritized and real-time upload
•Stores image to database upon capture with other sensed data•Images annotation - fetches image, extracts selected feature, stores results•Classification – read computed features, classifies image using decision tree•Clustering – generates cluster ID using the capture time•Upload Ranker – ranks images based on configured upload policy
DCT ignored due to high cost on IPS Edge count not implemented File size used due to a high correlation with
number of edges AND normalized sum of DCT coefs.
DCT on IPS vs size on phone Edge count on IPS vs size on phone
82% of images correctly classified Only 14% of low quality images incorrectly
classified
Phones collecting multiple images per second can easily congest narrow upload channels
Extended periods of disconnect can result in a sizable backlog
Prioritizing upload order can improve usability of the system• Reverse Chronological Order – uploads the most
recently captured image first• Prioritized Upload – ranked by quality and cluster
Not an issue if users do not attempt to view images for a duration of time
In event of rapid review of images, selective uploading can enhance interactivity and responsiveness
Can show best images throughout the day (best of clusters) or reverse chronological (reverse order) as opposed to chronological order
Waiting time before viewing the most recently captured image following different disconnection times.
Completion delay comparison following different disconnection times.
Within a cluster, prioritized method uploads high quality image first
Performance gain insignificant when every image is of low quality
Rewind: Leveraging Everyday Mobile Phones for Targeted Assisted Recall (UCLA Technical Report 2008)
Urban Sensing – CENS/UCLA• http://urban.cens.ucla.edu/projects/
dietsense/
Top Related