Life <--> LoggingLife <--> Logging

Melekam TsegayeMelekam TsegayeShaun BangayShaun BangayAlfredo TerzoliAlfredo Terzoli

Research area: Wearable, Pervasive and Ubiquitous ComputingResearch area: Wearable, Pervasive and Ubiquitous Computinghttp://www.cs.ru.ac.za/research/students/g98t4414/researchhttp://www.cs.ru.ac.za/research/students/g98t4414/research

25/11/200425/11/2004

Life loggingLife logging

Recording/capturing everything you see Recording/capturing everything you see and do. and do.

Past, present and future eventsPast, present and future events Why ? Why ?

for the sake of improving your memoryfor the sake of improving your memory Your memory defines who you are!Your memory defines who you are!

Amnesia?Amnesia?

Forms of life logsForms of life logs

Manual paper diariesManual paper diaries Electronic diaries on PDAsElectronic diaries on PDAs Audio/Video recordingsAudio/Video recordings Emergence of the Weblog - blogEmergence of the Weblog - blog

Current TrendsCurrent Trends Weblogs are mostly text and image based Weblogs are mostly text and image based

e.g. Blogger.com (blog via email)e.g. Blogger.com (blog via email) Next natural transition is audio bloggingNext natural transition is audio blogging

Individuals make an audio recording, publish Individuals make an audio recording, publish to a web server e.g. audioblog.comto a web server e.g. audioblog.com

Hardware needed to start life loggingHardware needed to start life logging Camera, microphone, local storage Camera, microphone, local storage

(Digicam/Cell phone)(Digicam/Cell phone) Web serverWeb server

audioblog.comaudioblog.com

Effortless audio loggingEffortless audio logging recording audio to a remote server from recording audio to a remote server from

your web browseryour web browser cell phonecell phone emailemail

audioblog.com recently introduced a video audioblog.com recently introduced a video blogging extensionblogging extension

Life-log applicationsLife-log applications

Nokia lifeblogNokia lifeblog Software for organizing video clips recorded Software for organizing video clips recorded

using a cell phone camerausing a cell phone camera

Life-log applications(2)Life-log applications(2) Microsoft sense camMicrosoft sense cam

Passive Capture, improved participation in eventsPassive Capture, improved participation in events• ACM workshop on Continuous archival and retrieval of personal experiences  New ACM workshop on Continuous archival and retrieval of personal experiences  New

York, USA October  2004 York, USA October  2004 Correlations and relationships from sensors used to manage captured imagesCorrelations and relationships from sensors used to manage captured images

• light intensity sensorlight intensity sensor• x/y tilt (tilt sensor )x/y tilt (tilt sensor )• motion sensor to avoid taking blurry pictures (accelerometer)motion sensor to avoid taking blurry pictures (accelerometer)• temperaturetemperature• infra red sensorinfra red sensor• clockclock• locationlocation• MMC card storage MMC card storage • 2xAA cell batteries, 12 hours2xAA cell batteries, 12 hours• Size of a pagerSize of a pager

Useful: quick replay of a day’s eventsUseful: quick replay of a day’s events

Our workOur work

wPim - Wearable audio enhanced wPim - Wearable audio enhanced personal information management systempersonal information management system

Record audio wherever you areRecord audio wherever you are Organize people and events (past, future and Organize people and events (past, future and

present)present) Retrieve using time of capture and textual Retrieve using time of capture and textual

descriptorsdescriptors

Current WorkCurrent Work Once you capture your memories how do you retrieve Once you capture your memories how do you retrieve

segments of interest for later use ? segments of interest for later use ? using time, location ?using time, location ?

We want to analyze data produced by multiple sensors We want to analyze data produced by multiple sensors at time of av capture and extract useful information at time of av capture and extract useful information to be used to auto tag complex streams such as video and audio to be used to auto tag complex streams such as video and audio

Assembling a large number of sensors and conducting Assembling a large number of sensors and conducting experiments can be difficultexperiments can be difficult

Some sensors are not available in a convenient formSome sensors are not available in a convenient form We waste too much time dealing with hardware issues and We waste too much time dealing with hardware issues and

produce poor software solutionsproduce poor software solutions

Current Work(2)Current Work(2) The approach we take in our attempts to solve the problems The approach we take in our attempts to solve the problems

mentioned earlier ismentioned earlier is To define multiple virtual sensors that sense environmental and user To define multiple virtual sensors that sense environmental and user

statesstates make the virtual sensors generate data (essentially, simulate the make the virtual sensors generate data (essentially, simulate the

process sensing data) process sensing data) extract useful contextual information from the generated dataextract useful contextual information from the generated data make use of the extracted information to build a contextual tag databasemake use of the extracted information to build a contextual tag database use the information in the tag database to automatically tag complex use the information in the tag database to automatically tag complex

data streams such as video and audio streamsdata streams such as video and audio streams Not simulation only, we want to be able use a combination of virtual Not simulation only, we want to be able use a combination of virtual

and real sensors to run our experimentsand real sensors to run our experiments E.g. A wearable life logging facility that uses a combination of virtual and E.g. A wearable life logging facility that uses a combination of virtual and

real sensorsreal sensors

PrototypePrototype

sensors for: Video, Audio, Locationother sensors and I/O devices through a multi-channel (a2d) data capture device.

ProblemsProblems Social issuesSocial issues

PrivacyPrivacy LawsLaws Hostility and fear from societyHostility and fear from society First person experience capture vs big brother First person experience capture vs big brother

scenarioscenario Hardware limitationsHardware limitations

Wireless network connectivity is not yet ubiquitous, Wireless network connectivity is not yet ubiquitous, bandwidth limitationsbandwidth limitations

long lasting power sourceslong lasting power sources comfortable wearable hardwarecomfortable wearable hardware

Questions Questions ??