Context-Awareness based on Lifelog
description
Transcript of Context-Awareness based on Lifelog
Context-Awareness based on Lifelog
Sangkeun Lee
Intelligent Database Systems Lab.
Seoul National University
Copyright 2009 by CEBT
Introduction
The whole story begins with my survey ‘A Survey of Context-Aware Systems’
Context & Context-awareness
General Process in Context-Aware Systems
History of Context-aware Systems
– Generations of Context-aware Systems
– Trend Analysis
Let’s briefly revisit the survey first
2
Copyright 2009 by CEBT
Context & Context-aware Systems
Context
Any information that can be used to characterize the situation of entities relevant to the interaction between a user and an application (Dey and Abowd ,2001)
Context-aware Systems
The systems that use context to provide relevant information and/or services to the user, where relevancy depends on the user’s task (Dey and Abowd ,2001)
3
Copyright 2009 by CEBT
General Process in Context-Aware Systems
Generally, processing context-awareness can be summarized into following four steps
Sense
– Getting values from context sources (e.g. sensors)
– Context-sources doesn’t have to be hardware devices
– (e.g. virtual sensors, other applications)
Abstraction
– Context Aggregation Temperatures of previous 10 minutes -> Very Cold
– Context Interpretation GPS signal -> City name
(Recognize)
– Understanding current context & trigger the most suitable action
Action
– Better user’s experience
4
Copyright 2009 by CEBT
Trend Analysis
5
Copyright 2009 by CEBT
Insights from the Survey
What makes context-Awareness so difficult?
Realizing useful Application/Domain Specific context-aware systems is not a big problem
– Using simple, non-flexible context models
– In a restricted & known scenarios (e.g. museum, …)
– Not a ‘true’ context-awareness for a ubiquitous environment
However, realizing a flexible and general context-aware system is a big problem
– If you use ontological context model,
then you get flexibility and expressiveness of representing context information,
but recognizing (understanding) the represented context data causes computational costs, less scalability – not practical!
6
Copyright 2009 by CEBT
Approach to achieve feasibility and flexibility at the same time?
As I mentioned earlier, context-aware systems are
The systems that use context to provide relevant information and/or services to the user, where relevancy depends on the user’s task (Dey and Abowd ,2001)
Recommender system
– Recommender systems or recommendation engines form or work from a specific type of information filtering system technique that attempts to present information items that are likely to be of interest to the user. (Wikipedia)
– Similar ! but there are many working systems They use user’s ratings, logs, previous histories, ….
By getting hints from recommender systems that are practically used in many domains, we present a practical way of processing context-awareness by separating ‘Recognize’ step into two steps ‘Collect’ & ‘Match’
Existing approach : Sense - Abstraction – Recognize – Action (Analytic)
Practical Approach : Collect - Sense – Abstraction - Match – Action (Empiric)
7
Copyright 2009 by CEBT
Profile
Query
Context
t1
t2
t3
t4
t5
t2
t4
t6
Informa-tion
t7
t8
t9
Service
t10
t11
t12
Sense
Practical Context-awareness –an Empiric Approach
Match
Collect
Copyright 2009 by CEBT
Practical Context-awareness –an Empiric Approach
Is it reasonable to do this? Yes!
– Context history can be used to establish trends and predict future context values. [Anind K. Dey and Gregory D. Abowd, 2001]
– “Context history is generally believed to be useful, but it is rarely used” [Guanling Chen and David Kotz, 2001]
– “Context histories may be used to establish trends and predict future context values.” [Matthias Baldauf, 2007]
Many systems keeps recording context history (although they do not utilize them)
– Context Toolkit, CoBrA, CASS, SOCAM and CORTEX save sensed context data persistently in a database.
9
Copyright 2009 by CEBT
Collect & Match
My research consists of two parts
Collect
– Sense and store context histories
– LifeLogOn: Log on to Your Lifelog Ontology! (ISWC ‘09)
– Entity-Event Lifelog Ontology Model (EELOM)for Lifelog Ontology Schema Definition (APWEB ‘10)
– LifeLogOn: A Practical Lifelog System for Building and Exploiting Lifelog Ontology (UMC ‘10)
Match
– Find the most suitable information/services based on current context & context histories (logs) using available techniques such as Collaborative filtering, Context-aware collaborative filtering, contents based matching,
Simple If-then rules, many other heuristics
– A flexible framework for context-aware matching is needed
10
Copyright 2009 by CEBT
Collect – LifeLogOn: A Lifelog System
Integrate currently available logs from different devices and create semantic relationships among logs
Transforms relational log data and metadata into instance-level Ontology and stores in knowledge base
Music Listening LogsMovie Watching Logs
GPS LogsPhone Call LogsE-mail History
Schedules
Music Listening Logs
Movie Watching Logs
GPS LogsPhone Call Logs
E-mail History Schedules
* Sematnic Relationships
LifeLogOn
Copyright 2009 by CEBT
Collect – ‘Entity-Event Lifelog Ontol-ogy Model’
EELOM consists of domain, entity and events A domain is a group of entities and events, and it has a domain name for representing it.
An entity is a thing which has a separate existence and can be uniquely identified (e.g. a person, a location, a device, a timestamp) An entity is represented as a set of attributes. Entities can be shared among domains.
An event is an activity or interaction that can be uniquely identified. It is represented as a set of en-tities, and each entity has its role for the event.
Why not RDF? OWL?
- It does not allow complex relationships (e.g. hierarchy of entities),- but it is still flexible and expressive
enough to cover logs of many domains
- Easy to construct index structure by considering everything as entities & events
Copyright 2009 by CEBT
Collect – LifeLogOn Overview
Log ID E-mail Time Date Event
453 2412 1:11PM 09/03/21 Played
454 2151 12:06AM 09/04/22 Played
455 2151 12:10AM 09/04/22 Stopped
Title Genre Artist BPM Tags
453 2412 1:11PM 09/03/21 Played
454 2151 12:06AM 09/04/22 Played
455 2151 12:10AM 09/04/22 Stopped
Music Domain
Music
Atmosphere
Album Genre
Artist
BPMTitle
Play
User
Time DateOther
Context
MusicLocation
Stop
User
Time DateOther
Context
MusicLocation
Title Genre Artist BPM Tags
453 2412 1:11PM 09/03/21 Played
454 2151 12:06AM 09/04/22 Played
455 2151 12:10AM 09/04/22 StoppedTitle Genre Artist BPM Tags
453 2412 1:11PM 09/03/21 Played
454 2151 12:06AM 09/04/22 Played
455 2151 12:10AM 09/04/22 Stopped
Title Genre Artist BPM Tags
453 2412 1:11PM 09/03/21 Played
454 2151 12:06AM 09/04/22 Played
455 2151 12:10AM 09/04/22 Stopped
Last.fm Music Log
iTunes Music Log
AMG Metadata
CDDB Metadata
GPS Log from Cell phone
Temperature Log from Forecast.com
Music@1Groove
Hyori’s back KpopHyori
9010 minute
Play@1
Location@1Korea
Kangwon
12:31:42Chunchon
User@1
Male
Hiphop Seoul,KoreaSinging
27Matt
Time@130
1202
Play@2Time@212
901
Location@2Korea
Seoul
55:24:66Seoul
Music@2Bright
Yellow Submarine OldpopBeatles
110 Yesterday
take@1Photo@1SonyCD2
640KB
Home.jpg640x480
LifeLogOn Instance-Level Ontology GeneratorLifeLogOn Instance-Level Ontology Generator
Log–Ontology Schema Mapping
OntologySchema
OntologyGenerator
OntologyInstances
RelationalLogData
Ontology Schema Definition Tool
Ontology Schema Definition Tool
Visualization & Browsing ToolVisualization & Browsing Tool
Log to Ontology Mapper
Log to Ontology Mapper
Copyright 2009 by CEBT
What can we use LifeLogOn for?
You can create your own lifelog ontology without understanding any ontology languages
You can search something not sure about Find a song you listened at your birthday
party and but only know the filename of the photos at the party
Find photos that you took when you are talking with your friend on the phone, saying "It's so beautiful here!"
Find any events and entities those you know their context information such as time, location,...
... even more
14
Copyright 2009 by CEBT
We have the context history collec-tion…
15
Many issues remain- Building a lifelog ontology using real data set- Implementing Loggers (maybe on iPhone, …)- Log Ontology Abstraction/Summarization- Better Visualization- Entity Matching, …
Copyright 2009 by CEBT
Profile
Query
Context
t1
t2
t3
t4
t5
t2
t4
t6
Informa-tion
t7
t8
t9
Service
t10
t11
t12
Sense
Let’s Get Back to Context-awareness
Match
Collect
Copyright 2009 by CEBT
Paper Reading
Context-Aware Collaborative Filtering System: Predicting the User’s Preference in the Ubiquitous Computing Environment - Annie Chen, 2005
predicts a user’s preference in different context situations based on past experiences
시간 상황 정보를 고려한 협업 필터링을 이용한 음악 추천 – 2009, 이동주
Context-Aware Recommendation by Aggregating User Context – 2009, 신동민
Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions – 2005, Gediminas Adomavicius
And so on, …
17
Copyright 2009 by CEBT
Problem Definition
Recommender System
Estimating ratings for the items that have not been seen by a user
User (explicitly/implicitly - logs) rates Item
Input: User, Ratings Output: Item
Context-aware Recommender System (similar but different!)
Predicting a user’s preference in different context situations based on past experiences
User (explicitly/implicitly) rates Item + each rating has Context Info
Input: User, Current Context Output: Item
18
Copyright 2009 by CEBT
Problem Definition
Context-awareness based on Lifelog Ontology
In our Lifelog Ontology, there are no classification of users, items, contexts…
All the things in the ontology are Events & Entities
19
Copyright 2009 by CEBT
Problem Definition
Assume that the system knows the current context (situation)
Context-aware applications are such as Find out the most suitable songs to the current user
Find out the most suitable person to be invited to the current location
Find out the most similar songs to the song that currently user’s listening to
Find out the most related anything to the current situation
…
Relatedness can be defined in various ways
We can conclude that a context-aware system based on lifelogs is a system that allows dynamically & flexibly composing recommendation based on lifelogs
20
Copyright 2009 by CEBT
Context-Awareness Based on Lifelogs
21
Our Lifelog model
Lifelog Ontology LO is a set of events EV
A context history is represented by a event EV that are composed of a set of entities EN
– e.g. {Sangkeun Lee, Yesterday, 2009/12/24, Summer, Seoul, SNU}
A current context C is represented by a set of entities EN
– e.g. {2009/12/25, Hyo-ri Lee, Winter, Hungry, Happy, Busan, …}
Copyright 2009 by CEBT
Simple Flexible Match Algorithm (on-going work)
Collaborative Filtering (e.g. Music recommendation)
22
User
Similar User
Based on logs
SongsUser’s rat-
ing
Rating prediction based on similar user’s rait-ings
Copyright 2009 by CEBT
Simple Flexible Match Algorithm (on-going work)
Generalized Model
Raiting is one possible relatedness between users & songs
User similarity is one possible relatedness between users
Entity – Entity Relatedness Can be calculated by similarity of events related to each entity
Is it only suitable for same type of entities? Not sure.
23
Entity
Event
Event
Entity
……
User
Song
Song
User
… …
Song
Location
Time
…
Copyright 2009 by CEBT
Simple Flexible Match Algorithm 1 (on-going work)
Collaborative Context-aware Filtering
24
Entity
Similar Entity
EventsEntity relatedness in current context
Predict relatedness based on similar entity’s related-ness
C
E1
E2
E3E4
E5
E6
C
E1
E2
E3E4
E5
E6
C
E1
E2
E3E4
E5
E6
Entity relatedness in current context
Copyright 2009 by CEBT
Simple Flexible Match Algorithm 2 (on-going work)
Context-Aware Filtering
Find out the most related entities to current context
Context Similarity Can be defined as an aggregation of entity relatedness
Entity Rank The most related entity to current context
25
Current Context can be represented in a set of entities
(Assume that all the entities are the categorical values)
C
E1
E2
E3E4
E5
E6
CH1
E1
E2
E3E4
E5
E6
CH1
E1
E2
E3E4
E5
E6
CH1
E1
E2
E3E4
E5
E6
CH1
E1
E2
E3E4
E5
E6
CH1
E1
E2
E3E4
E5
E6
CH1
E1
E2
E3E4
E5
E6
Event Rank
User Time Location Tempera-ture
…
1 Matt Summer Chunchon Hot …
2 Kang Summer Chunchon Hot …
3 Matt Winter Chunchon Hot …
4 Steve Fall Seoul Hot …
5 … … … … …
Event ranking based on similarity to current context
Entity Rank can be calculated by weighted sum
Copyright 2009 by CEBT
Simple Flexible Match Algorithm (on-going work)
Context-Aware Filtering
Flexible?– Context Similarity, Entity Relatedness can be defined in various
ways by controlling Weight parameters of context similarity, weight parameters of entity relatedness
– You can decide what to recommend (Any entity can be recommended)
26
Copyright 2009 by CEBT
Simple Match is Not the Only Way – Graph Tra-verse
Assume that Current Context is
{Matt Lee, DSC_2128, 2009/07/23, 018-2144-8842}
Result Entity: matt183/018-2144-8842/
Entity: SNU. Main Gate/37:27:14.60N:126:57:12.33E/37:27:14.60N:126:57:12.33E/
Entity: GS24-TB/
Entity: Green House/37:27:27.29N:126:57:62.52E/37:27:27.29N:126:57:62.52E/
Entity: Jay Yeon/Female/[email protected]/101-1106 Suwon apt. Suwon Si, Republic Of Korea/Deejay09/010-4255-5541/
Entity: 2009/01/04/
Entity: Lazy Rhapsody/The Duke: Creole Rhapsody (1931-1932) (disc 1)/Duke Ellington & His Orchestra/196440/
Entity: SNU. Rear Gate/37:27:13.79N:126:57:18.23E/37:27:13.79N:126:57:18.23E/
Entity: 12:35:45/
Entity: Warren Gong/[email protected]/138-3 Naksung apt. BongChun dong, Kwanak-gu, Seoul Si, Republic Of Korea/gongfit/019-258-8163/
Entity: SNU Library/37:27:72.11N:126:57:11.32E/37:27:72.11N:126:57:11.32E/
27
Copyright 2009 by CEBT
Simple Match is Not the Only Way – Simple If-then Rules
Simple if then rules
sometimes much powerful and useful then other recommendation algorithm
Many context-aware actions can be defined as simple if-then rules
– Alert me when ‘Sangkeun Lee’ is in ‘IDS Lab’
– Turn on light when ‘Sangkeun Lee’ arrives at home
– ….
CAUTION : should not require complex reasoning or inference
28
Copyright 2009 by CEBT
Conclusion & Discussion
The Ideal Context-awareness
Find out the most related anything to the current situation
Considering everything useful
Ignoring information noise
…
29