Leveraging the Semantics of Tweets for Adaptive Faceted...

Delft University of Technology

Leveraging the Semantics of Tweets for Adaptive Faceted Search on Twitter

ISWC, Bonn, Germany, Oct 27th 2011

Fabian Abel1, Ilknur Celik1, Geert-Jan Houben, Patrick Siehndel2

1Web Information Systems, TU Delft, the Netherlands 2L3S Research Center, Hannover, Germany

2 Adaptive Faceted Search on Twitter

Personalized Recommendations

Personalized Search

Adaptive Systems

What we do: Science and Engineering for the Personal Web

Social Web

Analysis and User Modeling

user/usage data

Semantic Enrichment, Linkage and Alignment

domains: news social media cultural heritage public data e-learning


200,000,000 number of tweets published per day


1 number of tweets that are interesting for me now


Searching on Twitter


Issues with Multiple Keywords Search


Let’s try to search with One Keyword


Page 1


Page 2


Page 3


Page 60!!

tweet I was looking for


Page 60!!

tweet I was looking for

Next Saturday @thatsimpsonguy aka Guilty Simpson will be performing at Area51 in my hometwon Eindhoven. #realliveshit #iwillspinrecords about 9 hours ago via Blackberry

Music Artist

Locations


Is there an easier way?

Locations more...

Events more...

Music Artists: + Guilty Simpson + Bryan Adams + Elton John + Golden Earring + Rihanna + The eagles + 3 Doors Down more...

Current Query:

Results: 1. Yskiddd: Next saturday

@thatsimpsonguy aka Guilty Simpson will be performing at Area51 in my homeytown Eindhoven. #realliveshit #iwillspinrecords2

2. Usee123: Cool #EV3door7980 !!! http://bit.ly/igyyRhL

3. sanmiquelmusic: This Saturday I'm joining @KrusadersMusic to Intents

Eindhoven Music

Expand Query:

Faceted Search can help (hypothesis)


Challenges


Facets of a Tweet

@bob: Julian Assange got arrested

Creator @bob Location Delft, the Netherlands Creation time Nov 29th 2011

Facet type Facet Value


Facets of a Tweet


Creator @bob Location Delft, the Netherlands Creation time Nov 29th 2011

Facet type Facet Value

Challenge 1: How to infer facets that describe the content of a tweet?


Faceted Search: selecting facet-value pairs

Locations + Aachen + Aalborg + Aalesund + Aarhus + Aasiaat + Abaiang + Abakan more...

Events more...

Music Artists more…

Current Query:





Music

Expand Query:




Events more...


Current Query:





Music

Expand Query:

Number of selectable facet values may be

very high!




Events more...


Current Query:





Music

Expand Query:

Number of selectable facet values may be

very high!

Challenge 2: How to adapt the faceted search interface to the current demands of a user?


Adaptive Faceted Search Framework


Adaptive Faceted Search Framework

Adaptive Faceted Search

Twitter posts

Semantic Enrichment

User and Context Modeling

user

How to adapt the facet-value pair ranking to the

current demands of the user?

How to represent the content of a

tweet? facet extraction


Facet Extraction and Semantic Enrichment


Julian Assange

Julian Assange Tweet-based enrichment

powered by




Julian Assange


Julian Assange arrested Julian Assange, the founder of WikiLeaks, is under arrest in London…

Link-based enrichment

powered by




Julian Assange


Julian Assange arrested Julian Assange, the founder of WikiLeaks, is under arrest in London…

Link-based enrichment

powered by

Julian Assange

London

WikiLeaks

Julian Assange

Julian Assange

London WikiLeaks


Impact of Link-based enrichment


Impact of Link-based enrichment

Representation of tweets:

significantly more facets per tweet with link-based

enrichment


Faceted Search Strategies

• Challenge: most-relevant facet-value pair should appear at the top of the ranking

• Baseline: hashtag-based keyword search

Locations 1. Aachen 2. Aalborg 3. Aalesund 4. Aarhus … 2145. Eindhoven

Locations 1. Eindhoven 2. Delft 3. Amsterdam 4. Rotterdam 5. London …





• Faceted Search Strategies: 1. Occurrence frequency: count occurrence frequencies of FVP (baseline)



facet-value pair

current hit list of matching tweets

number of tweets that contain the FVP






2. Personalization: adapt ranking to user profile ( different user

modeling strategies possible; here: entire tweeting history of the user)



facet-value pair



Personalized FVP ranking stratey

facet-value pair



time June 27 July 4 user

User Profile FVP weight

6

4

3

(location, Delft)

(event, JazzBaltica)

(person, ChetBaker)

weight in user profile =

rank of the FVP








3. Diversification: increase variety among the top-ranked FVPs



facet-value pair




facet-value pair





6

4

3

(location, Delft)


(person, ChetBaker)


rank of the FVP


Genre + Blues + Jazz + JazzMusic + Rock more...

Genre + Blues + Jazz + Rock + Classic more...

minimize overlaps









4. Time-sensitivity: adapt FVP ranking to temporal context



facet-value pair




facet-value pair





6

4

3

(location, Delft)


(person, ChetBaker)


rank of the FVP


Genre + Blues + Jazz + JazzMusic + Rock more...

Genre + Blues + Jazz + Rock + Classic more...

minimize overlaps




time June 27 July 4 June 20

occu

rre

nce

fr

eq

ue

ncy

of

FV

P


(event, FrenchOpen)

Event + JazzBaltica + FrenchOpen more...

search









4. Time-sensitivity: adapt FVP ranking to temporal context

• Semantic enrichment: (i) tweet-based and (ii) link-based enrichment




Research Questions

1. How well does faceted search that is supported by the semantic enrichment perform in comparison to keyword search?

2. What strategy performs best in ranking facet-value pairs that allow users to find relevant tweets on Twitter?

3. How do the different building blocks of the faceted search framework influence the performance?


Dataset

time Nov 15 Dec 15 Jan 15 Feb 15

20,000 Twitter users

30,000,000 tweets

4 months

more than:

Egyptian revolution

Jan 25


Evaluation Framework

• User Simulation Model [cf. Koren et al., WWW’08]: • Input: search settings = { (user who searches, relevant target tweet) }

• Drill down search result list until no more FVPs can be applied or less than 10 tweets match the query

• Simulating click behavior: first-matching FVP is selected ( user knows

target resource)

• Ground truth relevant target tweet = tweet that has been re-tweeted by the user

• Metrics: • Succes@k: probability that relevant FVP appears in the top k (the higher

the Succes@k, the faster the search and fewer the user effort)

• MRR: mean reciprocal rank of the target tweet when the user selected it


Faceted-search vs. hashtag-based (keyword) search


Faceted-search vs. hashtag-based (keyword) search

Faceted search based on semantic enrichment of

tweets outperforms hashtgag-based search

significantly.


Results: Overview


Results: Overview Personalized strategy achieves ~12% better

performance than other semantic strategies (and 2 x

better than hashtag-based)


Impact of link-based enrichment


Impact of link-based enrichment

Personalized strategy outperforms baseline

significantly

Link-based enrichment improves quality for both

strategies


Impact of time-sensitivity


Impact of time-sensitivity

Time-sensitivity based ranking improves quality for both frequency and

diversification strategies


Application of the Faceted Search Framework


Twitcident.com Twitter-based crisis management system

1.

2.

3. 4.

Semantic enrichment allows for: 1. Grouping tweets

into incidents 2. Faceted search 3. Thematic Views 4. Analysis


Conclusions

What we did:

• Adaptive Faceted Search on Twitter + Evaluation Framework • Analysis and Evaluation (+ Application in Twitcident)

Findings:

1. Semantic Enrichment allows for structured representation of the content of tweets basis for faceted search

2. Faceted search performs significantly better than hashtag-based keyword search

3. Different building blocks for making faceted search on Twitter adaptive improve the search quality:

a) Link-based enrichment: more discoverable tweets, better search performance

b) Personalization leads to significant improvements

c) Time-sensitivity improves performance as well


Thank you!

Twitter: @fabianabel

http://wis.ewi.tudelft.nl/iswc2011/

Leveraging the Semantics of Tweets for Adaptive Faceted...

Documents

Transcript of Leveraging the Semantics of Tweets for Adaptive Faceted...