Download - Searching Twitter: Separating the Tweet from the Chaff

Transcript
Page 1: Searching Twitter: Separating the Tweet from the Chaff

Searching Twitter: Separating the Tweet from the ChaffJonathan Hurlock & Max L. Wilson

Page 2: Searching Twitter: Separating the Tweet from the Chaff

You sure can!

Page 3: Searching Twitter: Separating the Tweet from the Chaff
Page 4: Searching Twitter: Separating the Tweet from the Chaff
Page 5: Searching Twitter: Separating the Tweet from the Chaff
Page 6: Searching Twitter: Separating the Tweet from the Chaff
Page 7: Searching Twitter: Separating the Tweet from the Chaff

How do I follow

my Interests?

http://www.flickr.com/photos/stevegarfield/5397972626/

Page 9: Searching Twitter: Separating the Tweet from the Chaff
Page 10: Searching Twitter: Separating the Tweet from the Chaff

Yet more DataMeta Data, Profile Data, Linked Data

Page 11: Searching Twitter: Separating the Tweet from the Chaff

Any of it Useful?Who cares how much data there is!

“I think the challenge not only for twitter, but for the technology industry at large. Is building more relevant filters, in real time. Like being able to surface valuable information immediately. No matter who it is, whoʼs listening or whoʼs broadcasting, is a really really hard problem, and it makes twitter alot more meaningful[... ]Weʼve gotten really really good at being able to put content in, into media [...] getting it out in a relevant, valueable way, in real time is still very difficult.” - Jack Dorsey (Creator of Twitter)

Page 12: Searching Twitter: Separating the Tweet from the Chaff

Why Twitter?Where is the value?

ƒ

!₱₧£

₤₣

!

₠₥

¢

$

₡ƒ! ₱

£

₣ !₠₥

¢

$

!

Page 13: Searching Twitter: Separating the Tweet from the Chaff

Lets go back...

http://www.flickr.com/photos/milesdeelite/5309712846/

Page 14: Searching Twitter: Separating the Tweet from the Chaff

Lets go back...Great Scott!

http://www.flickr.com/photos/milesdeelite/5309712846/

Page 15: Searching Twitter: Separating the Tweet from the Chaff

Asking FriendsHey, what are you doing?

you me&

Page 16: Searching Twitter: Separating the Tweet from the Chaff

Social SearchWhat is everyone else doing?

you me&

Page 17: Searching Twitter: Separating the Tweet from the Chaff

Social SearchWhat is everyone else doing?

you me&

friendfriendfriend

friend

Page 18: Searching Twitter: Separating the Tweet from the Chaff

Existing KnowledgeNo need to reinvent the wheel

you me&

bob lisa&

Meredith Ringel Morris, Jaime Teevan, and Katrina Panovich. 2010. What do people ask their social networks, and why?: a survey study of status message & behavior. In Proceedings of the 28th international conference on Human factors in computing systems (CHI '10). ACM, New York, NY, USA, 1739-1748.

Page 19: Searching Twitter: Separating the Tweet from the Chaff

Existing KnowledgeNo need to reinvent the wheel

you

mebob

lisa

&

Meredith Ringel Morris, Jaime Teevan, and Katrina Panovich. 2010. What do people ask their social networks, and why?: a survey study of status message & behavior. In Proceedings of the 28th international conference on Human factors in computing systems (CHI '10). ACM, New York, NY, USA, 1739-1748.

Page 20: Searching Twitter: Separating the Tweet from the Chaff

Lets go back to the networkRemember...

you me&

Page 21: Searching Twitter: Separating the Tweet from the Chaff

and if we take a step back...Please mind the gap

friendfriendfriend

friend you me

Page 22: Searching Twitter: Separating the Tweet from the Chaff

We start to see interesting things...

Page 23: Searching Twitter: Separating the Tweet from the Chaff

Which have value!

Page 24: Searching Twitter: Separating the Tweet from the Chaff

Location, experiences, temporal data

http://en.wikipedia.org/wiki/File:Plane_crash_into_Hudson_River_(crop).jpg

http://www.flickr.com/photos/gcaw/5445225362/

http://www.flickr.com/photos/seanhobson/3256437306/

http://www.flickr.com/photos/mdid/4560003881/http://www.flickr.com/photos/24423474@N08/4999891492/ Yardi, Sarita and Boyd, Danah. ICWSM 2010.

Tweeting from the Town Square: Measuring Geographic Local Networks

Page 25: Searching Twitter: Separating the Tweet from the Chaff

Location, experiences, temporal data Political upheaval,

http://en.wikipedia.org/wiki/File:Plane_crash_into_Hudson_River_(crop).jpg

http://www.flickr.com/photos/gcaw/5445225362/

http://www.flickr.com/photos/seanhobson/3256437306/

http://www.flickr.com/photos/mdid/4560003881/http://www.flickr.com/photos/24423474@N08/4999891492/

emergency events .. so what are you tweeting now?

Yardi, Sarita and Boyd, Danah. ICWSM 2010. Tweeting from the Town Square: Measuring Geographic Local Networks

Page 26: Searching Twitter: Separating the Tweet from the Chaff

Twitter SearchHow do you find useful information?

Page 27: Searching Twitter: Separating the Tweet from the Chaff

Displaying ResultsRealtime

Page 28: Searching Twitter: Separating the Tweet from the Chaff

Time, ReTweets, Location, Popularity? RTDisplaying Results

http://www.flickr.com/photos/publicenergy/394124407/

Page 29: Searching Twitter: Separating the Tweet from the Chaff

Time, ReTweets, Location, Popularity? RTDisplaying Results

http://www.flickr.com/photos/publicenergy/394124407/

Page 30: Searching Twitter: Separating the Tweet from the Chaff

Displaying ResultsMaking sense of the data.

Page 31: Searching Twitter: Separating the Tweet from the Chaff

Displaying ResultsMaking sense of the data.

Michael S. Bernstein, Bongwon Suh, Lichan Hong, Jilin Chen, Sanjay Kairam, Ed H. Chi. Eddi: Interactive Topic-based Browsing of Social Status Streams. In Proc. of ACM User Interface Software and Technology (UIST) conference, Oct. 2010. New York, NY.

Page 32: Searching Twitter: Separating the Tweet from the Chaff

Displaying ResultsMaking sense of the data.

Diakopoulos, N.; Naaman, M.; Kivran-Swaine, F.; , "Diamonds in the rough: Social media visual analytics for journalistic inquiry," Visual Analytics Science and Technology (VAST), 2010 IEEE Symposium on , vol., no., pp.115-122, 25-26 Oct. 2010

Page 33: Searching Twitter: Separating the Tweet from the Chaff

Not necessarily useful!Interestingness

http://www.flickr.com/photos/wwarby/2460655511/

Naveed, Nasir and Gottron, Thomas and Kunegis, Jérôme and Alhadi, Arifah Che (2011) Bad News Travel Fast: A Content-based Analysis of Interestingness on Twitter. pp. 1-7. In: Proceedings of the ACM WebSci'11, June 14-17 2011, Koblenz, Germany.

Page 34: Searching Twitter: Separating the Tweet from the Chaff

What makes us unique?How we are different?

Page 35: Searching Twitter: Separating the Tweet from the Chaff

usefulness

What constitutes a useful Tweet?Finding Usefulness!

http://www.flickr.com/photos/edduddiee/4346349664/

Page 36: Searching Twitter: Separating the Tweet from the Chaff

How did we go about this?The Method

Page 37: Searching Twitter: Separating the Tweet from the Chaff

3 Information Seeking TasksInformation Seeking

http://www.flickr.com/photos/ivyfield/4731067396/

http://www.flickr.com/photos/anniemole/241655156/

http://www.bbc.co.uk/proms/2010/share/badgewidget.shtml

Teevan, J., Ramage, D., & Morris, M. R. (2011). #TwitterSearch: a comparison of microblog search and web search. WSDM '11: Proceedings of the fourth ACM international conference on Web search and data mining (pp. 35-44). New York, NY, USA: ACM.

Page 38: Searching Twitter: Separating the Tweet from the Chaff

They were really nice people!20 Participants

Page 39: Searching Twitter: Separating the Tweet from the Chaff

A simple, easy to understand interfaceSearch Interface

Page 40: Searching Twitter: Separating the Tweet from the Chaff
Page 41: Searching Twitter: Separating the Tweet from the Chaff

To help us provide more insightThink aloud + Interviews

It’s useful because...

I didn’t because...

Page 42: Searching Twitter: Separating the Tweet from the Chaff

Lots and lots of it!Analysis K∑

Page 43: Searching Twitter: Separating the Tweet from the Chaff

Inductive Coding = Lots of Post-its!Grounded Theory

Glaser, B. G., & Strauss, A. L. (2009).The Discovery of Grounded Theory: strategies for qualitative research.Piscataway, New Jersey, USA: Transaction Publishers.

Page 44: Searching Twitter: Separating the Tweet from the Chaff

Cohen... Fleiss....Kappa Analysis

Landis, R. J., & Koch, G. G. (1977). The Measurement of Observer Agreement for Categorical Data. Biometrics , 33 (1), 159-174.

Page 45: Searching Twitter: Separating the Tweet from the Chaff

Multi Coded KappaExtended Kappa Analysis

Harris, J. K., & Burke, R. C. (2005). Do you see what I see? An application of inter-coder reliability in qualitative analysis. American Public Health Association 133rd Annual Meeting & Exposition. Washington, DC, USA: American Public Health Association.

0.73 (Substantial Agreement) Between Evaluators

&0.62 (Substantial Agreement)

with Independent Untrained Coder

Page 46: Searching Twitter: Separating the Tweet from the Chaff

Useful & Not-UsefulWhat did we find?

Page 47: Searching Twitter: Separating the Tweet from the Chaff

In Tweet ContentIn Tweet ContentExperience Someone reporting a personal experience, but not necessarily suggestion / direction.

Direct Recommendation

Someone making a direct recommendation, but not necessarily relaying a personal experience.

Social Knowledge Containing information that is spreading socially, or becoming general knowledge.

Specific Information Where facts are listed directly in tweets e.g. prices, times etc.

Reflection on TweetReflection on TweetEntertaining The reader finds them amusing.

Shared Sentiment The reader agrees with the author of the tweet.

RelevantRelevantTime The time is current

Location The location is relevant to the query.

Useful

Page 48: Searching Twitter: Separating the Tweet from the Chaff

TrustTrustTrusted Author The twitter account has a reputation / following

Trusted Avatar The visual appearance cultivates trust.

Trusted Link A link to a trustworthy recognisable domain.

LinksLinksActionable Link The user can perform a transaction by using the link (heavily dependent on trust)

Media Link The link is to rich multimedia content.

Useful Link The link provides valuable information content, e.g. authoritative information, educated reviews

Meta TweetMeta TweetReTweeted Lots Its information that others have passed on lots

Conversation Its part of a series of tweets, and they all need to be useful

Useful (cont.)

Page 49: Searching Twitter: Separating the Tweet from the Chaff

Tweet ContentTweet ContentNo Information Absence of anything, event, factual points

Introspective Personal content and personal thoughts for no social benefit

Off Topic Result not related to the query give / TF-IDF irrelevant

Too Technical The content requires specific domain knowledge the resader doesn’t possess

Poorly Constructed Tweets that may have grammatical / spelling errors, or malformed URLs.

Bad TweetsBad TweetsSPAM Irrelevant or inappropriate messages

Wrong Language Messages sent in a foreign language of that to the reader

Dead Link A URL which does not work i.e. a 404

Not RelevantNot RelevantTime Out of date content

Location Wrong geographic location

Not Useful

Page 50: Searching Twitter: Separating the Tweet from the Chaff

TrustTrustUn-truested Author An author the reader feels at un-eased by or suspicious of.

Un-trusted Link A link the reader feels is suspicious

SubjectiveSubjective

Perspective Oriented A tweet that is perspective centric, meaning the author is providing their view or projecting an attitude on a subject matter or to a subject / reader.

Disagree with Tweet A conflict of aggreement between the reader and the author

Not Funny A tweet that is aimed to be humorous, which the reader does not feel is humorous.

Meta TweetMeta TweetQnA Part of a conversation, reader desires the whole convo. not just the question or the answer.

Repeated Content the reader has seen before.

Not Useful (cont.)

Page 51: Searching Twitter: Separating the Tweet from the Chaff

Interesting findsInsights

http://www.flickr.com/photos/foxmulderven/3063598624

Page 52: Searching Twitter: Separating the Tweet from the Chaff

Where could we see the impact of this work?The Possible Impact

Page 53: Searching Twitter: Separating the Tweet from the Chaff

A work in progressSearch System

Page 54: Searching Twitter: Separating the Tweet from the Chaff

So just remember.Conclusions

Page 55: Searching Twitter: Separating the Tweet from the Chaff

Thank you for ListeningJonathan Hurlock

Max L. Wilson

@jonhurlock

@gingdottwit

http://moourl.com/LikedTheTalk

Like the talk? Then please tweet it, by quickly visiting: