Social curation as corpora for large- scale …...Statistics Social curation as corpora for...

Since 2013-02-20 / Copyright 2013 NTT Communication Science Laboratories. All Rights Reserved.

Social curation as corpora for large-

scale multimedia content analysis

Akisato Kimura

NTT Communication Science Laboratories

[email protected] @_akisato

Acknowledgment

Social curation as corpora for large-scale multimdia content analysis 2

• I really appreciate the following collaborators

for their great commitments for this work.

Prof. Kevin Duh

@ NAIST

Dr. Katsuhiko

Ishiguro

（石黒勝彦）

Mr. Koh

Takeuchi

（竹内孝）

Ms. Kaori

Kataoka

（片岡香織）

Summary


• Social curation has a great potentialfor analyzing large-scale multimedia contents.

1. Curated contents share a specific context, unlike common social media contents.

2. Quite a large amount of collection is

freely available.

Agenda


1. What is “Social Curation”?

2. Why is “Social Curation” emerging?

3. How does “Social Curation” work?

4. What is the next?


What is “Social Curation”?

What is “curation”?


(From Wikipedia)

• Traditionally, a curator of a cultural heritage

institution (e.g., gallery, museum, library or

archive) is a content specialist responsible for an

institution's collections and involved with the

interpretation of heritage material.

Standard flow of social media


Creators generate and

upload contents to social

services

Consumers enjoy

created contents

Process of social curation


Creators generate and

upload contents to social

services

Curator picks up

contents and summarizes

in a list

Consumers enjoy

summarized lists


Curators : the significance


Curator picks up

contents and summarizes

in a list


Curator makes significant differences

from existing social media contents.

Statistics


“Pew Internet : Social Networking (full detail)“

– 46% of adult internet users post original photos or videos

online that they themselves have created. We call them creators.

– 41% of adult internet users take photos or videos that they

have found online and repost them on sites designed for sharing images

with many people. We call them curators.

– Overall, 56% of internet users do at least one of the creating or curating

activities we studied and 32% of internet users do both

creating and curating activities.

(Cf. http://pewinternet.org/Commentary/2012/March/Pew-Internet-Social-Networking-full-detail.aspx)

Example 1 (in Japan)


(cf. http://matome.naver.jp/odai/2134491562701539701)

• Specialized to Twitter messages

• One of the most popular

curation services in Japan


Example 2 (in Japan)


(cf. http://matome.naver.jp/odai/2134491562701539701)

• One of the most popular curation services in Japan

• Can deal with various multimedia contents

(microblogs, news, images, music, videos…)

• Provide incentive compensation plans for curators


Example 3


(cf. http://storify.com/weatherchannel/meteor-slams-earth-injuring-hundreds)

• Quite similar to Naver Matome,

but more sophisticated

• Can deal with

various multimedia contents

• One of the most popular

curation

services in the world


Example 4


• Specialized to images

• One of the most emerging curation

services in the world


(cf. http://pinterest.com/akisatoo/)

Example 5


(cf. http://www.pearltrees.com/#/N-u=1_334641&N-p=23176992&N-s=1_3109322&N-fa=3109322&N-f=1_3109322)


Lots of curation services available


(Cf. http://www.youbrandinc.com/ultimate-lists/ultimate-list-content-curation-tools-platform/)


Social Curation Summit



(Cf. http://www.mediabistro.com/socialcurationsummit/)


Why is “Social Curation” emerging?

Win-win-win relationships


[Curators]

• Convenient as a scrap board

• Useful for self-branding and

(stealth) marketing



[Consumers]

• Avoid being drown in a sea of

social media

• Enjoy more focused stories



[Service providers]

• Probe individual preferences

& “hidden voices” of users

Service providers

Social curation as a second media


• Social curation

= social media as sensors + focused storytelling

Social mediaSocial curation

Mass media

Automatic social curation


(Cf. http://paper.li/g9ine/1307853228)

Automatic social curation (in Japan)


(Cf. http://gunosy.com)

Uniqueness of Japanese culture


• Japanese social media users are used to be

familiar to social curation tools as “まとめサイト”.

• But we haven’t noticed they are social curation

services, we regarded them as a kind of SNSs.

Social media as corpora


• Feasible to exploit them as sensors

for real-world events and internet memes


Sadilek+ “Modeling Spread of

Disease from Social Interactions,”

Proc. ICWSM12

(Best Paper Award candidates)

(Cf. http://www.youtube.com/watch?v=3S2rq2SKTSw)

Social curation as corpora


1. Curated contents are manually selected.

– Only “credible” contents survive in the process.


Social curation as corpora


2. Curated contents share the same context.

– Effectively present curator’s interest or perspective.


(Cf. http://storify.com/cbccommunity/how-has-texting-changed-your-life)


How does “Social Curation” work?

Content analysis with social curation


• Despite the excitement of social curation,

few researches have been performed so far.

– Most previous work dealing with social curation

comes from sociology, not computer science.

– Most previous work from CS communities handled

social media contents, not socially curated contents.

• Social curation has so much room to grow

as a promising research topic!!


3 aspects for handling social curation


Data resourcesKnowledge &

findingsMethods

The most significant aspect

among all the aspects.Difference resources yield

difference analysis results.

Any sophisticated methods

aren’t required, since data

is clean and well organized.

2 policies for social curation analysis


• Wisdom or diversity of crowdsVegetables

Face

Green Fruits

FunnyStrange

Our contributions


Togetter Pinterest

Wisdom of

crowds

Finding topic-wise

power users[Takeuchi+ IBIS12]

Estimating

image interestingness [Ishiguro+ ICDM12]

Image context discovery[Kimura+ to appear]

Diversity,

personalization

Assisting social curation [Duh+ ICWSM12]

User preference analysis[Kataoka+ PRMU13]

Policies Data resources

Assisting social curation



Demo video


Contributions


• Data

– Togetter

• Finding

– Curated contents are supervised corpora.

– This can be acquired through large corpus analysis.

• Method

– Learning to rank : popular IR technique

Corpus analysis of Togetter



# curated contents in a listsufficiently long for analysis

# users involved in a listInvolves several users,

only social curation could capture it.

Median40 tweets

90%250 tweets

Median6 users

90%60 users

Corpus analysis of Togetter



Occupied by curators’ tweets

Curators gathers lots of tweets

by other people

Findings from the stats


• Lists are very diverse

– in terms of size, topic, and purpose.

• Lists are rather large and elaborate

– by curators’ manual efforts.

• Lists are probably suitable as corpora

– for analyzing microblog messages.


Assisting social curation


• Content discovery is time-consuming

• Let curators focus on filtering & presentation


retrieve

tweet

tweet

Partially Curated List(queries)

more?

aggregate

Rankingmanual

inspection

tweet

tweet

tweet

tweet

Twitter Timelines

suggest new contents

Finding topic-wise power users



Demo (snapshot)


Contributions


• Data

– Togetter

• Findings

– Lists are diverse, but each list has a specific topic.

– Only informative tweets would be gathered into a list.

– Topically credible users frequently appear in lists.

• Method

– Novel extension of non-negative matrix factorization

Method in detail


• Formulated as a factorization of

multiple matrices correlated to each other

Factorization

X

Y

Z

lists

users

words

popularity

0.0017% non-zero

1.23% non-zero

1.82M

160K

230K 2

W

A

BH

lists

users

words

Power users and lists

about a topic “space”

Power users and lists

about a topic “Hadoop”

Qualitative analysis


Authority Profile #follow

hadukino Space and astronomy n China 1,321

ohnuki_tsuyoshi Space developer 5,477

5thstar Space pilot finalist 2,386

ShinyaMatsuura Writer on space development 18,609

sctracker Spacecraft tracker 847

Topical words

宇宙(space) 衛星(satellite) 地球(earth) ロケット(rocket) 軌道(orbit) JAXA はやぶ

さ(Hayabusa) 探査(exploration) 打ち上げ(launch) 飛行(flight) 移籍(transfer) あかつ

き(Akatsuki)

Topic 1 “Space”

Authority Profile #follow

understeer Solutions Architect 2,368

yutuki_r Cloud, Hadoop, NoSQL 800

kimtea NoSQL, Google AppEngine 2,742

repeatedly D, Ruby, Python, Opera 4,081

siena Web techniques and design 1,065

Topical words

使う(use) Hadoop クラウド(cloud) さん(Mr.) AWS ない(not) 話(story)データ(data) これ

(this)いい(good) 開発(development) セッション(session) PHP処理(processing) DB

サーバ(server)

Topic 2 “Hadoop”

Qualitative analysis


Estimating interestingness of images


Contributions


• Data

– Togetter

• Domain knowledge & findings

– A lot of lists contained an image content.

– View counts of images can be regarded as a

quantitative and measurable proxy of interestingness.

• Methods

– Dispense with expensive CV/PR techniques

Formulation


• View count prediction by curation information


Results


• Easily available social curation features

performed greatly.


Individual preference analysis



Contributions


• Data

– Pinterest

• Domain knowledge & findings

– Every curated image collection (board) reflects

individual preference of a user.

• Methods

– Saliency-based object detection

– Image clustering based on a topic model (LDA)

Pinterest vs. Flickr



Service style (1) Content sharing

(2) Social network

(1) Social curation

(2) Social network

Network structure User-centric network Content-centric network

Resources Uploading original

contents is a major style.

Most contents come from

outside, including Flickr.

Popularity Still a mainstream, but in

a falling trend.

Emerging, becoming a

mainstream.

Research progress Frequently used in CS

researches.

Few researches have

been performed.

Framework


• Corpora for personalized image retrieval


Qualitative results


• Confirm many boards reflect user preferences


Images in a board (probably reflecting user preferences)

Proposed method

(MAP estimation)

Feature-based NN

Random sampling

Application examples


• Query = “Christmas”Storage (board) of User 1

Retrieval results for User 1

Storage (board) of User 2

Retrieval results for User 2


What is the next?

Future of social curation services


• We are now at the dawn of social curation

– But, it’s a “social curation” bubble, coming from

over-expectation for a new trend.

– A selection process will start soon.

• The key would be inter-service partnerships.

– Remember, why service providers do it :

Probe user preferences & “hidden voices”

– Fundamental limitation of a single service


Future of social curation services


• Movements of web giants cannot be ignored

– Information stored in social curation services would

be really demanding for portal sites, SNSs and ECs.

• Will become a segment-market

1. Marketing : Collaborating with ads and E-commerce.

2. Portal : Entertaining users


Future of social curation as corpora


• At the predawn in CS communities

– Finding a promising corpus yields 3 papers :-)

(Analysis, method, application)

• There is a high bar to approach

– Requires various knowledge and techniques in a wide

range of research areas

• NLP, IP, SP, IR, ML,

network analysis, human activity etc.


Future of social curation as corpora


• The fastest way to learn domain knowledge isto be a user.

• Multi-platform analysis would be promising.

– Fundamental limitation of a single platform.

– The point is the way of identifying users.

• Which do you want, wisdom or diversity?

– Wisdom : An alternative to crowdsourcing

– Diversity : Analyzing contents from multiple views


Summary


• Social curation has a great potentialfor analyzing large-scale multimedia contents.

1. Curated contents share a specific context, unlike common social media contents.

2. Quite a large amount of collection is

freely available.


• The presentation slides will be uploaded soon.

http://bit.ly/prmu2013FebCuration

Akisato Kimura, Ph.D

NTT Communication Science Laboratories

[email protected] @_akisato


Appendix

参考文献


• Duh+ “Creating stories: Social curation of Twitter messages,” Proc. ICWSM2012.

• Takeuchi+ “Stacked non-negative matrix factorization for social media analysis,” Proc. IBIS2012 (In Jap).

• Ishiguro+ “Towards automatic image understanding and mining via social curation,” Proc. ICDM2012.

• Kataoka+ (Yesterday’s talk, In Jap).

Social curation as corpora for large- scale …...Statistics Social curation as corpora for...

Documents

Transcript of Social curation as corpora for large- scale …...Statistics Social curation as corpora for...