#FAIL!

THINGS THAT DIDN‘T WORK OUT IN SOCIAL MEDIA RESEARCH

- AND WHAT WE CAN LEARN FROM THEM

Workshop at Web Science Conference, Oxford, June 29, 2015.

• Workshop hashtag: #fail2015a

• Conference hashtag: #websci15

• Workshop website: http://failworkshops.wordpress.com

• Etherpad: https://pad.okfn.org/p/fail

WELCOME

Luca Rossi @LR

Taha Yasseri @TahaYasseri

Katrin Weller @kwelle

0

100

200

300

400

500

600

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Twitter

Facebook

YouTube

Blogs

Wikis

Foursquare

LinkedIn

MySpace

Number of publications per year, which mention the respective social media platform‘s name in their title. Scopus Title Search. For details: http://kwelle.wordpress.com/2014/04/07/bibliometric-analysis-of-social-media-research/

CURRENT SOCIAL MEDIA RESEARCH

2008-2013 papers on Twitter and elections: data sources

Weller, K. (2014). Twitter und Wahlen: Zwischen 140 Zeichen und Milliarden von Tweets. In: R. Reichert (Ed.), Big Data: Analysen zum digitalen Wandel von Wissen, Macht und Ökonomie (pp. 239-257). Bielefeld: transcript.

4

Data source number No information 11

Collected manually from Twitter website (Copy-Paste / Screenshot)

6

Twitter API (no further information) 8

Twitter Search API 3

Twitter Streaming API 1

Twitter Rest API 1

Twitter API user timeline 1

Own program for accessing Twitter APIs 4

Twitter Gardenhose 1

Official Reseller (Gnip, DataSift) 3

YourTwapperKeeper 3

Other tools (e.g. Topsy) 6

Received from colleagues 1

CURRENT SOCIAL MEDIA RESEARCH

• Social media platforms and users – a moving target…

• Best practices and pitfalls in social media research are mainly discussed informally. Few possibilities to share unsuccessful approaches.

• Researchers with lots of different disciplinary backgrounds enter the field. Different fields of expertise, few interdisciplinary exchange of approaches.

• Limited possibilities for data sharing / validation and reproduction of results.

CURRENT PROBLEMS

OUR SOLUTION: #FAIL! WORKSHOPS

• First workshop at WebSci15

• Traveling on to different conferences. Up next is #ir16 (Internet Research, Phoenix, October 2015).

• Collect various examples for things that can go wrong and share them with different communities learn from experiences.

WHAT COULD GO WRONG?

More information: Kinder-Kurlanda, K., & Weller, K. (2014). “I always feel it must be great to be a hacker!” The role of interdisciplinary work in social media research. Proceedings of the ACM Web Science Conference 2014, Bloomington, USA 2014. Weller, K., & Kinder-Kurlanda, K (2014). “I love thinking about ethics!” Perspectives on ethics in social media research. In Proceedings of Internet Research 15: Boundaries and Intersections, Daegu, South Korea 2014. Weller, K., & Kinder-Kurlanda, K. (2015). Uncovering the Challenges in Collection, Sharing and Documentation: The Hidden Data of Social Media Research? In Standards and Practices in Large-Scale Social Media Research: Papers from the 2015 ICWSM Workshop. Proceedings Ninth International AAAI Conference on Web and Social Media Oxford University, May 26, 2015 – May

29, 2015, 28-37. Ann Arbor, MI: AAAI Press.

Upcoming examples from project “THE HIDDEN DATA OF SOCIAL MEDIA RESEARCH”, more information: Kinder-Kurlanda, K., & Weller, K. (2014). “I always feel it must be great to be a hacker!” The role of interdisciplinary work in social media

research. Proceedings of the ACM Web Science Conference 2014, Bloomington, USA 2014. Weller, K., & Kinder-Kurlanda, K. (2015). Uncovering the Challenges in Collection, Sharing and Documentation: The Hidden Data of Social Media

Research? In Standards and Practices in Large-Scale Social Media Research: Papers from the 2015 ICWSM Workshop. Proceedings Ninth International AAAI Conference on Web and Social Media Oxford University, May 26, 2015 – May 29, 2015, 28-37. Ann Arbor, MI: AAAI Press.

research design I

“I mean we’ve looked at the Arab spring […]. And we were just tracking #Egypt and #Lybia as keywords, as hashtags. What was quite obvious

early on was that we missed a lot of the early tweets around Egypt because they were still

using #25jan or #jan25 as the hashtag around the first big popular demonstration on the 25th

of January […] we didn’t really realize that.”

research design II

“My questions are limited to what I can do.”

dependencies

• Platforms and APIs

• Other software

death by opportunities

“To be fair, I have collected a lot of data that I haven't done anything with. I thought it would

be interesting, and it would be interesting, just I haven't had time to look at everything. […] so I

have lots of datasets still sitting on my computer waiting for me to actually do something with.”

traditional approaches meet new data

• Platforms and APIs

• Other software

ethics

“the quantity of the data we’re analyzing doesn’t really allow us to use the same

approaches that have been developed over the last centuries for social sciences. We cannot ask

for permission”

OUR AIM

• Categorization: Identify different reasons for failed approaches in social media research guidebook?

Today:

- 4 presentations

- Think about your own experiences! - … in connection to each presentation

- … in general

2:00 pm Welcome and introduction

2:15 pm Presentation by Taha Yasseri:

“The double-edged sword of statistical significance”

2:45 pm Presentation by Michael Bossetta and Anamaria Dutceac Segesten:

“Tracing Eurosceptic Party Networks via Hyperlink Network Analysis and

#FAIL!ng: Can Web Crawlers Keep up with Web Design?”

3:15 pm break

3:30 pm Presentation by Elodie Crespel:

“Extending data collection with web browser extension”

4:00 pm Presentation by Marie Van Cranenbroeck: “

Managing and Using Unstable Data in a Social Science Research about

Museums and Audiences on Social Media”

4:30 pm: Discussion and conclusions

PROGRAM

• Other experiences?

• Main categories of #fail cases?

• Top 3 take away messages for next workshop?

DISCUSSION

WHERE TO GO FROM HERE?

• Next steps – lessons learnt for future workshop organisation

• Which additional conferences?

• Publication?

THANKS!

To our

• authors

• participants

• reviewers