Making More Sense Out of Social Data Harith Alani h+p://people.kmi.open.ac.uk/harith/
@halani
harith-alani
@halani
4th Workshop on Linked Science 2014— Making Sense Out of Data (LISC2014) ISWC 2014 -‐ Riva del Garda, Italy
Topics
• Social media monitoring"• Behaviour role analysis"• Semantic sentiment "• Engagement in microblogs"• Cross platform and topic studies"• Semantic clustering"• Application examples"
Take home messages • Social media has many more challenges and
opportunities to offer"
• Fusing semantics and statistical methods is gooood"
• Studying isolated social media platforms is baaaad … or not good enough … anymore!"
Sociograms • Capturing and graphing social
relationships"• Moreno founder of sociograms and
sociometry"• Assessing psychological well-being
from social configurations of individuals and groups"
h+p://diana-‐jones.com/wp-‐content/uploads/EmoRons-‐Mapped-‐by-‐New-‐Geography.pdf
Friendship Choices Among Fourth Graders (from Moreno, 1934, p. 38
Computational Social Science
Behaviour role Analysis “A field is emerging that leverages the capacity to collect and analyze data at a scale that may reveal pa+erns of individual and group behaviours.”
Original slide by Markus Strohmaier h+p://gking.harvard.edu/files/LazPenAda09.pdf
“what does exisRng sociological network theory, built mostly on a foundaRon of one-‐Rme “snapshot” data, typically with only dozens of people, tell us about massively longitudinal data sets of millions of people .. ?”
!
Social semantic linking …. in 2003
Alani, H.; Dasmahapatra, S.; O'Hara, K.and Shadbolt, N. IdenRfying communiRes of pracRce through ontology network analysis. IEEE Intelligent Systems, 18(2) 2003.
• Domain ontologies
• SemanRcs for integraRng people, projects, and publicaRons
• IdenRfy communiRes of pracRce
• Browse evoluRon of social relaRonships and collaboraRons
Linking scientists …. in 2005
• Who is collaboraRng with whom?
• How funding programmes impacted collaboraRons over Rme?
Alani, H.; Gibbins, N.; Glaser, H.; Harris, S. and Shadbolt, N. Monitoring research collaboraRons using semanRc web technologies. ESWC, Crete, 2005.
data sources
gatherers and
mediators
ontology knowledge repository (triplestore)
applicaRons
Bigger data, greater sociograms
Social Media
Jan 29, 2013
In-house Social Platforms
Tools for monitoring social networks
Reputation Monitoring
• http://www.robust-project.eu/videos-demos "
Challenges and Opportunities • Integration"
– How to represent and connect this data?"
• Behaviour"– How can we measure and
predict behaviour?"– Which behaviours are good/bad
in which community type?"
• Community Health"– What health signs should we
look for? "– How to predict this health?"
• Engagement"– How can we measure and
maximise engagement? "
• Sentiment"– How to measure it? "– Track it towards entities and
contexts? "
Patterns
SemanRc Web & Linked Data
SemanRc SenRment Analysis
Macro/Micro Behaviour Analysis
ini#ators)
lurkers)
followers)
leaders)
Community Engagement
Table 1: Correlation Coefficients of dimensions
Dispersion Engagement Contribution Initiation Quality PopularityDispersion 1.000 0.277 0.168 0.389 0.086 0.356Engagement 0.277 1.000 0.939** 0.284 0.151 0.926**Contribution 0.168 0.939** 1.000 0.274 0.086 0.909**Initiation 0.389 0.284 0.274 1.000 -0.059 0.513Quality 0.086 0.151 0.086 -0.059 1.000 0.065Popularity 0.356 0.926** 0.909** 0.513 0.065 1.000
Figure 7: Cumulative density functions of each dimension showingthe skew in the distributions for initiated and in-degree ratio
same forum and do not deviate away, at the other ex-treme very few users are found to post in a large rangeof forums. For initiated (initiation) and in-degree ratio(popularity) the density functions are skewed towardslow values where only a few users initiate discussionsand are replied to by large portions of the community.Average points per post (quality) is also skewed to-wards lower values indicating that the majority of usersdo not provide the best answers consistently.These plots indicate that feature levels derived from
these distributions will be skewed towards lower values,for instance for initiated the definition of high for thisfeature is anything exceeding 1.55x10−5.The distribution of each dimension is shown in Fig-
ure 8 for each of the 11 induced clusters. We assessthe distribution of each feature for each cluster againstthe levels derived from the equal-frequency binning ofeach feature, thereby generating a feature-to-level map-
Figure 8: Boxplots of the feature distributions in each of the 11 clus-ters. Feature distributions are matched against the feature levels de-rived from equal-frequency binning
ping. This mapping is shown in Table 2 where certainclusters are combined together as they have the samefeature-level mapping patterns (i.e. 5,7 and 8,9). Wethen interpreted the role labels from these clusters, andtheir subsequent patterns, as follows:
• 0 - Focussed Expert Participant: this user typeprovides high quality answers but only within se-lect forums that they do not deviate from. Theyalso have a mix of asking questions and answeringthem.
• 1 - Focussed Novice: this user is focussed within afew select forums but does not provide good qual-ity content.
• 2 - Mixed Novice: is a novice across a mediumrange of topics
6
StaRsRcal Analysis
Technologies
MODELLING AND LINKING SOCIAL MEDIA DATA
June 25, 2013
• SIOC is an ontology for representing and integrating data from the social web"
• Simple, concise, and popular"
sioc-project.org
Semantically-Interlinked Online Communities (SIOC)
SRll seeking the one size that’ll fit all
SIOC for Discussion forums • SIOC is well
tailored to fit discussion forum communities"
• Needs extension to fit other communities, such as microblogs and Q&A"
Twitter in SIOC • Microblogs"
• No forum structure"
IBM Connections in SIOC
SAP Community Network in SIOC
BEHAVIOUR ROLES
h+p://www.smrfoundaRon.org/wp-‐content/uploads/2008/12/disRnguishing-‐a+ributes-‐of-‐social-‐roles.png
Why we monitor behaviour? • Understand role of people in a community
• Monitor impact of behaviour on community evolution
• Forecast community future
• Learn which behaviour should be encouraged or discouraged
• Find the best mix of behaviour to increase engagement in an online community
• See which users need more support, which ones should be confined, and which ones should be promoted
Linking networks
27/
<?xml version="1.0"?>!<rdf:RDF! xmlns="http://tagora.ecs.soton.ac.uk/schemas/tagging#"! xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"! xmlns:xsd="http://www.w3.org/2001/XMLSchema#"! xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"! xmlns:owl="http://www.w3.org/2002/07/owl#"! xml:base="http://tagora.ecs.soton.ac.uk/schemas/tagging">! <owl:Ontology rdf:about=""/>! <owl:Class rdf:ID="Post"/>! <owl:Class rdf:ID="TagInfo"/>! <owl:Class rdf:ID="GlobalCooccurrenceInfo"/>! <owl:Class rdf:ID="DomainCooccurrenceInfo"/>! <owl:Class rdf:ID="UserTag"/>! <owl:Class rdf:ID="UserCooccurrenceInfo"/>! <owl:Class rdf:ID="Resource"/>! <owl:Class rdf:ID="GlobalTag"/>! <owl:Class rdf:ID="Tagger"/>! <owl:Class rdf:ID="DomainTag"/>! <owl:ObjectProperty rdf:ID="hasPostTag">! <rdfs:domain rdf:resource="#TagInfo"/>! </owl:ObjectProperty>! <owl:ObjectProperty rdf:ID="hasDomainTag">! <rdfs:domain rdf:resource="#UserTag"/>! </owl:ObjectProperty>! <owl:ObjectProperty rdf:ID="isFilteredTo">! <rdfs:range rdf:resource="#GlobalTag"/>! <rdfs:domain rdf:resource="#GlobalTag"/>! </owl:ObjectProperty>! <owl:ObjectProperty rdf:ID="hasResource">! <rdfs:domain rdf:resource="#Post"/>! <rdfs:range =…!
Linking people via sensors, social media, papers, projects
• Integration of physical presence and online information"• Semantic user profile generation"• Logging of face-to-face contact"• Social network browsing"• Analysis of online vs offline social networks"Alani, H.; Szomszor, M.; Ca+uto, C.; den Broeck, W.; Correndo, G. and Barrat, A.. Live social semanRcs. ISWC, Washington, DC, 2009
0"
0.2"
0.4"
0.6"
0.8"
1"
1.2"
1" 5" 9" 13" 17" 21" 25" 29" 33" 37" 41" 45"
H.Index"F2F"Degree"F2F"Strength"
Online+offline social networks
• What’s your social configura-on? • What does it say about you? • And what you’ll become?
Barrat, A.; C., Ca+uto; M., Szomszor; W., Van den Broeck and Alani, H. Social dynamics in conferences: analyses of data from the Live Social SemanRcs applicaRon. ISWC, Shanghai, China, 2010.
h+p://www.tehowners.com/info/Popular%20Culture%20&%20Social%20Media/Online%20CommuniRes.jpg
Clustering for identifying emerging roles
– Map the distribution of each feature in each cluster to a level (i.e. low, mid, high)
– Align the mapping patterns with role labels
Table 1: Correlation Coefficients of dimensions
Dispersion Engagement Contribution Initiation Quality PopularityDispersion 1.000 0.277 0.168 0.389 0.086 0.356Engagement 0.277 1.000 0.939** 0.284 0.151 0.926**Contribution 0.168 0.939** 1.000 0.274 0.086 0.909**Initiation 0.389 0.284 0.274 1.000 -0.059 0.513Quality 0.086 0.151 0.086 -0.059 1.000 0.065Popularity 0.356 0.926** 0.909** 0.513 0.065 1.000
Figure 7: Cumulative density functions of each dimension showingthe skew in the distributions for initiated and in-degree ratio
same forum and do not deviate away, at the other ex-treme very few users are found to post in a large rangeof forums. For initiated (initiation) and in-degree ratio(popularity) the density functions are skewed towardslow values where only a few users initiate discussionsand are replied to by large portions of the community.Average points per post (quality) is also skewed to-wards lower values indicating that the majority of usersdo not provide the best answers consistently.These plots indicate that feature levels derived from
these distributions will be skewed towards lower values,for instance for initiated the definition of high for thisfeature is anything exceeding 1.55x10−5.The distribution of each dimension is shown in Fig-
ure 8 for each of the 11 induced clusters. We assessthe distribution of each feature for each cluster againstthe levels derived from the equal-frequency binning ofeach feature, thereby generating a feature-to-level map-
Figure 8: Boxplots of the feature distributions in each of the 11 clus-ters. Feature distributions are matched against the feature levels de-rived from equal-frequency binning
ping. This mapping is shown in Table 2 where certainclusters are combined together as they have the samefeature-level mapping patterns (i.e. 5,7 and 8,9). Wethen interpreted the role labels from these clusters, andtheir subsequent patterns, as follows:
• 0 - Focussed Expert Participant: this user typeprovides high quality answers but only within se-lect forums that they do not deviate from. Theyalso have a mix of asking questions and answeringthem.
• 1 - Focussed Novice: this user is focussed within afew select forums but does not provide good qual-ity content.
• 2 - Mixed Novice: is a novice across a mediumrange of topics
6
Table 2: Mapping of cluster dimensions to levels
Cluster Dispersion Initiation Quality Popularity0 L M H L1 L L L L2 M H L H3 H H H H4 L H H M5,7 H H L H6 L H M M8,9 M H H H10 L H M H
• 3 - Distributed Expert: an expert on a variety oftopics and participates across many different fo-rums
• 4 - Focussed Expert Initiator: similar to cluster0 in that this type of user is focussed on certaintopics and is an expert on those, but to a large ex-tent starts discussions and threads, indicating thathis/her shared content is useful to the community
• 5.7 - Distributed Novice: participates across arange of forums but is not knowledgeable on anytopics
• 6 - Focussed Knowledgeable Member: con-tributes to only a few forums, has medium-levelexpertise (i.e. he/she is neither an expert nor anovice) and has medium popularity
• 8,9 - Mixed Expert: medium-dispersed user whoprovides high-quality content
• 10 - Focussed Knowledgeable Sink: focusseduser who has medium-level expertise but who getsa lot of the community replying to them - hence asink. Differs from cluster 6 in terms of popularity.
6. Analysis: Community Health
Deriving a community’s role composition providescommunity operators and hosts with amacro-level viewof how their community is operating and how it is func-tioning. Understanding what is a healthy and unhealthycomposition in a community involves analysing how agiven role composition has been associated with com-munity activity, interaction or some other measure in thepast and reusing that knowledge. Forums and communi-ties operating within the same platform may also differsuch that what turns a community healthy in one loca-tion may be different from another. In this section wedescribe how community analysis is possible throughour presented approach to derive the role compositionof a community using semantic rules.
6.1. Experimental Setup
To demonstrate the utility of our approach we anal-ysed each of the 33 SAP communities from 2009through to 2011. Figure 9 shows how our dataset wasdivided into the tuning section - i.e. the first half of2008 in which we derived our clusters and aligned themto roles (as described in Section 5) - and the analysissection. We began with the 1st January 2009 as our col-lect date by taking a feature window 6 months prior tothis date (going back to the 2nd half of 2008) in whichwe measured the behaviour dimensions for each com-munity’s users. In order to gauge the role compositionin a community over time we move our collect date onone week at a time and use the 6-months prior to thisdate as our feature window. As Figure 9 demonstrateswe repeat this process until we reach 2011.
Figure 9: Windows used for a) tuning of the clusters and the derivationof roles and b) the analysis of community health. Role compositionis derived every week from 2009 onwards using a 6-month windowgoing back from the collection date.
By measuring the behaviour dimensions of individ-ual users in individual communities we are able to inferthe roles of the users using the semantic rules describedin Section 4. This provides a micro-level assessment ofthe roles that individual users assume. We can then lookat the macro-level by deriving the role composition of agiven community at a given point in time by measuringhow many users have a specific role. Such role compo-sition analysis allows for predictions to then be made.To demonstrate the application of such analysis we per-formed three distinct experiments (each designed to ex-plore one of our three aforementioned research ques-tions):
1. Composition Analysis: assesses the average rolecomposition in each community and clusters thembased on the compositions. We also pick out eachcommunity’s most popular role and measure whatpercentage of the community that role covers.
2. Activity Increase/Decrease: we perform a binaryclassification task such that at timestep t = k + 1we predict whether the community’s activity (i.e.number of posts) has increased or decreased since
7
• 1 - Focussed Novice: focussed within a few select forums but does not provide good quality content.
• 2 - Mixed Novice: a novice across a medium range of topics
• 3 - Distributed Expert: expert on a variety of topics and participates across many different forums
….
Mapping of cluster dimensions to levels
Encoding Roles in Ontologies with SPIN
• Bottom Up analysis"– Every community member is
classified into a “role”"– Unknown roles might be identified"– Copes with role changes over time "
iniRators
lurkers
followers
leaders
Structural, social network, reciprocity, persistence, participation
Feature levels change with the dynamics of the community
Associations of roles with a collection of feature-to-level mappings e.g. in-degree -> high, out-degree -> high
Run rules over each user’s features and derive the community role composition
Behaviour role extraction from Social Media Data
Angeletou, S; Rowe, M, and Alani, H. Modelling and analysis of user behaviour in online communiRes. ISWC 2011, Bonn, Germany
Correlation of behaviour roles with community activity
Forum on CommuRng and Transport Forum on Rugby Forum on Mobile Phones and PDAs
• How certain behaviour roles impact activity in different community types?"
Community types • So do communities of different types behave differently?
• Analysed IBM Connections communities to study participation, activity, and behaviour of users
• Compare exhibited community with what users say they use the community for – Does macro behaviour match micro needs?
Community types Community
Wiki Page Blog Post Forum Thread
Wiki Edit Blog Comment Forum Reply
Bookmark Tag
File
§ Data consists of non-private info on IBM Connections Intranet deployment
§ Communities: § ID § Creation date § Members § Used applications
(blogs, Wikis, forums)
§ Forums: § Discussion threads § Comments § Dates § Authors and
responders
Community types • Muller, M. (CHI 2012) identified five distinct community types in
IBM Connections:"– Communities of Practice (CoP): for sharing information and network"– Teams: shared goal for a particular project or client"– Technical Support: support for a specific technology"– Idea Labs Communities: for focused brainstorming "– Recreation Communities: recreational activities unrelated to work.
"• Our data consisted of 186 most active communities:"
– 100 CoPs, 72 Teams, and 14 Technical Support communities "– No Ideas of Recreation communities"
Behaviour roles in different community types
Macro
Rowe, M. Fernandez, M., Alani, H., Ronen, I., Hayes, C., Karnstedt, M.: Behaviour Analysis across different types of Enterprise Online Communities. WebSci 2012
• Members of Team communities are more engaged, popular, and initiate more discussions
• Technical Support community members are mostly active in a few communities, and don’t initiate or contribute much!
• CoP members are active across many communities, and contribute more
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Churn Rate
FPR
TPR
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
User Count
FPR
TPR
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Seeds / Non−seeds Prop
FPR
TPR
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Clustering Coefficient
FPR
TPR
• Machine learning models to predict community health based on compositions and evolution of user behaviour
• Churn rate: proportion of community leavers in a given time segment.
• User count: number of users who posted at least once.
• Seeds to Non-seeds ratio: proportion of posts that get responses to those that don’t
• Cluster coefficient: extent to which the community forms a clique.
Health categories
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Churn Rate
FPR
TPR
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
User Count
FPR
TPR
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Seeds / Non−seeds Prop
FPR
TPR
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Clustering Coefficient
FPR
TPR
False Positive Rate
False Positive Rate False Positive Rate
False Positive Rate
True
Pos
itive
Rat
e Tr
ue P
ositi
ve R
ate
True
Pos
itive
Rat
e Tr
ue P
ositi
ve R
ate
The fewer Focused Experts in the community, the more posts will received a reply! There is no “one size fits all” model!
Behaviour roles and community health
Rowe, M. and Alani, H. What makes communiRes Rck? Community health analysis using role composiRons. SocialCom 2012, Amsterdam, The Netherlands.
SEMANTIC SENTIMENT ANALYSIS
Semantic sentiment analysis on social media
• Range of features and statistical classifiers have been used in social media sentiment analysis in recent years
• Semantics have often been overlooked
– Semantic Features
– Semantic Patterns
• Semantic concepts can help determining sentiment even when no good lexical clues are present
Sentiment Analysis
hate negative honest positive inefficient negative Love positive …
Sentiment Lexicon
I hate the iPhone
I really love the iPhone
Lexical-Based Approach
Learn Model
Apply Model
Naïve Bayes, SVM, MaxEnt , etc.
Training Set
Test Set
Model
Machine Learning Approach
Semantic Concept Extraction • Extract semantic concepts from tweets data and incorporate
them into the supervised classifier training.
Fig. 1. Measuring correlation of semantic concepts with negative/positive sentiment. These se-mantic concepts are then incorporated in sentiment classification.
OpenCalais and Zemanta. Their experimental results showed that AlchemyAPI per-forms best for entity extraction and semantic concept mapping. Our datasets consist ofinformal tweets, and hence are intrinsically different from those used in [10]. There-fore we conducted our own evaluation, and randomly selected 500 tweets from the STScorpus and asked 3 evaluators to evaluate the semantic concept extraction outputs gen-erated from AlchemyAPI, OpenCalais and Zemanta.
No. of Concepts Entity-Concept Mapping Accuracy (%)Extraction Tool Extracted Evaluator 1 Evaluator 2 Evaluator 3AlchemyAPI 108 73.97 73.8 72.8Zemanta 70 71 71.8 70.4OpenCalais 65 68 69.1 68.7Table 2. Evaluation results of AlchemyAPI, Zemanta and OpenCalais.
The assessment of the outputs was based on (1) the correctness of the extractedentities; and (2) the correctness of the entity-concept mappings. The evaluation resultspresented in Table 2 show that AlchemyAPI extracted the most number of conceptsand it also has the highest entity-concept mapping accuracy compared to OpenCalaisand Zematna. As such, we chose AlchemyAPI to extract the semantic concepts fromour three datasets. Table 3 lists the total number of entities extracted and the number ofsemantic concepts mapped against them for each dataset.
STS HCR OMDNo. of Entities 15139 723 1194No. of Concepts 29 17 14
Table 3. Entity/concept extraction statistics of STS, OMD and HCR using AlchemyAPI.
Impact of adding semantic features • Incorporating semantics increases accuracy against the
baseline by:
– 6.5% for negative sentiment,
– 4.8% for positive sentiment
– F1 = 75.95%, with 77.18% Precision and 75.33% Recall
Saif, H., He, Y. and Alani, H. SemanRc senRment analysis of twi+er. ISWC 2012, Boston, US.
Destroy(((Invading(Germs((
Nega%ve'Concept'Nega%ve'
• OK, but what about such cases?
• Can semantics help?
Semantic Pattern Approaches • Apply syntac-c and seman-c processing techniques
• Use external semanRc resources (e.g. Dbpedia, Freebase) to idenRfy semanRc concepts in Tweets
Trojan Horse
Threat
Hack
Code
Malware
Program
Dangerous
Harm
Spyware
• Extract clusters of similar contextual semanRcs and senRment, and use as pa+erns in senRment analysis
Features
MaxEnt Classifier
Accuracy F-Measure
Minimum Maximum Average Minimum Maximum Average
Syntactic
Twitter Features -0.23 3.91 1.24 -0.25 4.53 1.62POS -0.89 2.92 0.79 -0.91 5.67 1.25Lexicon -0.44 4.23 1.30 -0.38 5.81 1.83Average -0.52 3.69 1.11 -0.52 5.33 1.57
Semantic
Concepts -0.22 2.76 1.20 -0.40 4.80 1.51LDA-Topics -0.47 3.37 1.20 -0.68 6.05 1.68SS-Patterns 0.70 9.87 3.05 1.23 9.78 3.76
Average 0.00 5.33 1.82 0.05 6.88 2.32Table 6: Win/Loss in Accuracy and F-measure of using different features for sentiment classifica-tion on all nine datasets.
classifier described in Section 4.2. Note that STS-Gold is the only dataset among theother 9 that provides named entities manually annotated with their sentiment labels(positive, negative, neutral). Therefore, our evaluation in this task is done using theSTS-Gold dataset only.
Features Accuracy
Positive Sentiment Negative Sentiment Neutral Sentiment Average
P R F1 P R F1 P R F1 P R F1Unigrams 48.28 92 79.31 85.19 6.67 7.69 7.14 22.22 25 23.53 40.3 37.33 38.62LDA-Topics 58.62 92 79.31 85.19 31.82 53.85 40 36.36 25 29.63 53.39 52.72 51.6Semantic Concepts 55.17 92 79.31 85.19 25 38.46 30.3 30.77 25 27.59 49.26 47.59 47.69SS-Patterns 60.34 92 79.31 85.19 34.78 61.54 44.44 40 25 30.77 55.59 55.28 53.47
Table 7: Accuracy and averages of Precision, Recall, and F measures of entity-level sentiment classification using differentfeatures.
Table 7 reports the results in accuracy, precision (P), recall (R) and F1 measure ofpositive, negative and neutral sentiment classification performances from using unigrams,semantic concepts, LDA-Topics and SS-Patterns features. Generally, our SS-Patternsoutperform all other features including word unigrams in all measures. In particular,merely using word unigrams for classification gives the lowest performance of 48.24%and 38.62% in accuracy and average F1. However, augmenting the feature space withSS-Patterns improves the performance significantly by 12.06% in accuracy and 14.85%in average F1. Our SS-Patterns also outperform LDA-Topics and semantic conceptsfeatures by at least 1.72% and 1.87% in accuracy and average F1.
As for per-class sentiment classification performance, we observe that all featuresproduce high and similar performances on detecting positive entities. This is becauseclassifiers trained from either feature set fail in detecting the sentiment of the sameentities. Moreover, it seems that detecting negative and neutral entities are much moredifficult tasks than detecting positive ones. For example, unigrams perform very poorlyin detecting negative entities with a F1 less then 8%. Although the performance improvesa lot by using SS-Patterns, it is still much lower than the positive classification perfor-mance. For neutral sentiment classification the performance is the lowest with unigrams(F1 = 23.53%) while it is the highest with SS-Patterns (F1 = 30.77%). Such varyingperformance might be due to the uneven sentiment class distribution in the entity dataset.As can be noted from Table 2, positive entities constitute 50% of the total number ofentities while the neutral and negative entities form together the other 50%.
Win/Loss in Accuracy and F-‐measure of using different features for senRment classificaRon on all nine datasets.
Based on 9 Twi+er datasets
Tweet-Level Sentiment Analysis
Hassan S., He, Y., Miriam F.and Harith A., SemanRc Pa+erns for SenRment Analysis of Twi+er, ISWC 2014, Trento, Italy
55.00
57.00
59.00
61.00
63.00
65.00
67.00
Accuracy F1
Unigrams LDA-‐Topics SemanRc Concepts SS-‐Pa+erns
Gold standard of 58 enRRes
Entity-Level Sentiment Analysis
Hassan S., He, Y., Miriam F.and Harith A., SemanRc Pa+erns for SenRment Analysis of Twi+er, ISWC 2014, Trento, Italy
ENGAGEMENT ANALYSIS
ONLINE ENGAGEMENT
Forum on a celebrity
Forum on transport
Different Engagement Patterns
Different Engagement Parameters
Different Engagement Parameters
… “few people took part”
• 309 invitees from media, academia, and public engagement bodies"
• 2 invitees contributed to the site, with
2 edits!!
Recipe for more engaging posts?
Ask the (Social) Data
• What’s the model of good/bad tweets?"• What features are associated with each group?"
Feature Engineering
initial tweet that generates a reply. Features which describe seed posts can bedivided into two sets: user features - attributes that define the user making thepost; and, content features - attributes that are based solely on the post itself.We wish to explore the application of such features in identifying seed posts, todo this we train several machine learning classifiers and report on our findings.However we first describe the features used.
4.1 Feature Extraction
The likelihood of posts eliciting replies depends upon popularity, a highly subjec-tive term influenced by external factors. Properties influencing popularity includeuser attributes - describing the reputation of the user - and attributes of a post’scontent - generally referred to as content features. In Table 1 we define user andcontent features and study their influence on the discussion “continuation”.
Table 1. User and Content Features
User FeaturesIn Degree: Number of followers of U #
Out Degree: Number of users U follows #List Degree: Number of lists U appears on. Lists group users by topic #Post Count: Total number of posts the user has ever posted #
User Age: Number of minutes from user join date #Post Rate: Posting frequency of the user PostCount
UserAge
Content FeaturesPost length: Length of the post in characters #Complexity: Cumulative entropy of the unique words in post p λ
of total word length n and pi the frequency of each word!
i∈[1,n] pi(log λ−log pi)
λUppercase count: Number of uppercase words #
Readability: Gunning fog index using average sentence length (ASL) [7]and the percentage of complex words (PCW). 0.4(ASL+ PCW )
Verb Count: Number of verbs #Noun Count: Number of nouns #
Adjective Count: Number of adjectives #Referral Count: Number of @user #
Time in the day: Normalised time in the day measured in minutes #Informativeness: Terminological novelty of the post wrt other posts
The cumulative tfIdf value of each term t in post p!
t∈p tfidf(t, p)Polarity: Cumulation of polar term weights in p (using
Sentiwordnet3 lexicon) normalised by polar terms count Po+Ne|terms|
4.2 Experiments
Experiments are intended to test the performance of different classification mod-els in identifying seed posts. Therefore we used four classifiers: discriminativeclassifiers Perceptron and SVM, the generative classifier Naive Bayes and thedecision-tree classifier J48. For each classifier we used three feature settings:user features, content features and user+content features.
Datasets For our experiments we used two datasets of tweets available on theWeb: Haiti earthquake tweets4 and the State of the Union Address tweets.5 The
4 http://infochimps.com/datasets/twitter-haiti-earthquake-data5 http://infochimps.com/datasets/tweets-during-state-of-the-union-address
• Focus Features"– Topic entropy: the distribution of the author across community forums"– Topic Likelihood: the likelihood that a user posts in a specific forum given his post history"
• Measures the affinity that a user has with a given forum"• Lower likelihood indicates a user posting on an unfamiliar topic"
initial tweet that generates a reply. Features which describe seed posts can bedivided into two sets: user features - attributes that define the user making thepost; and, content features - attributes that are based solely on the post itself.We wish to explore the application of such features in identifying seed posts, todo this we train several machine learning classifiers and report on our findings.However we first describe the features used.
4.1 Feature Extraction
The likelihood of posts eliciting replies depends upon popularity, a highly subjec-tive term influenced by external factors. Properties influencing popularity includeuser attributes - describing the reputation of the user - and attributes of a post’scontent - generally referred to as content features. In Table 1 we define user andcontent features and study their influence on the discussion “continuation”.
Table 1. User and Content Features
User FeaturesIn Degree: Number of followers of U #
Out Degree: Number of users U follows #List Degree: Number of lists U appears on. Lists group users by topic #Post Count: Total number of posts the user has ever posted #
User Age: Number of minutes from user join date #Post Rate: Posting frequency of the user PostCount
UserAge
Content FeaturesPost length: Length of the post in characters #Complexity: Cumulative entropy of the unique words in post p λ
of total word length n and pi the frequency of each word!
i∈[1,n] pi(log λ−log pi)
λUppercase count: Number of uppercase words #
Readability: Gunning fog index using average sentence length (ASL) [7]and the percentage of complex words (PCW). 0.4(ASL+ PCW )
Verb Count: Number of verbs #Noun Count: Number of nouns #
Adjective Count: Number of adjectives #Referral Count: Number of @user #
Time in the day: Normalised time in the day measured in minutes #Informativeness: Terminological novelty of the post wrt other posts
The cumulative tfIdf value of each term t in post p!
t∈p tfidf(t, p)Polarity: Cumulation of polar term weights in p (using
Sentiwordnet3 lexicon) normalised by polar terms count Po+Ne|terms|
4.2 Experiments
Experiments are intended to test the performance of different classification mod-els in identifying seed posts. Therefore we used four classifiers: discriminativeclassifiers Perceptron and SVM, the generative classifier Naive Bayes and thedecision-tree classifier J48. For each classifier we used three feature settings:user features, content features and user+content features.
Datasets For our experiments we used two datasets of tweets available on theWeb: Haiti earthquake tweets4 and the State of the Union Address tweets.5 The
4 http://infochimps.com/datasets/twitter-haiti-earthquake-data5 http://infochimps.com/datasets/tweets-during-state-of-the-union-address
Classification of Posts
Seed Posts Non-Seed Posts
§ Binary classification model
§ Trained with social, content, and combined features § 80/20 training/testing
§ Identify best feature types, and
top individual features, in predicting post classification
Engagement on Boards.ie • Which posts are
more likely to stimulate responses and discussions?"
• What impacts engagement more; user features, post content, forum affinity?"
• Which individual features are most influential?"
Top Features for Engagement on Boards.ie
Rowe, M.; Angeletou, S. and Alani, H. AnRcipaRng discussion acRvity on community forums. SocialCom 2011, Boston, MA, USA.
• Content features were key!"• Best predictions were achieved when combining user, content, and focus features"• URLs (Referral Count) in a post negatively impact discussion activity"• Seed Posts (posts that receive replies) are associated with greater forum likelihood"activity levels, and because it has already been used in other
investigations (e.g., [14]).Boards.ie does not provide explicit social relations be-
tween community members, unlike for example Facebook andTwitter. We followed the same strategy proposed in [3] forextracting social networks from Digg, and built the Boards.iesocial network for users, weighting edges cumulatively by thenumber of replies between any two users.
TABLE IDESCRIPTION OF THE BOARDS.IE DATASET
Posts Seeds Non-Seeds Replies Users1,942,030 90,765 21,800 1,829,465 29,908
In order to take derive our features we required a windowof !-days from which the social graph can be compiled andrelevant measurements taken. Based on previous work overthe same dataset in [14], we used a similar window of 188days (roughly 6-months) prior to the post date of a given seedor non-seed post. For instance, if a seed post " is made attime #, then our window from which the features (i.e., userand focus features) are derived is from # − 188 to # − 1. Inusing this heuristic we ensure that the features compiled foreach post are independent of future outcomes and will notbias our predictions - for example a user may increase theiractivity following the seed post which would not be a trueindicator of their behaviour at the time the post was made.Table I summarises the dataset and the number of posts (seeds,non-seeds and replies) and users contained within.
V. CLASSIFICATION: DETECTING SEED POSTS
Predicting discussion activity levels are often hindered byincluding posts that yield no replies. We alleviate this problemby differentiating between seed posts and non-seeds through abinary classification task. Once seed posts have been identifiedwe then attempt to predict the level of discussion that suchposts will generate. To this end, we look for the best classifierfor identifying seed and non-seed posts and then search for thefeatures that played key roles in distinguishing seed posts fromnon-seeds, thereby observing key features that are associatedwith discussions.
A. Experimental Setup
For our experiments we are using the previously describeddataset collected from Boards.ie containing both seeds andnon-seeds throughout 2006. For our collection of posts webuilt the content, user, and focus features listed in section IIIfrom the past 6 months of data leading up to the date on whichthe post was published - thereby ensuring no bias from futureevents in our dataset. We split the dataset into 3 sets using a70/20/10% random split, providing a training set, a validationset and a test set.
Our first task was to perform model selection by testing fourdifferent classifiers: SVM, Naive Bayes, Maximum Entropyand J48 decision tree, when trained on various individual fea-ture sets and their combinations: user features, content features
and focus features. This model selection phase was performedby training each classifier, together with the combination offeatures, using the 70% training split and labelling instancesin the held out 20% validation split.
Once we had identified the best performing model - i.e.,the classifier and combination of feature set that produces thehighest $1 value - our second task was to perform featureassessment, thereby identifying key features that contributesignificantly to seed post prediction accuracy. For this wetrained the best performing model from the model selectionphase over the training split and tested its classification accu-racy over the 10% test split, dropping individual features fromthe model and recording the reduction in accuracy followingthe omission of a given feature. Given that we are performinga binary classification task we use the standard performancemeasures for such a scenario: precision, recall and f-measure- setting % = 1 for an equal weighting of precision andrecall. We also measure the area under the Receiver OperatorCharacteristic curve to gauge the relationship between recalland fallout - i.e., false negative rate.
TABLE IIRESULTS FROM THE CLASSIFICATION OF SEED POSTS USING
VARYING FEATURE SETS AND CLASSIFICATION MODELS
! " #1 "$%User SVM 0.775 0.810 0.774 0.581
Naive Bayes 0.691 0.767 0.719 0.540Max Ent 0.776 0.806 0.722 0.556J48 0.778 0.809 0.734 0.582
Content SVM 0.739 0.804 0.729 0.511Naive Bayes 0.730 0.794 0.740 0.616Max Ent 0.758 0.806 0.730 0.678J48 0.795 0.822 0.783 0.617
Focus SVM 0.649 0.805 0.719 0.500Naive Bayes 0.710 0.737 0.722 0.588Max Ent 0.649 0.805 0.719 0.586J48 0.649 0.805 0.719 0.500
User + Content SVM 0.790 0.808 0.727 0.509Naive Bayes 0.712 0.772 0.732 0.593Max Ent 0.767 0.807 0.734 0.671J48 0.795 0.821 0.779 0.675
User + Focus SVM 0.776 0.810 0.776 0.583Naive Bayes 0.699 0.778 0.724 0.585Max Ent 0.771 0.806 0.722 0.607J48 0.777 0.810 0.742 0.617
Content + Focus SVM 0.750 0.805 0.729 0.511Naive Bayes 0.732 0.787 0.746 0.658Max Ent 0.762 0.807 0.731 0.692J48 0.798 0.823 0.787 0.662
All SVM 0.791 0.808 0.727 0.510Naive Bayes 0.724 0.780 0.740 0.637Max Ent 0.768 0.808 0.733 0.688J48 0.798 0.824 0.792 0.692
B. Results: Model Selection
1) Model Selection with Individual Features: The resultsfrom our first experiments are shown in Table II. Lookingfirst at individual feature sets - e.g., SVM together withuser features - we see that content features yield improvedpredictive performance over user and focus features. On dis-cussion forums content appears to play a more central role
• Lower informativeness is associated with seed posts"
– i.e. seeds use language that is familiar to the community"
Top Features for Engagement on Twitter
which we found to be 0.674 indicating a good correlation between the two listsand their respective ranks.
Table 4. Features ranked by Information Gain Ratio wrt Seed Post class label. Thefeature name is paired within its IG in brackets.
Rank Haiti Union Address1 user-list-degree (0.275) user-list-degree (0.319)2 user-in-degree (0.221) content-time-in-day (0.152)3 content-informativeness (0.154) user-in-degree (0.133)4 user-num-posts (0.111) user-num-posts (0.104)5 content-time-in-day (0.089) user-post-rate (0.075)6 user-post-rate (0.075) user-out-degree (0.056)7 content-polarity (0.064) content-referral-count (0.030)8 user-out-degree (0.040) user-age (0.015)9 content-referral-count (0.038) content-polarity (0.015)10 content-length (0.020) content-length (0.010)11 content-readability (0.018) content-complexity (0.004)12 user-age (0.015) content-noun-count (0.002)13 content-uppercase-count (0.012) content-readability (0.001)14 content-noun-count (0.010) content-verb-count (0.001)15 content-adj-count (0.005) content-adj-count (0.0)16 content-complexity (0.0) content-informativeness (0.0)17 content-verb-count (0.0) content-uppercase-count (0.0)
Fig. 3. Contributions of top-5 features to identifying Non-seeds (N) and Seeds(S).Upper plots are for the Haiti dataset and the lower plots are for the Union Addressdataset.
The top-most ranks from each dataset are dominated by user features includ-ing the list-degree, in-degree, num-of-posts and post-rate. Such features describea user’s reputation, where higher values are associated with seed posts. Figure3 shows the contributions of each of the top-5 features to class decisions in thetraining set, where the list-degree and in-degree of the user are seen to correlateheavily with seed posts. Using these rankings our next experiment explored theeffects of training a classification model using only the top-k features, observing
• Top are list-degree, in-degree, informativeness, and #posts"
"• Top are list-
degree, time of posting, in-degree, and #posts"
Rowe, M., Angeletou, S., Alani, H. PredicRng Discussions on the Social SemanRc Web. ESWC, Crete, 2011
which we found to be 0.674 indicating a good correlation between the two listsand their respective ranks.
Table 4. Features ranked by Information Gain Ratio wrt Seed Post class label. Thefeature name is paired within its IG in brackets.
Rank Haiti Union Address1 user-list-degree (0.275) user-list-degree (0.319)2 user-in-degree (0.221) content-time-in-day (0.152)3 content-informativeness (0.154) user-in-degree (0.133)4 user-num-posts (0.111) user-num-posts (0.104)5 content-time-in-day (0.089) user-post-rate (0.075)6 user-post-rate (0.075) user-out-degree (0.056)7 content-polarity (0.064) content-referral-count (0.030)8 user-out-degree (0.040) user-age (0.015)9 content-referral-count (0.038) content-polarity (0.015)10 content-length (0.020) content-length (0.010)11 content-readability (0.018) content-complexity (0.004)12 user-age (0.015) content-noun-count (0.002)13 content-uppercase-count (0.012) content-readability (0.001)14 content-noun-count (0.010) content-verb-count (0.001)15 content-adj-count (0.005) content-adj-count (0.0)16 content-complexity (0.0) content-informativeness (0.0)17 content-verb-count (0.0) content-uppercase-count (0.0)
Fig. 3. Contributions of top-5 features to identifying Non-seeds (N) and Seeds(S).Upper plots are for the Haiti dataset and the lower plots are for the Union Addressdataset.
The top-most ranks from each dataset are dominated by user features includ-ing the list-degree, in-degree, num-of-posts and post-rate. Such features describea user’s reputation, where higher values are associated with seed posts. Figure3 shows the contributions of each of the top-5 features to class decisions in thetraining set, where the list-degree and in-degree of the user are seen to correlateheavily with seed posts. Using these rankings our next experiment explored theeffects of training a classification model using only the top-k features, observing
HaiR Earthquake
State Union Address
former dataset contains tweets which relate to the Haiti earthquake disaster,covering a varying timespan. The latter dataset contains all tweets publishedduring the duration of president Barack Obama’s State of the Union Addressspeech. Our goal is to predict discussion activity based on the features of a givenpost by first identifying seed posts, before moving on to predict the discussionlevel.
Within the above datasets many of the posts are not seeds, but are insteadreplies to previous posts, thereby featuring in the discussion chain as a node.In [13] retweets are considered as part of the discussion activity. In our workwe identify discussions using the explicit “in reply to” information obtainedby the Twitter API, which does not include retweets. We make this decisionbased on the work presented in boyd et.al [4], where an analysis of retweetingas a discussion practice is presented, arguing that message forwards adhere todifferent motives which do not necessarily designate a response to the initialmessage. Therefore, we only investigate explicit replies to messages. To gatherour discussions, and our seed posts, we iteratively move up the reply chain - i.e.,from reply to parent post - until we reach the seed post in the discussion. Wedefine this process as dataset enrichment, and is performed by querying Twitter’sREST API6 using the in reply to id of the parent post, and moving one-step ata time up the reply chain. This same approach has been employed successfullyin work by [12] to gather a large-scale conversation dataset from Twitter.
Table 2. Statistics of the datasets used for experiments
Dataset Users Tweets Seeds Non-Seeds RepliesHaiti 44,497 65,022 1,405 60,686 2,931
Union Address 66,300 80,272 7,228 55,169 17,875
Table 2 shows the statistics that explain our collected datasets. One canobserve the difference in conversational tweets between the two corpora, wherethe Haiti dataset contains fewer seed posts as a percentage than the Uniondataset, and therefore fewer replies. However, as we explain later a later section,this does not correlate with a higher discussion volume in the former dataset. Weconvert the collected datasets from their proprietary JSON formats into triples,annotated using concepts from our above behaviour ontology, this enables ourfeatures to be derived by querying our datasets using basic SPARQL queries.
Evaluation Measures The task of identifying seed posts is a binary classifi-cation problem: is this post a seed post or not?. We can therefore restrict ourlabels to one of two classes: seed and non-seed. To evaluate the performance ourmethod we use four measures: precision, recall, f-measure and area under theReceiver Operator Curve. Precision measures the proportion of retrieved postswhich were actually seed posts, recall measures the proportion of seed postswhich were correctly identified and fallout measures the proportion of non-seedposts which were incorrectly classified as seed posts (i.e., false positive rate). We
6 http://dev.twitter.com
neg pos
05
1015
2025
30
Length
neg pos0.0
0.5
1.0
1.5
Complexity
neg pos
010
2030
40
Readability
neg pos
−4−2
02
4
Polarity
• Top influential features do not match those found for Board.ie or for two non-random Twitter datasets"
Top Features for Engagement on Twitter – Earth Hour 2014
neg pos
510
1520
2530
Length
neg pos
0.6
0.8
1.0
1.2
1.4
complexity
neg pos
−4−3
−2−1
01
23
polarity
neg pos
01
23
45
67
mentions
!
Fernandez, M., Cano, E., and Alani, H. Policing Engagement via Social Media. CityLabs workshop, SocInfo, Barcelona, 2014
Top Features for Engagement on Twitter – Dorset Police
• Top 4 features share 3 with Twitter Earth Hour dataset"
Publications about social media
by Katron Weller -‐ h+p://kwelle.files.wordpress.com/2014/04/figure1.jpg
Moving on … § How can we move on
from these (micro) studies?
§ Are results consistent across datasets, and platforms?
§ One way forward is: § Multiple platforms § Multiple topics
Papers studying single/multiple social media platforms
Survey done on all submi7ed papers to Web Science conferences
Papers studying single/multiple social media platforms
Survey done on all submi7ed papers to Web Science conferences
Papers studying single/multiple social media platforms
Survey done on all submi7ed papers to Web Science conferences
Papers studying single/multiple social media platforms
Survey done on all submi7ed papers to Web Science conferences
Apples and Oranges
• We mix and compare different datasets, topics, and platforms
• Aim is to test consistency and transferability of results
7 datasets from 5 platforms
Seed posts are those that receive a reply Non-seed posts are those with no replies
Pla1orm Posts Users Seeds Non-‐seeds Replies
Boards.ie 6,120,008 65,528 398,508 81,273 5,640,227
Twi+er Random 1,468,766 753,722 144,709 930,262 390,795
Twi+er (HaiR Earthquake)
65,022 45,238 1,835 60,686 2,501
Twi+er (Obama State of Union Address)
81,458 67,417 11,298 56,135 14,025
SAP 427,221 32,926 87,542 7,276 332,403
Server Fault 234,790 33,285 65,515 6,447 162,828
Facebook 118,432 4,745 15,296 8,123 95,013
Data Balancing Pla1orm Seeds Non-‐seeds Instance Count
Boards.ie 398,508 81,273 162,546
Twi+er Random 144,709 930,262 289,418
Twi+er (HaiR Earthquake)
1,835 60,686 3,670
Twi+er (Obama State of Union Address)
11,298 56,135 22,596
SAP 87,542 7,276 14,552
Server Fault 65,515 6,447 12,894
Facebook 15,296 8,123 16,246
Total 521,922
For each dataset, an equal number of seeds and non-seed posts are used in the analysis.
Classification Results
Feature P R F1
Social 0.592 0.591 0.591
Content 0.664 0.660 0.658
Social+Content 0.670 0.666 0.665
(Random) (HaiR Earthquake)
(Obama’s State Union Address)
P R F1
0.561 0.561 0.560
0.612 0.612 0.611
0.628 0.628 0.628
P R F1
0.968 0.966 0.966
0.752 0.747 0.747
0.974 0.973 0.973
Feature P R F1
Social 0.542 0.540 0.539
Content 0.650 0.642 0.639
Social+Content 0.656 0.649 0.646
P R F1
0.650 0.631 0.628
0.575 0.541 0.521
0.652 0.632 0.629
P R F1
0.528 0.380 0.319
0.626 0.380 0.275
0.568 0.407 0.359
Feature P R F1
Social 0.635 0.632 0.632
Content 0.641 0.641 0.641
Social+Content 0.660 0.660 0.660
§ Performance of the logisRc regression classifier trained over different feature sets and applied to the test set.
Effect of features on engagement Boards.ie
β
−2−1
012
Twitter Random
β
−0.50.00.51.0
Twitter Haiti
−6e+16−4e+16−2e+16
0e+002e+164e+166e+16
Twitter Union
β
−0.8−0.6−0.4−0.2
0.00.2
Server Fault
β
−1.0−0.5
0.00.51.01.52.0
SAP
β
−10
−5
0
5
β
−0.10.00.10.20.30.40.5
In−degreeOut−degreePost CountAge
Post RatePost LengthReferrals CountPolarity
ComplexityReadabilityReadability FogInformativeness
Logistic regression coefficients for each platform's features
Comparison to literature
§ How performance of our shared features compare to other studies on different datasets and platforms?
Positive impact Negative impact
Mismatch Match
Comparison to literature
Positive impact Negative impact
Mismatch Match
Comparison to literature
Let’s Share More Data!
Semantic Clustering • Statistical models play important roles in social data
analyses
• Keeping such models up to date often means regular, expensive, and time consuming retraining
• Semantic Features are likely to decay more slowly than lexical features
• Could adding semantics to the models extend their value and life expectancy?
Cano, E., He, Y., Alani, H. Stretching the Life of Twitter Classifiers with Time-Stamped Semantic Graphs. ISWC 2014, Trento, Italy.
Semantic Representation of a Tweet
<dbp:Barack_Obama>
American
dbprop:nationality
<skos:Nobel_Peace_Price_laureates>
dcterms:subject
<dbo:PresidentOfUnitedStateofAmerica>
rdf:type
<dbp:Hosni_Mubarak>
<skos:PresidentsOfEgypt>
<dbp:CNN>
<dbp:Egyptian_Arabic>
<skos:Arab_republics>
<dbp:Egypt>
<skos:English-language_television_stations>
dcterms:subject
dcterms:subject
dbprop:languages<dbp:Country>
rdf:type
rdf:type
Evolution of Semantics • Renewed DBpedia Graph snapshots are taken over time"• Semantic features updated based on new knowledge in
DBpedia"
v3.6 v3.7 v3.8
<Barack_Obama>
<Budget_Control_Act_of_2011>
<UnitedStatesPresidentialCandidates>
<Hawaii>
<MechelleObama>
spouse
birth1place
wikiPageWikiLink
wikiPageWikiLink
Experiments
Cross-‐Epoch
2010-‐2011 2010-‐2013 2011-‐2013 Average
F1 F1 F1 BoW 0.634 0.481 0.261 0.458
Category 0.683 0.539 0.524 0.582
Property 0.665 0.557 0.502 0.603
Resource 0.774 0.544 0.445 0.587
Class 0.691 0.665 0.669 0.675
Extending fitness of model to proceedings epochs
• 12,000 annotated tweets"• Adding Classes as clustering features provide best performance"
Same-‐epoch
2010-‐2010 2011-‐2011 Average
BoW 0.831 0.875 0.845
APPLICATIONS
1. "Fish where the fish is"– one interface to access multiple SNS"– layman monitoring of users and topics "
2. "My consistency first"– communicating with users in own
constituency"– find local groups, events, and topics"
3. "What are their needs, complaints, and preferences?"– what citizens talk about, complain about"– what are the top 5-10 topics of the day"
4. Who should I talk to?"– who are the influential citizens"– whom to engage with"
5. What about Tomorrow?"– which topics will get hotter?"– which discussions are likely to grow
further?"
6. Presence and popularity"– what writing recipe to follow to reach more
people"
7. Privacy"– concerns on citizens’ privacy when
extracting info"– concerns on their own privacy with 3rd
party SNS access tools"
What policymakers really want from Social Media?
Interviews with 31 policymakers
Wandhöfer, T.; Taylor, S.; Alani, H.; Zoshi, S.; Sizov, S.; et al. Engaging poliRcians with ciRzens on social networking sites: the WeGov Toolbox. IJEGR, 8(3), 2012
Demo
Monitoring SCN "Monitoring of evolution of community activities and level of contributions in SAP Community Networks – SCN "
SCN Behaviour
"Community managers can monitor behaviour composition of forums, and its association to activity evolution "
For Education
https://twitter.com/OpenUniversity/status/346911297704714240
FB Groups
Sentiment
Macro Behaviour
Micro Behaviour
Topics
Course tutors Behaviour Analysis
SenRment Analysis
Topic Analysis
Real Rme monitoring
• How acRve the engaged the course group is?
• How is senRment towards a course evolving?
• Are the leaders of the group providing posiRve/negaRve comments?
• What topics are emerging? • Is the group flourishing or
diminishing? • Do students get the answers
and support they need?
Thomas, K.; Fernández, M.; Brown, S., Alani, H. OUSocial2: a plaxorm for gathering students’ feedback from social media. (Demo) ISWC 2014, Trento, Italy.
DEMO
Thanks to colaborators
Thanks to ..
Gregoire Burel
Miriam Fernandez
Smitashree Choudhury
Hassan Saif Lara Piccolo Thomas Dickensen
Elizabeth Cano
Matthew Rowe
Keerthi Thomas
Sofia Angeletou
Heads-up
Semantic Patterns for Sentiment Analysis of Twitter Thursday 16:00 - Session: Social Media"
Semantic Patterns for Sentiment Analysis of Twitter Thursday 15.40 - Session: Social Media"
OUSocial2: a pla1orm for gathering students’ feedback from social media (DEMO) The Topics they are a-‐Changing — Characterising Topics with Time-‐Stamped Seman\c Graphs (POSTER)"!Automa\c Stopword Genera\on using Contextual Seman\cs for Sen\ment Analysis of Twi_er (POSTER)
User Profile Modeling in Online Communities !Sunday 2:05 pm - SWCS Workshop"
Top Related