+
Did you get it? Sanath Bhandary Akhil Bhiwal
#Self Organization
in Humans
#Crowd Sourcing
#Game With a
Purpose
#Human
Computation
#Computational Linguistics
+Self Organization in Humans
n As in natural phenomenon like ants, people too organize themselves into patterns to get tasks done.
+Self Organization in Humans
n As in natural phenomenon like ants, people too organize themselves into patterns to get tasks done.
+
Motivation
+Self Organization in Humans
n Millions of people play games and spent countless hours for entertainment.
n To give you an example:
Time spent in 1 year > 9 Billion Time took to built this = 7 million hours
+Self Organization in Humans
n Millions of people play games and spent countless hours for entertainment.
n To give you an example:
Time spent in 1 year > 9 Billion Time took to built this = 20 million hours
+
n Could we use this human time to do something useful, while people still get entertained.
n We think, yes!
n So, we designed a game which people play purely for entertainment.
n As a side effect of their playing this game, they solve a problem which computers currently can’t do easily.
Self Organization in Humans
+Problems hard for computer – Easy for Humans
n Machine translation.
n Identifying objects in given image (Computer vision).
n Detecting sarcasm in given text.
n And many more…
+Detecting Sarcasm in Text
n Sarcasm transforms the polarity of an apparently positive or negative utterance into its opposite.
n Why it’s difficult: Some of the best approaches for computational linguistics relies on machine learning techniques which require large dataset. However, the currently available datasets are small so limits training of algorithms.
n Our aim in this project is to construct a corpus of text for computational linguistics researcher to train their existing algorithms or create more accurate computer linguistic algorithms.
+Did you get it? Game Rules
n Web-based multiplayer game
n 2 users to play a single instance of game.
n Players have limited communication with each other and can’t know their partner’s identity.
n Each user gets a small paragraph (at max 4 sentences).
n User categorizes it as positive, negative or sarcastic. User can also give two words which she finds useful to identify that. Each matching word with other partner fetches bonus score.
+
Username:
Login
+
Did you get it?
Play
Pack 1 1-5 Points
Pack 3 10-15 Points
Pack 2 5-10 Points
Pack 4 15-25 Points
Your Score 480
Highest Score 5150
+
Did you get it?
Words
Sarcastic
Positive Negative
Pass Submit
Sometimes I need what only you can provide: your absence.
Time Left 1:30
Score 155
This text is:
Key
Points: 4
+
Did you get it?
Words
Sarcastic
Positive Negative
Pass Submit
Marriage is the chief cause of divorce!
Time Left 0:22
Score 985
This text is:
Key
Points: 6
+
Did you get it?
Words
Sarcastic
Positive Negative
Pass Submit
The 100% American is 99% idiot.
Time Left 2:55
Score 65
100% 99%
This text is:
Key
Points: 5
+Did you get it? Game Rules: Reiteration
n Web-based multiplayer game
n 2 users to play a single instance of game.
n Players have limited communication with each other and can’t know their partner’s identity.
n Each user gets a small paragraph (at max 4 sentences).
n User categorizes it as positive, negative or sarcastic. User can also give two words which she finds useful to identify that. Each matching word with other partner fetches bonus score.
+The failure points!
n Language apart from English.
+
Did you get it?
Words
Sarcastic
Positive Negative
Pass Submit
Por favor, tráeme un vaso de agua.
Time Left 9:58
Score 0
This text is:
Key
Points: 23
+The failure points!
n Language apart from English.
n Multiple recurrence of same word.
+
Did you get it?
Words
Sarcastic
Positive Negative
Pass Submit
Where she sits she shines, and where she shines she sits.
Time Left 5:15
Score 480
This text is:
Key
Points: 25
+How it works?
n User starts with a initial score of zero.
n User chooses the pack he wants to play with. Pack 1 being the easiest and Pack 4 being the most difficult.
n Each time user agrees with his/her partner on a particular text, their score increases by points mentioned below the text.
n If one user clicks on Pass, other user also have to do the same. He can’t choose any other option until he passes that question.
+How it works?
n Bonus Score: n Let’s say the score of given text is X points.
n If players have submitted one key word same, both gets a bonus score of X.
n If players submitted both key words same, they get a bonus score of 3X.
n Key words are useful metadata to train computational linguistics algorithm.
+
Did you get it?
Words
Sarcastic
Positive Negative
Pass Submit
The 100% American is 99% idiot.
Time Left 2:55
Score 65
100% 99%
This text is:
Key
Points: 5
+How it works?
n Dataset: n Twitter dataset with userid (@userid) and hashtag (#hashtag)
removed.
n In practice, any sort of data can be used. Eg. Product reviews, opinions, etc.
n Implementation: n JavaScript and Nodejs (>90%) + Python (<10%)
n Mongodb for storing data
n Platform: Heroku
+How it works?
n When N players agrees that a specific text to be of positive/negative/sarcastic type, we tag that text with specified type.
n When users submit result for specific text, the points associated with text gets recalculated by: n Points = Points + Pheromone * weighted sum
n When users agree, pheromone = 1 else pheromone = -1.
n Weighted sum adjusts the points of the text.
In our game:
N = 10
Weighted sum = 0.02
+Self Organization of Players
n Task: Constructing corpus for computational linguistics researchers
n Agents: Players
n Program: Incentive structure – Points and Entertainment
n Patterns formation: Clusters around different difficulty packs
n Communication: Pheromone based model
+Self Organization of Players
+
n Inspired from Pheromone based Model
n Pheromones are values that are used to alter the points associated with each sentence on the basis of the following formula
n Pt = Pt-1 + (pheromone intensity)*W Pt : Points associated with sentence at time t Pt-1 : Points associated with sentence at time t-1 Pheromone intensity =1 when players disagree Pheromone intensity =-1 when players agree W : Weighted constant 0<W<1
Communication Scheme
+Agents
n Users are synonymously referred as agents
n Agents have two operations : sense_sentence_score(sentence_id) modify_sentence_score(sentence_id)
+Clustering
n Driven by incentive model
+Self Organization: Different Perspective
Robot Based Model Human Based Model
Task Moving a rock Creating metadata for ML algorithms
Agent Robots Humans
Program Set of rules Incentive structure (more points or entertainment or both)
Communication mechanism
Electromagnetic radiation Pheromone deposit
Patterns Local interactions causing global behavior
People forming clusters
Fault-Tolerant Yes Yes. System is independent of agents
Dependence on Initial Conditions
It depends Yes
+Conclusion
n Play game.
n Better still, collaborate with us to further develop this game. Email us for accessing code via GitHub.
n When robots become dominant, they will still need humans. So, our species is safe.
+Thank you!
Top Related