Carrots for Couch PotatoesImproving recommendations by
motivating the crowd
@fabianabel
Definition 1“Recommender system = black box that knows the answer to the ultimate question…of life, the universe and everything.”
Hypothesis 1 “The more obscure a recommender system, the higher the chance that its users are happy with the system.”
Definition 2.1 “Data Scientist = folks that can program the smartest recommender systems.”
Hypothesis 2 “Nobody needs an interaction designer.”
Definition 2.2 “Interaction Designer = folks that think about what users actually want to do.”
Definition 3 “Couch potatoes = users who do not provide input to a recommender system, but have high expectations towards the quality of the system.”
Hypothesis 3 “The quality of a recommender system increases with the number of couch potatoes that are *using* the system.”
Goals of Recommender Systems
Make users happy and surprise them with new and relevant content.
[user perspective]
Deliver content so that monetary success of the business is maximized.
[business perspective]
Problem space
Challenges
• Understanding the users
• Understanding the items
• Coding a good (ensemble of) recommendation algorithm(s)
• Evaluation
• Presentation of recommendations
• …
recommender
system
users
items
recommender system
Example from xing.com
Delete item
Hide entire box
(1) Less-Like-This(2) Collaborative
filtering
deletions?
Clicks + bookmarks
(1) More-Like-This(2) Collaborative
filtering
positive feedback
interactions exploited by…
negative feedback
AB test* resultsC
TR
Control group
Group with Less-Like-This
filtering
-3%
?
*AB tests on XING- are done in front-end and back-end
components- typically 50:50 random splits (others:
specific groups; inter-leaving)- Run for days to weeks significance
level: p-value < 0.01- Validation includes AA comparison,
BA/BA test, repeating AB test
More cookies!
People used “delete” to get more recommendations.
Hypothesis 2 is wrong!
Hypothesis 2 “Nobody needs an interaction designer.”
How can we collect more valuable explicit feedback from
our couch potatoe users?
Related Work
1. First explicit feedback is collected right from the beginning during on-boarding, e.g.: select 3 favorites rate 10 items (5-star rating scale)
2. Continuous collection of explicit feedback user control, e.g.: ratings (lightweight) revising ratings, taste preference questionnaire (advanced)
3. Understanding why a user liked or disliked an item, e.g.: emphasizing topics blocking topics
Explicit feedback is key!
1. From the beginning2. Continuously 3. Understanding why…
Hypothesis 3 is wrong!
Hypothesis 3 “The quality of a recommender system increases with the number of couch potatoes that are *using* the system.”
We need to motivate our couch potatoes to contribute to improve
our recommender sytems!
Feedback app
Only means of motivation:1. Promise: “this will enhance your recommendations”2. Progress bar
Feedback App
Pros• Continuous stream of
explicit feedback
• Decoupled from the actual system
Cons• Attracts “Haters” more than
fans
• Decoupled from the actual system
In addition, we also want feedback mechanisms that are more integrated into the natural interaction flow of the system.
explanations feedback
Is this job for me?
Is Hypothesis 1 wrong as well?
Hypothesis 1 “The more obscure a recommender system, the higher the chance that its users are happy with the system.”
Conclusions
Thank you!
@fabianabel