Cybernetics, human-in-the-loop and probabilistic modelling for recommender systems

Eliezer de Souza da SilvaPhD candidate, Department of Computer Science, NTNUhttps://eliezersilva.blog/

Talk presented at BRAIN NTNU eventhttps://brainntnu.no/portfolio/brain-talks-big-data2-2/

2019

2

Initial thoughts

“A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.”

― Robert A. Heinlein

3

Initial thoughts• Plan• Reason• Execute• Evaluate• Control• Learn• Communicate with language• Sense• Creativity• Feel• Capacity for love/empathy• Social life

4

Initial thoughts

5

Cybernetics

“It is my thesis that the physical functioning of the living individual and the operation of some of the newer communication machines are precisely parallel in their analogous attempts to control entropy through feedback.”

Norbert Wiener

6

What about a machine?

• Idea 1: substitute human labour• Idea 2: extend human capabilities• Not necessarily mutually exclusive• Extending modes of acting over the

environment

7

Model of possible interactions

Environment

8

Model of possible interactions

Environment

Multiple Feedback Loops

9

Model of possible interactions

Environment

Multiple Feedback Loops

Low Hanging fruit: model each possible interactions / feedback loop

● Language● Image● Movements● Sounds● Text● Spatio-temporal

dynamics

● Prediction● Learning ● Control

10

The (success) story so far:● Classification with lots of data and labels

○ Vision○ Text / language

● Combining simulation and learning:○ Recent advances of AlphaZero and AlphaGo

● Representation learning with lots of data:○ Machine Translation and word embedding

● Computational efficient inference / modelling○ Variational inference, stochastic gradient

descent, probabilistic programming

11

Road ahead:

● Learning with little data and little amount of meta-information / labels○ Semi-supervised learning

● Fusion and multiple interactions○ Transfer learning○ Relationship learning○ Language(s) for logic + statistical learning

● Better use of causal / counterfactual models○ Combining probabilistic & neural models

with internal state and world simulation

12

Cross-fertilization with neuroscienceKarl Friston: Free energy principle

13

Some examples in recommender system research

14

Basic setup● Prediction of unseen entries: recommendation of items to users given

user interaction with some items

User 1 User 2 User 3

Item 1 Item 2 Item 3 Item 4

15

Poisson Matrix Factorization

16

Joint modelling of user social network and item topic content● User social network

○ Homophily○ Item exposure positively influenced by peers

(positive “peer-pressure”)● Item content analysis

○ Enrich items latent factors with topic model○ Cold start items○ Preferences can be influenced by topics

17

User 1 User 2 User 3

Item 1 Item 2 Item 3 Item 4 Item 5

Topics

User 4

User Social

Network

Adding context via shared factors

18

[Topics, Words]

[Topics, Users]

[Items, Topics]

[Items, Words]

[Items, Users]

Observed

Latent

Content-based social Poisson factorization

Content-Based Social Recommendation with Poisson Matrix Factorization. Eliezer de Souza da Silva; Helge Langseth; Heri Ramampiaro. ECML-PKDD 2017.

19

Item Recommendations

• Top-M items for each user:– Approximate expected value of user-item matrix for each

unseen item for ranking

Rud User preferences

Shared item topic

intensity

Item topic offset

Weighted sum of social network neighbors interactions with item

20

Results

PoissonMF-CS (K =10) and Gaussian-based models

PoissonMF-CS (K =10) and other Poisson factorization models

21

Hierarchical RNN and (time) Point process for multi-session rec.

Time is of the essence: A joint Hierarchical RNN and Point Process model for time and item predictions. Bjørnar Vassøy; Massimiliano Ruocco; Eliezer de Souza da Silva; Erlend Aune.. WSDM 2019

22

Results

23

Beyond prediction: feedback loops and bias in recommendation

How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility. Chaney et al. RecSys 2018

24

Recommendation as Reinforcement Learning

DRN: A Deep Reinforcement Learning Framework for News Recommendation. Zheng et al. WWW2018.

Master thesis at Norwegian Open AI Lab:

- Olav Nymoan- Massimiliano Ruocco

25

A society of human-machine cooperation?

● Internal simulation of human and other AI agents states

○ Theory of mind● Counterfactual reasoning● System 2 type of thinking and

reasoning● Empathy / emotional

understanding● Explicit connection of behavior,

purpose and teleology (Wiener et al)

○ Bias is present even in purpose-blind systems

○ AI Risks● Human-centric engineering

(Michael I. Jordan)○ statistical and

computational thinking and modelling for automatic decision-making

Augmenting Human Capabilities to New Dimensions. Harri Valpola (The Curious AI Company), talk at Slush 2017

26

Conclusion

• Model including different sources of information are promising for designing new algorithms

• Computational efficient approximate inference is essential for scalable models

• Work with flexible probabilistic programming languages• Look ahead with a map of possible loops of

human-machine-environment interactions is a good strategy for advancing the area

27

Nordic Probabilistic AI Summer School 2019• 1st week of June• Focus on probabilistic programming (Pyro), variational

inference methods (classic and modern variations), and deep generative networks.

• Lectures and tutorials• http://www.probabilistic.ai/• https://www.facebook.com/probabilisticai

28

Questions?

@zeh_silva