STR-MT-IJCAI2011

IJCAI, Industry DayBarcelona :: July 22, 2011

Top 10 Lessons learnedDeveloping, Deploying and Operating

Real-World Recommender Systems

Marc TorrensChief Innovation Officer

Sunday, July 17, 2011


Agenda

2

About Strands

What is a Recommender?

The Business Perspective

Top Lessons Learned



Agenda

3

About Strands



Top Lessons Learned



About Strands

4

Year 2003...

Provide Recommendations in the Music Space

• implicit preferences!

What to play?What to synch?



Strands develops technologies to better understand people’s taste and help them discover things they like and didn’t know about.

About Strands

5

2003 2004 2005 2006 2007 2008 2009 2010

music

peoplevideosmusic

Strands RecommenderStrands FitnessStrands Finance

Same mission evolving through different domains

2011

RecSys’06summer school

RecSys’07Minneapolis

RecSys’08Lausanne

RecSys’09NY

RecSys’10Barcelona

RecSys’11Chicago



About Strands

6

Understanding consumer habits at the commerce

(transaction) level

Understanding consumer preferences in real-world activities

Understanding consumer behavior at

the product level

Highly-targeted Product Placement



About Strands

7



About Strands

8

• BBVA, Spain• ING, Netherlands• BMO, Canada



About Strands

9



About Strands

10



Agenda

11

About Strands



Top Lessons Learned



Why Personalize?

12

The Paradox of Choice by Barry Schwartz



Personalization is great ...but

13

The Filter Bubble by Eli Pariser



Personalization is great ...but

13

The Filter Bubble by Eli Pariser

IMPORTANT

UNCOMFORTABLE

CHALLENGING

OTHER POINTS OF VIEWS




14

A Recommender selects the product that if acquired by the buyer maximizes value of both

buyer and seller at a given point in time




15

A Recommender processes information and transforms it into actionable knowledge




15


Recommender Components




15


Knowledge Base





15


Knowledge Processing Application

Knowledge Base





15


Business Control

& Analytics


Knowledge Base





15


UserInterface

Business Control

& Analytics


Knowledge Base





15


UserInterface

Business Control

& Analytics


Knowledge Base


25%25%25%25%




15


UserInterface

It has a certain level of autonomy presenting recommendations to the end user

Business Control

& Analytics


Knowledge Base


25%25%25%25%



Agenda

16

About Strands



Top Lessons Learned




17




17

1. Do I need a recommender?




17

1. Do I need a recommender?2. How do I get one?




17

1. Do I need a recommender?2. How do I get one?3. Is it performing well?




17

1. Do I need a recommender?2. How do I get one?3. Is it performing well?4. Was it a good idea after all?



Agenda

18

About Strands



Top Lessons Learned



LESSON 1

Make sure it is needed.

19



LESSON 1Make sure it is needed.

20




21




22

X

X

X X X

000s in both products and customers

low medium high

low

med

ium

high

Div

ersi

ty o

f the

Cat

alog

Diversity of the Customers

OK




23

low

med

ium

high

ROI

random top 10 recommender sophisticatedrecommender

even moresophisticatedrecommender




23

low

med

ium

high

ROI

showing this impact is already challenging!

random top 10 recommender sophisticatedrecommender

even moresophisticatedrecommender



LESSON 2

It must make “strategic” sense.

24



LESSON 2It must make “strategic” sense.

25

Is the best recommendation for the customer the best recommendation for the business?




26

• Relevant vs Useful• Correctness is often too obvious to be useful• Riskier recommendations have less chances of being known

• customer perspective

• business perspective• Short-term gain vs long-term return




27

UserInterface

Business Control

& Analytics


Knowledge Base

? %? %




27

UserInterface

Business Control

& Analytics


Knowledge Base

? %? %

• How much business logic goes into Recommender?




27

UserInterface

Business Control

& Analytics


Knowledge Base

? %? %


• What’s the right level of autonomy a recommender must have?




27

UserInterface

Business Control

& Analytics


Knowledge Base

? %? %


• What’s the right level of autonomy a recommender must have?

• How can the business control recommendations?




28

1. Do I need a recommender?2. How do I get one?3. Is it performing well?4. Was it a good idea after all?



LESSON 3

Choose the right partner.

29



LESSON 3Choose the right partner.

30




30

Small company Select a vendor




30


Medium company Hire a copule of PhD students!




30



Large company Partner with an experienced vendor




30



Large company Partner with an experienced vendor

Tech company Organize a contest!




31

2000 2002 2003 2004 2006 2007 2008 20092001 20102005




31

2000 2002 2003 2004 2006 2007 2008 20092001 20102005

000s licensing + 000s integration

Price

$99 / month




31

2000 2002 2003 2004 2006 2007 2008 20092001 20102005

Vendors

hundreds vendors

few vendors

000s licensing + 000s integration

Price

$99 / month



LESSON 4

Cold start? Be creative!

32



LESSON 4Cold start? Be creative!

33

With the advent of the Internet the start for a Recommender isn’t so cold anymore



LESSON 5

Data and algorithms.

34



LESSON 5Data and Algorithms.

35

Which really makes the difference?Ingredients or Receipe?



LESSON 5Data and Algorithms.

36

X Xbad good

bad

good

Dat

a Q

uant

ity a

nd Q

ualit

y

Algorithm Performance

OK



LESSON 6

Finding correlated items is easy, deciding what, how, and when to present to the user is hard.

37



LESSON 6Finding correlated items is easy,

deciding what, how, and when to present to the user is hard.

38

Math Art



LESSON 7

Don’t wast time calculating nearest neighbors.

39



LESSON 7Don’t waste time calculating nearest neighbours.

40

Let people tell you!



LESSON 8

Don’t wait too long to get ready to scale.

41



LESSON 8Don’t wait too long to get ready to scale.

42

When is the right moment?• if you do too soon and recommendations don’t take off...• if you do too late and recommendations do take off...



LESSON 9

Choose the right feedback mechanism.

43



LESSON 9Choose the right feedback mechanism.

44




45




46




47

• Implicit Ratings vs Explicit Ratings

• Implicit Semantics vs Explicit Semantic of Ratings




48

The Ideal (explicit) Rating system...

feedback

Good Badso-so*

*optional at it may help to confirm some implicit actions.

actions

I have it (i knew it, i saw it)

Don’t show it any more

Show it to me later

has it (knew it, saw it)my friend




49



LESSON 10

Measure everything.

50



LESSON 10Measure everything.

514th ACM Conference on Recommender SystemsBarcelona :: September 30, 2010



52

LESSON 10Measure everything.



So, what have we learned?

53

UserInterface

Business Control

& Analytics


Knowledge Base





53

UserInterface

Business Control

& Analytics


Knowledge Base


50%




53

UserInterface

Business Control

& Analytics


Knowledge Base


50%20%




53

UserInterface

Business Control

& Analytics


Knowledge Base


50%20%5%




53

UserInterface

Business Control

& Analytics


Knowledge Base


50%20%5%25%



Questions?

54

Thank you!


STR-MT-IJCAI2011

Documents

Transcript of STR-MT-IJCAI2011