Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter...

28
Drowning in Data or Swimming with Insight Matthieu Chouard KXEN - SVP Field Operations EMEA Laurent Tessier KXEN - Pre-Sales Manager EMEA May 29 th 2013

Transcript of Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter...

Page 1: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Drowning in Data or

Swimming with Insight

Matthieu Chouard KXEN - SVP Field Operations EMEA

Laurent Tessier KXEN - Pre-Sales Manager EMEA

May 29th 2013

Page 2: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Telecom Financial Services

Retail e-Business

500+ Leading Brands Use KXEN

Who is going to churn?

What’s the Next Best Action to take?

Who can we win back profitably?

What segments exist in my customers?

Who should I send a catalog to?

Which promotions should I push?

Which products should I recommend?

Which friends should I recommend?

Which songs/movies should I recommend?

Which accounts are dormant?

What’s the Next Best Offer to make?

How can we boost assets per household?

Page 3: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Analyst Leadership

“KXEN is a disruptor in the

predictive analytics market.”

Leader “KXEN’s focus on automating key modeling and

analytical tasks is a 'blessing’”

“KXEN customers build predictive

models 3x faster.”

Strong Performer “KXEN’s low-touch

approach to predictive will

boom in popularity.”

Vendor Rating

Customer Analytics Wave

Predictive Benchmark

Big Data Predictive Wave

Page 4: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Access to Data is Exploding

CRM ERP Billing

Profile Products

Purchase History

CDR Usage

Before 2010 (Transactions)

Now (Behaviors)

Web Mobile Social

Media

Call

Logs Email

Ad

Servers

Page 5: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Access to Data is Exploding C

RM

ER

P

Ca

mp

aig

n

Handcrafted

E

CRM ERP Billing

Profile Products

Purchase History

CDR Usage

Before 2010 (Transactions)

Now (Behaviors)

Web Mobile Social

Media

Call

Logs Email

Ad

Servers

Page 6: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Access to Data is Exploding C

RM

ER

P

Ca

mp

aig

n

Handcrafted

100’s of Derived Attributes

E

CRM ERP Billing

Profile Products

Purchase History

CDR Usage

Before 2010 (Transactions)

Now (Behaviors)

Web Mobile Social

Media

Call

Logs Email

Ad

Servers

Page 7: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Behavioral Data Adds Massive Insight C

RM

ER

P

Ca

mp

aig

n

Handcrafted

100’s of Derived Attributes

10,000’s of Potential Attributes

Big Data

E

CRM ERP Billing

Profile Products

Purchase History

CDR Usage

Before 2010 (Transactions)

Now (Behaviors)

Web Mobile Social

Media

Call

Logs Email

Ad

Servers

Page 8: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

20 Variables -Demographics / Account Information -Simple Aggregates (e.g. Account Balance, Total Usage)

The More (Data), The Better (Quality)

Page 9: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

The More (Data), The Better (Quality)

100 Variables -Pivoting Transactions (e.g. Calls by Type) -Time-Sensitive Aggregates (e.g. Calls by Week)

Page 10: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

The More (Data), The Better (Quality)

200 Variables -Social Network Analysis (e.g. Calls in First Circle) -Community Detection (e.g. Community Churn Rate)

Page 11: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Traditional approach does NOT work

Unless:

• You have an army of Data Scientists

• You have no time constraint

• You can plan all your actions in advance

• You don’t need to be proactive

• Your data is not volatile

Page 12: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Mass Production = Industrialisation & Automation

• Automated Dataset production

• Automated Data Quality handling

– Missing values, Outliers etc…

• Automated Data Preparation handling

– Grouping, Banding etc…

• One method

– Manufacturers do not use different processes

to build one item

– The best process is used to manufacture

each item

• In-database Production

• Control over the whole process

– Robustness

– Quality control over time

15 years

of R&D

Page 13: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

KXEN InfiniteInsight® architecture

Model transformed in SQL Code

Dataset automation

SQL Code

Automated Dataset Production Text coding

Scoring Regression

Segmentation Forecasting

Model in SQL code for In-database production

Other languages for real-time purposes

Link Analysis Influencers

Communities

Product recommandations

Productionization Automated control &

maintenance

To real time

environments

Page 14: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Faster Models and Better Models

Page 15: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Data Warehouse Web & Social Media

Next Big Thing?

Total Campaigns

Easy Productive Big Data

Time to Market

We needed a solution that would allow us to

scale with the hyper growth of our business.

KXEN innovation makes it possible

- Rafał Kudliński

BI Director

Page 16: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

KXEN Helps Drive a Smarter Obama Campaign

Needed speed to scale to huge and dynamic

online and offline data

Allowed OFA to run campaigns through email

and social media sites (FB, Twitter)

Optimized fundraising campaigns for higher

contributions

Identified segments of voters who were

persuadable

- Rayid Ghani

Chief Scientist at Obama for America 2012

Obama For America (OFA) Campaign

The timely and accurate insights provided by KXEN

led to more effective and quicker targeting and in the

end, more votes.

Page 17: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

The Goal: Target Voters and Volunteers

TV Advertising

Fundraising emails

Visits of Volunteers

Social Campaign

Page 18: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

A Strategy Based on Big Data Analytics

• Set up infrastructure to collect relevant big data – From multiple sources (email campaigns, Facebook,

Twitter, vote history, socio-demographics…)

– With massive volumes

• Analyze and obtain answers to key questions – Who’s likely to vote? Who’s likely to vote for Obama?

– Who’s likely to turn out?

– Who influences who on social channels?

Page 19: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Boosting Fundraising & Recruitment Campaigns

• Fundraising and promotion campaigns

– Determine who is most likely to donate funds

– Determine the right message to send, at the right time and through

the right channel

– Financing of Adverts and of overall marketing activities

• Volunteer engagement campaigns

– Determine who can be engaged for field campaigns

• Visits, call center, adverts distribution…

Page 20: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Vote Influence Campaigns

• What type of campaign to run on social media?

• Who has influence on Facebook

– When you share content on Facebook, your followers click on it

Page 21: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

KXEN’s Public Press Release

KXEN’s Predictive Analytics Helps Drive a Smarter

Obama Campaign

Predictive insight on “big data” gives campaign

strategic advantage

Data scientists use agile predictive analytics to

optimize email and social media interactions

http://www.kxen.com/News+and+Events/Press+

and+News/Press/2013-01-29-OBAMA

Page 22: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Who is Allegro?

• The biggest non eBay Marketplace platform

worldwide, leader in Eastern Europe

• The Allegro Group operates 75 websites in 14

countries across Eastern Europe:

– 15 Millions Buyers

– 30K Professional Sellers

– 15 Millions Offers every day

– 20 Millions unique website visitors

– 15 Millions listed products

– 100 Millions request per day

– 500 Millions daily page views

Page 23: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Challenges

• Low click-through (CTR) and conversion rates of

generalized “bestseller” rules.

• Need to increase the income and gross

merchandise volume (GMV) through cross-selling

and up-selling

• Offer personalized product recommendations to its

20+ million distinct website visitors.

• None of existing solutions on the market meet

requirements for a new recommendation system

because of data volume and versatility

Page 24: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Solutions: KXEN’s InfiniteInsight®

• Personalized product recommendations

– foundation of the company’s website

– Using links analysis between products and creating

weightings based on:

• visitor click paths

• items placed in shopping carts

• purchase transactions

• Propensity models for newsletters

Page 25: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Solutions: KXEN’s InfiniteInsight®

• Allegro chose KXEN’s InfiniteInsight® for:

– its ability to analyze huge volumes of online data,

– its ease of use

– its flexibility to adapt to unique website requirements

– Its capacity to be integrated with:

• Oracle Exadata

• Neolane’s campaign management system

Page 26: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Allegro Recommendation Architecture

Production Database

Analytical Datamart

Data Replication

Loading Transactions

Writing Rules

As tables Application of Business

rules

Reco Tables

Or Precalculated

Reco

Website Apps

Data Collection

Querying of

Reco tables

Recommendation display performance is defined by production system

performance

Recommendation design is an offline process run when required i.e. daily in this context

Page 27: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Results:

• Boosted e-Commerce KPI’s:

– Page views per visit: up 30%

– Click-through rates: up 500% (from 0,54% to 3,09%)

– Conversion rates: up 40x (compared to “bestseller” rules)

• 80Millions/day personalized product recommendations

• 60,000 rules/week from in average:

– 120,000 product links (shoppers that “watched” similar products)

– 45,000 purchase links (shoppers that purchased similar products).

• Response time < 200ms

Page 28: Drowning in Data or Swimming with Insight BigData - Allegro Obama.pdf · KXEN Helps Drive a Smarter Obama Campaign Needed speed to scale to huge and dynamic online and offline data

Demo