VoltDB and Flytxt Present: Building a Single Technology Platform for Real-Time and Iterative...

page

VOLTDB AND FLYTXT PRESENT: BUILDING A SINGLE TECHNOLOGY PLATFORM FOR REAL-TIME AND ITERATIVE ANALYTICS ON FAST + BIG DATA

page© 2015 VoltDB PROPRIETARY

OUR SPEAKERS

Ryan Betts CTO at VoltDB

2

Prateek Kapadia CTO at Flytxt


VOLTDB OVERVIEW

Mike Stonebraker

FASTWorld Record Cloud Benchmark:

YCSB (Yahoo Cloud Serving Benchmark) - 2.4 million tps (transactions per second)

Other Stonebraker Companies

Customers

3

Technology•  In-Memory (but data is durable to disk)•  Scale-Out shared-nothing architecture•  Reliability and fault tolerance

•  SQL + Java with ACID•  Hadoop and data warehouse integration•  Open source and commercially licensed (24X7)

Founded by winner of the 2014 ACM Turing Award


Collect Explore

AnalyzeAct

4

Big Data analytic results:

1.  Discoveries: seasonal predictions, scientific results, long-term capacity planning

2.  Op.miza.ons: market segmentation, fraud heuristics, optimal customer journey


FAST DATA – BIG DATA

5

Fast Data Pipeline

Ingest Export

Big Data

Real-Time Analytics & Decisions

Fast Data: the velocity side of Big Data

Milliseconds


DATA ARCHITECTURE FOR FAST + BIG DATA

Enterprise Apps

ETL

CRM ERP Etc.

Data Lake (HDFS, etc.)

BIG DATA SQL on Hadoop

Map Reduce

Exploratory Analytics

BI Reporting

Fast Operational Database

FAST DATA

Export Ingest / Interactive

Real-time Analytics

Fast Serve Analytics

Decisioning

6


7

“89% of marketers surveyed plan to

compete primarily on the basis of customer experience by 2016.”

Source: Gartner 2014 survey, Companies > $50M in revenue


FAST DATA SOURCES AND DRIVERS

Mobile

IoT

Social

Sensors

Logs

Data is doubling every two years

•  26 billion connected devices by 2020 (Gartner 2014)

•  37% of most data will be processed at the edge in milliseconds (Cisco IoT Study 12/11/14)

Mobile

IoT

8


THE FAST DATA PIPELINE

9

Calculations Serving of Results

Real Time, Per Event, Interactive


STREAMING: REAL TIME ANALYTICS

•  Operational analytics and monitoring

•  RT analytics enabling user-facing applications

•  KPI for internal BI/Dashboards

10


STREAMING OPERATORS NEED STATE

Require State

•  Filter

•  Join

•  Aggregate

•  Group By

Stateless

•  Partition

11


REAL-TIME ANALYTICS

12

Database

Metadata (Dimension table)

Session state (Fact table) •  Operational analytics and

monitoring

•  RT analytics enabling user-facing applications

•  KPI for internal BI/Dashboards

•  In-memory MPP SQL over ODBC/JDBC

•  Cheap + correct materialized views for streaming aggregations

SQL, Views

Ingest


INTEGRATING WITH EXPORT TARGETS

13

•  Local file system export•  JDBC export•  Kafka export•  RabbitMQ export•  HDFS export•  HTTP export•  Extensible API


EXPORT FORMATS

•  CSV

•  TSV

•  Avro container

•  Raw data


DATA PIPELINES WITH EXPORT

15

Database

Metadata (Dimension table)

Session state (Fact table)

•  Filtering (ex: only RFID / iBeacon readings that show change from previous location).

•  Sessionization

•  Common version re-writing

•  Data enrichment

•  MPP streaming Export

•  Row data, Thrift messages, CSV

•  OLAP, HDFS and message queues

Export

page© 2015 VoltDB PROPRIETARY page

FLYTXT AND VOLTDB

16


FLYTXT OVERVIEW

17

}  Vision: Create >10% measurable economic value for Communica8on Services Providers through Big Data Analy8cs

}  Flytxt’s internal and external mone8za8on solu8ons increase revenue, reduce churn and improve customer experience

}  Dutch company with corporate office in Dubai, global delivery centres in India and regional presence in Mexico City, Johannesburg, Singapore, Dhaka and Nairobi.

Partners Operators Customers and Partners

Brands

Sample text

Awards & Achievements Vision, Mission & Impact


FLYTXT’S INTEGRATED ANALYTICS SOLUTION ARCHITECTURE: BIG DATA, ITERATIVE AND REAL-TIME ANALYTICS

Files and BI outputs

Event Filter Trigger Detector

Real 2me Trigger Engine (RTE)

Processed Events

Con2nuous Insight Engine (CIE)

Data Fusion Engine

Batch Analy2cs

Triggered Rules

Scheduled Rules

Rendering Engine

Persistence Store

KPIs, Insights, Recommenda2ons, Ac2ons & Rules

Hadoop 2

Hadoop 2, Hbase Jboss, Apache, SMPP sim, Tomcat, Ext JS, Hornet Q

Hadoop 2, Spark, Pig, Hive, Mahout, COIN-‐OR, Weka, MLlib

VoltDB, Drools

Response/ Input

Capture Network / BSS / OSS

GUI

N/W Integra2on

Network / BSS / OSS

Chan

nels

Ope

rator

Subscribers

Triggers

Lookup Subscriber Insights

Scan Subscriber Insights

Itera2ve Analy2cs

2

Streaming Data


BIG DATA ANALYTICS USE CASE: BEST FIT PRODUCT RECOMMENDATION

19

Profile, Usage

Data Fusion Engine

Batch Analy2cs

Persistence Store

Hadoop 2


Best Fit Product Recommenda2on

Usage Product U8lity

Business Constraints Model

Ranking

Context 1

Context 2

Context 3

Recommended Offers P1 P2 P3

P3 P2 P1

P5 P4 P7

Ranking based on Similari8es

Objec8ve: Recommend best fit product to subscribers based on usage and business objec8ves

Recommended Offers Recommended Offers

Hadoop 2, Hbase


0

5

10

15

20

Offer-‐1 Offer-‐2 Offer-‐3 Offer-‐4 Offer-‐5

Conversion

Rate in Ju

l-‐2014

in %)

0

5

10

15

20

25

Conversion

Rate in

Jul-‐2

014 (in

%)

Circles Rule-‐based campaign Best-‐fit recommenda8on (fair)

CASE STUDY: CONTEXTUAL PRODUCT RECOMMENDATION FOR TIER 1 OPERATOR

Recommenda2on Personas: CLV (HVC, MVC, LVC), Vola8le, Early Adopter, Frequent Handset Changer, Heavy Data user, Social Media Fan, Bollywood Fan, Music Fan, Sports Fan, poten8al iPad buyer, Interna8onal Caller Etc……. Objec2ves: Cross sell, Upsell, S8mulate recharge/usage/Service adop8on Etc… Offers: Data Plan, 3G plan, VAS usage, Interna8onal Calling packs, Bundle offers, Recharge s8mula8on, Seeding, ebill subscrip8on etc…… Channels: IVR, In store, Retailer, WAP portal, Customer care portal


ITERATIVE ANALYTICS USE CASE: MICRO-SEGMENTATION

21


Data Fusion Engine

Itera2ve Analy2cs

Persistence Store

Spark, MLlib


Micro-‐segmented Offers

Gaussian Mixture Model

Segment Offers

Segment Offers Segment Offers

Soi Clustering

S1 P3

S2 P8

S4 P9

Clustering to enable micro segmentation for

personalized offers

Hard Clustering

Soft Clustering

Hadoop 2, Hbase

page© 2015 VoltDB PROPRIETARY 22

USE CASE: REAL-TIME ANALYTICS SUPPLEMENTED BY BIG DATA, ITERATIVE ANALYTICS


Event Filter Trigger Detector

Processed Events


Data Fusion Engine

Analy2cs Engine

Triggered Rules

Scheduled Rules

KPIs, Insights, Recommenda2ons, Ac2ons & Rules

Hadoop 2

Hadoop 2, Hbase

Hadoop 2, Spark, Pig, Hive, Mahout, COIN-‐OR, Weka, MLlib

VoltDB, Drools

Itera2ve Analy2cs

Persistence Store

Big Data+ Itera2ve analy2cs

Real-‐2me analy2cs

Real-‐8mes Ac8ons

Balance Threshold

Recharge

Usage

Network

Behaviou

r

Preferen

ce

Recommen

da8o

n

Contextual Needs

Streaming Data


CASE STUDY: REAL-TIME TRIGGER BASED MICRO-SEGMENTED OFFERS

23

Drop in overall ARPU

Drop in data usage AR

PU Slabs

VAS

SMS

Voice usage

DATA

Outgoing

Incoming Long term inac2vity

Short term inac2vity

Product Affinity

Local

Long distance

Drop in O/G MoU

Drop in balance

Leg-‐wise Usage

Segments

Usage Behaviour

Client Objec2ve •  Improve customer

engagement for ARPU enhancement

Solu2on

•  Marke8ng Program based on usage behaviour driven micro-‐segmenta8on and tripwire monitoring

Data Analyzed

•  Customer usage history, ARPU charts, spend palerns and preferences

Impact •  2% increase in month-‐on-‐

month revenue •  28% higher revenues &

MOU


QUESTIONS?

•  Use the chat window to type in your questions or hashtag #VoltDBFlytxt

•  Know more about Flytxt•  Visit www.Flytxt.com

•  Try VoltDB yourself:Ø  Free trial of the Enterprise Edition:

•  www.voltdb.com/download

Ø  Try VoltDB in the CloudØ  Amazon’s Cloud Formation

Ø  Open source version is available on github.com

24

VoltDB and Flytxt Present: Building a Single Technology Platform for Real-Time and Iterative...

Documents

Transcript of VoltDB and Flytxt Present: Building a Single Technology Platform for Real-Time and Iterative...