CURIO:

A Mobile Marketing Platform

Ülker ÖZGEN ÇİFTÇİ

TURKCELL

About Turkcell

9 COUNTRIES, 71.3 MILLION MOBILE SUBSCRIBERS

EUROPE’S SECOND LARGEST OPERATOR

400 DEVELOPERS

Content

• Mobile marketing platform : Curio

• Curio’s architecture (Storm + Kafka + Cassandra)

• Use cases about Cassandra

About Curio

• Mobile marketing platform

• Now serving 80+ mobile applications in production

• Nearly 100 million transactions/day

• Real time interaction with users (via Push Notifications)

Example Analytics Data

Apache Kafka

• Distributed publish-subscribe messaging system

• Open sourced by Linkedin

Apache Kafka – Features

• Fast

• Scalable

• Durable

• Distributed by Design

Apache Storm

• Distributed fault-tolerant realtime computation system

• Open sourced by Twitter

• Written in Java

Apache Storm – Features

• Runs "Topologies"

• Clustered Structure

• Master node Nimbus

• Worker node Supervisor

• State is kept in Zookeeper

Apache Storm – Features II

• Integrates with any queueing and database system

• Kestrel, RabbitMQ / AMQP, Kafka, JMS..

• Simply connect with your database

• Simple API / Trident API

Apache Storm – Features III

• Scalable

• Benchmarks clocked Storm at over 1.000.000 tuples/second/node

• Fault-tolerant

• Guarantees your data will be processed (exactly once is guaranteed by

Trident API)

Curio Topologies

• Visit Topology (heavy reads & writes to Cassandra)

• 24 parallel and partitioned tasks processing raw data

• 12 parallel consolidating tasks processing the pre-processed data

• Push Topology

• 5 paralel tasks for sending push notifications

Topology To Cassandra

• All stored in Cassandra

• Mobile application launching/closing (creating and ending session)

• Page navigations (creating and ending screen hits)

• Event triggers (creating events)

• Counted values (relevant summary tables)

Curio Use Cases

• Use Case – I

• Calculating online user counts in real time

• Use Case – II

• Calculating active user counts in real time

Use Case I:

Counting Online Users

Requirement

• Counting online users for each mobile application

• Within a session timeout duration a user is online if:

• Opens a session

• Navigates through screens

• Triggers events

First Implementation

• Store online requests into a single table

• Default compaction strategy for the table is :

SizeTieredCompactionStrategy

First Implementation (cont.)

• Insert with a TTL for each request that is encountered as "online"

• Do deletion for session end requests

• Use a count query when online counts are requested

First Problem

• Storm performs insertion, update and deletions to "visit_online" table

• The performance of these queries got 100 times worse than before.

• The cause is stated as "SizeTieredCompactionStrategy"

Solution I

• Use "LeveledCompactionStrategy"

• The storm queries returned back to normal values as 50msec/tuple

Second Problem

• Count queries started getting timeout under heavy traffic

• ex: the applications who has 500.000 transactions/day

The Solution

• Re-design online table :

• Add new column desc ordered to identify "online" status :

valid_through

• TTL duration is changed to 3 days (no session_timeout any more)

The Solution (cont)

• Do not perform manual deletions

• Insert single row for each transaction (session, screen, event)

• No more count query! Calculation performed in java by selecting less

records with:

Use Case II:

Counting Active Users

Requirement

• Users have unique "visitor_code" for each application

• The most required report is "active user count" requires performance

• Active users = new users + returning users

• New users can be summed up daily

• Returning users need calculation with time interval

Solution

• Create a new table for storing session intervals

• Use a column for storing previous session time

previous_timestamp

• This column is ordered DESC

Solution

• Calculate returning users by:

• Running query for each date in the selected date interval

• Select records whose previous login time is less than the start time

of query

View from GUI

Thank You

@ulkeroz