background image: 960x540 pixels - send to back of slide and set to 80% transparency
Advanced Analytics for Telecommunications Bob Glithero, Principal Product Marketing Manager Vineet Goel, Product Manager
background image: 960x540 pixels - send to back of slide and set to 80% transparency
Agenda
• Pivotal – Hortonworks Partnership • Challenges in Customer Experience • HDB: Hadoop-Native Analytics Database
for Hortonworks Data Platform • Sample Use Cases • For More Information
Pivotal HDB + Hortonworks Hadoop Partnering for Faster Value from Data
● Leaders in open-source Hadoop ● Managing, analyzing, and operationalizing data at
scale ● Joint support for ODPi promotes interoperability in
Hadoop
+Pivotal and Hortonworks’ strategic partnership marries Pivotal’s best-in-class SQL on Hadoop, analytical database, with Hortonworks’ best-in class expertise and support for Hadoop.
Managing Experience is Complicated
Then • Basic handsets, embedded applications • Simpler services - voice, SMS, WAP • Experience influenced mostly inside the network
Now • From phones to hand-held computers • Massive data volume, velocity, and variety from millions of apps and
services • MNOs held responsible for all aspects of service, whether inside or
outside the network
CSPs Increasingly Competing on QoE
Trying to understand how network performance impacts experience
When service is degraded, CSPs need to quickly understand:
Is the problem inside or outside the network? Which subscribers are impacted? What needs attention first?
Common Operator Challenges
Network Operations Customer Care Marketing
Increase monetization, offset voice, SMS revenue loss
Reduce churn and credits, cost to serve
Reduce complexity, increase visibility, increase QoE
background image: 960x540 pixels - send to back of slide and set to 80% transparency
Operators are turning to their data to solve these challenges How do we analyze data in an efficient, cost-effective way to transform customer experience?
High performance, interactive SQL queries on Hadoop HDB: The Hadoop Native SQL Database
● Highly efficient MPP (massively parallel processing)
● Low-latency
● Petabyte scalability
● ACID transaction support
● SQL-92, 99, 2003 compatibility
● Advanced cost-based optimizer
DATA LAKE SQL App
BUSINESS ANALYSTS
DATA SCIENTISTS
Advanced Analytics Performance
Exceptional MPP performance, low latency, petabyte scalability, ACID
reliability, fault tolerance
Most Complete Language Compliance
Higher degree of SQL compatibility, SQL-92, 99, 2003, OLAP, leverage
existing SQL skills
Best-in-class Query Optimizer
Maximize performance and do advanced queries with confidence
Elastic Architecture for Scalability
Scale-up/down or scale-in/out, expand/shrink clusters on the fly
Tightly integrated w/MADlib Machine
Learning Advanced MPP analytics, data science at
scale, directly on Hadoop data
HDB / HAWQ Advantages
MAD
● DiscoverNewRela/onships● EnableDataScience● AnalyzeExternalSources● QueryAllDataTypes!
Mul/-levelFaultTolerance
GranularAuthoriza/on
ResourcePools+YARN
Mul$-tenancy+Security
ANSISQLStandard
OLAPExtensions
JDBCODBCConnec/vity
MPPArchitecture
OnlineExpansion
Hadoop/HDFS
PetabyteScale
Cost-BasedOpXYZizer
DynamicPipelining
ACID+Transac/onal
AmbariManagement
MachineLearning
DataFedera/on
LanguageExtensions
Hardened,10+YearsTested,Produc/onProven
Opera$ons+Extensibility
HDFSNa/veFileFormats
● ManageMul/pleWorkloads● PetabyteScaleAnaly/cs● Sub-secondPerformance
● LeverageExis/ngSkills&Tools
● EasilyIntegratewithOtherTools
Compression+Par//oning
Core
compliance
● WellIntegratedwithHortonworksDataPlaZorm
HDB + HDP Marketecture
13
Faster Insight with In-Database Analytics
Pivotal HDB / Apache HAWQ (incubating) Low-latency, MPP analytic
database with full ANSI SQL support running natively on
Hortonworks HDP
Apache MADlib (incubating) Scale out, SQL-based
machine learning within HDB/HAWQ, Greenplum, and
PostgreSQL databases
+
14
Top MADlib Use Cases
• Fraud detection • Risk analysis • Customer experience • Marketing • Predictive maintenance
Telco uses HDB to analyze and improve call quality
2bn call records per day • Overwhelmed traditional data warehouse
Hadoop and HDB • 5x data stored at half the cost • Familiar SQL interface to analyze 3 months
worth of dropped call data
DATA LAKE
16
How could a network operations team apply analytics to improve experience for its network services?
What Data Is Needed?
Service Assurance Customer Care Marketing • Network Performance data (GTP probe data)
• HTTP Click Stream Records
• Flow Records
• Network & Device Reference Data
• Topology and location
• HTTP Click Stream Records
• Flow Records
• Network Performance data (GTP probe data)
• CRM data (account, device information)
• Service Request Records
• HTTP Click Stream Records
• Flow Records
• CRM data (account, device information)
Constructing KQIs from performance indicators
84% Speed Latency Effective Throughput
Integrity
Drops Time-Outs Cut-Offs Failures
Retainability
Failure % Response time Access time
Accessibility Voice QoE
Data capture Data science
• xDRs • NetFlow • Probes
Data processing
Accessibility
Quality
Retainability
In-Database Analytics with HDB and MADlib
Application/ Content Data
• Raw Usage • Logs • (HTTP, Flow, Other)
HDFS
HBase
Hive
HDB/HAWQ In-DB Analytics Network Data
• Probes (GTP-C/U) • xDRs
• Case management • CRM • Billing • Device inventory • Network topology • Geolocation maps B/OSS Data
PXF
PXF
MPP Query Execution
ANSI SQL
• SQL-based • Over 50 data science
functions • UDFs
• Offline modeling • Batch queries • Reporting/viz with
SQL-based tools
+
Native or PXF
20
How could marketing teams use analytics to better target subscribers for promotions and advertising?
Blended Mobile ARPU is Declining
Loss of voice and SMS ARPU from competition, free apps Data revenues not offsetting voice, SMS losses MNOs seeking new monetization options
Source: IHS Technology Mobile ARPU Forecast, 2016
Need for Behavioral Insights
• CSPs need to maximize subscriber yields to offset declining revenues
• Marketers have little information to market to anonymous prepaid subscribers
• Need to protect current revenue from competition from over-the-top (OTT) apps and services
Morning: New York • Starts on Samsung Galaxy S6 • On CNN, sees news on earthquake • Donates via Red Cross Society • Later: Switches to iPad – same account
plan • Checks market close on WSJ.com
A Day in the Life: User Perspective
Evening: Boston • Checks Facebook page • Streams Netflix
SubscriberId StartTimeStamp EndTimeStamp URL User AgentRK2FQ9PWZVW52 2015 04 28 06 37 04 512 2015 04 28 06 37 04 543http://www.cnn.com Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1 RK2FQ9PWZVW52 2015 04 28 06 37 05 546 2015 04 28 06 37 04 623http://www.cnn.com/world Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1
RK2FQ9PWZVW52 2015 04 28 06 37 19 529 2015 04 28 06 37 19 599http://www.cnn.com/2015/04/28/asia/flight-delhi-nepal-earthquake/index.html Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1
RK2FQ9PWZVW52 2015 04 28 06 37 23710 2015 04 28 06 37 23 770http://www.cnn.com/2015/04/28/asia/kathmandu.jpgMozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1 RK2FQ9PWZVW52 2015 04 28 06 37 45919 2015 04 28 06 37 45988http://adclick.g.doubleclick.net/pics/click/?= Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1
RK2FQ9PWZVW52 2015 04 28 06 37 34957 2015 04 28 06 37 34996http://www.google-analytics.com/__utm.gif?utmwv=4.9mi Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1
RK2FQ9PWZVW52 2015 04 28 06 42 09 883 2015 04 28 06 42 10 467http://www.cnn.com/2015/04/25/world/nepal-earthquake-how-to-help/index.html Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1 ….. (images being loaded here) …….
RK2FQ9PWZVW52 2015 04 28 06 43 03 234 2015 04 28 06 06 12 334http://www.nrcs.org Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) ….. ….. ….. ….. …
RK2FQ9PWZVW52 2015 04 28 09 45 05 7322015 04 28 09 45 05
812 http://wsj.comMozilla/5.0 (iPad; CPU OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari
….. ….. ….. … …
RK2FQ9PWZVW52 2015 04 28 17 03 14 204 2015 04 28 17 03 14 269http://wsj.comMozilla/5.0 (iPad; CPU OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari
….. ….. ….. … …RK2FQ9PWZVW52 2015 04 28 18 19 56 459 2015 04 28 18 19 56 509https://69.63.178.45
….. ….. ….. … …RK2FQ9PWZVW52 2015 04 28 21 23 25 754 2015 04 28 21 23 25 876http://23.13.201.71 netflix-ios-app
A Day in the Life: Data Perspective
• Capture and collate raw subscriber data • Sessionize and enrich clickstream data with location, device. and other data, calculate subscriber
usage metrics
SubscriberId DeviceNAME PUBLISHERCategory-
SubcategoryApplication
Name SESSION START SESSION END PAGE_VIEWS HITS BYTES LOCATION
RK2FQ9PWZVW52Samsung Galaxy S6 CNN NewsNews-International
News CNN App 2015 04 28 06 37 04 512 2015 04 28 06 42 10 467 4 45 539123 NY
RK2FQ9PWZVW52 Samsung Galaxy S6 Red CrossNon Profit &
Charities-Institutions Browser 2015 04 28 06 43 03 234 2015 04 28 06 53 03 874 2 7 383372 NY
RK2FQ9PWZVW52 Apple iPadWall Street
JournalNews-Business &
Finance News Safari Browser 2015 04 28 09 45 05 732 2015 04 28 09 55 05 732 4 40 600272 NY
RK2FQ9PWZVW52 Apple iPadWall Street
JournalNews-Business &
Finance News Safari Browser 2015 04 28 17 03 14 204 2015 04 28 17 23 14 204 5 35 801714 NY
RK2FQ9PWZVW52
Apple iPad Facebook
Social Media & Networking-Social
Networking -2015 04 28 18 19 56 459
2015 04 28 18 23 21 459
318 5041054 Boston
RK2FQ9PWZVW52
Apple iPad Netflix
Media & Entertainment-Online
Video Ne&lixApp2015 04 28 21 23 25 876
2015 04 28 23 23 24 325
6 2330 295121789 Boston
Compute subscriber-level metrics and aggregates
…enrich with information about content (websites or apps) and categorization, devices, and locations
Aggregation and Enrichment
Insights: Marketing to Prepaid Users
• With data science, operators can infer gender and approximate age from subscriber activity
• Classify according to segmentation schemes (e.g., who does unknown subscriber resemble from their activity)
We can offer advertisers anonymized subscriber info mapped to standard marketing/advertising categories (e.g., IAB) based on activity
Marketing Questions We Can Answer with Analytics
• How will subscribers respond to changes in pricing?
• How do we market to anonymous pre-paid subscribers?
• Who’s likely to respond to an offer?
• Which OTT apps threaten our own branded apps?
• Which groups should we target with advertising?
Pivotal and Hortonworks are partnering to help
companies use their data for better
customer outcomes
Learn more
• Videos: bit.ly/MADlibvideos • Project: madlib.incubator.apache.org • Downloads: bit.ly/getMADlib
• Videos: bit.ly/HDBvideos • Project: hawq.incubator.apache.org • Commercial: pivotal.io/pivotal-hdb • Downloads: bit.ly/getHDB
Top Related