An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival...
-
Upload
truongkhanh -
Category
Documents
-
view
226 -
download
2
Transcript of An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival...
![Page 1: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/1.jpg)
An Introduction to Survival Analysis
Dr Barry Leventhal
Transforming Data
Dr Barry Leventhal
Henry Stewart Briefing on Marketing Analytics
19th November 2010
![Page 2: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/2.jpg)
Agenda
• Survival Analysis concepts
• Descriptive approach
• 1st Case Study – which types of customers lapse early
• Predicting survival times
Transforming Data
• Predicting survival times
• 2nd Case study – lifetimes of mobile phone customers
• Business applications of survival analysis
• Applications to different industries and problems
• Summary of business benefits
![Page 3: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/3.jpg)
Tracking the Customer Lifecycle - Financial Services
Forminga Family
Moving upthe Ladder
Golden Years
Income ChangeRetirement Annuity
Transforming Data
Starting Out
FinancialIndicators
MortgageLoan ProtectionJoint Accounts
Life InsuranceLoansHigher monthly debits
InvestmentsIncreased monthly depositsRetirement Plans
AnnuityMove home
![Page 4: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/4.jpg)
Tracking the Customer Lifecycle – Telco
YoungAdults
Middle Aged
Golden Years
Simpler handsetSkype to grandchildren
Transforming Data
Kids
FeaturesFunky PhonePay as You GoHeavy texting
Pay MonthlySmart PhoneData users
Good to talkBluetoothLocation-based services
grandchildrenEmergency services
![Page 5: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/5.jpg)
What is Survival Analysis?- Analysis of TIME
• To understand length of time before an event occurs
• To predict time till next event
• To analyse duration of time in a particular state
“Event” can be:
Transforming Data
“Event” can be:
• Customer churn
• Take-up new product
• Default on credit
• Make next purchase
• …
![Page 6: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/6.jpg)
How does Survival Analysis differ from Churn Analysis?
Churn Analysis
• Examines customer churn within a set time window e.g. next 3 or 6 months
• Predicts likelihood of customer to churn during the defined window
Survival Analysis
• Examines how churn takes place over time
• Describes or predicts retention likelihood over
Transforming Data
the defined window
• No indication about subsequent risk of churn
• Does not provide information on customer lifetime value
retention likelihood over time
• Identifies key points in customer lifecycle
• Informs customer lifetime value
![Page 7: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/7.jpg)
The value of understanding both Churn and Survival Time
Churn
• Act on imminent event
• Understand combination of factors that are causing the current high
Survival
• Plan the customer lifecycle
• Understand how to extend time as customer before churn is imminent
Transforming Data
the current high probability of churn
• Understand why some customers churn
• Understand why some customers are retained longer than others
• Act on predicted changes in survival time
![Page 8: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/8.jpg)
Customer Survival – a Censored Data Problem
• You know most about the customers you’ve lost
• You want to predict the future retention of customers you haven’t yet lost
Lapsed
Transforming Data
timenow
?
CensoredCase
LapsedCase
![Page 9: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/9.jpg)
Terminology used in Survival Analysis
• Hazard Function
– the risk of churn in a time interval after time t, given that the customer has survived to time t
– usually denoted as: h(t)
• Survival Function
Transforming Data
• Survival Function
– the probability that a customer will have a survival time greater than or equal to t
– usually denoted as: S(t)
• Hazard and Survival functions are mathematically linked - by modelling Hazard, you obtain Survival
![Page 10: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/10.jpg)
Example Hazard Function – the classic “Bathtub” curve
Transforming Data
![Page 11: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/11.jpg)
100%
80%
60%
Example Survival Curve
80% probability of surviving beyond 7 years
Survival Probability
50% probability of surviving beyond 8 years
Transforming Data
60%
40%
20%
0%
Area under curve = expected survival time
Time in years
beyond 8 years
![Page 12: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/12.jpg)
Descriptive Survival Analysis
• Compute the survival curve for your customer base– Understand ‘natural patterns’ in customer survival
– Identify key points where survival rates fall
• Compare survival curves between– Demographic groups
– Customer segments
Transforming Data
– Customer segments
– Sales channels
– Product plans, etc
• Identifies key factors influencing ‘time till churn’
• Enables you to predict monthly numbers of churners – but does not identify which customers will churn
• Most widely used method: Kaplan-Meier
![Page 13: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/13.jpg)
1st Case Study Which types of customers lapse early?
• Financial services company cross-selling Personal Accident insurance via telemarketing
• Company experienced an increase in monthly lapse rates and reduction in retention levels
Transforming Data
rates and reduction in retention levels
• Wanted to understand which types of customers were lapsing early and identify optimal intervention point for reducing lapse rates
![Page 14: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/14.jpg)
Descriptive Survival Analysis – by Age Bands
• Survival chances increase with Age
- the older the customer, the longer they are likely to retain PA insurance
Transforming Data140 6 12 18 24 30 36
Results have been disguised
![Page 15: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/15.jpg)
• Hazards Model
– a model for predicting the hazard of an individual
• Cox Proportional Hazards Model
– a particular form of hazards model, for predicting hazard as a combination of survival time and individual characteristics
Predicting Survival Times
Transforming Data
combination of survival time and individual characteristics
h(t,x,b) = ho(t) . exb
Baselinehazard
Individual effect:data value x,regression coefficient b
![Page 16: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/16.jpg)
Case Study Example: Survival Model for European Pre-pay Mobile Phone Operator
• Data from the Data Warehouse extracted for a sample of pre-pay mobile customers
• Both active customers and previous churners were represented
Transforming Data
represented
• Wide range of variables and attributes were extracted, that could help to explain length of customer relationship
Source of Case Study: Teradata Partners User Group Conference
![Page 17: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/17.jpg)
Example data for Pre-pay Survival Analysis
• Calling data
– Inbound / Outbound
– Home / Roam
– Voice / SMS (inbound and outbound)
– Voice Mail usage
In-network / Out of network
• Top-up data
– Frequency of top-ups
– Time between top-ups
– Value of top-ups
• Customer data
– Age
Transforming Data
– In-network / Out of network
– Dropped calls
– Customer care interactions
– Product usage
– Volatility of call patterns
– Age
– Gender
– Geodemographic data -postcodes
– Handset information
– Registered
![Page 18: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/18.jpg)
Example Results: Key factors that influence lifetime of a pre-pay customer
• Prepayment top-up behaviour
– High value prepayments
– Medium value prepayments
– Frequent prepayments made
• Calling behaviour in home calling area
– Value of outbound voice calls
– Number of inbound calls and text messages
Transforming Data
– Number of inbound calls and text messages
– Use of added-value services, such as voicemail
– Out of network outbound voice calls
• Customer Demographics
– Gender
– Age
– Geodemographic segments
• Quality issues
![Page 19: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/19.jpg)
Example Results: How Factors Influence Survival – Customers making frequent pre-payments
0.70
0.80
0.90
1.00
Overall Survival Probability
Transforming Data
0.40
0.50
0.60
0.70
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 56 59
Months of Survival
Overall Survival Probability
Frequent Prepayments - Yes Frequent Prepayments - No Survival - Mean Values
![Page 20: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/20.jpg)
Example Results: How Factors Influence Survival – Customers making high-value pre-payments
0.70
0.80
0.90
1.00
Overall Survival Probability
Transforming Data
0.40
0.50
0.60
0.70
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 56 59
Months of Survival
Overall Survival Probability
High Value Prepayment - Yes High Value Prepayment - No Survival - Mean Values
![Page 21: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/21.jpg)
Outputs from Predictive Analysis
• Survival curve – all customers and sub-sets
• Key factors influencing “time till churn”
Survival model – can apply to individual customers
Transforming Data
• Survival model – can apply to individual customers
– Customers should be regularly rescored, and their scores saved and monitored
![Page 22: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/22.jpg)
Business Applications of Survival AnalysisCustomer Management
• Examine and act on predicted customer survival rates over time:
– Identify customers whose predicted survival rates are low or rapidly falling
Transforming Data
– Examine implications if a key behaviour could be changed
– Take the right marketing actions aimed at influencing behaviours with greatest impact on predicted survival rates
– Address some behaviours by modifying service design or terms of use
![Page 23: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/23.jpg)
Frequent Prepayments0
20 Euro Prepayment0
30 Euro Prepayment0
Recent Outbound Voice
Calls 2
Outbound Voice Calls 2
What are the implications of changes in the customer’s behaviour on predicted survival?
0.3000
0.4000
0.5000
0.6000
Transforming Data
Outbound Voice Calls 2
Months ago 8
Recent Inbound Voice Calls2
Recent Text Messages Sent2
Text Messages Sent 2
Months ago 3
Recent Voicemail Use5
Recent Out-of-network
Voice Calls 5
0.0000
0.1000
0.2000
s(t0) s(t1) s(t2) s(t3) s(t4) s(t5) s(t6) s(t7) s(t8) s(t9) s(t10) s(t11) s(t12)
C4New Customer 4
![Page 24: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/24.jpg)
Frequent Prepayments0
20 Euro Prepayment0
30 Euro Prepayment0
Recent Outbound Voice
Calls 2
Outbound Voice Calls 2
1
0.4000
0.5000
0.6000
0.7000
0.8000
What are the implications of changes in the customer’s behaviour on predicted survival?
Transforming Data
Outbound Voice Calls 2
Months ago 8
Recent Inbound Voice Calls2
Recent Text Messages Sent2
Text Messages Sent 2
Months ago 3
Recent Voicemail Use5
Recent Out-of-network
Voice Calls 5
0.0000
0.1000
0.2000
0.3000
s(t0) s(t1) s(t2) s(t3) s(t4) s(t5) s(t6) s(t7) s(t8) s(t9) s(t10) s(t11) s(t12)
C4 New Customer 4
![Page 25: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/25.jpg)
Frequent Prepayments0
20 Euro Prepayment0
30 Euro Prepayment0
Recent Outbound Voice
Calls 2
Outbound Voice Calls 2
1
0.4000
0.5000
0.6000
0.7000
0.8000
0.9000
What are the implications of changes in the customer’s behaviour on predicted survival?
Transforming Data
Outbound Voice Calls 2
Months ago 8
Recent Inbound Voice Calls2
Recent Text Messages Sent2
Text Messages Sent 2
Months ago 3
Recent Voicemail Use5
Recent Out-of-network
Voice Calls 5
0.0000
0.1000
0.2000
0.3000
0.4000
s(t0) s(t1) s(t2) s(t3) s(t4) s(t5) s(t6) s(t7) s(t8) s(t9) s(t10) s(t11) s(t12)
C4 New Customer 4
![Page 26: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/26.jpg)
Further Business Applications
• Business Planning – Forecast monthly numbers of lapses and use to monitor current lapse rates
• Lifetime Value prediction– Derive LTV predictions by combining expected survival times with monthly revenues
Transforming Data
with monthly revenues
• Active customers– Predict each customer’s time to next purchase, and use to identify “active” vs. “inactive” customers
• Campaign evaluation– Monitor effects of campaigns on survival rates
![Page 27: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/27.jpg)
Applications to different industries and business problems
• Telco – customer lifetime and LTV
• Insurance – time to lapsing on policy
• Mortgages – time to mortgage redemption
Transforming Data
• Mail Order Catalogue – time to next purchase
• Retail – time till food customer starts purchasing non-food
• Manufacturing - lifetime of a machine component
• Public Sector – time intervals to critical events
![Page 28: An Introduction to Survival Analysisbarryanalytics.com/Downloads/Presentations/Survival Analysis.pdf · An Introduction to Survival Analysis Dr Barry Leventhal Transforming Data Henry](https://reader031.fdocuments.net/reader031/viewer/2022022421/5a8a69c47f8b9ac87a8c4b29/html5/thumbnails/28.jpg)
Business Benefits of Survival Analysis
• Improved planning and budgeting through better understanding of future events over time
• Ability to plan timing of churn-related customer communications
Transforming Data
• Greater ability to manage customer lifecycles
• Better understanding of factors causing customers to stay for different lengths of time, enabling those factors to be influenced - either by improving service design or at customer level