PA

GE

1

October 30, 2017

Julia Minkowski Principal, Fraud Analytics, Signifyd

Real-Time Fraud Detection:

Strategies for Speed and Actionability

PA

GE

2 Agenda

Use Case:

Fraud Prevention in E-Commerce What problem should your team be solving?

Best Practices:

Turning Data Mining Strategy into Action

PA

GE

3

PA

GE

4 What is special about Fraud Prevention?

1 Fraud is performed by

organized criminal groups

using sophisticated

technologies and logistics

PA

GE

5 What is special about Fraud Prevention?

2 Hard to detect: target has low

frequency (2 in 10,000)

PA

GE

6 What is special about Fraud Prevention?

3 The cost of mistakes is very

high

60% increase

in customer

attrition if you

misclassify (False Positives)

$ Losses if

you fail to

detect fraud (False

Negatives)

PA

GE

7 What is special about Fraud Prevention?

4 The environment changes

fast, so you need to adapt

quickly

PA

GE

8 What is special about Fraud Prevention?

5 Fraud prevention is a great

field for the application of

predictive analytics

PA

GE

9 Analytics for Fraud Prevention

Explore

& Understand

Anticipate

&

Control

Monitor

1 2 3

PA

GE

11

Risk Management: Goals and Constraints

Goals

Help the merchants to expand to

more profitable markets

(international, cross-selling), while

keeping loss rates constant, and

their customers happy

Constraints

• Build a flexible system that adapts to new

fraud patterns

• Service the existing client base

• On-board new merchants

• Minimize the time that the production systems

will be off-line or reset

• Build the next-generation of strategies with

very limited resources

PA

GE

12

Data Miner Survey by Rexer Analytics

While 6 out 10 data miners report the data is available for analysis within days of capture, the time to deploy the models takes substantially longer. For 60% of the respondents, the deployment time will range between 3 weeks and 1 year.

Everyone

might forget

about

deployment –

but it is a most

important

component!

PA

GE

13

In Fraud Mitigation – Speed is the Key

How long can you wait to deploy a solution?

PA

GE

14

Evolution of Model Deployment

14

3 months to collect data, build and deploy a model

2 weeks to estimate model

1 week to install rules 4 hours to estimate a model

1-2 days to install rules

4 hours to build a model

Few hours to implement

Same day analysis and rules deployment

2014 2016 2017 2018

PA

GE

15

Agenda

Use Case:

Fraud Prevention in E-Commerce What problem should your team be solving?

Best Practices:

Turning Data Mining Strategy into Action

PA

GE

16

Best Practices in Analytics

16

Select Best Option(s)

Success Factors and Constraints

• ROI /Cost

• Profitability

• Operations

1. Identify Benefits & Constraints

Install into Production

• Run A/B testing

• Start Small and Increase Gradually

Data

Scientist

3.Turn Strategy into Action

IT

Manager

Select the Appropriate Infrastructure

• DB Architecture

• Modeling techniques

2. Develop the Strategy

Provide Actionable Insights

Estimate Impact for the Business Track Benefits and KPI

• Test Predictive Models

• Simulate scenarios (Monte Carlo) Score

models on KPI

Collect & Process Data

• Run Descriptive Analytics

• Identify patterns

Business

Manager

• Align your Team’s Incentives

Involve Key Stakeholders

PA

GE

17

Involve the Right Stakeholders

17

Business Manager

Analyst / Data Scientist IT Manager

• Preserve Service Level

Agreement

• Reduce Operational Risk

• Preserve Budget

PA

GE

18

Conflict of Interests?

18

Cannot agree on success factors?

Wonder why?

PA

GE

19

IT Manager’s Strategy

19

• Preserve Service Level Agreements (SLA)

• Stable systems

• Ease of roll-back

• Minimize Operational risk

• Control Costs

19

PA

GE

20

Analyst / Data Scientist’s Mind

20

• Estimate the Best Model Possible

• Improve Detection Rates

• Better Algorithms, Faster Hardware

• Big(ger) Data!

• Explore New Algorithms

• Put some power behind it !!

PA

GE

21

Business Manager’s Mind

21

• Maximize Productivity: Build for specific

needs

– What is the cost?

– What is the impact on customer

experience?

– Why does it take so long?

– And: Don’t talk to me in Tech-Speak !

“First we ran a chi- square test, and then we converted the categorical data

to ordinal, next we ran a logistic regression, and then we lagged the

economic data by a year…”

Source: Davenport, Tom (2013), “Keep up with your quants", Harvard Business Review, Issue Jul-Aug 2013

PA

GE

22

Communication issue

Presenting a solution What the analyst sees What the audience sees

What the audience

remembers

What the presenter

remembers Feedback on the

solution

Source: Eric Hixson, PhD, Cleveland Clinic, 2014

PA

GE

23

Managing the Quants (Tip for Managers)

23

• Define clearly the objective and constraints

• Implement SMART* goal setting

• Establish a timeline for delivery then multiply x 2

• Get familiar with basic analytics concepts

• Coursera, EDX, Lynda, TDWI

• Make sure you understand enough to explain to other

executives: you will champion this initiative and negotiate the

budgets

* SMART Goal setting involves establishing Specific, Measurable, Achievable, Realistic and Time-

targeted goals. Wikipedia, 2016

PA

GE

24

Taking Care of Business (Tip for Analysts)

24

• Communicate clearly business level information

• When and what is the expected result

• Present the key concept in 2 phrases

• Avoid technical language for communication

• If asked for more details, then present the “How”

• Provide a Business Dashboard

• Provide the $$ metrics profit/loss reduction

• Show the impact of algorithms deployed / provided

• Current vs. Historical

• Pick the right model - the model that maximizes the

ROI

Source: Davenport, Tom (2013), “Keep up with your quants", Harvard Business Review, Issue Jul-Aug 2013

PA

GE

25

Tracking Performance: Dashboard

25

Our dashboards tracked the key performance metrics:

• Historical Trends for Fraud Rates and $ Losses (Business KPI)

• Percentage of Transfers affected by Risk Mitigation (Business KPI)

• % of population affected by policy and % of fraud prevented (KPI for Analytics)

• Fraud detection rates for models and rules installed (KPI for Analytics)

PA

GE

26

Super-Leader characteristics

26

Source: Alexander Linden, Key Trends and Emerging Technologies in Advanced Analytics, Gartner 2014

PA

GE

27

Key Takeaways On Fraud Analysis and Modeling

When dealing with fraud, the speed to implement a new model is the most important factor

Improvements in accuracy may be lost due to delays in deployment; systems with fast turnaround have better ROI than complex algorithms with long implementation times

Turning Strategy into Action

Involving the key stakeholders early in the process maximizes your chance for success. Once you have aligned the incentives for the team, selecting the appropriate techniques and infrastructure becomes much simpler

It is crucial for business managers to correctly define the problems and objectives, asking the right questions and learning the basic analytical concepts

For data scientists it is important to select their models and projects based on the expected business impact and to translate their findings into the relevant metrics

27

1

2

1

2

3

PA

GE

28

THANK YOU!

28