AI & ML Hype, Reality and Results - SAS

Copyright © SAS Inst itute Inc. A l l r ights reserved.

SAS® FINANCIAL CRIMES EXECUTIVE FORUM Toronto, 2018

Changing Face of AI & ML for Financial Crimes - Hype, Reality and Results

Michael Ames, Sr. Director of Fraud, Compliance and Investigative Solutions , SAS


Past to Present1950s 1980s 2010’s Present

Evolution

Classical

Modern

Artificial Intelligence

Machine Learning

Deep Learning

Neural Network

Artificial Intelligence


Statistics quantifies numbersDescribes data (mean, median, mode, standard deviation etc.) Draws conclusions from data (hypothesis test, deriving estimates etc.)

Machine Learning predicts with modelsProcess of fitting equations to data with the goal of predictive accuracy “Deep Learning” is a specific class of machine learning models

Artificial Intelligence provides the appearance of behavior through automation Computer programs to mimic “human” behavior and interactionsAutomate processes for productivity, efficiency and accuracy

Data Mining & Data Science finds patterns to explain a phenomenonPractical application of computer programs, Statistics and Machine LearningDevelop AI bots, systems, and applications


Reality: Machine Learning is about building a SYSTEM• Capacity to explore, test and create • Applying a level of analytic rigor • Using appropriate technology• Make it repeatable


Real-world data science with disparate tools and

techniques…


or another approach…


Is the juice worth the squeeze?

Picture of hand squeezing orange

Is the Juice worth the squeeze?


SBM Cracks down on insurance claims fraud Toolkit Approach

BUSINESS GOALS• Increase accuracy of fraudulent claim detection

• Identify organized crime rings

• Adapt to evolving fraud patterns

Toolkit of Advanced Analytics & Data Management • Predictive Models (Trees, Logistic Regression)

• Unsupervised techniques (Clustering)

• Anomaly detection (Clustering)

• Network and Graph analytics

• Alert management

• Data management

RESULTS• In just nine months, SBM uncovered US $86 million in potential fraud.

• Near-real-time scoring, detects fraud early in the cycle before payments occur.

"Our focus on analytics over business rules has led to the discovery of 259 million TL [US $86 million] in potential fraud cases within the first nine months of using the solution. ”

Aydin Satici

General Manager SBM

https://www.sas.com/en_us/customers/sbm-tr.html

https://www.sas.com/en_us/customers/sbm-tr.html


Try it you’ll like it …

Picture of broccoli or cauliflower

Try it you’ll like it…


Reduce AML False Positives & Quality Checks3-month study into the art of the possible

FP Reduction Eliminated - 70% of wire alerts - 77% of structuring alerts - 92% of dormant account alerts - 95% of loan activity alerts - 86% of ATM activity alerts

Identified 15k mislabeled retail &

commercial accounts

And 5k retail accounts being used as commercial

Improved Surveillance Accuracy - 2x retail accounts- 6.5x high net worth accounts - 19X phishing alerts


Unregistered MSBs - Proof of Concept1.7B observations to find 416 MSBs

25 Confirmed Fraud MSBs

89 previously unknown & un-regulated MSBs

Aggregation of 1.7B observations in 9min 45sec

Dozens of Cases Referred for Investigation

416 Potential MSBs Identified

81% Overlap with prior intelligence


There is gold in those hills!


Nets improves fraud-detection rate by 50 percent Neural Networks in Action

BUSINESS GOALS• Prevent fraud by stopping suspicious transactions before they are processed.

• Distinguish between true fraud and genuine cardholder spending.

• Avoid freezing customers’ accounts due to false positives.

Application of Segmented Neural Ensembles • SAS’ Signature Approach for Feature Engineering

• Predictive Machine Learning (Ensembles, Neural Networks, Neural Wavelets)

RESULTS• Improved fraud-detection rate by 50 percent.

• Reduced card fraud by 50 to 70 percent.

• False positives have been cut in half.

“Since our analysis team began using SAS Fraud Management, we’ve increased our fraud-detection rate by 50 percent and reduced card fraud by 50 to 70 percent for cards under the optional prevention program – all while cutting false positives in half.”

Kaspar Kock Kristensen

Senior Vice President of Fraud and Dispute Services

https://www.sas.com/en_us/customers/nets.html

Copyr i g ht © 2016, SAS Ins t i tu t e Inc . A l l r ights reser ve d .

What is SAS doing?


Platform for AI & ML

• Enabling

• APIs & ServicesDeploy, Provision, Use

Self-service

Innovative

• Analytical Lifecycle

• Governance

• Integrated

Capability & Consumption Based

Private & Public

Elastic

• Languages


Fresh new look


Code in your preferred language … SAS, R, Python


Adaptive Machine Learning and Intelligent Agents

For Compliance, Fraud, Waste and Abuse

Company Conf ident ia l – For Internal Use OnlyCopyright © SAS Inst itute Inc. A l l r ights reserved.

Machine Learning & AI We know what is required…• Analytic Rigor

• Documented, stable & repeatable process

• “Develop on one set test on another”

• Supervised Machine Learning • Uncover patterns from known outcomes

• Un-Supervised Machine Learning • Uncover patterns from data unknown outcomes

• Explainable / Digestible for Analysts and Regulators • Scorecard based Visualizations

• Natural Language Generation

• Monitoring & Evaluation • Champion challenger & retraining

• Ability to learn and adapt • Automated “system” outcomes

• Curated “alternative system” and or “hand-labeled” outcomes


What are we doing ? Improving accuracy and effectiveness…

Machine Learning

• Highly Accurate Supervised & Unsupervised Machine Learning Methods

• Providing customers with some of our best in-class ML Methods, Feature Engineering

- “AdaptiveQHyperBoost”

• White-box Scorecard Visualizations

• Simple Explainable

• White-box Narrative Generation

• “Plain English” narrative

• Model Validation Report

• Charts, graphs, repeatable with defensible language

• Scoring API & A-Store Model(s)

• Integration / deployable to anywhere SAS can score

• “Adaptive Learning” to Monitor / Retrain / Adapt

• The “intelligent agent”, adapts to changes overtime (automatically)

• Integration with Visual Investigator

• Outcomes & dispositions fed back into “the agent”


Machine Learning & AI Intelligent Agents & Automation

• Automation both Naïve and Predictive

• Naïve – when X occurs do A,B&C

• Predictive – when X, Y & Z occur, predict the actions to take: - Action A (90%) – automatically perform

- Action B (70%) – automatically perform

- Action C (50%) – prompt analyst to perform

- Action D (25%) – suppress

• Agent - Predicts Intent and or Action

• Based on analyst activities and trends in the data

• Augmenting Search, Exploration and Investigation

• Given this type of Alert/Case, suggest and gather

- Data

- Visualizations

- Narratives

- Checks

Automation


Is the juice worth the squeeze?

Picture of hand squeezing orange

Where do you start?

5 ways to look past the shiny-object phase...

Copyr i g ht © 2015, SAS Ins t i tu t e Inc . A l l r ights reser ve d .

#1 - Start with a problem, not the solution…

Before launching an AI/ML program, identify concrete business problems, then consider if AI can help. For example, rather than ask, “What can we use

AI for?”, think, “Where could we make our operations more efficient?” or “What decisions are we making that we could drive with data?”


#2 Have the Capacity and Will to experiment!


#3 Have realistic expectations

Failure is often more valuable than success…

#3 Aim high and have realistic expectations


#4 Demand a White-box

It’s not magic. If a data scientist can’t explain what they are doing or what their AI / ML

product, service or methods are up to in terms you understand, don’t buy it!


#5 Govern the process…


Keys to success with AI and ML

- Competition drives innovation… - Have capacity to expand on demand… - Experiment relentlessly… - Govern from end to end … - Insist on a white-box … - Rinse repeat …

“It’s the journey, not the destination”

sas.com


Thank you.

http://www.sas.com/


SAS® Visual Investigator: Adaptive Learning and Intelligent Agent System

Projects

jconrad

Table_Eight Table_Eleven Table_Five Table_Four

Select a table to analyze, or upload a file by drag and drop or browse for a file.

Table_Nine Table_One Table_Seven Table_Six

Table_Ten Table_Thirteen Table_Three Table_Twelve

Table_Two

Size: 1,864 KB

Last updated: Jul 12, 2017 2:14:42 PM

Size: 1,864 KB


Size: 1,864 KB


Size: 1,864 KB


Size: 1,864 KB


Size: 1,864 KB


Size: 1,864 KB


Size: 1,864 KB


Size: 1,864 KB


Size: 1,864 KB


Size: 1,864 KB


Size: 1,864 KB


Size: 1,864 KB


NameSort by:

One Click

Filter:

Project Six

SAS® Adaptive Learning and Intelligent Agent System 1

Table1 Target2 Variables3



Projects

jconrad

Select a target variable.

Project Six

1

Continue without a Target

Fraud Flag Account Status Gender

StateTransaction Type

One Click

DistinctSort by:Filter:


Categories (Binary)

Measures

Categories

Type: Numeric

Role: Input

Level: Binary

Type: Numeric

Role: Input

Level: Binary

Type: Numeric

Role: Input

Level: Binary

Type: Character

Role: ?????

Level: ?????

Type: Character

Role: ?????

Level: ?????

- Machine Learning ProjectSAS® Adaptive Learning and Intelligent Agent System



Projects

jconrad

Train Model Training the model may take a long time.

Candidate Variable Name Type Importance

Training candidate State Character 100

Training candidate ATM Deposit Sum 10 Numeric 87

Training candidate ATM Credit Count 1 Numeric 85

Training candidate ATM Deposit Sum 30 Numeric 75

Training candidate ATM Deposit Count 30 Numeric 67

Training candidate ATM Credit Sum 10 Numeric 65

Training candidate Cash Deposit 1 Numeric 54


Training candidate ATM Credit Count 10 Numeric 45

Training candidate ATM Credit Sum 30 Numeric 32


Training candidate Age Numeric 31

Training candidate Transaction Type Character 28

Training candidate Gender Binary 27

Training candidate Account Status Binary 25

Project Six

1

Training candidate

Specify variables that are candidates for training the models.

Distinct Missing

20 0%

One Click

State

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0

500

1000

1500

2000

2500

3000

Per

cen

t o

f Ev

ents

Co

un

t

State

Events

Count Percent of Events

Cal

ifo

rnia

Co

nn

ecti

cut

Del

awar

e

Flo

rid

a

Geo

rgia

Haw

aii

Illin

ois

Lou

isia

na

Mar

ylan

d

Mas

sach

use

tts

Mic

hig

an

Mis

sou

ri

Nev

ada

New

Jer

sey

New

Yo

rk

Oh

io

Pen

nsy

lvan

ia

Texa

s

Vir

gin

ia

Was

hin

gto

n


Role

InputLevel

Nominal

- Machine Learning ProjectSAS® Adaptive Learning and Intelligent Agent System



Projects

jconrad

Version 1

Project Six

1

Generate ScoresMake ChampionNew ModelPropertiesModels Publish

Distribution Event Non-Event Actual Event

Variable Bins Scorecard Points Count Percent Count Percent Count Percent Count Percent

Account Status Active 67 500 50 600 60 400 40

Inactive 60 500 50 400 40 600 60

ATM Credit Count 1 >= 0 < 20 42 200 20 200 20 800 80

>= 20 < 40 35 200 20 200 20 200 80

>= 40 < 60 33 200 20 200 20 200 80

>= 60 < 80 50 200 20 200 20 200 80

>= 80 < 100 49 200 20 200 20 200 80

ATM Credit Sum 10 >= 0 < 20 43 200 20 200 20 200 80

>= 20 < 40 100 200 20 200 20 200 80

>= 40 < 60 96 200 20 200 20 200 80

>= 60 < 80 49 200 20 200 20 200 80

>= 80 < 100 68 200 20 200 20 200 80

ATM Deposit Count 1 >= 0 < 20 49 200 20 200 20 200 80

>= 20 < 40 50 200 20 200 20 200 80

>= 40 < 60 33 200 20 200 20 200 80

>= 60 < 80 50 200 20 200 20 200 80

>= 80 < 100 68 200 20 200 20 200 80

ATM Deposit Count 10 >= 0 < 20 60 200 20 200 20 200 80

>= 20 < 40 93 200 20 200 20 200 80

>= 40 < 60 85 200 20 200 20 200 80

Results Sample Output Model Scorecard Variables Details

0.62Threshold:0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

- Machine Learning Project

AI & ML Hype, Reality and Results - SAS

Documents

Transcript of AI & ML Hype, Reality and Results - SAS