Hunting criminals with hybrid analytics -- October 2015
-
Upload
seattle-daml-meetup -
Category
Engineering
-
view
337 -
download
3
Transcript of Hunting criminals with hybrid analytics -- October 2015
David Talby
@davidtalby
SVP Engineering, Atigeo
HUNTING CRIMINALS WITH HYBRID ANALYTICS, SEMI-SUPERVISED LEARNING, AND AGENT FEEDBACK
Claudiu Branzan
@melcutz
Principal Data Science Lead, Atigeo
2
3
IT’S NOT (JUST) ABOUT MONEY
33
4
WHAT WE’RE UP AGAINST
44
50+Schemes(and counting)
99.9999%‘Good’ messages
6+Monthsper case
Needle in a haystack
Hybrid analytics
No training data
Semi-supervised learning
Adversarial learning
Online feedback
5
WHY HYBRID ANALYTICS?
55
Ignore more rules
Unusual timing of
eventsUnusualpersonal network
Teamwork & scale
Think & talk differently
6
(BITS OF) THE TOOLBOX
66
Rule Inference
Time Series
AnalysisLink Analysis
Ensemble Learning
Natural Language
7
CAN WE SEE SOME CODE PLEASE?
7 7
Freely available IPython notebooks
Open source libraries & open data
Jump-start via AWS Marketplace
8
STREAM PROCESSING
88
Kafka
Email Stream
Account transactions Stream
Email NLP Features
People graph
Transactions time series
9
SAMPLE EMAIL PATTERNS
1 0
SAMPLE NATURAL LANGUAGE ANNOTATORS
Understand vocabulary
– Jargon
– Code words
– Multi-lingual
Understand grammar
– Who are we talking about?
– Past, present or future?
– Compound sentences
Understand context
– Email: Re:, Fwd:, attachments
– SMS & IM have their own grammar
1 1
SAMPLE GRAPH FEATURESStandard algorithms like KMeans don’t work on “haystacks”
1 2
SAMPLE GRAPH FEATURESBregman Bubble Clustering
1 3
USER ANALYSIS ITERATION
Email NLP Features
User graph
Transactionstime series
Graph Features
Time SeriesFeatures
NLP Features
Agent Feedback
Trai
n /
Tes
t C
lass
ifie
r
1 4
Really
• Makes the world a better place • Needle in a very large haystack– Actually needs a petabyte-scale platform
• Multi-modal: no single trick works– Hybrid analytics
• No labeled data– Semi-supervised learning
– Cold start problem
• Sparse & high-dimensional
– Graph based features & change over time
• Adversarial– Feedback & online learning
Technically
SUMMARY: WHY HUNTING CRIMINALS IS COOL
1414
1 5 15
@Atigeo
@melcutz
@davidtalby
© 2015 Atigeo, Corporation. All rights reserved. Atigeo and the xPatterns logo are trademarks of Atigeo. The information herein is for informational purposes only and represents the current view of Atigeo as of the date of this presentation. Because Atigeomust respond to changing market conditions, it should not be interpreted to be a commitment on the part of Atigeo, and Atigeo cannot guarantee the accuracy of any information provided after the date of this presentation. ATIGEO MAKES NO WARRANTIES,EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
APPENDIXIn case the live demo gets cold feet on stage
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6