AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS....
Transcript of AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS....
![Page 1: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/1.jpg)
AI- and Data-Driven Support
For FINTECH
Beng Chin OOI
www.comp.nus.edu.sg/~ooibc
0
![Page 2: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/2.jpg)
Contents
• AI and QoS
• GIT for Data: Facilitating AI Services without Data Sharingo Our Software Stack
• Financial News Analytics o An Example on using AI/ML
1
![Page 3: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/3.jpg)
MAS Fintech Challenges
Integrated cross platform payments
Seamless season parking fee payments
Cashless schools
Secure digital authentication
Common industry APIs
Engaging investor education
Aggregated financial statements manager
Automated insurance claims
Mobile banking for visually impaired
Smart portfolio management
Robot-advisor 2.0
Data-driven investment solutions
Fraudulent behaviour detection
and prediction
![Page 4: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/4.jpg)
The Reality of Exploiting AI
The actual implementation of the ML algorithm is usually less than 5% lines of code in a real, non-trivial application
The main effort (i.e. those 95% LOC) is spent on:Data cleaning & annotationData extraction, transformation, loadingData integration & pruningParameter tuningModel training & deployment… …
This blurs the line between DB and “non-DB” processing, and calls for better integration
These are what we have been doing!
![Page 5: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/5.jpg)
The BIG Data Analytics Pipeline*
AcquisitionExtraction/Cleaning/
AnnotationIntegration
Interpretation/Visualization
Analytics/Modeling
Data Science
Application of AI/ML
Big Data*Alexandros Labrinidis, H. V. Jagadish:Challenges and Opportunities with Big Data. PVLDB 5(12): 2032-2033 (2012)
![Page 6: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/6.jpg)
AI
AI
![Page 7: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/7.jpg)
In the nutshell
AITraining Prediction
![Page 8: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/8.jpg)
Preparation and QoS
AITraining PredictionData Cleansing
QoSMonitoring
![Page 9: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/9.jpg)
The Workflow
AITraining PredictionData Cleansing
QoSMonitoring
Dat
a Co
llect
ions
Query Interactions
![Page 10: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/10.jpg)
AITraining PredictionData Cleansing
QoSMonitoring
Dat
a Co
llect
ions
Query Interactions
Original Data
TrainingData
AI Model
Prediction History
QoSStatistics
![Page 11: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/11.jpg)
Scaling AI
AITraining PredictionData Cleansing
QoSMonitoringD
ata
Colle
ctio
nsQ
uery Interactions
Original Data
TrainingData
AI Model
Prediction History
QoSStatistics
How to coordinate collaborative data cleansing?
How to support machine/deep learning under privacy concern?
How to preform timely quality control for online prediction?
How to manage data for multi-user/application?
![Page 12: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/12.jpg)
GIT FOR DATAForkCloud
11
![Page 13: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/13.jpg)
Data and Learning
• Due to PDPA/GDPR, data is not widely available
• Even if it is stored in the cloud, data cannot be seen by
external AI experts and developers
• Without data, models cannot be well designed or trained
!!
• No Free Lunch Theorem [1997]• there is no useful model/algorithm for all the data distributions
![Page 14: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/14.jpg)
Big Data Analytics in Banking
Customer Experience
Operations Optimization
Scattered Data
Risk Analytics
No Single View of the Customer
Fraud Identification Governance
![Page 15: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/15.jpg)
Risk Analytics
Banking Data
External Information
Third-party Analytics
• Customer basic information• Customer asset status• Customer transaction history• Customer trading pattern (e.g., trading
accounts, trading frequency, …)• Customer credit history• ……
![Page 16: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/16.jpg)
Risk Analytics
Banking Data
External Information
Third-party Analytics
• Customer basic information• Customer asset status• Customer transaction history• Customer trading pattern (e.g., trading
accounts, trading frequency, …)• Customer credit history• ……
![Page 17: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/17.jpg)
Facilitating AI Services
Git for data• Data-driven services to facilitate development of domain-specific AI
solutions• Data cleansing, curation and integration services• Data versioning to facilitate collaboration, data provenance and
improvement of quality data
Applications / Target market• “Git” for data – cleaning and curation, privacy-preserving crowdsourcing for
service, data protection and provenance, data immutability• AI-as-a-Service in the cloud• Tools for data scientists and business analysts to conduct analytics on
sensitive data
![Page 18: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/18.jpg)
The ForkCloud Stack
ForkBaseGit for Data
Data Store
Data Cleansing
AI Development
AI Service
Access Control
![Page 19: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/19.jpg)
Software Stack and Architecture
ForkBase
DICEData Cleansing & Integration
Apache SINGADeep Learning
CohAnaCohort Analysis
RafikiAutomatic
Machine/Deep Learning Service
PANDAAI as a Service
Access Control
CDASCrowdsourcing
Data Owner Data Processing Vendor
Analytics Provider
![Page 20: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/20.jpg)
NUHS DISCOVERY AI Sandbox
MASTERMAPPER
MAPFORCEAUTO
General complications model
Lab test model
ForkBaseWorking storage
LSI database
CSIdatabase
Extracted trial data
AUGURIUM
Readmissions
Disease Progression Model
Pre-Processing layerExpandable storage
SPH
CSI
LSI
SDSD
REDCAP
I2B2
Database sector
CDOC|CCDR
TissueRepositor
y
LSI
CSI
SPH
Active sector
AuguriumLearningdatabase
Learning sector
SPHdatabase
RA learningdatabase
DPMlearningdatabase
![Page 21: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/21.jpg)
Financial News Analytics For fintech applications
![Page 22: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/22.jpg)
STOCK PRICE VS BREAKING NEWS
![Page 23: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/23.jpg)
This Photo by Unknown author is licensed under CC BY-SA.
Risk
![Page 24: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/24.jpg)
Risk
![Page 25: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/25.jpg)
Opportunity
This Photo by Unknown author is licensed under CC BY-SA-NC.
![Page 26: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/26.jpg)
Opportunity
![Page 27: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/27.jpg)
Over 30,000 news publishers indexed by newsapi.orgImage from https://newsapi.org/sources
TOO MANY NEWS ARTICLES/PUBLISHERS
![Page 28: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/28.jpg)
Many articles
![Page 29: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/29.jpg)
This Photo by Unknown author is licensed under CC BY-SA.
Few articles
![Page 30: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/30.jpg)
Tasks
News deduplication
Knowledge graph construction
Risk/opportunity discovery
Stock price/index prediction
![Page 31: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/31.jpg)
Clustering if many articles
Knowledge graph if few articles
Risk/opportunity prediction
![Page 32: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/32.jpg)
News deduplication by clustering
• Document Clustering• Extract representation of the article/document
• TF.IDF, word2vector -> Document vector, etc.• Apply clustering algorithms
• K-means, DBSCAN, etc.
• Challenges• Too many articles are generated every day -> Big data• Clustering algorithms are expensive
• Hierarchical clustering• Identify articles of the same company• Apply clustering algorithms over the articles about the same company
![Page 33: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/33.jpg)
Hierarchical clustering
News deduplication by clustering
![Page 34: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/34.jpg)
Knowledge Graph• Knowledge
• Relation tuples, (A, relation, B).• Tim Cook, CEO of Apple, has attended the meeting. (Tim Cook, CEO,
Apple)• Foxconn that makes Apple’s iPhones… (Foxconn, Supplier, Apple)
• Application • News extrapolation
• (Person A, CEO, Company B), (Company B, Supplier, Company C)• If the big company C has risks, then the small company B is likely to have
troubles • Stock price/index analysis
• The drop of price for Company C would affect that of Company B
![Page 35: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/35.jpg)
Knowledge Graph Construction• Subtasks
• Named Entity Recognition• Locate and classify named entity mentioned in unstructured text into
pre-defined categories, such as person, organizations, locations...• e.g., [John]person bought 500 shares of [Apple Inc.]organization In [2015]time.
• Entity Linking• Link the mention into the entity in the knowledge base.• e.g., ”Paris is the capital of France” Paris is referring to this entity.
• Relation Prediction• Classify the semantic relationship between entities• e.g., "Yesterday, Foo Inc. announced their acquisition of Bar Corp." ->
(Foo Inc., acquire, Bar Corp)
![Page 36: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/36.jpg)
Knowledge Graph Construction
• NLP is challenging• Ambiguous words
• e.g., Apple & apple• Uppercase words impacts
• e.g. A and B Jointly Developed … "B Jointly Developed" can be mistakenly recognized as one entity
• Multiple relations in one sentence• e.g., Mary married with a guy, who is the chairman of Bar Corp.
![Page 37: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/37.jpg)
Knowledge Extraction --- Deep Learning
• Supervised learning• Select the interested relations
• Key Person (e.g., employee, founder, CEO, chairman, head, etc)• Related Company (e.g., supplier, partner, competitor, etc)• Location
• Select relevant entities• PERSON, LOCATION, ORGANIZATION
• Train neural networks that map (sentence, entities) -> relation• Bill Gates is the founder of Microsoft. -> (Bill Gates, founder,
Microsoft)• Mondrian takes 6.6% stake in Sheng Siong (Mondrian,
shareholder, Sheng Siong)
![Page 38: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/38.jpg)
Knowledge Extraction --- DL Data Flow
![Page 39: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/39.jpg)
Knowledge Extraction --- Distant Supervision
• Steps• Find text (sentences) discussing the entities of a relation
• Zuckerberg says he won't step down as Facebook CEO • Apple’s Tim Cook says tech regulation ‘inevitable’…
• not talking about the CEO relation noisy sample• Learn over the noisy text to correlate
• the descriptions, e.g., step down, CEO, Facebook• the relation, CEO
• Infer the relation of entities in new sentences• Microsoft CEO Satya Nadella sells 30 percent of common stock
• Returns: Satya Nadella, CEO, Microsoft
![Page 40: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/40.jpg)
Risk/opportunity detection
• Sentiment analysis• Too general
• Risk• Rating downgrade• Sanctions• More
• Opportunity• New market• New product• More
text labelUnilever to increase investment in Greece New
market/opportunityJapan cryptocurrency exchange plans Europe, Asia sales hubs
New market/opportunity
Trump administration unveils sanctions aimed at starving North Korea of resources
Sanctions/risk
Why Did Maxim Group Downgrade Domino’ Pizza? Rating downgrade/risk
![Page 41: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/41.jpg)
Risk/opportunity detection
• Document classification
• Annotate the articles as training data• Expensive• Inconsistency between annotators
• Data cleaning and pre-processing is important• Cluster articles for fast annotate• Rule-based annotation• …
• Deep learning models for classification• CNN, RNN• BERT (bidirectional encoder representations from
transformers)
![Page 42: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/42.jpg)
Stock Prediction
• Traditional solutions for stock prediction are based on time-series models.
• With the recent success of deep neural networks in modeling sequential data, deep learning has become a promising choice for stock prediction.
41
![Page 43: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/43.jpg)
Related Work
42
Topic Type of Data Number of stocks
Temporal Relational Ranking for Stock Prediction - Feng et al (2019)
Stock selection Stock Prices 1700
A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction – Yao et al (2017)
Index Prediction Stock Prices 100
Deep Learning for Event-Driven Stock Prediction - Xiao et al (2015)
Index Prediction News and Stock Prices 500
Stock price prediction via discovering multi-frequency trading patterns – Zhang et al (2017)
Index Prediction Stock Prices 50
![Page 44: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/44.jpg)
Optimized Trade Execution
The goal: is to sell (respectively, buy) V shares of a given stock within a fixed time period (or horizon) H, in a manner that maximizes the revenue received (respectively, minimizes the capital spent).
Trade-offs: such as between the speed of execution and the prices obtained for the shares.
Virtually every strategy's profitability will depend on how well it is implemented.
44
![Page 45: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/45.jpg)
Methodology
45
![Page 46: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/46.jpg)
Experiment results
46
![Page 47: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/47.jpg)
Fraud Detection
47
![Page 48: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/48.jpg)
Summary
• AI has provided great improvement in prediction on a
variety of applications, including high stack applications• However, unlike games, machines cannot learn by playing out every
possibility, and also, there are just too many external factors
• AutoML, NAS, Model Verification have reduced the
barrier for adoption, but a lot more needs to be done
• Data is a new oil, and data quality determines its value
but, sensitive data is problematic to handle• GIT for Data for data provenance and governance, and supporting AI
development without disclosing the data48
![Page 49: AI- and Data-Driven Support For FINTECH · Scaling AI Training AI. Prediction. Data Cleansing. QoS. Data Collections. Monitoring. Query Interactions. Original Data. Training Data.](https://reader035.fdocuments.net/reader035/viewer/2022062415/5fd0722b9cd7235cdf278f19/html5/thumbnails/49.jpg)
49
Thanks!