Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases
-
date post
21-Oct-2014 -
Category
Technology
-
view
3.699 -
download
0
description
Transcript of Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases
![Page 1: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/1.jpg)
1
Finance and Manufacturing Big Data Use Cases and Solutions
Sanjay SharmaPrincipal Architect
August 2013
![Page 2: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/2.jpg)
Impetus Big Data Services
2
Big Data Platform Implementation Operations and Visualization
Business Analyticsand Data Science
Solution Architecture, POC and Production planning
Technology strategy, Use Case development & Validation
BUSINESS PROCESS
MANAGEMENT
Assessment
Solution Modeling
Solution Analysis
People, Process,
Technology Impact
Analyze & Optimize
Objectives & Strategy Model
![Page 3: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/3.jpg)
© 2013 Impetus Technologies
Big Data : Value Drivers
![Page 4: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/4.jpg)
© 2013 Impetus Technologies
Financial World: Key Challenges
Business Challenges
• Fraud• Regulatory
Compliances• Customer Insights• Risk Handling
Technical Challenges
• Reduce IT costs• Do more with
LESS• Unstructured data
![Page 5: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/5.jpg)
© 2013 Impetus Technologies
Financial World: Some Use Cases
Fraud Detection and
Analysis
Risk Management
Customer Insights/Mana
gement
Micro targeting
Trade/Payments Analytics
Long Term ‐Storage & Analytics
New Opportunities
Reputational Risk
Marketing Campaigns
![Page 6: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/6.jpg)
© 2013 Impetus Technologies
Financial World: Technical Solution Challenges
• Regulatory Compliances• Machine Learning based supervised and
unsupervised analytics
Large Data Storage and Advanced
Analytics
• Mash up structured and unstructured data• Enrich TX with NLP/Text analyticsUnstructured Data
• Customer level Profiling/ Recommendations
• Transaction/Trade Ticker level analytics
Individual Level Analytical Processing
![Page 7: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/7.jpg)
© 2013 Impetus Technologies
Big Data Solution: Building Blocks
![Page 8: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/8.jpg)
© 2013 Impetus Technologies
Big Data Use Case Solution: Example
• Major Credit Card Company• Traditionally Oracle/ DB2 + SAS +
Informatica• Huge data– Existing solutions/ infrastructure
inadequate to meet new business requirements and data growth cost effectively
– Successfully revamping to Big Data Platform
![Page 9: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/9.jpg)
© 2013 Impetus Technologies
Big Data Use Case Solution: Example
Step 1: Hadoop POC with Sentiment
Analysis
Step 2: Hadoop Distribution Selection
Step 3: Production-ized Recommendation
Engine
Step 4: Production-ized Negative Fraud
Detection ($100+mlln)Step 5: …
Hive
Hadoop Platform
HBase
Oozie
Mahout + NLP
Transactions
Unstructured Sources
(Email, Surveys, CRM notes, Social Feeds etc.)
Sqoop
Flume
MapRed
BI/ Visualization
Analytics- SAS/R
Real Time DeliveryRDBMS/API
RDBMS/MPP
Informatica
ETL/ELTL
Advantages= PB scale= ETL and Analytical Offloading= Prescriptive Analytics than Predictive
![Page 10: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/10.jpg)
© 2013 Impetus Technologies
Manufacturing Domain: Challenges
Business Challenges
• Early preventive maintenance/repair• Real time/ near real time
actionable response• Improve productivity/Margin
Recovery• Reduce wastes, improve
efficiency• Improve Yield
• Supply Chain• Optimize supply chain
Technical Challenges
• High Ingestion Rates• Sensor/ tool data with sub
second ingestion requirements
• Millions of read/writes per second
• Complex log formats• Semi-structured data
• Huge amount of data• TB/PB of data storage for
deeper analytics
![Page 11: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/11.jpg)
© 2013 Impetus Technologies
Manufacturing Domain: Architectural Building Blocks
Real Time AnalyticsStreaming Analytics + CEP
Hadoop
Advanced Analytics
Rules and Alerts Machine Learning
Loopback for RuleEnhancement/ Enrichment
NoSQL + Search
Machine Data
![Page 12: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/12.jpg)
© 2013 Impetus Technologies
Machine Data
Ingestion Engine
(Real time + Batch
components)
Real Time Processing
Engine(CEP/Analytics/
Rule Engine)
Real Time Data
Storage Engine
(Store + Indexing/Searc
h)
Business Process Engine(Business Process+ Rule management)
Kafka/ Storm Storm + Esper Cassandra + Solr
JBoss Drools/jBPM
Manufacturing Domain: Data Ingestion/ Streaming – Customer Example
![Page 13: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/13.jpg)
© 2013 Impetus Technologies
HBase + Elastic SearchSpark Streaming
Hadoop
Hive/PIG/Shark
Rules and Alerts Machine Learning
Loopback for RuleEnhancement/ Enrichment
NoSQL + Search
Machine Data
Manufacturing Domain: Reference Architecture-Cloud– Customer Example
![Page 14: Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5446f33dafaf9f51178b481f/html5/thumbnails/14.jpg)
© 2013 Impetus Technologies
• Software Solutions and Services Company• Leader in Innovation led Technology services• 17 years of customer success, 1500 people
across US/ India• Big Data, Enterprise Mobility, Test and
Performance Engineering, Carrier Grade Large Systems
• Vendor neutral, open source contributor with Big data accelerator products
Impetus Technologies