T22.Fujitsu World Tour India 2016-Business Intelligence and Data Analytics in Banking, Financial,...

0 Copyright 2016 FUJITSU

Fujitsu for Business Intelligence and Data Analytics Pankaj Sharma – Solution Architect – HPC & Analytics

Fujitsu World Tour 2016


Why Big Data?


Big Data

Big is a relative term – depends upon the context

When we have to deploy a cluster to do store and analyze

When we take analysis of the data to the realms of High Performance Computing

Structured and unstructured Data


Big Data continued ….

We cannot escape BigData – Either analyze in parallel or perish

Abundance of unstructured Data – difficult to use any DB

Getting the data to processor bottleneck

Parallel and Cluster computing on data


Where? Fujitsu’s Foot steps


Use Cases.....Examples

Customer Intimacy Operational Efficiency

Risk Management Innovation

Categories

Market Basket Analysis

Predictive Maintenance

Enterprise Data Hub

IoT Applications

Monitoring and Analysis of unused data

360° View

Targeted Advertisement

Sentiment Analysis

Recommendations

Cybersecurity

Fraud Detection

Compliance Auditing


Cybersecurity Target Industries: Large Public Networks, Government, Financial, Telecom, Utilities, etc.

Real Time

Detect

Attacks in

Real Time

Prevent

Threats in

Real Time


Fraud Detection Target Industries: Financial, Insurance, Government, etc.

Patterns

Words

External

Information

Internal

Information


Targeted Advertising Target Industries: Retail, Manufacturing, Service, Financial, etc.

Personalized Offers to Consumers

Data sources: • Social networks • CRM • Sales Orders • POS • Web Logs • Online profiles • etc.

Social Media

Data

Multi-

Structured

Data


Trace and interprets online discussions in social media to Gather customer feelings on: Your organization

Your Brand

Your Products and Services

Your Competition

Other events that may impact corporate performance: politics, disasters, large-scale events, etc.

Sentiment Analysis

or ?

Decide /

Improve

Target Industries: Retail, Manufacturing, Service, Financial, etc.

Like

Dislike

Want

Need


Real Time

Recommendations

Widely in the retail sector

Looks at customers’ purchase behavior of products, and offer a "best next offer" recommendation when they purchase a product

Not only checking which combinations are most likely, but also, identifying a closely related peer consumer group

Target Industries: Retail, Service, Social Networks, etc.

Web logs, Cart Data, User Data

Recommendation Profiles Web Logs


Market Basket Analysis

Match products purchased together for cross-selling, up-selling and promotions

Big data adds more context: Time of day

Music played in a store

Store visit duration

Store traffic

Weather, etc.

Target Industries: Retail

Multi-

Structured

Data

Social Media

Operational

Data

Other Data


Predictive Maintenance Target Industries: Manufacturing, Natural Resources, Utilities, Telecommunication

Real Time

Improve Operations, Reduce Waste,

Identify Bottlenecks

Control Manufactured Product Quality

Monitor Asset Performance

Avoid Operations Downtime

Predict Asset Failure

Machine / Sensor Data

Predictive Models/Machine Learning


Usage Monitoring Target Industries: Manufacturing , Natural Resources, Transport, Utilities

Monetize Data

Monitor/Control

Optimize

Machine / Sensor Data


DW Optimization

Offload and accelerate ELT/ETL workloads

Improve Storage TCO, store more data, and tackle data growth challenges

Scale-out to meet performance and/or capacity with low cost commodity hardware

Insight Enrichment

Perform analytics on all your data assets and discover new insights

Enable Enterprise Data Hub and eliminate information silos

Keep all your data assets virtually “forever”

Offer business users self-service analytics and reduce time-to-information, and development costs

Use Case: Enterprise Data Hub BI/Data Warehouse Augmentation and Optimization

Target Industries: Generic


Fujitsu Big Data Technologies


Apache Hadoop Ecosystem

Fujitsu Infra Unified, Elastic, Resilient, Secure

Workload Management (YARN)

Batch

Processing (MR, Pig, Hive)

Analytic

SQL (Hive, Impala)

Search

Engine Solr

Machine

Learning (Mahout, Spark)

Stream

Processing (Spark)

3rd Party

Apps (Datameer,

Tableau,

elasticsearch,…)

DA

TA

MA

NA

GE

ME

NT

(H

adoop, F

alc

on, O

ozie

…)

SY

ST

EM

MA

NA

GE

ME

NT

(Z

oo

ke

ep

er, A

mb

ari, …

)

File System Online NoSQL

Da

ta A

cce

ss

Secu

red

Pla

tform

Integration on open source stack (Flume, Storm, Sqoop, Falcon, WebHDFS, [Pentaho, Informatica, Other Connectors])


Cluster with Hadoop

• Holds the metadata for the HDFS NameNode

• Performs housekeeping functions for the NameNode

Secondary NameNode

• Stores the actual HDFS data blocks and runs analytic compute DataNode

• Manages MapReduce jobs JobTracker YARN

• Monitors individual Map and Reduce tasks TaskTracker


Multi-Structured Data Storage and Batch Processing

Structured Data

•Databases •Transaction •CRM •ERP •…

Unstructured Data

•Machine Data •Web Data •Social Media •Documents •E-Mails •Audio/Visual •…

Multi-Function

•Self-Service Analytics •Advanced Analytics •DW Landing + ETL Stage •Dynamic Archival •…


Real-Time/In-Memory Data Analytics

Real-Time Data Analysis Non Real-Time Data Lake

and Batch Processing

Network Logs

Web Logs

Machine Data

Market Data

Hadoop

Real-Time Data Collection


Fujitsu Big Data Professional Services


Why Fujitsu?

The world‘s fourth-largest IT services provider and No.1 in Japan*

Committed to deliver local service globally

We do everything in ICT, and work with you to deliver end-to-end solutions

Global best-practice-based delivery of Business Transformation and Information Technology Projects

A one-stop shop for Big Data where organizations can get optimized Big Data infrastructure solutions including hardware, software and services – all from a single source

PC/Mobile

Services

Networks Software

Servers


Big Data Professional Services

• FUJITSU Big Data Assessment Workshop • FUJITSU Big Data Strategy Consulting Service • FUJITSU Services for Hadoop • FUJITSU Big Data Analytics Services

Consulting Services

• FUJITSU Integrated System PRIMEFLEX for Hadoop • Other Platforms on request

Integration Services


Fujitsu Big Data Strategy Consulting Service

Offers the expertise and skills required to build your Big Data Strategy Identify and document your Big Data requirements

Identify the most promising best practices, use cases, and technologies

Assess and align Big Data to organizational assets and strategic objectives

Identify gaps

Document strategic recommendations for Big Data initiatives and their implementation plan

Service Deliverables Big Data Strategy Plan for the organization

Recommendation on the best-of-breed solution and reference architecture

Strategy implementation road map

Service description Best Practice-based service following a 5-step approach

The Benefit

Helps you identify, implement, and validate use cases and their outcomes

Remove Big Data technical and analytical skill gaps

Provide Big Data analytics development services for data transformation, analysis, and reporting

Duration: Scope Dependent

1- Requirement Definition and Project Planning

2- Organizational Assets assessment

3- Gap Analysis

4- Reference Architecture definition

5- Strategy and Road Map Documentation


Fujitsu PRIMEFLEX for Hadoop Deployment Service

Introduction to Fujitsu Integrated System PRIMEFLEX for Hadoop

Overview of the hardware and software stack

Definition and configuration of the required infrastructure resources: Servers

Network

OS

Definition of Hadoop Cluster Configuration Configuration of nodes

Configuration of networking parameters

Definition of Enterprise Integration of Hadoop Cloudera Manager cluster configuration

Base services and security

Define Testing

Definition of Datameer configuration Datameer application configuration

Base Security and Tuning

Enterprise Integration

The service delivers a formal configuration report, subsequent implementation of the configuration in the IT infrastructure environment, and concludes with a handover of the running solution to the Customer

T22.Fujitsu World Tour India 2016-Business Intelligence and Data Analytics in Banking, Financial,...

Technology

Transcript of T22.Fujitsu World Tour India 2016-Business Intelligence and Data Analytics in Banking, Financial,...