Big data using Public Cloud
-
Upload
imc-institute -
Category
Technology
-
view
717 -
download
0
Transcript of Big data using Public Cloud
![Page 1: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/1.jpg)
Big Data Using Public Cloud
Assoc. Prof. Dr. Thanachart NumnondaExecutive DirectorIMC Institute18 August 2015
![Page 2: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/2.jpg)
2
Internet of ThingsCloud Computing
Big Data
![Page 3: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/3.jpg)
3
Data created every minute
Source: Trendwise Analytics
![Page 4: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/4.jpg)
4
Data!
The New York Stock Exchange generate about1TB of new trade data per day.
A commercial aircraft generates 3GB of flightsensor data in 1 hour.
Vodafone generates 3TB of Call Detail Record(CRDs) per day.
Between 2009 and 2014, the total number of U.S.online banking households will increase from 54million to 66 million.
![Page 5: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/5.jpg)
5
Big Data
Source: http://www.datasciencecentral.com/
![Page 6: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/6.jpg)
6
![Page 7: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/7.jpg)
7
Sample Data v.s. Big Data
Can you judge a persons life expectancy?
Given:
– DNA
– Medical records
– Food
– Lifestyle (smoking, drinking, driving, exercise)
![Page 8: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/8.jpg)
8
IT Infrastructure
Analytics
Data Sources
![Page 9: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/9.jpg)
9
“B ัy 2015, 20% of Global 1000 organizationsWill have established a strategic focus on
information infrastructure ”
Gartner
![Page 10: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/10.jpg)
10
Big Data Technology !!
![Page 11: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/11.jpg)
11
Big Data Landscape
Source: Big Data in the Enterprise. When to Use What?
![Page 12: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/12.jpg)
12
A scalable fault-tolerant distributed system for data storage and processing
Completely written in javaOpen source & distributed under Apache license
What is Hadoop?
![Page 13: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/13.jpg)
13
Hadoop Environment
Source: Hadoop in Practice; Alex Holmes
![Page 14: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/14.jpg)
14
Hadoop Distribution
Microsoft Azure
![Page 15: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/15.jpg)
15
Big Data Future Architecture
Sscial Media Images e-mails Crawlers ERP CRM LOB APPs
Unstructured and Structured Data
Data Warehouse / NewSQL
Hadoop OnCloud
Hadoop OnPrivateServer
Connectors
SSRS
BI Platform
Familiar End User ToolsSpreadsheet Predictive Analytics
Data Market Place
NoSQL
Petabytes of Data(Unstructured)
Hundreds of TB of Data(structured)
![Page 16: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/16.jpg)
16
Issue with Big Data Infrastructure
Large investment
Scalabilty
ROI
Business Cases
![Page 17: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/17.jpg)
17
Big Data on Cloud
Using IaaS to leverage Cloud Vms
Using Big Data as a Services
![Page 18: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/18.jpg)
18
Big Data using IaaS
![Page 19: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/19.jpg)
19
![Page 20: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/20.jpg)
20
![Page 21: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/21.jpg)
21Source: http://www.datadansandler.com/
![Page 22: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/22.jpg)
22
Big Data Services on Cloud
Amazon Elastic Mapreduce
Microsoft Azure Hadoop
![Page 23: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/23.jpg)
23
Big Data as a Service
![Page 24: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/24.jpg)
24
Database as a Service
Amazon RDS
IBM SQL Database for Bluemix
Microsoft SQL Database
Google CloudSQL
![Page 25: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/25.jpg)
25
NoSQL as a Service
Amazon DynomoDB
Google Cloud DataStore
Microsoft Azure DocumentDB
Cloudant on IBM Bluemix.
Mongo DB on Heroku
![Page 26: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/26.jpg)
26
Amazon DynomoDB
![Page 27: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/27.jpg)
27
Hadoop as a Service
Amazon Elastic Map Reduce
Rackspace Cloud Big Data Platform
Qubole
Google Cloud Platform
IBM Bluemix: Analytic on Hadoop
Microsoft Azure HDInsight
![Page 28: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/28.jpg)
28
![Page 29: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/29.jpg)
29
![Page 30: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/30.jpg)
30
Big Data on Amazon EMR
![Page 31: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/31.jpg)
31
Amazon EMR
![Page 32: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/32.jpg)
32
![Page 33: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/33.jpg)
33
![Page 34: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/34.jpg)
34
Hadoop on Google
![Page 35: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/35.jpg)
35
![Page 36: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/36.jpg)
36
Analytic as a Service
Google Big Query
Amazon Machine Learning
Azure Machine Learning
BIME: BI as a Service
IBM Watson Analytic
![Page 37: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/37.jpg)
37
![Page 38: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/38.jpg)
38
![Page 39: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/39.jpg)
39
Google BigQuery
![Page 40: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/40.jpg)
40
Big Data on Cloud Roadmap
Step 1: Build the business case
Step 2: Assess your Big Data applicationworkloads
Step 3: Develop a technical approach fordeploying and managing Big Data in the cloud
Step 4: Address governance, security, privacy,risk,
Step 5: Deploy, integrate, and operationalizeyour cloud-based Big Data infrastructure
Source : Deploying Big Data Analytics Applications to the Cloud: Roadmap for Success: CSCS
![Page 41: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/41.jpg)
41
Sample applications
Enterprise applications already hosted in thecloud
High-volume external data sources thatrequire considerable preprocessing
Tactical applications beyond your on-premises, Big Data capabilities
Elastic provisioning of very large but short-lived analytic sandboxes
Source : Deploying Big Data Analytics Applications to the Cloud: Roadmap for Success: CSCS
![Page 42: Big data using Public Cloud](https://reader031.fdocuments.net/reader031/viewer/2022032700/55d6de35bb61eb53538b47a7/html5/thumbnails/42.jpg)
42
www.facebook.com/imcinstitute