AIOps深度案例分享 - elasticsearch · AIOps深度案例分享 Elastic Meetup 成都分享会 [email protected] &
Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.
-
Upload
leona-campbell -
Category
Documents
-
view
411 -
download
0
Transcript of Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.
![Page 1: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/1.jpg)
Microsoft Big Data 解決方案與案例分享
DBI 312
Rich HoTechnical Architect微軟技術中心
![Page 2: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/2.jpg)
Agenda
• What is Big Data?• Microsoft Big Data Strategy• Key Benefits of Microsoft Big Data• Demo• Case Study
![Page 3: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/3.jpg)
What is Big Data ?
![Page 4: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/4.jpg)
By 2015, organizations that build a modern information management system will outperform their peers financially by 20 percent.
– Gartner, Mark Beyer, “Information Management in the 21st Century”
10x increase every five years
85% from new
data types
Dataexplosion
VolumeVelocityVariety
Easy Accessibility of External Data
Hadoop
Cloud
Cheap, Distributed Storage &
Processing
The world of Data is changing
![Page 5: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/5.jpg)
Three V in Big Data
Relational Data
10x increase every 5 years
VOLUME
4.3 connected devices per adult
VELOCITY
VARIETY
85% from new data types
Big Data
![Page 6: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/6.jpg)
We are moving to Big Data
Any Data, Any Size, Anywhere
Traditional Data
Big Data
Data Volume Petabytes+Terabytes
Data Variety UnstructuredStructured
Data Velocity StreamingBatch
Garbage? +++++-----
![Page 7: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/7.jpg)
With Big Data, says:
以前我們將尿布和啤酒放在一起而創造新業績
現在我們將比老爸更早一步知道他女兒懷孕了 !
![Page 8: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/8.jpg)
Traditional data flow
![Page 9: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/9.jpg)
New exploratory data flow
![Page 10: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/10.jpg)
What is Big Data?
Increasing Data Volumes
Increasing Data & Analysis Complexing
Emerging Technologies
![Page 11: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/11.jpg)
Microsoft Big Data Strategy
![Page 12: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/12.jpg)
Microsoft Big Data Stacks
NON-RELATIONAL
100111
DATA MANAGEMENT
SHAREAND GOVERN
DISCOVERAND RECOMMEND
TRANSFORMAND CLEAN
INSIGHTS
DATA ENRICHMENT
OPERATIONAL
SELF-SERVICE MOBILE
PREDICTIVE
REAL-TIMECOLLABORATIVE
MA
RK
ETPLA
CE
Exte
rnal D
ata
and S
erv
ices
RELATIONAL MULTIDIMENSIONAL STREAMING
![Page 13: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/13.jpg)
Fast
L
oad
Source Systems
Historical Data(Beyond Active Window)
Summarize & Load
Big Data Sources (Raw, Unstructured) Data & Compute
Intensive Application
ERPCRM LOB
APPS
Integrate/Enrich
Enterprise Data Warehouse
Enterprise ETL with SSIS, DQS, MDS
Distributed environments for Non-structured data
Data Marts
Reporting
Multi-dimensional Analysis
Business Insights
Interactive Reports
Performance Scorecards
Crawlers
Bots
Devices
Sensors
Online Transaction
Enterprise Big Data Blueprint
![Page 14: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/14.jpg)
Hadoop on Windows
Hadoop on Azure
![Page 15: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/15.jpg)
Fast
L
oad
Source Systems
Historical Data(Beyond Active Window)
Summarize & Load
Big Data Sources (Raw, Unstructured) Data & Compute
Intensive Application
ERPCRM LOB
APPS
Integrate/Enrich
Hadoop on Windows Azure
Hadoop on Windows Server
Business Insights
Interactive Reports
Performance Scorecards
Crawlers
Bots
Devices
Sensors
Enterprise Data Warehouse
Data Marts
Reporting
Multi-dimensional Analysis
Enterprise ETL with SSIS, DQS, MDS
Online Transaction
Microsoft Big Data Platform
![Page 16: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/16.jpg)
Fast
L
oad
Source Systems
Historical Data(Beyond Active Window)
Summarize & Load
Big Data Sources (Raw, Unstructured) Data & Compute
Intensive Application
ERPCRM LOB
APPS
Integrate/Enrich
Hadoop on Windows Azure
Hadoop on Windows Server
Business Insights
Interactive Reports
Performance Scorecards
Crawlers
Bots
Devices
Sensors
Enterprise Data Warehouse
Data Marts
Reporting
Multi-dimensional Analysis
Enterprise ETL with SSIS, DQS, MDS
Online TransactionSQL Server
EE
SQL Server PDW
SQL Server FT
Microsoft Big Data Platform
![Page 17: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/17.jpg)
Fast
L
oad
Source Systems
Historical Data(Beyond Active Window)
Summarize & Load
Big Data Sources (Raw, Unstructured) Data & Compute
Intensive Application
ERPCRM LOB
APPS
Integrate/Enrich
Hadoop on Windows Azure
Hadoop on Windows Server
Business Insights
Interactive Reports
Performance Scorecards
Crawlers
Bots
Devices
Sensors
Enterprise Data Warehouse
Data Marts
Reporting
Multi-dimensional Analysis
Enterprise ETL with SSIS, DQS, MDS
Online Transaction
SQL Azure
Microsoft Big Data on Public Cloud
![Page 18: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/18.jpg)
Hadoop on Azure (CTP)
![Page 19: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/19.jpg)
Request a New Cluster (4 nodes, 2 TB)
![Page 20: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/20.jpg)
Allocating NN & DN
![Page 21: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/21.jpg)
Starting NN & DN
![Page 22: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/22.jpg)
NN & DN ready! (after 9 mins)
![Page 23: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/23.jpg)
Hadoop on Azure ready! (after 13 mins)
![Page 24: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/24.jpg)
Access Hadoop on Azure through RDP
![Page 25: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/25.jpg)
Key Benefits of Microsoft Big Data
![Page 26: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/26.jpg)
Microsoft Big Data Benefits
• New Hadoop-based distribution on Windows / Azure
• Hive ODBC driver
• Hive add-in for Excel
Broader Access of Hadoop
• Analyze Big Data with familiar tools (Excel, PowerPivot, Power View)
• JavaScript based simple programming
Breakthrough Insights
• System Center Integration
• Active Directory Integration
Enterprise Ready
![Page 27: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/27.jpg)
Breakthrough Insights
Hive ODBC Driver integrates Hadoop to SQL Server Analysis Services, PowerPivot, and Power View
Breakthrough Insights
Benefits
Key
Featu
res
![Page 28: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/28.jpg)
Broader Access of Hadoop
Hive add-in for Excel
Hive ODBC driver
Broader Access of Hadoop
Benefits
Key
Featu
res
![Page 29: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/29.jpg)
Enterprise Ready (SCOM Integration)
System Center to monitor performances of Head nodes and Data nodes
Enterprise Ready
Benefits
Key
Featu
res
![Page 30: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/30.jpg)
1. Hadoop on Azure - JavaScript Console - Job Execution2. Excel Hive Connectivity3. Windows Marketplace & Hadoop on Azure Integration
![Page 31: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/31.jpg)
Case Study: Mining Big Mobility Data
![Page 32: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/32.jpg)
Microsoft Big Data Case Study: T-Drive
![Page 33: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/33.jpg)
Microsoft Big Data Case Study: T-Drive
![Page 34: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/34.jpg)
Video
Microsoft Research AsiaBig Data Case Study
![Page 35: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/35.jpg)
Thank You
![Page 36: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/36.jpg)
Connect. Share. Discussshttp://www.microsoft.com/taiwan/techdays2012/
Microsoft Certification & Training Resources
http://www.microsoft.com/learning/zh/tw/
Resources for IT Professionals
http://social.technet.microsoft.com/Forums/zh-tw/categories
/
Resources for Developershttp://social.msdn.microsoft.com/Forums/zh-tw/categories
/
Resources
![Page 37: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/37.jpg)
![Page 38: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/38.jpg)
請協助完成「本課程問卷」,並在離開教室時交給工作人員!
填妥大會背包中的大會問卷,可於活動第三天兌換問卷禮哦!
感謝您的合作。
![Page 39: Microsoft Big Data 解 決方案與案例分享 Rich Ho Technical Architect 微軟技術中心.](https://reader033.fdocuments.net/reader033/viewer/2022061504/56649e8f5503460f94b940ba/html5/thumbnails/39.jpg)