Introduction to Big Data. Reference: What is “Big Data”?What is “Big Data”?
Big data taiwan_supply_chain_and_communities_20130912
-
Upload
jazz-yao-tsung-wang -
Category
Documents
-
view
288 -
download
0
description
Transcript of Big data taiwan_supply_chain_and_communities_20130912
![Page 1: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/1.jpg)
台灣台灣 Big DataBig Data 產業供應鏈產業供應鏈
與人才社群發展現況分析與人才社群發展現況分析Big Data in Taiwan : Supply Chain and Communities
Jazz Yao-Tsung Wang王耀聰 王耀聰 <[email protected]><[email protected]>
Co-founder of Hadoop.TWFree Software Lab,STD,NCHC,NARL,Taiwan
![Page 2: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/2.jpg)
2
WHO AM I ? WHO AM I ? 這傢伙是誰啊這傢伙是誰啊 ? JAZZ ?? JAZZ ?• 講者介紹:
– 國網中心 王耀聰 副研究員 /交大電控八九級碩士
– 今天我是以台灣 Hadoop社群傳教士的身份發言– [email protected]
• 所有投影片、參考資料與操作步驟均在網路上– http://trac.nchc.org.tw/cloud
– 由於雲端資訊變動太快,愛護地球,請減少不必要之列印。
FOSS 使用者Debian/Ubutnu
Access GridMotion/VLC
Red5Debian Router
DRBL/ClonezillaHadoop
推廣者
DRBL/ClonezillaPartclone/TuxbootHadoop Ecosystem
行動力薄弱的開發者
TRTC WSU/Haduzilla /
Hadop4Win / Ezilla
![Page 3: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/3.jpg)
3
Supply Chain of Cloud Computing雲端產業的產業供應鏈雲端產業的產業供應鏈
![Page 4: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/4.jpg)
4
Relation of different departments !! Relation of different departments !! 各科系人才如何供給雲端產業各科系人才如何供給雲端產業
基本硬體基本硬體建設組件建設組件基本硬體基本硬體建設組件建設組件
資料中心資料中心機房維運機房維運資料中心資料中心機房維運機房維運
軟體服務軟體服務供應商供應商軟體服務軟體服務供應商供應商
行動裝置行動裝置共通平台共通平台行動裝置行動裝置共通平台共通平台
應用軟體應用軟體供應商供應商應用軟體應用軟體供應商供應商
\
\
\ 電機電機電機電機
資工資工資工資工
機械機械機械機械 土木土木土木土木
資工資工資工資工資管資管資管資管
資管資管資管資管統計統計統計統計
電機電機電機電機 資工資工資工資工
資工資工資工資工 資管資管資管資管美術美術美術美術 其他其他其他其他
![Page 5: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/5.jpg)
5
巨量資料的奇幻漂流 Life of Big Data
![Page 6: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/6.jpg)
6
Supply Chain of Big Data Industry巨量資料產業的上下游供應鏈
Open Data
Storage
MapReduce
Query
Web 2.0
Mobile
IoT
Analytics
處理巨量資料的資訊架構SMAQ (Storage, MapReduce and Query)
參考來源: The SMAQ stack for big data , Edd Dumbill , 22 September 2010 , http://radar.oreilly.com/2010/09/the-smaq-stack-for-big-data.html
![Page 7: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/7.jpg)
7
Big Data 很重要,但該去那兒找人才?
美國軟體就業市場分析,根據indeed與 simply hired兩間公司的趨勢觀察,都得到一樣的結果:Big Data > Cloud Computing > Hadoop > NoSQL
Big Data
Cloud Computing
![Page 8: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/8.jpg)
8
自由軟體社群最重要的是「人」!產業供應鏈也一樣!
Community EvangelistConnecting PeopleConnecting Values
< Mission >Connecting People to
build a Value Supply Chain
![Page 9: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/9.jpg)
9
巨量資料的社群關聯 Relations between Communities
Hadoop
![Page 10: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/10.jpg)
10
Hadoop related Facebook Group in Taiwan
2013/09/04 Hadoop.TW 986 members
Hadoop.TWhttps://www.facebook.com/groups/hadoop.tw
![Page 11: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/11.jpg)
11
Hadoop related Facebook Group in Taiwan
2013/09/04 Hadoop in Taiwan 118 members
Hadoop in Taiwanhttps://www.facebook.com/groups/hadoopintaiwan/
![Page 12: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/12.jpg)
12
巨量資料的社群關聯 Relations between Communities
上游連結資料集來源
![Page 13: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/13.jpg)
13
Open Data Taiwan Facebook Group
2013/09/04 Open Data Taiwan 614 members
OpenData / Taiwan ( ODTWN )https://www.facebook.com/groups/odtwn/
![Page 14: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/14.jpg)
14
巨量資料的社群關聯 Relations between Communities
下游連結資料應用
![Page 15: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/15.jpg)
15
Taiwan R User Group @ Meetup
Taiwan R User Group http://www.meetup.com/Taiwan-R/
2013/09/04 Taiwan R User 384 members
![Page 16: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/16.jpg)
16
NoSQL Taiwan Facebook Group
2013/09/04 NoSQL Taiwan 1,343 members
NoSQL Taiwanhttps://www.facebook.com/groups/306552142710977/
![Page 17: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/17.jpg)
17
NoSQL & Big Data Architecture
2013/09/04 NoSQL-BigData 230 members
NoSQL & BigData Architecturehttps://www.facebook.com/groups/423848814337101/
![Page 18: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/18.jpg)
18
HBase Taiwan Facebook Group
2013/09/04 HBase.TW 134 members
HBase 小聚https://www.facebook.com/groups/289369481132604/
![Page 19: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/19.jpg)
19
MongoDB Taiwan Facebook Group
2013/09/04 MongoDB.TW 267 members
MongoDB 小聚https://www.facebook.com/groups/142553245867411/
![Page 20: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/20.jpg)
20
JavaScript Taiwan Facebook Group
2013/09/04 JavaScript.TW 4,328 members
JavaScript.TWhttps://www.facebook.com/groups/javascript.tw/
![Page 21: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/21.jpg)
21
Supply Chain of Big Data Industry
JavaScript.TW 4328 members
NoSQL Taiwan 1343 members
Open Data Taiwan 614 members
Hadoop Taiwan 986 members
Open Data
Storage
MapReduce
Query
Web 2.0
Mobile
IoT
Analytics Taiwan R User 384 members
![Page 22: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/22.jpg)
22
SNA of Big Data Communities 巨量資料社群的群聚分析
node = facebook id每個點代表一個臉書帳號
edge = membership每一邊代表社群歸屬
SocialNetworkAnalysis SNA @ 2013/03/22
![Page 23: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/23.jpg)
23
SNA of Big Data Communities 巨量資料社群的群聚分析
SNA @ 2013/08/31
5 個月以後,OpenData.TW 人數翻了 5 倍
與 NoSQL.TW跟 Hadoop.TW的連結也變強!
![Page 24: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/24.jpg)
24
SNA of Big Data Communities 巨量資料社群的群聚分析
JavaScript.TW 人數非常龐大!代表台灣有很多人對前端工程有興趣。
NoSQL.TW
Hadoop.TW
OpenData.TW
巨觀來看,對於資料庫有興趣的人數比較高!
SNA @ 2013/08/31
玩 OpenData 的人不少都需要前端視覺化的技術。
跨領域人才?
玩後端的人比例上仍舊比較少。
![Page 25: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/25.jpg)
25
結 語 Conclusion
Open Data
Storage
MapReduce
Query
Web 2.0
Mobile
IoT
JavaScript.TW 4328 members
NoSQL Taiwan 1343 members
Open Data Taiwan 614 members
Hadoop Taiwan 986 members
設法找出需求強化供應鏈
還算健康持續互動
需求明確市場飽和
缺乏成功案例
巨量資料產業尚未成型
仍有待打通產業供應鏈
有待法令開放
紅海
藍海
![Page 26: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/26.jpg)
26
就讀科系決定您未來在巨量資料產業的地位
電機
資訊
數學數學
統計統計
商商 做決策
資料科學家
分析軟體
![Page 27: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/27.jpg)
27
大家都說「資料是金礦」,讓我們拿採礦當類比
Open Data資料集
分析資料的合法性
資料鑑價?
個資法
商業模式
金礦
開採權
含金度
提煉廠 分析工具軟體 SMAQ
開採成本 總擁有成本 軟硬體投資
國際金價 提供給客戶的價值 產品通路
![Page 28: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/28.jpg)
28
http://www.huffingtonpost.com/deb-roy/the-birth-of-a-word_b_2639625.html
http://www.ted.com/talks/deb_roy_the_birth_of_a_word.html
個人化的巨量資料
Big Data Can Get Very Personal
巨量資料的未來? Future of Big Data ?
![Page 29: Big data taiwan_supply_chain_and_communities_20130912](https://reader034.fdocuments.net/reader034/viewer/2022051818/54b767544a795957768b45fc/html5/thumbnails/29.jpg)
29
問題與討論
Questions?Questions?Slides - Slides - http://trac.nchc.org.tw/cloudhttp://trac.nchc.org.tw/cloud
Questions?Questions?Slides - Slides - http://trac.nchc.org.tw/cloudhttp://trac.nchc.org.tw/cloud
Jazz WangJazz WangYao-Tsung WangYao-Tsung Wang
[email protected]@nchc.org.tw
Jazz WangJazz WangYao-Tsung WangYao-Tsung Wang
[email protected]@nchc.org.tw
知識有價,熱情無價!
社群也一樣需要永續經營