Computing at scale
-
Upload
jerjou -
Category
Technology
-
view
114 -
download
3
description
Transcript of Computing at scale
![Page 1: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/1.jpg)
Computing at Scale:Data Exploration
Jerjou Cheng, Barry Brumitt
![Page 2: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/2.jpg)
HELLO
/jur'jō/ɯoɔ˙ǝlƃooƃ@noɾɹǝɾ
MY NAME IS
Developer Programs Engineer(Developer Relations)
Google Storage
![Page 3: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/3.jpg)
Michael PIERROThttp://www.freephotobank.org/v/sky-stars/cloud/Cloud-19.jpg.html
![Page 4: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/4.jpg)
Computing at Scale:Data Exploration
Jerjou Cheng, Barry Brumitt
![Page 5: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/5.jpg)
Overview
• Google App Engine• Google Storage for Developers• Prediction API• BigQuery
![Page 6: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/6.jpg)
Introductions
![Page 7: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/7.jpg)
Who are theseservices for?
![Page 8: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/8.jpg)
A World without Clouds
• Build a web applicationoStartup costsoMaintenance / reliabilityoScaling
Michael Scheltgenflickr.com/mscheltgen/
![Page 9: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/9.jpg)
Google App Engine
Easy to startEasy to maintainEasy to scale
![Page 10: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/10.jpg)
Users
![Page 11: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/11.jpg)
gigy Socialize - traffic
![Page 12: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/12.jpg)
Overview
• Google App Engine• Google Storage for Developers• Prediction API• BigQuery
![Page 13: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/13.jpg)
Overview
• Google App Engine• Google Storage for Developers• Prediction API• BigQuery
![Page 14: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/14.jpg)
A World without Clouds
• Store data• Reliability• Sharing• Large objects
Michael Scheltgenflickr.com/mscheltgen/
![Page 15: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/15.jpg)
Google Storage for Developers
• Google infrastructure• You control access to your data• Store massive data in Google's cloud• Easy interface
![Page 16: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/16.jpg)
Example
![Page 17: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/17.jpg)
Internal use cases
• Content hosting
• Sharing
• Data Import Google BigQuery
Google Prediction API
![Page 18: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/18.jpg)
Overview
• Google App Engine• Google Storage for Developers• Prediction API• BigQuery
![Page 19: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/19.jpg)
Prediction API
• Cloud-hosted machine learning as service• Simple interface over complex analysis• Predict results in real-time
![Page 20: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/20.jpg)
How does it work?
"english" The quick brown fox jumped over the lazy dog.
"english" To err is human, but to really foul things up you need a computer.
"spanish" No hay mal que por bien no venga.
"spanish" La tercera es la vencida.
"english" To be or not to be, that is the question.
"spanish" La fe mueve montañas.
The Prediction APIfinds relevantfeatures in the sample data during training.
The Prediction APIlater searches forthose featuresduring prediction.
![Page 21: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/21.jpg)
Prediction API
1. Upload
2. Train
Upload your training data toGoogle Storage
Build a model from your data
Make new predictions
prediction/v1.1/training?data={}POST : a training request
prediction/v1.1/training/{}/predictGET : model infoPOST : a prediction request
Use the API, gsutil or any compatible utility to upload your data to Google Storage
3. Predict
![Page 22: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/22.jpg)
Example
![Page 23: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/23.jpg)
Prediction API
• Google's machine learning algorithms• Available as RESTful HTTP service• Predict results in real-time
![Page 24: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/24.jpg)
Overview
• Google App Engine• Google Storage for Developers• Prediction API• BigQuery
![Page 25: Computing at scale](https://reader033.fdocuments.net/reader033/viewer/2022061223/54c6cb6c4a795943608b4583/html5/thumbnails/25.jpg)
GET /information HTTP/1.0
• To request access and get more information, go to:o http://code.google.com/appengineo http://code.google.com/apis/bigqueryo http://code.google.com/apis/predicto http://code.google.com/apis/storage