Data exploration at the speed of thought · [email protected] Data exploration at the speed of...
23
Nico Gaviola Head of Healthcare and Lifesciences UKIE [email protected] Data exploration at the speed of thought Lessons learned from inside Google
Transcript of Data exploration at the speed of thought · [email protected] Data exploration at the speed of...
Nico Gaviola Head of Healthcare and Lifesciences UKIE [email protected]
Data exploration at the speed of thought Lessons learned from inside Google
Google’s m ission is t o organize t he wor ld’s inform at ion and m ake it universally accessible and useful.
Sundar Pichai CEO, Google
uploads per minute
users
search index
query response time
500hrs
1B+
100PB+
0.25s
Google computing scale
Hitting the limits, early on...
The Anatom y of a Large -Scale Hype rtextua l Web Search Engine 1996, Se rgey Brin and Lawrence Page Com pute r Science Departm ent, Stanford Unive rsity, Stanford , CA 94305
2012 2013 2002 2004 2006 2008 2010
Google Research Publications referenced are available here: http://research.google.com/pubs/papers.html The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, 2009 http://research.google.com/pubs/pub35290.html
GFS
MapReduce
BigTable
Single Node to Cluster
Google’s Data Research
2002 2004 2006 2008 2010 2012 2014 2016
GFS
MapReduce TensorFlow
BigTable
Dremel
Colossus
Flume
Megastore
Spanner
Millwheel
PubSub
F1
Google’s Data Products
2002 2004 2006 2008 2010 2012 2014 2016
ML
PubSub
DataFlow
DataStore
DataFlow
Cloud Storage
BigQuery
BigTable
DataProc
Cloud Storage
Programming
Resource provisioning
Performance tuning
Monitoring
Reliability Deployment & configuration
Handling growing scale
Utilization improvements
Typical Big Data Jobs
Big Data with Google Focus on ins ights . Not infras tructure.
Programming
Understanding
Google’s Big Data Vision
Pay $5 per TB
Active contributor to numerous OSS projects
Make migrations eas ier with open APIs
Cus tomers s hould us e us becaus e they love us , not becaus e they are unable to move off
Open Source & APIs
12
Confidential & Proprietary Google Cloud Pla tform 13
You own your data and remain Data Controller
You can delete or remove your data a t
any time
Google does not s hare your content or
pers onal information google.com/ privacy
Strict Internal Policies : a ll acces s es to
cus tomer or cons umer data applications are
logged
Internal data acces s auditing tracks
Googlers
Google Security Model & You!
Example
“Right at the start of the partnership we were able to reduce time to insight from 96 hours to 30 minutes by using BigQuery”
Gary Sanders Head of Digital Analytics
What’s Next?
“Machine learning is a core, trans formative way by which we’re re-thinking how we’re doing everything”
Sundar Pichai CEO, Google
15% reduction in PUE
Fully trained, easy to use Machine Learning models
Cloud Trans la te
Cloud Vis ion
Cloud Speech
Cloud Natural Language Stay tuned…
Use your own data to train models
Cloud Storage BigQuery Cloud Datalab
Cloud Machine Learning
Develop, Model, Train, Tes t
One more thing
Free training courses coming near you!
Thank you!
