1Data & Analytics
3rd Generation Data Platform: From Buckets of Bits to Understanding
2Data & Analytics
Data is Everything.How well you use your data can determine the degree of your success.
3
Street name
Street number
Street View
Sign
Business facade
Sign Business
name
Traffic light
Traffic sign
Street number
4Data & Analytics
3rd Gen Data Platforms Challenges
Data access to a variety of data sources.
Develop and build analytic models.
Data preparation, exploration and visualization.
Deploy models and integrate them into
business processes and applications.
High performance and scalability for both
development and deployment.
Perform platform, project and model management.
55
– Doug Cutting, Hadoop Co-Creator
“Google is living a few years in the future and sending the
rest of us messages”
6Data & Analytics
Google Cloud Platform VisionSingle-node computing
“Some assembly required”True, on-demand cloud
An actual, global elastic cloud
3rd Wave
Invest your energy in great apps
Colocation
Your kit, someone else’s building.
Yours to manage.
1st WaveToday's Cloud:
Virtualized Data CentersStandard virtual kit, for rent. Still
yours to manage.
2nd Wave
7Data & Analytics
Bridging the Waves Analyze StoreCapture
BigQuery(large scale SQL)
Cloud Machine Learning
Cloud Pub/Sub
Logs, App Engine
BigQuery streaming
Process
Cloud Dataflow(stream and batch)
CloudStorage(objects)
Cloud SQL(SQL)
CloudDatastor
e(NoSQL)
BigQuery(structured
)
Cloud Dataproc (Hadoop & Ecosystem)
Cloud Bigtable(NoSQLHBase)
Cassandra
hBase MongoDBRabbit MQ
Kafka
Wave 2
Wave 3
Visualise
Cloud DataLab (iPython/Jupyter)
Data Studio 360
Tableau
Qlik
8Data & Analytics
Exploration &
Collaboration
Databases Storage
Data Preparatio
n & Processin
g
Analytics
Advanced Analytics
& Intelligenc
e
Google Cloud Data Platform
Mobile apps
Sensors and
devices
Web apps
Relational
Key-value
Document
SQL
Wide Column
ObjectStream processing
Batch processing
Data preparation
Federated query
Data catalog
Data exploration
Data visualizatio
nDevelopers
Data scientists
Business analysts
Development
environment for
Machine Learning
Pre-Trained Machine Learning models
Data Ingestion
Messaging
Logs
9Data & Analytics
Data Preparatio
n & Processing
Cloud Dataflow
Cloud Dataproc
Exploration &
Collaboration
Google BigQuery
Cloud Datalab
Google Analytics 360
Cloud Dataproc
Google Cloud Data Platform
Mobile apps
Sensors and
devices
Web apps
Developers
Data scientists
Business analysts
Data Ingestion
Cloud Pub/Sub
App Engine
Databases/Storage
Cloud SQL
Cloud Bigtable
Cloud Datastore
Cloud Storage
Analytics
Google BigQuery
Google Analytics 360
Cloud Dataproc
Google Drive
Advanced Analytics & Intelligenc
e
Cloud Machine Learning
Translate API
Vision API
Speech API
10Data & Analytics
Managed Data Services - Focus on Insight vs InfrastructurePB+ Scale, No-Ops, Batch & Streaming of Data
Insights/Programming
Resource Provisioning
Performance Tuning
Monitoring
Reliability
Deployment & Configuration
Handling Growing Scale
Utilization Improvemen
ts
Insights/Programming
proprietary & confidential | not for distribution
"We are very excited about the productivity benefits offered by Cloud Dataflow and
Cloud Pub/Sub. It took half a day to rewrite something that had previously
taken over six months to build using Spark"Paul Clarke, Director of Technology, Ocado
http://googlecloudplatform.blogspot.co.uk/2015/08/Announcing-General-Availability-of-Google-Cloud-Dataflow-and-Cloud-Pub-Sub.html
Hadoop + Local SSD
5X the IOPS at 0.5 the cost of AWS local SSD
Up to 1.5TB per instance
680,000 read IOPS and < 1ms latency 1
2
3
13Data & Analytics
– Mattias P Johansson, Software Engineer, Spotify
“With Google Cloud Platform, we benefitted by having a virtual supercomputer on demand, without having to deal with all the usual space, power, cooling and networking issues. Just a few years ago, we would have needed to use the largest supercomputers on the planet to do what we’re now able to do with Google”
– Mark Johnson, CEO, Descartes Labs
“Right at the start of the partnership we were able to reduce time to insight from 96 hours to 30 minutes by using BigQuery.”– Gary Sanders, Head of Digital Analytics, Lloyds Banking
Group
“Everyone involved unanimously picked GCP. It came down to this: we believe the core technology is better.”
– Peter Bakkam, Platform Lead, Quizlet
Do you feel this way about your Data Warehouse?
14Data & Analytics
Data Warehouses/Lakes
Machine Intelligence
Data Warehouse is the foundation of something bigger
Predictive +
Prescriptive analytics
=Advanced analytics
Cloud
On Premises Machine
LearningAPIs
Train your own
Models
15Data & Analytics
Automatically categorize, and automatically extract value
Evaluate the model by applying it against additional manually categorized data, correct and tune
Machine intelligence is already making a huge differenceand there are many, many more opportunities
Capture lots of examples of correct evaluations for that
categorization, and use them to train an ML
model
Identify categorizations that
provide value, categories you’re
already evaluating for by hand today
1 2 3 4
16| THE LEADERS CIRCLE
Rapidly accelerating use of deep learning at Google
AlphaGoAndroidAppsGmailMapsPhotosRoboticsSpeechSearchTranslationYouTubeand many others ...
Used across areas:
2012 2013 2014 2015
1500
1000
500
0
Number of directories containing model description files
171717
BETAGA BETA
CloudTranslate API
CloudVision API
CloudSpeech API
Natural Language API
GA
Ready to use Machine Learning models
18Data & Analytics
Machine learning will drive every successful huge IPO win in the next 5
years.“ ”Eric SchmidtExecutive Chairman, Alphabet Inc
19Data & Analytics
Now
Stores and Analyzes
Next
Understands
20Data & Analytics 20
Thanks!
Top Related