GCP UpdatesWhat's new in Google Cloud Platform

Anuja Shukla, Technical Lead for Cloud Platform, Google India

Google Innovations in Software

2002 2004 2006 2008 2010 2012

ColossusMapReduce

SpannerBig Table

Dremel

GFS

2013 2014

Dataflow

Kubernetes

Google Cloud Platform

NetworkingCompute Big Data Management Storage MobileDeveloper

Tools

Compute

ManagementNetworkingCompute Big Data Storage MobileDeveloper

Tools

Compute Engine

Container Engine

App Engine

Compute Engine: Features

• Sub-hour Billing & Sustained-Use Discounts

• Up to 10TB Persistent Disk

• Over 44 Instance Types

• Advanced Networking

• Instance Metadata and Startup Scripts

• Load Balancing

• Monitoring

• Snapshotting

Container Engine: Features

• Kubernetes based container cluster service

• Deploy, run and manage Docker images

• Get started quickly with base images

• Resource isolation and prioritization

• Scale quickly and efficiently

• Automatically heal applications

• Securely store images in a private registry

App Engine: Features

• Popular Programming Language Support, or BYO

runtime

• Access Underlying VM

• Flexible and Scalable Application Storage

• Integrated Services - Cron, Tasks Queues,

Memcache, Datastore

• Auto-scaling

• Versioning and Traffic Splitting

• Local Developer Tools

• Third-party Frameworks and Extensions

Networking

ManagementNetworkingCompute Big Data Storage Mobile

Cloud Interconnect

Cloud Interconnect: Features

• Carrier Interconnect

• Direct Peering

• Secure VPN

• Software defined network adapts based on

observed workload

Big Data

ManagementNetworkingCompute Big Data Storage Mobile

Big QueryCloud

Pub/SubCloud

Dataflow

BigQuery: Value

• Performance: Ingest data at 100K

rows/second and process real-time queries

• Ease of use: No administration for

performance and scale

• Scale: No need to worry about growing data.

Unlimited storage with pay as you go pricing

model

• Non-technical analysts can drive queries on

massive datasets using BI tools

BigQuery: Features

• Interactive query performance: Query multi-

terabyte datasets in an ad hoc manner

• SQL: Familiar SQL-like query syntax and intuitive

user interface

• Data mashup: Query across diverse datasets

• Highly Available: Data replication in multiple

geographies. Data is available and durable even in

the case of extreme failure modes

• Secure: Access to data is controlled using

customer-owned ACLs

Cloud Pub/Sub: Features

• Unified messaging: Durability and low-latency

delivery in a single product

• Global presence: Connect services located

anywhere in the world

• Flexible delivery options: Both push- and pull-style

subscriptions supported

• Data reliability: Replicated storage and guaranteed

at-least-once message delivery

• Data security and protection: Encryption of data on

the wire and at rest

Cloud Dataflow: Features

• Unified programming model for both batch and

stream-based data analysis

• Managed scaling: Manages the lifecycle of

required compute resources

• Reliable & consistent processing: Built-in

support for fault-tolerant execution

• Monitoring: Provides lifecycle statistics

including in flight information like real time

pipeline throughput, real time step lag and real

time worker log inspection

Management

Monitoring

ManagementNetworkingCompute Big Data Storage Mobile

Cloud Monitoring: Features

• Core metrics and dashboards

• Near real-time insight

• Pinpoint issues and bottlenecks

• Alerting via email, SMS and more

• Build custom metrics and dashboards

• Endpoints to test functionality

• Lightweight agent for deep insight

Storage

ManagementNetworkingCompute Big Data Storage Mobile

Cloud Storage

Cloud SQL

CloudDatastore

Easy-to-use storage options

NoSQL SQL Blob Block

Cloud Storage: Features

• 3 storage options

– Standard: The highest level of durability,

availability and performance

– DRA: High level of durability, availability and

performance

– Nearline: High performance data archiving,

online backup, and disaster recovery

Cloud Datastore: Features

• Auto-scale

• Schemaless Access

• SQL-like Capabilities

• Authentication That Just Works

• Fast and Easy Provisioning

• RESTful Endpoints

• ACID Transactions

• Local Development Tools

• Built-in Redundancy

Cloud SQL: Features

• Fully managed

• Ease of Use

• Highly Reliable

• Flexible Charging

• Security, Availability, Durability

• EU and US Data Centers

• Easy Migration & Data Portability

• Control

Mobile

ManagementNetworkingCompute Big Data Storage MobileDeveloper

Tools

Firebase

Firebase

Firebase: Value

• Best way to launch, build and grow your

mobile strategy

• Rapid development & iteration: time-to-value is key

for mobile projects

• Backend-less development frees you to focus

on your app’s user experience

• Your data syncs seamlessly even when offline:

let Firebase handle spotty network connections

• Cross-platform mobile development

Firebase: Features

• Seamless data sync w/offline support

• Client libraries across all major platforms

• Built-in security and easy authentication

• Host your entire application end to end

• Rule-based triggers: create business rules without

writing any server-side code

Firebase Demo

Developer Tools

ManagementNetworkingCompute Big Data Storage MobileDeveloper

Tools

Cloud Debugger

• Debug running

applications in production

without stopping

• Easy to use developer

inspection tools

• Safe for production: No

user noticeable effects

Cloud Trace

• Visualize time spent in

your application

• Quickly identify

application performance

bottlenecks

• Compare performance

across releases to spot

degradation

Google Ecosystem + APIs

• Take advantage of Google’s entire ecosystem of services:

Search

Web analytics

Monetization

App Distribution

Google Cloud Platform (GCP) - Cloud ContinuumA practical guide to move to cloud

Mani Doraisamy, Founder of Guesswork.co,Google Developer Expert (GDE) for Cloud Platform

Introduction

Mani Doraisamy

- Founder - Guesswork.co

- Creator of 2 successful products on Google Cloud

- One of the top 10 Google Experts in the world

Which one is your app?

Build from scratch

Running on premise

Existing App Greenfield App

Motivation

Zero maintenance

Cost

Existing App Greenfield App

What should you choose?

Platform(App engine)

VM(GCE)

Existing App Greenfield App

Can I have both?

Platform(App engine)

VM(GCE)

Somewhere here

Existing App Greenfield App

Compute as a continuumbuild and deploy vm images

● Curated runtimes● Rich services● Auto-everything● … just add code

● Managed collections

● Declarative + Dynamic

PlatformCluster

● Basic atom

● Run anything

VM

More agility

More Flexibility

build and deploy containers build and deploy apps

processes, microservices, jails...

Containers?

Process Isolation

key concepts

What is Kubernetes

● 0.14.1 released

● 250+ contributors

● over 6800+ github

stars

● CoreOS

● HP

● IBM

● Mesosphere

● Microsoft

● Pivotal

● Red Hat

● SaltStack

● VMWare

● Binary releases

● Logging and monitoring

● Richer scheduling

o Labels based constraints

o Smart bin-packing

o Run-to-completion

● Updated service model

o IP per service

● DNS integration

Project Partners Features

Pods

● Group of containers

● Settings in a template

➔ Reuse across environments

➔ Repeatable, manageable

Behavior Benefits

Web Server

Content Management

Server

Labels

Dashboard

show: FE

● Metadata with semantic meaning

● Membership identifier

➔ Allow for intent of many users (e.g. dashboards)

➔ Build higher level systems …

Behavior Benefits

Pod Pod

frontend

Pod

frontend

Pod Pod

FE BI, FE v2

Dashboard

show: v2

more at labels.md

Replication Controllers

Behavior Benefits

ReplicationController

#Pods → 2label selector: v1

Pod Pod

frontend

Pod

frontend

Pod Pod

v1 v1

ReplicationController

#Pods → 1label selector: v2

v2

more at rc.md

● Keeps Pods running

● Gives direct control of Pod #s

➔ Restarts Pods, desired state

➔ Fine-grained control for scaling

Demo

Demo

DatabaseBatch

processing

Cache

WebserversWebservers

WebserversWebservers

Services

Pod

Behavior Benefits

Pod

FE FE

Pod

FE

Service

label selector: Front End

more at services.md

➔ Clients shielded from implementation details

➔ Independently control each, build for resiliency

● Stable address

● Decoupled from Controllers

Distributed Process

Scheduler

Wrap Up

Do I have to manage it

myself?

Google Container Engine

1. Inspired by a

decade of using

containers

2. Reimagines cluster

computing

3. Designed for multi-

cluster world

Run an optimized clusterKubernetes

Open Source

Workload

API & Tools Google Container Engine (GKE)

Kubernetes

● Growing Community

● Linux containers support

● Open source orchestration

create and manage

Container Engine

● Provisioned cluster in seconds

● Optimized cluster environment

● Runs and manages Kubernetes

GKEMaster GKE Nodes

Virtual Machines

Google Compute EngineNew service for cluster-based compute

● Provisioned cluster in seconds. Fully configured.

● Fine-grained control over cluster. Desired state

● Designed for multi-cloud. Runs Kubernetes.

Status

● Today: Alpha open to the public

● Coming soon: SLA, Pricing

Resources

● Google Container Engine: http://cloud.google.com/container-engine

● Kubernetes: http://kubernetes.io