Kubernetes in Production

DOXLON - October 2016

Me

My Work

Established in 1844

Adaptation is in Our History

Inspiring Mission

Passionate People

From ISBN to FQDN

Embrace the Challenge!

What is Bitesize?

AWS CA East Pearson Toronto

Canada

AWS US East AWS US West Pearson Iowa Pearson Denver

Americas

AWS EU Central AWS EU West Pearson London Pearson Slough

EMEA

AWS AP Southeast 1 AWS AP Southeast 2 Pearson Singapore Pearson Australia

APAC

AWS CH East Azure China East Azure China North Pearson Shanghai

China

Pearson Engineering Application Development

Pearson Product Global Go To Market Plan

Delivery PipelineInfrastructure Platform

Cloud DeploymentInfrastructure Platform

Global Delivery Service Supports- Software Lifecycle Automation- Assurance of Standards- Integrated Quality and Security- Rapid Go To Market Capability- Faster Roadmap Execution

How a consistent Infrastructure platform enables global products

Container Runtime Environment

Kubernetes Minions

Application Production NamespaceScheduled across multiple IaaS Availability Zones

App Pod

App Container

App Pod

App Container

App Pod

App Container

Internet (www.pearson-app.com)

AWS Elastic Load Balancer

Kubernetes Load Balancer Minions

Core PaaS Services

Ingress Load Balancer NamespaceScheduled across multiple IaaS Availability Zones

Ingress Pod

Ingress Container

Ingress Pod

Ingress Container

Ingress Pod

Ingress Container

Consul Vault

Containers ingest config & secrets dynamically on spawn

App Replication Controller

SSL Certs and Ingress Config Stored in Consul/Vault

Standard Delivery Pipeline

Docker Registry

Application Dev Namespace

Application Stage Namespace

Application Prod Namespace

VM1 VM2 VM3Cloud Formation

Template

POD A POD B

POD A POD B

POD A POD B

Build Manifests

Application Source

NodeJS <YAML>

Jenkins ContainerRunning in Namespace

Code Commit

The build manifests control all the pipeline steps and deploy activities.

CISO & QA teams load test suites in from separate git repos co-ordinated here.

Current Status

● First app live in two regions

● Second app due live in December

● Roadmap for 50 apps by July 2017

● 25-50% cost reduction in cloud running costs

● 200k global learners depending on us by Q1’17

● Release velocity for early adopters increasing

● Driving a culture change with our developers

“I’m walking, but capable of face planting into a

wall with zero notice…”

Why Kubernetes

http://kubernetes.io/case-studies/pearson/

Our Open Source CriteriaApache 2.0 License Preference

Strong Anchor Sponsor

Sustainable Contribution Model

Non-Commercial Motivation

High Development Activity

Quarterly Release Cadence

Established Implementation Patterns

Reason #1Upfront Complexity

Pays Off in Capability

Reason #2Low Risk Open Source

Community Model

Reason #3Its Google…!

Challenges and Roadmap

What Are We Up To Next?Multi-Region Deployments - Watching Ubernetes Closely

Containerised Databases - Volume Manager + PetSets FTW

Zero-Downtime Platform Releases - Almost done!!

Metadata Frameworks - Sentient Platforms aka Skynet...

Security - Just because

What Are We Struggling With?Ready… Steady… Go... - We need the EA features in GA now!

Upgrades - See above...

Kube-DNS - Step 1, check DNS… Step 2, check it again...

Networking - Not enough choice, not enough differentiation

Security - Just because

And Finally...