Kubernetes: My BFFMrunal Patel @mrunalpRajat Chopra @rajatchopinJonathan Yu @jawnsy

Image used under CC-BY: https://flic.kr/p/5MQQVH

Software Defined Infrastructure

$ whoami

Mrunal Patel @mrunalp

Open Containers maintainer(OCI); runc maintainer; Creator of OCI Tools project; Container Security; Docker contributor.Previously: Cisco, Wipro, NCSU

Rajat Chopra @rajatchopin

Hardware hacker turned software geek; Creator of OpenShift SDN and ose-contiv; Glider Pilot

Previously: Mentor Graphics, Texas Instruments, Cypress Semiconductors, UC Berkeley

Jonathan Yu @jawnsy

OpenShift Online developer; Level 11 on Pokemon Go

Previously: IBM, Ciena, WesternU2

Why containers?

3

Physical Hardware

Virtual MachineContainer

“Pets” (RHEV) “Cattle”

Density Lowest Highest

Lifetime Years Hours

Startup Time Weeks Seconds

Isolation Best Good

Cost Highest Lowest

Increasing Density

Shorter Runtime

Lower startup time overhead

Less (but still good) isolation

From “call the CFO” to “basically free”

Containers are fairly low-level

4

A few servers...

Image source: EmojiOne font, Red Hat official brand logos, “lime guy” meme

Dozens of VMs... 1000s of containers...

We need Orchestration!

● Scheduling● Scaling● Networking● Persistent Storage● Monitoring● Logging

We will show some of these by example...

5

The origin of kubernetes

● Built on lessons learnt by Google over the years (yeah, right)● Declarative (system reconciles observed/desired state)● Scales to massive clusters (1000 nodes/250 pods per node)● Plugin-based architecture (networking, scheduling, runtimes, storage)

6

Down the orchestration rabbit hole...

7

“I have an app…”

“... Now just go run it somewhere for me!”

8

host1.red.ht host2.red.ht

“Cool, that was easy!”

9

host1.red.ht host2.red.ht

“Wow, our app was super successful! Run another instance for me!”

“... wait, that’s not what I wanted.”

I need scheduling!

10

Scheduling

Actually, I want my app to run on the “right” machine.

Some example criteria:

● A different machine: avoid Single Point of Failure (SPOF)● Cost: least expensive capacity● Performance: most lightly-loaded machine● … and so on, with arbitrary labels

11

host1.red.ht

host2.red.ht

“So, how does external traffic reach my containers?”

Ingress

https://app.red.ht

DNS: 203.0.113.7

172.30.1.1

172.30.2.10

Public Network Cluster Network

12

Ingress

● Each container* gets its own cluster-internal IP address● Ingress routes traffic from the external world to the internal network

○ Enables high availability (failover) and load balancing○ Conceptually similar to: AWS ELB, HAProxy, nginx, F5 BIG-IP, …

○ When instances are added or removed, the pool of containers that can handle requests is dynamically adjusted

○ Handles sticky sessions

* technically pods, but the distinction isn’t relevant here

13

“Okay, now my app needs a database. Can kubernetes manage that, too?”

host1.red.ht host2.red.ht

“Since containers are ephemeral, how do we store data? What if containers are rescheduled on a different machine?”

Persistent Storage

14

Containers can “claim” persistent storage from a pool of volumes:

EBS pDisk VHD Gluster

Ceph NFS Flocker Cinder

tmpfs… and more, it’s pluggable!

● A Persistent Volume (PV) defines available storage capacity that persists across container crashes/restarts

● A Persistent Volume Claim (PVC) binds a PV to a container; storage is available until the claim is released

15

“Okay, but how will my app know how to find the database?”

host1.red.ht host2.red.ht

Services

16

● Expose a set of containers as a virtual IP and cluster-internal DNS● Containers are dynamically selected by a label

○ For example: selector.name = “frontend”

● Connections are automatically load-balanced● For some services, like MongoDB, instances can be scaled up and down

dynamically (transparently to clients!)

Service: mongodb.svc.cluster.local (172.30.0.10)

17

What would this look like in practice?

tier=frontend

tier=backend

172.30.1.5172.30.10.2

172.30.1.7

… A service could have 1 pod, or 100

18

“Wait! What are pods? We’ve been talking about containers so far...”

Pods

● Pods are the unit of scale in kubernetes● A “Pod” is simply a set of containers that work together

○ Shared IP address○ Always deployed side-by-side

● If you only define one container per pod, you can treat them the same

19

Replica Sets and Autoscaling

● Replica Sets ensure that a certain number of pods are running○ If a pod crashes, it is restarted automatically○ If a node disappears, pods are rescheduled on remaining cluster members

● Autoscaling dynamically adjusts replica count based on usage○ We can handle the “Reddit” or “Hacker News” effect automatically○ Unused capacity is returned to the pool for other applications

20

Deployments

● Deployments provide declarative updates to pods and replica sets● Allow rollbacks to older versions in history● Support rolling updates for zero-downtime deployment● Labels support more sophisticated scenarios, like canary deployments,

blue/green, and A/B testing

21

22

“Great! But I have some batch processes that I need to run. Can kubernetes help me with those, too?”

Jobs

● Pods that run batch operations that are expected to terminate● For example:

○ Machine learning algorithms (Big Data, Analytics)○ Compute-intensive preprocessing, such as video transcoding

● Future work: Scheduled/periodic jobs (like cron or Chronos)

23

Secrets and ConfigMap

“I can’t put configuration files or TLS private keys in my container image, so what do I do?”

● Secrets are designed to hold sensitive information (passwords, keys)○ Different credentials can be used with the same container image○ Useful for separating development and production secrets

● ConfigMap is similar to Secrets, but for other configuration data● Both can be mounted as read-only volumes or environment variables

24

How do all these concepts relate to AWS, Azure, etc?

25

Kubernetes OpenShift AWS Azure OpenStack

Pods EC2 Container Service

Azure Container Service Virtual Machine

Replica Sets, Autoscaling Autoscaling Groups Availability Set Heat

Registry Atomic Registry EC2 Container Registry ? ?

Overlay/SDN (Contiv, Flannel, OpenShift-SDN) Virtual Private Cloud Virtual Network Neutron

Ingress/Routes Elastic Load Balancer Load Balancing Neutron

Pluggable Block Storage (platform-agnostic) Elastic Block Storage Virtual Hard Disk Cinder

Bottom line: Kubernetes enables portable infrastructure-agnostic deployments (on-premises, private cloud, public cloud, multiple federated clouds)

How do all these projects relate to each other?

26

Storage

Security

Networking

Control Groups (cgroups), Namespaces, SELinux

Open Containers (OCI), LXC, systemd-nspawn

Cont

aine

r Ru

ntim

es

Registry

OpenShift SDN

$ oc cluster up

So… This stuff sounds cool, but how do I try it?

● Minikube (Kubernetes in a VM via Vagrant)

● Nanokube (Kubernetes via Docker-in-Docker)

● OpenShift “oc cluster up” (OpenShift Origin via Docker-in-Docker)

27

Questions?

Mrunal Patel @mrunalp

https://www.linkedin.com/in/mrunalhttps://github.com/mrunalpmpatel@redhat.com

Rajat Chopra @rajatchopin

https://www.linkedin.com/in/rajat-chopra-7311913https://github.com/rajatchoprarchopra@redhat.com

Jonathan Yu @jawnsy

https://www.linkedin.com/in/jawnsyhttps://github.com/jawnsyjawnsy@redhat.com

28

What is OpenShift?

29

OpenShift is Kubernetes for the Enterprise. It adds PaaS capabilities (builds).

OpenShift Originhttps://github.com/openshift/origin

Register for Dev Preview: https://www.openshift.com/devpreview/