MAKEMONEYMATTER.

Kubernetes Operations

By Kasper Nissen (@phennex), DevOps Engineer @thelunarway

MAKEMONEYMATTER.

Kubernetes Operations

By Kasper Nissen (@phennex), DevOps Engineer @thelunarway

Vision

We're living in the era of mobile/digital only – we believe banking and commerce should to.

Therefore, our vision is to rethink the interaction with money and defining a complete new category - by introducing a new money app.

It’s the complex coordination between banking services and commerce use:

– How I save money. – How I get money. – How I spend money.

Product “We own the data.

Kasper NissenDevOps & Infrastructure Engineer @thelunarway

Experience

DevOps & Infrastructure Engineer @ LEGO (CITMABIS) (oursourced by IT Minds) for 5 months

Senior/Software Engineer @ IT Minds (~4 years part time)

Master thesis: KubeCloud - A Small-Scale Tangible Cloud Computing Environment.Interview with Software Engineering Daily: bit.ly/2paZ5lg

Blogging about Cloud Native Tech @ www.kubecloud.io

M. Eng. Computer Technology from Aarhus University - Department of Engineering.

B. Eng. Information and Communication Technology from Aarhus University - School of Engineering

WHAT IS KUBERNETES?

Written by: Matt Butcher Illustrated by: Bailey Beougher

WHAT DOES IT DO?

Node Node Node Node Node

Node Node Node Node Node

big dataapp Bapp A database

datacenter

Cluster Manager

WHY ARE WE USING IT AT LUNAR WAY?

Freedom Squads can deploy and more or less implement how they see fit

Autonomous services Squads can work independent of other squads

Continuous Delivery Kubernetes allows us to deploy multiple times a day. It’s easy to rollback in case something went wrong

Flexibility We run many different type of workloads in the cluster. Gives us mobility to become cloud agnostic

Scalable infrastructure Scaling the infrastructure is easy, both on node and container level

High availability Kubernetes takes care of container failures, AWS Auto Scaling groups takes care of node failures

Easy maintenance We are using Kubernetes Operations to help us spin up our clusters, and maintain them.

👍🎉

THAT’S GREAT,BUT HOW DO I GET STARTED? 🤓

KUBERNETES OPERATIONS AKA. KOPS

WHAT IS THAT? 😱

THINK OF IT AS KUBECTL FOR CLUSTERS 👏

kubectl get pods

NAME READY STATUS RESTARTS AGE dns-controller 1/1 Running 0 13d etcd-server-events-ip 1/1 Running 0 13d etcd-server-ip 1/1 Running 0 13d kube-apiserver-ip 1/1 Running 0 13d kube-controller-manager 1/1 Running 0 13d

kops get cluster

$ kops get cluster NAME CLOUD ZONES private-k8s-cluster aws eu-west-1a,eu-west-1b,eu-west-1c

WHAT CAN I DO WITH IT?

kops helps you create, destroy, upgrade and maintain production-grade, highly available, Kubernetes clusters from the command line

WHERE?

AWS (Officially supported) GCP (alpha) VMWARE vSPHERE (alpha)

FEATURES?

• Automates the provisioning of Kubernetes clusters in (AWS)

• Deploys Highly Available (HA) Kubernetes Masters • Supports upgrading from kube-up • Built on a state-sync model for dry-runs and automatic

idempotenc • Ability to generate configuration files for AWS

CloudFormation and Terraform configuration • Supports custom Kubernetes add-ons • Command line autocompletion • Community supported!

LET’S BUILD SOME CLUSTERS

PREREQUISUITES.

IAM USER AmazonEC2FullAccess AmazonRoute53FullAccess AmazonS3FullAccess IAMFullAccess AmazonVPCFullAccess

ROUTE53 HOSTED ZONE SETUP Follow the guide:

https://github.com/kubernetes/kops/blob/master/docs/aws.md

https://github.com/kaspernissen/kops-demo

Videos: http://bit.ly/2tmWOCF

DEMO 1: SINGLE MASTER IN PUBLIC ZONE 3 NODES

s3://phennex-state-store

VPC

api.public-single-master.phennex.com

kubectlkops

autoscaling group autoscaling group

eu-west-1a

pub

lic

$ kops create cluster \ --name $KOPS_NAME \ --state $KOPS_STATE_STORE \ --dns-zone $PUBLIC_HOSTED_ZONE_ID \ --node-count 3 \ --zones eu-west-1a \ --master-zones eu-west-1a \ --cloud aws \ --node-size t2.medium \ --master-size t2.medium \ --yes

DEMO 2: MULTIPLE MASTERS IN PRIVATE ZONES 3 NODES

s3://phennex-state-store VPC

api.private-multiple-masters.phennex.com

kubectlkops

autoscaling group

autoscaling group autoscaling group autoscaling group

bastion.private-multiple-masters.phennex.com

eu-west-1a eu-west-1b eu-west-1c

pri

vate

pub

lic

$ kops create cluster \ --name $KOPS_NAME \ --state $KOPS_STATE_STORE \ --node-count 3 \ --dns-zone $PRIVATE_HOSTED_ZONE_ID \ --zones eu-west-1a,eu-west-1b,eu-west-1c \ --master-zones eu-west-1a,eu-west-1b,eu-west-1c \ --cloud aws \ --node-size t2.medium \ --master-size t2.medium \ --bastion \ --topology private \ --networking weave \ --yes

AWESOME! THAT WAS EASY… 👍

WHAT IF YOU WANT YOUR CLUSTER TO RUN IN AN EXISTING VPC?

http://kubecloud.io/setup-ha-k8s-kops/

MANY MORE CONFIGURATION OPTIONS TO CHOOSE FROM…

--cloud-labels ”Team=Dev” --image some-image (Ubuntu, CentOS, RHEL7, CoreOS) --networking calico, flannel --node-security-groups --master-security-groups --admin-access <SOME IP> --ssh-public-key=/Users/.. --out=. \ --target=terraform

WHAT ABOUT UPGRADING?

$ kops upgrade cluster

YOU CAN ALWAYS EDIT YOUR CLUSTER CONFIGURATION $ kops edit cluster $ kops edit ig --name=<CLUSTER> nodes

ROLLING UPDATES(default pretty bad!) Use feature flag:

$ export KOPS_FEATURE_FLAGS="+DrainAndValidateRollingUpdate" $ kops rolling-update cluster

SCALING THE CLUSTER $ kops edit ig --name $KOPS_NAME nodes

EXTRA… (if we have time)

CLEAN UP!

OTHER SOLUTIONS

COREOS TECHTONIC OPENSHIFT KISMATIC KUBERMATIC

Thank you! Catch me on Twitter @phennex

Or in the Slack Community: Cloud Native DK (https://cloudnative-dk.herokuapp.com/ )