Unleashing K8S to reduce complexities of an entire middleware

platform

Director - Architecture, WSO2Afkham Azeez

Director - Cloud Architecture, WSO2Lakmal Warusawithana

WSO2 Helps Build a Connected Business

Enterprise middleware platform

WSO2 Carbon

So what has this session got to do with Kubernetes?

Why are these guys at KubeCon?

Credits: http://texas-blooms.com/valentines-day-flowers-a-guys-guide/`

Kubernetes use cases for WSO2

o Multi-tenancy

o Microservices

o Scaling

WSO2 Carbon Multitenancy

● User management

● Data isolation

● Execution isolation

Shared process multitenancy in Carbon

8

Issues with Shared Process MT

● Difficult to control how much resources a tenant can use

● Complex Java Security management

● Too many security restrictions at runtime

Kubernetes to the rescue!

● K8S Namespaces

● K8S Quota

● K8S Health Monitoring

● K8S Rolling Update

● K8S Secret Sharing and Volume Mounting

● K8S Autoscaling

● K8S Identity and Access Management

Execution Isolation with K8S Namespaces

● Tenant mapped to a k8s namespace

● Namespace provides the scope for pods, services, and replication controllers in the cluster

● Users of tenant interacting with one namespace do not see the content in another namespace

● Different authorization rules for each namespace.

K8S Resource Controlling using Quota● Tenant creation assigned a Resource Quota for each

namespace

● Compute Resource Quota○ Total cpu limits of containers○ Total memory limits of containers

● Object Count Quota○ Total number of pods○ Total number of services○ Total number of replication controllers○ Total number of secrets○ Total number of persistent volume claims

K8S Resource Controlling using Quota$ kubectl describe quota quota

Name: quota

Resource Used Hard

-------- ---- ----

cpu 0m 20

memory 0 1Gi

pods 5 10

replicationcontrollers 5 20

resourcequotas 1 1

services 3 5

K8S Health Monitoring● Process Health Checking

○ The Kubelet constantly asks the Docker daemon if the container process is still running, and if not, the container process is restarted

● Application Health Checking○ HTTP Health Checks - The Kubelet will call a web hook. If it returns

between 200 and 399, it is considered success, failure otherwise.

○ Container Exec - The Kubelet will execute a command inside your container. If it exits with status 0 it will be considered a success

○ TCP Socket - The Kubelet will attempt to open a socket to your container. If it can establish a connection, the container is considered healthy, if it can't it is considered a failure.

K8S Rolling Update● Tenant's application artifacts are burned into the docker

image● New artifacts create new docker images with new

versioning/tag number● Update replication controller using rolling-update

○ It will create new rc with a pod template that uses the new docker image

○ Scale the old and new replication controllers until the new controller replaces the old. This will kill the current pods one at a time, spinning up new ones to replace them

K8S Secret Sharing● Objects of type secret are intended to hold sensitive information, such as

passwords, OAuth tokens, and ssh keys● Secret volumes are backed by tmpfs (a RAM-backed filesystem) so they

are never written to non-volatile

apiVersion: v1kind: Secretmetadata: name: mysecrettype: Opaquedata: password: dmFsdWUtMg0K username: dmFsdWUtMQ0K

K8S Autoscaling

K8s Identity and Access Management with WSO2 Identity Server

● User Roles○ Carbon Super Admin - k8s Admin○ Carbon Tenant Admin - k8s project administrator○ Carbon Tenant Users - k8s developer

● User Store - LDAP

● Authentication

● Authorization

Ops work

● Planing to use kubectl for deploying and managing WSO2 multitenant Products

● We believed all necessary ops functionality is available in kubectl

● If we see some gaps will hoping to contribute back to the community

WSO2 Microservices Server (MSS)

● Lightweight & fast Java microservices server

● Default deployment mode is based on Docker & Kubernetes

● GitHub: https://github.com/wso2/product-mss

● 1.0-alpha available for download https://github.com/wso2/product-mss/releases

WSO2 Microservices Server - TPS

WSO2 Microservices Server - Memory Usage

Pet store sample

Pet store sample - deployment view

24

Contact us !