Creating an Advanced Load Balancing Solution for Kubernetes with NGINX

Andrew Hutchings — Technical Product Manager, NGINX, Inc., @LinuxJedi

About LinuxJedi• Kubernetes user for 4 days

• Worked at HP on OpenStack LBaaS and ATG

• Worked on several Open Source DBs

• Alopecia sufferer

Goals

• Basic and advanced load balancing

• Current load balancing options in Kubernetes

• Ingress resource

• Implementing an Ingress controller for NGINX

• Load balancing demo: exposing Kubernetes services to the Internet

Basic Load Balancing

A load balancer distributes request among healthy servers

LB

Server 1 Server 2 Server 3

Basic Load Balancing

HTTPHTTP

Layer 7

TCPTCP UDPUDP

Layer 4

Advanced Load Balancing

• SSL termination

• Active health checks

• Security

• Bandwidth limits

• Logging

• Real-time statistics

• Session Persistence

• Content-based routing

• and more…

Load Balancing in Kubernetes

Internal

• kube-proxy

External

• NodePort• LoadBalancer• External IPs• Service loadbalancer• Ingress

Internal: Kube-proxyapiVersion: v1kind: Servicemetadata: name: backend-servicespec: ports: - port: 80 targetPort: 80 protocol: TCP selector: app: backend

apiVersion: v1kind: Servicemetadata: name: backend-servicespec: ports: - port: 80 targetPort: 80 protocol: TCP selector: app: backend

# env | grep -i backendBACKEND_SERVICE_SERVICE_HOST=10.3.246.245BACKEND_SERVICE_SERVICE_PORT=80…

# env | grep -i backendBACKEND_SERVICE_SERVICE_HOST=10.3.246.245BACKEND_SERVICE_SERVICE_PORT=80…

# nslookup backend-service…Name: backend-serviceAddress 1: 10.3.246.245

# nslookup backend-service…Name: backend-serviceAddress 1: 10.3.246.245

$ kubectl get svcNAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGEbackend-service 10.3.246.245 <none> 80/TCP app=backend 6m

$ kubectl get svcNAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGEbackend-service 10.3.246.245 <none> 80/TCP app=backend 6m

Internal: Kube-proxy

kube-proxykube-proxy

BB

kube-proxykube-proxy

BB

kube-proxykube-proxy

BB

Features

• TCP/UDP• Health checks• Client IP session affinity

External: NodePortapiVersion: v1kind: Servicemetadata: name: backend-servicespec: type: NodePort ports: - port: 80 targetPort: 80 protocol: TCP selector: app: backend

apiVersion: v1kind: Servicemetadata: name: backend-servicespec: type: NodePort ports: - port: 80 targetPort: 80 protocol: TCP selector: app: backend

$ kubectl create -f backend-service-nodeport.yamlYou have exposed your service on an external port on all nodes in yourcluster. If you want to expose this service to the external internet, you mayneed to set up firewall rules for the service port(s) (tcp:31107) to serve traffic.

$ kubectl create -f backend-service-nodeport.yamlYou have exposed your service on an external port on all nodes in yourcluster. If you want to expose this service to the external internet, you mayneed to set up firewall rules for the service port(s) (tcp:31107) to serve traffic.

External: NodePort

Features

• TCP/UDP• Health checks

kube-proxykube-proxykube-proxykube-proxy

BB

kube-proxykube-proxy

BB

NodePortNodePort NodePortNodePort NodePortNodePort

BB

External: LoadBalancerapiVersion: v1kind: Servicemetadata: name: backend-servicespec: type: LoadBalancer ports: - port: 80 targetPort: 80 protocol: TCP selector: app: backend

apiVersion: v1kind: Servicemetadata: name: backend-servicespec: type: LoadBalancer ports: - port: 80 targetPort: 80 protocol: TCP selector: app: backend

$ kubectl describe svc backend-serviceName: backend-serviceNamespace: defaultLabels: <none>Selector: app=backendType: LoadBalancerIP: 10.3.249.155LoadBalancer Ingress: XXX.YYY.ZZZ.IIIPort: <unnamed> 80/TCPNodePort: <unnamed> 32074/TCPEndpoints: <none>Session Affinity: None

$ kubectl describe svc backend-serviceName: backend-serviceNamespace: defaultLabels: <none>Selector: app=backendType: LoadBalancerIP: 10.3.249.155LoadBalancer Ingress: XXX.YYY.ZZZ.IIIPort: <unnamed> 80/TCPNodePort: <unnamed> 32074/TCPEndpoints: <none>Session Affinity: None

External: LoadBalancer

Features

• TCP• Health checks• Client IP session affinity

(GCE)kube-proxykube-proxykube-proxykube-proxy

BB

kube-proxykube-proxy

BB

NodePortNodePort NodePortNodePort NodePortNodePort

BB

Cloud LB

Cloud LB

External: External IPsapiVersion: v1kind: Servicemetadata: name: backend-servicespec: externalIPs: - 10.240.0.2 - 10.240.0.3 ports: - port: 80 targetPort: 80 protocol: TCP selector: app: backend

apiVersion: v1kind: Servicemetadata: name: backend-servicespec: externalIPs: - 10.240.0.2 - 10.240.0.3 ports: - port: 80 targetPort: 80 protocol: TCP selector: app: backend

$ kubectl get nodes -o json | grep -A 1 "InternalIP" "type": "InternalIP", "address": "10.240.0.2"-- "type": "InternalIP", "address": "10.240.0.3"-- "type": "InternalIP", "address": "10.240.0.4"

$ kubectl get nodes -o json | grep -A 1 "InternalIP" "type": "InternalIP", "address": "10.240.0.2"-- "type": "InternalIP", "address": "10.240.0.3"-- "type": "InternalIP", "address": "10.240.0.4"

External: External IPs

Features

• TCP/UDP• Health checks

kube-proxykube-proxykube-proxykube-proxy

BB

kube-proxykube-proxy

BB BB

8080

10.240.0.2 10.240.0.3 10.240.0.4

8080

External: service LoadBalancerhttps://github.com/kubernetes/contrib/tree/master/service-loadbalancer

1 or more HAProxy, each deployed in a pod

Services -> HAProxy configuration

svcA -> /svcAsvcB -> /svcB

Features

• TCP/UDP, HTTP• URL Mapping• SSL Termination (via Annotations)• Session Persistence (via Annotations)• Multiple algorithms (via Annotations)

External: IngressapiVersion: extensions/v1beta1kind: Ingressmetadata: name: hello-ingressspec: rules: - host: hello.example.com http: paths: - path: /a backend: serviceName: backend-a servicePort: 80 - path: /b backend: serviceName: backend-b servicePort: 8080

apiVersion: extensions/v1beta1kind: Ingressmetadata: name: hello-ingressspec: rules: - host: hello.example.com http: paths: - path: /a backend: serviceName: backend-a servicePort: 80 - path: /b backend: serviceName: backend-b servicePort: 8080

• hello.example/a -> backend-a:80• hello.example/b -> backend-b:8080

External: Ingress

apiVersion: extensions/v1beta1kind: Ingressmetadata: name: hello-ingressspec: tls: - hosts: - hello.example.com secretName: hello-secret rules: - host: hello.example.com

. . .

apiVersion: extensions/v1beta1kind: Ingressmetadata: name: hello-ingressspec: tls: - hosts: - hello.example.com secretName: hello-secret rules: - host: hello.example.com

. . .

apiVersion: v1kind: Secretmetadata: name: hello-secrettype: Opaquedata: tls.crt: <base-64 encoded crt> tls.key: <base-64 encoded key>

apiVersion: v1kind: Secretmetadata: name: hello-secrettype: Opaquedata: tls.crt: <base-64 encoded crt> tls.key: <base-64 encoded key>

New in 1.2: TLS support

External: Ingress

Features

• HTTP Load Balancing• SSL Termination• Content-based routing

How to use it

Ingress Controller must be deployed

External: Ingress

IngressController

IngressController

IngressResources

IngressResources Load Balancer Load Balancer

watches configures

External: Ingress

Cloud Load Balancers

• GCE HTTP Load Balancer

Software Load Balancers

• NGINX

https://github.com/kubernetes/contrib/tree/master/ingress/controllers

NGINX• Layer 4/Layer 7 Load Balancer

• Advanced algorithms

• SSL termination

• Content-based routing

• Limits

• HTTP/2 gateway

• Logging

• Security

• Real-time statistics*

• Layer 7 Session Persistence*

• Dynamic reconfiguration*

* NGINX Plus

Also a webserver and cache

NGINX Ingress ControllerapiVersion: extensions/v1beta1kind: Ingressmetadata: name: hello-ingressspec: rules: - host: hello.example.com http: paths: - path: /a backend: serviceName: backend-a servicePort: 80 - path: /b backend: serviceName: backend-b servicePort: 8080

apiVersion: extensions/v1beta1kind: Ingressmetadata: name: hello-ingressspec: rules: - host: hello.example.com http: paths: - path: /a backend: serviceName: backend-a servicePort: 80 - path: /b backend: serviceName: backend-b servicePort: 8080

upstream backend-a { server 10.3.246.245:80;}

upstream backend-b { server 10.3.246.249:8080;}

server { listen 80; server_name hello.example.com;

location /a { proxy_pass http://backend-a; }

location /b { proxy_pass http://backend-b; }}

upstream backend-a { server 10.3.246.245:80;}

upstream backend-b { server 10.3.246.249:8080;}

server { listen 80; server_name hello.example.com;

location /a { proxy_pass http://backend-a; }

location /b { proxy_pass http://backend-b; }}

NGINX Ingress Controller

1. Watch for Ingress resources

2. Watch for Services and Endpoints: to get IP address of a service or its endpoints in case of a headless service

3. Watch for Secrets

NGINX Ingress Controller

IngressIngress

EndpointsEndpoints

ServiceService

SecretSecret

IngressIngressaffects

Changes in1. Regenerate configuration for the

Ingress

2. Reload NGINX

NGINX Ingress Controller

• NGINX Plus supports re-resolving DNS names in runtime every X seconds

• Doesn’t fail when a name can’t be resolved

• Simplifies implementation: no need to watch for Services and Endpoints

NGINX Ingress Controller• As an example we took the GCE HTTP Load Balancer Ingress Controller

— https://github.com/kubernetes/contrib/tree/master/ingress/controllers/gce

• Written in Go

• Different implementations for NGINX and NGINX Plus

• Deployed in the same container as NGINX. the Controller starts first and then launches NGINX.

NGINX Ingress Controller

• HTTP Load Balancing

• SSL Termination

• Content-based routing

Features

• Advanced algorithms

• Limits

• Access Control

• Logging

• Limits Real-time statistics (NGINX Plus)

• Layer 7 Session Persistence (NGINX Plus)

• Dynamic reconfiguration (NGINX Plus)

• and more

Features, supported by changing NGINX templates

Demo

kube-proxykube-proxykube-proxykube-proxy

TT

kube-proxykube-proxy

CC

80, 44380, 443

BBCC TT

Demo

• tea-rc and tea-svc

• coffee-rc and headless coffee-svc

• Ingress resource cafe-ingress with TLS

• Secret cafe-secret

• NGINX Plus Ingress Controller nginx-plus-ingress-rc

NGINX Ingress Controller

• Expose more NGINX features via Kubernetes resources (Annotations and Config Maps)

• Make it production-ready

• Improve it based on your feedback

Wishlist

The End● Resources: http://tiny.cc/nginx-ingress● NGINX: https://www.nginx.com/● My site: http://linuxjedi.co.uk/● Twitter: @LinuxJedi● Freenode: LinuxJedi● Email: linuxjedi@nginx.com