Kubernetes @ Nanit
-
Upload
chen-fisher -
Category
Technology
-
view
61 -
download
5
Transcript of Kubernetes @ Nanit
![Page 1: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/1.jpg)
Kubernetes @ nanitChen Fisher, VP Engineering
![Page 2: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/2.jpg)
nanitSmart baby monitor
![Page 3: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/3.jpg)
![Page 4: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/4.jpg)
API
Clipper
Focus
Gateway
Grant
Mailman
RTMPRabbitMQ
S3
Stampa
State Department
Timeline
Taper
Algorithms
Communication
![Page 5: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/5.jpg)
MicroservicesDocker
![Page 6: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/6.jpg)
docker-compose up -d
![Page 7: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/7.jpg)
Orchestration
![Page 8: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/8.jpg)
Orchestration
- Service Discovery
- High Availability
- Resource Management
- Port Management
API
Clipper
Focus
Gateway
Grant
Mailman
RTMPRabbitMQ
S3
Stampa
State Department
Timeline
Taper
Algorithms
![Page 9: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/9.jpg)
ECS vs Kubernetes
![Page 10: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/10.jpg)
https://railsadventures.wordpress.com/2015/12/06/why-we-chose-kubernetes-over-ecs/-- Erez Rabih
![Page 11: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/11.jpg)
Service Discovery / ECS
- Offers no service discovery
- One can use external tools like Consul
(https://aws.amazon.com/blogs/compute/service-discovery-via-consul-with-amazon-ecs/)
![Page 12: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/12.jpg)
Service Discovery / Kubernetes
- Has built-in service discovery (http://kubernetes.io/docs/user-guide/services/)
- One can simply refer to a service by its name
- Can group and isolate services by namespaces
![Page 13: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/13.jpg)
High Availability / ECS
- Uses ELB health checks
- Limited to HTTP/TCP health checks
- If not using ELB then only recovers on container exit
- Must create ELB for every service to get health checks
![Page 14: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/14.jpg)
High Availability / Kubernetes
- Offers exec health check in addition to http
- One can perform application level health check
![Page 15: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/15.jpg)
Port Management / ECS
- Maps an ELB to a service which exposes a container port on host
- Cannot have more than one container with same exposed port on same host
- Must manually manage container ports
![Page 16: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/16.jpg)
- Assigns random port for every container
- Maps ELB to the container’s random port
Port Management / Kubernetes
![Page 17: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/17.jpg)
Kubernetes
- AWS
- Kube 1.2.2
- Two clusters: Staging & Production
- 20+ m4-large instances on production (pre-launch)
- 100+ pods
- RDS, DynamoDB outside of cluster
![Page 18: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/18.jpg)
From Dev to Prod
![Page 19: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/19.jpg)
Dev and test locallyclojure, ruby, go, c/c++, elixir
![Page 20: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/20.jpg)
Push to Githubpull requests
![Page 21: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/21.jpg)
Jenkins picks up and testsmarks commit on Github
![Page 22: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/22.jpg)
Manual merge to master
![Page 23: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/23.jpg)
Deployment via Slack
![Page 24: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/24.jpg)
![Page 25: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/25.jpg)
Deployment / directory structure
● nanit/
○ api/■ app/
■ Dockerfile■ code...
■ kube/■ dep.yml
■ Makefile
![Page 26: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/26.jpg)
apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: {{APP_NAME}}spec: replicas: {{REPLICAS}} template: metadata: labels: app: {{APP_NAME}} spec: volumes: - name: some-secret secret: secretName: some-secret containers: - name: {{APP_NAME}} image: {{IMAGE_NAME}}...
Deployment / kube yaml file
![Page 27: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/27.jpg)
Deployment / Makefile
APP_NAME=apiIMAGE_TAG=$(shell git log -n 1 --pretty=format:%h app)IMAGE_NAME=nanit/$(APP_NAME):$(IMAGE_TAG)
define generate-dep sed -e 's/{{IMAGE_NAME}}/$(IMAGE_NAME)/g' kube/dep.ymlendef
docker: sudo docker pull $(IMAGE_NAME) || (sudo docker build -t $(IMAGE_NAME) app && sudo docker push $(IMAGE_NAME))
deploy: docker $(call generate-dep) | kubectl apply -f -
migrate: …
test:...
![Page 28: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/28.jpg)
Deployment / config
from Makefile:REPLICAS?=$(shell curl -s config/$(NANIT_ENV)/api/replicas)
from kube yaml file:spec: replicas: {{REPLICAS}}
![Page 29: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/29.jpg)
Post deployment(Monitoring & Alerts)
![Page 30: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/30.jpg)
Post deployment (Monitoring & Alerting)
- Currently using built in Kibana & Grafana
- Alerting to Slack using proprietary service
![Page 31: Kubernetes @ Nanit](https://reader034.fdocuments.net/reader034/viewer/2022042618/589e2b241a28ab5c128b54ff/html5/thumbnails/31.jpg)
Thank You!