Post on 07-Jan-2017
CI/CD with Kubernetes
Sandeep Parikh@crcsmnkyHead of Solutions, US EastGoogle Cloud Platform
Why Is This ImportantBulletproof process to go from code to production
Centralized, repeatable building/tagging/pushing
Iterate quickly and reliably
Hands off my kubectl
Continuous Integration
TestCode Commit Push TestBuild
The ideal development flow
Continuous Integration
TestCode Commit Push TestBuild
The ideal development flow
TestD’oh!Code Commit Push Test
Crap!Build
Whoops!
The real development flow
Continuous Delivery
TestBuild Deploy
Testing
Staging
Continuous Deployment
DeployApproveDeliver $$$
Why Continuous?Fail fast
The build broke!
Triage fast
Why did the build break?
Deliver faster
Better software in customers hands!
Tools
Jenkins
Kubernetes Cluster (GKE)
Kubernetes Cluster
Node 1 Node 3Node 2
Kubernetes Cluster (GKE)
Jenkins Leader
Node 1 Node 3Node 2
Jenkins Leader
Kubernetes Cluster (GKE)
Jenkins Service
Node 1 Node 3Node 2
Jenkins Leader
Jenkins Service
Kubernetes Cluster (GKE)
Jenkins Ingress
Node 1 Node 3Node 2
Jenkins Leader
Jenkins Service
Google Cloud Load Balancer
Kubernetes Cluster (GKE)
Jenkins Builders
Node 1 Node 3Node 2
Jenkins Leader
Jenkins Builder
Jenkins Builder
Jenkins Builder
Jenkins Builder
Jenkins Service
Jenkins Builder
Google Cloud Load Balancer
Microservices
Backend
Frontend
Kubernetes Cluster (GKE)
Node 1
Microservices
Node 2
Kubernetes Cluster (GKE)
Node 1
Microservices
Node 2
BackendBackend Backend
Kubernetes Cluster (GKE)
Microservices
Node 2Node 1
Backend
Backend
Service
Backend Backend
Google Cloud Load Balancer
Kubernetes Cluster (GKE)
Microservices
Node 2Node 1
Backend
Frontend Frontend
Backend
Service
Frontend
Backend Backend
Google Cloud Load Balancer
Kubernetes Cluster (GKE)
Microservices
Node 2Node 1
Backend
Frontend Frontend
Backend
Service
Frontend
Frontend
Service
Backend Backend
Google Cloud Load Balancer
Microservices & NamespacesVirtual clusters backed by the same physical cluster
Divide cluster resources by different use cases
Supports quotas for managing resources
Test, Staging, Production, etc.
Pods
Time
Microservices & Rolling Update DeploymentsReplace Replication Controller Deployment A with Replication Controller Deployment B by updating one Pod at a time.
A
B
A
B
AB A
B
A
B
A
B
Rolling Update vs. DeploymentsImperative vs Declarative
Rollback
kubectl edit and kubectl apply
Server-side
...
Workflow
Kubernetes Workflow
CreatePushPackage Expose
Mapping To JenkinsWorkflow plugin, Freestyle doesn’t fit
Define a flexible, extensible, script-based CD pipeline
Groovy for scripting actions
Supports human input/approval
node('docker') { checkout scm // Kubernetes cluster info def cluster = 'gtc' def zone = 'us-central1-f' def project = 'REPLACE_WITH_YOUR_PROJECT_NAME' // Run tests stage 'Go tests' docker.image('golang:1.5.1').inside { sh('go get -d -v') sh('go test') } // Build image with Go binary stage 'Build Docker image' def img = docker.build("gcr.io/${project}/gceme:${env.BUILD_TAG}") sh('gcloud docker -a') img.push()
Jenkinsfile
// Deploy image to cluster in dev namespace stage 'Deploy to QA cluster' docker.image('buildpack-deps:jessie-scm').inside { sh('apt-get update -y ; apt-get install jq') sh('export CLOUDSDK_CORE_DISABLE_PROMPTS=1 ; curl https://sdk.cloud.google.com | bash') sh("/root/google-cloud-sdk/bin/gcloud container clusters get-credentials ${cluster} --zone ${zone}") sh('curl -o /usr/bin/kubectl https://storage.googleapis.com/.../release/v1.0.1/bin/linux/amd64/kubectl ; chmod +x /usr/bin/kubectl') sh("kubectl --namespace=staging rollingupdate gceme-frontend --image=${img.id}") sh("kubectl --namespace=staging rollingupdate gceme-backend --image=${img.id}") sh("echo http://`kubectl --namespace=staging get service/gceme --output=json | jq -r '.status.loadBalancer.ingress[0].ip'`> staging") }
Jenkinsfile
// Deploy to prod if approved stage 'Approve, deploy to prod' def url = readFile('staging').trim() input message: "Does staging at $url look good? ", ok: "Deploy to production" sh('gcloud docker -a') img.push('latest') docker.image('buildpack-deps:jessie-scm').inside { sh('apt-get update -y ; apt-get install jq') sh('export CLOUDSDK_CORE_DISABLE_PROMPTS=1 ; curl https://sdk.cloud.google.com | bash') sh("/root/google-cloud-sdk/bin/gcloud container clusters get-credentials ${cluster} --zone ${zone}") sh('curl -o /usr/bin/kubectl https://storage.googleapis.com/.../release/v1.0.1/bin/linux/amd64/kubectl ; chmod +x /usr/bin/kubectl') sh("kubectl --namespace=production rollingupdate gceme-frontend --image=${img.id}") sh("kubectl --namespace=production rollingupdate gceme-backend --image=${img.id}") sh("echo http://`kubectl --namespace=production get service/gceme --output=json | jq -r '.status.loadBalancer.ingress[0].ip'`") }}
Jenkinsfile
Workflow
Repo
Kubernetes Cluster (GKE)Github
Jenkins
µservice
µservice
Default Staging
Production
Workflow
Repo
Kubernetes Cluster (GKE)Github
Jenkins
µservice
µservice
Default Staging
Production1
Push1
Workflow
Repo
Kubernetes Cluster (GKE)Github
Jenkins
µservice
µservice
Default Staging
Production1 2
Push
Build
1
2
Workflow
Repo
Kubernetes Cluster (GKE)Github
Jenkins
µservice
µservice
Default Staging
Production1 2
3
Push
Build
Clone
1
2
3
Workflow
Repo
Kubernetes Cluster (GKE)Github
Jenkins
µservice
µservice
Default Staging
Production1 2
3
4
Push
Build
Clone
Stage
1
2
3
4
Workflow
Repo
Kubernetes Cluster (GKE)Github
Jenkins
µservice
µservice
Default Staging
Production1 2
3
5
4
Push
Build
Clone
Stage
Approve
1
2
3
4
5
Workflow
Repo
Kubernetes Cluster (GKE)Github
Jenkins
µservice
µservice
Default Staging
Production1 2
3
5
4
Push
Build
Clone
Stage
Approve
Deploy
1
2
3
4
5
6
6
Questions, Comments@crcsmnkyRepo
https://github.com/GoogleCloudPlatform/continuous-deployment-on-kubernetesUpdates coming (built pre-Jenkins 2.0, which includes Workflow)Changes include Ingress, Deployments, Jenkins+K8S Plugin (auto-spawn builders)Docs
Automated Image Builds with Jenkins, Packer, and KubernetesDistributed Load Testing using KubernetesReal-time data analysis with Kubernetes, Google Cloud Pub/Sub, and BigQuery
Resources
Join the Google Cloud Platform User Group