Post on 11-Jul-2015
©2014 CloudBees, Inc. All Rights Reserved
CI and CD Across the Enterprise with Jenkins
Cyrille Le Clerc, CloudBees
©2014 CloudBees, Inc. All Rights Reserved
About Me
Cyrille Le Clerc
@cyrilleleclerc
Open Source
CTO
DevOps, Infra as Code, Con9nuous Delivery
Product Management
©2014 CloudBees, Inc. All Rights Reserved
Agenda
• Con9nuous Delivery & Jenkins Workflow • Scaling Jenkins • Con9nuous Delivery with Jenkins and Puppet and Chef
©2014 CloudBees, Inc. All Rights Reserved
Continuous Delivery
©2014 CloudBees, Inc. All Rights Reserved
Continuous Delivery
• “Reliable So+ware Releases through Build, Test, and Deployment Automa=on”
• DONE = Shippable into Produc9on
©2014 CloudBees, Inc. All Rights Reserved
Continuous Integration Steps
Con$
nuou
s Integra$o
n Source Code
Quality Analysis
Compile &Unit Tests
Binaries
©2014 CloudBees, Inc. All Rights Reserved
Continuous Delivery Steps
Con$
nuou
s Delivery
DEV
DEV
OPS
QA
Source Code
Quality Analysis
Compile &Unit Tests
Binaries
Perfs Environment
Func Test Environment
UAT Environment
Production Environment
GO LIVE!
Release!
Staging Environment
©2014 CloudBees, Inc. All Rights Reserved
Continuous Delivery Pipeline
Continuous Delivery Pipeline
Compile &Unit Tests
Perfs Tests
Selenium Tests Deploy UAT
Sonar Analysis
Deploy Staging Deploy Production
©2014 CloudBees, Inc. All Rights Reserved
Jenkins: Hub of Continuous Delivery
• Connect the dots
• It is the process Source Code
Perfs Environment
Func Test Environment
UAT Environment
Binaries
Quality Analysis
Compile &Unit Tests
©2014 CloudBees, Inc. All Rights Reserved
CD Pipeline è Workflow
Continuous Delivery Pipeline
Compile &Unit Tests
Perfs Tests
Selenium Tests Deploy UAT
Sonar Analysis
Deploy Staging Deploy Production
©2014 CloudBees, Inc. All Rights Reserved
What we need
• Complex pipelines involving mul9ple stages • Non-‐sequen$al logic such as loops, forks … • Long-‐running builds must survive outages • Interac$ons with humans including pauses, input • Restartable builds in case of transient errors • Reusable defini$ons to avoid duplica9on • Comprehensible scripts with one clear defini9on
©2014 CloudBees, Inc. All Rights Reserved
Workflow until today
• Job chaining
• Build Flow Plugin
• Build Pipeline Plugin
• Build Delivery Pipeline
©2014 CloudBees, Inc. All Rights Reserved
Workflow until today
• Many atomic jobs • Hard to share variables/state between jobs • Limited logic • Mix build triggers, parameterized build …
©2014 CloudBees, Inc. All Rights Reserved
Build Flow Plugin
• Did have scriptability and extensibility • Did not address configura9on “sprawl” • Disjointed view of what really ran • No ability to survive restarts • Almost good enough but could not go further
©2014 CloudBees, Inc. All Rights Reserved
New Workflow Syntax
©2014 CloudBees, Inc. All Rights Reserved
Demo time! https://github.com/cyrille-leclerc/spring-petclinic/blob/petclinic-1.0.0-clc/src/main/jenkins/workflow.groovy
©2014 CloudBees, Inc. All Rights Reserved
CD Pipeline è Workflow
Continuous Delivery Pipeline
Compile &Unit Tests
Perfs Tests
Selenium Tests Deploy UAT
Sonar Analysis
Deploy Staging Deploy Production
©2014 CloudBees, Inc. All Rights Reserved
Key features
• En9re flow is one concise Groovy script – For loops, try-‐finally, fork-‐join …
• Can restart Jenkins while flow is running • Allocate slave nodes and workspaces
– As many as you want, when you want
• Stages throYle concurrency of builds • Human input/approval integrated into flow • Standard project concepts: SCM, ar9facts, plugins
©2014 CloudBees, Inc. All Rights Reserved
Groovy DSL vs. Graphical Workflow
• Familiar control flow construc9on • Familiar “tools” for building abstrac9ons
Func9ons, classes, variables, …
• Workflow in version control • As opposed to:
Graphical workflow designer
©2014 CloudBees, Inc. All Rights Reserved
Project setup
• One workflow is defined as a job • Single script for all steps • Build triggers & parameters like regular projects • Each workflow execu9on is a regular Jenkins build displayed in regular Jenkins views
• Graphical visualiza9on of actual build possible Not of visualiza9on job defini9on but of build execu9on
©2014 CloudBees, Inc. All Rights Reserved
Non-sequential logic
try {} catch {}
©2014 CloudBees, Inc. All Rights Reserved
Interaction with humans
©2014 CloudBees, Inc. All Rights Reserved
Restartable build / checkpoint
©2014 CloudBees, Inc. All Rights Reserved
Visualization
©2014 CloudBees, Inc. All Rights Reserved
Stages
• Semaphore • Visualiza9on
©2014 CloudBees, Inc. All Rights Reserved
Comprehensible scripts
©2014 CloudBees, Inc. All Rights Reserved
Scaling with Jenkins
©2014 CloudBees, Inc. All Rights Reserved
• How do I scale Jenkins from 10 to 1000 jobs? Projects? Users?
• How do I comply with corporate
rules?
• How do I keep Jenkins flexible for project teams?
Problem Statement: Scale!
©2014 CloudBees, Inc. All Rights Reserved
Typical Jenkins installations
Let’s think about this for a few minutes
©2014 CloudBees, Inc. All Rights Reserved
The top down “structured” and “lets grow vertically” approach
5 projects 0 Slaves
25 projects 5 Slaves
75 projects 50 Slaves
150 projects 400 Slaves
©2014 CloudBees, Inc. All Rights Reserved
Vertical Scaling issues
When is the next failure? Single point of failure
©2014 CloudBees, Inc. All Rights Reserved
The “organic” agile growth – “let each group do what they want to” approach
Horizontally… virally
©2014 CloudBees, Inc. All Rights Reserved
• Share executors between masters – Not possible today
Horizontal Issues: I Would Like to…
Master-‐b
Master-‐a
Master-‐c
©2014 CloudBees, Inc. All Rights Reserved
• Share security information – Security realm and roles
• Push configurations & upgrades
Master-‐b
Master-‐a
Master-‐c
Horizontal Issues: I Would Like to…
©2014 CloudBees, Inc. All Rights Reserved
Jenkins Opera9ons Center
oc.cloudbees.com Sample Topology
Master-‐b
Master-‐a
Master-‐c
Shared Roles , Security Realm and versions
AYached-‐master client
AYached-‐master client
AYached-‐master client
pushed to client master
pushed to client master
©2014 CloudBees, Inc. All Rights Reserved
Demo time!
©2014 CloudBees, Inc. All Rights Reserved
Continuous Delivery traceability with Jenkins and puppet and Chef
xxx Environment
©2014 CloudBees, Inc. All Rights Reserved
Puppet environments
Prod style environment: – Master & agents
Dev Environment
Puppet Agentpetclinic.war
Application Serverwar file + puppet manifests
& modules
puppet apply
“Production style” Environment
Puppet Master
Puppet Agentpetclinic.war
Application Serverwar file + puppet manifests
& modules
Dev environment: – Standalone agents + “puppet apply”
©2014 CloudBees, Inc. All Rights Reserved
Test environment with Puppet and Vagrant
Dev Environment
Puppet Agentpetclinic.war
Application Serverwar file + puppet manifests
& modules
vagrant provisionpuppet apply
Vagrant to create “clean” puppet managed servers
©2014 CloudBees, Inc. All Rights Reserved
CI with Jenkins and Vagrant and Puppet
Jenkins builds war file and puppet modules and deploys via vagrant
puppet deployment
modulejava code Dev Environment
Puppet Agentpetclinic.war
Application Serverwar file + puppet manifests
& modules
vagrant provisionpuppet apply
©2014 CloudBees, Inc. All Rights Reserved
CI with Jenkins and Vagrant and Puppet
Jenkins builds war file and puppet modules and deploys via vagrant
puppet deployment
modulejava code Dev Environment
Puppet Agentpetclinic.war
Application Serverwar file + puppet manifests
& modules
vagrant provisionpuppet apply
puppet deployment
modulejava code Dev Environment
Puppet Agentpetclinic.war
Application Serverwar file + puppet manifests
& modules
vagrant provisionpuppet apply
notification
©2014 CloudBees, Inc. All Rights Reserved
Jenkins – Puppet Integration
©2014 CloudBees, Inc. All Rights Reserved
Demo time! https://github.com/CloudBees-community/vagrant-puppet-petclinic/tree/vagrant-puppet-petclinic-0.1.0 jenkins-workflow-with-vagrant-and-puppet.groovy: https://gist.github.com/cyrille-leclerc/7f621c341e193622c844
©2014 CloudBees, Inc. All Rights Reserved
Jenkins & Puppet
• Jenkins Deployment No9fica9on Plugin – hYps://wiki.jenkins-‐ci.org/display/JENKINS/Deployment+No9fica9on+Plugin
• Puppet module for Jenkins tracking – hYps://github.com/jenkinsci/puppet-‐jenkinstracking
• Puppet Labs plugin for Jenkins – hYps://wiki.jenkins-‐ci.org/display/JENKINS/Puppet+Plugin
• CloudBees DevOps page – hYps://developer.cloudbees.com/bin/view/Main/DevOps
• This demo – hYps://github.com/CloudBees-‐community/vagrant-‐puppet-‐petclinic – Jenkins Workflow with Vagrant and Puppet: hYps://gist.github.com/cyrille-‐leclerc/7f621c341e193622c844
©2014 CloudBees, Inc. All Rights Reserved
Jenkins & Chef?
• Similar concepts • Jenkins Deployment No9fica9on Plugin
– hYps://wiki.jenkins-‐ci.org/display/JENKINS/Deployment+No9fica9on+Plugin
• Puppet module for Jenkins tracking – hYps://github.com/jenkinsci/puppet-‐jenkinstracking
• Jenkins Chef Tracking – hYps://wiki.jenkins-‐ci.org/display/JENKINS/Chef+Tracking+Plugin
• CloudBees DevOps page – hYps://developer.cloudbees.com/bin/view/Main/DevOps
©2014 CloudBees, Inc. All Rights Reserved
Conclusion
• Jenkins workflow to build Con9nuous Delivery pipelines
• Jenkins Opera9ons Center by CloudBees to scale Jenkins
• Puppet & Chef deployments can be integrated in Jenkins
©2014 CloudBees, Inc. All Rights Reserved