Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenkins, r10k and Git...

Continuously deliver your puppet code withjenkins, r10k and git

Toni Schmidbauer

October 14, 2014

whoami

I [email protected]

I http://stderr.at

I http://github.com/tosmi

Agenda

I A short story about configuration management

I What is continuous delivery

I Tools used to achieve continuous delivery

I DEMO

I Things to improve

A short story about configuration management (CM)

I We manage a very diverse environment of UNIX/LinuxSystems (Solaris 10/11 SPARC/i386, AIX, RHEL/CentOS5/6/7)

I We’ve got around 1000 nodes in total

I Before CM we had strict standards on how to manage thesesystems

I The problem:count(team members) == count(standards)

I So configuration management is the solution to all ourproblems

The solution to all our problems

I Broke our systems

The solution to all our problems

I Broke our systems

WHY????

Problems with our old CM system

I Deployments sucked

I Deployment via manual tagging and checkout, so mistakeshappened

I Deployment in stages, but we always had to cross our fingers

I Testing sucked

I No UnittestsI No acceptance tests

I No immediate feedback if things where OK or not

I Systems installed without CM are hard to bring under CMcontrol

I Every system was a special case

I Deployments suckedI Deployment via manual tagging and checkout, so mistakes

happenedI Deployment in stages, but we always had to cross our fingers

I Testing sucked

I Testing suckedI No UnittestsI No acceptance tests

So what’s our solution?

Continuous delivery

I is a pattern for getting software from development to release

I this pattern is called the deployment pipeline

1Continuous Delivery: Jez Humble, David Farley

Continuous delivery

I is a pattern for getting software from development to release

I this pattern is called the deployment pipeline

1Continuous Delivery: Jez Humble, David Farley

The deployment pipeline

Commit StageAutomated unittesting

Automated acceptancetesting

Tag release Deployment

Why are we using continuous delivery

I When you automate your deployment,

I less mistakes will happen and the same mistake will nothappen twice

I less mistakes means less stress when deployingI less stress means you are going to deploy more oftenI more deployments means a more flexible environment

I When you automate your deployment,I less mistakes will happen and the same mistake will not

happen twice

I less mistakes means less stress when deployingI less stress means you are going to deploy more oftenI more deployments means a more flexible environment

happen twiceI less mistakes means less stress when deploying

I less stress means you are going to deploy more oftenI more deployments means a more flexible environment

happen twiceI less mistakes means less stress when deployingI less stress means you are going to deploy more often

I more deployments means a more flexible environment

happen twiceI less mistakes means less stress when deployingI less stress means you are going to deploy more oftenI more deployments means a more flexible environment

Tools to build a deployment pipeline

Jenkins

I Jenkins is an Open Source continuous integration server

I It’s purpose is to execute and monitor jobs

I Jobs are shell scripts or any other thing that’s executable andreturns 0 on success

I Many plugins available to extend Jenkins (e.g. git,build-pipeline, monitor)

I You can link jobs together, thats our pipeline

Jenkins II

Monitoring with Jenkins

I git push triggers the deployment pipeline

I one central repository for internal modules

I gitolite for access controlI 3 main branches

I developmentI testingI production

I feature branches for new site local modules

I hiera data is in the same repository

I git push triggers the deployment pipeline

I one central repository for internal modules

I gitolite for access controlI 3 main branches

I developmentI testingI production

I feature branches for new site local modules

I hiera data is in the same repository

GIT repository layout

I modules/:I where r10k stores external (forge, github) modules

I site/:I site local modules, we do not want to share

I hiera/:I our hiera yaml files

I Puppetfile:I config file for r10k that specifies which external modules we

I Vagrantfile:I To boot a development puppet environment on your local

workstation

GIT workflow

Puppet developer

Development

Puppet Master

Testing

Puppet Master

Production

Puppet Master

Feature

Branch

Testing

Branch

Production

Branch

Production Nodes

Testing Nodes (~40)

Dev Nodes (7−8)

1 Features Branches get automatically created on Puppet Master (Dynamic Environments)

2 Development Branch gets deployed on commit via Jenkins

3 Testing Branch gets deployed via GIT tag

Jenkins

It’s all the same for Hiera yaml files!

Development

Branch

pushing to testing triggers a deployment

pushing to production triggers a deployment

Production Branch gets deployed via GIT tag

I a tool to deploy puppet environments and modules

I every git branch gets deployed to a corresponding puppetenvironment

I it also downloads and installs modules from puppetforge orgithub

I in the current version (1.3.2) dependencies have to bemanaged manually

Example Puppetfile

1 f o r g e ’ f o r g e . puppe t l ab s . com ’

3 mod ’ puppe t l ab s /ntp ’ , ’ 3 . 1 . 2 ’mod ’ puppe t l ab s / p o s t g r e s q l ’ , ’ 3 . 4 . 2 ’

5 mod ’ puppe t l ab s / s t d l i b ’ , ’ 4 . 3 . 2 ’mod ’ puppe t l ab s / f i r e w a l l ’ , ’ 1 . 1 . 3 ’

7 mod ’ puppe t l ab s /apache ’ , ’ 1 . 1 . 1 ’mod ’ puppe t l ab s / lvm ’ , ’ 0 . 3 . 2 ’

9 mod ’ n o s o l u t i o n s /tsm ’ , ’ 0 . 2 . 2 ’mod ’ saz / sudo ’ , ’ 3 . 0 . 6 ’

11 mod ’ s p i e t t e / s e l i n u x ’ , ’ 0 . 5 . 4 ’

13 mod ’ concat ’ ,: g i t => ’ h t t p s : // g i t hub . com/ puppe t l ab s / puppe t l abs−concat ’ ,

15 : commit => ’ feba3096c99502219043b8161bde299ba65e7b8a ’

You are able to pin to a git tag / branch / commit hash

a word on testing

I you must have unit tests for your puppet code: rspec-puppet

I for acceptance tests there’s puppetlabs/beaker

I you need to test everything to get most out of the buildpipeline

I we testI internal puppet modulesI hiera dataI puppet configuration

I all internal modules are required to have rspec tests

rspec-puppet example

samplemodule/manifests/init.pp

1 c l a s s samplemodule ( $message = ’ de f au l tmes sage ’ ) {n o t i f y { ’ samplemessage ’ :

3 message => ”This i s the sample module , my message i s : $message ” ,}

samplemodule/spec/classes/samplemodules spec.rb

1 r e q u i r e ’ s p e c h e l p e r ’

3 d e s c r i b e ’ samplemodule ’ , : t ype => : c l a s s docon t e x t ’ w i th d e f a u l t paramete r s ’ do

5 i t { shou l d c o n t a i n n o t i f y ( ’ samplemessage ’ ) }end

beaker example

1 r e q u i r e ’ s p e c h e l p e r a c c e p t a n c e ’

3 d e s c r i b e ’ p r o f i l e s : : o s s ba s e c l a s s ’ doi t ’ s hou l d work wi th no e r r o r s ’ do

5 a p p l y man i f e s t ( ’ i n c l u d e p r o f i l e s : : o s sbase ’ , : c a t c h f a i l u r e s => t r u e )end

Do try this at home

I You need:I Vagrant from http://vagrantup.comI VirtualboxI Git client

I You have to run:I git clone

https://github.com/tosmi/puppetcamp2014.gitI cd puppetcamp2014I vagrant upI vagrant ssh

Things we have to improve

I We need more test Systems (CentOS/RHEL/Solaris/AIX?)

I We need more acceptance tests

I Once again use stages in production

I Our documentation

Things puppetlabs should improve

I Puppetlabs should package beaker as a rpm/deb whatever,gems suck in production

Summary

I Continuous Delivery is implemented via a deployment pipeline

I Within the pipeline everything is automated

I When everything is automated you need to test everything

I When everything is automated and well tested deployingbecomes easy

I Tools we are using to implement a deployment pipelineI Jenkins: to execute jobsI GIT: version controlI Gitolite: access control and authorization for GITI r10k: install puppet modules / create puppet environments

from branchesI rspec-puppet: unittests for puppetI puppetlabs/beaker: acceptance tests for puppet

Summary

Thanks for your attention!

Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenkins, r10k and Git...

Software

Transcript of Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenkins, r10k and Git...