1

Scaling DevOps with Puppet and Perforce

Christoph Leithner

Founder

CELIX Hard- & Software

VertriebsgmbH

http://www.celix.at

Logo area

2

Agenda

• What is DevOps?

• IT Automation with Puppet

• Scaling DevOps with Perforce

• Agile Flow of Change for Puppet Modules

3

What is DevOps?

4

DevOps

What is DevOps?

Source: http://commons.wikimedia.org/wiki/File:Devops.png by Rajiv Pan

Technology

Operations

DevOps

5

Area 1: Extend delivery to production

DEV OPS

Area 2: Extend operations feedback to project

Area 3: Embed project knowledge into operations

Area 4: Embed operations knowledge into project

Source: http://www.jedi.be/blog/2012/05/12/codifying-devops-area-practices/ by Patrick Debois

DevOps Areas - codified by Patrick Debois

6

• IT Automation: Puppet

• Shared Version Control: Perforce

• Build and Deploy: Atlassian Bamboo

• Process Management: Atlassian JIRA

DevOps Tools - Overview

7

IT Automation with Puppet

8

Define. 1 2 Simulate.

3 Enforce.

4 Report.

How Puppet works

9

Database Web Server Application Server Security Mo

du

les

Via Puppet Dashboard Custom External Source (CMDB, LDAP, etc.)

Define your

resources in

modules.

1

Assign resource

relationships

automatically.

2

Reuseable,

composable

configurations.

3

Web Servers Database Servers Application Servers

Define Composable Deployments with Puppet

10

user { 'dave':

ensure => present,

uid => '507',

gid => 'admin',

shell => '/bin/zsh',

home => '/home/dave',

managehome => true,

}

Puppet Resource Abstraction Layer (RAL)

11

class ssh {

package { ‘ssh’:

ensure => latest,

}

file { ‘/etc/ssh/sshd_config’:

source => ‘puppet:///modules/ssh/sshd_config’,

}

service { ‘sshd’:

ensure => running,

}

Package[‘ssh’]->File[‘/etc/ssh/sshd_config’]~>Service[‘sshd’]

Package[‘ssh’]~>Service[‘sshd’]

}

class ssh Puppet RAL Example:

12

module_name

manifests

files

templates

lib

spec

tests

Modulefile

Puppet Modules – Deployments as Code

13

name 'puppetlabs-apache'

version '0.6.0'

source 'git://github.com/puppetlabs/puppetlabs-apache.git'

author 'puppetlabs'

license 'Apache 2.0'

summary 'Puppet module for Apache'

description 'Module for Apache configuration'

project_page 'https://github.com/puppetlabs/puppetlabs-apache'

## Add dependencies, if any:

dependency 'puppetlabs/firewall', '>= 0.0.4'

dependency 'puppetlabs/stdlib', '>= 2.2.1'

Puppet Modulefile for puppetlabs-apache

14

http://forge.puppetlabs.com

15

puppet module install puppetlabs-apache --version 0.6.0

firewall

stdlib

apache

Puppet Module Subcommand

16

puppet module install puppetlabs-ntp --version 0.2.0

puppet module install puppetlabs-apache --version 0.6.0

firewall

ntp

stdlib

apache

Sets of Modules

17

puppet module install puppetlabs-ntp --version 0.2.0

puppet module install puppetlabs-apache --version 0.6.0

firewall

Puppet Environment

ntp

stdlib

apache

Multiple Environments on a single Puppet Master

Sets of Modules – Puppet Environments

18

Option A: Version each Module you need to change locally

A Git Repository for each Puppet Module you change

firewall

Puppet Environment

ntp

stdlib

apache

Versioning Puppet Modules/Environments

19

Option B: Version the Environment Folder

A single Git Repository for the Puppet Environment

firewall

Puppet Environment

ntp

stdlib

apache

Versioning Puppet Modules/Environments

20

Option C:

A single Git Repository for the Puppet Environment that uses Git

Submodules to include the Git Repositories for the Modules

firewall

Puppet Environment

ntp

stdlib

apache

Version the Environment Folder and all Modules

Versioning Puppet Modules/Environments

21

main

forge

apache

stdlib

firewall

Managing Puppet Environments with Perforce Git Fusion

apache

stdlib

firewall

environment

22

main

forge

apache

stdlib

firewall

Managing Puppet Environments with Perforce Git Fusion

environment

Puppet Environment

apache

stdlib

firewall

23

feature_4

release_12.3

forge

feature_1

hotfix_1

release_13.1

feature_2 feature_3

Mainline Model for Puppet Modules

mainline

24

• Modularity – Puppet Environment Repository is

assembled from multiple Module Repositories

• Easy Workflow - NO Git Submodules

• Central Change History in Perforce

• Fine grained Permissions

Perforce Git Fusion - Benefits

25

Puppet Console

Puppet Module Forge

Build and Deploy Version Management Process Management

Puppet Agent Facter

NODE

Puppet Agent Facter

NODE

Puppet Agent Facter

NODE

Puppet Agent Facter

NODE

Puppet Agent Facter

NODE

Puppet Master

Modules/Environments

Puppet Architecture

26

Catalog 1

2

3

4 Report

Collector Report

Report

Facts SSL secure

Encryprion

on all data

transport

Puppet

Master

Node

Puppet Data Flow

27

Scaling DevOps

Agile Flow of Change for Puppet Modules

28

• More Teams/Services adopt DevOps Practices

• Number of Environments and Modules grows

• Fine grained Security becomes more important

• Central Change history is required

• Processes are standardized

• Environments are standardized

• Puppetized Deployments are used across all Stages

What means Scaling DevOps?

29

release_12.3

rails_webapp

hotfix_1

release_13.1

feature_1 feature_2

apache

zend_webapp

java_app

mainline

Agile Flow of Change for Puppet Modules

30

rel_12.3

java_app

rel_13.1

zend_webapp apache

feature_2 feature_1

rails_webapp

main

apache_only

Agile Flow of Change for Puppet Modules

31

apache

rails_webapp

zend_webapp

All trademarks and logos are property of their respective owners

main

rails

webapp rails

webapp zend

webapp zend

webapp

Perforce Streams Modularity

rails

webapp zend

webapp

32

Development Quality Assurance Staging Production

Feature Stream Service Stream Mainline Stream Release Stream/Tag

Build Automation - The Puppet Environment Lifecycle

33

Puppet Console

Puppet Module Forge

Build and Deploy Version Management Process Management

Puppet Agent Facter

NODE

Puppet Agent Facter

NODE

Puppet Agent Facter

NODE

Puppet Agent Facter

NODE

Puppet Agent Facter

NODE

Puppet Master

Modules/Environments

Summary

34

Thank you.

perforce@celix.at