Migrating to Puppet 4.0

0.27

2.7.x

3.x

3.x future parser

4.x

Puppet 4.0 Language

henrik.lindberg@puppetlabs.com

@hel

notice 1 => 1notice 1 + 2 => syntax errornotice(1 + 2) => 3notice 1,2,3 => 1 2 3notice [1,2,3] => 1 2 3notice [1,2],3 => 1 2 3notice 5 in [5] => truenotice (4+1) in [5] => error, not a stringnotice 5 in [4+1] => falsenotice 5.0 in [5] => falsenotice(0xff =~ /5/) => falsenotice((0xfe+1) =~/5/) => truenotice { a=>10 } => error, no title

Godzilla on TypeJuggling….

Puppet 4.0• Language Cleanup– Sanity, Principle of Least Surprise– Expressions, Expressions, Expressions

• Features– Misc enhancements– Resource Expression Features– Iteration– Type System / Optional Typing– Embedded Puppet Templates (EPP)– Heredoc– Data in Modules and Environments

Cleanup

• Language Specification – yes we have one– https://github.com/puppetlabs/puppet-specifications

• Numbers are Numbers• No magic to-string for =~ !~ in• Upper case bare words are Type References• Empty string is not undef (and thus "thruty")• Interpolation follows specified rules• +=, []= removed, no more mutation

Migrating

Signs Your Goldfish is Dead

– Goldfish skin easily cracks when flexed

– Goldfish is completely dried out

– Eyes of goldfish are concave instead of convex

– Pupils of goldfish are gray– Body parts of the goldfish

are clearly missing

4.0

+ future

3.8

2.7

To 4.x from < 3.8 no parser=future• First Migrate to latest 3.8.x– Coming from < 3.0 (released 2012)

• Dynamic Scoping removed • Address Deprecations

– Switch to Directory Environments• Legacy Environments removed in 4.x

– Use PE Catalog Preview with same code in two environments • (demos on youtube)• Turn on future parser in one to help with migration to 4.x Language• FOSS users can use similar free catalog compare tools (with more manual work)

– Do changes in backwards compatible way– Update modules– Gradually migrate nodes – "future parser" production safe

To 4.x from 3.8 with parser=future

• Relatively painless– Standup a new 4.x master

• Packaging and paths changed in 4.x– The "all in one" agent– Same Ruby everywhere– New location for config files, hiera data etc.

• Enjoy new PE 2015.3 / 4.3 features– Application Management– Data in Modules

Migrating to Puppet 4.0

Language Changes

concat/append/merge/delete 1(6)• Concatenate Arrays with +

[1,2,3] + [4,5,6] => [1,2,3,4,5,6]

• Merge Hashes with +{a=>1} + {b=>2} => {a=>1, b=>2}

• Delete with -[1,2,3] – [2, 3] => [1]{a=>1, b=>2, c=>3} – [a,c] => {b=>2}

• Append to Array with <<[1,2,3] << 4 => [1,2,3,4]

[1,2,3] << [4,5,6] => [1,2,3,[4,5,6]]

unfold/splat 2(6)• Unfold with unary *

$a = [2,3]$b = [1, *$a, 4] => [1,2,3,4]

• Unfold in case option, selector and call$a = [1,2,3]case 1 { *$a : { # 1 or 2 or 3

}}notice *$a => 1,2,3notice $a => [1,2,3]

substrings 3(6)

• Substring in string'cd' in "abcde" => true

• Substring with []"xyzabcdef"[3,3] => "abc""xyzabcdef"[3] => "a""xyzabcdef"[3,-2] => "abcde"

regexp string/interpolate 4(6)

• Matches with Regexp in String form $a = "example.com" $url =~ "http://$a.*"

Error message improvements 5(6)

• Detailed Error Messages– Semantic validation unless lex or syntax errors – Outputs position on line– Can report more than one error

Expression Based Grammar 6(6)

• if, unless, case are expressions notice if 1 > 2 { true } else { false } # => false

$a = case 2 { 1, 2, 3: { yes } default: { no } } # => $a == yes

Resource Expression

Local Defaultsfile { default: mode => '444', owner => 'admin'; title: . . . ;}

Unfold Hash of attributesfile { default: * => $defaults_hash; 'tmp/foo': mode => '666', * => $params_hash;}

Create Resources Equiv. in PuppetResource[$type] { default: * => $defaults_hash; $titles: * => $params_hash;}

LOOOOOOOPS

• Iterate over:– Arrays– Hashes– Strings– Integer ranges

• Implemented as functions taking callable code blocks (lambdas) = open design

• Calls can now be expressed from left to right using '.' notation

each

• Do something with each element• Returns LHS

[1,2,3].each |$x| { notice $x }

each([1,2,3]) |$x| { notice $x }

map

• Transform each element• Returns transformed result

[1,2,3].map |$x| { $x*10 }

=> [10,20,30]

filter

• Produces elements that match• Returns filtered result

[1,2,3].filter|$x| { $x >= 2 }

=> [2,3]

reduce

• Transforms / reduces many to one• Feeds seed/previous result into next iteration • Returns transformed result

[1,2,3].reduce |$result, $x| {$result + $x

}

=> 6

What do you see? Shout out the answer….

Next one….

Are you ready?

Two more…

ok?

The Puppet Type System

Cow

Integer

Puppet Types

• Puppet Types are first order objects (they can be assigned and passed around in variables)

• Uses syntax familiar from Resource – i.e. Class, File, User, where [ ] applied to the type makes it more specific – e.g. File['foo']

Example Integer

# All integersInteger

# All integers >=42Integer[42]

# All integers >=42 and <=142Integer[42,142]

Automatic type checking!

define mytype(Integer[80,443] $port){ # port guaranteed to be integer # and between 80 and 443 # otherwise an error}

Manual type checking!define mytype($port) { assert_type(Integer[80,443], $port) |$expected, $got| { warn("Bad port $got, expected $expected. Using port 80.") 80 } }

• Code block called if given does not match• …do what you want, fail, warn, return default

Operations on Type• Since a Type is a kind of Pattern…– Match with =~ and !~– Match in case expression

• Since Types are defined in a hierarchy:– Compare types with <, <=, >, >=

# is $x an integer ?$x =~ Integer

# is $t more specific than Integer$t = Integer[80, 144]$t < Integer

Type HierarchyAny |- Scalar | |- Numeric | | |- Integer[from, to] | | |- Float[from, to] | | | |- String[from, to] | | |- Enum[*strings] | | |- Pattern[*patterns] | | | |- Boolean | |- Regexp[pattern_string]

Type HierarchyAny |- Collection | |- Array[T] | | |- Tuple[T*, from, to] | | | |- Hash[K, V] | | |- Struct[{ key => T, ...}] | |- Variant[T*] |- Optional[T] | |- Undef |- Default | |- Type[T]

Type HierarchyAny |- CatalogEntry | |- Resource[type_name, title] | |- Class[class_name] | |- Undef |- Data | |- Scalar | |- Array[Data] | |- Hash[Scalar, Data] | |- Undef

EPP

EPP – Templates in Puppet

• Same template markup as ERB – Logic is Puppet instead of Ruby

AND• Can be parameterized !• Use functions

epp(template)inline_epp(string)

• instead oftemplate()inline_template()

Example EPP

$x = 'human'inline_epp('This is not the <%= $x %> you are looking for.', { 'x' => 'droid'})

# => 'This is not the droid you are looking for.'

<%- |$x = 'human'| -%>This is not the <%= $x %> you are looking for.

$x = 'human'inline_epp('This is not the <%= $x %> you are looking for.')

# => 'This is not the human you are looking for.'

Heredoc

Puppet Heredoc

• For more detailed control over a block of text• No, or selected set of escapes• Interpolation or no interpolation• Can be syntax checked by parser (JSon in core,

can add plugin language support)• Control over left margin

Heredoc - Syntax

@( ["]<endtag>["] [:<syntax>] [/<escapes>] )<text>[|][-] <endtag>

ENDS-HEREanything not in <text>

"ENDS-HERE"with interpolation

:jsonsyntax check result

/tsrn$L turns on escape/ turns on all

|set left margin

-trim trailing

t tabs spacer returnn new-line$ $L <end of line>

Puppet Heredoc Example

#.........1.........2.........3.........4.........5....$a = @(END) This is indented 2 spaces in the source, but produces a result flush left with the initial 'T' This line is thus indented 2 spaces. | END

#.........1.........2.........3.........4.........5....foo(@(FIRST), @(SECOND)) This is the text for the first heredoc FIRST This is the text for the second SECOND

Ruby API

Ruby API

• 4x Function API– type checked– dispatch to impl based on given types– more powerful– safer

• Binder– composable type safe injection– for plugins and data (e.g. syntax checkers, data

providers)

Summary

• Language Cleanup• More strict• New Features• Better Error Messages• Iteration• Type System• Puppet Templates – EPP• Heredoc

Enjoy Puppet 4

Links

• github/puppetlabs/puppet-specifications• http://puppet-on-the-edge.blogspot.com/• Twitter @hel• IRC helindbe

Just use PE the next time, ok?