Puppet getting started by Dirk Götz
-
Upload
netways -
Category
Technology
-
view
1.134 -
download
4
description
Transcript of Puppet getting started by Dirk Götz
www.netways.de // blog.netways.de // @netways
Make IT do more with less
27 NOVEMBER 2013 | PUPPET CAMP
PUPPET GETTING STARTED
DIRK GÖTZ | NETWAYS GMBH
www.netways.de // blog.netways.de // @netways
Make IT do more with less
AGENDA
■ Brief introduction
■ Configuration management
■ Components
■ Design your environment
■ Design your workflow
■ Design your module
www.netways.de // blog.netways.de // @netways
Make IT do more with less
BRIEF INTRODUCTION
www.netways.de // blog.netways.de // @netways
Make IT do more with less
BRIEF INTRODUCTION TO NETWAYS
• Founded in 1995
• Open source since 1997
• 40 employees
• Specialised in open source systems
management and open source data
center infrastructure
www.netways.de // blog.netways.de // @netways
Make IT do more with less
Puppet Camp 2014
• 11 April 2014
• 110 attendees (April 2013)
• 170 attendees (November 2014)
Open Source Data Center Conference
• 8 – 10 April 2014
• 120 attendees (2013)
• 2 tracks of presentations & workshops
NETWAYS CONFERENCES
www.netways.de // blog.netways.de // @netways
Make IT do more with less
CONFIGURATION MANAGEMENT
www.netways.de // blog.netways.de // @netways
Make IT do more with less
Golden images
Software solutions
CONFIGURATION MANAGEMENT
Manual configuration
Self-made scripts
for i in $(cat host.cfg)dossh user@$i uname -a
done
Types of „Automation“
www.netways.de // blog.netways.de // @netways
Make IT do more with less
THE PUPPET WAY
1. Define
2. Simulate
3. Enforce
4. Report
Desired StateCurrent State
www.netways.de // blog.netways.de // @netways
Make IT do more with less
THE PUPPET WAY - DEFINE
■ Using Puppet Domain Specific Language (DSL)
■ Describe relationship between resources
■ Create modular reuseable code
www.netways.de // blog.netways.de // @netways
Make IT do more with less
THE PUPPET WAY - SIMULATE
■ Simulate deployment
■ Without changes to your system
■ Test and fix it before enforcing
$ sudo puppet apply ntp.pp --verbose --noopInfo: Loading facts in /var/lib/puppet/lib/facter/root_home.rbInfo: Loading facts in /var/lib/puppet/lib/facter/pe_version.rbInfo: Loading facts in /var/lib/puppet/lib/facter/vmware.rbInfo: Loading facts in /var/lib/puppet/lib/facter/last_run.rbInfo: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rbInfo: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rbInfo: Loading facts in /var/lib/puppet/lib/facter/puppi_projects.rbInfo: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rbError: Could not find template 'ntp/ntp.conf.erb' at /tmp/ntp.pp:9 on node puppet.localdomainError: Could not find template 'ntp/ntp.conf.erb' at /tmp/ntp.pp:9 on node puppet.localdomain
www.netways.de // blog.netways.de // @netways
Make IT do more with less
THE PUPPET WAY - ENFORCE
■ Compares current state and desired state
■ Changes to desired state
■ Idempotency
$ sudo puppet agent -tInfo: Retrieving pluginInfo: Loading facts in /var/lib/puppet/lib/facter/root_home.rbInfo: Loading facts in /var/lib/puppet/lib/facter/pe_version.rbInfo: Loading facts in /var/lib/puppet/lib/facter/vmware.rbInfo: Loading facts in /var/lib/puppet/lib/facter/last_run.rbInfo: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rbInfo: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rbInfo: Loading facts in /var/lib/puppet/lib/facter/puppi_projects.rbInfo: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rbInfo: Caching catalog for puppet.localdomainInfo: Applying configuration version '1384768191‚Notice: Finished catalog run in 0.54 seconds
Desired StateCurrent State
www.netways.de // blog.netways.de // @netways
Make IT do more with less
THE PUPPET WAY - REPORT
■ Report changes and metrics
■ Many different presentation formats
www.netways.de // blog.netways.de // @netways
Make IT do more with less
COMPONENTS – BASIC TOOLS
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET WORKFLOW
SSL-Encryption
www.netways.de // blog.netways.de // @netways
Make IT do more with less
FACTER
■ Open source Ruby library by Puppet Labs
■ Provides information on your host
■ CLI returns key-value pairs
$ facterarchitecture => x86_64augeasversion => 0.9.0blockdevice_sr0_model => QEMU DVD-ROMblockdevice_sr0_size => 1073741312blockdevice_sr0_vendor => QEMUblockdevices => sr0,vdadomain => localdomainfacterversion => 1.7.3filesystems => ext4,iso9660fqdn => puppet.localdomain
$ facter osfamilyRedHat
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET
■ Open source Ruby library by Puppet Labs
■ Client server based
• REST-API• X509 certificates
■ Platform independent
• Puppet Domain Specific Language• Providers for different platforms
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET DOMAIN SPECIFIC LANGUAGE
■ Abstracts resources
■ Resource types
• user, group, • file, package, service, • exec, • …and many more• possible to create your own
■ Type and title/name pairs must be unique
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET PROVIDERS
■ Resource Abstraction Layer
• Resource types• Providers
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET – MORE ABSTRACTION
■ Class groups resources in one manifest
■ Syntax constructs possible
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET – SYNTAX CONSTRUCTS
■ Variables (but are more like constants)
■ Logical structures
• selectors• case statements• if/elseif/else statements
■ Dependencies
• require/before• subscribe/notify
■ Inheritance
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET – MORE ABSTRACTION
■ Module groups classes and corresponding files
modulename|-- files – static files|-- lib – custom facts, functions, resources|-- manifests – manifests containing classes|-- spec – tests for rspec-puppet|-- templates – dynamic files|-- tests – examples declaring classes
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET FILESERVER
■ File transfer from master
• puppet:///modules/modulename/filename
■ “Magic“ mount point for modules
■ Additional mount points possible
■ Recursion for directories possible
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET PLUGIN SYNC
■ Option on the agent
■ Syncs from all modules
• custom facts• custom resources (types and providers)
■ Before the actual Puppet run occurs
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET TEMPLATES
■ Function executed on server
■ ERB (Plain text with embedded Ruby)
• variables (including facts)• conditionals• iterations• access to tags and classes• usage of Puppet functions
■ Combination of multiple templates possible
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET NODE DECLARATION
■ Connects system (node) with functionality (classes)
• Single hosts
• Mulitiple hosts (by regular expression)
• Default
• Inheritance possible
www.netways.de // blog.netways.de // @netways
Make IT do more with less
COMPONENTS – BASIC GUI
www.netways.de // blog.netways.de // @netways
Make IT do more with less
BASIC GUI – ADDED FUNCTIONALITY
■ Reporting target
• Collects reports• Graphical presentation
■ External Node Classifier (ENC)
• Graphical node declaration• Adds groups
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET - REPORTING
■ Enable on agent
■ Choose target on server
• http/https• log• tagmail• store• rrdgraph• puppetdb
■ Multiple targets possible
■ Create your own
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET - ENC
■ External source for node declaration
• Script returning yaml• LDAP
■ Merged with internal node declaration
$/etc/puppet/node.rb puppet.localdomain---environment: productionparameters: foreman_env: productionowner_name: Admin Userdomainname: ""owner_email: root@localdomainroot_pw: $1$default$hCkak1kaJPQILNmYbUXhD0puppetmaster: ""
classes: profiles::default:
dn: cn=testserver,ou=Hosts,dc=madstop,dc=comobjectClass: deviceobjectClass: ipHostobjectClass: puppetClientobjectClass: topcn: testserverenvironment: testingipHostNumber: 192.168.0.50description: My test serverl: dc1puppetClass: testingpuppetVar: owner_name=„Admin user“
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET DASHBOARD
■ Open source ruby web interface
• Version <= 1.2 by Puppet Labs• Version >= 2 by community
http://puppetlabs.com/presentations/story-dashboard-20
www.netways.de // blog.netways.de // @netways
Make IT do more with less
COMPONENTS – ORCHESTRATION
www.netways.de // blog.netways.de // @netways
Make IT do more with less
ORCHESTRATION
■ Centralized execution on multiple systems
■ Different solutions
• ssh loops• func• fabric• capistrano• mcollective• … and many more
www.netways.de // blog.netways.de // @netways
Make IT do more with less
MCOLLECTIVE
■ Open source Ruby framework by Puppet Labs
■ Facter for grouping
■ Middleware for communication
$ mco package status httpd -F kernel=Linux
* [ ============================================================> ] 3 / 3
centos63a version = httpd-2.2.15-15.el6.centos.1ubuntu1204a version = httpd-purgedmaster version = httpd-purged
---- package agent summary ----Nodes: 3 / 3
Versions: 1 * 2.2.15-15.el6.centos.1, 2 * purgedElapsed Time: 0.12 s
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET ENTERPRISE CONSOLE
■ Ruby web interface based on dashboard
• Auditing• Live management
www.netways.de // blog.netways.de // @netways
Make IT do more with less
COMPONENTS – DEPLOYMENT
www.netways.de // blog.netways.de // @netways
Make IT do more with less
DEPLOYMENT
■ Config management needs running agent
■ (Agent needs running operating system)
■ Different solutions:
• Kickstart / Autoyast / Preseed / Jumpstart• Cobbler / kickstand• Razor• Puppet Cloud Provisioner• Foreman• … and many more
www.netways.de // blog.netways.de // @netways
Make IT do more with less
RAZOR
■ Open source Ruby library by Puppet Labs/EMC
■ Client
• micro-kernel for PXE boot
■ Server
• REST-API• CLI
■ Policy connects
• host profiles• operating system• config management
{"name": "centos-for-small","repo": { "name": "centos-6.4" },"installer": { "name": "centos" },"broker": { "name": "noop" },"enabled": true,"hostname": "host${id}.example.com","root_password": "secret","max_count": "20","rule_number": "100","tags": [{ "name": "small", "rule": ["<=", ["num", ["fact",
"processorcount"]], 2]}]}
www.netways.de // blog.netways.de // @netways
Make IT do more with less
CLOUD PROVISIONER
■ Open source Ruby library by Puppet Labs
■ Instantiates cloud instances
• EC2• VMware• OpenStack• Google Compute Engine
■ Installs Puppet using SSH
www.netways.de // blog.netways.de // @netways
Make IT do more with less
FOREMAN
■ Open source Ruby web interface by Ohad Levy
(Red Hat)
• Reporting target• ENC• Provisioning
www.netways.de // blog.netways.de // @netways
Make IT do more with less
FOREMAN – SMART PROXIES
■ Connects GUI and Backends
• puppet• puppetca• tftp• dhcp• dns
www.netways.de // blog.netways.de // @netways
Make IT do more with less
FOREMAN – COMPUTE RESOURCES
■ Integrates virtualization and cloud platforms
• Libvirt• oVirt / RHEV• VMware• EC2• Google Compute Engine• OpenStack• Rackspace
www.netways.de // blog.netways.de // @netways
Make IT do more with less
COMPONENTS – MORE FEATURES
www.netways.de // blog.netways.de // @netways
Make IT do more with less
EXPORTED RESOURCES
■ One node creates resource
■ Another node realizes that resource
■ Resource needs to be stored:
• Stored configs (deprecated)• PuppetDB
■ Use cases:
• host entries• sshkey management• monitoring / backup• other centralized services
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPETDB
■ PostgreSQL and Java based data warehouse solution
by Puppet Labs
■ Collects
• Facts• Catalogs• Reports (optional)
■ Used for
• Inventory service• Exported resources
www.netways.de // blog.netways.de // @netways
Make IT do more with less
HIERA
■ Open source Ruby library by Puppet Labs
■ Hierarchical data look-up separates code and data
■ Integrated in Puppet >3 / addon for Puppet <3
$cat hiera.yaml---:backends:- yaml
:yaml::datadir: /etc/puppet/hieradata
:hierarchy:- `hosts/${::fqdn}´- `location/${::location}´- common
$cat hosts/specialhost.localdomainntp::server = 192.168.23.23
$ cat location/rz2ntp::server = 192.168.2.23yum:mirror = 192.168.2.42
$ cat commonntp::server = 192.168.0.23yum::mirror = 192.168.0.42proxy::server = [`192.168.0.237´,`192.168.0.238´]
www.netways.de // blog.netways.de // @netways
Make IT do more with less
COMPONENTS – SOFTWARE MANAGEMENT
www.netways.de // blog.netways.de // @netways
Make IT do more with less
SOFTWARE MANAGEMENT
■ Provider needs defined sources for software
■ Local mirror reduces traffic
■ Many different tools
• rsync / createrepo• updian• Spacewalk / Red Hat Satellite / Suse Manager• pulp / katello• … and many more
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PULP
■ Open source Python repository management
by Red Hat
■ Server:
• Import and upload of content• rpm and puppet modules• Publish web-based or ISO images
■ Client:
• Server-side management and reporting
www.netways.de // blog.netways.de // @netways
Make IT do more with less
KATELLO
■ Java web interface by Red Hat
■ Combines:
• candlepin (subscription management)• pulp (software management)• Foreman + Puppet (configuration management)
www.netways.de // blog.netways.de // @netways
Make IT do more with less
DESIGN YOUR ENVIRONMENT
www.netways.de // blog.netways.de // @netways
Make IT do more with less
CHOICES
■ Support / packages needed?
■ Best method to report?
■ Resources to export?
■ Deployment needed?
■ Software management needed?
■ Node declaration or ENC?
www.netways.de // blog.netways.de // @netways
Make IT do more with less
SCALE UP
■ Puppet easily scales up
■ Start simple, grow with your environment
■ Rule of thumb:
number of nodes * catalog compile time (in seconds)number of masters = ------------------------------------------------------------------------------
cores per master * run interval (in seconds)
www.netways.de // blog.netways.de // @netways
Make IT do more with less
SCALE UP
■ Only one certficate authority
• Only needed for registration
■ Load balance Puppet traffic
■ GUI only required for users
• But ENC / Reporting always required
■ Orchestration
www.netways.de // blog.netways.de // @netways
Make IT do more with less
SCALE UP – EXAMPLE: PUPPET ENTERPRISE
www.netways.de // blog.netways.de // @netways
Make IT do more with less
DESIGN YOUR WORKFLOW
www.netways.de // blog.netways.de // @netways
Make IT do more with less
EDITOR
■ vim – text editor
• vim-puppet – synthax highlight• tabular – style guide conformity• puppet-lint – style guide conformity• syntastic – validation
■ Geppetto – eclipse based IDE
synthax hightlight, style guide conformity, module creation
www.netways.de // blog.netways.de // @netways
Make IT do more with less
STYLEGUIDE
■ Official guidehttp://docs.puppetlabs.com/guides/style_guide.html
• readability• work with and without features• simple and robust• shareable / useable by others
■ Create your own
• based on official guide• make your own rules
www.netways.de // blog.netways.de // @netways
Make IT do more with less
TESTING
■ puppet parser validate – syntax
■ puppet-lint – style guide conformity
■ puppet apply --noop – simulate
■ vagrant – simulate
■ rspec-puppet – expected results
www.netways.de // blog.netways.de // @netways
Make IT do more with less
VERSION CONTROL SYSTEM
■ Pre-Commit / Pre-Receive
• use for validation and review
■ Post-Commit
• checkout in environment
■ Versioning for change management
■ Solutions:
• Version control: svn, git, bazaar• Validation & Review: gerrit, jenkins/hudson
www.netways.de // blog.netways.de // @netways
Make IT do more with less
STAGING
■ Puppet environments
• Different versions of modules
■ Use your own versioning
• config_version: script returns version string
www.netways.de // blog.netways.de // @netways
Make IT do more with less
DOCUMENTATION
■ Inline
• RDoc markup• Console output• HTML generation
■ README
• Markdown
■ Modulefile
• used by Puppet module tool
www.netways.de // blog.netways.de // @netways
Make IT do more with less
DESIGN YOUR MODULE
www.netways.de // blog.netways.de // @netways
Make IT do more with less
CONSIDERATIONS
■ Where to start?
■ Use cases?
• different platforms• different roles• small adjustments• one-time or multiple objects
■ ‚Part of it‘ or separate?
• Dependencies
■ Templates or files?
■ Write your own?
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET FORGE
■ Community platform for modules
• Web platform• Command line tool
■ Module information:
• Author• Project homepage and issue tracker• Tags• Releases and download count• Test results
www.netways.de // blog.netways.de // @netways
Make IT do more with less
PUPPET FORGE – EXAMPLE
example42
■ 96 modules on forge / more on github
■ always same layout
■ supports: Red Hat, CentOS, Fedora, Ubuntu, Debian,
Mint, SLES, OpenSuSE, FreeBSD
■ adds monitoring, firewall and puppi (own script library)
www.netways.de // blog.netways.de // @netways
Make IT do more with less
MODULE DESIGN – MORE ABSTRACTION
■ Classes abstract resources
■ Modules abstract classes
■ Nodes contain the logic
Not maintainable!
node 'basil.puppetlabs.vm' {class { 'apache’:version => 'latest’,
}class { 'motd': }class { 'ssh’: }if $::operatingsystem == 'solaris' {class { 'users’:default_shell => '/bin/false’,
}}else {class { 'users': }
}Class['ssh’] -> Class['users’]
}
www.netways.de // blog.netways.de // @netways
Make IT do more with less
MODULE DESIGN – MORE ABSTRACTION
■ Classes abstract resources
■ Modules abstract classes
■ Profiles contain the logic
• abstract the modules• separate implementation
from technology
class profiles::application {include tomcatinclude mysqlinclude componenta
}class profiles::application::x inheritsprofiles::application {include componentbcomponentb::resource { 'name':ensure => present,
}}class profiles::application::y inheritsprofiles::application {include componentcinclude componentd
}class profiles::application::z inheritsprofiles::application {include componentbinclude componentdinclude dependencyClass['dependency'] -> Class['componentd']
}
www.netways.de // blog.netways.de // @netways
Make IT do more with less
MODULE DESIGN – MORE ABSTRACTION
■ Classes abstract resources
■ Modules abstract classes
■ Profiles contain the logic
■ Roles contain business logic
• no logic, just profiles• separate business role from implementation
■ Node has exactly one role
• No Puppet know-how neededfor node declaration
class role::webapp {include profiles::baseinclude profiles::customappinclude profiles::test_tools
}
node ‘web1.example.com’ {include role:webapp
}
www.netways.de // blog.netways.de // @netways
Make IT do more with less
QUESTIONS & ANSWERS
www.netways.de // blog.netways.de // @netways
Make IT do more with less
NETWAYS GmbH
Deutschherrnstrasse 15-19
90429 Nürmberg
Phone: +49 911 92885-0
Fax: +49 911 92885-77
Email: [email protected]
Website: www.netways.de
Twitter: twitter.com/netways
Facebook: facebook.com/netways
Blog: blog.netways.de
QUESTIONS & ANSWERS