Puppet At Twitter - Puppet Camp Silicon Valley
-
Upload
puppet-labs -
Category
Documents
-
view
874 -
download
0
description
Transcript of Puppet At Twitter - Puppet Camp Silicon Valley
![Page 1: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/1.jpg)
Puppet @Twitter
Andrew Hamilton
@ahamilton55
![Page 2: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/2.jpg)
Who am I?
![Page 3: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/3.jpg)
What we’ll cover
![Page 4: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/4.jpg)
The Dev in DevOps
![Page 5: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/5.jpg)
Version Control
![Page 6: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/6.jpg)
Peer review of code
![Page 7: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/7.jpg)
Review Boardhttp://www.reviewboard.org/
![Page 8: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/8.jpg)
Define your style
![Page 9: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/9.jpg)
OWNERS
![Page 10: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/10.jpg)
OWNERS example
Moving from Puppet 2.7 to Puppet 3.x
Need to go from:
to:source => ‘puppet:///<module>/<filename>’
source => ‘puppet:///modules/<module>/<filename>’
![Page 11: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/11.jpg)
But of course…
![Page 12: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/12.jpg)
You can automate all of this!
![Page 13: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/13.jpg)
Review Board – git review tools
Style Guide – puppet-lint
OWNERS – Script a hook and git review tools
Version control hooks
![Page 14: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/14.jpg)
And always iterate on this
![Page 15: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/15.jpg)
Branching
![Page 16: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/16.jpg)
Head
Our true testing branch
Where development happens
Systems don’t live on this branch
![Page 17: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/17.jpg)
Testing
Works along with our canary system for deployments
Major changes can sit here for a couple days to weeks
![Page 18: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/18.jpg)
Production
99% of our systems are on the production branch
Testing on puppet and head should prevent any error from propagating to production
![Page 19: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/19.jpg)
Workflow
Head
Develop
Review Board
Commit
Test
Testing Production
Cherry pick
Review Board
Test
Commit
Cherry pick
Review Board
Commit
![Page 20: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/20.jpg)
But it could be better
![Page 21: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/21.jpg)
puppetmasterd
![Page 22: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/22.jpg)
Each host runs 3 puppetmasterd processes
Each can serve between ~1k – 2k hosts
We use round-robin DNS
![Page 23: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/23.jpg)
ENC
![Page 24: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/24.jpg)
Audubon
![Page 25: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/25.jpg)
Facts
Mostly deprecated, legacy
Pulled info from Facter
![Page 26: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/26.jpg)
Attributes
Basic host information
Two levels - server, role
Allows for inheritance
![Page 27: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/27.jpg)
Groups
Allows for a logical grouping of hosts
Tells you things like role, rack, cluster, etc
![Page 28: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/28.jpg)
Pulling in info from the ENC is easy
We have functions that pull in each type of change in a manifest
attribute(‘<attr_name>’, ‘<default_value>’)
group(‘<group_name>’, ‘<default_value>’)
![Page 29: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/29.jpg)
Our ENC data store is also what controls the puppet branch our hosts use
We set a group called puppet_branch which is used by our client cronjob
![Page 30: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/30.jpg)
Agents
![Page 31: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/31.jpg)
1. Find branch the host is using
2. Find the port that branch uses on masters
3. Get a master from DNS
1. Health check the master
2. If health continue, else repeat until timeout
4. Run puppet apply against the master
and port
![Page 32: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/32.jpg)
Modules
![Page 33: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/33.jpg)
Team Modules
Base Modules
Service Modules
![Page 34: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/34.jpg)
Create a module for each system service
Chain these together in your service modules
If you do something often, create a module that every group can use
![Page 35: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/35.jpg)
Build for self-service
![Page 36: Puppet At Twitter - Puppet Camp Silicon Valley](https://reader033.fdocuments.net/reader033/viewer/2022052301/5498d396b47959db498b4571/html5/thumbnails/36.jpg)
Looking forward