Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired...
Transcript of Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired...
![Page 1: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/1.jpg)
Deploying Complex Stacks with AnsibleDevconf 2020 - 2020-01-24
Will Foster • @sadsfae github.com/sadsfae • https://hobo.house
![Page 2: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/2.jpg)
● Client-less configuration management system● Written in Python● Uses SSH as a transport mechanism● Uses YAML for logic and tasks● Uses Jinja2 for templating
What is Ansible?
![Page 3: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/3.jpg)
● Save time and resources so you can do other things● Significantly lower deployment time for apps / services● Reduce complexity and human error via automation
Why should I use Ansible?
![Page 4: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/4.jpg)
● Strive for idempotency. ○ Ansible should do nothing if in desired state
● Template as many configuration files as possible● Break deployment pieces/objectives into logical parts● Make liberal use of configuration variables● Aim for an open-ended design and choice● Use Ansible provided modules wherever possible
Configuration Management Goals
![Page 5: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/5.jpg)
How Baby Yoda Writes Ansible
![Page 6: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/6.jpg)
● Deploy a full all-in-one ELK/EFK 6.8.x stack○ Elasticsearch (search engine, time-series datastore)○ Logstash (data collection, log parsing engine)
○ Kibana (analytics, visualization)
○ Nginx (web reverse proxy)
● We’ll use CentOS7
● Code here: github.com/sadsfae/ansible-elk
Complex Stack Example: ELK/EFK
![Page 7: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/7.jpg)
● host-02 (client) →○ Send system logs via Filebeat to Logstash on host-01
● host-01 (server) → ○ Logstash accepts system logs over SSL/TLS○ Logstash filters logs and sends to Elasticsearch○ We visualize events in Kibana Web UI
DEMO: ELK/EFK Deployed via Ansible
![Page 8: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/8.jpg)
You’ve had your snack, you’ve played with the buttonNow it’s time to put your jammies on.
![Page 9: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/9.jpg)
![Page 10: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/10.jpg)
Ansible Facts can Customize your Environment
![Page 11: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/11.jpg)
● Example: automatically tune Elasticsearch JVM heapsize based on amount of physical memory detected
Use System Facts to Adjust Configuration
![Page 12: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/12.jpg)
Handling Service Dependencies in Ansible
![Page 13: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/13.jpg)
● When components depend on other components, check they are available before proceeding
● Can be done by HTTP return code, port, or web content
Make use of Service Dependency Checking
![Page 14: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/14.jpg)
● wait_for, until and uri Ansible modules are useful for this
Make use of Service Dependency Checking
![Page 15: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/15.jpg)
● Parent service availability checks during a playbook run
Make use of Service Dependency Checking
![Page 16: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/16.jpg)
● Another example: checking raw output matches what we want before proceeding
Dependency Checking Example - Dell Racadm
![Page 17: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/17.jpg)
Using Variables for Conditional Logic
![Page 18: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/18.jpg)
● Complex stacks will inevitably grow to require more deploy options● Maximize the usage of conditional vars to provide choice● Expand options for deployment flexibility (see all.yml)
../install/group_vars/all.yml → → → →
vars_files Make your Playbook more Flexible
![Page 19: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/19.jpg)
Organizing your Playbook and role Hierarchy
![Page 20: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/20.jpg)
Common Design Hierarchy for Large Playbooks
![Page 21: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/21.jpg)
Try to use one role per major component
![Page 22: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/22.jpg)
● Use branches to support older versions/series of the stack● Use branches to support deployment to different environments● Gitlab.com offers free, private repositories each with 10Gb of space
Keep your Playbooks in an SCM (Git, etc).
![Page 23: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/23.jpg)
Automate Client Operations When Possible
![Page 24: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/24.jpg)
● Make it easy to automate client integration● e.g. SSL/TLS certificate retrieval, client applications / libraries
Make sure you’re automating client-side
![Page 25: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/25.jpg)
CI / CD and Ansible Lint
![Page 26: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/26.jpg)
● Use ansible-lint to test your playbooks
ansible-lint install/*.yml -v
ansible-lint install/roles/*/*/*.yml
![Page 28: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/28.jpg)
Troubleshooting and Debugging Tips
![Page 29: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/29.jpg)
● Useful to determine registers, variable output and stdout
Using Debug in a Playbook
![Page 30: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/30.jpg)
● Debug can be useful for informational messages
Using Debug in a Playbook
Running ansible-playbook --check tells you what it would do
![Page 31: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/31.jpg)
● Upgrade ELK Stack to 7.x+ (currently at 6.8.x)
● Support multi-node deployments
ansible-elk roadmap
![Page 32: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/32.jpg)
We use ELK Stack for recording QUADS data
![Page 33: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/33.jpg)
● Manages bare-metal systems and network switch automation based on schedules set in the future
● Powers automation within the Red Hat Scale Lab
● https://quads.dev● github.com/redhat-performance/quads
QUADS is an automation framework
![Page 34: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/34.jpg)
● github.com/sadsfae
● https://hobo.house
● Twitter: @sadsfae
● Freenode IRC: sadsfae
Thank you for attending!
Questions, Comments, Discussion?
![Page 35: Deploying Complex Stacks with Ansible · 1/20/2020 · Ansible should do nothing if in desired state Template as many configuration files as possible Break deployment pieces/objectives](https://reader033.fdocuments.net/reader033/viewer/2022051410/6030b23d58464312af4bfe23/html5/thumbnails/35.jpg)