Post on 21-Jul-2015
OpenStack Vancouver Summit
Learn you some Ansible for Great Good!
Juergen Brendel (@brendelconsult) , David Lapsley (@devlaps)
May 21, 2015
Unified test and deployment environments
Dev, Test, Deploy
Dev, Test, Deploy
Why didn't we
catch this bug
in testing?
Dev, Test, Deploy
Why didn't we
catch this bug
in testing?
We can't
reproduce the
issues.
Dev, Test, Deploy
Why didn't we
catch this bug
in testing?
We can't
reproduce the
issues.
I don't have
access to our
test server.
Dev, Test, Deploy
Why didn't we
catch this bug
in testing?
How do I setup
my
development
environment?
We can't
reproduce the
issues.
I don't have
access to our
test server.
Dev, Test, Deploy
Why didn't we
catch this bug
in testing?
How do I setup
my
development
environment?
We can't
reproduce the
issues.
I don't have
access to our
test server.
“It works for me...”
(shrug)
Dev, Test, Deploy
Why didn't we
catch this bug
in testing?
How do I setup
my
development
environment?
We can't
reproduce the
issues.
I don't have
access to our
test server.
“It works for me...”
(shrug)Wouldn't this be nice instead?
Single command: Dev environment created
Single command: Test environment created
Summary
• Configuration management background
• Ansible introduction
• Rise of APIs
• Unified test and deployment environments
• Demonstration
Configuration Management Tools
Overview
Configuring servers
How do you configure a server?
Arcane
magic
Configuring servers
How do you configure a server?
Arcane
magic
Configuring servers
How do you configure a server?
Manual
instructions
Arcane
magic
Configuring servers
How do you configure a server?
Manual
instructions
Scripts
Arcane
magic
Configuring servers
How do you configure a server?
Manual
instructions
Scripts
CM tools
Arcane
magic
Configuring servers
How do you configure a server?
Manual
instructions
Scripts
CM tools
Automation!
CM Tools
Describe the desired state
Ensure all
system packages
are updated.
CM Tools
Describe the desired state
Ensure all
system packages
are updated.
CM Tools
Describe the desired state
Ensure package
“apache” is
installed.
Ensure all
system packages
are updated.
CM Tools
Describe the desired state
Ensure that user
“xyz” exists.
Ensure package
“apache” is
installed.
Ensure all
system packages
are updated.
CM Tools
Describe the desired state
Ensure that user
“xyz” exists.
Ensure package
“apache” is
installed.Have latest
sources: Clone
repo, update if it
exists already.
Ensure all
system packages
are updated.
CM Tools
Describe the desired state
Ensure that user
“xyz” exists.
Ensure package
“apache” is
installed.Have latest
sources: Clone
repo, update if it
exists already.
Ensure package
“postgres” v9.1 is
installed.
Ensure all
system packages
are updated.
CM Tools
Describe the desired state
Ensure that user
“xyz” exists.
Ensure package
“apache” is
installed.Have latest
sources: Clone
repo, update if it
exists already.
Ensure package
“postgres” v9.1 is
installed. Ensure DB
“app_data” exists
with password
“****”.
Configuration Management Tools
Varieties
Puppet (2005)“powerful, feature-rich, enterprise-y”
Chef (2009)
CM Tools variety
Puppet (2005)“powerful, feature-rich, enterprise-y”
Chef (2009)
Salt (2011)“simple, fast, good for most things”
Ansible (2012)
CM Tools variety
Puppet (2005)“powerful, feature-rich, enterprise-y”
Chef (2009)
Salt (2011)“simple, fast, good for most things”
Ansible (2012)
Fabric (2011)“not really CMS tools”
Scripts
CM Tools variety
Ansible Overview
• “Orchestration engine” for CM and deployment
• Written in Python
• Uses YAML
• “Playbooks”
• Config specs or explicit commands
Ansible overview
• Key Points:• No central configuration server
• No key management
• No agent to install on target machine
• Explicit order
Ansible simplicity
• Key Points:• No central configuration server
• No key management
• No agent to install on target machine
• Explicit order
• Requirements:• Need SSH access (with key or password)
• Need Python installed on target machine
Ansible simplicity
Ansible architecture
Server 1
Server 2Server 3
Server 4
Server 5
Your laptop
Ansible Overview
Modules
Modules
Hundreds of them. They know how to do stuff…
Command
Shell
Script
Modules
Hundreds of them. They know how to do stuff…
Command
Shell
Script
Copy
Sync
Templates
Line ops
Modules
Hundreds of them. They know how to do stuff…
Command
Shell
Script
Copy
Sync
Templates
Line ops
Install packages
Users and groups
Networking
Services
Modules
Hundreds of them. They know how to do stuff…
Command
Shell
Script
Copy
Sync
Templates
Line ops
Install packages
Users and groups
Networking
ServicesRepositories
Message queues
Monitoring
Notification
Modules
Hundreds of them. They know how to do stuff…
Command
Shell
Script
Copy
Sync
Templates
Line ops
Install packages
Users and groups
Networking
ServicesRepositories
Message queues
Monitoring
Notification
Web servers
Database servers
Cloud infra
Ansible Overview
How does it
work?
How does it work?
Laptop
How does it work?
Python Module
“Install Apache”
Laptop
run playbook
How does it work?
Python Module
“Install Apache”
Run ModuleLaptop
ssh
How does it work?
Python Module
“Install Apache”
Run Module
(then delete)
Run ModuleLaptop
ssh
How does it work?
Python Module
“Install Apache”
Run Module
(then delete)
Run ModuleLaptop
return results
Ansible Overview
Details
Inventory and groups
Define hosts, organized in groups
Inventory and groups
Define hosts, organized in groups
by function
by location
by hosting provider
...
[europe]
server1.somehoster.co.uk
server2.otherhoster.de
[north-america]
host-a.serverhost.com
host-b.serverhost.com
[frontend]
server1.somehoster.co.uk
host-b.serverhost.com
[backend]
server2.otherhoster.de
host-a.serverhost.com
Adhoc commands
Single commands, applied to groups
Adhoc commands
Single commands, applied to groups
$ ansible -i hosts europe –a “uname -a”
$ ansible -i hosts frontend -a “/sbin/reboot” -f 3
• Tell Ansible what to do
Playbooks
---
- hosts: frontend
sudo: yes
tasks:
- name: Update the system
apt: pkg=nginx state=latest
- name: Create the user account
user: name=appuser shell=/bin/bash state=present
- name: Copy files to remote user's home
copy: >
src=files/names.txt dst=/home/appuser
owner=appuser mode=0644
• Provide input to Ansible templates
Variables
---
- hosts: all
sudo: yes
vars:
username: appuser
tasks:
- name: Create the user account
user: >
name={{ username }}
shell=/bin/bash
state=present
• Simple layout for arranging Ansible playbooks, variables, templates, metadata, etc.
Simple Project Layout
/
my_hosts
group_vars/
all
frontend
backend
europe
north-america
site.yml
• Best practices layout for arranging Ansible playbooks, variables, templates, metadata, etc.
• Better suited for larger projects
• More extensible
Best Practice Project Layout
/
ansible.cfg
deploy_hosts
staging_hosts
group_vars/
all
frontend
backend
europe
north-america
host_vars/
server1.somehoster.co.uk
host-b.serverhost.com
site.yml
roles/
common/
tasks/
main.yml
handlers/
main.yml
templates/
sshd_config.j2
files/
my_script.sh
vars/
main.yml
web/
...
db/
...
Playbooks with roles
---
- hosts: frontend
sudo: yes
roles:
- common
- web
The rise and rise of APIs
The rise and rise of APIs
APIs
The rise and rise of APIs
APIs
Local
The rise and rise of APIs
APIs
Local Infrastructure
The rise and rise of APIs
APIs
Local Infrastructure Services
The rise and rise of APIs
APIs
Local Infrastructure Services
Ansible 'cloud' modules
Public cloud
OpenStack
Amazon AWS
Google Compute
Azure
Digital Ocean
Linode
Private cloud
• OpenStack
• Eucalyptus
• Vsphere
• Docker
• libvirt
Example: AWS Modules
EC2 / infrastructure
• Instances
• Images
• VPCs
• Load balancers
Services
• S3
• Route 53
• Databases
• Cache
• Create instances via AWS and OpenStack
Example: Create instances
- name: Booting EC2 guests
ec2:
key_name: my-key
group: my-security-group
instance_type: t2.micro
image: ami-120abc90
region: us-east-1
count: 1
register: ec2results
- name: Booting OpenStack guests
nova_compute:
state: present
login_username: "{{ openstack_username }}"
login_password: "{{ openstack_password }}"
login_tenant_name: "{{ openstack_tenantname }}"
name: "{{ cluster_id }}-{{ item }}"
image_id: "{{ openstack_image_id }}"
key_name: "{{ openstack_keyname }}"
wait_for: 60
flavor_id: "{{ openstack_flavor_id }}"
nics:
- net-id: "{{ openstack_internal_net_id }}”
register: openstack_guests
• Add hosts to inventory
Example: Create instances
- local_action:
module: ec2
key_name: my-key
group: my-security-group
instance_type: t2.micro
image: ami-120abc90
region: us-east-1
count: 3
register: ec2results
- local_action:
module: add_host
hostname: {{ item.public_ip }}
groupname: my-server-group
with_items: ec2results.instances
Unified test and deployment environments
Vagrant
Use Vagrant to spin-up VMs local (VirtualBox, VMware, etc.)
cloud (EC2)
Use Ansible as 'provisioner'
Make an inventory file with just your VM
Point at same playbook as before
Vagrant
• Tells vagrant which VMs to construct
• How to construct them:• RAM
• Virtual CPUs
• Network interfaces (public, private, static, nat’d)
Vagrant config: Vagrantfile
Vagrant.configure(2) do |config|
config.vm.box = "saucy64"
config.vm.box_url = "http://cloud-
images.ubuntu.com/vagrant/..."
config.vm.host_name = "myapp-test"
config.vm.network "private_network", ip: "192.168.1.2”
config.vm.provision "ansible" do |ansible|
ansible.playbook = "site.yml"
#ansible.verbose = "vvvv"
ansible.inventory_path = "vagrant_hosts"
ansible.host_key_checking= false
end
end
• Specifies which VMs/Groups Ansible should run against
Inventory: Vagrant Hosts
[vagrant]
vagrant_host
ansible_ssh_host=192.168.1.2
[frontend-hosts]
vagrant_host
[applayer-hosts]
vagrant_host
[backend-hosts]
vagrant_host
[db-access:children]
applayer-hosts
backend-hosts
[appserver-access:children]
frontend-hosts
applayer-hosts
Vars: group_vars/vagrant
Variables that only apply to Vagrant instances
---
ansible_ssh_user: vagrant
Create and configure VMs
$ vagrant up
...
$ vagrant provision
Unified test and deployment environments
Cattle, not
pets!
Desired development/deployment workflow
- Local unit tests
- Local provisioning with Vagrant + Ansible
- Integration tests
Local dev
and testing
Desired development/deployment workflow
- Create/update cloud
staging servers with
Ansible
- Provision servers with
Ansible
Local dev
and testing
Cloud
testing
Desired development/deployment workflow
- Create/update cloud production
servers with Ansible
- Provision servers with Ansible
Local dev
and testing
Cloud
testing
Cloud deployment
Demo
Dev Environment
Cacher (apt/pip)
MCP MHV1 MHV2
Ansible
Git cache
• Same Ansible playbooks can be used to provision application locally or in the cloud
Key Takeaways
• Same Ansible playbooks can be used to provision application locally or in the cloud
• With cloud APIs and Ansible modules (OpenStack, AWS, Rackspace, …) playbooks can also be used to provision infrastructure
Key Takeaways
References
• Questions: jbrendel@cisco.com, dlapsley@cisco.com
• Ansible playbooks: http://bit.ly/devstack-ansible
• Ansible docs: http://docs.ansible.com/
• Ansible source: https://github.com/ansible/ansible
• Vagrant: http://www.vagrantup.com/
• Example project: http://bit.ly/ansible-devstack
@brendelconsult, @devlaps
Thank You