fabricControl your linux nodes with python

What is it?

• Command-line tool for deployment automation and system administration as a ssh wrapper

• Executes to local or remote

✓ local: not a ssh connection. uses python subprocess module.

✓ run or sudo: when executing against remote, it can execute as the current user or as current user with sudo rights.

What is it?

• Simply specify python functions in fabfile.py

• Install it

✓ pip install fabric

✓ Once installed, fab command is available in your system path.

✓ Start writing your python function in fabfile.py

automation

• Daily tasks• Server setup• System update and maintenance• New project creation• Deployment processes

get started

• Download and install virtualbox https://www.virtualbox.org

• Download and install vagrant http://www.vagrantup.com

cd ~/VirtualBox\ VMsmkdir debsqueeze64cd debsqueeze64vagrant init debsqueeze64 http://www.emken.biz/vagrant-boxes/debsqueeze64.boxvagrant up

get started

• This downloads and creates an image of debian 64-bit on your computer.

• We will use fabric on our local machine to interact with this debian virtual machine

• See http://www.vagrantbox.es to download other prepared boxes.

Python Virtualenv

• Create an isolated python project environment

Mac OS X: package management with port or brew

sudo port -v install virtualenv_select py27-virtualenv sudo port -v install py27-virtualenvwrappersudo port select --set virtualenv virtualenv27

(optional)

Python Virtualenv

• Edit your ~/.bash_profile export WORKON_HOME=$HOME/.virtualenvsexport PROJECT_HOME=$HOME/worksource `which virtualenvwrapper.sh`

• $HOME -- this refers to your user home or “~” (i.e., on my computer, “~” means “/Users/calvin” directory)

(optional)

Python Virtualenv

• Create our python virtualenv for our fabric tutorial

mkvirtualenv --distribute --no-site-packages learnfabriccd $PROJECT_HOMEgit clone git://github.com/calvinchengx/learnfabric.gitcd learnfabricpip install -r requirements.txt

(optional)

learn fabric

• Once we have our debian virtual machine and optionally our python virtualenv ready, we begin

git clone git://github.com/calvinchengx/learnfabric.gitcd learnfabricpip install -r requirements.txtcp -rf ~/VirtualBox\ VMs/debsqueeze64/.vagrant .

• This installs fabric and installs python-vagrant (a wrapper around vagrant commands so we can programmatically control our vagrant instance)

learn fabric

• Once we have our debian virtual machine and optionally our python virtualenv ready, we begin

git clone git://github.com/calvinchengx/learnfabric.gitcd learnfabricpip install -r requirements.txtcp -rf ~/VirtualBox\ VMs/debsqueeze64/.vagrant .

• This installs fabric and installs python-vagrant (a wrapper around vagrant commands so we can programmatically control our vagrant instance)

fabric functionfab mytask

[vagrant@127.0.0.1:2222] Executing task 'mytask'[vagrant@127.0.0.1:2222] run: echo $USER[vagrant@127.0.0.1:2222] out: vagrant[vagrant@127.0.0.1:2222] out:

Done.Disconnecting from vagrant@127.0.0.1:2222... done.

the code

import vagrantfrom fabric.api import env, task, runv = vagrant.Vagrant()v.up()env.hosts = [v.user_hostname_port()]env.key_filename = v.keyfile()env.disable_known_hosts = True # useful when vagrant box ip changes.# our first fabric function@taskdef mytask(): run('echo $USER')

fabric core

• local() runs a command locally, via python subprocess

• run() runs a command remotely, via ssh• sudo() runs a command remotely as sudo, via

ssh• put() copies a file from local to remote, via ssh• get() copies a file from remote to local, via ssh

fabric AUth

• All authentication is ssh-based• SSH keys help us avoid typing in passwords -

place fabric ssh user (env.user)’s public key in remote node’s authorized_keys

• Avoid directly using root user• Give your fabric ssh user sudo rights instead

(env.user)

fabric config• from fabric.api import env

• fabric environment is simply a dictionary containing host information, roles, user (env.user); and

• any other custom information you would like to include.

• $HOME/.fabricrc allows custom configuration, e.g.

user = ssh_user_name where “ssh_user_name” is any value you so desire to pass in to env.user when a fab function runs.

Fabric ROLEDEFS# code

from fabric.api import envenv.roledefs = { ‘web’: [‘100.0.0.1’, ‘100.0.0.2’], ‘db’: [‘100.0.0.3’], ‘media’: [‘100.0.0.4’]}

# command line

fab -R web mytask

Fabric host(S)# code

from fabric.api import envenv.roledefs = { ‘web’: [‘100.0.0.1’, ‘100.0.0.2’], ‘db’: [‘100.0.0.3’], ‘media’: [‘100.0.0.4’]}

# command line

fab -H 100.0.0.1,100.0.0.2 mytask

map roles to tasks# codefrom fabric.api import envfrom fabric.decorators import rolesenv.roledefs = { ‘web’: [‘100.0.0.1’, ‘100.0.0.2’], ‘db’: [‘100.0.0.3’], ‘media’: [‘100.0.0.4’]}

@roles(‘web’)def mytask(): run(‘uptime’)

# command line

fab mytask # already mapped to -R web

handling failures• Do not abort upon failure; just give a warning

# codefrom fabric.context_managers import settings

def mytask(): with settings(warn_only=True): run(‘rm /var/www/proj/releases/current’) run(‘ln -s /var/www/proj/releases/deployed

/var/www/proj/releases/current’)

Lazy sysadmin• A generic command

# code

@roles(‘all’) def invoke(command): “””

Invoke an arbitrary command “””

sudo(command)

# command line

fab invoke:“aptitude update” fab invoke:”aptitude upgrade”

parallel execution• A generic command

# code

@roles(‘all’) def invoke(command): “””

Invoke an arbitrary command “””

sudo(command)

# command line

fab -P invoke:“aptitude update” fab -P invoke:”aptitude upgrade”

python web app

Example deploy@task(default=True)@set_target_envdef deploy(email=False): """ `fab -R all deploy` or fab -H mysite.com deploy`. Execute a deployment to the given groups of hosts or host """ if not chk_req(): return if git_branch_check() or test_host_check(): manage_release('Deployment start') git_archive_and_upload_tar() pip_requirements() collectstatic(deploy=True) symlink_current() webserver() migrate_db() # post-deployment tasks manage_release('Deployment end') _releases_cleanup() email_on_success(trigger=email)# command line

fab -R all deploy

quick refErence

• fab -H host1.com task• fab -H host1.com task:arg1,arg2• fab -P -H localhost, host1.com, host2.com task• fab -R web task # roles defined by

env.roledefs• fab -R web task:arg1,arg2• fab -P -R web task• fab -l• more options explained via fab --help

devops

• Bridging the gap between code implementation by developers and deployment to staging and/or live servers by sysadmin

• Automation leads to continuous integration (e.g. Jenkins CI Server) and continuous delivery

• Support heterogeneous development environment in combination with vagrant (developers test against vagrant, which has same distro as staging and production hosts)

referenceS• My reusable fabric functions

https://github.com/fabric-colors/fabric-colorshttps://fabric-colors.readthedocs.org/en/latest/index.html

• Code that accompanies this set of slideshttps://github.com/calvinchengx/learnfabric

• Fabric code and documentationhttps://github.com/fabric/fabrichttp://fabric.readthedocs.org

• Multiple vagrant boxes for testing your server confighttp://awaseroot.wordpress.com/2012/05/06/script-for-adding-multiple-vagrant-boxes/

q&A

• Get in touch @calvinchengx

• http://calvinx.com