Plone deployment made easy

Pawel LewickiKim Chee Leong

Goldmund, Wyldebeast & Wunderliebe

{ lewicki, leong } @gw20e.com

Outline

● From manual to automatic deployment● How Puppet and Fabric are used for

deployment● Deployment demo in screencast● Puppet server orchestration● Fabric in detail

From manual to automatic deployment

our path to automatic deployment

● Mark eggs/modules using git tag● Update mr developer sources with tag

● SSH to server● Clone new buildout in releases directory● Bootstrap buildout and execute● Switch previous release to new release● If on a cluster set-up, rinse and repeat

Typical Plone release @GWW

● All previous steps in one Fabric script● Allows ‘dummy-proof’ releases

Run:

fab make_tag

fab deploy

fab switch

And Bob is your uncle!

Plone release @GWW using Fabric

xkcd - automationhttp://xkcd.com/1319/

● Avoid repetitive tasks● Error-proof● Immune to typo’s● Consistent environments● Time efficient

Why automatic deployment?

● Avoid repetitive tasks● Error-proof● Immune to typo’s● Consistent environments● Time efficient

Why automatic deployment?

nothing* should be done manually on the server

How Puppet and Fabricare used for deployment

● Puppet is used for server orchestration● Puppet prepares the servers● When Puppet is done, Fabric is used to

deploy buildouts

Initial server setup with Puppet

Puppet diagram

● Uses tst, acc and prd layers● Knows configuration for buildout● Deploys and runs buildouts● Does several other tasks

Buildout deployment with Fabric

Fabric diagram

Deployment screencastusing Puppet and Fabric

Puppet server orchestration

paving the way for Fabric

Example Puppet config

node 'ploneconf.puppet' {

package { 'apache2': ensure => installed }

appie::app { "ploneconfapp":

envs => {

prd => { uid => 3011 },

acc => { uid => 3012 },

tst => { uid => 3013 },

},

webserver => 'apache2',

accountinfo => $gw20e::user_accounts,

}

class { 'ssh':

server_options => {

'PasswordAuthentication' => 'no',

'PermitRootLogin' => 'no',

},

}

}1 / n

● User accounts for TAP environments:o app-mysite-prd in /opt/APPS/mysite/prdo app-mysite-acc in /opt/APPS/mysite/acco app-mysite-tst in /opt/APPS/mysite/tst

● Webserver config is mapped to each application user$ cat /etc/apache2/sites-enabled/zzz-app-mysite-prdInclude /opt/APPS/mysite/prd/sites-enabled/

● SSH Keys of developers are synced● Buildout cache is enabled

Puppet tasks

Fabric in detailusing gww.buildout and

gww.buildout-fabric

● Automatic deployment● Push buildout config using jinja templates● Extended TAP-layers● Switching between releases

● But also; supports git branches, virtual host for webserver, copying zodb to local buildout.

Plone Fabric features

1. Update current symlinkrm -f ~/current

ln -s ~/releases/20141028 ~/current

2. Start supervisor daemon in new buildout~/releases/20141028/bin/supervisord

3. For each service in supervisor:a. Stop service in old buildout

b. Start service in new buildout

4. Shutdown supervisor in old buildout~/releases/20130101/bin/supervisorctl shutdown

Release switch using Fabric

● Fabric is controlled from the local buildout:

./bin/fab -l

Available commands:

deploy Create new buildout in release dir

switch Switch supervisor to latest buildout

test Test if the connection is working

● Deploying to test environment:

./bin/fab deploy:layer=tst,branch=new-feature

Example deployment config (1/2)

prdfrontend = dict(

hosts=prd['hosts'],

buildout='releases/frontend',

varnish={'port': 48083},

haproxy={'port': 48082, 'instances': prd['instances'], },

webserver='apache2',

sitename='ploneconfapp-prd.puppet',

site_id=prd['site_id'],

)

prdbackend = dict(

hosts=prd['hosts'],

buildout='releases/backend',

zeo=dict(base='/data1/APPS/ploneconfapp/prd', **prd['zeo']),

)

Example Deployment config (2/2)

prd = dict(

hosts=['app-ploneconfapp-prd@ploneconf.puppet',],

buildout=_datestamped('releases/%Y-%m-%d'),

current_link='current',

auto_switch=False,

modules=_modules,

third_party_modules=_third_party_modules,

zeo={

'ip': ‘ploneconf.puppet’, 'port': 48081,

},

instances={

'ports': {'instance0': 8080, 'instance1': 8081},

'ipaddresses': _servers,

},

site_id=_site_id,

credentials={'username': 'admin', 'password': 'secret', },

remote_configs={'clockusers': 'clockuser.cfg'},

sentry={

'dsn': 'https://usr:passwd@sentry.gw20e.com/29',

'level': 'ERROR',

},

)

Example jinja template / buildout

[supervisor]

programs +=

...

{% if varnish %}

60 varnish ${buildout:directory}/bin/varnish

true

{% endif %}

{% if varnish %}

[varnish]

port = {{ varnish.port }}

{% endif %}

Example jinja template / apache

{% if varnish %}

{% set port = varnish.port %}

{% elif haproxy %}

{% set port = haproxy.port %}

{% else %}

{% set port = instances.ports.instance0 %}

{% endif %}

<VirtualHost *:80>

ServerName {{ sitename }}

RewriteEngine on

ProxyPass / http://localhost:{{ port

}}/VirtualHostBase/http/{{ sitename }}:80/

ProxyPassReverse / http://localhost:{{ port }}/../http/{{ sitename }}:80/

</VirtualHost>

● https://github.com/search?q=ploneconf2014_gww

● Puppet module: o http://git.io/2NLMtg (shortened url)o https://github.com/Goldmund-Wyldebeast-Wunderliebe/puppet-ap

pie

● Fabric module:o http://git.io/kVdsnw (shortened url)o https://github.com/Goldmund-Wyldebeast-Wunderliebe/gww.buildo

ut-fabric

● GWW Plone buildouto http://git.io/PzdjqQ (shortened url)o https://github.com/Goldmund-Wyldebeast-Wunderliebe/gww.buildo

ut

Github repositories

Thank you!

Goldmund, Wyldebeast & Wunderliebe

{ lewicki, leong } @gw20e.com