Visuel à insérer ici
WEB SUMMER CAMP 2016
Rovinj
eZ Publish 5: from zero to automated deployment
(and no regressions!) in one afternoon
WHO AM IThe short version
• « the guy who has written a lot of answers
on the eZPublish forums »
• Working at Kaliop in London since 2014
• Principal consultant at eZSystems for 7 years
• Love coffee and bicycles ?
• @gggeek
• Little known fact: today it’s my birthday!
VOTRE SCHEMA
TALK IS CHEAPShow me the code
https://github.com/kaliop-uk/websummercamp2016
HOUSE RULESMarek and Jakub are the Lethal Enforcers Minions Gentle Assistants
• Interrupting is welcome at any time
• fall behind: the assistants are authorized
to stop me for all the time needed for you
to catch up with the rest
• fall asleep: the assistants are authorized to
kill you empty your pockets pinch your ear
WHO ARE YOU?The short version, please
• Any experience with eZPublish ?
• Any experience with eZPublish Legacy?
• Did anybody attend the Docker workshop
from Marek on Wednesday ?
• Any Docker experience at all ?
• Who has been up all night partying?
VOTRE SCHEMA
SUMMARY
1. INTRO TO DOCKER
2. (MOAR) DOCKER!
3. MANAGING ENVIRONMENTS
4. MANAGING DB CHANGES
5. A DEPLOYMENT SCRIPT
6. TESTING THE SITE
Getting started is easyGetting off the hook on the other hand…
"just like a VM, but lighter"
all it takes to get started:
docker run -ti ubuntu:15.10 bash
Getting started is easy
what has just happened:
• the Docker daemon connected to a public repository to download an
image (https://hub.docker.com/)
• a container was started using that image
• the bash program was started in the container, and its I/O attached to
the shell in execution
Getting started is easy
• ideal for testing a command / script in a different environment
• makes it very easy to deploy (some types of) applications as self-
contained packages
• not created for multi-process applications, nor for Continuous
Integration scenarios
• but it is getting there quickly
Terminology
• container: a container is a runtime instance of a Docker image
• image: Docker images are the basis of containers (recursion?)
• an image is an ordered collection of root filesystem changes and the
corresponding execution parameters for use within a container runtime
• dockerfile: a Dockerfile is a text document that contains all the
commands you would normally execute manually in order to build a
Docker image
• volume: a data volume is a specially-designated directory within one or
more containers that bypasses the Union File System
• in other words: a volume is a directory on the host which gets mounted into the
container
Sample DockerfileReference at: https://docs.docker.com/engine/reference/builder/
FROM UBUNTU:15.10
RUN apt-get update && apt-get install -y php5 php5-curl php5-gd
ADD src/ /var/www/html/
ADD scripts/start.sh /start.sh
EXPOSE 443 80
CMD ["/start.sh"]
Docker philosophyDefinition according to Me™, again
• one process per container
• containers should be immutable (or close to it)
Useful one-liners
• docker images lists all images present on the host computer
• docker ps lists all running containers
• docker ps -a lists all containers, including stopped ones
• docker rm <cont.> removes a container
• docker rmi <image> removes an image
Where the magic stops
• how to share application data with the container?
• how to access the apps in the container (map container ports to the
host)?
• how to start multiple containers together and connect them?
• how to access the log files from the application?
Then things get tricky
• mapping of user ids to properly handle filesystem permissions
• linking containers when there are circular dependencies (solved with
recent Docker versions)
• applying security updates to existing containers
• and more…
• for some more of the fine details, see f.e. this blog post:
http://blog.kaliop.com/en/blog/2015/05/27/docker-in-real-life-the-tricky-
parts/
Docker Compose to the rescue!
• a single file describes multiple containers
• a single command to start/stop them all together
• permits further automation (via eg. introduction of variables)
Sample docker-compose.ymlReference docs: https://docs.docker.com/compose/compose-file/
version: '2'
services:web:
build: images/webhostname: webports:
- "80:80"volumes:
- ./site/:/var/www/siteenv_file:
- docker-compose.env
cli:build: images/clihostname: clienvironment:
- SSH_AUTH_SOCK=/ssh-agentcap_add:
- SYS_PTRACE
Why use Docker for eZPublish website development?Because you get to go to the Summer Camp, duh!
• easier to share with a distributed dev team than a whole VM
just a bunch of scripts, not multi-MB image
easy to fix/improve and redistribute
• easy to convert to the same version of the OS/Apps used in production
• forces separation of applications to separate environments
closer to a production environment (for non-trivial sites)
good to avoid surprises when deploying
Introducing the stack
• apache
• mysql
• memcached
• solr
• php cli
• varnish
• nginx
plus
• control panels for all of the above
• phpmyadmin
• maildev (an smtp server for development purposes)
• dockerui (a container monitoring the whole container stack [Inception!])
How it worksAll the things someone else has figured out for you
• configuration of the applications is mounted as volume, not copied into
the image
a simple service restart is sufficient to take changes into account
• application source code is kept on the host computer
no risk of losing changes when rebuilding containers
no need to have containers running GUI or IDEs
• all log files accessible from the host, in one place
• use ssh-agent to avoid developers storing their own private keys inside
the containers
How it works, IIAll the things someone else has figured out for you
• allow developers to change some variables, while providing good
defaults
a configuration file is provided, which has all the variables that developers
can customize
a 2nd configuration file has to be created with local changes (empty is fine)
ex: id of the user account on the host computer, or github credentials
the config file with local changes is not committed to git
• not tied to a specific project
can be quickly converted for usage of Drupal instead of eZPublish
two of them can be run in parallel
How it works, IIIAll the things someone else has figured out for you
• the containers have to replicate as much as possible a production
environment
the base images used are Debian or Ubuntu (the platforms we generally
deploy on)
the applications are started using service start and service stop
commands
cronjob tasks definitions are kept as part of the sources of the stack
On Disk layout
• config configuration for services running in containers
• data data persisted by the apps in containers (eg: mysql db)
• images Dockerfiles and files used to build the images
• logs log files from services running in containers
• site the website installation folder
• src source code for tools which are part of the stack
On Disk layout and volume mounts( incomplete example )
• config/…
• data/…
• images X
• logs/…
• site
• src X
Mysql container
/etc/mysql/conf.d/
/var/lib/mysql/
/var/log/mysql/
Web container
/etc/apache2/sites-available
/etc/php5/apache2/
/var/log/apache/
/var/www/site/
Cli container
/tmp/cron.d
/etc/php5/cli
/var/log/php/
/var/www/site/
Where to store per-environment configuration files
• Symfony: excellent native support
• eZPublish legacy: somewhat, hackish support (eg. in extensions?)
• Webserver config etc… : no standard solution
All-in-the-stack solution
• Store eZPublish legacy settings and app. settings in a specific dir
in the source code
• Use symlinks to deploy them
• But not manually!
• kaliop/ezpublish5universalinstaller
• Advantage: only one git repo to take care of
• Advantage: supports an ‘override’ logic
Sharing the Database
Many possibilities:
A. All developers connect to the same db
B. The database is committed to git
C. Managing changes via scripts
D. Live DB replication (really ???)
E. More ?
Connecting to a shared database
• good for teams which are *not* geographically distributed
• works when developers do not blow up the database with junk
content: needs some developer discipline
• works only up to the 1st deployment to UAT env;
then 2 dbs have to be managed anyway
Committing the database to git
• good for when the development database does not contain a
huge number of contents and assets
• good for when the development database does not change too
frequently
• developers might be working on different versions
of the db: needs some developer discipline
• works only up to the 1st deployment to UAT env;
then 2 dbs have to be managed anyway
Managing database changes via script
• Safest option
• Good for when the development database does not change too
frequently
• Perfect after deployment to UAT/PROD
• kaliop/ezmigrationbundle
Automating deployments
• do I even have to explain why it is a good idea ?
• use the same deployment script during development as in
production
so that it gets well tested
• good for automated testing too
eZPublish deployment tasks
1. alter webserver config to disallow access
2. get the latest version of the source code
3. composer install
4. apply changes to database
5. clear Opcache cache (just in case)
6. custom scripts?
7. reindex content (optional)
8. clear shared caches
always: inner to outer
Memcached
Varnish
9. alter webserver config to allow access
eZPublish deployment tasks
extras:
1. run security checks (Symfony has a built-in script)
2. notify monitoring systems of deployment (eg: NewRelic)
You do write tests for CMS-
powered websites, don’t you?
A. HAHAHA!
B. Are you joking?
C.No, really?
06
Going against the grainHow much non-project-specific code have you used on this project ?
UI
Unit
???
Going against the grainFunctional testing rocks the world?
• Use Behat if you *really* have customers / PMs / Analysts willing to read
the scenarios
• Functional tests can be written in PHPUnit too
• Functional testing is slow
Run it on a CI platform
On every commit is perfect, but before deployment is good enough
Use Selenium-as-a-service if you can afford it (even if you can’t)
THANK YOU
@gggeek
https://github.com/kaliop-uk
https://github.com/kaliop
http://www.kaliop.co.uk/info/ez.html
Top Related