Always Be Clear Always Be Complete Always Be Concise Always Be Considerate.
Always be shipping
-
Upload
francesco-uliana -
Category
Technology
-
view
4.813 -
download
0
Transcript of Always be shipping
![Page 1: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/1.jpg)
Always be shippingMicroservice application Continuous Deployment with Docker
Linux Day Roma 2015
Oct 24th 2015
David Rossi e Francesco Uliana
demo code: github.com/francescou/docker-continuous-deployment1 / 66
![Page 2: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/2.jpg)
Who we areFrancesco UlianaTechnologist at CNR - Reti e Sistemi Informativi
Java dev, FP languages, DevOps, IoT
@warrior10111 | www.uliana.it/francesco | github.com/francescou
David RossiSysadmin at CNR - Reti e Sistemi Informativi
GNU\Linux & FOSS evangelist
2 / 66
1 / 66
![Page 3: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/3.jpg)
2 / 66
Software deployment maturity
3 / 66
![Page 4: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/4.jpg)
3 / 66
Software deployment maturity - level 0build on developer machine
little/no SCM usage (e.g. git, svn)
deploy over FTP
4 / 66
![Page 5: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/5.jpg)
4 / 66
Software deployment maturity- level 1building automation tool (e.g. GNU Makefile, Ant, Maven,Rake)
proficient SCM usage (branches, tags)
5 / 66
![Page 6: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/6.jpg)
5 / 66
Software deployment maturity- level 2Unit testing (e.g. jUnit, TestNG)
Continuous Integration (e.g. Jenkins, Bamboo, CircleCI)
Artifact Repository
6 / 66
![Page 7: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/7.jpg)
6 / 66
Software deployment maturity- level 3externalized (and versioned) application configuration
extensive (and up to date) documentation about the releaseprocess
how to scale?
how to diff instances?
7 / 66
![Page 8: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/8.jpg)
7 / 66
Continuous Deployment
8 / 66
![Page 9: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/9.jpg)
8 / 66
Continuous Deploymentbuild software in such a way that it can be released toproduction at any time. (Martin Fowler)
short cycles
software can be reliably released at any time
9 / 66
![Page 10: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/10.jpg)
9 / 66
Key pointsautomated testing
Continuous Integration (CI)
rapidly, reliably and repeatedly push outenhancements/bugfixes at low risk and with minimal manualoverhead
10 / 66
![Page 11: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/11.jpg)
10 / 66
Benefits1. Accelerated Time to Market
2. Improved Productivity and Efficiency
3. Reliable Releases
4. Improved Product Quality
11 / 66
![Page 12: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/12.jpg)
11 / 66
Requirements and constraints1. good knowledge of several tools
2. less "agility"
3. team maturity
12 / 66
![Page 13: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/13.jpg)
12 / 66
Tools of the trade
13 / 66
![Page 14: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/14.jpg)
13 / 66
14 / 66
![Page 15: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/15.jpg)
14 / 66
lightweight
Linux
containers
15 / 66
![Page 16: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/16.jpg)
15 / 66
Diagram source: Docker Inc.
16 / 66
![Page 17: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/17.jpg)
16 / 66
Dockerdeployment of applications inside software containers
layer of abstraction and automation of operating-system-level virtualization
> 25k stars on github
> 1100 contributors
RESTful API
17 / 66
![Page 18: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/18.jpg)
17 / 66
Inside a Docker Container1. your application code
2. runtime
3. system tools
4. system libraries
5. configuration
18 / 66
![Page 19: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/19.jpg)
18 / 66
Hands on
19 / 66
![Page 20: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/20.jpg)
19 / 66
Steps1. Create a Dockerfile
2. Build container image
3. Run the image
20 / 66
![Page 21: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/21.jpg)
20 / 66
Docker Build (1/3)francesco ~/consul-template-nginx $ ls -ltotal 12-rw-r--r-- 1 francesco francesco 498 6 ago 15.19 Dockerfile-rw-r--r-- 1 francesco francesco 507 6 ago 15.43 nginx.conf.tpl-rw-r--r-- 1 francesco francesco 93 6 ago 15.19 README.md
21 / 66
![Page 22: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/22.jpg)
21 / 66
Docker Build (2/3)francesco ~/consul-template-nginx $ cat Dockerfile
FROM nginx:1.9.3
WORKDIR /opt/consul-template-nginx
RUN apt-get update && apt-get install -y curl
RUN curl --location-trusted https://github.com/hashicorp/consul-template/.../consul-template_....tar.gz | tar -xz
COPY nginx.conf.tpl ./
CMD service nginx start && ./consul-template/consul-template -consul consul:8500 -template './nginx.conf.tpl:/etc/nginx/nginx.conf:service nginx reload'
22 / 66
![Page 23: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/23.jpg)
22 / 66
Docker Build (3/3)francesco ~/consul-template-nginx $ docker build -t myapplication:1.0 .
23 / 66
![Page 24: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/24.jpg)
23 / 66
Docker Rundocker run --name my-nginx -p 8080:80 myapplication:1.0
24 / 66
![Page 25: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/25.jpg)
24 / 66
Just a single container?
25 / 66
![Page 26: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/26.jpg)
25 / 66
usually...multiple services
26 / 66
![Page 27: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/27.jpg)
26 / 66
multiple processes
27 / 66
![Page 28: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/28.jpg)
27 / 66
multiple containers!
28 / 66
![Page 29: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/29.jpg)
28 / 66
Case study: web application4 Containers:
1. web frontend (e.g. apache, nginx)
2. web backend
3. cache (e.g. redis)
4. Database (e.g. mysql, postgres)
29 / 66
![Page 30: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/30.jpg)
29 / 66
Docker Compose
30 / 66
![Page 31: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/31.jpg)
30 / 66
31 / 66
![Page 32: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/32.jpg)
31 / 66
docker-compose.ymlweb: image: node volumes: - ./app:/opt/ links: - db ports: - "3000:3000" command: node /opt/app/index.jsdb: image: mongo ports: - "27017:27017"
32 / 66
![Page 33: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/33.jpg)
32 / 66
More complex docker-compose.ymlpostgresql: image: sameersbn/postgresql:9.4-3 environment: - DB_USER=gitlab - DB_PASS=password - DB_NAME=gitlabhq_production volumes: - /srv/docker/gitlab/postgresql:/var/lib/postgresqlgitlab: image: sameersbn/gitlab:8.0.4 links: - redis:redisio - postgresql:postgresql ports: - "10080:80" - "10022:22" environment: - TZ=Asia/Kolkata - GITLAB_TIMEZONE=Kolkata - .............. volumes: - /srv/docker/gitlab/gitlab:/home/git/dataredis: image: sameersbn/redis:latest volumes: - /srv/docker/gitlab/redis:/var/lib/redis
33 / 66
![Page 34: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/34.jpg)
33 / 66
Manual Docker Compose deployment
docker-compose up -d
34 / 66
![Page 35: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/35.jpg)
34 / 66
...but(few seconds) downtime!
you still need manual intervention
35 / 66
![Page 36: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/36.jpg)
35 / 66
Automated application Deploymentcontinuous integration tool (e.g. Jenkins) should performdocker-compose up -d on each "release" build
36 / 66
![Page 37: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/37.jpg)
36 / 66
Docker Compose UI
37 / 66
![Page 38: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/38.jpg)
37 / 66
38 / 66
![Page 39: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/39.jpg)
38 / 66
Docker Compose UIgithub.com/francescou/docker-compose-ui
docker-compose REST API wrapper
open source (MIT)
93 stars, 12 forks
39 / 66
![Page 40: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/40.jpg)
39 / 66
Continuous Deployment Lifecycle(dev perspective)git clone myrepo.gitgit co -b awesomefeatureimplement featuremerge requestmerge request acceptance triggers Jenkins Build
mvn clean installdocker build -t application:1.1curl -X POST http://docker:5000/api/v1/projects --data'{"id":"myapp"}' -H'Content-type: application/json'
40 / 66
![Page 41: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/41.jpg)
40 / 66
Zero downtime deployments
41 / 66
![Page 42: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/42.jpg)
41 / 66
42 / 66
![Page 43: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/43.jpg)
42 / 66
Live Demosee github.com/francescou/docker-continuous-deployment
43 / 66
![Page 44: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/44.jpg)
43 / 66
Once in productionmonitoring
alerting
log analysis
44 / 66
![Page 45: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/45.jpg)
44 / 66
HA, a philosophy...
45 / 66
![Page 46: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/46.jpg)
45 / 66
by xkcd.com
# Last step...yum erase openssh-server
46 / 66
![Page 47: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/47.jpg)
46 / 66
Habits are changing...Sysadmin scope of interest becomes broader
OS is the last part, like installing java
Sysadmin needs developers procedural skills
Keyword: Automation and Planning
47 / 66
![Page 48: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/48.jpg)
47 / 66
Production1. Provisioning Bare Metal
2. Hardware Lifecycle management
The sysadmin must support different vendors andheterogeneous tecnologies
Use abstraction from hardware (VMs help a lot!)
Avoid lock-in tecnologies !!!
48 / 66
![Page 49: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/49.jpg)
48 / 66
Service must be Continuous1. Automation and planning
2. Monitoring and alerting
3. Backup & Restore
4. Failure (Hardware & Software) solving
49 / 66
![Page 50: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/50.jpg)
49 / 66
Context Separation1. Production, Staging, Development, ...
For the sysadmin... all is CRITICALDevelopers does't feel happy when their builds fail
50 / 66
![Page 51: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/51.jpg)
50 / 66
Planning1. Scheduled to... NOW!
Time is a thing you cannot buy in black market!
Infrastructure automation gives flexibility shortening timeto production
Dev and Ops can plan time for a beer... :)
51 / 66
![Page 52: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/52.jpg)
51 / 66
52 / 66
![Page 53: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/53.jpg)
52 / 66
Opensource tools:1. Provisioning and configuration management service
+
2. Hardware inventory service
53 / 66
![Page 54: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/54.jpg)
53 / 66
Opensource tools:3. Discovery, monitoring, metrics database and alertingservice
+
4. Log service and log parsing dashboard
+
54 / 66
![Page 55: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/55.jpg)
54 / 66
55 / 66
![Page 56: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/56.jpg)
55 / 66
ELK
56 / 66
![Page 57: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/57.jpg)
56 / 66
Zabbix: Cassandra nodes storage
57 / 66
![Page 58: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/58.jpg)
57 / 66
Zabbix: Apache metrics
58 / 66
![Page 59: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/59.jpg)
58 / 66
Machine Lifecycle1. Foreman builds machine (metal or VMs) with dynamictemplates
2. High monitoring automation with Zabbix low discoveryrules and templates
3. Metrics exposed to third party tools (like Grafana)
4. Production: log server configured for docker containers
59 / 66
![Page 60: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/60.jpg)
59 / 66
Enterprise
60 / 66
![Page 61: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/61.jpg)
60 / 66
Habits to avoid (-)(-) Server is a holy place but Ssh is allowed
(-) Environment separation only at physical level (when planned)
(-) Lack of strategy for log services.
(-) Limited sharing of the know-how between teams. Cooperation is limitedto problem solving
(-) Vertical scaling. Expensive hardware. Complexity increases exponentialy
61 / 66
![Page 62: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/62.jpg)
61 / 66
Habits to acquire (+)(+) Ssh is not to be used in production
(+) Containers permits isolation, icc off
(+) Use version control system and build environment for infrastructure
(+) Common tecnologies permit coral planning and better relationshipsbetween Ops and Dev
(+) Horizontal scaling. Commodity hardware. Complexity increases linearly
62 / 66
![Page 63: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/63.jpg)
62 / 66
Google commodity server
63 / 66
![Page 64: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/64.jpg)
63 / 66
Links:Syslog-ng server and compose ui
http://bitfieldconsulting.com/category/tags/devops
http://www.slideshare.net/albertspijkers/linux-containers-and-redhat-7pdf
64 / 66
![Page 65: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/65.jpg)
64 / 66
TakeawaysImmutable containers are awesome
Automate all the things
Abstract from Hardware
65 / 66
![Page 66: Always be shipping](https://reader031.fdocuments.net/reader031/viewer/2022022202/58794bef1a28abb1418b4f21/html5/thumbnails/66.jpg)
65 / 66
Q&A time
66 / 66