Transcript of Scaling your jenkins master with docker
- 1. Scaling Your Jenkins Master with Docker London, UK 23-24
June
- 2. #jenkinsconf Contents WHO: Wyplay WHAT: Docker, Docker in
Jenkins, Jenkins in Docker WHY: Situation Before, Issues HOW:
Setup, Usage WHERE: Future 2
- 3. #jenkinsconf Contents WHO: Wyplay WHAT: Docker, Docker in
Jenkins, Jenkins in Docker WHY: Situation Before, Issues HOW:
Setup, Usage WHERE: Future 3
- 4. #jenkinsconf WHO am I I Worked in Queueing Networks,
Simulation, Smartcards, now TV Software for Operators Develop
Set-top-box OS (Canal+, Proximus, SkyIt,..) 80% of Developers
4
- 5. #jenkinsconf Contents WHO: Wyplay WHAT: Docker, Docker in
Jenkins, Jenkins in Docker WHY: Situation Before, Issues HOW:
Setup, Usage WHERE: Future 5
- 6. #jenkinsconf WHAT: Docker Company (Docker Inc) Open Source
(Apache license) platform for building, shipping, running
distributed applications Technology: manage and run lightweight
Linux containers Isolation Portability, Shipping 6
- 7. #jenkinsconf WHAT: Docker in Jenkins 7 Docker Plugin
Dynamically provision slaves Run build Tear-down slave Slaves in
the Cloud
- 8. #jenkinsconf WHAT: Jenkins in Docker Official Jenkins LTS
docker image From Cloudbees, sponsor of DockerCon Open Source
https://github.com/jenkinsci/docker Easy to use docker pull jenkins
docker run -p 8080:8080 jenkins 8
- 9. #jenkinsconf Contents WHO: Wyplay WHAT: Docker, Docker in
Jenkins, Jenkins in Docker WHY: Situation Before, Issues HOW:
Setup, Usage WHERE: Future 9
- 10. #jenkinsconf WHY: Situation Before 10 One Master 34
attached slaves 527 jobs 78 plugins admin users Fast-paced
environment. No real control
- 11. #jenkinsconf WHY: Issues 11 Performance Reliability
Inability to upgrade Downtime => Projects spin-off masters
Security
- 12. #jenkinsconf Contents WHO: Wyplay WHAT: Docker, Docker in
Jenkins, Jenkins in Docker WHY: Situation Before, Issues HOW:
Setup, Usage WHERE: Future 12
- 13. #jenkinsconf HOW: Setup for one master (1) Three
possibilities for /var/jenkins_home inside the container: docker
run -p 8080:8080 jenkins mounted from a host directory: docker run
-v :/var/jenkins_home jenkins mounted from a data volume container:
docker run -v /var/jenkins_home --name=data busybox true docker run
--volumes-from=data jenkins 13
- 14. #jenkinsconf HOW: Setup for one master (2) Summary for one
master: pull the official Jenkins image: $ docker pull jenkins
create the data volume and master containers: $ docker run -v
/var/jenkins_home --name=frog-data busybox true run the master
container: $ docker run -t -i -u root -p 8084:8080 -p 50004:50004
--volumes-from=frog-data --name=frog jenkins 14
- 15. #jenkinsconf HOW: Setup n masters Launch an httpd
dispatcher (e.g., Apache) 443 (https) vhosts(name) 808i Launch
docker containers: the Data Volume containers, with names the
Jenkins masters containers linked with --volumes- from and 808i /
5000i ports map each container port 8080 to host ports 8080+i
15
- 16. #jenkinsconf HOW: Usage 16 Pressed Json DRAFT Apache
cert.crt privat.key frog-data polka-data frog Pressed Json DRAFT
pressed .json LIVE Pressed Json DRAFT Pressed Json DRAFT polka
Pressed Json DRAFT Pressed Json DRAFT pressed .json LIVE Pressed
Json DRAFT Pressed Json DRAFT Frog.conf ServerName frog-ci.com
ProxyPass localh:8084 X-F-Port443 g6-data tools-data 443 8084 8080
8080 808i --volumes -from --volumes -from 50004 Slaves
- 17. #jenkinsconf HOW: Admin Usage (1) Install plugins (from
/tmp/plugins) $ docker run -t -i --rm --volumes-from=frog-data -v
/tmp:/tmp ubuntu bash root@1a210c0a1846:/# cd
/var/jenkins_home/plugins root@1a210c0a1846:/# cp /tmp/plugins/* .
root@1a210c0a1846:/# exit Similar technique for: full backup debug
data (SCM Sync git repos, Plots csv, etc.) 17
- 18. #jenkinsconf HOW: Admin Usage (2) Change image stop
container, destroy, create a new one $ docker stop frog $ docker rm
frog $ docker run --volumes-from=frog-data --name frog $ docker
logs -f frog 18
- 19. #jenkinsconf HOW: Admin Usage (3) Three levels of backup
Vmware: dump of entire docker server Docker: copy all Data Volume
Jenkins: SCM Sync plugin Three levels of monitoring Vmware, Docker,
Jenkins Security: Role Strategy plugin 19
- 20. #jenkinsconf Contents WHO: Wyplay WHAT: Docker, Docker in
Jenkins, Jenkins in Docker WHY: Situation Before, Issues HOW:
Setup, Usage WHERE: Future 20
- 21. #jenkinsconf WHERE: do we go from here? Orchestration
manual setup ok for a few more => orchestration tool (e.g.,
Docker Compose) 21
- 22. #jenkinsconf WHERE: do we go from here? 22 Docker vs
Config/Automation tools (chef, puppet, ansible, saltstack) Before:
use ansible to setup hardware/VM, install packages, deploy code,
run app. After: use ansible to setup hardware/VM, install Docker,
run containers. use Dockerfiles to install packages, deploy code,
run app.
- 23. #jenkinsconf WHERE: do we go from here? Monitoring (docker
level) Find a way to Configure System programmatically LDAP Plugins
upgrades TCP port for slaves Markup formatter SCM Sync, Roles, IRC,
etc. 23
- 24. #jenkinsconf Thank You @cmuller13 on twitter 24