Watson Customer Engagement
20. – 21. Juni 2017
IBM Labor Böblingen
Docker und IBM Digital
Experience in Docker
Container
1
Watson Customer Engagement
v
What is docker
VMs vs. containers
Terminology
Docker components
2
Introduction
6/22/2017
Watson Customer Engagement
• An implementation of an application container
– Containers isolate the differences between applications inside the
container everything outside the container can be standardized
– Lightweight “virtualization” solution
– Almost no performance overhead
– Mainly based on namespaces & cgroups
• Allows you to package an application with all of its dependencies into a
standardized unit for software development.”
• Can be moved between environments and run without changes
3
What is docker?
Watson Customer Engagement4
VMs vs Containers
App
A
Hypervisor (Type 2)
Host OS
Server
Guest
OS
Bins/
Libs
App
A’
Guest
OS
Bins/
Libs
App
B
Guest
OS
Bins/
Libs
VM
Guest
OS
Guest
OS
Host OS
Server
Bins/Libs
Ap
p A
Bins/Libs
Ap
p B
Ap
p B
’
Ap
p B
’
Ap
p B
’
Container
Containers are isolated,
but share OS and, where
appropriate, bins/libraries
…faster, less overheadA
pp
B
Watson Customer Engagement
• Image
– A read-only snapshot of a container stored in Docker Hub to be used as
a template for building containers
– Series of read-only storage layers
• Container
– The standard unit in which the application service resides
– Based on an image
– At start time a r/w layer is creation on top of the image‘s read-only
layers
5
Terminology
Watson Customer Engagement
• Docker Hub/Registry/Store
– Stores, distributes and shares container images
– Available in SaaS or Enterprise to deploy anywhere you choose
• Docker Engine
– A program that creates, ships and runs application containers
– Runs on any physical and virtual machine or server locally, in private or
public cloud
– Client communicates with Engine to execute commands
6
Docker components
Watson Customer Engagement
• Docker Compose
– Tool to define and run multiple containerized applications (like DX+DB2)
– Allows to define networks, storage etc.
• Docker swarm
– Manages a cluster of containers
• Docker Machine
– Tool for provisioning and managing your Dockerized hosts (hosts with
Docker Engine on them)
• Docker on Cloud
7
Docker components
Watson Customer Engagement
• Marathon (container management) / Mesos (Hardware abstraction)
• Kubernetes
• Cloud native options
• Bluemix
• ...
8
Docker orchestration
Watson Customer Engagement
v
Create your server / VM
Install docker engine
Install docker compose
Install docker registry
Build docker images
Created compose files
9
How to get started
6/22/2017
Watson Customer Engagement
• Community Edition (CE) and Enterprise Edition (EE)
• Available on Linux, Cloud, Windows, and OS X
– For RedHat and CentOS yum installation
– Windows has two options
• Docker for Windows (Win10 + Hyper-V)
• Docker Toolbox (runs Virtualbox underneath)
• Is the runtime environment for the containers
– Default filesystem size for containers is 10G – for DX containers use
40G (--storage-opt dm.basesize=40G) with LVM storage driver
10
Install docker engine
Watson Customer Engagement
• Requires docker engine
• docker-compose based start is recommended
• Pull it using curl and make it executable– curl -L
https://github.com/docker/compose/releases/download/${version}/docker-
compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
– chmod +x /usr/local/bin/docker-compose
• Test the installation– docker-compose –version
11
Install docker compose
Watson Customer Engagement
• A central place to store and distribute docker images
– Runs in a docker container
• Docker Hub is a free registry to use
– Be aware of licencing issues
• Secured by default
• It‘s recommeded to use the compose version
• User / Password for access is via authorization file (similar to HTTP
Server)
– Run docker login $(hostname):5000 to verify the registry12
Install docker registry
Watson Customer Engagement
v
Manual build
Automated build using a docker-
file
Footer13
Build docker images
6/22/2017
Watson Customer Engagement
• Start the base image
• Run the installation commands inside the container
• Exit the container
• Commit the changes
• Push the container to the registry
14
Build docker images - Manual build
Watson Customer Engagement
• Create a docker file
– Every step creates a new layer / image impacts image size
• Run docker build command
– Images can get tagged with a name
• Tag the image (unless done during build)
– Tags are in <name>:<tag> format
– <name> format: [<registry-host>:<registry-port>/]<image-name>
• Push the image to the registry
15
Build docker images - Automated build
using a docker-file
Watson Customer Engagement
• System defined networks
• User defined networks
– User defined networks allow a better isolation of the containers
• All containers on a network can communicate with each other
• Containers can be connected to multiple network
• List the existing networks by runningdocker network ls
16
Docker network
Watson Customer Engagement
• Are directories on the host file-system (i.e. Outside of the container)
• Can be seens as mounting a host-filesystem over a containers directory
– Content of the containers directory which is mapped to the volume is
copied to the host file-system at initialization time
• Needed to
– Persist changes
– Share data with other containers
17
Docker volumes
Watson Customer Engagement
• Docker compose uses a .yml file to define the runtime environment
• Environment variables can be defined in a „.env“ file
• Using the compose file we can combine multiple containers to a working
application
– WebSphere Portal container + DB2 container DX application
• No synchronization between containers by compose
– Your start script must cover dependencies
• Starting the .yml via docker-compose– docker-compose –p 1 -f ./docker-compose_wasfp10_cf12_derby.yml up -t60
18
Creating the compose file
Watson Customer Engagement
• Virtualization Policy for IBM Software
http://www-01.ibm.com/software/support/virtualization_policy.html
• Virtualization Support Policy for WebSphere Application Server
http://www-01.ibm.com/support/docview.wss?uid=swg21242532
• Virtualization Support Policy for DX
http://www-01.ibm.com/support/docview.wss?uid=swg21243549
• This includes VMWare, Docker, KVM, PowerVM Hypervisors, …
• Rarely ever have we encountered software issues with virtualization
other than performance sizing, communication or other environment related issues
20
Support
Watson Customer Engagement
• We have added testing with Docker containers and published an article on a possible setup here: https://developer.ibm.com/digexp/docs/docs/customization-administration/ibm-digital-experience-on-docker-containers/
– Setup is for development with local Derby as DB
• Internally we have done additional setups with DB2 in a separate container as well as other scenarios like farming or cluster
21
DX and Docker
Watson Customer Engagement
• Options to install:
– Install Portal into a Docker container and save as image
– Link or copy an existing install into a Docker container and save as an image
• Options to configure:
– Database, http server inside containers or not – possible orchestration with docker swarm and docker compose
– Clustered or Farmed or standalone
22
Install and Config
Watson Customer Engagement
• Based on Market demand we will keep on testing new Container software solutions and publish how to articles
• Planning to publish to a public Container registry as well
• Currently can easily build images locally based on need (e.g. with local DB2 or remote or …)
23
DX Container roadmap
Watson Customer Engagement
v
Image hierarchy
Image size
Start / stop
Build performance
Use LVM instead of loop device
Automation of deployments
Topologies
Footer24
Lessons learned
6/22/2017
Watson Customer Engagement
• Images are build on a layered basis
25
Lessons learned - Image hierarchy
Docker Hub
CentOS
Customized
CentOS
yumDB2
db2DB2
Custom
modifications
imclIIM
WAS 8.5.0.9
WP v8.5.0.0
imcl
WAS 8.5.0.10
WP v8.5.0.0
imcl
WAS 8.5.0.9
WP v8.5.0.9
CF09
WAS 8.5.0.9
WP v8.5.0.10
WAS 8.5.0.9
WP v8.5.0.9.c
Cust.
WAS 8.5.0.9
WP
v8.5.0.10.c
Cust.
WAS 8.5.0.10
WP v8.5.0.9
CF09
WAS 8.5.0.10
WP v8.5.0.10
WAS 8.5.0.10
WP v8.5.0.9.c
Cust.
WAS 8.5.0.10
WP
v8.5.0.10.c
Cust.
CF10
CF10
Watson Customer Engagement
• Images are a series of read-only storage layers
• Each layer adds to the image size
• Each step in the build file creates a new layer
26
Lessons learned - Image size
FROM registry.host.de:5000/centos_iim:1.8.5
MAINTAINER Hermann Huebler “[email protected]"
LABEL name="registry.host.de:5000/test" version="1.0"
RUN echo "192.168.57.17 httpHost" >> /etc/hosts && \
mkdir -p /iimrepo/pkgs/wp85 && \
cd /iimrepo/pkgs/wp85 && \
echo "Downloading WSP_Enable_8.5_Setup.zip ..." && \
curl -o /iimrepo/pkgs/wp85/WSP_Enable_8.5_Setup.zip
http://httpHost/wp85/bin/WSP_Enable_8.5_Setup.zip && \
echo "Unzipping WSP_Enable_8.5_Setup.zip" && \
unzip -o ./WSP_Enable_8.5_Setup.zip > /tmp/unzip.log 2>&1
RUN rm -rf /iimrepo/
RUN echo "Done ..."
ENTRYPOINT ["/bin/startContainer.sh"]
FROM registry.host.de:5000/centos_iim:1.8.5
MAINTAINER Hermann Huebler “[email protected]"
LABEL name="registry.host.de:5000/test2" version="2.0"
RUN echo "192.168.57.17 httpHost" >> /etc/hosts && \
mkdir -p /iimrepo/pkgs/wp85 && \
cd /iimrepo/pkgs/wp85 && \
echo "Downloading WSP_Enable_8.5_Setup.zip ..." && \
curl -o /iimrepo/pkgs/wp85/WSP_Enable_8.5_Setup.zip
http://httpHost/wp85/bin/WSP_Enable_8.5_Setup.zip && \
echo "Unzipping WSP_Enable_8.5_Setup.zip" && \
unzip -o ./WSP_Enable_8.5_Setup.zip > /tmp/unzip.log 2>&1 && \
rm -rf /iimrepo/ && \
echo "Done ..."
ENTRYPOINT ["/bin/startContainer.sh"]
6.078
GB941.4
MB
registry.host.de:5000/centos_iim:1.8.5
938 MB
Watson Customer Engagement
• Start script of the container is specified in the ENTRYPOINT
["/bin/startContainer.sh"] becomes PID 1
– Can be overwritten in the compose file
• Stopping the container is by sending a kill signal
– The PID 1 in the container gets this signal forwarded[root@wp85-derby /]# ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 21:03 ? 00:00:00 bash /bin/startContainer.sh
– The script running as PID 1 must handle the kill signal for a controlled
container stop
– DB2 & DX start scripts start a daemon Script must stay active
27
Lessons learned - Container start / stop
Watson Customer Engagement
:
trap "echo '**CAUGHT TRAP**' ; \
/opt/IBM/WebSphere/wp_profile/bin/stopServer.sh WebSphere_Portal; \
exit 0" HUP INT QUIT TERM
:
##
## start service in background here
/opt/IBM/WebSphere/wp_profile/bin/wp85up.sh &
WPPID=$!
echo "Background PID=${WPPID}"
wait ${WPPID}
rc=$?
28
Lessons learned - Container start / stop
Watson Customer Engagement
• Installation binaries must be copied to the container for build
• Docker provides a COPY instruction
– DX binaries are too big and cause problems
– Very slow for large files – goes thru the docker daemon
• Providing the binaries via an HTTP server and pulling via curl
– Significantly faster solution
– More stable
29
Lessons learned – Build performance
Watson Customer Engagement
• By default docker engine uses a loop device to mount host file-system
– Easy setup
– Size of DX images causes troubles
– Hangs
• Docker provides the ability to mount LVM logical volumes
– Recommended way for build system
– See https://docs.docker.com/engine/userguide/storagedriver/device-
mapper-driver/ for instructions
• Use fast disks (SSD) to improve build performance
30
Lessons learned – Use LVM instead of
loop device
Watson Customer Engagement
• To allow automation of image builds application deployment automation is
a must
• We used IBM Autodeploy to deploy the portal
31
Lessons learned – Automation of
deployments
Build file
.ear
Watson Customer Engagement
• Works nicely for single server installations
– With derby and or DB2 database (on the same server)
• Farming with shared or copied profile directory seems a good fit
– Consider the managed pages limitations
• Clustering difficult due to networking requirements
32
Lessons learned – Topologies
Watson Customer Engagement
• Install Docker Engine
https://docs.docker.com/engine/installation/
• Docker and the Device Mapper storage driver
https://docs.docker.com/engine/userguide/storagedriver/device-mapper-
driver/
• Docker Compose
https://docs.docker.com/compose/
• Docker Registry
https://docs.docker.com/registry/
• IBM Digital Experience on Docker Containers
https://developer.ibm.com/digexp/docs/docs/customization-
administration/ibm-digital-experience-on-docker-containers/33
For Additional Information
Watson Customer Engagement
• IBM AutoDeploy
http://appscc.ibm-sba.com/autodeploy/latest/
• Virtualization Policy for IBM Software
http://www-01.ibm.com/software/support/virtualization_policy.html
• Virtualization Support Policy for WebSphere Application Server
http://www-01.ibm.com/support/docview.wss?uid=swg21242532
• Virtualization Support Policy for DX
http://www-01.ibm.com/support/docview.wss?uid=swg21243549
34
For Additional Information
Top Related