Docker Hong Kong Meetup (Jul 2017)
Introduction to Docker
Clarence Ho
Independent Software EngineerDocker HK Meetup [email protected]@gmail.com
3
Topics
• Introduction to Docker• Latest Features of Docker• Docker Adoption• Docker Editions• Demo• Open Discussion
What is Docker?
Introduction to Docker
5
A brief explanation of Containers
An image is a lightweight, stand-alone, executable package that includes everything needed to run a piece of software• Contains the application executable and their dependencies• Built with instructions from a Dockerfile
A container is a runtime instance of an image – what the image becomes in memory when actually executed• Run apps natively on the host machine’s kernel• Running in a discrete process (isolated environment)• Containers on the same machine share a single kernel
6
Containers vs Virtual MachineVirtual Machine Diagram Container Diagram
7
Container vs VM - Performance Benchmark(Just for reference)
On a modest Intel server (16GB Ram)• 536 Linux Containers• 37 KVM Virtual Machines
Reference: https://insights.ubuntu.com/2015/06/11/how-many-containers-can-you-run-on-your-machine/
8
Virtualization
9
Containerization
10
Benefits of Containers
• More efficient in resource utilization− The same computing resources can run more containers than VMs− Containers organically consume the resources they need (bound by the
maximum value assigned). For VM, it will take up all the resources assigned when startup
• Better for cloud deployment (Microservices and Devops)− It’s a general practice to have separate images for difference components
for the same application (e.g. DB, App Server, Web Server)− More easy to deploy/upgrade/scale an individual component, without
impacting others
Latest Features of Docker
(Content based on Dockercon 2017)
12
Latest Features of Docker
• Versioning and Release Schedule• Builder• Runtime• Swarm Mode• Compose
Version and Release Schedule
Latest Features of Docker
14
New Versioning
15
New Release Schedule
Builder
Latest Features of Docker
17
Multi-Stage Builds
Traditional Dockerfile that includes build tools:
➜ Target is to reduce the size of Docker image
FROM alpine
RUN apk add make g++
ADD . /src
RUN cd /src && make
EXPOSE 80
ENTRYPOINT /usr/local/bin/app
18
Multi-Stage BuildsA Dockerfile that use multi-stage build:
➮ Final image will not include the build tools and libraries
FROM alpine AS build-env
RUN apk add make g++
ADD . /src
RUN cd /src && make
FROM busybox
COPY --from=build-env /src/build/app /usr/local/bin/app
EXPOSE 80
ENTRYPOINT /usr/local/bin/app
Runtime
Latest Features of Docker
20
Data Management Commands
• docker system df
➜ docker system sub-command added
$ docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 5 1 2.777 GB 2.647 GB (95%)
Containers 1 1 0 B 0B
Local Volumes 4 1 3.207 GB 2.261 GB (70%)
• docker system prune
• docker container/image/network/volume prune
Demo
22
Docker Playground
• Play with Docker− http://labs.play-with-docker.com
• Github− https://github.com/play-with-docker/play-with-docker
Swarm Mode
Introduction to Service Orchestration
24
Introduction to Service Orchestration
• Management− Need a manager to maintain the cluster state, and serve requests for
container management (schedule/stop/scale up/scale down)• Security
− All nodes within the cluster should be able to communicate securely• Service Discovery
− Need to be able to identify and locate a container service by using DNS• Load Balancing
− Need to be able to scale up/down containers with auto load balancing• Networking
− Able to segregate the network for different scenarios• Update/Rollback
− Support update and rollback of container services across the cluster
⌘ Container Services need Orchestration
25
Docker’s answer to Service OrchestrationDocker Swarm mode
26
Docker Swarm ModeSecurity - All managers and nodes communicates via TLS
27
Docker Swarm ModeLoad Balancing - Ingress Routing Mesh
28
Docker’s answer to Service OrchestrationLoad Balancing - External Load Balancer
29
Docker’s answer to Service OrchestrationLoad Balancing - Service to Service Communication
30
Introduction to Service Orchestration
• A DNS server was embedded in a Swarm cluster• Swarm mode has an internal DNS component that
automatically assigns each service in the swarm a DNS entry
• The swarm manager uses internal load balancing to distribute requests among services within the cluster based upon the DNS name of the service
Service Discovery with DNS
Swarm Mode
Latest Features of Docker
32
Service Rollback on Failure
“rollback” action added to --update-failure-action(in addition to “pause” and “continue”)
with all the associated flags
--rollback-delay--rollback-failure-action--rollback-max-failure-ratio--rollback-monitor--rollback-parallelism
swarm mode improvement
33
Topology Aware Scheduling
docker service create --replicas=6 postgresdocker service create --replicas=2 webapp
swarm mode improvement
34
Topology Aware Scheduling
docker service create --replicas=6 --placement-pref-add=rack postgresdocker service create --replicas=2 --placement-pref-add=rack webapp
swarm mode improvement
docker node update --label-add rack SFO-1 docker node update --label-add rack SFO-2
35
Service Logsswarm mode improvement
$ docker service create --replicas 2 --name redis redis$ docker service logs redisredis.2.najk8sq1klac@node2 | _.-``__ ''-._redis.2.najk8sq1klac@node2 | _.-`` `. `_. ''-._ Redis 3.2.8 (00000000/0) 64 bitredis.1.lfkijq3fx3q8@node1 | _.-``__ ''-._redis.2.najk8sq1klac@node2 | .-`` .-```. ```\/ _.,_ ''-._redis.1.lfkijq3fx3q8@node1 | _.-`` `. `_. ''-._ Redis 3.2.8 (00000000/0) 64 bitredis.2.najk8sq1klac@node2 | ( ' , .-` | `, ) Running in standalone moderedis.1.lfkijq3fx3q8@node1 | .-`` .-```. ```\/ _.,_ ''-._redis.2.najk8sq1klac@node2 | |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379redis.1.lfkijq3fx3q8@node1 | ( ' , .-` | `, ) Running in standalone moderedis.2.najk8sq1klac@node2 | | `-._ `._ / _.-' | PID: 1redis.1.lfkijq3fx3q8@node1 | |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379redis.2.najk8sq1klac@node2 | `-._ `-._ `-./ _.-' _.-'redis.1.lfkijq3fx3q8@node1 | | `-._ `._ / _.-' | PID: 1...
Swarm Mode - Secrets ManagementLatest Features of Docker
37
Securely Distributing Passwords
● Service often require sensitive information (like passwords, keys, etc.)
● Need a way to securely distribute such information across the cluster
38
Securely Distributing PasswordsThe Old Way
Pass as environment:$ docker service create -e password=TOTALLYSECURE dockercon
Password is stored on host and mount by container as volume:$ docker service create -v some/host/dir:/password dockercon
39
Securely Distributing PasswordsThe Old Way > Pass as environment > Problem
A developer need to debug the service, and the environment is dump into a debug log file.
40
Securely Distributing PasswordsThe Old Way > Save Secret in Volume > Problem
Volume must exist on every node that service needs to run on.
When service is rescheduled, secret stay on the host!
41
Docker SecretsSecrets are stored in the Raft Store
The Raft log is encrypted and secure
42
Docker SecretsSecrets are stored in the Raft Store
The encryption key of the Raft log can be further encrypted for added security
$ docker swarm update --autolock=true
43
Docker SecretsCreate a new secret
$ docker secret create my-password password.file
44
Docker SecretsUpon creation, secret shared across managers via the Raft Store
45
Docker SecretsUpdate service to use the secret
$ docker service update --secret-add=my-password Dockercon
46
Docker SecretsSecret only sent to nodes running the serviceStored in tmpfs mounted into the container
47
Docker SecretsNode failureService instance need to be rescheduled
48
Docker SecretsSecret moves with the serviceDead worker node does not have secret
49
Docker SecretsSecrets are new first-class objectsThe right way is also the easy way
Docker Compose
Latest Features of Docker
51
Compose to Swarm
It is now possible to deploy services using compose files directly from docker
➜ docker stack sub-command added
● docker stack deploy --compose-file docker-compose.yml <my_stack>
● docker stack list
● docker stack rm <my_stack>
52
Compose Format Version 3
Main differences from v2 are:
docker-compose.yml improvements
● Removed the non-portable options○ build○ volume-from○ …
● Added Swarm specific options
○ replicas
○ mode
○ ...
53
Long Syntax for Portsdocker-compose.yml improvement
ports:- 3000- 3000-3005- 49100:22- 9090-9091:8080-8081- 127.0.0.1:8001:8001- 127.0.0.1:5005-5010:5005-5010- 6060:7060/udp
Old Format (for port publishing):
54
Long Syntax for Portsdocker-compose.yml improvement
ports:- target: 6060
published: 7060protocol: udp
New Format (for port publishing):
55
Long Syntax for Volumesdocker-compose.yml improvement
volumes: - /var/lib/mysql - /opt/data:/var/lib/mysql - ./cache:/tmp/cache - datavolume:/var/lib/mysql - ~/configs:/etc/configs/:ro
Old Format (for volume mounting):
56
Long Syntax for Volumesdocker-compose.yml improvement
volumes: - type: bind source: ~/configs target: /etc/configs read_only: true
New Format (for volume mounting):
Docker Adoption
(Content based on Dockercon 2017)
58
What a Difference 3 Years Makes
Docker in Enterprise
Docker Adoption
60
Docker in in the Enterprise
Docker on Windows
Docker Adoption
62
Docker on Windows Server 2016● Now 98% of enterprise workloads supported by Docker● Proven benefits of Docker on Linux available to Windows Server
developers and IT Pros● One Docker platform and one adoption journey for all enterprise
applications and infrastructure● Docker CS Engine with Windows Server 2016 at no additional cost
63
Docker on Windows Server 2016
Docker EE is free and support by Microsoft directly
64
Windows and Hyper V Containers
65
Windows vs Linux Containers (Docker Store)
Oracle in Docker Store
Docker Adoption
67
Oracle on Docker Store
68
Oracle Database Enterprise Edition
Available as Docker imageFree for development and testing
Modernizing Traditional ApplicationsDocker Adoption
70
Legacy to Containerized AppThe proper way
71
I Want to Escape from VM ASAP, what to do?A faster way ⇨ Image2Docker
72
Sample Use Case2 applications (1 Linux, 1 Windows) running on VM
73
Sample Use Case2 applications (1 Linux, 1 Windows) running on VM
74
Sample Use Case2 applications (1 Linux, 1 Windows) running on VM
75
Sample Use Case
Steps:
76
Image2Docker - Linux
make preparemake buildmake builtin-prep
sudo bin/v2c-darwin64 build -n img.vmdk
https://github.com/docker/communitytools-image2docker-linux
77
Image2Docker - Windows
Install-Module Image2DockerImport-Module Image2Docker
ConvertTo-Dockerfile ` -ImagePath c:\iis.vhd ` -OutputPath c:\i2d2\iis ` -Artifact IIS
https://github.com/docker/communitytools-image2docker-win
78
Create a Hybrid Swarm
79
Deployment
Docker Editions
(Content based on Dockercon 2017)
Community and Enterprise EditionsDocker Editions
82
Enterprise and Community Editions
83
Docker Enterprise Edition (EE)CaaS enabled platform for the modern software supply chain
84
Docker Enterprise Edition (EE)Docker EE Components
85
Docker Enterprise Edition (EE)Docker EE Architecture
86
Docker Enterprise Edition (EE)
Docker EE Plans● Basic● Standard● Advanced
87
Docker Enterprise Edition (EE)Image - Promotion Branching
88
Docker Enterprise Edition (EE)Image - Scanning
89
Docker Enterprise Edition (EE)Image - Scanning Result (UCP)
90
Docker Enterprise Edition (EE)Mixed Windows/Linux Cluster
Docker for Various Platforms
Docker Editions
92
Docker CE and EESupported Platforms
93
Docker for various PlatformsExample : Docker for AWS
94
Docker for various PlatformsExample : Docker for Google Cloud (GCP)
Docker Cloud
Docker Editions
96
Docker Cloud• Manage Build and Images
− Provides a hosted registry service− Link to your source code repository
• Swarm Mode (Beta)− Provision swarms or register existing swarms to popular cloud providers− Support multiple providers in a single user interface− Use your Docker ID to authenticate and securely access personal or team
swarms• Standard Mode
− Link to your hosts, upgrade the Docker Cloud agent, and manage container distribution
− Deploy and manage nodes, services, and applications in Docker Cloud• Pricing
− Contact Docker
97
Docker CloudDocker Cloud provisions Docker CE Editions
98
Docker CloudProvision Swarms for multiple cloud providers
99
Docker CloudSwarm management
100
Docker Cloud vs Enterprise EditionFeature Docker EE Docker Cloud
Docker Engine Version Docker EE Docker CE, Docker EE (Basic)
Private Image Registry Your own registry Host by Docker
User Interface Docker UCP(Universal Control Plane)
Docker Cloud UI
Image Security Scan Support Support
User Security Create your own user/group,Role based access control
Docker ID
Docker Datacenter Included (Standard, Advance) Not included
Automated Development Pipelines Included Not included
Private Cloud Full Support Partially Support (Bring your own Swarm)
Pricing Visit Docker site Contact Docker
✦ Contact Docker for latest information
Service Orchestration (Alternatives)Docker Editions
102
Container Service Orchestration PlatformAlternatives
• Public Cloud Providers− Amazon EC2 Container Service− Google Container Engine (based on Kubernetes)
• Redhat Openshift− Redhat Enterprise Linux, Docker, Kubernetes
• CoreOS− Container Linux, Quay Container Registry, Tectonic Kubernetes
• Apache Mesos− DC/OS (Datacenter Operating System)
• IBM, HPE, Oracle, etc.
Demo
104
Docker Playground
• Play with Docker− http://labs.play-with-docker.com
• Github− https://github.com/play-with-docker/play-with-docker
105
Sample Application
• Github− https://github.com/clarenceh/docker-contact
Final Words
107
Let’s Keep the Meetup Running
• Let’s work together to keep the meetup active• Speakers WANTED• Share with each other about your Docker journey• Reach out for venues for deep dive
− Workshops− The best way to learn is to do some real stuff
• Containerize your application• Setup a Docker Swarm cluster• Use Docker Compose to deploy your stack
Hey, I need HELP!!!
Open Discussions
THANK YOU
Top Related