Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical...
Transcript of Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical...
![Page 1: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/1.jpg)
1
Cloud based networks Containers
Simon Csaba
BME-TMIT
![Page 2: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/2.jpg)
CONTAINERS
2
![Page 3: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/3.jpg)
Virtualization and performance?
» Motivation: » Virtualization = smthg(runs_smthg)
» Some kernel tasks are executed two times
» Increase the performance: decrease the overhead of „smthg”
3
![Page 4: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/4.jpg)
Container metaphore: logistics
» Logistic (management) problem » Many transport platforms, many product types
» How many package variants are required?
4
![Page 5: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/5.jpg)
Container metaphore: intermodal container
5
» Logistic (management) problem » Many transport platforms, many product types
» How many package variants are required?
» Use only one: the container, as a transportation standard
![Page 6: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/6.jpg)
„Transporting” the code in virtualized systems
6
» Trasnportation platform => execution environment
» Product type => computation task
![Page 7: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/7.jpg)
Application containers
7
![Page 8: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/8.jpg)
Linux containers solve everything (hm..)
8
![Page 9: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/9.jpg)
Paranthesis: case study (Why should we replace clouds with Docker?)
9
![Page 10: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/10.jpg)
Why Docker?
My World Needed To Change » 5+ individual teams building “micro services” in Java and Scala » Frictionless deployment of “micro-services” using Chef & AWS » 25+ separate “micro-services” deployed in the previous 18 months » Each service is typically deployed to a single AWS virtual machine » Each service is deployed 6x - dev, test, staging (2x) and production
(2x) » 25+ “micro-services” became nearly 150 AWS virtual machines
10
![Page 11: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/11.jpg)
Why Docker? COST!
The AWS bill is too damn high! » Decline in the global price of oil causing churn in our business » 6 AWS virtual machines per service isn’t sustainable with our budget » AWS monthly bill started to gain visibility from sr. management and
the board
11
![Page 12: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/12.jpg)
Why Docker? WASTE!
We weren’t using the compute and memory resources purchased from AMZN!
» Nearly all “micro-services” were at 1% CPU utilization » Nearly all “micro-services’ were only using 40% of memory (JVM) » 150+ virtual machines essentially sitting idle
12
![Page 13: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/13.jpg)
Why Docker? LOCK IN!
How would we leave AMZN if we wanted to? » Could we use Drillinginfo IT’s Openstack platform? » What about alternate IaaS providers like Rackspace or Azure? » What about Container as a Service (CaaS) providers like Joyent,
Tutum or Profitbricks? » What about using Amazon’s Container Service?
13
![Page 14: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/14.jpg)
My World Needs To Change - Problem Statement
“How can we deploy fewer virtual machines while increasing the density and utilization of services per machine without locking us into a specific IaaS provider?”
14
![Page 15: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/15.jpg)
Why Docker Is Important - Before Containers
Very inefficient use of memory and CPU resources
15
![Page 16: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/16.jpg)
Why Docker Is Important - After Containers
Isolated
services in
fewer
VMs...
… and use
VMs more
efficiently.
16
![Page 17: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/17.jpg)
Why Is Docker Important?
Docker container technology provides our “micro-services” platform: » Increased density of isolated “micro-services” per virtual machine
(9:1!) » Containerized “micro-services” are portable across machines and
providers » Containerized “micro-services” are much faster than virtual machines
17
![Page 18: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/18.jpg)
End of case study
18
![Page 19: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/19.jpg)
Introduction: Linux containers
Container = Operation System Level virtualization method for Linux
Kernel
P1
Guest1
P2
Container
Management
Tools
Namespace
Set 1
P1
Guest2
P2
Namespace
Set 2 API/ABI
19
![Page 20: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/20.jpg)
Introduction: motivation
Why do we need it? Better performance
multi-tenant environment
Kvm
Host-OS
Emulator-Lay
Guest-OS
App
Container
Host-OS
App
![Page 21: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/21.jpg)
LINUX NAMESPACES
21
![Page 22: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/22.jpg)
Namespaces
Isolating system resources
6 namespaces in Linux Kernel
Mount
UTS
IPC
Net
Pid
User
22
![Page 23: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/23.jpg)
P3 P2 P1
Mount
Namespace2
Mount
Namespace1
Mount Namespace
Own file system
/proc/<p1>/mounts / /dev/sda1 /home /dev/sda2
/proc/<p3>/mounts / /dev/sda3 /boot /dev/sda4
23
/proc/<p2>/mounts / /dev/sda1 /home /dev/sda2
![Page 24: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/24.jpg)
UTS Namespace
UTS = UNIX Timesharing System
Own uts-info
UTS namespace1
ostype: Linux
osrelease: 3.8.6
version: …
hostname: uts1
domainname: uts1
UTS namespace2
ostype: Linux
osrelease: 3.8.6
version: …
hostname: uts2
domainname: uts2
Same
Modified
24
![Page 25: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/25.jpg)
P3 P2 P1 P4
IPC
namespace2
IPC
namespace1
IPC Namespace
IPC: InterProcess Communication
shared memory
Semaphore
message queue
25
![Page 26: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/26.jpg)
Net Namespace 1/2
Net namespace: networking resources
Net Namespace1
Net devices: eth0
IP address: 1.1.1.1/24
Route
Firewall rule
Sockets
Proc
sysfs
…
Net Namespace2
Net devices: eth1
IP address: 2.2.2.2/24
Route
Firewall rule
Sockets
Proc
sysfs
…
26
![Page 27: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/27.jpg)
Net Namespace 2/2
» Separated by the Kernel
» In order to connect two namespaces
» routing
27
![Page 28: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/28.jpg)
PID Namespace
PID: Process ID
Hierarchical system
PID namespace1 (Parent)
(Level 0)
PID Namespace2 (Child)
(Level 1)
PID Namespace3 (Child)
(Level 1)
P2
pid:1
pid:2
P3
P4
ls /proc
1 2 3 4
ls /proc
1
ls /proc
1
pid:4
P1
pid:1
pid:3
pid:1
28
![Page 29: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/29.jpg)
User Namespace
User attributes linked to secure access kuid/kgid: Original uid/gid, Global
uid/gid: user id from „user” namespace mapped to kuid/kgid
Only the parent user NS can setup the mapping
User namespace1
uid:
10-14
uid_map
10 2000 5
kuid:
2000-2004
User namespace2
uid:
0-9
uid_map
0 1000 10
kuid:
1000-1009
29
![Page 30: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/30.jpg)
User Namespace
Create, stat file
User
namespace
root
#touch
/file
Disk /file (kuid:1000)
uid_map:
0 1000 10
root
#stat
/file
File : “/file” Access: uid
(0/root)
30
![Page 31: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/31.jpg)
CGROUPS
31
![Page 32: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/32.jpg)
Linux cgroups
» Limiting the resource usage » Storage (mem)
» Compute (cpu)
» Communication (blkio)
» Devices (dev)
32
![Page 33: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/33.jpg)
LXC
33
![Page 34: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/34.jpg)
System API/ABI
Proc /proc/<pid>/ns/
System Call
clone
unshare
setns
34
![Page 35: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/35.jpg)
Proc
/proc/<pid>/ns/ipc: ipc namespace
/proc/<pid>/ns/mnt: mount namespace
/proc/<pid>/ns/net: net namespace
/proc/<pid>/ns/pid: pid namespace
/proc/<pid>/ns/uts: uts namespace
/proc/<pid>/ns/user: user namespace
If the proc file of two processes are the same, then they belong to the same namespace
35
![Page 36: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/36.jpg)
System calls
clone int clone(int (*fn)(void *), void *child_stack,
int flags, void *arg, …);
6 flag:
CLONE_NEWIPC,CLONE_NEWNET,
CLONE_NEWNS,CLONE_NEWPID,
CLONE_NEWUTS,CLONE_NEWUSER
36
![Page 37: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/37.jpg)
System calls
clone
new process (process2) and IPC in namespace2
Mount1
P1 P2 IPC2
(new created)
Others1
37
IPC1 clone(,, CLONE_NEWIPC,)
Mount1
Others1
![Page 38: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/38.jpg)
unshare int unshare(int flags);
New namespace from „user space”, stepping into a new NS
38
System calls
![Page 39: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/39.jpg)
unshare
Creating net namespace2
39
Mount1
P1 P1 Net2
(new created)
Others1
Net1 unshare(CLONE_NEWNET)
Mount1
Others1
System calls
![Page 40: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/40.jpg)
setns int setns(int fd, int nstype);
Defines the NS the new process will belong to
@fd: file descriptor of namespace(/proc/<pid>/ns/*)
@nstype: type of namespace.
40
System calls
![Page 41: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/41.jpg)
setns
Chaging PID namespace of P2
PID1 P1
P2
PID2
setns(open(/proc/p1/ns/pid,) , 0)
P2
41
PID1 P1
PID2
System calls
![Page 42: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/42.jpg)
Libvirt LXC
Libvirt LXC: userspace container management tool
Implemented as libvirt driver
Container management
Creating NS
Handling private file system within a container
Creating the devices of a container
Resources controlled through cgroup
42
![Page 43: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/43.jpg)
Comparison
Lightweigth virtualization, only one OS
„host share the same kernel with guest”
43
Container KVM
performance Great Normal
OS support Linux Only No Limit
Security Normal Great
Completeness Low Great
![Page 44: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/44.jpg)
Open issues
/proc/meminfo, cpuinfo… Kernel space (cgroup)
User space (low efficiency)
New namespace proposals under discussion Audit (user namespace?)
Syslog (is it required?)
44
![Page 45: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/45.jpg)
Bandwidth TC Qdisc
On host (how to map a container to NICs)
On container (user can modify it)
Netfilter How to handle ingress bandwidth?
Disk quota Uid/Gid Quota (many users)
Project Quota (xfs OK)
45
Open issues
![Page 46: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/46.jpg)
DOCKER
46
![Page 47: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/47.jpg)
What is Docker?
47
• Docker = Linux container engine
• Open Source project
• First release (early beta): 3/2013 by dotCloud
• Later renamed to Docker Inc
• Python code, later refactored in Go
• https://www.docker.io/
• git repository: https://github.com/dotcloud/docker.git
![Page 48: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/48.jpg)
Docker terminology
» Docker image = one file group corresponding to a VM, which contains any extension (lib, db, config, etc.) required to run the planned app
» Container = run-time Docker image instance
» Registry = image repository » By default is local (on-host)
» Docker Inc. Supports a global public on-line repository (similar to github)
48
![Page 49: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/49.jpg)
What is Docker?
49
![Page 50: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/50.jpg)
Docker Engine
» start docker service
» It executes every „docker command”
» Keeps track the locally stored docker images
50
![Page 51: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/51.jpg)
Docker system overview
51
![Page 52: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/52.jpg)
[Source: https://docs.docker.com/terms/layer/]
Docker images
» Composed of layers
» Union file system
» One single image from layers
» Created based on a template
» Dockerfile
» Starting point: base image (e.g. ubuntu, fedora, etc.)
» Own command to add new layers
» Visulaization of different layers of an image: » https://imagelayers.io/
52
![Page 53: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/53.jpg)
Docker Machine
» Handling containers in remote hosts
![Page 54: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/54.jpg)
Docker Machine
» Handling containers in remote hosts » Own cli (docker-machine)
![Page 55: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/55.jpg)
Docker Compose
» Starting multiple services (containers)
» Dockerfile -> application specific
details
» docker-compose.yml
» docker-compose up
55
![Page 56: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/56.jpg)
Docker workflow 1/2
» Single dev environment (local machine or container)
» All services run in containers (eg. DB) » And run the same way
» Testing in „real” deployment conditions » Build in seconds
» Run immediately
56
![Page 57: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/57.jpg)
Docker workflow 2/2
» If local build OK, then » Upload to registry (public/private)
» Automatized run
» In production, enterprise environment, too
» Simple shift between dev and production
» In case of errors: Rollback
» Use an earlier working version
57
![Page 58: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/58.jpg)
a.) Docker images - (run/commit)
» 1) docker run ubuntu bash
» 2) apt-get install this and that
» 3) docker commit <containerid> <imagename>
» 4) docker run <imagename> bash
» 5) git clone git://.../mycode
» 6) pip install -r requirements.txt
» 7) docker commit <containerid> <imagename>
» 8) repeat steps 4-7 as necessary
» 9) docker tag <imagename> <user/image>
» 10) docker push <user/image>
58
![Page 59: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/59.jpg)
a.) Pro/con
» Pro
– Well-known technologies and steps
– roll back/forward – as required
» Con
– Manual process
– Iterative steps „add on”, hard to remember
– Complete re-build prone to errors
59
![Page 60: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/60.jpg)
» RUN apt-get -y update
» RUN apt-get install -y g++
» RUN apt-get install -y erlang-dev erlang-manpages erlang-base-hipe ...
» RUN apt-get install -y libmozjs185-dev libicu-dev libtool ...
» RUN apt-get install -y make wget
» RUN wget http://.../apache-couchdb-1.3.1.tar.gz | tar -C /tmp -zxf-
» RUN cd /tmp/apache-couchdb-* && ./configure && make install
» RUN printf "[httpd]\nport = 8101\nbind_address = 0.0.0.0" >
» /usr/local/etc/couchdb/local.d/docker.ini
EXPOSE 8101
CMD ["/usr/local/bin/couchdb"]
docker build -t author_name/couchdb
60
b.) Docker files
![Page 61: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/61.jpg)
b.) Advantages
» Easy to learn
» Easy re-build » Caching system
» build process described in a single file
61
![Page 62: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/62.jpg)
Docker – why is fast?
62
![Page 63: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/63.jpg)
Docker
» Multi-arch, multi-OS
» Stable control API
» Stable API plugin
» Resiliency
» Signed
» Organized in clusters, scalable
63
![Page 64: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/64.jpg)
Docker vs. VM » Latency: Applications with a low tolerance for latency are going to do better on physical. This
something we see quite a bit in financial services (trading applications are prime example).
» Capacity: VMs made their bones by optimizing system load. If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here.
» Mixed Workloads: Physical servers will run a single instance of an operating system. So, you if you wish to mix Windows and Linux containers on the same host, you’ll need to use virtualization
» Disaster Recovery: Again, like capacity optimizations, one of the great benefits of VMs are advanced capabilities around site recovery and high availability. While these capabilities may exist with physical hosts, the are a wider array of options with virtualization.
» Existing Investments and Automation Frameworks : A lot of the organizations have already built a comprehensive set of tools around things like infrastructure provisioning. Leveraging this existing investment and expertise makes a lot of sense when introducing new elements.
» Multitenancy: Some customers have workloads that can’t share kernels. In this case VMs provide an extra layer of isolation compared to running containers on bare metal.
» Resource Pools / Quotas: Many virtualization solutions have a broad feature set to control how virtual machines use resources. Docker provides the concept of resource constraints, but for bare metal you’re kind of on your own.
» Automation/APIs: Very few people in an organization typically have the ability to provision bare metal from an API. If the goal is automation you’ll want an API, and that will likely rule out bare metal.
» Licensing Costs: Running directly on bare metal can reduce costs as you won’t need to purchase hypervisor licenses. And, of course, you may not even need to pay anything for the OS that hosts your containers.
64
![Page 65: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/65.jpg)
Advantages of Docker
» Easy installation
» Every app, many environments
» Repeatable build
» Great hype, strong community, fast bugfixes
» New virtualization processes
» Con
» Docker container type » Host OS dependent
» „Orchestration”
» Networking
65
![Page 66: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/66.jpg)
Docker cons
» Docker container type » Host OS dependent
» „Orchestration”
» Networking
» But: continous upgrades, developments » E.g., Docker on Windows, Docker Swarm Mode
» Favored by the hype and strong community support
66
![Page 67: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/67.jpg)
Security?
» Docker over REST API / HTTP? » Authentication!
» Docker daemon runs with root priviliges » Containers are then OK (in user space)
» Can they „reach back”?
» docker-1.3 --cap-add, --cap-drop
» man capabilities
» „overview of Linux capabilities”
» „Starting with kernel 2.2”
» „per-thread attribute”
» More developments in the make » Docker daemon
67
![Page 68: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/68.jpg)
Sources
» Docker story in a nutshell: http://www.infoworld.com/article/3025870/paas/the-sun-sets-on-original-docker-paas.html
» Docker overview:
http://www.linuxjournal.com/content/docker-lightweight-linux-containers-consistent-development-and-deployment
» „The Docker Book”
http://www.dockerbook.com/#toc
» Docker Meetup @Budapest http://www.ustream.tv/recorded/60277876
68
![Page 69: Cloud based networks · If your containerized app doesn’t consume all the capacity on a physical box, virtualization still offers a benefit here. » Mixed Workloads: Physical servers](https://reader034.fdocuments.net/reader034/viewer/2022042307/5ed3ace789ea24219c3ce8bd/html5/thumbnails/69.jpg)
Docker state machine
» die, kill ≠ destroy
69