Stonefly is the innovator of worldwide storage standard protocol

DOCKER EXPLAINEDDocker Basics & Fundamentals

By Ahmed Murkaz

Stonefly Inc.26250 Eden Landing Hayward, California, USA

Docker Explained

Absract

This course will cover the basics and fundamentals of Docker and Docker infrastruc-

ture, how to manage your Docker containers and images; you will learn the advanced

usage of Docker and the Docker containers and your Docker infrastructure, you will

learn how to run your containers in an enterprise environment with external storage

providers and with advanced networking infrastructure. You will learn how to run

your Docker containers on a public cloud just like Amazon or Google cloud engine.

The difference between traditional virtualization and containers will be covered,

what's the difference between full containers or OS containers and application con-

tainers. You will learn the basics of a Docker image, and how a Docker image looks

like, and you will learn the basics of a Docker infrastructure and what parts it com-

prises.

1

Docker Explained

What is tradi�onal virtualiza�on and how it works?

In traditional virtualization you use your general infrastructure, your servers, storages,

switches and so on, and you put an “Operating System” (OS) like Windows or Linux

onto your servers and after that you put a so-called “Hypervisor” on top of the OS. In

case of Linux you will install the KVM, and in case of Windows or Windows Core

Server you will install the Hyper-V feature onto that. So Hyper-V and KVM are hypervi-

sors, but there are several specialized OSs which consist the “Host Operating System”

and the “Hypervisor” in one (Just like the VMware ESX or XenServer which provides

you with an OS and a hypervisor itself in one installable package).

2

After you install your “Host Operating System” and the “hypervisor”, you can create

Virtual Machines (VMs) and the hypervisor will emulate the hardware to these VMs. It

will emulate the Motherboard, CPU, memory, disk, and Network Interface Card (NIC),

and you will install the “Guest OS” into these VMs. The guest OS installation is simple,

Docker Explained

there are no special requirements because the hardware is emulated, however, it

comes with a price and performance penalty as everything is emulated. So when you

write data onto the disk in the VM, first you have to write it out in the VM disk, but the

VM disk is a real file on the host OS, so you have to write out that data onto the real

disk into a real file. There is at least double write or cures and the same happens with

the NIC and so on.

We have specialized drivers called para-virtualized drivers and when you install it into

the “Guest OS” it boosts the performance as it evades the performance penalty by

talking directly to the disk or the file on the “Host Operating System”. It helps you a

lot but still it's just like a real server, so you will have to administer the server's VM just

like a normal server. You have to patch all the applications of the OS.

Traditional virtualization has the advantage that you can install different OSs on top

each other. Let's say you have an Ubuntu host OS with a KVM Hypervisor, you can

install a windows server or a Linux server or a FreeBSD server on top of that. It doesn't

matter because there is a separate kernel for all the VMs.

3

Docker Explained

In the first place traditional virtualization is used to consolidate the hardware and the

infrastructure, so we have a better usage of the infrastructure, because you can put

more OSs, more machines on the real servers, since in real life 80% or 90% of all the

time the OS is just idle, or just uses a 10% or 20% of the hardware resources. If you

put more machines, more servers into one real server you have better average con-

sumption, you will spend less money by putting more servers on one physical ma-

chine.

One physical machine consumes less electricity than three different servers. This is

the main reason behind the traditional virtualization; better usage of the infrastruc-

ture. On the second thought you can just think about different software versions and

different libraries which are not compatible with each other.

So let's say you have four applications and you want to put them onto one server, but

they are requiring different versions of a software which are not compatible with

each other. Consider that you want to use MySQL 6 and you want to use MySQL 5,

PHP 7 and PHP 5.6 and they are not compatible with each other. But your applica-

tions maybe they just require different versions and you can just manage it but it's

very hard, because you can install them into different directories, work with different

environment variables, but it will be very hard to maintain patch and update all

these things.

On the other hand you use traditional virtualization to isolate the processes and to

give the different applications different virtual machines. That's much easier to

manage, i.e. different virtual machines with applications than one real server with five

different applications. We have lots of tools to orchestrate and schedule these virtual

machines. You can just install them with the help of a “Chef” or “Puppet” or anything

like that so you can automate all these processes. Yet still you have performance and

management overhead as well because you have to patch all the OSs and it con-

sumes more hard disk, more storage and so on.

Why use tradi�onal virtualiza�on?

4

Docker Explained

So we have another virtualization method called containers. The first occurrence of

the containers happened just like Full containers of OS containers, and they are

almost the same as the traditional virtualization because you use them to isolate and

separate the different roles, different customers or different applications and you

install whole OSs into the containers. You have a full boot sequence, there's an SSHD

or a remote desktop. If it is a windows you have a MySQL server and an Apache server

in one container, and you just put another version of a particular server and PHP and

MySQL going to another container and they're with their full boot sequence, and you

can use the same upgrade methods just like with the traditional virtualization. You

can use almost the same orchestration tools, you can use “Chef” and “Puppet” to

deploy these containers onto the “Host Operating System”.

There's a very big difference between traditional virtualization and a container virtual-

ization. This difference is that you cannot use different OS on top of each other. You

cannot use a Linux on top of a windows server, and you cannot install a windows

server on top of an “Ubuntu” host OS

Containers

5

Docker Explained 6

Docker Explained

With a container virtualization you have the real servers and the host OS, and you

have a container management service just like V-server, Alex-C and so on. There are

lots of different flavors of this container management service but one thing is coming

in all of them that they isolate different processes. So when you install guest OS you

just install it into a directory of the host OS, and when you boot the container it just

starts the processes and the processes are isolated from each other. So if you have

this green guest OS and this orange guest OS, they are just processes and there's no

hardware emulation.Despite the obvious benefits of cloud storage, many organiza-

tions remain wary of moving to the cloud. Traditional storage may be costly,

resource-intensive and cumbersome, but it also performs well, is secure, highly avail-

able, and just works.

7

Docker Explained

So in traditional virtualization because the hardware, i.e. the motherboard, CPU and

everything is emulated, you can install different Kernels into these virtual machines,

but with containers you use the same Kernel on the host OS. So the Kernel is one you

cannot use different kernels in guest OSs other than the host OS because a container

is just a process and the processes in the container are just processes on the host OS,

they are just separated from each other and separated from the host OS processes as

well, and you can put limits on them. So you can limit the memory and CPU usage of

a container, but you can only run Linux container on top of Linux OS, and you can run

only windows containers on a windows server. But it has a very big advantage that is

it’s very fast. It is nearly as fast as a real server because there is no emulation.

It's much faster than a traditional virtualization. If you run Linux on Linux, it doesn't

mean that if your host OS is an “Ubuntu” that you cannot run CentOS Linux as a guest

OS, only the Kernel will be the same but all the libraries, server versions and so on will

be the guest OSs’ version.

8

Docker Explained 9

So you can run CentOS, SUSE or a different version of “Ubuntu” on an “Ubuntu” host

OS but still it looks like a real server with all the boot sequences and all the softwares.

You can manage and login to the guest OSs as we have discussed before.

The very big difference between an application container, an Operating System (OS)

container and the traditional virtualization is that you won’t ship full OSs in an appli-

cation container. We have several application container engines, but Docker is the

most famous and the most mature.

Full Containers vs. Applica�on Containers

So you still use your servers and you put an OS on top of that, let's say you will install

an “Ubuntu” operating system, but you will install the Docker engine on top of that

What is an applica�on container?

Docker Explained 10

and we are not ship and deploy for guest OSs with full boot sequences, SSH, Daemon

or mail server Daemon or an apache web server in one package or in one directory,

you just put your application and all the required libraries and software in one pack-

age. So if you have a WordPress installed in an OS container, you would install an SSH

daemon, an apache web server, a PHP library to MySQL and mail server on top of

that guest OS.

With an application container you use different containers and let's say again that

WordPress example works like the following; that you will put your WordPress and

your custom theme into the green application container, and we will put the MySQL

server into another container let's say the orange one.

Then you link them together, so you will only pack your WordPress and WordPress

team into one container just like the apache web server and the PHP FISE, and you

will use standard MySQL image in another container.

Docker Explained

Why is it good for you?

If you are a developer you can just pack your application into one standardized

format and you can just deploy your station anywhere Docker is supported. Let's

say you have your custom software package into the red application container

and it doesn't really matter whether you deploy it onto Amazon, Microsoft Azure,

Google Cloud Engine (GCE) or your own laptop, it will work the same way because

all the required dependencies, binaries, and libraries are in one package.

11

You can just compare an application container to a portable executable container,

that if the runtime environment supports your application, you can run it just like

portable executable versions of Mozilla Firefox. You can run it on any windows version;

it’s the same.

If you have a Docker container and this Docker container consists of Linux binaries

and Linux applications, you can run it on any server that supports the Docker Engine

and the operating system is Linux and it works in the same way with windows server

as well.

Docker Explained 12

From a developer point of view it’s just very easy to pack your applications into one

package and run it anywhere, and from the infrastructure guys’ point of view if you

want an infrastructure engineer you don't have to work with dependency house, and

you don't have to provide several scripts just to maintain state of the servers.

You only have to provide the Docker Engine and the basic infrastructure and devel-

opers will take care of everything else. As the application container just like a Docker

container is very lightweight, you can run more applications on one OS on one server

than with OS containers over the traditional virtualization. A traditional virtualization

just depends on your hardware, but sometimes you can just run 10-20 virtual ma-

chines (VMs) on one server, however, with a Docker application container you can run

just like 100 containers; It just depends on how much memory it uses and how much

memory do you have, but you have the possibility, and this application container

format is very good at running the so-called Micro-Services.

So if you have one web application and you need to deploy like 50 more from that

web application you just deploy 50 containers and you don't have to wait for an OS

installation or use it from a template, because you just deploy your web application

and Docker Engine will take care of all the other things. Another big advantage

beside the standard format and the lower resource usage, is that it's much easier to

deploy new versions, because let's say App 1 and App 2 are just two different versions

but from the same software. When you want to deploy a newer version from your

software you just deploy the App 2 container and just stop the App 1 and if anything

goes wrong it's very easy to downgrade or roll back the software installation because

you just start up the App 1 container and just shut down the App 2 container. And

when you deploy you don't have to wait for infrastructure guys, and as an infrastruc-

ture guy you won't have extra work just to make sure that the newer version of a

software can be rolled out. These are the biggest advantages of a Docker environ-

ment.

Docker Explained 13

Basics of a Docker image

Docker Explained 14

The Docker environment works with Docker images, so the containers are the run-

ning versions of an image and an image is executable and it has several layers. When

you start a container you will start up an image and make it run and let's say in this

example you have an Apache container and you have that Apache image but

“Apache” needs an OS as well to run all the libraries, so it doesn't require a full OS

because the host OS provides all of the memory management and so on, so you only

have to have the binaries required to run the Apache. You put your Apache into one

image and maybe it has dependencies so it has references to another parent image

as shown in this example from Docker.com.

It will just reference to that other image and it just grabs that image as well, and that

image references to the Debian “Base Image” and the Debian “Base Image” is not

referenced to anything because it's a base image. When you want to run the Apache

container you grab the Apache image and the image will point you to other images

and at the end Docker will put all the layers on top of each other.

It will just grab the “Base Image”, put the “emacs image” on top of that and the

“Apache image” on top of that, and finally it will create a “read/writeable” layer

Docker Explained 15

on top of that so any change will be written into that “read/writeable” layer because

the images are immutable. This means you can use the same image for several con-

tainers. So if you want to run 100 Apache containers, you use the same image and

they will be only different with the “read/writeable” layer, because every image will

have a different “read/writeable” layer. But this layer is ephemeral. This means when

you remove the container that “writeable” layer will be lost. If you delete all the

Apache binaries you just remove the container and start a new container from the

same image, and because the images is immutable all the changes are lost, because

they leave only interactive container, nothing is lost.

Everything is just happening in the “writeable” layer but not in the “Base Image”, and

because you can use the same image for several containers, it consumes less storage

space because in a full container or in a traditional virtualization if you have an image

with one gigabyte in size and you start or deploy 100 containers it would consume

100 gigabytes. But in a Docker infrastructure if you have an image with one gigabyte

in size and you deploy 100 containers you still use the same image because only the

“read/writeable” layer is different. It will only consume one gigabyte disk space and

the “read/writeable” layer’s disk space.

All you have to know is that Docker containers are coming from images and these

images are immutable and the “Containers” are ephemeral or the “read/writeable”

layer of the containers are ephemeral.

So if you remove a container everything will be lost which has been written into this

“read/writeable” layer.

You will see in later that there are persistent data stores, what you can use with con-

tainers. So if you have a real application you won’t lose your data if you just remove a

container. So don't be scared when you hear that the “read/writeable” layer is ephem-

eral and the images are immutable because you can store your changes.

Docker Explained 16

Docker infrastructure

Docker Explained 17

At the end let's see how a Docker infrastructure looks like. This is a very simple Docker

infrastructure because you have your Docker host let's say that's an “Ubuntu” or

CentOS Linux and you have the Docker Daemon on top of that Docker host.

When you want to start up a container right now you know that Docker uses images

to create containers. So the Docker daemon will look for the images locally, and if it

doesn't find the images it will pull down from the so-called registry, so you store your

images on a registry and these images will be downloaded when necessary and will

be stored on a Docker host to create containers from these images.

So the Docker client just issues comment to Docker demon and the Docker demon

will behave just instructed. These are the basics of the infrastructure. The image for

the container can be pulled from a public registry or a private registry. It just depends

on the configuration, and it will just start up a container.

Later you can remove the locally stored images, and you can upgrade the images

from the registry and so on.