Presentacin de PowerPoint

Docker hosting on FIWARE LabRemote management with local docker client

Kenneth NaginIBMCloud Chapter Lead, FIWAREnagin@il.ibm.com

This tutorial will explain how you can easily setup your Docker environment to develop and deploy your services on the FIWARE Lab.

FIWARE has developed many features with which you can compose your services;

Features related to:- data context management- integration of data and media content- IoT- Apps for data visualization and publication- Advanced Web UI- Software defined networking- Security- and more.

All of the features known as Generic Enablers or GEs for short have packaged as Docker containers.These FIWARE containers can be leveraged to compose your own FIWARE based services.You can develop and deploy your services on the FIWARE Cloud.

The FIWARE Cloud, also known as the FIWARE Lab, has lots to offer.Besides the raw power in terms of cores, memory, storage, and bandwidth. It has 18 regions in Europe. Most recently we have added regions in Mexico and Brazil. All of this can be harnessed to host your Docker containers.

The FIWARE Cloud supports both VM hosting and Docker hosting. So why would you want to use Docker over VMs?Docker containers are much smaller than VMs. They take up a fraction of the space.Most organization could fit all of their required containers on a single Docker host.In the world dominated by VMs, modular design would call for one component per VM.With Docker you can pack the same components referred as Docker micro-services on a single Docker host.It is much easier to manage a single Docker host than multiple VMs.The start-up time for each micro-service is much faster than what would be required for a VM.Likewise, integrating the micro-services into a single application is much easier.Beyond all that the Docker eco-system makes it easier to reuse other Docker containers and tools for managing the docker lifecycle.

5FIWARE Lab Cloud Docker Hosting OverviewDocker Hub: cloud service for managing and sharing Docker container images, including FIWARE services, i.e. Generic Enablers (GEs) (http://catalogue.fiware.org/enablers);Docker Engine: creates and runs Docker containers; pulls/pushes images from/to Docker Hub;Docker Compose: defines and runs multi-container applications;.Docker Swarm: manages a pool of Docker hosts using the full suite of Docker tools. Because Docker Swarm serves the standard Docker API, any tool that already communicates with a Docker daemon, e.g. Docker-Compose, can use Swarm to transparently scale to multiple hosts.Docker Machine: creates and manages Docker hosts locally or on cloud providers (including OpenStack). It can be used to create and manage Docker swam clusters.FIWARE Lab Docker Hosting: Docker Containers, Docker machines, and Docker Swarm clusters can be hosted on the FIWARE lab, but remotely created and managed by local Docker clients;

The primary elements of the Docker Ecosystem are Docker Hub, Docker Engine, Docker Compose, Docker Swarm and Docker Machine.Docker Hub is a cloud service for managing and sharing Docker container images, including FIWARE services, known as Generic Enablers (GE); You can browse for the GEs in the fiware catalogue or search for their docker containers in the Docker Hub.Docker Engine, or simply Docker, creates and runs Docker containers; It supports automatic pulling of docker images rom Docker Hub. It then caches the images locally to speed up subsequent pulls. You can make run time changes to your images and then push them back on to Docker Hub to save them or share them with other developers.Docker Compose allows you to define and run multi-container applications; Most GEs describe how they can be used to compose new applications.Docker Swarm: manages a pool of Docker hosts using the full suite of Docker tools. Because Docker Swarm serves the standard Docker API, any tool that already communicates with a Docker daemon, e.g. Docker-Compose, can use Swarm to transparently scale to multiple hosts.Docker Machine: creates and manages Docker hosts locally or on cloud providers (including OpenStack). It can be used to create and manage Docker swam clusters. Since the FIWARE Cloud is based on OpenStack, docker machine can be used to create and manage docker hosts on the FIWARE cloud.Docker containers, Docker hosts, and Docker Swarm clusters can be hosted on the FIWARE lab.

Simple Docker hosting on FIWARE Remotely Managed by Docker Client

Region

.

Region

.

clientclientdocker-enginedocker-machinedocker-composedocker-swarmdocker api, openstack api, sshany tool basedondocker-api

docker cli

One of the key take-aways of this tutorial is not only can you host and manage docker on FIWARE, but you can do this remotely from your local Docker client;

In this illustration we show a local Docker client remotely controlling Docker hosts from a work station.

The Docker hosts may be spread over multiple FIWARE regions and each region may contain multiple Docker hosts.

Most of the communication between the client and FIWARE is through the Docker REST API.

But Docker Machine also uses the OpenStack API to allocate resources and SSH to provision and configure the Docker hosts.

All of this communication is done securely using a combination of Transport Layer Security (TLS), token based authentication and authorization with Openstack Keystone, and SSH encryption.

Once the Docker host have be created other tools like compose can be used to create and deploy complex Docker services.

Swarm can be used to manage a cluster of Docker hosts running on FIWARE.

The rest of this tutorial will be devoted to showing you how to accomplish all of this.

7Set up stepsSign up fiware account (https://account.lab.fiware.org/sign_up/)(Optionall) Request Community UpgradeThe orgranization automatically associated with your account must be able to allocate at least on floating point IP and access at least one FIWARE region. You can create other organizations.In Cloud service Allocate a floating point IPCreate Security Group that exposes ports for daemon, ssh, and any other ports that are required. Install docker and docker machine on your local computerUse docker-machine to create hosts & swarm clusters on the desired FIWARE regions

As I said earlier, most of FIWARE Docker hosting can be accomplish remotely from your docker client.

But there are a few set up steps that require you to use the FIWARE Cloud Portal.1st you must obtain a FIWARE account.Optionally request a community account which will give you more privileges and a more stable environment.The organization associated with your account must be able to allocate at least one floating point IP and access at least one FIWARE region. You can create other organizations.Once you have an account you must create a security groupNext you must install docker and docker machine on your local work station.We will use docker machine to create docker hosts and swarm clusters on FIWARE regions.

Details on how to accomplish all this follows.

Signing up for a FIWARE account is simple. Just submit the required information. You should be authorized within a day.

Your login account must be authorized to access the Cloud.Notice the name of your default project name is your name followed by cloud. The project name is used to identify you when creating Docker hosts; this will be discussed later.Finally, double click on the cloud tab to prepare your project to create docker hosts.

FIWARE set up: Allocate at least one public ip

On the FIWARE Cloud page double click on the security tab on the left pane.Use the Floating IP tab to an Allocate an IP to the Project.

FIWARE set up: Create Security Group

While in the Security tab create a security group.

FIWARE set up: Create Security Group

Required:Docker Daemon Port 2376SSH: Port 22Optional:Auto allocated user ports: 32768-33768Other User Ports e.g. 8080 Docker Swarm Master Port: 3376

New create rules to open incoming ports:The required ports are 2376 and 22.The Docker Daemon Port listens on 2376. Opening the docker daemon port allows docker clients to communicate with docker remotely.The ssh daemon listens on port 22. Docker Machine uses ssh to provision and configure the docker host. If necessary you can replace this port with another.Optional you can open other ports to be used by the docker containers to interact with the outside world:Ports 32768-33768 are auto allocated by docker when creating containers.Of course you can also open other ports. For instance 8080 for a web service.We also have observed that the Docker Swarm Master uses 3376.

FIWARE set up: List images

Using the image tab you can list the available images on the FIWARE region.We will use the Ubuntu Server 14.04.1 (x64) image for the tutorial.But the base_ubuntu_14.04 also works.

14FIWARE Setup: Get image ID

You can get more details about the image by selecting it.Notice the image id.You can either specify the image name or image id to tell docker machine which image to use.

We are now done with the FIWARE setup phase of the tutorial.

We will revisit the Cloud portal for illustration purposes, but once the setup is complete you cando all of you docker management remotely without the GUI.

15docker-machine create docker host on fiwareexport OS_REGION_NAME=export OS_REGION_NAME=Spain2>docker-machine create -d openstack --openstack-flavor-id="2" --openstack-image-id=$imageID --openstack-net-name="node-int-net-01" --openstack-floatingip-pool="public-ext-net-01" --openstack-sec-groups="docker-machine-sg" --swarm --swarm-master --swarm-discovery token://$TOKEN FIWARE-SPAIN2

We then create the swarm master.We export the OS_REGION_NAME to indicate which region to create the master.There is no change to the other environment variable used previously.The docker-machine create command requires three new flags:--swarm--swarm-master--swarm-discovery discovery requires the token as input

29Docker Swarm: create slave>export OS_REGION_NAME=Crete>docker-machine create -d openstack --openstack-flavor-id="2" --openstack-image-id=$imageID --openstack-net-name="node-int-net-01" --openstack-floatingip-pool="public-ext-net-01" --openstack-sec-groups="docker-machine-sg" --swarm --swarm-discovery token://$TOKEN FIWARE-Crete

We next create a swarm slave on the Create region.The swarm-master flag is not included indicating its role as a slave.We create another slave in the Budapest2 region, but it is not shown.

docker swarm docker-machine ls

The docker-machine ls command is used to list the cluster.Notice that FIWARE-Spain2 is the master and the other docker hosts are slaves.

>eval $(docker-machine env --swarm FIWARE-SPAIN2) >docker info

We use the eval command to have docker machine alter the docker environment so that docker will work remotely with the swarm master running of Spain2.The docker info commands shows some details about the swarm cluster/

>eval $(docker-machine env --swarm FIWARE-SPAIN2) >docker run hello-world>docker ps -a

This slide shows how running docker hello-world multiple times gets spread over the cluster.

>eval $(docker-machine env --swarm FIWARE-SPAIN2) >docker-compose up -d>docker ps

>curl http://147.27.60.99:32768/employees/resources/employees/

Finally in this slide we show how docker-compose also can used to launch multi-container services on the cluster,

>eval $(docker-machine env --swarm FIWARE-SPAIN2) >docker-compose stop>docker-compose ps

Docker-compose also can stop the multi-container services running on the cluster.

Simple Docker hosting on FIWARE Remotely Managed by Docker Client

Region

.

Region

.

clientclientdocker-enginedocker-machinedocker-composedocker-swarmdocker api, openstack api, sshany tool basedondocker-api

docker cli

In summary, this tutorial has shown how you can host and remotely manage docker on FIWARE.

You can compose new services based on the FIWARE GEs and deploy them on the FIWARE cloud.