VM-Cloud management softwarePablo Moreno

EBI-EMBASSYOpenStack

Internal network:internal ports 80, 8080 and 22 allowedproxy

wordpress jenkins

w4mwordpress

volume

jenkins volume

floating ip: ports 80 & 22 (EBI only) allowed

Architechture

VM OS

cont

. too

l 1

cont

. too

l 2

cont

. too

l N

...

OS orchestration

inst

alle

d to

ols

Cloud framework

Create/download image

Set hardware

Build for target (i.e. amazon)

Orchestrate provisions

Run in target

Vagrant packer cloud-init ansible docker

Orchestrate VMs

kubernetes

Orchestrate Services

Timeline

PackingOS Download Deployment Running

● packer○ ansible○ cloud-init○ shell

● cloud-init● shell

● ansible● kubernetes

(on amazon, openstack, vagrant, vm-ware, etc.)

single_image_file.vmdk

Vagrant vs packerVagrant

● Normally a single target● Few targets available● Aim: run the VMs locally● Best for development phase of

provisioning● Prepackaged base boxes.

Packer

● Multiple target (“builders”).● Plenty of targets available● Aim: produce VM-images to deploy● Once provisioning development done, use

to deploy in production.● Can create its own base boxes.

Pac

ker e

xam

ple

image_for_qemu.img

image_for_vbox.img

image_for_aws.ami

docker_export.tar

Ansible vs cloud-initAnsible

1. Declarative syntax for lots of modules, plus shell commands

2. Requires ssh access3. Agent-less (but requires python!)4. Configuration by roles

Cloud-init

1. Declarative syntax for a few modules, otherwise run shell commands

2. Console access, injected on the appliance3. Requires cloud-init installed.4. No roles

$ docker run <image> <argument_to_entrypoint>

CoreOS● Minimal installation● Automatic updates.● Cluster awareness.● Read-only root file system.

Minimal OS difficulties

Welcome to CoreOS 845!$ which apt-get$ which python$ which myFavouriteTool$ which docker/usr/sbin/docker

docker best practices : directory

docker best practices: do you really need them all?

docker best practices: one container, one process

FROM ubuntuENTRYPOINT [“top”,“-b”]CMD [“-c”]

$ docker run -it -rm top_container -H

docker best practices: more● # of layers balance● how to group commands● linking deprecated -- use networking● etc.

http://docs.docker.com/engine/articles/dockerfile_best-practices/

https://github.com/BioDocker/biodocker/wiki/Best-Practices

Kubernetes● Container cluster orchestration

Data flow problem?

Kubernetes : services abstraction

http://kubernetes.io/v1.0/docs/user-guide/services.html

Stateless vs Stateful

chromatograms aligned chromatograms

new pipeline galaxy + new pipeline

Our architecture?

VM: Ubuntu LTS

cont

. too

l 1

cont

. too

l 2

cont

. too

l N

...

OS orchestration

inst

alle

d to

ols

Orchestration? At what level?container deployment?communication between nodes?VM deployment to multiple targets?

VM: Minimal OS

cont

. too

l 1

cont

. too

l 2

cont

. too

l N

...

Cloud framework Cloud framework