Webinar: Enabling Microservices with Containers, Orchestration, and MongoDB

Containers and Orchestration - Enabling

Microservices

Andrew Morgan ([email protected])

4th May 2016

mailto:[email protected]

Agenda

1. What are containers?

2. Containers vs. VMs

3. Business benefits

4. Docker – The most popular container technology

5. Microservices

6. Orchestration

7. Security

8. MongoDB considerations

9. Implementing a MongoDB Replica Set

10.Use Cases

11.Resources

Containers

Real world shipping containers

• Same container efficiently transports goods by

road, rail and sea

• Contents remain untouched through all those

environments; no repacking needed

• Ubiquitous and standardized

• Simple to use – open, fill, close

• Contents of each container safe from the others

• Know how much space the container consumes

ContainersSoftware containers

• Build an image containing the full application stack

only once

• Spin up many containers from the same image in

multiple environments

– Laptop, data center, cloud

– Development, QA, production, support

• Simple to use & efficient

• Contents of each container isolated from the

others

– Storage, memory, CPU, namespace

• Constrain resources available to each container

Sounds like a Virtual Machine?

• Both technologies allow you to create an image and then spin up multiple, isolated

instances

• But

– Each VM contains full operating system, libraries… With containers these can be

shared

• Faster, less memory, less storage, spin up in seconds

• Run many container on a single host

– Enables microservices

• Each container has a narrow, specialized focus with well defined interfaces with

other containers

– Developer-focused tools and management APIs

• Integrate with automation systems such as Chef and Puppet

• Used by DevOps to cover entire software lifecycle

VM VMVM

VMs Containers

Bare Metal

Host Operating System

Hypervisor

Guest OS

Libraries

Apps

Service

Guest OS

Libraries

Apps

Service

Guest OS

Libraries

Apps

Service

Container ContainerContainer

Bare Metal

Host Operating System

Docker Engine

Libraries

Libraries

Apps

Libraries

Apps

Service ServiceService

Containers – Business Benefits

• DevOps & Continuous Delivery– Low impact & risk; update one container at a time

• Replicating environments– Instantiate clones for development, QA, production, support…

• Accurate Testing– Confident your stack exactly matches what’s in production

• Scalability– Add and remove containers based on demand

• Isolation– Safely run multiple environments on the same hosts

• Performance– Minimal impact from container overhead

• High Availability– Redundancy from multiple containers fulfilling a role

DockerThe most popular container technology

• Simple to use and has a rich ecosystem

• 100,000+ images available from Docker Hub

– Including mongo hub.docker.com/_/mongo/

– Syncs with GitHub projects

• Define new images built upon base images

• Define interfaces between containers

• LINUX only (on OS X & Windows, transparently runs in

a VM)

• Runs on bare metal, VMs and cloud. Cloud providers

supply the Docker infrastructure (e.g. Google Container

Engine)

docker run -d mongo

https://hub.docker.com/_/mongo/

MicroservicesMicroservices built by combining multiple

containers

• Build sophisticated services from many small,

focused processes (containers)

– Well defined APIs between components

– Each component can use different libraries,

middleware & programming languages

• Modular, decoupled architecture simplifies

maintenance and enables reuse

• Fault tolerant

• Scalable

Cmglee

https://commons.wikimedia.org/wiki/User:Cmglee

Orchestration

Automated deployment, connecting and

maintenance of multiple containers

• Provision hosts

• Instantiate containers

• Reschedule failed containers

• Link containers through defined interfaces

• Expose services to the outside world

• Scale out and back in

Orchestration Technologies

• Docker Machine

– Provisions hosts and installs Docker Engine

– Used to run Docker on Windows and OS X

• Docker Swarm

– Provides single, virtual Docker host by clustering multiple Docker hosts

– Often uses Docker Machine

• Docker Compose

– Deploys multi-container applications; including dependencies

– Targets development, testing, and staging environments

Orchestration – Kubernetes

Created by Google, feature-rich and widely

adopted

• Automated container deployment and ‘replication’

• On-line scale out/in

• Rolling upgrades

• HA – automatic rescheduling of failed containers

• Exposure of network ports to external apps

• Load balancing over groups of containers

providing a service

• Provided as a service by Google Compute Engine

Orchestration – KubernetesTerms

• Cluster: collection of nodes (bare-metal servers or

VMs)

• Pod: group of containers and volumes collocated in the

same host. Containers share namespace & IP address

• Volume: map ephemeral directories within container to

external, persistent storage

• Service: Load balancer and ambassador for containers

– exposes port and external IP address

• Labels: used to tag resources; labels are then

referenced from other resources using selectors

• Replication Controller: ensures the requested number

of containers are always running

Orchestration – Apache MesosDesigned to scale to 10,000s of physical

servers; used by Twitter, Airbnb & Apple

• Developer writes code to turn application into a

framework to run on Mesos

• Less feature rich than Kubernetes; considers many

functions such as load balancing, rescheduling,

and scaling to be a higher level function

– Project exists to run Kubernetes as a Mesos

framework

• Foundation for distributed systems

– Apache Aurora, Chronos, Marathon

Choosing an Orchestration Framework

Factors to consider…

• Integration with existing DevOps frameworks?

• Number of hosts?

• Bare metal, VMs or cloud deployment?

• Automated High Availability?

• Grouping and load balancing?

• Existing skills?

• Install your own orchestration framework or use as

a service?

SecurityContainers provide opportunities to improve

security

• Containers provide isolation; resources can only

be accessed from outside through explicitly

provided APIs

• Resources can be rationed

• A container’s role can be very narrow – remove

anything not required

• Images and containers should be kept current;

rolling upgrades with Kubernetes or Aurora

• Typically log into container as root so restrict

access

MongoDBOrchestrating MongoDB containers requires

special treatment as it’s a distributed,

stateful application…

• State should survive rescheduling; use

Kubernetes’ volumes abstraction

• Replica Set members must communicate with

each other; expose external IP addresses/ports

which survive rescheduling

• Replica Set must be initialized from exactly one

member

• MongoDB must still be monitored and backed up –

MongoDB Cloud Manager

Volumename:

mongo-persistent-storage1

pdName: mongodb-disk1

mongodb

-disk1

Containername: mongo-node1

image: mongo

command: mongod –replSet my_replica_set

containerPort: 27017

volumeMounts:

name: mongo-persistent-storage1

mountPath: /data/db

Docker Hub Registry

Podlabels: [name: mongo-node1; instance: rod]

ReplicationControllername: mongo-rc1

labels: [name: mongo-rc]

replicas: 1

selector: [name: mongo-node1]

Service: LoadBalancername: mongo-svc-a

labels: [name: mongo-svc-a]

ports: [port: 27017, targetPort: 27017]]

selector: [name: mongo-node, instance: rod]Exte

rnal

IP

Ad

dre

ss104.1.1.1

App

104.1.1.1:27017

Volumename:

mongo-persistent-storage2

pdName: mongodb-disk2

mongodb

-disk2

Containername: mongo-node2

image: mongo



volumeMounts:

name: mongo-persistent-storage2

mountPath: /data/db

Docker Hub Registry

Podlabels: [name: mongo-node2; instance: jane]

ReplicationControllername: mongo-rc2

labels: [name: mongo-rc]

replicas: 1

selector: [name: mongo-node2]

Service: LoadBalancername: mongo-svc-b

labels: [name: mongo-svc-b]


selector: [name: mongo-node, instance: jane]Exte

rnal

IP

Ad

dre

ss104.1.4.5

App

104.1.4.5:27017

ReplicationControlername: mongo-rc-europe

labels: [name: mongo-europe]

replicas: 1

selector: [name: mongo-node]

Volumename:

mongo-persistent-storage

pdName: mongodb-disk-europe

mongodb-

disk-europe

Containername: mongo-node

image: mongo



volumeMounts:

name: mongo-persistent-storage

mountPath: /data/db

Podlabels: [name: mongo-node]

Service:LoadBalancername: mongo-svc-europe

labels: [name: mongo-svc-europe]



ReplicationControlername: mongo-rc-asia

labels: [name: mongo-asia]

replicas: 1


Volumename:


pdName: mongodb-disk-asia

mongodb-

disk-asia


image: mongo



volumeMounts:


mountPath: /data/db


Service:LoadBalancername: mongo-svc-asia

labels: [name: mongo-svc-asia]



ReplicationControlername: mongo-rc-us

labels: [name: mongo-us]

replicas: 1


Volumename:


pdName: mongodb-disk-us

mongodb-

disk-us


image: mongo



volumeMounts:


mountPath: /data/db


Service:LoadBalancername: mongo-svc-us

labels: [name: mongo-svc-us]



Square Enix: DaaS

• Multi-tenant OnLine Suite

• DaaS to studios & developers, exposed as an API

• Manages data shared by all gaming titles

– Player profiles

– Credits

– Leaderboards

– Competitions

– Catalog

– Cross-platform messaging

API Layer

App Layer

MongoDB Shared Data Service

On-Prem Private Cloud

Square Enix: Public Cloud

API Layer

App Layer



• In-App functionality provisioned to private clusters on AWS

– Game state

– Player metrics

– Game-specific content & features

• Elastically scalable

Square Enix:

MongoDB Cloud Manager

API Layer

App Layer



Run entire estate

with just 2 x admins

Resources

• Case Study – FuboTV

https://www.mongodb.com/blog/post/leaf-in-the-wild-leading-

soccer-streaming-service-fubotv-scales-its-business-with-

mongodb-docker-containers-and-kubernetes

• Case Study – Square Enix

https://www.mongodb.com/blog/post/leaf-in-the-wild-square-

enix-scales-tomb-raider-hitman-absolution-deus-ex-and-

more-on-mongodb

• “Enabling Microservices – Containers &

Orchestration Explained” white paper

https://www.mongodb.com/collateral/microservices-

containers-and-orchestration-explained

https://www.mongodb.com/blog/post/leaf-in-the-wild-leading-soccer-streaming-service-fubotv-scales-its-business-with-mongodb-docker-containers-and-kubernetes

https://www.mongodb.com/blog/post/leaf-in-the-wild-square-enix-scales-tomb-raider-hitman-absolution-deus-ex-and-more-on-mongodb

https://www.mongodb.com/collateral/microservices-containers-and-orchestration-explained

Webinar: Enabling Microservices with Containers, Orchestration, and MongoDB

Technology

Transcript of Webinar: Enabling Microservices with Containers, Orchestration, and MongoDB