CoreOS Battle Stories

CoreOS Battle StoriesJari Kolehmainen, Founder

Background

© 2015 Kontena, Inc.

What is Kontena?

Open Source container platform built to maximize developer happiness. Works on

any cloud, easy to setup, simple to use.

How does it work?

Kontena Grid A number of physical or virtual machines – Kontena Nodes – create a Kontena Grid. The nodes may be located anywhere; in single data center, different AZs or different cloud providers.

Overlay Network Kontena will automatically create an overlay network powered by Weave and connect all nodes of a Grid. Overlay network enable services to communicate with each other in multi-host, multi-AZ environment.

Service Discovery Kontena has a built-in service discovery powered by etcd. It is used to automatically assign DNS addresses for any services running in Kontena. It is also used by Kontena’s load balancer for zero-downtime operation.

Orchestration Kontena’s orchestrator is distributing, running and monitoring all Kontena Services in a Grid. Services may be stateless or stateful, and they are automatically distributed across Nodes in a Grid.

Containerized Workloads With Kontena, all containerized workloads are described as Services. Kontena Service is composed of containers based on the same image file. Services may be scaled and linked together to create complex elastic apps.

OS

Docker

Kontena Nodes & Agent Kontena Agent may be installed to any machine capable of running Docker. It

is running as a privileged container in a machine.

Kontena Master Kontena Master is orchestrating the

entire Kontena system. It provides APIs used by Kontena CLI, Web UI and third

party integrations.

Kontena Master may be installed as high-availability setup if needed.

What about OS?


Perfect OS: Requirements

• Minimal footprint • Container native • Zero maintenance • Stable • Secure


Why Container OS?

• Our expectation from an OS has changed • “Pets vs Cattle”

• Maintaining the system should be easy • It needs to be more secure than traditional OS

Finding the Right OS


Choices (Back Then)

• Boot2Docker • CoreOS • Project Atomic • DIY (not an option, really)


Boot2Docker

• Based on Tiny Core Linux • Small (24MB download, 5s to boot) • No automatic updates • Not recommended for production use


CoreOS

• Based on Gentoo • Minimal (~100MB) • Designed for containers • Focus on security and stability • Automatic updates


Project Atomic

• Not a new Linux distribution • Framework to create on OS from RHEL, CentOS

and Fedora • Designed for containers • Focus on security and stability

CoreOS

Is not just an OS…


CoreOS Project

• etcd • rkt • fleet • locksmith • flannel • many more…


CoreOS Host

Automation


Kontena Provisioning Goals

• single command that “just works” • register host to etcd cluster • register host to Kontena Grid • should work on any infrastructure


CoreOS Problems

• configuration management • etcd cluster / discovery • etcd security • coordinated auto-updates


CoreOS & Configuration Management

• Chef • Puppet • Ansible • Saltstack • ??

CloudInit


Bootstrapping with CloudInit

• de-facto way to initialize cloud instances • integrated to CoreOS • only sane way to bootstrap


Etcd: The Hard Parts

• discovery • security (tls certificates) • central services vs workers • maintenance


Initial Implementation

• run etcd inside a container • bind etcd only to localhost & overlay network • use public discovery service


Etcd: Current Implementation

• run etcd inside a container • bind etcd only to localhost & overlay network • master coordicates etcd discovery

• static ip’s


Etcd: Future Improvements

• automatic failover with magic • support for external etcd cluster

• compose.io

http://compose.io

Automatic Updates


Automatic Updates

• several update strategies • best-effort, etcd-lock, reboot, off

• our pick: best-effort • if etcd is running, locksmith coordinates the

reboots • otherwise just reboot once update is available


Automatic Updates

• chaos monkey for free! • updates also kontena-agent • works like a charm

Overlay Network


Overlay Network Options

• Flannel by CoreOS • Weave Net by Weaveworks • Calico • Docker Overlay Network • Most likely you need one of these


Flannel

• bundled with CoreOS • depends on working etcd • has multiple backends to choose from


Weave Net

• simple setup • optional encryption • multicast, multi-hop, fast datapath • dns • requires “some” external coordination

• needs information about other peers


Our Pick: Weave Net

• can start before etcd • makes it possible to expose etcd only to overlay net • secure communication between nodes

• not dependant on infrastructure features • easy-ish to orchestrate • “just works”

Summary


CoreOS Summary

• the “OS” part is currently best option for containers • etcd is a must, but a little hard to handle • pick orchestrator that hides all the complexities • automate everything

Thank You!www.kontena.io

CoreOS Battle Stories

Technology

Transcript of CoreOS Battle Stories