Weave User Group Talk - DockerCon 2017 Recap
106
Patrick Chanezon, @chanezon Docker for Devs and Ops What’s new and What’s next May 2017
-
Upload
patrick-chanezon -
Category
Technology
-
view
2.179 -
download
0
Transcript of Weave User Group Talk - DockerCon 2017 Recap
Patrick Chanezon, @chanezon
Docker for Devs and Ops
What’s new and What’s next
May 2017
French
Polyglot
Platforms
Software Plumber
San Francisco
Developer Relations
@chanezon
Docker
The world needs
tools of mass innovation
A programmable Internet would be the ultimate
tool of mass innovation
A commercial product,
built on
a development platform,
built on
infrastructure,
built on
standards.
Docker is building a stack to program the Internet
Docker is building a stack to program the Internet
CE
EE
Docker for Developers
The best tools…
1. Get out of the way
The best tools…
1. Get out of the way
2. Adapt to you
The best tools…
1. Get out of the way
2. Adapt to you3. Make the powerful
simple
The best tools…
enterprise edition
Ubuntu
Fedora
Mac
Azure
CentOS
Windows 10
AWS
Debian
community edition
Ubuntu
Windows Server
Azure
CentOS
Suse
Red Hat
AWS
Oracle Linux
Bettertools for developers
Docker removes frictionin the development cycle
How to remove developer friction in 3 easy steps
Step 1.
Developer
complains
about detail
Step 2.
Fix Detail
Step 3.
Repeat
FOREVER
In the developer experience, details add up...
Example #1
My container images are too big!“My container images are too big!”
Introducingmulti-stage builds
Example #1
“My container images are too big!”
Build smaller images withmulti-stage builds
First stage: complete build environment
Second stage: minimal runtime environment
One Dockerfile, one build
FROM big-buildbase……
FROM tiny-runbase
…COPY --from=0/artifact /run/app …
One Dockerfile,one build
Stage 0: large build environment
Stage 1: minimal run environment
Copy artifacts from one stage to the nextOnly copy what you need!
Build smaller images withmulti-stage builds
MAC AWS
“I wish it was easier to
take my app from desktop
to cloud”
Example #2
IntroducingDESKTOP-TO-CLOUD
“I wish it was easier to
take my app from desktop
to cloud”
Example #2
Desktopintegration
27
Built-in collaboration with Docker
Cloud & Docker ID
docker.com/getdocker
edge channel
Available in an Edge release near you
enterprise edition
Ubuntu
Fedora
Mac
Azure
CentOS
Windows 10
AWS
Debian
community edition
Ubuntu
Windows Server
Azure
CentOS
Suse
Red Hat
AWS
Oracle Linux
Docker for Ops
Going to Production is Hard
Going to Production SECURELY is EXTREMELY Hard
Challenges to a
secure production:
Distributed Systems1
Challenges to a
secure production:
Distributed Systems1
Solutions:
Distributed systems are just
more systems.
Use the same tools.
Solutions:
Challenges to a
secure production:
Distributed Systems1 Secure orchestration
Orchestration
Container Runtime
OS
Infrastructure Management
Let’s talk about secure orchestration
Application Services
Raft StoreNode
IdentitySecrets
Routing
MeshEncrypted Networking
Application Services
Core Orchestration Engine
Secure Orchestration with SwarmKit
Secure Node
Introduction
SwarmKit
SWMTKN-1-mx8suomaom825bet6-cm6zts22rl4hly2
Known
Prefix
Token
Version
Hash
of Root CA
Random
Secret
Cryptographic
Node Identity
SwarmKit
MTLS
Between All
Nodes
SwarmKit
Cluster
Segmentation
SwarmKit
Encrypted
Networks
SwarmKit
Secure
Secret
Distribution
SwarmKit
Moby
Orchestration
Container Runtime
OS
Infrastructure Management
Container Platform Layers
Application Services
Docker is a platform made of components
Raft StoreNode
IdentitySecrets
Routing
Mesh
Overlay
Networking
Swarm Orchestration
Engine
Application Services
12,000,000,000
11,000,000,000
10,000,000,000
9,000,000,000
8,000,000,000
7,000,000,000
6,000,000,000
5,000,000,000
4,000,000,000
3,000,000,000
2,000,000,000
1,000,000,000
Notary
runC
containerd
HyperKit , VPNKit, DataKit
SwarmKit
libcontainer
libnetwork
InfraKit
2013 2014 2015 2016 2017
1M2014
PULLS
1B2015
PULLS
6B2016
PULLS
12B2017
PULLS
linuxKit
LinuxKitA toolkit for building secure, portable and lean operating systems for containers
Taking Dockermulti-platform
“I want Docker for X”
Desktop Server Cloud
I want Docker for…
Not every platform provides a Linux subsystem
Not every platform provides a Linux subsystem
Orchestration
Container Runtime
Linux Subsystem
Infrastructure Management
Application Services
The container movement needs asecure, lean, portable subsystem
The container movement needs
a secure, lean, portable Linux subsystem.
introducing
Only works with
containers
- Smaller attack
surface
- Immutable
infrastructure
- Sandboxed system
services
- Specialized patches
and configuration
Incubator for
security innovations
- Wireguard,
Landlock, KSPP
- MirageOS type
safe system
daemons
Community-first
security process
- Linux is too big
for any one
company to
secure it
- Participate in
existing Linux
security efforts
1. LinuxKit: a SECURE Linux subsystem
- Minimal size, minimal boot time
- All system services are containers
- Everything can be removed or
replaced
2. LinuxKit: a LEAN Linux subsystem
- Desktop, server, IoT, mainframe
- Intel & ARM
- Bare metal & virtualized
3. LinuxKit: a PORTABLE Linux subsystem
Docker and Microsoft collaborate to bringLinux containers to Windows
+ +
https://github.com/linuxkit/linuxkit
Get Started with LinuxKit
MobyAn open framework to assemble specialized container systems without reinventing the wheel.
Pioneers 2013 - 2014
Production Model: open-source!
Use case: cloud native apps on Linux server
Early Adopters 2015 - 2016
Production Model: OPEN COMPONENTS
Mainstream 2017 - 2018Containers are spreading to every category of computing:
server, datacenter, cloud, IoT, desktop, mobile…
Case study:
Specializing Docker for the mainstream
Desktop Server Cloud
The open component model shows its limits…
The auto industry has solved this problem: COMMON ASSEMBLIES.
Scaling the Docker production model: share components AND
ASSEMBLIES.
It’s time to take our ecosystem to the next level…
By collaborating on components AND COMMON ASSEMBLIES.
– Library of 80+ components
– Package your own
components as containers
– Reference assemblies
deployed on millions of nodes
– Create your own assemblies
or start from an existing one
A framework to assemble
specialized container
systems without
reinventing the wheel.
Docker uses Moby for its
open-source
– Thousands of contributors,
hundreds of patches/week
– Component development
– Specialized assembly
development
– Integration tests
– Architecture design
– Integration with other projects
– Experimentation and bleeding
edge features
Docker uses Moby for its
open-source...
and so can you!
– Community-run
– Open governance inspired by
the Fedora project
– Plays well with existing
projects - no donation
necessary!
Moby and Docker
What it means for you
Moby helps you
innovate without tying
you to Docker
System BuildersDocker Users
Docker will better leverage
the ecosystem to innovate
faster for you
Moby transforms multi-month R&D projects into weekend projects.
locked-down Linux with remote attestation
Weekend project #1:
Notary
custom CI/CD stack
Weekend project #2:
Notary Registry Docker Builder
+
custom CI/CD stack + Debian+ Terraform
Weekend project #3:
Notary Docker Builder
+
Registry
“RedisOS”
Weekend project #4:
"RedisOS"for Windows
"RedisOS"for Mac
"RedisOS"for bare metal
HyperKit
bare metal
Etcd clustering on Google Cloud
Weekend project #5:
SSHD
Kubernetes on the Mac
Weekend project #6:
HyperKit
Getting Started
- Blog https://mobyproject.org/blog
- Twitter @moby
- Github moby/moby
Let’s take containers mainstream!
InfraKitA toolkit for building declarative, self-healing infrastructure.
What is it?
90
• Launched at LinuxCon, Berlin in October, 2016.
• Toolkit for building declarative, self-managing
distributed applications
• Active management with active controllers
• scaling groups, rolling updates
• monitoring / health checks
• connecting nodes to L4 / ingress
• Declarative infrastructure
Architecture
CLI
API
container orchestration
Where does it fit?
92
kubectl run nginx --image=nginx
gcloud container node-pools list --zone us-
central1-f --cluster MyWorkers
aws autoscaling update-auto-scaling-group
--auto-scaling-group-name MyWorkers
docker create service nginx …
infrakit group describe workers
az vmss create --resource-group vmss-
test-1 --name MyWorkers
container orchestration
infrastructure orchestrationinfrastructure orchestration
list, err :=
group.Controller.Describe(“workers”)
App Opscontainer orchestrationApp Ops
One console across environments
93
kubectl run nginx --image=nginx docker create service nginx …
infrakit group describe workers
container orchestration
infrastructure orchestration
list, err :=
group.Controller.Describe(“workers”)
AWS RackHDAZ GCP OneVIEWMAASKVM VMW
Cloud Ops Hardware OpsCluster Ops
Configuration
Example config file (zk.conf): Group configuration = Instance + Flavor
{"Properties": {
/* raw configuration */
}}
{"groups" : {
"my_zookeeper_nodes" : {"Properties" : {
"Instance" : {"Plugin": "instance-vagrant","Properties": {
"Box": "bento/ubuntu-16.04"}
},"Flavor" : {
"Plugin": "flavor-zookeeper","Properties": {
"type": "member","IPs": ["192.168.1.200", "192.168.1.201", "192.168.1.202"]
}}
}}
}}
Current Status
Support more platforms
96
• Compute:
• Bare-metal: HP OneView, MAAS, RackHD
• Public cloud: AWS, GCP
• MacOS X (HyperKit); Docker containers
• Coming soon: Azure, IBM, Digital Ocean,
Packet, libvirt
• Other resource types
• AWS - vpc, subnets, gateways, etc.
Improve usability
97
• Templates
• Complex scripts and configuration in any format;
no more escape quotes in JSON
• Fetch templates from remote repositories
• Playbooks
• CLI - flags, prompts — config driven and
dynamic
• Share “playbooks” from remote repositories
Improve core system
98
• High Availability — Swarm Mode or etcd
• New Plugin types — Metadata and Events
• Metadata: cluster-wide sysfs and reflection
• Events - publish / subscribe
• Remote client access: infrakit -H host:port to remote cluster
Road Map
Use Cases
100
• Support container orchestration
• bootstrapping + day N management
• API for cluster autoscaling
• k8s, Docker Swarm Mode
• Bare-metal + GPU provisioning
• IoT — LinuxKit integration / custom kernel
deployment
Improve usability
101
• Finalize API / Schema for 1.0
• Make it easy to consume
• Simplify setup - fewer daemons and binaries
• Embeddable / vendor API
• Sensible CLI for stable / experimental features
• Make it easy to extend / contribute
• metadata / instance plugins
• playbooks / reusable templates
• community CI / compatibility testing
• Documentation
Improve core system
102
• Provisioning of diverse resource types
• networks / proxies / load balancers
• GPU
• Stability / performance of core controllers
• Asynchronous messaging - mqtt, natsd, amqp
• Monitoring + Health check SPI
Support more platforms
103
• Direct libvirt / KVM / CUDA
• Better bare-metal / hardware ops integration
• Kernel image build pipeline — LinuxKit
Build, test, and deploy clusters from infrastructure
definitions to kernel images
Get involved
https://github.com/docker/infrakit
dockercommunity.slack.com: #infrakit
THANK YOU
