Docker Berlin Meetup June 2015: Docker powering Radical Agility @ Zalando Tech
55
-
Upload
henning-jacobs -
Category
Technology
-
view
1.109 -
download
3
Transcript of Docker Berlin Meetup June 2015: Docker powering Radical Agility @ Zalando Tech
DockerPowering Radical AgilityDocker Berlin Meetup [email protected] @try_except_
DOCKER-BASED DEPLOYMENT
STUPS CLOUD PLATFORM
HISTORY
RADICAL AGILITY & ARCHITECTURE PRINCIPLES
AGENDA
15 countries3 fulfillment centers15+ million active customers2.2+ billion € revenue 2014130+ million visits per month8.000+ employees
ONE OF EUROPE’S LARGEST ONLINE FASHION RETAILERS
Visit us: tech.zalando.com
A BRIEF HISTORY OF ZALANDOTECHNOLOGY
A BRIEF HISTORY OF ZALANDO TECH
ZALANDO PLATFORM
~70% of all applications
WAR deployment
Single deployment tool
On-premise data center
MAIN PRODUCTION STACK SINCE 2010
Platform
THE CHALLENGE
Platform team
request serversdeploy
Platform
THE CHALLENGE
80+ delivery teams
Platform team
deploy
request serversrequest storage
RADICAL AGILITY
GOAL
DELIVER AMAZING PRODUCTS EFFICIENTLY AT SCALE, AND FEELING GREAT ABOUT IT.
3 PRINCIPLES
PURPOSE
AUTONOMY
MASTERY
ARCHITECTURE
AN ARCHITECTURE FOR INNOVATION
API FIRST
REST
SAAS
MICROSERVICES
CLOUD
STUPSSTUPS To Unleash Penguin Swarms
AWS
STUPS
DOCKERDEPLOY
SSH ACCESS
AUDIT REPORTS
FULL AWS ACCESS
A PLATFORM ON TOP OF AMAZON WEB SERVICES
AUTONOMY AND COMPLIANCE
STUPS offers maximum freedom for developers while enabling near-real-time audit compliance for every single application.
One AWS account per Team
Deployment with Docker
Managed SSH Access
REST/OAuth 2.0 mandatory
Supports Traceability of Changes
STUPS IN A NUTSHELL
Public Internet
*.foo.example.org *.bar.example.org
Team “Foo” Team “Bar”ELB ELB
EC2Instance
EC2InstanceEC2
InstanceEC2Instance
EC2InstanceEC2
InstanceData Center LB
EC2InstanceEC2
InstanceLegacyInstances
ISOLATED AWS ACCOUNTS
DEPLOYMENT
IMMUTABLE STACKS
AWS
DEPLOYMENT WITH SENZA
Senza CLI
Pier One
docker pull
docker push
Taupage
FROM zalando/openjdk:8u40-b09-4
EXPOSE 8080
COPY target/hello-world.jar /COPY target/scm-source.json / CMD java $(java-dynamic-memory-opts) ↲ -jar /hello-world.jar
DOCKERFILE
$ docker build -t ↲ pierone.example.org/myteam/hello-world:0.2 .
$ pierone loginGetting OAuth2 token "pierone".. OKStoring Docker client configuration in ~/.dockercfg.. OK
$ docker push pierone.example.org/myteam/hello-world:0.2
DOCKER BUILD & PUSH
$ pierone tags myteam hello-world
Team │Artifact │Tag │Created│By | myteam hello-world 0.1-andre-test 13d ago ahartmann
myteam hello-world 0.1 3d ago ahartmann
myteam hello-world 0.2 3m ago hjacobs
$ pierone scm myteam hello-world 0.2
Tag│Author │URL │Revision │Status│Created│By |
0.2 hjacobs git:git@github.. 442b7502 10m ago hjacobs
VERIFY IMAGE UPLOAD
PIER ONE DOCKER REGISTRY✓ S3 backend to store images ✓ OAuth2 integration ✓ Team repositories ✓ Immutable tags & scm-source.json ✓ JVM-based (Clojure) ✓ Command line interface (Python)
SENZA: DEFINITION YAML
SenzaInfo:
StackName: hello-world
Parameters:
- ImageVersion:
Description: "Docker image version of Hello World."
SenzaComponents:
- Configuration:
Type: Senza::StupsAutoConfiguration # auto-detect network setup
- AppServer: # will create a launch configuration and ASG with scaling triggers
Type: Senza::TaupageAutoScalingGroup
InstanceType: t2.micro
SecurityGroups: [app-hello-world]
ElasticLoadBalancer: AppLoadBalancer
TaupageConfig:
runtime: Docker
source: "stups/hello-world:{{Arguments.ImageVersion}}"
ports:
8080: 8080
SENZA: STACK DEPLOYMENT
$ senza create hello-world.yaml 1 0.2
Generating Cloud Formation template.. OK
Creating Cloud Formation stack hello-world-1.. OK
$ senza events hello-world.yaml 1Stack Name│Ver.│Resource Type │Resource ID │Status │Status Reason │Event Time
hello-world 1 CloudFormation::Stack hello-world-1 CREATE_IN_PROGRESS User Initiated 10m ago
...
hello-world 1 CloudFormation::Stack hello-world-1 CREATE_COMPLETE 6m ago
docker run -d --log-driver=syslog ↲ --restart=on-failure:10 ↲ -e DB_SUBNAME=.. ↲ -v /meta:/meta:ro ↲ -e CREDENTIALS_DIR=/meta/credentials ↲ -p 8080:8080 -p 7979:7979 ↲ -u 999 ↲ pierone.example.org/stups/pierone:0.5
TAUPAGE: DOCKER COMMAND LINE
SENZA: MANAGE STACKS
LOGGING
docker run .. --log-driver=syslog ..
/etc/rsyslog.d/24-application.conf:syslogtag, startswith, "docker" ↲ /var/log/application.log
/etc/logrotate.d/..Don’t forget log rotation..
TAUPAGE: DOCKER SYSLOG
APPLICATION LOGS: TAUPAGE SUPPORTS LOGENTRIES AND SCALYR
SSH ACCESS
SSH ACCESS: TIME-LIMITED ACCESS TO ANY TEAM SERVER
MONITORING
TODO: Screenshot
ZMON
ZMON APPLIANCE
*.foo.example.org *.bar.example.org
Team “Foo” Team “Bar”
EC2Instance
EC2InstanceEC2
InstanceEC2
Instance
ZMON Appliance
ZMON Appliance
KairosDB
EC2Instance
EC2Instance
ZMONController
ELB ELB
HYSTRIX TURBINE
FULLSTOP: REPORT VIOLATIONS
DOCKER?
● Ubuntu & OpenJDK base image● Log to STDOUT● Config via environ. vars (+ KMS decryption)● Non-root execution● Persistence via EBS mounts● Immutable stacks, no orchestration● DNS endpoints, etcd e.g. for Hystrix streams
RECAP: DOCKER IN STUPS
STUPS Frontpagehttp://stups.ioSTUPS Documentationhttp://docs.stups.ioGitHub Repositorieshttps://github.com/zalando-stupsTrying out Senza and Taupagehttp://docs.stups.io/en/latest/user-guide/standalone-deployment.html
LINKS
QUESTIONS?
http://stups.io@try_except_
BACKUP
STUPS COMPONENTS
● ELB forinbound traffic
● NAT instancesfor outbound
● HTTPS Only● Internal subnets
for app instances
DMZ DMZ DMZ
internalinternal
eu-west-1a eu-west-1b eu-west-1c
ELB
EC2
internal
EC2
NAT
STUPS: AWS ACCOUNT VPC SETUP
Pier One Docker Reg.
build
approve
EC2 Instances
Docker Container
Application “myapp”issue_management: Jira
Application Version “1.0”artifact: docker/myart:1.0
Taupage AMI
Ticket System
Kio Application RegistryTicket System
SCM
Image “docker/myart:1.0”commit: afb123Issue “ABC-123”
spec: [...]
Commit “afb123”msg: ABC-123..
✓ specs approved✓ artifact tested✓ artifact approved
STUPS: TRACEABILITY
