The Big SwitchRewiring Zalando’s Infrastructure outside DatacentersForgeRock Identity Summit 2015 - Half Moon Bay - CA

ABOUT US

Jan Löffler● Head of Platform Engineering● twitter: @jlsoft2● email: jan.loeffler@zalando.de

ABOUT US

Christian Kunert● Security Engineer● twitter:@noahk3lly● email: christian.kunert@zalando.de

ONE of EUROPE’S LARGEST ONLINE FASHION RETAILERS

15 countries3 fulfillment centers15+ million active customers2.2+ billion € revenue 2014130+ million visits per month8.000+ employees

Visit us: tech.zalando.com

ENVIRONMENT

THE GOOD OLD DAYS

Or, how to build a wall in 27 easy steps

file:///Users/kwalckermaye/Downloads/Mobile-Developers-look-ov-008.jpg

file:///Users/kwalckermaye/Downloads/desktop_death-600x369.jpg

file:///Users/kwalckermaye/Downloads/072358-wired.gif.jpeg

file:///Users/kwalckermaye/Downloads/the-death-of-the-desktop.jpg

TOPIC 1

WHERE TO GO

Building walls is an obsession of mankind, for a good reason.

However, someone will always build a bigger ladder.

THE PAST

DATACENTER ENVIRONMENT

DataCenter IGütersloh, Germany

DataCenter IIBerlin, Germany

DataCenter IIIBerlin, Germany

Global Traffic Management

DATACENTER ENVIRONMENT

DataCenter IGütersloh, Germany

DataCenter IIBerlin, Germany

DataCenter IIIBerlin, Germany

APP 1

APP 2

APP 3

APP 4

APP 5

APP 6

APP 1

APP 2

APP 3

APP 4

APP 5

APP 6

APP 1

APP 2

APP 3

APP 4FW FW

THE LOST HIGHWAY

CLOUD PROJECTS

2013/14 2014

Pequod

2013

Noah’s ARKzCloud

TOPIC 1

WHERE TO GO

THIS NEEDS TO STOP

Doing it yourself is not the most sensible thing.

Amazon invested already thousands of engineering hours… we must utilize this.

(Eric Bowman)

RADICAL AGILITY

GOAL

DELIVER AMAZING PRODUCTS EFFICIENTLY AT SCALE, AND FEELING GREAT ABOUT IT.

LEADERSHIP

FROM CONTROL & COMMANDTO PURPOSE AND TRUST

ARCHITECTURE

AN ARCHITECTURE FOR INNOVATION

API FIRST

REST

SAAS

MICROSERVICES

CLOUD

BACK TO THE DRAWING BOARD

Securing REST APIs - The Candidates

Basic Auth

● Very simple, supported by all tools.

● More or less no transport overhead.

● Stateless.

SAML

● OASIS standard

● Used by AWS to authenticate users

● Assertions can express sophisticated use cases

Kerberos

● There are no passwords on the network

● Flexible lifetime and must be revalidate after it expired

● Works with Postgres Databases

OAuth 2.0

● Open standard for Authorization

● Provides client applications a delegated access on behalf of a resource owner

● Specifies a process for resource owners to authorize access to third party resources

Notariat● Claim-based approach similar to SAML

using a PKI.● Authentication can be implemented for

different sources (SAML, Kerberos, ... )● Rotating the signing keys

UNFORTUNATELY

STOPPING FOR SOME ELEVENSES

file:///Users/kwalckermaye/Downloads/Mobile-Developers-look-ov-008.jpg

file:///Users/kwalckermaye/Downloads/desktop_death-600x369.jpg

file:///Users/kwalckermaye/Downloads/072358-wired.gif.jpeg

file:///Users/kwalckermaye/Downloads/the-death-of-the-desktop.jpg

TOPIC 1

WHERE TO GO

[Me]: Want to try OpenAM?

[H]: Sure, why not, When?

[Me]: How about now?

[H]: Now works for me…

DECEMBER 2014

IT COULD WORK

ProjectStart

WE KNOW WHAT - LEAVES THE QUESTION - HOW?

December 2014

March 2015

HackWeek

Initial TelCo

PoC

January 2015

February 2015

First Delivery

April 2015

33

LET’S ADD A LITTLE PRESSURE

CATCHING OUR BREATH

Delivery OAuth 2.0✓ 30.04.2015

GoLive for all Zalando✓ 28.05.2015

MOVING TO AWS IN A NUTSHELL

One AWS account per Teamsecured via SSL and OAuth 2.0

Deployment based on Docker

Usage of REST+OAuth mandatory

ISOLATED AWS ACCOUNTS

Public Internet

*.foo.zalan.do *.bar.zalan.do

Team “Foo” Team “Bar”ELB ELB

EC2Instance

EC2InstanceEC2

InstanceEC2Instance

EC2InstanceEC2

InstanceDatacenter LB

EC2InstanceEC2

InstanceLegacyInstances

PLANS ARE USELESS

BUT PLANNING IS EVERYTHING

Unified IdentityBeing in control of account, data and access regardless of its source

Unified PasswordOne password only to manage all accounts

Unified FlowsAbility to authenticate and authorize reliably for any identity

Unified cohesive architectureKnow you can trust an identity, without being aware of the protocol

The Vision

“Employee”

THE MISSION

ADS

OpenAM

AWS

DCITR/GTH

OpenDJ

OpenDJ

OpenDJ

OpenIDM

HR

Cust.DB

Brand CMS

Role Mgmt.

“Customer”

“Others”

OpenIG

THE PROJECT PLAN

Phase IIINew South Wales

Phase ITasmania

Phase IIVictoria

Phase IVQueensland

End of April End of July ETA October ETA December

Employee

Services

API’s

Roles Partner/Brands

Customer

Portal

Provisioning

■ Team Info■ Service Management■ Token Retrieval

All written in GOLangFollow 12FactorApp Guides

APIs

all can be reached via a common domain:https://auth.zalando.com

GTM

PHYSICAL INFRASTRUCTURE

F5 Load-Balancer F5 Load-Balancer F5 Load-Balancer Elastic Load-Balancer

Office Berlin

OpenAMService

API

Team API

config-store

sessionstore

saestore

employeestore

AD brandsstore

OpenAMService

API

Team API

config-store

sessionstore

saestore

employeestore

AD brandsstore

OpenAMService

API

Team API

config-store

sessionstore

saestore

employeestore

brandsstore

OpenAMService

API

Team API

config-store

sessionstore

saestore

employeestore

AD brandsstore

OpenIDM

DC Berlin

DC Gütersloh

AWS

Cloud Deployment

• Mai Get AWS tokens via SAML/OAuth

• Piu Request SSH access to a server

• Senza Cloud formation based deploy

TOOL OVERVIEW

Fork us on Github https://github.com/zalando-stups

AWS ACCOUNT SETUP

DMZ DMZ DMZ

internalinternal

eu-west-1a eu-west-1b eu-west-1c

ELB

EC2

internal

• ELB for inbound traffic

• NAT Instances for outbound

• HTTPS Only• Internal VPC with

own subnet

EC2

NAT

VP

CV

PC

Mai$ mai create stupsIdentity provider: https://aws.zalando.netAvailable roles:1) AWS Account 600231584188 (zalando-hackweek): Shibboleth-PowerUser2) AWS Account 786011980701 (zalando-stups): Shibboleth-PowerUserPlease select (1-4): 2‘stups’ profile created.$ mai login stups # logs in and stores keys for ‘stups’ profile$ mai Shibboleth-PowerUser $ mai --set-default stups # define ‘stups’ to be the default$ mai # login to default (‘stups’ in this case)$ mai --env stups # instead of storing, print env variablesAWS_ACCESS_KEY_ID=ASIAIA2JMCGTEH64IK2AAWS_SECRET_KEY=265nbjuqugAMWeZbS9ABhd3m6F2oik/dj37fonyl

Piu$ piu --even https://even.stups.zalan.do \ # you can specify defaults --odd odd-eu-central-1.stups.zalan.do \ johndoe@172.31.148.155 \ health debuggingssh -tA johndoe@odd.stups.zalan.do ssh johndoe@172.31.148.155$ piu defaults https://even.stups.zalan.do odd-eu-central-1.stups.zalan.do johndoe # store all defaultsssh -tA johndoe@odd.stups.zalan.do ssh johndoe@172.31.148.155$ piu 172.31.148.155 health debugging # uses all the defaultsssh -tA johndoe@odd.stups.zalan.do ssh johndoe@172.31.148.155$ piu --odd odd-eu-west-1.zalan.do 172.31.148.155 fun project restart # overwritablessh -tA johndoe@odd.stups.zalan.do ssh johndoe@172.31.148.155

Senza$ senza create kio.yaml b123 DockerImageVersion 0.1.0-SNAPSHOT $ senza show kio.yaml # shows DNS weights 90% 180 kio-b122 10% 20 kio-b121 ? 0 kio-b123$ senza weight kio.yaml \ # sets DNS weights kio-b121:0 \ kio-b123:10$ senza delete kio.yaml b121 # deletes a stack$ senza cf-template kio.yaml b123 DockerImageVersion 0.1.0-SNAPSHOT # prints the effective cf template… cf json …$ senza manifest kio.yaml b123 DockerImageVersion 0.1.0-SNAPSHOT # prints the effective manifest… manifest yaml …

Documentation

http://greendale.readthedocs.org

http://stups.readthedocs.org

Open Source

https://github.com/zalando/

https://github.com/zalando-stups

QUESTIONS?