Zalando: Bootstrapping a Cassandra Cluster in a Dynamic Cloud Environment

Click here to load reader

download Zalando: Bootstrapping a Cassandra Cluster in a Dynamic Cloud Environment

of 47

  • date post

    11-Feb-2017
  • Category

    Technology

  • view

    1.125
  • download

    0

Embed Size (px)

Transcript of Zalando: Bootstrapping a Cassandra Cluster in a Dynamic Cloud Environment

  • Bootstrapping Cassandra in a Dynamic Cloud Environment

    Luis Mineiro - @voidmaze Thorbjrn Gruda - @ThorbyG

  • AgendaWho We Are and What We Do

    How Do We Run a Tech Company

    Cloud Infrastructure for Autonomous Teams

    Cassandra in a Dynamic Cloud Environment

  • 15 countries 3 fulfilment centers 15+ million active customers 2.2+ billion revenue 2014 130+ million visits per month 8.000+ employees

    ONE OF EUROPES LARGEST ONLINE FASHION RETAILERS

    Visit us: tech.zalando.com

    Tech hubs in Berlin, Dublin, Dortmund and Helsinki

  • #CassandraSummit

    Zalando Technology

  • #CassandraSummit

    Environment

    Credits to our colleague Kolja Wilcke

  • #CassandraSummit

    Environment

    Credits to our colleague Kolja Wilcke

  • #CassandraSummit

    Fashion Store Website

  • #CassandraSummit

    Fashion Store Apps

    iOSAndroid Windows Mobile

  • #CassandraSummit

    Shoes in Space

    http://thespaceshoe.com/

    http://thespaceshoe.com/

  • #CassandraSummit

    Zalando Infrastructure

    Based on https://flic.kr/p/bX5E4c

    2 datacenters thousands of production instances

    serving 15 countries

    https://flic.kr/p/bX5E4c

  • #CassandraSummit

    RadicalAgility

  • #CassandraSummit

    Whats That?

    Radical Agility is Zalando Tech's approach to running a technology organization -

    an approach based on the three pillars of autonomy, mastery and purpose

    http://zln.do/ra-video

    https://goo.gl/qosqj2

  • Compliance Innovation

    #CassandraSummit

  • #CassandraSummit

    STUPSA Cloud Infrastructure for Autonomous Teams

  • #CassandraSummit

    What Is It?

    The STUPS platform is a set of tools and components to provide a convenient

    and audit-compliant Platform-as-a-Service (PaaS) for multiple autonomous teams on top of Amazon Web Services (AWS)

    Learn more at https://stups.io

    https://stups.io

  • #CassandraSummit

    A Platform on Top of AWS

    Learn why at https://stups.io/why

    https://stups.io

  • #CassandraSummit

    Relevant Requirements

    All STUPS applications are Docker containers

    We can only use audit-compliant AMIs

    STUPS is dynamic. VPC is DHCP only

  • #CassandraSummit

    Challenges Ahead

  • #CassandraSummit

    Docker Container

    Plenty of options out there

    None was created specifically for STUPS

    They should be immutable

  • #CassandraSummit

    Audit-Compliant AMI

    We cant use DataStaxs AMIs

    Cloud Engineering team publishes audit-compliant AMIs - Taupage

    Taupage has its own bootstrapping process

  • #CassandraSummit

    Dynamic Environment

    Clusters require special nodes - Seeds

    EC2 instances get a random IP address

    EC2 instances come and go

  • #CassandraSummit

    How can I help you?

  • #CassandraSummit

    Id like to bootstrap Cassandra

  • #CassandraSummit

    YADCYet Another Docker Cassandra

    Kept the best options from the other

    recipes out there

    Added STUPS specific options

  • #CassandraSummit

    and then?

  • #CassandraSummit

    Run Container in Taupage

    With this Cassandra container

    we should be ready to run in our

    audit-compliant AMI - Taupage

  • #CassandraSummit

    You wish!

  • #CassandraSummit

    Id like to discover seed nodes dynamically

  • #CassandraSummit

    Discovery

    STUPS makes use of etcd to register EC2 instances

    Applications can use etcd for discovery and distributed locking

  • #CassandraSummit

    and then?

  • #CassandraSummit

    Custom SeedProviderWe created the EtcdSeedProvider

    seed_provider: - class_name: org.zalando.cassandra.locator.EtcdSeedProvider parameters: - url: http://example.org/v2/keys/cassandra/my-cluster-name/seeds

    Dynamically updates list of seed nodes from etcd

  • #CassandraSummit

    and then?

  • #CassandraSummit

    Senza TemplateWe use the STUPS senza tool

    to instrument AWS CloudFormation Bootstraps a Cassandra Cluster

    with one simple command

    $ senza create https://goo.gl/qyTy7b \ my-new-cluster \ etcd.example.org

  • #CassandraSummit

    STUPS Appliance

    CloudFormation

    Cluster Details Name: my-new-cluster Size: 3

    etcd.example.org

  • #CassandraSummit

    STUPS Appliance

    CloudFormation

    Cluster Details Name: my-new-cluster Size: 3

    etcd.example.org

    Auto Scaling Group

    Min, Max, Desired = 3

  • #CassandraSummit

    STUPS Appliance

    CloudFormation

    Cluster Details Name: my-new-cluster Size: 3

    etcd.example.org

    Auto Scaling Group

    Min, Max, Desired = 3

    SeedNode #1

  • #CassandraSummit

    STUPS Appliance

    CloudFormation

    Cluster Details Name: my-new-cluster Size: 3

    etcd.example.org

    Auto Scaling Group

    Min, Max, Desired = 3

    SeedNode #1

    Node #2

  • #CassandraSummit

    STUPS Appliance

    CloudFormation

    Cluster Details Name: my-new-cluster Size: 3

    etcd.example.org

    Auto Scaling Group

    Min, Max, Desired = 3

    SeedNode #1

    Node #2

    Node #3

  • #CassandraSummit

    STUPS OpsCenterCluster discovers and registers into OpsCenter while bootstrapping

  • #CassandraSummit

    What about operations?

  • #CassandraSummit

    Appliance Operations

    First release focused only on initial bootstrapping

    Some things Just Work

  • #CassandraSummit

    Adding NodesJust adjust your Auto Scaling Group

    Or trigger CloudWatch ScaleUp actions

  • #CassandraSummit

    Useless piece of sh*t

  • #CassandraSummit

    Upcoming FeaturesBetter distribution of seed nodes per AZ

    Scheduled backups

    HTTP Health check

    JMX metrics and instrumentation over HTTP

    Replacement of dead nodes

  • #CassandraSummit

    All of this is open source

  • #CassandraSummit

    We Want to Hear From You

    https://github.com/zalando/stups-cassandra

    https://github.com/zalando/stups-opscenter

    https://github.com/zalando/cassandra-etcd-seed-provider

    Try it out

    Help us find issues

    Send us pull requests

    https://github.com/zalando/stups-cassandrahttps://github.com/zalando/stups-opscenterhttps://github.com/zalando/cassandra-etcd-seed-provider

  • Thank you for listeningCheck out our blog https://tech.zalando.com

    Our many open source products https://github.com/zalando The STUPS stack https://stups.io

    Got more questions? You can reach us on twitter @ZalandoTech

    Were hiring!

    Special thanks to Jessie Dude. No Continuum Transfunctioners were harmed during the production of these slides.

    https://tech.zalando.comhttps://github.com/zalandohttps://stups.io