kubernetes at eBay

eBay cloud services

● Cloud Infrastructure and Platform Services

● Data Services● Database Infrastructure● Analytics Infrastructure

Our portfolio Manages½ Million+ Compute Cores, 150K+ compute servers200+ PB of Data InfrastructureeBay.com

Data and Infra services handlesBillions of data Queries per day20 Billion+ Image Downloads Per Day300+ TB of logs per day2 Million Monitoring Metrics Per Second

app platforms current lifecycle - IaaS oriented

current Host Runtime - Standardized Deployment Agent

Kubelet + Container Runtime

current static scheduling

dynamic scheduling with a central cluster manager

static middleboxes bottleneck

distribute network and other framework functions

kubernetesOpensource

Container based runtime

Declarative, application centric abstractions

Watchable, updateable central state store

Well defined cloud provider and infrastructure plugin model

Support from various IaaS clouds (Openstack, public...)

Awesome community with great thought leadership

Waiting for the evolution of “XPaaS on Kubernetes” marketplace

challenges

Challenges: Networking

IP Routability ✔

IP Density

Multi-tenancy

overlay based

L3 routed model

Plans: Networking

BGP to host

Layer 7 Load Balancing

IPV6

Overlay

Challenges: Security

Integrate with cloud IAM (Openstack Keystone) ✔

Container security (selinux, seccomp, Docker Content Trust)

Distributed firewalls

Native Public Key Infrastructure on Kubernetes (CFSSL/Lemur)

Policy based IAM

Challenges: Storage

Integrate with cloud block storage (Openstack Cinder ) ✔

Local storage leases

Challenges: Scale

Currently has 5-20K servers per AZ

Cluster Federation

Plans: Scale

Scale testing lab - work with scale SIG

Ubernetes

Static provisioning using detached Kubelet

thanks

araveendrann

spothanis

uruddaraju

thuc.nguyen

@

ashw7n @