© 2019 DIAMANTI | CONFIDENTIAL | DO NOT …...© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 4...
Transcript of © 2019 DIAMANTI | CONFIDENTIAL | DO NOT …...© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 4...
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 2
Challenges in Deploying Kubernetes on Hyperconverged Infrastructure (HCI)
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 3
Presenters
Naren NarendraDirector
Product Marketing
Naveen SethFounding Engineer
Hiral PatelFounding Engineer
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 4
Agenda
1. Background with AWS EC2 evolution - example
2. Requirements for Kubernetes on bare metal HCI
3. Demo
4. Q&A
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 5
Picture source: Evolution of the EC2 Host, https://bit.ly/2lV2WTk (@awsgeek, Jerry Hargrove, AWS)1 https://perspectives.mvdirona.com/2009/10/vl2-a-scalable-and-flexible-data-center-network/ (James Hamilton, DE, AWS, 2009)
Networking is, in effect, “in the way” and blocking the efficient optimization of the most valuable resources in the data center. 1
The Beginning
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 6
Picture source: Evolution of the EC2 Host, https://bit.ly/2lV2WTk (@awsgeek, Jerry Hargrove, AWS)Reference blog: https://perspectives.mvdirona.com/2019/02/aws-nitro-system/ (James Hamilton, DE, AWS, Feb 2019)
1GHDD
1G/10GSSD
10G/100GNVMe
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 7
Source: Powering Next-Gen EC2 Instances: Deep Dive into the Nitro System - AWS re:Invent 2018, https://bit.ly/2ltCPmaReference blog: https://perspectives.mvdirona.com/2019/02/aws-nitro-system/ (James Hamilton, DE, AWS, Feb 2019)
AWS Nitro Cards
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 8
● Host is heavily taxed on utilization and performance, applications starve
● Noisy neighbors● Some NW acceleration● Very High TCO
Hyperconverged Infrastructure (HCI) Evolution
HYPERVISOR
STORAGE
NETWORKING
VM VM VM VM ….
Hyperconverged 1.0
HYPERVISOR
STORAGE
NETWORKING
VM VM VM VM ….
Hyperconverged 1.5
HYPERVISOR
NETWORKING
VM VM VM VM ….
Hyperconverged 2.0
STORAGE
NETWORKING
Hyperconverged 3.0
STORAGE
C1 C2 C3
C4 C5 C6
C7 Cn
….
SO
FTW
AR
E
H
AR
DW
AR
E
● Host is heavily taxed on utilization and performance, applications starve
● Noisy neighbors● No SLA guarantee● Highest TCO
● Host is heavily taxed on utilization and performance, applications starve
● Noisy neighbors● Storage offload, NW
acceleration● High TCO
● Cloud native● Applications get >95% host
utilization, Hypervisor removed● Storage and NW acceleration● No noisy neighbors● Guaranteed SLA (Latency, BW)● Inherently secure● Lowest TCO
(Outpost + Nitro)
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 9
NETWORKING
Hyperconverged 3.0
STORAGE
C1 C2 C3
C4 C5 C6
C7 Cn
….
NETWORKING
Hyperconverged 3.0+
STORAGE
C1 C2
C4 C5
C7
….
C3
C3
KVM1
KVM2
Bare-Metal Containers
+ Container-Native
Virtualization
“Container-Native Virtualization”
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 10
HCI Requirements
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 11
HCI Requirements for Containerized Applications
Multiple applications on a node ⇒ Guaranteed SLAs (Jitter Free)
NETWORKING
STORAGE
● SR-IOV○ Hardware queues
● Performance Tiers○ Min guarantees○ Max limits
● SR-IOV○ Hardware queues
● Performance Tiers○ Min guarantees○ Max limits
Networking Storage
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 12
HCI Requirements for Kubernetes
NETWORKING
STORAGE
NETWORKING
STORAGE
NETWORKING
STORAGE
Multiple application instances across a Kubernetes cluster
● Static and Dynamic endpoint provisioning
● Multiple endpoints● Endpoint visibility● Separation of control
and data planes● Availability zones
● Static and Dynamic provisioning
● Synchronous mirroring● Snapshots / Restore● Backup / Restore● Availability zones
Networking (CNI) Storage (CSI)
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 13
HCI Requirements for Kubernetes Clusters
NETWORKING
STORAGE
NETWORKING
STORAGE
NETWORKING
STORAGE
NETWORKING
STORAGE
NETWORKING
STORAGE
NETWORKING
STORAGE
High Availability Zones ⇒ Campus Clusters
● Zone aware subnet management ● Zone aware mirror placement
Networking (CNI) Storage (CSI)
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 14
HCI Requirements for “yet-to-be-containerized” Legacy Applications
NETWORKING
STORAGE
NETWORKING
STORAGE
NETWORKING
STORAGE
KVM
KVM
KVM
KVM
KVM
KVM
“Container-Native Virtualization”
● Feature parity● Performance parity● Kubernetes managed
● Feature parity● Performance parity● Kubernetes managed
Networking (CNI) Storage (CSI)
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 15
HCI Requirements for Kubernetes Beyond Networking and Storage
Cloud Native Networking
Cloud Native Storage
CSI
CNI
ORCHESTRATION
RUNTIME
GKE AKS SECURITY (RBAC, LDAP, AD)
OPERATING SYSTEM
ORCHESTRATION
CONTAINER RUNTIME
MANAGEMENTMONITORING
LOGGING
SERVICE DISCOVERY
ROLE BASED ACCESS CONTROL
NETWORKING
STORAGE
CONFIGURATION MANAGEMENT
IMAGE REGISTRY
NETWORKING STORAGE
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 16
Demo
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 17
Demo Flow
1. Demo setup
2. Deploy multi-instance WordPress application using Kubernetes
3. Deploy KVM for legacy application (yet-to-be-containerized) using Kubernetes
4. I/O isolation with QoS for performance
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 18
Demo Flow
1. Demo setup
2. Deploy multi-instance WordPress application using Kubernetes
3. Deploy KVM for legacy application (yet-to-be-containerized) using Kubernetes
4. I/O isolation with QoS for performance
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 19
Diamanti Demo Cluster
ETCDKUBERNETES APISERVER
ETCD ETCD
CONTROLLER MANAGERSCHEDULER
SCHEDULER EXTENSIONSAPISERVER/CONTROLLERS
Master Components
NODE AGENT + PLUGINS
KUBELET KUBELET
NODE AGENT + PLUGINS NODE AGENT + PLUGINS
KUBELETINGRESS INGRESS Node Agents
Node2Node1 Node3
INGRESS
KVM CONTROLLER
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 20
Demo Flow
1. Demo setup
2. Deploy multi-instance WordPress application using Kubernetes
3. Deploy KVM for legacy application (yet-to-be-containerized) using Kubernetes
4. I/O isolation with QoS for performance
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 21
WordPress Application Deployment
ETCD
MYSQL
WORDPRESS
KUBERNETES APISERVERETCD ETCD
CONTROLLER MANAGERSCHEDULER
SCHEDULER EXTENSIONSAPISERVER/CONTROLLERS
Master Components
NODE AGENT + PLUGINS
KUBELET KUBELET
NODE AGENT + PLUGINS NODE AGENT + PLUGINS
KUBELET
Third Party Backup Storage
WORDPRESS
INGRESS INGRESS Node Agents
S1S2
Diamanti IO Layer
INGRESS
Node2Node1 Node3
WORDPRESS
Via Storage Classes in Kubernetes
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 22
Demo Flow
1. Demo setup
2. Deploy multi-instance WordPress application using Kubernetes
3. Deploy KVM for legacy application (yet-to-be-containerized) using Kubernetes
4. I/O isolation with QoS for performance
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 23
Container-Native Virtualization
● A VM runs inside a container
● Can co-exist with container workloads/pods
● Based on KVM
● Uses Kubernetes as Orchestrator
● Consistent I/O isolation and quality of service for containers and VMs using PCI pass-through
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 24
Container-Native Virtualization(CRD) and Pod Deployment on Kubernetes
API Server
KVM Controller
Kubectl / Proxy Kubelet
Pod (per VM)
Container
VM
Guest OS
App
RC
NGINX
Network controller Storage controller
Pod
Scheduler
Diamanti API Server
KVM
Pod
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 25
KVM Deployment
ETCDKUBERNETES APISERVER
ETCD ETCD
CONTROLLER MANAGERSCHEDULER
SCHEDULER EXTENSIONSAPISERVER/CONTROLLERS
Master Components
NODE AGENT + PLUGINS
KUBELET KUBELET
NODE AGENT + PLUGINS NODE AGENT + PLUGINS
KUBELET
Third Party Backup Storage
INGRESS INGRESS Node Agents
S1S2
Diamanti IO Layer
INGRESS
KVM CONTROLLER
KVM
Node2Node1 Node3
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 26
Demo Flow
1. Demo setup
2. Deploy multi-instance WordPress application using Kubernetes
3. Deploy KVM for legacy application (yet-to-be-containerized) using Kubernetes
4. I/O isolation with QoS for performance
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 27
“Isolation” with “QoS” for Network and Storage
ETCDKUBERNETES APISERVER
ETCD ETCD
CONTROLLER MANAGERSCHEDULER
SCHEDULER EXTENSIONSAPISERVER/CONTROLLERS
NODE AGENT + PLUGINS
KUBELET KUBELET
NODE AGENT + PLUGINS NODE AGENT + PLUGINS
KUBELETINGRESS INGRESS INGRESS
KVM CONTROLLER
Node2Node1 Node3
NODE AGENT + PLUGINS
KUBELETINGRESS
NETWORKING
STORAGE
NETWORKING
STORAGE
NETWORKING
STORAGE
NETWORKING
STORAGE
C3KVM1 C3KVM
2 C3KVM3
C3KVM4 C3KVM
9
….
C1 C2 C3
C4 C9
….
C10 C11 C12
C13 C18
….
C19 C20 C21
C22 C27
….
Node4
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 28
“Isolation” with “QoS” for Network and Storage
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 29
“Isolation” with “QoS” for Network and Storage
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 30
Q&A
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 31
To Learn More...
● www.diamanti.com
● @diamanticom
● https://www.linkedin.com/company/diamanti
© 2019 DIAMANTI | CONFIDENTIAL | DO NOT DISTRIBUTE 32
Thank You!