CNA1142BE Developer-Ready Infrastructure from or …...Pivotal Cloud Foundry 101 6 war PCF Elastic...
Transcript of CNA1142BE Developer-Ready Infrastructure from or …...Pivotal Cloud Foundry 101 6 war PCF Elastic...
Merlin Glynn (Vmware)Ramiro Salas (Pivotal)
CNA1142BE
#VMworld #CNA1142BE
Developer-Ready Infrastructure from VMware and Pivotal
VMworld 2017 Content: Not fo
r publication or distri
bution
• This presentation may contain product features that are currently under development.
• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.
• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
• Technical feasibility and market demand will affect final delivery.
• Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Disclaimer
2
VMworld 2017 Content: Not fo
r publication or distri
bution
Agenda
1 Pivotal Cloud Foundry 101 Why do my Developers want it?
2 Site Reliability Engineering (SRE) The role of the Platform Operator
3 Ops: PCF & vSphere
4 Ops: PCF Network & Security
5 Ops: PCF Monitoring & Logging
6 Ops: PKS & Developer Ready Infrastructure
3
VMworld 2017 Content: Not fo
r publication or distri
bution
Pivotal Cloud Foundry 101Why do my Developers want It?
VMworld 2017 Content: Not fo
r publication or distri
bution
What Pivotal Cloud Foundry solves for …
5
Ag
ilit
y &
Co
st
Savin
gs
Cloud Native Maturity
NetworkingStorage
ServersVirtualization
O/SMiddleware
RuntimeDataApps
Apps T
eam
Manage A
ll
Old School
Apps M
anages
NetworkingStorage
ServersVirtualization
O/S
MiddlewareRuntime
DataApps
IaaS + Containers
Ops
Apps M
anages
NetworkingStorage
ServersVirtualization
O/S
MiddlewareRuntime
DataApps
Cloud Native Platform
Ops
PC
F
Pla
tform
_O
psVMworld 2017 Content: N
ot for publicatio
n or distribution
Pivotal Cloud Foundry 101
6
war
PCF Elastic Runtime
Availability Zone 1 Availability Zone 2 Availability Zone 3
Staging
Root
FS
Build
Pack
war
`cf push`
Drop
let
A
I
A
Imyapp.foo.com
PCF Routing PCF Routing PCF Routing
“Here is my source code
Run it on the cloud for me
I do not care how”
Developer
URL Request:
myapp.foo.com
VMworld 2017 Content: Not fo
r publication or distri
bution
PCF Elastic Runtime
Pivotal Cloud Foundry 101
7
Availability Zone 1
myapp.foo.com
PCF Diego
VM=Cell
VM=Cell
PCF Routing
VM=
GoRouter
A
I
A
I
• Running Containers are called ‘Application Instances’ (AIs)
• PCF Diego schedules & maintains the health of containers on special VMs called Diego Cells.
• GoRouters are another type of VM that balances & forwards requests to the correct Cell AIs
How do the Containers run?URL Request:
myapp.foo.com
VMworld 2017 Content: Not fo
r publication or distri
bution
Pivotal Cloud Foundry 101
8
PCF Elastic Runtime
Availability Zone 1 Availability Zone 2 Availability Zone 3
A
I
A
I
PCF Routing PCF Routing PCF Routing
Developer
• Self Service & Automated Scaling
• Diego Dynamically keeps apps Healthy in case of IaaS faults
• No Intervention from the Developer and no Tickets ☺
• PCF also provides auto recovery when IaaS is repaired
CF API
`cf scale myapp –i 3`
A
Imyapp.foo.comA
I
Cluster Cluster ClustervSphere
VMworld 2017 Content: Not fo
r publication or distri
bution
Pivotal Cloud Foundry 101
9
PCF Elastic Runtime
Availability Zone 1 Availability Zone 2 Availability Zone 3
A
I
A
Imyapp.foo.com
• PCF Service Broker
– Self Service to app services
– PCF Managed & External
• Binds creds to app (12 Factor)
Developer PCF Mysql PCF RabbitMQ PCF Spring Svcs
DATA MSG BUSNetflix
OSS
User Provided
CF API
`cf create-service mysql`
jdbc url & creds
vcap env {}
`cf bind-service mysql`
VMworld 2017 Content: Not fo
r publication or distri
bution
BOSH
Pivotal Cloud Foundry 101
10
PCF Elastic Runtime
Availability Zone 1 Availability Zone 2 Availability Zone 3
A
I
A
I
PCF Routing PCF Routing PCF Routing
• Pivotal Cloud Foundry is a distributed system made up of many VMs
• Automated deployment and health management is provided by Pivotal Operations Manager & BOSH
• Enables API driven & repeatable deployment
CF API
`bosh deploy mypcf`
A
Imyapp.foo.comA
I
Cluster Cluster ClustervSphere
Platform
Operator
Developer
Ops Manager
(OVA)
VMworld 2017 Content: Not fo
r publication or distri
bution
Site Reliability Engineering (SRE)The role of the Platform Operator
VMworld 2017 Content: Not fo
r publication or distri
bution
Key Developer Ready Infrastructure Personas
12
Platform
Operator
Developer
IT
Operator
– SRE (Site Reliability Engineering)
– Deploy, Scale, Operate Platform
– Innovation of Business Capability as Cloud native Apps
– Develop, Deploy, Scale, Monitor Apps
– Physical Infrastructure is Operated
– Network & Security Control Policy is defined
Cloud-Native App Services
Automation
Monitoring, Security & Logging
Cloud-NativePlatform
PhysicalInfrastructure
App Lifecycle
VMworld 2017 Content: Not fo
r publication or distri
bution
Key Developer Ready Infrastructure Personas
13
Developer
IT
Operator
– SRE (Site Reliability Engineering)
– Deploy, Scale, Operate Platform
– Innovation of Business Capability as Cloud native Apps
– Develop, Deploy, Scale, Monitor Apps
– Physical Infrastructure is Operated
– Network & Security Control Policy is defined
• Platform Operators
– Site Reliability Engineers (SRE)’s
• Role Shift
– In most case the VI Admins (IT Ops), are becoming the Platform Operators
• Cloud Native Applications at scale can & should be kept running by a 2 Pizza Team mentality (DevOps in Action)
Platform
Operator
VMworld 2017 Content: Not fo
r publication or distri
bution
14
Site Reliability Engineers
Platform
Operator
• Platform is Reliable
• Capacity Is planned for
• Platform is Secured & Controlled
• Platform is Auditable
• Developers are Agile
• Platform as Code{}
– SRE (Site Reliability Engineering)
– Deploy, Scale, Operate Platform
What do they Do?
% of Time Spent
Traditional Ops Coding Ops
They combine the knowledge &
skills of the IT Ops guys with the
mandate of providing ops
processes as code {}
VMworld 2017 Content: Not fo
r publication or distri
bution
Ops: PCF & vSphere
VMworld 2017 Content: Not fo
r publication or distri
bution
vSphere Fundamentals for PCF
16
BOSH
AZ1 AZ2 AZ3
Platform
Operator
Ops Manager
(OVA)
Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2
go_rtr go_rtr go_rtr go_rtr go_rtr go_rtr
ESX
Cluster
Developer
PCF OrgPCF Space
App App
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
ESX
Cluster
ESX
Cluster
• PCF is many services deployed as many VMs, called instances
• PCF distributes instances across availability zones (AZ)
• vSphere Clusters & Resource Pools map to PCF AZs
VMworld 2017 Content: Not fo
r publication or distri
bution
vSphere Fundamentals for PCF
17
BOSH
AZ1 AZ2 AZ3
Platform
Operator
Ops Manager
(OVA)
Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2
go_rtr go_rtr go_rtr go_rtr go_rtr go_rtr
ESX
Cluster
Developer
PCF OrgPCF Space
App App
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
ESX
Cluster
ESX
Cluster
• VMotion Helps keep load balanced within clusters
• vSphere HA helps recover instances
VMworld 2017 Content: Not fo
r publication or distri
bution
vSphere Fundamentals for PCF
18
BOSH
AZ1 AZ2 AZ3
Platform
Operator
Ops Manager
(OVA)
Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2 Cell_0 Cell_1 Cell_2
go_rtr go_rtr go_rtr go_rtr go_rtr go_rtr
ESX
Cluster
Developer
PCF OrgPCF Space
App App
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
ESX
Cluster
ESX
Cluster
• vSAN, VMFS, NAS Datastores are supported
• Storage SVMotion is NOT recommended
Data
store
Data
store
Data
store
VMworld 2017 Content: Not fo
r publication or distri
bution
vSphere Fundamentals for PCF
19
BOSH
AZ1
Platform
Operator
Ops Manager
(OVA)
Cell_0 Cell_1 Cell_2
go_rtr go_rtr
Developer
PCF OrgPCF Space
App App
OTHER
INSTANCES
OTHER
INSTANCES
OTHER
INSTANCES
ESX
Cluster
• Single PCF Availability Zone deployments are supported
• Limited Platform availability
• Usually smaller scale deployments
VMworld 2017 Content: Not fo
r publication or distri
bution
vSphere Fundamentals for PCF
20
BOSH
AZ1 AZ2 AZ3
Platform
Operator
Ops Manager
(OVA)
ESX
Cluster
ESX
Cluster
ESX
Cluster
• Most instances do not deploy dynamically
• Platform Operator needs to plan for scale based on Application Instances (AIs)
• https://pcfsizer.cfapps.io
100 AIs
VMworld 2017 Content: Not fo
r publication or distri
bution
Ops: PCF Network & Security
VMworld 2017 Content: Not fo
r publication or distri
bution
Network Fundamentals for PCF
22
VTEP VLAN VTEP VLAN
ESX
Cluster
ESX
Cluster
ESX
Cluster
VTEP VLAN
• NSX Logical Switches provide a single Layer 2 over Layer 3 Network.
• Operator can have a VLAN per Physical Fault Domain (Cluster)
• Logical Switch presents as a simple single network to PCF
NSX Logical Switch 192.168.10.0/24
NSX Logical Switch Primer
172.16.101.1/24 172.16.102.1/24 172.16.103.1/24
VMworld 2017 Content: Not fo
r publication or distri
bution
Network Fundamentals for PCF
23
ERT
AZ1 AZ2 AZ3
SVC1 Infra
• NSX Logical Switches give PCF a common Layer 2 Network across AZs
• Recommend 1 Switch per deployment, sometimes called a PCF ‘tile’
VTEP VLAN VTEP VLAN
ESX
Cluster
ESX
Cluster
ESX
Cluster
VTEP VLAN
Elastic RunTime Deployment
MySQL DeploymentBOSHOperations
Manager
192.168.0.0/28 192.168.8.0/22 192.168.12.0/24
VMworld 2017 Content: Not fo
r publication or distri
bution
Network Fundamentals for PCF
24
ERT
AZ1 AZ2 AZ3
SVC1 Infra
• NSX Distributed Logical Router (DLR) Allows for hundreds of connected Logical Switches
– NSX Edge has a 10 Interface limit
– Optimized East & West traffic
VTEP VLAN VTEP VLAN
ESX
Cluster
ESX
Cluster
ESX
Cluster
VTEP VLAN
Elastic RunTime Deployment
MySQL DeploymentBOSHOperations
Manager
192.168.0.0/28 192.168.8.0/22 192.168.12.0/24
NSX Distributed Logical Router
VMworld 2017 Content: Not fo
r publication or distri
bution
Network Fundamentals for PCF
25
ERT
AZ1 AZ2 AZ3
SVC1 Infra
• PCF requires a production grade external load balancer
• NSX Edge provides:
– SNAT & DNAT
– SSL Term & Acceleration
– HTTP & TCP Load Balancing
• RCF 1918 on the Logical Switches = Repeatability
VTEP VLAN VTEP VLAN
ESX
Cluster
ESX
Cluster
ESX
Cluster
VTEP VLAN
Elastic RunTime Deployment
MySQL DeploymentBOSHOperations
Manager
192.168.0.0/28 192.168.8.0/22 192.168.12.0/24
NSX Distributed Logical Router
Routing Logical Switch
NSX EdgePUBLIC VIPS
INTERNAL NAT
VMworld 2017 Content: Not fo
r publication or distri
bution
26
ERT
AZ1 AZ2 AZ3
SVC1 Infra
VTEP VLAN VTEP VLAN
ESX
Cluster
ESX
Cluster
ESX
Cluster
VTEP VLAN
Elastic RunTime Deployment
MySQL DeploymentBOSH
NSX
Distributed Firewall
(DFW)192.168.0.0/28 192.168.8.0/22 192.168.12.0/24
NSX Distributed Logical Router
Routing Logical Switch
NSX EdgePUBLIC VIPS
INTERNAL NAT
• Use DFW
– Single Policy Engine across all Logical Switches
– Use for Internal East/West and Egress Control
Network Security & Controls
• Use Edge for Perimeter Ingress Control
VMworld 2017 Content: Not fo
r publication or distri
bution
Network Security & Controls
PCF OrgPCF Space
AppA AppB AppC
cf create-security-group SECURITY-GROUP PATH-TO-RULES-FILE cf create-security-group dev-mssql mssql.json
PCF Application Security Groups (ASG):
– Uses iptables in the Diego Cell Server
– Controls Egress only at the container source level
– Can control any IP address as the target
• Operator Declares in the Platform
[ {
"protocol": "tcp",
"destination": "10.0.11.0/24",
"ports": "1-65535"
},
{
"protocol": "udp",
"destination": "10.0.11.0/24",
"ports": "1-65535"
} ]
Platform
Operator
Prod Mssql
192.168.11.10
Prod Mssql
10.0.11.10
VMworld 2017 Content: Not fo
r publication or distri
bution
Network Security & Controls
PCF OrgPCF Space
AppA AppB AppC
cf allow-access SOURCE-APP DESTINATION-APP --protocol PROTOCOL --port PORT• cf allow-access “AppA” “Appc” --protocol TCP --port 443
Developer
PCF Container to Container Networking:
– Creates and Overlay (VXLAN)
– Controls ingress & egress between Ais(containers)
– Uses CNI
• NSX-T in development
• “batteries included” protocol today
– Developer can Declare in CI/CD
VMworld 2017 Content: Not fo
r publication or distri
bution
Ops: PCF Monitoring & Logging
VMworld 2017 Content: Not fo
r publication or distri
bution
Monitoring & Logging
30
METRICS
LOGS
Metrics & Logs will emit from many sources:
• PCF Platform
• PCF Applications
• vSphere
• NSX
• Physical & Logical
Platform Operator MUST leverage ALL of them
VMworld 2017 Content: Not fo
r publication or distri
bution
Monitoring & Logging
31
Developer
Virtual Data Center
– I need to keep my apps healthy
– I need self service to my Apps Log’s
– I need to instrument my Apps (APM)
Platform
Operator
– I need to keep the Platform healthy
– I need to plan capacity
– I need to watch & Alert on KPIs
– I need to audit & Report
vRops
vRealize Operations (vRops) KPI Visualization & Alerting for:
PCF vSphere NSX
VMworld 2017 Content: Not fo
r publication or distri
bution
Monitoring & Logging
32
Developer
Virtual Data Center
– I need to keep my apps healthy
– I need self service to my Apps Log’s
– I need to instrument my Apps (APM)
Platform
Operator
– I need to keep the Platform healthy
– I need to plan capacity
– I need to watch & Alert on KPIs
– I need to audit & Report
vRLI
vRops
vRNI
vRealize Log Insight (vRLI) Log Aggregation & Alerting for:
PCF vSphere NSX
vRealize Network Insight (vRLI) Network & Security Reporting for:
Physical & Logical Networks
VMworld 2017 Content: Not fo
r publication or distri
bution
Monitoring & Logging
33
Developer
Virtual Data Center
– I need to keep my apps healthy
– I need self service to my Apps Log’s
– I need to instrument my Apps (APM)
Platform
Operator
– I need to keep the Platform healthy
– I need to plan capacity
– I need to watch & Alert on KPIs
– I need to audit & Report
vRLI
vRops
vRNI
vRealize Log Insight (vRLI) Long term log Aggregation & Alerting for:
Applications Running in PCF
VMworld 2017 Content: Not fo
r publication or distri
bution
Monitoring & Logging
34
Developer
Virtual Data Center
– I need to keep my apps healthy
– I need self service to my Apps Log’s
– I need to instrument my Apps (APM)
Platform
Operator
– I need to keep the Platform healthy
– I need to plan capacity
– I need to watch & Alert on KPIs
– I need to audit & Report
vRLI
vRops
Wavefront
vRNI
Wavefront by VMware, Application Performance Monitoring (APM) for:
Applications Running in PCF
VMworld 2017 Content: Not fo
r publication or distri
bution
Wrapping It up w/ PKSand DRI …
VMworld 2017 Content: Not fo
r publication or distri
bution
Kubernetes 101
36
K8s Cluster
Worker
`kubectl apply –f myapp.yml`
Developer
Worker
kube-proxyMaster
etcd
kube-proxy
Service: nodeport
POD POD
Load Balancer
URL Request:
myapp.foo.com/k8siscool
Docker
Registry
VMworld 2017 Content: Not fo
r publication or distri
bution
K8s Cluster
WorkerWorker
Kubo
37CONFIDENTIAL
WorkerWorker
kube-proxyMaster
etcd
kube-proxy
service
POD POD
Load Balancer
Platform
Operator
BOSH
Day 1
Day 2
Master
etcdetcd
What is KUBO?
• Kubernetes Powered By BOSH
• Can Be deployed independent of PCF
• Can Deploy & Manage Multiple K8S Clusters
VMworld 2017 Content: Not fo
r publication or distri
bution
38
VMware PKS
Kubernetes on BOSH (Kubo)
BOSH
NSX
Analytics Automation
SecurityOperations
Monitoring
GCP
Service Brokermasteretcd worker
Logging
vSANvSphere
masteretcd workerContainer
Registry
(PKS)
VMworld 2017 Content: Not fo
r publication or distri
bution
PCF + PKS
Developer
BOSH
PCF + PKS
• Integrated PCF Service Broker “I need a K8S cluster”
`cf create service mykubo`
Why Kubernetes + PCF…
– App Packaging Need Only
– Need Specific hardware and image stack (example GPU)
– Complex multi VM Data services with perisistent diskj … like ELK … not a good fit for PCF Elastic Runtime staging
VMworld 2017 Content: Not fo
r publication or distri
bution
PCF + PKS
Developer
BOSH
PCF + PKS
• Integrated PCF Service Broker
• Integrated PCF Routing
“I need to route to my K8S Service”
`http://myk8sapp.io`
VMworld 2017 Content: Not fo
r publication or distri
bution
Developer Ready Infrastructure
Compute Network Storage
Application
Services
Container
ServicesA
uto
ma
tio
n
Platform
Operator
Developer
App Monitoring
• Automation
• Day 2 Operations
• Control
• Application Services or Container Services
• Application Logging & Monitoring
Solves for DevOps Reqs …
App Logging
Platform
Monitoring
Platform
Logging
Self Service
VMworld 2017 Content: Not fo
r publication or distri
bution
Developer Ready Infrastructure
vSphere NSX vSAN
Pivotal Cloud Foundry
PCF
PKS
BOSH powered KubernetesB
OS
H
Platform
Operator
Developer
Wavefront
Self Service
• Automation
• Day 2 Operations
• Control
• Application Services or Container Services
• Application Logging & Monitoring
Solves for DevOps Reqs …
vRLI (Dev)
vRops
vRLI (Ops)
vRNI
VMworld 2017 Content: Not fo
r publication or distri
bution
43
VMworld US Key Focus Description
CNA1509BU DRI Developer-Ready Infrastructure from VMware & Pivotal
CNA1612BU PCF & PKSUse Cases: Deploying real-world workloads on Kubernetes and Pivotal Cloud
Foundry
CNA2006BU DRIDeep Dive: Architecting Container Services with VMware and Pivotal
Developer Ready Infrastructure
CNA2080BU PKS Deep Dive: How to Deploy and Operationalize Kubernetes
CNA3429BU PKSBasics of Kubernetes on BOSH: Run Production-grade Kubernetes on the
SDDC
CNA3430BU PCFYour Enterprise Cloud-Native App Platform: An Introduction to Pivotal Cloud
Foundry
MGT2871BUPCF & vRops,
vRLI
Bridging the Operations Gap Between the Software-Defined Data Center
and Pivotal CF for VMware Deployments
NET1523BU PCF & NSX Integrating NSX and Cloud Foundry
PAR4411PU DRIEmerging Technologies with VMware and Pivotal - presented jointly by
VMware, Pivotal and Special Guest Speakers from Cognizant and WWT
Developer Ready Infrastructure @ VMworld
VMworld 2017 Content: Not fo
r publication or distri
bution
VMworld 2017 Content: Not fo
r publication or distri
bution
Thank You &Any Questions …
VMworld 2017 Content: Not fo
r publication or distri
bution
VMworld 2017 Content: Not fo
r publication or distri
bution