KubernetesContinuedCloud-nativePhDCourseatLTH

Fall2019LarsLarsson

KubernetesUndertheHood

etcd OpenIDConnect

DesignPatterns

HelmPackageManager

KubernetesUndertheHood

• Architecture• Networking• Security• NetworkPolicies• Role-BasedAccessControltoKubernetesAPI

• Storage• Extensions

Architecture

Networking

Networking::PodtoPod

• ContainerNetworkInterface(CNI)• Manydifferentproviders• Differentfunctionality(L2orL3inOSIterms)• Flannel,Calico,andWeavemostcommon

• Essentiallytunnelinter-Podtrafficine.g.UDPpacketsbetweennodes• UseprotocollikeARP,BGP,orevenjustdatastoredetcdfordestination

• Readmore• ClusterNetworking

Networking::Services

• ServicesgetvirtualIPsreachableoneachhost• ForwardstraffictoregisteredEndpoints(Podsw/successfulliveness)• Managedbykube-proxy

• Serviceimplementations• (user-space–nolongerused)• Iptables(commondefault)• IPVS(high-performanceandhighscalability)

• Furtherreading• Services

Security::NetworkPolicies

Security::NetworkPolicies

• Limitincomingandoutgoingtrafficto/fromPods• Think"firewall"

• RequiressupportfromCNIprovider!

• ExampleYAMLdefinitiononnextslide

• Furtherreading• NetworkPolicies

Security::NetworkPolicy::ExampleapiVersion:networking.k8s.io/v1kind:NetworkPolicymetadata:name:test-network-policynamespace:defaultspec:podSelector:matchLabels:role:dbpolicyTypes:-Ingress-Egressingress:-from:-ipBlock:cidr:172.17.0.0/16except:-172.17.1.0/24-namespaceSelector:matchLabels:project:myproject-podSelector:matchLabels:role:frontendports:-protocol:TCPport:6379egress:-to:-ipBlock:cidr:10.0.0.0/24ports:-protocol:TCPport:5978

Podsthispolicypertainsto

AllowincomingfromIPrange...

...orfromanyinthisnamespace...

...orany"frontend"labeledPod...

...forTCPtrafficonthisport.

AllowoutgoingonlytothisIPrangeandonlyTCPtrafficonthisport

Security::Role-BasedAccessControl

Security::Role-BasedAccessControl

• AllAPIcallshappenwithincontextofsomeuser• Canassignrolestousersviaabinding

• Role(namespaced)vs.ClusterRole(notconfinedtonamespace)• RoleBindingvs.ClusterRoleBinding

• APIGroups,Resources,Verbs• Subjects{User,Group,ServiceAccount}

• Furtherreading• UsingRBACAuthorization• Authenticating

Security::Role-BasedAccessControl::ExampleapiVersion:rbac.authorization.k8s.io/v1kind:Rolemetadata:namespace:defaultname:pod-readerrules:-apiGroups:[""]#""indicatesthecoreAPIgroupresources:["pods"]verbs:["get","watch","list"]

apiVersion:rbac.authorization.k8s.io/v1kind:RoleBindingmetadata:name:read-podsnamespace:defaultsubjects:-kind:Username:jane#NameiscasesensitiveapiGroup:rbac.authorization.k8s.ioroleRef:kind:Role#thismustbeRoleorClusterRolename:pod-reader#mustmatchnameofRoleapiGroup:rbac.authorization.k8s.io

Storage

Storage

• PersistentVolumesupportviaContainerStorageInterface(CSI)• Cloud-specificones(e.g.Cinder)• NetworkedFileSystems(e.g.NFS,GlusterFS,…)• Proprietaryones(e.g.Portworx)

• StorageClasschosenforPersistentVolume

• Furtherreading• PersistentVolumes• StorageClasses• DynamicVolumeProvisioning

Extensions

Extensions

• CustomResourceDefinition(CRD)• AddcustomAPIobject• Immediatefirst-classsupportine.g.kubectl

• TypicallyCRDtiedtosomeController/Operator• ActsupondatamanagedinCRDobject

• Furtherreading• CustomResources• ExtendtheKubernetesAPIwithCustomResourceDefinitions

KubernetesUndertheHood

etcd OpenIDConnect

DesignPatterns

HelmPackageManager

etcd

• Distributedkey-valuestore

• ThecorecomponentinKubernetes• Singlesourceoftruth• StoresstateofallAPIobjectsandalleventsthatoccur• APIservicefrontendtoetcd

• Raftprotocol• Sensitivetoslownetworksandslowdiskperformance

• Furtherreading• Learningetcd• RaftConsensusAlgorithm(hascoolanimation!)

KubernetesUndertheHood

etcd OpenIDConnect

DesignPatterns

HelmPackageManager

OpenIDConnect

• Enablesthird-partyauthentication• Google,Facebook,Twitter,…

• KubernetescanuseOpenIDConnecttogetherwithRBAC• Dexeasytosetup

• ResourceServer:serviceyouwanttouse• Client:softwareyouusetoauthenticate(webbrowser!)• AuthenticationServer/IdentityProvider(IdP):servicethatauthenticatesyou

OpenIDConnectFlow

Imagecredits:M&SConsulting

KubernetesUndertheHood

etcd OpenIDConnect

DesignPatterns

HelmPackageManager

DesignPatternsinKubernetes

Sidecar Ambassador Adapter

Leaderelection Workqueue Scatter/gather

DesignPattern::Sidecar

• Oneprocesspercontainer• Sodoyoubakeinallfunctionalityintoone?

• Sidecarisahelpertomainprocess• E.g.Synchronizingfilecontents,samplingrequests,managinglogfiles...

• Composability!

DesignPattern::Ambassador

• Proxiesand(typically)simplifiescallsfrommainprocesstoaservice

• Hidescomplexity

• Reducescoupling

DesignPattern::Adapter

• Proxiesand(typically)simplifiescallstomainprocessfromaservice

• Hidescomplexity

• Reducescoupling

• E.g.ametricscollectionservicecanassumethatallourPodsarecompatiblewithsomeprotocolX–eithernativelyorviaanadapter

DesignPattern::Leaderelection

• Distributedalgorithms(upcominglecture!)oftenneedaleaderorcoodinator• Thisisverycomplex(andtheoreticallyimpossible)!• Whyre-inventthewheel?Justoffera"leaderelection"containerandinterfacewithit!

p0

p1

p2p3

p4

DesignPattern::Workqueue

• Dataprocessingcanoftenbedoneinparallel

• Moreworkers=lesswaitingonresults

• Createaworkqueueandspawnasmanyworkersasyoucan(afford)• Eachworkerisassignedtasksandmarksascompleted• Workerfailure?Workitemnotmarkedasfinished,sootherworkerclaimsit

DesignPattern::Scatter/gather

• Dataprocessingcanoftenbedoneinparallel

• Moreworkers=lesswaitingonresults

• Haveworkersworkonsub-problems!• Coordinatingprocesscollectssub-resultsandcreatesoverallresult

KubernetesUndertheHood

etcd OpenIDConnect

DesignPatterns

HelmPackageManager

HelmPackageManager

• Fullyfeaturedapplicationscangetquitecomplex!

• Helmletsyoudeployapplicationsas-a-whole("Charts")• Templatelanguagefordifferencesindeploymentsandvariables• Manageslife-cycleofentireapplication,notitsparts

• Tiller:(optionalbutuseful)server-sidecomponent• Helm:command-lineinterfacetool

• Furtherreading• Helmdocumentation

Summary

• Kubernetesarchitectureishighlymodular• Plug-and-play,usewhatyouwant,makeityours• ExtensibleAPIaswell

• etcdisthesinglesourceofalltruthinKubernetes

• OpenIDConnectletsapplicationsauthenticateusersviathirdparty

• Designpatternshelpyouquicklystandupaservicewithstandardcomponentsandabstractions

• Helmpackagemanagercanhelpdeployevencomplexapplicationsandmanagetheirlife-cycle

Nextweek-Distributedsystems